VMware : Streamlining VMware’s Open Source Licensing Compliance with Bazel

Description of icon when needed7 Min Read

By Daniel Machlab and Rehana Tabassum

There are hundreds of thousands of open source software projects to choose from today, making open source software, or OSS, a vital component of almost any codebase. (Daniel recently presented on the topic of license compliance-you can listen to the presentationhere.)

Regardless of the OSS used, product releases must comply with legal and license requirements of the open source software. This is true for VMware and it's true for anyone using open source components in their products. Today, you can go to VMware.com and find the Open Source License file (OSL) for everything we ship. But compiling that exhaustive list of components and determining compliance status can be cumbersome, time-consuming and error prone. Discovery adds time and friction to the development cycle and can delay product release. Ideally you want a scenario that enables discovery of the OSS components in software upfront-as early as possible in the development process. What follows is the blog form of Daniel's presentation.

The New OSS Compliance Workflow Rooted in Bazel

At VMware, we turned to Bazel, a fast, scalable, multi-language and extensible build systemto help solve this problem and eliminate many of the pain points present in the OSS compliance process. By creating and applying a new Bazel rule during the build process, all the OSS can be identified upfront, giving developers adeterministicBill of Materials (BOM). There is no need for a separate OSS scanning step in the post-build stage, thereby eliminating the toil and the margin of error associated with manual input by developers.

Second, when the OSS validation happens at build time, developers are quickly informed about any problems with OSS or the presence of any denied open source packages in the code base. OSS decisions are made earlier in the development and review process, making them less costly and less likely to impose release delays due to required remediation.

Third, Bazel's multi-language support allows us to have one tool that works cross-platform, relieving the team from scanning-pipeline setup and maintenance.

Let's take a look at how the Bazel OSS Compliance process works using our new oss_auditrule.

OSS Auditing Workflow

The process starts with a Bazel build that produces an .rpm package with a project's source files, using pkg_rpm. Before Bazel, we required a post-build step to scan the .rpm produced. Now, we simply add a new rule called oss_auditto the Bazel build.

The oss_auditrule uses aBazel aspectthat analyzes the dependency graph of a build and collects information about each package.

It consumes a list of previously approved and denied OSS packages - usually determined by a legal or security team - to alert developers when denied packages are discovered. After the aspect inspects all the dependencies, the oss_auditrule outputs two files: BOM YAML and BOM-issues.

The BOM YAML file includes information on each OSS package. The BOM-issues file contains a subset of OSS packages that have been denied for use by the legal or security teams, or that are still waiting for approval. Release managers and developers can use the BOM-issues to identify problems that may block or delay a product release.

Here's an example code snippet showing how to apply oss_auditto a project.

Example Code

oss_auditcan be added to a BUILD file to audit an RPM.In the example above, the code abides by the pkg_rpmrule, creating the RPM, and the oss_auditrule, auditing the RPM.

srcs Attribute

The solution can audit any build target, but is currently only aware of Java dependencies via metadata provided by rules_jvm_external. However, it can be extended to support other target types as well. We are currently developing a prototype that supports C++ and have plans to add support for other languages soon.

approved_list and denied_list Attributes

The approved and denied lists inform oss_auditabout package status and provide extra data for inclusion in the BOM, such as copyright notices and information about package use. All OSS packages will appear in the .BOM.yaml file, but any package that hasn't been approved,or is denied, will show up in the BOM-issues.yaml file.

For a demo of how oss_auditperforms, including how to suppress a package and update approved and denied lists, see mypresentation. (The project used in the demo is from thebazelbuildrepository on GitHuband I encourage you to try it yourself.)

Component BOMs

At VMware, our larger, more complex products are constructed of many sub-components. In this case, oss_auditgenerates multiple components BOMs. These component BOMs are then merged together to create one top-level BOM and BOM-issues files for the entire product. This gives product and component teams insight into what they are shipping at varying degrees of granularity.

Conclusion

OSS is a critical piece to every codebase. Using Bazel as our foundation, we're able to validate OSS usage at build time and produce a BOM containing all OSS used in a code base, giving developers immediate feedback, removing tedious extra steps, and providing a single cross-platform, product agnostic tool.

We encourage you to engage in the Bazel community to see what kind of OSS compliance solution would work for the ecosystem. Please visit ourGitHub project repo, check out the code and open an issue to continue this discussion!

Stay tuned to theOpen Source Blogand follow us onTwitterfor more deep dives into the world of open source contributing.