The build time of a project has a significant impact on a team’s development efficiency. The larger the code base, the longer it takes to build. And the longer the build time, the worse the developer experience becomes.
While SBT is a great build tool, its design (in particular the lack of reliable content-addressable build cache) makes it not very suited for large projects.
In this blog post I will share my expertise in Bazel, a build system to achieve fast builds, even in Google-scale repositories, through building simple Scala applications.
What is Bazel?
What is Bazel? It is an artifact-based build system that uses a declarative list of artifacts to build, specifies dependencies, and offers limited options for the build process.
This design allows Bazel to achieve its qualities, as in its motto "{Fast, Correct} - Choose two."
In this section, we will learn about the nature of Bazel by looking at what an artifact-based build system is and what “{Fast, Correct} choose two” means!
Artifact-based build system
Traditional build systems like Ant and Maven are called task-based build systems. In the build configuration for task-based build systems, we describe an imperative set of tasks like do task A, then do task B, and then do task C.
On the other hand, in artifact-based build systems such as Buck, Pants, and Bazel, we describe a declarative set of artifacts to build, a list of dependencies, and limited options for the build.
So, the basic idea of Bazel is your build is a pure function:
- Sources and dependencies are input.
- The artifact is an output.
- There are no side effects.
For more details about the concept of artifact-based build systems, I recommend reading through Chapter 18 of Software Engineering at Google.
{Fast, Correct} choose two
To better understand this claim, it is necessary to comprehend Bazel’s Hermeticity property.
In hermetic build systems like Bazel, when given the same input sources and configuration, it returns the same output.
Thanks to hermeticity, Bazel is able to provide reproducible builds: given the same inputs, Bazel always returns the same output on everyone’s computer.
And thanks to the reproducible build, Bazel can provide a remote cache feature to share the build cache within a team. With remote cache, we can build large projects fast by using the build cache shared across team members. Build reproducibility is also a critical aspect of supply chain security, required to achieve the highest level of SLSA compliance.
Bazel basics
In this tutorial, we are building a simple Scala application with Bazel. The complete source code is available here: https://github.com/tanishiking/bazel-tutorial-scala/tree/main/01_scala_tutorial
The project structure looks like this:
The Bazel configuration files are WORKSPACE and BUILD.bazel files.
- The WORKSPACE file is about getting stuff from the outside world into your Bazel project.
- BUILD.bazel files are about what is happening inside your Bazel project.
The WORKSPACE file
The WORKSPACE file contains the external dependencies (for both Bazel and JVM). For example, we download rules_scala, a Bazel extension for compiling Scala in the WORKSPACE file.
For more details, see the project README.
BUILD.bazel files
To define the build in Bazel, we write BUILD.bazel files.
Before jumping into the BUILD.bazel files, let’s take a quick look at Scala files to build. This project consists of two packages, each containing a single Scala file.
As you can see, lib.Greeting is a library module that provides the sayHi method, and cmd.Runner depends on lib.Greeting.
Next, let’s see how to write BUILD.bazel files to build these Scala sources.
scala_library
To build lib.Greeting in this example, we put the BUILD.bazel file adjacent to Greeting.scala, and we define a build target using the scala_library rule provided by rules_scala.
A rule in Bazel is a declaration of a set of instructions for building or testing code. For example, there’s a set of rules for building Java programs (that is natively supported by Bazel). rules_scala provides a set of rules for building Scala programs.
scala_library compiles the given Scala sources and generates a JAR file.
- The load statement imports a scala_library rule to the BUILD.bazel file.
- scala_library is one of the build rules in Bazel; we describe what to build using rules.
- The required attributes of scala_library are name and srcs.
bazel build
Now we have Scala sources to build and a BUILD.bazel configuration. Let’s build it using the bazel command line.
Build succeeded 🎉, but wait, what is //src/main/scala/lib:greeting?
Label
//src/main/scala/lib:greeting is something called a label in Bazel, and it points to the greeting in src/main/scala/lib/BUILD.bazel. In Bazel, we use a label to uniquely identify a build target.
A label consists of 3 components. For example, in @myrepo//my/app/main:app_binary,
- @myrepo// specifies a repository name. If we omit this part, the label refers to the repository that contains this BUILD.bazel file. Therefore, we can omit the @myrepo part when referring to a label defined within the same repository.
- my/app/main represents a path to the package (BUILD.bazel file) relative to the repository root.
- :app_binary is a target name.
That being said, //src/main/scala/lib:greeting points to a target that is in the same workspace, defined in a BUILD.bazel file located at src/main/scala/lib, and the target name is greeting.
Dependencies
Next, let’s build cmd.Runner that depends on lib.Greeting. This time, cmd.Runner depends on lib.Greeting, so we introduce a dependency between targets using a deps attribute.
The differences from the previous example are:
- We use scala_binary instead of scala_library
- scala_binary is an executable rule. Executable rules define how to build an executable from sources. This process might contain linking dependencies or listing the classpaths of dependencies.
- For example, the scala_binary rule builds an executable script from the sources and dependencies.
- Once the executable has been built, you can run it using the bazel run command. This will execute the executable file.
- We add the deps attribute to list all the dependencies.
- In this example, we add a label //src/main/scala/lib:greeting because cmd.Runner depends on lib.Greeting.
Now, we should be able to build the application by bazel build //… but it fails!
Visibility
Bazel has a concept of visibility. By default, all targets’ visibility is private, meaning only targets within the same package (i.e. same BUILD.bazel file) can access each other.
To make lib:greeting visible from cmd, add the visibility attribute to greeting.
//src/main/scala/cmd:__pkg__ is a visibility specification that grants access to the package //src/main/scala/cmd.
Now we can build the app:
As you can see, the scala_binary rule generates another file which is named runner in addition to runner.jar. This is a wrapper script for runner.jar, and we can easily run the JAR with this script.
Tip: build multiple targets
In the examples above, we specify a target’s label and build one target, but is it possible to build multiple build targets at the same time?
The answer is yes. We can use a wildcard to select multiple targets. For example, we can build all targets by $ bazel build //….
Now, we have learned the basics of Bazel by building a simple application in Scala, but how do we use third-party libraries in Bazel?
External JVM dependencies
Let’s now learn how to use third-party libraries from Maven by building a simple application that parses Scala programs using scalameta and pretty prints the AST using pprint.
In this example, we will use rules_jvm_external, one of the standard rules set to manage external JVM dependencies.
Note: we can download jars from Maven repositories using maven_jar, a natively supported rule by Bazel. However, I recommend using rules_jvm_extenal because it has a number of useful features over maven_jar.
The complete code example is available here: https://github.com/tanishiking/bazel-tutorial-scala/tree/main/02_scala_maven.
This project has only one Scala program.
To download scalameta and pprint from Maven repositories, we use rules_jvm_external. So, we have to download rules_jvm_external first.
To download rules_jvm_external, copy and paste setup statements from the release page to your WORKSPACE file, like this:
Now, Bazel can download the dependencies, but how can we use them?
To use the downloaded dependencies, we need to add them to the deps attribute of the build rules. rules_jvm_external automatically generates targets for the libraries under @maven repository in the following format:
> The default label syntax for an artifact foo.bar:baz-qux:1.2.3 is @maven//:foo_bar_baz_qux https://github.com/bazelbuild/rules_jvm_external#usage
Therefore, we can refer to com.lihaoyi:pprint_2.13:0.7.3 with the label @maven//:com_lihaoyi_pprint_2_13. So, put the following BUILD.bazel file adjacent to App.scala.
And build it!
Nice! 🎉 So, that’s how we use external JVM dependencies with rules_jvm_external.
Conclusion
In this article, we have shown how Bazel enables fast build in large repositories and introduced the basic concepts and usage of Bazel by building a simple Scala application.
As you saw, Bazel requires users to manage a lot more build configurations than other build tools such as sbt and Maven. However, this is an acceptable tradeoff for very large projects. considering the scalable build speed thanks to Bazel’s advantages, such as reproducible builds and remote cache.
I hope this article helps you take the first steps in getting started with Bazel.
If you want to learn more about Bazel, I recommend reading through the official Bazel starting guide and getting your hands dirty with your first Bazel project. There are many more interesting topics (such as Bazel testing) to learn about and explore!
