Published: Sep 22, 2022|15 min read15 minutes read
The entirely redesigned metaprogramming is a standout new feature of Scala 3. An additional dependency called “scala-reflect” made a few experimental yet widely used macro features available in Scala 2. However, redesigned macros and other newly added constructs now have built-in stable support in Scala 3. Of these, inline and transparent inline look the most intriguing.
But before we get ahead of ourselves, let’s start with the basics.
In a nutshell, metaprogramming is the activity of writing programs that modify and output other programs. You would be correct if you think that sounds suspiciously like a compiler’s job. But the difference is that metaprograms (created with metaprogramming) are executed at compile time and output programs that run at runtime.
There are many reasons why you would want to do this. Through some preprocessing, you may want to accelerate the user-facing runtime speed at the cost of developer-facing compilation speed. You may want to introduce more easily readable language constructs, like many metaprogramming libraries already do (e.g. quicklens). Or you want to automate some of the developer experience, deriving structures for use in other parts of the program. Finally, you might want to safely check semantic correctness by adding custom compile-time assertions.
None of the gains mentioned above matter if there is considerable friction in the metaprogramming development experience. This is why Scala 3 simplifies it and makes it more consistent with the rest of the language while still keeping its power.
The only direct compiler-supported form of metaprogramming in Scala 2 was the experimental support for macros, first introduced in Scala 2.10. However, this article includes the macro syntax for Scala version 2.11 and on, following a significant redesign. A basic example program containing a Scala 2 macro follows:
15 // unpack a static value from the method argument
16 val Literal(Constant(constValue: Int)) = number.tree
17
18 // construct a NonZeroNumber based on that value
19 if (constValue > 0) {
20 q"_root_.example.util.Positive($number)"
21 } else if (constValue < 0) {
22 q"_root_.example.util.Negative($number)"
23 } else {
24 c.abort(c.enclosingPosition, "Non zero value expected")
25 }
26 }
27}
1// example/util/NonZeroNumber.scala
2package example.util
3sealed trait NonZeroNumber
4case class Positive(value: Int) extends NonZeroNumber
5case class Negative(value: Int) extends NonZeroNumber
1// main.scala
2object Main {
3 def main(args: Array[String]): Unit = {
4 println(Macros.nonZeroNum(5)) // Positive(5)
5 println(Macros.nonZeroNum(15)) // Positive(15)
6 println(Macros.nonZeroNum(-5)) // Negative(-5)
7 // println(Macros.nonZeroNum(0)) // will cause a compile time error
As you can see, it works by running a method that then returns a program tree during the compilation, inlining the result into the runtime code. Notice how the returned code, while undoubtedly easier to write in the q”code” format (which is called a quasiquote) than directly constructing a program tree, can be somewhat surprising. The code is inlined “as is”, meaning you have to be very precise about the location of classes and objects used. Additionally, it is worth noting that a macro definition and a call to it cannot be located in the same compilation run, meaning that you will most likely have to create a separate project just for macro methods.
But even outside of those two things, couldn’t the same effect be achieved more simply? So let’s look at what Scala 3 brings to the table.
The newly introduced inline keyword before a def guarantees that any call to a method will be expanded directly into the method contents. It also provides access to a wealth of metaprogramming features, one of which is inline parameters. Inline parameters are used by putting the inline keyword before the definition of a method parameter, like this:
1inline def method(inline value: Int) = …
This means that the compiler replaces references of that parameter with its contents, which can be used to perform operations on constants in the compile time. This also includes folding if and match statements. We can ensure that they will be simplified by adding an inline keyword before them (inline if … and inline match …), which throws compile-time errors if constant folding cannot be done.
Inline methods also allow us to use scala.compiletime operations, like throwing custom compile-time errors. These errors will be thrown if it’s not removed as part of the unreachable code after inlining.
Let’s quickly try rewriting the previous Scala 2 macro, using only the Scala 3 operations:
7 scala.compiletime.error("Non zero value expected")
8 }
9}
As you can see, this was much easier to write and read than the initial Scala 2 example. We also did not have to split up the code into separate subprojects. Instead, we can organize the codebase however we want.
You may notice in the example above that despite knowing the precise returned type in the compile-time, we always resolve it to the NonZeroNumber type. This may be a problem, as later, we may have to manually cast to Positive or Negative types, defeating the entire purpose of the nonZeroNum assertion. For that reason, a transparent keyword was also introduced, which allows the compiler to decide the returned type after completing the expansion and constant folding of a method. The transparent keyword can be used like this:
7 scala.compiletime.error("Non zero value expected")
8 }
9}
Now val pos: Positive = nonZeroNum(5) would compile and val neg: Negative = nonZeroNum(5) would fail, just as we would expect. Without the transparent keyword both would fail, with only assigning the NonZeroNumber type being able to work, like in val pos: NonZeroNumber = nonZeroNum(5).
People experienced with Scala 2 macros may notice that inline def corresponds to blackbox macros, and transparent inline def corresponds to whitebox macros.
More complex metaprogramming methods may be impractical or impossible to write using only inline and scala.compiletime operations. Thankfully, macros also return, with improved semantics. As a basic comparison, the initial Scala 2 macro in Scala 3 can be rewritten like this:
10 // unpack a static value from the method argument
11 val constValue = value.valueOrAbort
12
13 // construct a NonZeroNumber based on that value
14 if (constValue > 0) {
15 '{ Positive($value) }
16 } else if (constValue < 0) {
17 '{ Negative($value) }
18 } else {
19 quotes.reflect.report.errorAndAbort("Non zero value expected")
20 }
21 }
22}
This time, while both a macro definition and a call to it cannot be put in the same file, it’s enough for them to be in two separate files as part of the same compilation run. More importantly, previously used quasiquotes were completely removed and replaced with all-new constructs: splices (denoted as ${code} or $symbol) and quotations (denoted as '{code} or 'symbol). While they both contain code, they are otherwise very different:
Splices contain code run during compile-time.
Quotations contain code to be run during runtime and which, before that, is resolved into a program tree.
Splices allow to, well, splice additional code into a quotation.
Quotations return an Expr[_] type representing the program tree.
Splices are also an entry point for a macro method implementation and this implementation must return an Expr[_] type.
All the above could have been done similarly through Scala 2’s quasiquotes, so what was the reason for those changes? Well, the most crucial part is that, unlike quasiquotes, quoted code blocks are typed, and their typing is consistent with the macro method itself.
Every splice generates a given Quotes instance for this consistency to happen, and every quoted block requires it. This is why we included a (using Quotes) in the method signature in both code snippets above. The Quotes object also becomes a gateway to the Quotes reflect API – a low-level API where we can construct program trees directly. We will come back to this in a while.
This redesign has many significant benefits. It allows the compiler to treat the quoted code inside the metaprogram in the same manner as any other code. This means that, unlike quasiquotes, a developer of a quoted code can get full IDE support, with completions, easy access to definitions etc., smoothing the development experience. Additionally, this results in fewer surprises regarding the types after inlining. You may recall that in Scala 2, we had to be precise about the namespaces and locations of the constructs used. Now, we can import something in the compilation scope and directly use it in the inlined quotation. Many errors that previously could only be found by the compiler after inlining a call to a macro will now be found earlier during compilation. Also, as always, it’s easier to track statically typed code – I am sure any Scala developer will agree.
Typed quotations illustrated
Since the first example was quite basic, in Scala 3, there is no point in using a macro when we can just use transparent inline and inline. For illustrative purposes, let’s do something more interesting. This time we will implement a CNF (Conjunctive Normal Form) mapper, which from a CNF string and a list of boolean expressions will create a resulting boolean expression. This will allow us to easily represent a scala logical formula in a canonical form. Perhaps it is easiest to think of it as a printf, except for booleans, where a method call like cnf("(0v1)^(2v-3)", true, false, boolVal, false) will be compiled to (true || false) && (boolVal || !false) and possibly simplified further by the compiler. For simplicity’s sake, we will not be checking the string input format for correctness.
As you can see, spliced Expr[T] (here Expr[Boolean]) becomes a T type (here Boolean) in a quoted code block, while all of the quoted code blocks in the example above are of type Expr[Boolean] due to returning Boolean`s.
Scala 3 quotations and Scala 2 quasiquotes can also be used to pattern match code, with frequent use cases analyzing structures passed into the macro method.
Accessing reflection API
As mentioned before, sometimes you may need to go more low level with the generated code constructs. For that purpose, both Scala 2 and 3 allow you to create custom program trees by hand. This tool provides the most freedom when creating macros (or even in Scala 3 metaprogramming in general) but is also the most difficult to tame. In Scala 3, it is contained inside of the quotes.reflect package (where quotes is a Quotes object instance created via a splice, as discussed before), and Scala 2 requires you to use the contents of context.universe (where context is a Context object created on the entry point of the macro). This means that in Scala 3, you must be careful which Quotes instance you use since every splice generates one. For example, something that was roughly implemented in Scala 2 like this:
1def example(c: Context) = {
2 import c.universe._
3 val customAst: tree = {
4 // create a custom AST tree
5 }
6
7 q"""
8 code
9 ${customAst}
10 code
11 """
12}
Has to be rewritten to something like this in Scala 3:
1def example(using Quotes) = {
2 '{
3 code
4 ${customAst()}
5 code
6 }
7}
8
9def customAst(using Quotes) = { // different Quotes instance than in the example method
10 import quotes.reflect._
11 // create a custom program tree
12}
In other words, in Scala 3, every custom program tree can only be inserted into a splice from which it was derived in the first place. This is a small price for all the convenience typed quotations give us.
Taking Scala 3’s metaprogramming further
As you can see, Scala 3 introduces many new metaprogramming concepts for a wide range of developers.
Inlines and transparent inlines drastically lower the barrier of entry for metaprogramming.
Macro methods allow for the same degree of freedom as before but with improved semantics. This helps with readability and keeps it consistent with the rest of the Scala language.
Lastly, there are still many concepts that are left unexplored in this blog post. One example is that inline functionality can be extended using compiler-generated Mirror type classes to obtain basic case class and case object type information. This, in several instances, may help you avoid macros altogether. In addition, multi-stage programming was introduced, using quotations and splices at runtime instead of compiling time. All of the above can be explored in the official Scala 3 documentation, which also provides a more thorough overview of macros and the Quotes reflect API, which we just barely touched upon here. The examples used in this blog post are all available in a GitHub repository.
