Since Akka is now a paid tool we also decided to move towards Pekko, so you can still use an open-source serialization helper.
Every message that leaves a JVM boundary in Pekko needs to be serialized first. However, the existing solutions for serialization in Scala leave a lot of room for runtime failures. These failures are the result of programmer's oversights. Such oversights aren’t reported as possible runtime errors in compile time. This is why VirtusLab is glad to introduce the Pekko Serialization Helper. It’s a toolkit with a Circe-based, runtime-safe serializer and a set of Scala compiler plugins to counteract the common caveats in Pekko serialization.
Although Pekko is a great tool to work with, it also has some downsides when it comes to serialization. Several situations might cause unexpected errors when working with standard Pekko serialization, such as:
- Missing serialization binding
- Incompatibility of persistent data
- Jackson Pekko serializer drawbacks
- Missing codec registration
The similarity between these situations are bugs in the application code caused by programmer’s mistakes. The Scala compiler, on its own, passes over these bugs, which can easily break your app in runtime. Fortunately, there is a way to catch them during compilation with Pekko Serialization Helper, or PSH.
How to enable Pekko Serialization Helper in your project
Before we can use Pekko Serialization Helper, we need to add the following line to the project/plugins.sbt file:
scala
addSbtPlugin("org.virtuslab.psh" % "sbt-pekko-serialization-helper" % Version)You can find the newest Version in PSH GitHub Releases.
Once this is done, let’s enable the sbt plugin in the target project:
scala
lazy val app = (project in file("app"))
.enablePlugins(PekkoSerializationHelperPlugin)
Pekko-specific objects that get serialized include: Messages, Events and States. We might encounter runtime errors during serialization. Pekko Serialization Helper assists with spotting these errors and avoiding them in runtime. Let’s see common runtime errors related to Pekko serialization.
If you want to read more about events and persistent states, follow this link.
1. Missing serialization binding
A proper serialization in Pekko follows a certain concept:
First, you need to define a Scala trait, to serialize a message, persistent state or event:
scala
package org
trait MySerSecond, bind a serializer to this trait in a configuration file:
scala
pekko.actor {
serializers {
jackson-json = "pekko.serialization.jackson.JacksonJsonSerializer"
}
serialization-bindings {
"org.MySer" = jackson-json
}
}A serialization error in runtime occurs if a class is not extended with the base trait bound to the serializer:
scala
trait MySer
case class MyMessage() // extends MySerNow let’s wire up Pekko Serialization Helper. The serializability-checker-plugin, part of PSH, detects messages, events and persistent states. It checks whether they extend the given base trait and reports an error when they don't.
This ensures that Pekko uses the specified serializer. The serializer protects a running application against an unintended fallback to Java serialization or outright serialization failure.
This plugin requires you to add an @SerializabilityTrait annotation to the base trait:
scala
@SerializabilityTrait
trait MySerializable
// It allows catching errors like these:
import Pekko.actor.typed.Behavior
object BehaviorTest {
sealed trait Command //extends MySerializable
def method(msg: Command): Behavior[Command] = ???
}If we enable serializability-checker-plugin and add an @SerializabilityTrait annotation to the base trait, the compiler will be able to catch errors like this during compilation:
console
test0.scala:7: error: org.random.project.BehaviorTest.Command is used as Pekko message
but does not extend a trait annotated with org.virtuslab.psh.annotation.SerializabilityTrait.
Passing an object of a class that does NOT extend a trait annotated with SerializabilityTrait as a message may cause Pekko to
fall back to Java serialization during runtime.
def method(msg: Command): Behavior[Command] = ???
^
test0.scala:6: error: Make sure this type is itself annotated, or extends a type annotated
with @org.virtuslab.psh.annotation.SerializabilityTrait.
sealed trait Command extends MySerializable
^
A common problem with persistence in Pekko is the incompatibility of already persisted data with schemas defined in a new version of an application.
The solution for this incompatibility is the dump-persistence-schema-plugin – another part of Pekko Serializer Helper toolbox. It is a mix of a compiler plugin and a sbt task. The plugin can be used for dumping schema of pekko-persistence to a file and detecting accidental changes of events (journal) and states (snapshots) with a simple diff.
If you want to dump a persistence schema for each sbt module where Pekko Serialization Helper Plugin is enabled, run:
bash
sbt ashDumpPersistenceSchemaIt saves the created dump into a yaml file. The default is target/<sbt-module-name>-dump-persistence-schema-<version>.yaml
Example dump
yaml
- name: org.random.project.Data
typeSymbol: trait
- name: org.random.project.Data.ClassTest
typeSymbol: class
fields:
- name: a
typeName: java.lang.String
- name: b
typeName: scala.Int
- name: c
typeName: scala.Double
parents:
- org.random.project.Data
- name: org.random.project.Data.ClassWithAdditionData
typeSymbol: class
fields:
- name: ad
typeName: org.random.project.Data.AdditionalData
parents:
- org.random.project.DataThen, a simple diff command can be used to check the difference between the version of a schema from develop/main branch and the version from the current commit. Such comparison lets us catch possible incompatibilities of persisted data.
3. Jackson Pekko Serializer drawbacks
One more pitfall is to use the Jackson Serializer for Pekko. Let’s dive into some examples that might occur when combining Jackson with Scala code:
Jackson Serializer – Scala example 1
Let’s take a look at a dangerous code for Jackson:
scala
case class Message(animal: Animal) extends MySer
sealed trait Animal
final case class Lion(name: String) extends Animal
final case class Tiger(name: String) extends AnimalThis code seems to be alright, but unfortunately it will not work with the Jackson serialization. At runtime, there will be an exception with a message such as: “Cannot construct instance of Animal(...)”. The reason behind it is that abstract types need to be mapped to concrete types explicitly in code. If you want to make this code work, you need to add a lot of Jackson annotations:
scala
case class Message(animal: Animal) extends MultiDocPrintService
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type")
@JsonSubTypes(
Array(
new JsonSubTypes.Type(value = classOf[Lion], name = "lion"),
new JsonSubTypes.Type(value = classOf[Tiger], name = "tiger")))
sealed trait Animal
final case class Lion(name: String) extends Animal
final case class Tiger(name: String) extends AnimalJackson Serializer – Scala example 2
If a Scala object is defined:
scala
case object TickThen there will be no exceptions during serialization. During deserialization though, Jackson will create another instance of object Tick’s underlying class instead of restoring the object Tick’s underlying singleton. This means, the deserialization will end up in an unexpected but unreported behavior…
scala
actorRef ! Tick
// Inside the actor:
def receive = {
case Tick => // this won't get matched !!
} // message will be unhandled !!Pekko Serialization Helper as alternative
Pekko Serialization Helper provides a more Scala-friendly serializer that uses Circe.
Use our Circe-based Pekko serializer, to get rid of problems as shown in the examples above. Circe Pekko Serializer comes with Pekko Serialization Helper toolbox. It uses Circe codecs that are derived using Shapeless and are generated during compilation. This ensures that the serializer doesn’t crash at runtime, as reflection-based serializers might do.
The Circe Pekko Serializer is easy to use, just add the following lines to project dependencies:
scala
import org.virtuslab.psh.PekkoSerializationHelperPlugin
lazy val app = (project in file("app"))
// ...
.settings(libraryDependencies += PekkoSerializationHelperPlugin.circePekkoSerializer)Then create a custom serializer by extending Circe Pekko Serializer base class:
scala
import org.virtuslab.psh.circe.CircePekkoSerializer
class ExampleSerializer(actorSystem: ExtendedActorSystem)
extends CircePekkoSerializer[MySerializable](actorSystem) {
override def identifier: Int = 41
override lazy val codecs = Seq(Register[CommandOne], Register[CommandTwo])
override lazy val manifestMigrations = Nil
override lazy val packagePrefix = "org.project"
}MySerializable in the example above is the name of the base trait
Last but not least, remember to add your custom serializer to the Pekko configuration. Shortly, add two following configurations to the .conf file - pekko.actor.serializers and pekko.actor.serialization-bindings:
scala
pekko {
actor {
serializers {
circe-json = "org.example.ExampleSerializer"
}
serialization-bindings {
"org.example.MySerializable" = circe-json
}
}
}From now on you’ll have a safe Circe-based serializer to cope with the serialization of your objects.
4. Missing Codec registration
Last situation in which unexpected runtime exceptions might occur during serialization is the missing registration of a codec.
scala
import org.virtuslab.psh.circe.CircePekkoSerializer
import org.virtuslab.psh.circe.Register
class ExampleSerializer(actorSystem: ExtendedActorSystem)
extends CircePekkoSerializer[MySerializable](actorSystem) {
// ...
override lazy val codecs = Seq(Register[CommandOne]) // WHOOPS someone forgot to register CommandTwo...
}console
java.lang.RuntimeException: Serialization of [CommandTwo] failed. Call Register[A]
for this class or its supertype and append the result to `def codecs`.
Pekko Serialization Helper can help by using the @Serializer annotation.
Pekko Serialization Helper toolbox includes the codec-registration-checker-plugin. It gathers all direct descendants of the class marked with @SerializabilityTrait during compilation and checks the body of classes annotated with @Serializer for any reference of these direct descendants.
Let’s take a look at how we can apply the plugin to check a class extending CircePekkoSerializer:
scala
import org.virtuslab.psh.circe.CircePekkoSerializer
import org.virtuslab.psh.circe.Register
@Serializer(
classOf[MySerializable],
typeRegexPattern = Register.REGISTRATION_REGEX)
class ExampleSerializer(actorSystem: ExtendedActorSystem)
extends CircePekkoSerializer[MySerializable](actorSystem) {
// ...
override lazy val codecs = Seq(Register[CommandOne]) // WHOOPS someone forgot to register CommandTwo...
// ... but Codec Registration Checker will throw a compilation error here:
// `No codec for `CommandOne` is registered in a class annotated with @org.virtuslab.psh.annotation.Serializer`
}The Plugin catches all missing codec registrations in compile-time.
Summary
Pekko Serialization Helper is the right tool to make Pekko serialization bulletproof by catching possible runtime exceptions during compilation. It is free to use and easy to configure. Moreover, it is already used in commercial projects, although it has not reached full maturity yet. If you want to know more, check out PSH readme on GitHub.
Curated by
Sebastian Synowiec
