Serialization with Jackson

Dependency

To use Jackson Serialization, you must add the following dependency in your project:

sbt
val PekkoVersion = "1.0.3"
libraryDependencies += "org.apache.pekko" %% "pekko-serialization-jackson" % PekkoVersion
Maven
<properties>
  <scala.binary.version>2.13</scala.binary.version>
</properties>
<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.apache.pekko</groupId>
      <artifactId>pekko-bom_${scala.binary.version}</artifactId>
      <version>1.0.3</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>
<dependencies>
  <dependency>
    <groupId>org.apache.pekko</groupId>
    <artifactId>pekko-serialization-jackson_${scala.binary.version}</artifactId>
  </dependency>
</dependencies>
Gradle
def versions = [
  ScalaBinary: "2.13"
]
dependencies {
  implementation platform("org.apache.pekko:pekko-bom_${versions.ScalaBinary}:1.0.3")

  implementation "org.apache.pekko:pekko-serialization-jackson_${versions.ScalaBinary}"
}

Introduction

You find general concepts for for Pekko serialization in the Serialization section. This section describes how to use the Jackson serializer for application specific messages and persistent events and snapshots.

Jackson has support for both text based JSON and binary formats.

In many cases ordinary classes can be serialized by Jackson without any additional hints, but sometimes annotations are needed to specify how to convert the objects to JSON/bytes.

Usage

To enable Jackson serialization for a class you need to configure it or one of its super classes in serialization-bindings configuration. Typically you will create a marker traitinterface for that purpose and let the messages extendimplement that.

Scala
source/**
 * Marker interface for messages, events and snapshots that are serialized with Jackson.
 */
trait MySerializable

final case class Message(name: String, nr: Int) extends MySerializable
Java
source/** Marker interface for messages, events and snapshots that are serialized with Jackson. */
public interface MySerializable {}

class MyMessage implements MySerializable {
  public final String name;
  public final int nr;

  public MyMessage(String name, int nr) {
    this.name = name;
    this.nr = nr;
  }
}

Then you configure the class name of the marker traitinterface in serialization-bindings to one of the supported Jackson formats: jackson-json or jackson-cbor

sourcepekko.actor {
  serialization-bindings {
    "com.myservice.MySerializable" = jackson-json
  }
}

A good convention would be to name the marker interface CborSerializable or JsonSerializable. In this documentation we have used MySerializable to make it clear that the marker interface itself is not provided by Pekko.

That is all that is needed for basic classes where Jackson understands the structure. A few cases that requires annotations are described below.

Note that it’s only the top level class or its marker traitinterface that must be defined in serialization-bindings, not nested classes that it references in member fields.

Note

Add the -parameters Java compiler option for usage by the ParameterNamesModule. It reduces the need for some annotations.

Security

For security reasons it is disallowed to bind the Jackson serializers to open ended types that might be a target for serialization gadgets, such as:

The deny list of possible serialization gadget classes defined by Jackson databind are checked and disallowed for deserialization.

Warning

Don’t use @JsonTypeInfo(use = Id.CLASS) or ObjectMapper.enableDefaultTyping since that is a security risk when using polymorphic types.

Formats

The following formats are supported, and you select which one to use in the serialization-bindings configuration as described above.

  • jackson-json - ordinary text based JSON
  • jackson-cbor - binary CBOR data format

The binary format is more compact, with slightly better performance than the JSON format.

Annotations

Constructor with single parameter

You might run into an exception like this:

MismatchedInputException: Cannot construct instance of `...` (although at least one Creator exists): cannot deserialize from Object value (no delegate- or property-based Creator)

That is probably because the class has a constructor with a single parameter, like:

Java
sourcepublic class SimpleCommand implements MySerializable {
  private final String name;

  public SimpleCommand(String name) {
    this.name = name;
  }
}

That can be solved by adding @JsonCreator or @JsonProperty annotations:

Java
sourcepublic class SimpleCommand implements MySerializable {
  private final String name;

  @JsonCreator
  public SimpleCommand(String name) {
    this.name = name;
  }
}

or

Java
sourcepublic class SimpleCommand implements MySerializable {
  private final String name;

  public SimpleCommand(@JsonProperty("name") String name) {
    this.name = name;
  }
}

The ParameterNamesModule is configured with JsonCreator.Mode.PROPERTIES as described in the Jackson documentation

Polymorphic types

A polymorphic type is when a certain base type has multiple alternative implementations. When nested fields or collections are of polymorphic type the concrete implementations of the type must be listed with @JsonTypeInfo and @JsonSubTypes annotations.

Example:

Scala
sourcefinal case class Zoo(primaryAttraction: Animal) extends MySerializable

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type")
@JsonSubTypes(
  Array(
    new JsonSubTypes.Type(value = classOf[Lion], name = "lion"),
    new JsonSubTypes.Type(value = classOf[Elephant], name = "elephant")))
sealed trait Animal

final case class Lion(name: String) extends Animal

final case class Elephant(name: String, age: Int) extends Animal
Java
sourcepublic class Zoo implements MySerializable {
  public final Animal primaryAttraction;

  @JsonCreator
  public Zoo(Animal primaryAttraction) {
    this.primaryAttraction = primaryAttraction;
  }
}

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type")
@JsonSubTypes({
  @JsonSubTypes.Type(value = Lion.class, name = "lion"),
  @JsonSubTypes.Type(value = Elephant.class, name = "elephant")
})
interface Animal {}

public final class Lion implements Animal {
  public final String name;

  @JsonCreator
  public Lion(String name) {
    this.name = name;
  }
}

public final class Elephant implements Animal {
  public final String name;
  public final int age;

  public Elephant(String name, int age) {
    this.name = name;
    this.age = age;
  }
}

If you haven’t defined the annotations you will see an exception like this:

InvalidDefinitionException: Cannot construct instance of `...` (no Creators, like default construct, exist): abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information

Note that this is not needed for a top level class, but for fields inside it. In this example Animal is used inside of Zoo, which is sent as a message or persisted. If Animal was sent or persisted standalone the annotations are not needed because then it is the concrete subclasses Lion or Elephant that are serialized.

When specifying allowed subclasses with those annotations the class names will not be included in the serialized representation and that is important for preventing loading of malicious serialization gadgets when deserializing.

Warning

Don’t use @JsonTypeInfo(use = Id.CLASS) or ObjectMapper.enableDefaultTyping since that is a security risk when using polymorphic types.

ADT with trait and case object

It’s common in Scala to use a sealed trait and case objects to represent enums. If the values are case classes the @JsonSubTypes annotation as described above works, but if the values are case objects it will not. The annotation requires a Class and there is no way to define that in an annotation for a case object.

The easiest workaround is to define the case objects as case class without any field.

Alternatively, you can define an intermediate trait for the case object and a custom deserializer for it. The example below builds on the previous Animal sample by adding a fictitious, single instance, new animal, an Unicorn.

Scala
sourcefinal case class Zoo(primaryAttraction: Animal) extends MySerializable

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type")
@JsonSubTypes(
  Array(
    new JsonSubTypes.Type(value = classOf[Lion], name = "lion"),
    new JsonSubTypes.Type(value = classOf[Elephant], name = "elephant"),
    new JsonSubTypes.Type(value = classOf[Unicorn], name = "unicorn")))
sealed trait Animal

final case class Lion(name: String) extends Animal
final case class Elephant(name: String, age: Int) extends Animal

@JsonDeserialize(`using` = classOf[UnicornDeserializer])
sealed trait Unicorn extends Animal
@JsonTypeName("unicorn")
case object Unicorn extends Unicorn

class UnicornDeserializer extends StdDeserializer[Unicorn](Unicorn.getClass) {
  // whenever we need to deserialize an instance of Unicorn trait, we return the object Unicorn
  override def deserialize(p: JsonParser, ctxt: DeserializationContext): Unicorn = Unicorn
}

The case object Unicorn can’t be used in a @JsonSubTypes annotation, but its trait can. When serializing the case object we need to know which type tag to use, hence the @JsonTypeName annotation on the object. When deserializing, Jackson will only know about the trait variant therefore we need a custom deserializer that returns the case object.

On the other hand, if the ADT only has case objects, you can solve it by implementing a custom serialization for the enums. Annotate the trait with @JsonSerialize and @JsonDeserialize and implement the serialization with StdSerializer and StdDeserializer.

Scala
sourceimport com.fasterxml.jackson.core.JsonGenerator
import com.fasterxml.jackson.core.JsonParser
import com.fasterxml.jackson.databind.DeserializationContext
import com.fasterxml.jackson.databind.SerializerProvider
import com.fasterxml.jackson.databind.annotation.JsonDeserialize
import com.fasterxml.jackson.databind.annotation.JsonSerialize
import com.fasterxml.jackson.databind.deser.std.StdDeserializer
import com.fasterxml.jackson.databind.ser.std.StdSerializer

@JsonSerialize(`using` = classOf[DirectionJsonSerializer])
@JsonDeserialize(`using` = classOf[DirectionJsonDeserializer])
sealed trait Direction

object Direction {
  case object North extends Direction
  case object East extends Direction
  case object South extends Direction
  case object West extends Direction
}

class DirectionJsonSerializer extends StdSerializer[Direction](classOf[Direction]) {
  import Direction._

  override def serialize(value: Direction, gen: JsonGenerator, provider: SerializerProvider): Unit = {
    val strValue = value match {
      case North => "N"
      case East  => "E"
      case South => "S"
      case West  => "W"
    }
    gen.writeString(strValue)
  }
}

class DirectionJsonDeserializer extends StdDeserializer[Direction](classOf[Direction]) {
  import Direction._

  override def deserialize(p: JsonParser, ctxt: DeserializationContext): Direction = {
    p.getText match {
      case "N" => North
      case "E" => East
      case "S" => South
      case "W" => West
    }
  }
}

final case class Compass(currentDirection: Direction) extends MySerializable

Enumerations

Jackson support for Scala Enumerations defaults to serializing a Value as a JsonObject that includes a field with the "value" and a field with the "type" whose value is the FQCN of the enumeration. Jackson includes the @JsonScalaEnumeration to statically specify the type information to a field. When using the @JsonScalaEnumeration annotation the enumeration value is serialized as a JsonString.

Scala
sourceobject Planet extends Enumeration {
  type Planet = Value
  val Mercury, Venus, Earth, Mars, Krypton = Value
}

// Uses default Jackson serialization format for Scala Enumerations
final case class Alien(name: String, planet: Planet.Planet) extends TestMessage

// Serializes planet values as a JsonString
class PlanetType extends TypeReference[Planet.type] {}
final case class Superhero(name: String, @JsonScalaEnumeration(classOf[PlanetType]) planet: Planet.Planet)
    extends TestMessage

Schema Evolution

When using Event Sourcing, but also for rolling updates, schema evolution becomes an important aspect of developing your application. The requirements as well as our own understanding of the business domain may (and will) change over time.

The Jackson serializer provides a way to perform transformations of the JSON tree model during deserialization. This is working in the same way for the textual and binary formats.

We will look at a few scenarios of how the classes may be evolved.

Remove Field

Removing a field can be done without any migration code. The Jackson serializer will ignore properties that does not exist in the class.

Add Optional Field

Adding an optional field can be done without any migration code. The default value will be NoneOptional.empty.

Old class:

Scala
sourcecase class ItemAdded(shoppingCartId: String, productId: String, quantity: Int) extends MySerializable
Java
sourcepublic class ItemAdded implements MySerializable {
  public final String shoppingCartId;
  public final String productId;
  public final int quantity;

  public ItemAdded(String shoppingCartId, String productId, int quantity) {
    this.shoppingCartId = shoppingCartId;
    this.productId = productId;
    this.quantity = quantity;
  }
}

New class with a new optional discount property and a new note field with default value:

Scala
sourcecase class ItemAdded(shoppingCartId: String, productId: String, quantity: Int, discount: Option[Double], note: String)
    extends MySerializable {

  // alternative constructor because `note` should have default value "" when not defined in json
  @JsonCreator
  def this(shoppingCartId: String, productId: String, quantity: Int, discount: Option[Double], note: Option[String]) =
    this(shoppingCartId, productId, quantity, discount, note.getOrElse(""))
}
Java
sourcepublic class ItemAdded implements MySerializable {
  public final String shoppingCartId;
  public final String productId;
  public final int quantity;
  public final Optional<Double> discount;
  public final String note;

  @JsonCreator
  public ItemAdded(
      String shoppingCartId,
      String productId,
      int quantity,
      Optional<Double> discount,
      String note) {
    this.shoppingCartId = shoppingCartId;
    this.productId = productId;
    this.quantity = quantity;
    this.discount = discount;

    // default for note is "" if not included in json
    if (note == null) this.note = "";
    else this.note = note;
  }

  public ItemAdded(
      String shoppingCartId, String productId, int quantity, Optional<Double> discount) {
    this(shoppingCartId, productId, quantity, discount, "");
  }
}

Add Mandatory Field

Let’s say we want to have a mandatory discount property without default value instead:

Scala
sourcecase class ItemAdded(shoppingCartId: String, productId: String, quantity: Int, discount: Double) extends MySerializable
Java
sourcepublic class ItemAdded implements MySerializable {
  public final String shoppingCartId;
  public final String productId;
  public final int quantity;
  public final double discount;

  public ItemAdded(String shoppingCartId, String productId, int quantity, double discount) {
    this.shoppingCartId = shoppingCartId;
    this.productId = productId;
    this.quantity = quantity;
    this.discount = discount;
  }
}

To add a new mandatory field we have to use a JacksonMigrationJacksonMigration class and set the default value in the migration code.

This is how a migration class would look like for adding a discount field:

Scala
sourceimport com.fasterxml.jackson.databind.JsonNode
import com.fasterxml.jackson.databind.node.DoubleNode
import com.fasterxml.jackson.databind.node.ObjectNode
import org.apache.pekko.serialization.jackson.JacksonMigration

class ItemAddedMigration extends JacksonMigration {

  override def currentVersion: Int = 2

  override def transform(fromVersion: Int, json: JsonNode): JsonNode = {
    val root = json.asInstanceOf[ObjectNode]
    if (fromVersion <= 1) {
      root.set("discount", DoubleNode.valueOf(0.0))
    }
    root
  }
}
Java
sourceimport com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.DoubleNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import org.apache.pekko.serialization.jackson.JacksonMigration;

public class ItemAddedMigration extends JacksonMigration {

  @Override
  public int currentVersion() {
    return 2;
  }

  @Override
  public JsonNode transform(int fromVersion, JsonNode json) {
    ObjectNode root = (ObjectNode) json;
    if (fromVersion <= 1) {
      root.set("discount", DoubleNode.valueOf(0.0));
    }
    return root;
  }
}

Override the currentVersioncurrentVersion() method to define the version number of the current (latest) version. The first version, when no migration was used, is always 1. Increase this version number whenever you perform a change that is not backwards compatible without migration code.

Implement the transformation of the old JSON structure to the new JSON structure in the transform(fromVersion, jsonNode)transform(fromVersion, jsonNode) method. The JsonNode is mutable so you can add and remove fields, or change values. Note that you have to cast to specific sub-classes such as ObjectNode and ArrayNode to get access to mutators.

The migration class must be defined in configuration file:

sourcepekko.serialization.jackson.migrations {
  "com.myservice.event.ItemAdded" = "com.myservice.event.ItemAddedMigration"
}

The same thing could have been done for the note field, adding a default value of "" in the ItemAddedMigration.

Rename Field

Let’s say that we want to rename the productId field to itemId in the previous example.

Scala
sourcecase class ItemAdded(shoppingCartId: String, itemId: String, quantity: Int) extends MySerializable
Java
sourcepublic class ItemAdded implements MySerializable {
  public final String shoppingCartId;

  public final String itemId;

  public final int quantity;

  public ItemAdded(String shoppingCartId, String itemId, int quantity) {
    this.shoppingCartId = shoppingCartId;
    this.itemId = itemId;
    this.quantity = quantity;
  }
}

The migration code would look like:

Scala
sourceimport org.apache.pekko.serialization.jackson.JacksonMigration
import com.fasterxml.jackson.databind.JsonNode
import com.fasterxml.jackson.databind.node.ObjectNode

class ItemAddedMigration extends JacksonMigration {

  override def currentVersion: Int = 2

  override def transform(fromVersion: Int, json: JsonNode): JsonNode = {
    val root = json.asInstanceOf[ObjectNode]
    if (fromVersion <= 1) {
      root.set[JsonNode]("itemId", root.get("productId"))
      root.remove("productId")
    }
    root
  }
}
Java
source
import org.apache.pekko.serialization.jackson.JacksonMigration; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.node.ObjectNode; public class ItemAddedMigration extends JacksonMigration { @Override public int currentVersion() { return 2; } @Override public JsonNode transform(int fromVersion, JsonNode json) { ObjectNode root = (ObjectNode) json; if (fromVersion <= 1) { root.set("itemId", root.get("productId")); root.remove("productId"); } return root; } }

Structural Changes

In a similar way we can do arbitrary structural changes.

Old class:

Scala
sourcecase class Customer(name: String, street: String, city: String, zipCode: String, country: String) extends MySerializable
Java
sourcepublic class Customer implements MySerializable {
  public final String name;
  public final String street;
  public final String city;
  public final String zipCode;
  public final String country;

  public Customer(String name, String street, String city, String zipCode, String country) {
    this.name = name;
    this.street = street;
    this.city = city;
    this.zipCode = zipCode;
    this.country = country;
  }
}

New class:

Scala
sourcecase class Customer(name: String, shippingAddress: Address, billingAddress: Option[Address]) extends MySerializable
Java
sourcepublic class Customer implements MySerializable {
  public final String name;
  public final Address shippingAddress;
  public final Optional<Address> billingAddress;

  public Customer(String name, Address shippingAddress, Optional<Address> billingAddress) {
    this.name = name;
    this.shippingAddress = shippingAddress;
    this.billingAddress = billingAddress;
  }
}

with the Address class:

Scala
sourcecase class Address(street: String, city: String, zipCode: String, country: String) extends MySerializable
Java
sourcepublic class Address {
  public final String street;
  public final String city;
  public final String zipCode;
  public final String country;

  public Address(String street, String city, String zipCode, String country) {
    this.street = street;
    this.city = city;
    this.zipCode = zipCode;
    this.country = country;
  }
}

The migration code would look like:

Scala
sourceimport org.apache.pekko.serialization.jackson.JacksonMigration
import com.fasterxml.jackson.databind.JsonNode
import com.fasterxml.jackson.databind.node.ObjectNode

import scala.annotation.nowarn

class CustomerMigration extends JacksonMigration {

  override def currentVersion: Int = 2

  @nowarn("msg=deprecated")
  override def transform(fromVersion: Int, json: JsonNode): JsonNode = {
    val root = json.asInstanceOf[ObjectNode]
    if (fromVersion <= 1) {
      val shippingAddress = root.`with`("shippingAddress")
      shippingAddress.set[JsonNode]("street", root.get("street"))
      shippingAddress.set[JsonNode]("city", root.get("city"))
      shippingAddress.set[JsonNode]("zipCode", root.get("zipCode"))
      shippingAddress.set[JsonNode]("country", root.get("country"))
      root.remove("street")
      root.remove("city")
      root.remove("zipCode")
      root.remove("country")
    }
    root
  }
}
Java
sourceimport org.apache.pekko.serialization.jackson.JacksonMigration;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ObjectNode;

public class CustomerMigration extends JacksonMigration {

  @Override
  public int currentVersion() {
    return 2;
  }

  @Override
  public JsonNode transform(int fromVersion, JsonNode json) {
    ObjectNode root = (ObjectNode) json;
    if (fromVersion <= 1) {
      ObjectNode shippingAddress = root.with("shippingAddress");
      shippingAddress.set("street", root.get("street"));
      shippingAddress.set("city", root.get("city"));
      shippingAddress.set("zipCode", root.get("zipCode"));
      shippingAddress.set("country", root.get("country"));
      root.remove("street");
      root.remove("city");
      root.remove("zipCode");
      root.remove("country");
    }
    return root;
  }
}

Rename Class

It is also possible to rename the class. For example, let’s rename OrderAdded to OrderPlaced.

Old class:

Scala
sourcecase class OrderAdded(shoppingCartId: String) extends MySerializable
Java
sourcepublic class OrderAdded implements MySerializable {
  public final String shoppingCartId;

  @JsonCreator
  public OrderAdded(String shoppingCartId) {
    this.shoppingCartId = shoppingCartId;
  }
}

New class:

Scala
sourcecase class OrderPlaced(shoppingCartId: String) extends MySerializable
Java
sourcepublic class OrderPlaced implements MySerializable {
  public final String shoppingCartId;

  @JsonCreator
  public OrderPlaced(String shoppingCartId) {
    this.shoppingCartId = shoppingCartId;
  }
}

The migration code would look like:

Scala
sourceclass OrderPlacedMigration extends JacksonMigration {

  override def currentVersion: Int = 2

  override def transformClassName(fromVersion: Int, className: String): String =
    classOf[OrderPlaced].getName

  override def transform(fromVersion: Int, json: JsonNode): JsonNode = json
}
Java
sourcepublic class OrderPlacedMigration extends JacksonMigration {

  @Override
  public int currentVersion() {
    return 2;
  }

  @Override
  public String transformClassName(int fromVersion, String className) {
    return OrderPlaced.class.getName();
  }

  @Override
  public JsonNode transform(int fromVersion, JsonNode json) {
    return json;
  }
}

Note the override of the transformClassName(fromVersion, className)transformClassName(fromVersion, className) method to define the new class name.

That type of migration must be configured with the old class name as key. The actual class can be removed.

sourcepekko.serialization.jackson.migrations {
  "com.myservice.event.OrderAdded" = "com.myservice.event.OrderPlacedMigration"
}

Remove from serialization-bindings

When a class is not used for serialization any more it can be removed from serialization-bindings but to still allow deserialization it must then be listed in the allowed-class-prefix configuration. This is useful for example during rolling update with serialization changes, or when reading old stored data. It can also be used when changing from Jackson serializer to another serializer (e.g. Protobuf) and thereby changing the serialization binding, but it should still be possible to deserialize old data with Jackson.

sourcepekko.serialization.jackson.allowed-class-prefix =
  ["com.myservice.event.OrderAdded", "com.myservice.command"]

It’s a list of class names or prefixes of class names.

Rolling updates

When doing a rolling update, for a period of time there are two different binaries running in production. If the schema has evolved requiring a new schema version, the data serialized by the new binary will be unreadable from the old binary. This situation causes transient errors on the processes running the old binary. This service degradation is usually fine since the rolling update will eventually complete and all old processes will be replaced with the new binary. To avoid this service degradation you can also use forward-one support in your schema evolutions.

To complete a no-degradation rolling update, you need to make two deployments. First, deploy a new binary which can read the new schema but still uses the old schema. Then, deploy a second binary which serializes data using the new schema and drops the downcasting code from the migration.

Let’s take, for example, the case above where we renamed a field.

The starting schema is:

Scala
sourcecase class ItemAdded(shoppingCartId: String, productId: String, quantity: Int) extends MySerializable
Java
sourcepublic class ItemAdded implements MySerializable {
  public final String shoppingCartId;
  public final String productId;
  public final int quantity;

  public ItemAdded(String shoppingCartId, String productId, int quantity) {
    this.shoppingCartId = shoppingCartId;
    this.productId = productId;
    this.quantity = quantity;
  }
}

In a first deployment, we still don’t make any change to the event class:

Scala
sourcecase class ItemAdded(shoppingCartId: String, productId: String, quantity: Int) extends MySerializable
Java
sourcepublic class ItemAdded implements MySerializable {
  public final String shoppingCartId;
  public final String productId;
  public final int quantity;

  public ItemAdded(String shoppingCartId, String productId, int quantity) {
    this.shoppingCartId = shoppingCartId;
    this.productId = productId;
    this.quantity = quantity;
  }
}

but we introduce a migration that can read the newer schema which is versioned 2:

Scala
sourceimport org.apache.pekko.serialization.jackson.JacksonMigration
import com.fasterxml.jackson.databind.JsonNode
import com.fasterxml.jackson.databind.node.ObjectNode

class ItemAddedMigration extends JacksonMigration {

  // Data produced in this node is still produced using the version 1 of the schema
  override def currentVersion: Int = 1

  override def supportedForwardVersion: Int = 2

  override def transform(fromVersion: Int, json: JsonNode): JsonNode = {
    val root = json.asInstanceOf[ObjectNode]
    if (fromVersion == 2) {
      // When receiving an event of version 2 we down-cast it to the version 1 of the schema
      root.set[JsonNode]("productId", root.get("itemId"))
      root.remove("itemId")
    }
    root
  }
}
Java
source
import org.apache.pekko.serialization.jackson.JacksonMigration; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.node.ObjectNode; public class ItemAddedMigration extends JacksonMigration { // Data produced in this node is still produced using the version 1 of the schema @Override public int currentVersion() { return 1; } @Override public int supportedForwardVersion() { return 2; } @Override public JsonNode transform(int fromVersion, JsonNode json) { ObjectNode root = (ObjectNode) json; if (fromVersion == 2) { // When receiving an event of version 2 we down-cast it to the version 1 of the schema root.set("productId", root.get("itemId")); root.remove("itemId"); } return root; } }

Once all running nodes have the new migration code which can read version 2 of ItemAdded we can proceed with the second step. So, we deploy the updated event:

Scala
sourcecase class ItemAdded(shoppingCartId: String, itemId: String, quantity: Int) extends MySerializable
Java
sourcepublic class ItemAdded implements MySerializable {
  public final String shoppingCartId;

  public final String itemId;

  public final int quantity;

  public ItemAdded(String shoppingCartId, String itemId, int quantity) {
    this.shoppingCartId = shoppingCartId;
    this.itemId = itemId;
    this.quantity = quantity;
  }
}

and the final migration code which no longer needs forward-compatibility code:

Scala
sourceimport org.apache.pekko.serialization.jackson.JacksonMigration
import com.fasterxml.jackson.databind.JsonNode
import com.fasterxml.jackson.databind.node.ObjectNode

class ItemAddedMigration extends JacksonMigration {

  override def currentVersion: Int = 2

  override def transform(fromVersion: Int, json: JsonNode): JsonNode = {
    val root = json.asInstanceOf[ObjectNode]
    if (fromVersion <= 1) {
      root.set[JsonNode]("itemId", root.get("productId"))
      root.remove("productId")
    }
    root
  }
}
Java
source
import org.apache.pekko.serialization.jackson.JacksonMigration; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.node.ObjectNode; public class ItemAddedMigration extends JacksonMigration { @Override public int currentVersion() { return 2; } @Override public JsonNode transform(int fromVersion, JsonNode json) { ObjectNode root = (ObjectNode) json; if (fromVersion <= 1) { root.set("itemId", root.get("productId")); root.remove("productId"); } return root; } }

Jackson Modules

The following Jackson modules are enabled by default:

sourcepekko.serialization.jackson {

  # The Jackson JSON serializer will register these modules.
  jackson-modules += "org.apache.pekko.serialization.jackson.PekkoJacksonModule"
  # PekkoTypedJacksonModule optionally included if pekko-actor-typed is in classpath
  jackson-modules += "org.apache.pekko.serialization.jackson.PekkoTypedJacksonModule"
  # PekkoStreamsModule optionally included if pekko-streams is in classpath
  jackson-modules += "org.apache.pekko.serialization.jackson.PekkoStreamJacksonModule"
  jackson-modules += "com.fasterxml.jackson.module.paramnames.ParameterNamesModule"
  jackson-modules += "com.fasterxml.jackson.datatype.jdk8.Jdk8Module"
  jackson-modules += "com.fasterxml.jackson.datatype.jsr310.JavaTimeModule"
  jackson-modules += "com.fasterxml.jackson.module.scala.DefaultScalaModule"
}

You can amend the configuration pekko.serialization.jackson.jackson-modules to enable other modules.

The ParameterNamesModule requires that the -parameters Java compiler option is enabled.

Compression

JSON can be rather verbose and for large messages it can be beneficial to compress large payloads. For the jackson-json binding the default configuration is:

source# Compression settings for the jackson-json binding
pekko.serialization.jackson.jackson-json.compression {
  # Compression algorithm.
  # - off  : no compression
  # - gzip : using common java gzip
  # - lz4 : using lz4-java
  algorithm = gzip

  # If compression is enabled with the `algorithm` setting the payload is compressed
  # when it's larger than this value.
  compress-larger-than = 32 KiB
}

Supported compression algorithms are: gzip, lz4. Use ‘off’ to disable compression. Gzip is generally slower than lz4. Messages larger than the compress-larger-than property are compressed.

Compression can be disabled by setting the algorithm property to off. It will still be able to decompress payloads that were compressed when serialized, e.g. if this configuration is changed.

For the jackson-cbor and custom bindings other than jackson-json compression is by default disabled, but can be enabled in the same way as the configuration shown above but replacing jackson-json with the binding name (for example jackson-cbor).

Using Pekko Serialization for embedded types

For types that already have a Pekko Serializer defined that are embedded in types serialized with Jackson the PekkoSerializationSerializerPekkoSerializationSerializer and PekkoSerializationDeserializerPekkoSerializationDeserializer can be used to Pekko Serialization for individual fields.

The serializer/deserializer are not enabled automatically. The @JsonSerialize and @JsonDeserialize annotation needs to be added to the fields containing the types to be serialized with Pekko Serialization.

The type will be embedded as an object with the fields:

  • serId - the serializer id
  • serManifest - the manifest for the type
  • payload - base64 encoded bytes

Additional configuration

Configuration per binding

By default the configuration for the Jackson serializers and their ObjectMappers is defined in the pekko.serialization.jackson section. It is possible to override that configuration in a more specific pekko.serialization.jackson.<binding name> section.

sourcepekko.serialization.jackson.jackson-json {
  serialization-features {
    WRITE_DATES_AS_TIMESTAMPS = off
  }
}
pekko.serialization.jackson.jackson-cbor {
  serialization-features {
    WRITE_DATES_AS_TIMESTAMPS = on
  }
}

It’s also possible to define several bindings and use different configuration for them. For example, different settings for remote messages and persisted events.

sourcepekko.actor {
  serializers {
    jackson-json-message = "org.apache.pekko.serialization.jackson.JacksonJsonSerializer"
    jackson-json-event   = "org.apache.pekko.serialization.jackson.JacksonJsonSerializer"
  }
  serialization-identifiers {
    jackson-json-message = 9001
    jackson-json-event = 9002
  }
  serialization-bindings {
    "com.myservice.MyMessage" = jackson-json-message
    "com.myservice.MyEvent" = jackson-json-event
  }
}
pekko.serialization.jackson {
  jackson-json-message {
    serialization-features {
      WRITE_DATES_AS_TIMESTAMPS = on
    }
  }
  jackson-json-event {
    serialization-features {
      WRITE_DATES_AS_TIMESTAMPS = off
    }
  }
}

Manifest-less serialization

When using the Jackson serializer for persistence, given that the fully qualified class name is stored in the manifest, this can result in a lot of wasted disk and IO used, especially when the events are small. To address this, a type-in-manifest flag can be turned off, which will result in the class name not appearing in the manifest.

When deserializing, the Jackson serializer will use the type defined in deserialization-type, if present, otherwise it will look for exactly one serialization binding class, and use that. For this to be useful, generally that single type must be a Polymorphic type, with all type information necessary to deserialize to the various sub types contained in the JSON message.

When switching serializers, there will be periods of time when you may have no serialization bindings declared for the type. In such circumstances, you must use the deserialization-type configuration attribute to specify which type should be used to deserialize messages.

Since this configuration can only be applied to a single root type, you will usually only want to apply it to a per binding configuration, not to the regular jackson-json or jackson-cbor configurations.

sourcepekko.actor {
  serializers {
    jackson-json-event = "org.apache.pekko.serialization.jackson.JacksonJsonSerializer"
  }
  serialization-identifiers {
    jackson-json-event = 9001
  }
  serialization-bindings {
    "com.myservice.MyEvent" = jackson-json-event
  }
}
pekko.serialization.jackson {
  jackson-json-event {
    type-in-manifest = off
    # Since there is exactly one serialization binding declared for this
    # serializer above, this is optional, but if there were none or many,
    # this would be mandatory.
    deserialization-type = "com.myservice.MyEvent"
  }
}

Note that Pekko remoting already implements manifest compression, and so this optimization will have no significant impact for messages sent over remoting. It’s only useful for messages serialized for other purposes, such as persistence or distributed data.

Additional features

Additional Jackson serialization features can be enabled/disabled in configuration. The default values from Jackson are used aside from the following that are changed in Pekko’s default configuration.

sourcepekko.serialization.jackson {
  # Configuration of the ObjectMapper serialization features.
  # See com.fasterxml.jackson.databind.SerializationFeature
  # Enum values corresponding to the SerializationFeature and their boolean value.
  serialization-features {
    # Date/time in ISO-8601 (rfc3339) yyyy-MM-dd'T'HH:mm:ss.SSSZ format
    # as defined by com.fasterxml.jackson.databind.util.StdDateFormat
    # For interoperability it's better to use the ISO format, i.e. WRITE_DATES_AS_TIMESTAMPS=off,
    # but WRITE_DATES_AS_TIMESTAMPS=on has better performance.
    WRITE_DATES_AS_TIMESTAMPS = off
    WRITE_DURATIONS_AS_TIMESTAMPS = off
    FAIL_ON_EMPTY_BEANS = off
  }

  # Configuration of the ObjectMapper deserialization features.
  # See com.fasterxml.jackson.databind.DeserializationFeature
  # Enum values corresponding to the DeserializationFeature and their boolean value.
  deserialization-features {
    FAIL_ON_UNKNOWN_PROPERTIES = off
  }

  # Configuration of the ObjectMapper mapper features.
  # See com.fasterxml.jackson.databind.MapperFeature
  # Enum values corresponding to the MapperFeature and their
  # boolean values, for example:
  #
  # mapper-features {
  #   SORT_PROPERTIES_ALPHABETICALLY = on
  # }
  mapper-features {}

  # Configuration of the ObjectMapper JsonParser features.
  # See com.fasterxml.jackson.core.JsonParser.Feature
  # Enum values corresponding to the JsonParser.Feature and their
  # boolean value, for example:
  #
  # json-parser-features {
  #   ALLOW_SINGLE_QUOTES = on
  # }
  json-parser-features {}

  # Configuration of the ObjectMapper JsonParser features.
  # See com.fasterxml.jackson.core.JsonGenerator.Feature
  # Enum values corresponding to the JsonGenerator.Feature and
  # their boolean value, for example:
  #
  # json-generator-features {
  #   WRITE_NUMBERS_AS_STRINGS = on
  # }
  json-generator-features {}

  # Configuration of the JsonFactory StreamReadFeature.
  # See com.fasterxml.jackson.core.StreamReadFeature
  # Enum values corresponding to the StreamReadFeatures and
  # their boolean value, for example:
  #
  # stream-read-features {
  #   STRICT_DUPLICATE_DETECTION = on
  # }
  stream-read-features {}

  # Configuration of the JsonFactory StreamWriteFeature.
  # See com.fasterxml.jackson.core.StreamWriteFeature
  # Enum values corresponding to the StreamWriteFeatures and
  # their boolean value, for example:
  #
  # stream-write-features {
  #   WRITE_BIGDECIMAL_AS_PLAIN = on
  # }
  stream-write-features {}

  # Configuration of the JsonFactory JsonReadFeature.
  # See com.fasterxml.jackson.core.json.JsonReadFeature
  # Enum values corresponding to the JsonReadFeatures and
  # their boolean value, for example:
  #
  # json-read-features {
  #   ALLOW_SINGLE_QUOTES = on
  # }
  json-read-features {}

  # Configuration of the JsonFactory JsonWriteFeature.
  # See com.fasterxml.jackson.core.json.JsonWriteFeature
  # Enum values corresponding to the JsonWriteFeatures and
  # their boolean value, for example:
  #
  # json-write-features {
  #   WRITE_NUMBERS_AS_STRINGS = on
  # }
  json-write-features {}

  # Configuration of the JsonFactory Visibility.
  # See com.fasterxml.jackson.annotation.PropertyAccessor
  # and com.fasterxml.jackson.annotation.JsonAutoDetect.Visibility
  # Enum values. For example, to serialize only public fields
  # overwrite the default values with:
  #
  # visibility {
  #   FIELD = PUBLIC_ONLY
  # }
  # Default: all fields (including private and protected) are serialized.
  visibility {
    FIELD = ANY
  }

  # Deprecated, use `allowed-class-prefix` instead
  whitelist-class-prefix = []

  # Additional classes that are allowed even if they are not defined in `serialization-bindings`.
  # This is useful when a class is not used for serialization any more and therefore removed
  # from `serialization-bindings`, but should still be possible to deserialize.
  allowed-class-prefix = ${pekko.serialization.jackson.whitelist-class-prefix}


  # settings for compression of the payload
  compression {
    # Compression algorithm.
    # - off  : no compression
    # - gzip : using common java gzip
    algorithm = off

    # If compression is enabled with the `algorithm` setting the payload is compressed
    # when it's larger than this value.
    compress-larger-than = 0 KiB
  }

  # Whether the type should be written to the manifest.
  # If this is off, then either deserialization-type must be defined, or there must be exactly
  # one serialization binding declared for this serializer, and the type in that binding will be
  # used as the deserialization type. This feature will only work if that type either is a
  # concrete class, or if it is a supertype that uses Jackson polymorphism (ie, the
  # @JsonTypeInfo annotation) to store type information in the JSON itself. The intention behind
  # disabling this is to remove extraneous type information (ie, fully qualified class names) when
  # serialized objects are persisted in Pekko persistence or replicated using Pekko distributed
  # data. Note that Pekko remoting already has manifest compression optimizations that address this,
  # so for types that just get sent over remoting, this offers no optimization.
  type-in-manifest = on

  # The type to use for deserialization.
  # This is only used if type-in-manifest is disabled. If set, this type will be used to
  # deserialize all messages. This is useful if the binding configuration you want to use when
  # disabling type in manifest cannot be expressed as a single type. Examples of when you might
  # use this include when changing serializers, so you don't want this serializer used for
  # serialization and you haven't declared any bindings for it, but you still want to be able to
  # deserialize messages that were serialized with this serializer, as well as situations where
  # you only want some sub types of a given Jackson polymorphic type to be serialized using this
  # serializer.
  deserialization-type = ""

  # Specific settings for jackson-json binding can be defined in this section to
  # override the settings in 'pekko.serialization.jackson'
  jackson-json {}

  # Specific settings for jackson-cbor binding can be defined in this section to
  # override the settings in 'pekko.serialization.jackson'
  jackson-cbor {}

  # Issue #28918 for compatibility with data serialized with JacksonCborSerializer in
  # Akka 2.6.4 or earlier, which was plain JSON format.
  jackson-cbor-264 = ${pekko.serialization.jackson.jackson-cbor}

}

Date/time format

WRITE_DATES_AS_TIMESTAMPS and WRITE_DURATIONS_AS_TIMESTAMPS are by default disabled, which means that date/time fields are serialized in ISO-8601 (rfc3339) yyyy-MM-dd'T'HH:mm:ss.SSSZ format instead of numeric arrays. This is better for interoperability but it is slower. If you don’t need the ISO format for interoperability with external systems you can change the following configuration for better performance of date/time fields.

sourcepekko.serialization.jackson.serialization-features {
  WRITE_DATES_AS_TIMESTAMPS = on
  WRITE_DURATIONS_AS_TIMESTAMPS = on
}

Jackson is still be able to deserialize the other format independent of this setting.