Serialization with Jackson

Dependency

To use Jackson Serialization, you must add the following dependency in your project:

sbt

val PekkoVersion = "1.0.1"
libraryDependencies += "org.apache.pekko" %% "pekko-serialization-jackson" % PekkoVersion

Maven

<properties>
  <scala.binary.version>2.13</scala.binary.version>
</properties>
<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.apache.pekko</groupId>
      <artifactId>pekko-bom_${scala.binary.version}</artifactId>
      <version>1.0.1</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>
<dependencies>
  <dependency>
    <groupId>org.apache.pekko</groupId>
    <artifactId>pekko-serialization-jackson_${scala.binary.version}</artifactId>
  </dependency>
</dependencies>

Gradle

def versions = [
  ScalaBinary: "2.13"
]
dependencies {
  implementation platform("org.apache.pekko:pekko-bom_${versions.ScalaBinary}:1.0.1")

  implementation "org.apache.pekko:pekko-serialization-jackson_${versions.ScalaBinary}"
}

Introduction

You find general concepts for for Pekko serialization in the Serialization section. This section describes how to use the Jackson serializer for application specific messages and persistent events and snapshots.

Jackson has support for both text based JSON and binary formats.

In many cases ordinary classes can be serialized by Jackson without any additional hints, but sometimes annotations are needed to specify how to convert the objects to JSON/bytes.

Usage

To enable Jackson serialization for a class you need to configure it or one of its super classes in serialization-bindings configuration. Typically you will create a marker traitinterface for that purpose and let the messages extendimplement that.

Scala

copysource/**
 * Marker interface for messages, events and snapshots that are serialized with Jackson.
 */
trait MySerializable

final case class Message(name: String, nr: Int) extends MySerializable

Java

copysource/** Marker interface for messages, events and snapshots that are serialized with Jackson. */
public interface MySerializable {}

class MyMessage implements MySerializable {
  public final String name;
  public final int nr;

  public MyMessage(String name, int nr) {
    this.name = name;
    this.nr = nr;
  }
}

Then you configure the class name of the marker traitinterface in serialization-bindings to one of the supported Jackson formats: jackson-json or jackson-cbor

copysourcepekko.actor {
  serialization-bindings {
    "com.myservice.MySerializable" = jackson-json
  }
}

A good convention would be to name the marker interface CborSerializable or JsonSerializable. In this documentation we have used MySerializable to make it clear that the marker interface itself is not provided by Pekko.

That is all that is needed for basic classes where Jackson understands the structure. A few cases that requires annotations are described below.

Note that it’s only the top level class or its marker traitinterface that must be defined in serialization-bindings, not nested classes that it references in member fields.

Security

For security reasons it is disallowed to bind the Jackson serializers to open ended types that might be a target for serialization gadgets, such as:

• java.lang.Object
• java.io.Serializable
• java.lang.Comparable.

The deny list of possible serialization gadget classes defined by Jackson databind are checked and disallowed for deserialization.

Formats

The following formats are supported, and you select which one to use in the serialization-bindings configuration as described above.

• jackson-json - ordinary text based JSON
• jackson-cbor - binary CBOR data format

The binary format is more compact, with slightly better performance than the JSON format.

Annotations

MismatchedInputException: Cannot construct instance of `...` (although at least one Creator exists): cannot deserialize from Object value (no delegate- or property-based Creator)

Polymorphic types

A polymorphic type is when a certain base type has multiple alternative implementations. When nested fields or collections are of polymorphic type the concrete implementations of the type must be listed with @JsonTypeInfo and @JsonSubTypes annotations.

Example:

Scala

copysourcefinal case class Zoo(primaryAttraction: Animal) extends MySerializable

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type")
@JsonSubTypes(
  Array(
    new JsonSubTypes.Type(value = classOf[Lion], name = "lion"),
    new JsonSubTypes.Type(value = classOf[Elephant], name = "elephant")))
sealed trait Animal

final case class Lion(name: String) extends Animal

final case class Elephant(name: String, age: Int) extends Animal

Java

copysourcepublic class Zoo implements MySerializable {
  public final Animal primaryAttraction;

  @JsonCreator
  public Zoo(Animal primaryAttraction) {
    this.primaryAttraction = primaryAttraction;
  }
}

@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type")
@JsonSubTypes({
  @JsonSubTypes.Type(value = Lion.class, name = "lion"),
  @JsonSubTypes.Type(value = Elephant.class, name = "elephant")
})
interface Animal {}

public final class Lion implements Animal {
  public final String name;

  @JsonCreator
  public Lion(String name) {
    this.name = name;
  }
}

public final class Elephant implements Animal {
  public final String name;
  public final int age;

  public Elephant(String name, int age) {
    this.name = name;
    this.age = age;
  }
}

If you haven’t defined the annotations you will see an exception like this:

InvalidDefinitionException: Cannot construct instance of `...` (no Creators, like default construct, exist): abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information

Note that this is not needed for a top level class, but for fields inside it. In this example Animal is used inside of Zoo, which is sent as a message or persisted. If Animal was sent or persisted standalone the annotations are not needed because then it is the concrete subclasses Lion or Elephant that are serialized.

When specifying allowed subclasses with those annotations the class names will not be included in the serialized representation and that is important for preventing loading of malicious serialization gadgets when deserializing.

Schema Evolution

When using Event Sourcing, but also for rolling updates, schema evolution becomes an important aspect of developing your application. The requirements as well as our own understanding of the business domain may (and will) change over time.

The Jackson serializer provides a way to perform transformations of the JSON tree model during deserialization. This is working in the same way for the textual and binary formats.

We will look at a few scenarios of how the classes may be evolved.

Remove Field

Removing a field can be done without any migration code. The Jackson serializer will ignore properties that does not exist in the class.

Add Optional Field

Adding an optional field can be done without any migration code. The default value will be NoneOptional.empty.

Old class:

Scala

copysourcecase class ItemAdded(shoppingCartId: String, productId: String, quantity: Int) extends MySerializable

Java

copysourcepublic class ItemAdded implements MySerializable {
  public final String shoppingCartId;
  public final String productId;
  public final int quantity;

  public ItemAdded(String shoppingCartId, String productId, int quantity) {
    this.shoppingCartId = shoppingCartId;
    this.productId = productId;
    this.quantity = quantity;
  }
}

New class with a new optional discount property and a new note field with default value:

Scala

copysourcecase class ItemAdded(shoppingCartId: String, productId: String, quantity: Int, discount: Option[Double], note: String)
    extends MySerializable {

  // alternative constructor because `note` should have default value "" when not defined in json
  @JsonCreator
  def this(shoppingCartId: String, productId: String, quantity: Int, discount: Option[Double], note: Option[String]) =
    this(shoppingCartId, productId, quantity, discount, note.getOrElse(""))
}

Java

copysourcepublic class ItemAdded implements MySerializable {
  public final String shoppingCartId;
  public final String productId;
  public final int quantity;
  public final Optional<Double> discount;
  public final String note;

  @JsonCreator
  public ItemAdded(
      String shoppingCartId,
      String productId,
      int quantity,
      Optional<Double> discount,
      String note) {
    this.shoppingCartId = shoppingCartId;
    this.productId = productId;
    this.quantity = quantity;
    this.discount = discount;

    // default for note is "" if not included in json
    if (note == null) this.note = "";
    else this.note = note;
  }

  public ItemAdded(
      String shoppingCartId, String productId, int quantity, Optional<Double> discount) {
    this(shoppingCartId, productId, quantity, discount, "");
  }
}

Add Mandatory Field

Let’s say we want to have a mandatory discount property without default value instead:

Scala

copysourcecase class ItemAdded(shoppingCartId: String, productId: String, quantity: Int, discount: Double) extends MySerializable

Java

copysourcepublic class ItemAdded implements MySerializable {
  public final String shoppingCartId;
  public final String productId;
  public final int quantity;
  public final double discount;

  public ItemAdded(String shoppingCartId, String productId, int quantity, double discount) {
    this.shoppingCartId = shoppingCartId;
    this.productId = productId;
    this.quantity = quantity;
    this.discount = discount;
  }
}

To add a new mandatory field we have to use a JacksonMigrationJacksonMigration class and set the default value in the migration code.

This is how a migration class would look like for adding a discount field:

Scala

copysourceimport com.fasterxml.jackson.databind.JsonNode
import com.fasterxml.jackson.databind.node.DoubleNode
import com.fasterxml.jackson.databind.node.ObjectNode
import org.apache.pekko.serialization.jackson.JacksonMigration

class ItemAddedMigration extends JacksonMigration {

  override def currentVersion: Int = 2

  override def transform(fromVersion: Int, json: JsonNode): JsonNode = {
    val root = json.asInstanceOf[ObjectNode]
    if (fromVersion <= 1) {
      root.set("discount", DoubleNode.valueOf(0.0))
    }
    root
  }
}

Java

copysourceimport com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.DoubleNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import org.apache.pekko.serialization.jackson.JacksonMigration;

public class ItemAddedMigration extends JacksonMigration {

  @Override
  public int currentVersion() {
    return 2;
  }

  @Override
  public JsonNode transform(int fromVersion, JsonNode json) {
    ObjectNode root = (ObjectNode) json;
    if (fromVersion <= 1) {
      root.set("discount", DoubleNode.valueOf(0.0));
    }
    return root;
  }
}

Override the currentVersioncurrentVersion() method to define the version number of the current (latest) version. The first version, when no migration was used, is always 1. Increase this version number whenever you perform a change that is not backwards compatible without migration code.

Implement the transformation of the old JSON structure to the new JSON structure in the transform(fromVersion, jsonNode)transform(fromVersion, jsonNode) method. The JsonNode is mutable so you can add and remove fields, or change values. Note that you have to cast to specific sub-classes such as ObjectNode and ArrayNode to get access to mutators.

The migration class must be defined in configuration file:

copysourcepekko.serialization.jackson.migrations {
  "com.myservice.event.ItemAdded" = "com.myservice.event.ItemAddedMigration"
}

The same thing could have been done for the note field, adding a default value of "" in the ItemAddedMigration.

Rename Field

Let’s say that we want to rename the productId field to itemId in the previous example.

Scala

copysourcecase class ItemAdded(shoppingCartId: String, itemId: String, quantity: Int) extends MySerializable

Java

copysourcepublic class ItemAdded implements MySerializable {
  public final String shoppingCartId;

  public final String itemId;

  public final int quantity;

  public ItemAdded(String shoppingCartId, String itemId, int quantity) {
    this.shoppingCartId = shoppingCartId;
    this.itemId = itemId;
    this.quantity = quantity;
  }
}

The migration code would look like:

Scala

copysourceimport org.apache.pekko.serialization.jackson.JacksonMigration
import com.fasterxml.jackson.databind.JsonNode
import com.fasterxml.jackson.databind.node.ObjectNode

class ItemAddedMigration extends JacksonMigration {

  override def currentVersion: Int = 2

  override def transform(fromVersion: Int, json: JsonNode): JsonNode = {
    val root = json.asInstanceOf[ObjectNode]
    if (fromVersion <= 1) {
      root.set[JsonNode]("itemId", root.get("productId"))
      root.remove("productId")
    }
    root
  }
}

Java

copysource
import org.apache.pekko.serialization.jackson.JacksonMigration;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ObjectNode;

public class ItemAddedMigration extends JacksonMigration {

  @Override
  public int currentVersion() {
    return 2;
  }

  @Override
  public JsonNode transform(int fromVersion, JsonNode json) {
    ObjectNode root = (ObjectNode) json;
    if (fromVersion <= 1) {
      root.set("itemId", root.get("productId"));
      root.remove("productId");
    }
    return root;
  }
}

Structural Changes

In a similar way we can do arbitrary structural changes.

Old class:

Scala

copysourcecase class Customer(name: String, street: String, city: String, zipCode: String, country: String) extends MySerializable

Java

copysourcepublic class Customer implements MySerializable {
  public final String name;
  public final String street;
  public final String city;
  public final String zipCode;
  public final String country;

  public Customer(String name, String street, String city, String zipCode, String country) {
    this.name = name;
    this.street = street;
    this.city = city;
    this.zipCode = zipCode;
    this.country = country;
  }
}

New class:

Scala

copysourcecase class Customer(name: String, shippingAddress: Address, billingAddress: Option[Address]) extends MySerializable

Java

copysourcepublic class Customer implements MySerializable {
  public final String name;
  public final Address shippingAddress;
  public final Optional<Address> billingAddress;

  public Customer(String name, Address shippingAddress, Optional<Address> billingAddress) {
    this.name = name;
    this.shippingAddress = shippingAddress;
    this.billingAddress = billingAddress;
  }
}

with the Address class:

Scala

copysourcecase class Address(street: String, city: String, zipCode: String, country: String) extends MySerializable

Java

copysourcepublic class Address {
  public final String street;
  public final String city;
  public final String zipCode;
  public final String country;

  public Address(String street, String city, String zipCode, String country) {
    this.street = street;
    this.city = city;
    this.zipCode = zipCode;
    this.country = country;
  }
}

The migration code would look like:

Scala

copysourceimport org.apache.pekko.serialization.jackson.JacksonMigration
import com.fasterxml.jackson.databind.JsonNode
import com.fasterxml.jackson.databind.node.ObjectNode

import scala.annotation.nowarn

class CustomerMigration extends JacksonMigration {

  override def currentVersion: Int = 2

  @nowarn("msg=deprecated")
  override def transform(fromVersion: Int, json: JsonNode): JsonNode = {
    val root = json.asInstanceOf[ObjectNode]
    if (fromVersion <= 1) {
      val shippingAddress = root.`with`("shippingAddress")
      shippingAddress.set[JsonNode]("street", root.get("street"))
      shippingAddress.set[JsonNode]("city", root.get("city"))
      shippingAddress.set[JsonNode]("zipCode", root.get("zipCode"))
      shippingAddress.set[JsonNode]("country", root.get("country"))
      root.remove("street")
      root.remove("city")
      root.remove("zipCode")
      root.remove("country")
    }
    root
  }
}

Java

copysourceimport org.apache.pekko.serialization.jackson.JacksonMigration;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ObjectNode;

public class CustomerMigration extends JacksonMigration {

  @Override
  public int currentVersion() {
    return 2;
  }

  @Override
  public JsonNode transform(int fromVersion, JsonNode json) {
    ObjectNode root = (ObjectNode) json;
    if (fromVersion <= 1) {
      ObjectNode shippingAddress = root.with("shippingAddress");
      shippingAddress.set("street", root.get("street"));
      shippingAddress.set("city", root.get("city"));
      shippingAddress.set("zipCode", root.get("zipCode"));
      shippingAddress.set("country", root.get("country"));
      root.remove("street");
      root.remove("city");
      root.remove("zipCode");
      root.remove("country");
    }
    return root;
  }
}

Rename Class

It is also possible to rename the class. For example, let’s rename OrderAdded to OrderPlaced.

Old class:

Scala

copysourcecase class OrderAdded(shoppingCartId: String) extends MySerializable

Java

copysourcepublic class OrderAdded implements MySerializable {
  public final String shoppingCartId;

  @JsonCreator
  public OrderAdded(String shoppingCartId) {
    this.shoppingCartId = shoppingCartId;
  }
}

New class:

Scala

copysourcecase class OrderPlaced(shoppingCartId: String) extends MySerializable

Java

copysourcepublic class OrderPlaced implements MySerializable {
  public final String shoppingCartId;

  @JsonCreator
  public OrderPlaced(String shoppingCartId) {
    this.shoppingCartId = shoppingCartId;
  }
}

The migration code would look like:

Scala

copysourceclass OrderPlacedMigration extends JacksonMigration {

  override def currentVersion: Int = 2

  override def transformClassName(fromVersion: Int, className: String): String =
    classOf[OrderPlaced].getName

  override def transform(fromVersion: Int, json: JsonNode): JsonNode = json
}

Java

copysourcepublic class OrderPlacedMigration extends JacksonMigration {

  @Override
  public int currentVersion() {
    return 2;
  }

  @Override
  public String transformClassName(int fromVersion, String className) {
    return OrderPlaced.class.getName();
  }

  @Override
  public JsonNode transform(int fromVersion, JsonNode json) {
    return json;
  }
}

Note the override of the transformClassName(fromVersion, className)transformClassName(fromVersion, className) method to define the new class name.

That type of migration must be configured with the old class name as key. The actual class can be removed.

copysourcepekko.serialization.jackson.migrations {
  "com.myservice.event.OrderAdded" = "com.myservice.event.OrderPlacedMigration"
}

Remove from serialization-bindings

When a class is not used for serialization any more it can be removed from serialization-bindings but to still allow deserialization it must then be listed in the allowed-class-prefix configuration. This is useful for example during rolling update with serialization changes, or when reading old stored data. It can also be used when changing from Jackson serializer to another serializer (e.g. Protobuf) and thereby changing the serialization binding, but it should still be possible to deserialize old data with Jackson.

copysourcepekko.serialization.jackson.allowed-class-prefix =
  ["com.myservice.event.OrderAdded", "com.myservice.command"]

It’s a list of class names or prefixes of class names.

Rolling updates

When doing a rolling update, for a period of time there are two different binaries running in production. If the schema has evolved requiring a new schema version, the data serialized by the new binary will be unreadable from the old binary. This situation causes transient errors on the processes running the old binary. This service degradation is usually fine since the rolling update will eventually complete and all old processes will be replaced with the new binary. To avoid this service degradation you can also use forward-one support in your schema evolutions.

To complete a no-degradation rolling update, you need to make two deployments. First, deploy a new binary which can read the new schema but still uses the old schema. Then, deploy a second binary which serializes data using the new schema and drops the downcasting code from the migration.

Let’s take, for example, the case above where we renamed a field.

The starting schema is:

Scala

copysourcecase class ItemAdded(shoppingCartId: String, productId: String, quantity: Int) extends MySerializable

Java

copysourcepublic class ItemAdded implements MySerializable {
  public final String shoppingCartId;
  public final String productId;
  public final int quantity;

  public ItemAdded(String shoppingCartId, String productId, int quantity) {
    this.shoppingCartId = shoppingCartId;
    this.productId = productId;
    this.quantity = quantity;
  }
}

In a first deployment, we still don’t make any change to the event class:

Scala

copysourcecase class ItemAdded(shoppingCartId: String, productId: String, quantity: Int) extends MySerializable

Java

copysourcepublic class ItemAdded implements MySerializable {
  public final String shoppingCartId;
  public final String productId;
  public final int quantity;

  public ItemAdded(String shoppingCartId, String productId, int quantity) {
    this.shoppingCartId = shoppingCartId;
    this.productId = productId;
    this.quantity = quantity;
  }
}

but we introduce a migration that can read the newer schema which is versioned 2:

Scala

copysourceimport org.apache.pekko.serialization.jackson.JacksonMigration
import com.fasterxml.jackson.databind.JsonNode
import com.fasterxml.jackson.databind.node.ObjectNode

class ItemAddedMigration extends JacksonMigration {

  // Data produced in this node is still produced using the version 1 of the schema
  override def currentVersion: Int = 1

  override def supportedForwardVersion: Int = 2

  override def transform(fromVersion: Int, json: JsonNode): JsonNode = {
    val root = json.asInstanceOf[ObjectNode]
    if (fromVersion == 2) {
      // When receiving an event of version 2 we down-cast it to the version 1 of the schema
      root.set[JsonNode]("productId", root.get("itemId"))
      root.remove("itemId")
    }
    root
  }
}

Java

copysource
import org.apache.pekko.serialization.jackson.JacksonMigration;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ObjectNode;

public class ItemAddedMigration extends JacksonMigration {

  // Data produced in this node is still produced using the version 1 of the schema
  @Override
  public int currentVersion() {
    return 1;
  }

  @Override
  public int supportedForwardVersion() {
    return 2;
  }

  @Override
  public JsonNode transform(int fromVersion, JsonNode json) {
    ObjectNode root = (ObjectNode) json;
    if (fromVersion == 2) {
      // When receiving an event of version 2 we down-cast it to the version 1 of the schema
      root.set("productId", root.get("itemId"));
      root.remove("itemId");
    }
    return root;
  }
}

Once all running nodes have the new migration code which can read version 2 of ItemAdded we can proceed with the second step. So, we deploy the updated event:

Scala

copysourcecase class ItemAdded(shoppingCartId: String, itemId: String, quantity: Int) extends MySerializable

Java

copysourcepublic class ItemAdded implements MySerializable {
  public final String shoppingCartId;

  public final String itemId;

  public final int quantity;

  public ItemAdded(String shoppingCartId, String itemId, int quantity) {
    this.shoppingCartId = shoppingCartId;
    this.itemId = itemId;
    this.quantity = quantity;
  }
}

and the final migration code which no longer needs forward-compatibility code:

Scala

copysourceimport org.apache.pekko.serialization.jackson.JacksonMigration
import com.fasterxml.jackson.databind.JsonNode
import com.fasterxml.jackson.databind.node.ObjectNode

class ItemAddedMigration extends JacksonMigration {

  override def currentVersion: Int = 2

  override def transform(fromVersion: Int, json: JsonNode): JsonNode = {
    val root = json.asInstanceOf[ObjectNode]
    if (fromVersion <= 1) {
      root.set[JsonNode]("itemId", root.get("productId"))
      root.remove("productId")
    }
    root
  }
}

Java

copysource
import org.apache.pekko.serialization.jackson.JacksonMigration;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ObjectNode;

public class ItemAddedMigration extends JacksonMigration {

  @Override
  public int currentVersion() {
    return 2;
  }

  @Override
  public JsonNode transform(int fromVersion, JsonNode json) {
    ObjectNode root = (ObjectNode) json;
    if (fromVersion <= 1) {
      root.set("itemId", root.get("productId"));
      root.remove("productId");
    }
    return root;
  }
}

Jackson Modules

The following Jackson modules are enabled by default:

copysourcepekko.serialization.jackson {

  # The Jackson JSON serializer will register these modules.
  jackson-modules += "org.apache.pekko.serialization.jackson.PekkoJacksonModule"
  # PekkoTypedJacksonModule optionally included if pekko-actor-typed is in classpath
  jackson-modules += "org.apache.pekko.serialization.jackson.PekkoTypedJacksonModule"
  # PekkoStreamsModule optionally included if pekko-streams is in classpath
  jackson-modules += "org.apache.pekko.serialization.jackson.PekkoStreamJacksonModule"
  jackson-modules += "com.fasterxml.jackson.module.paramnames.ParameterNamesModule"
  jackson-modules += "com.fasterxml.jackson.datatype.jdk8.Jdk8Module"
  jackson-modules += "com.fasterxml.jackson.datatype.jsr310.JavaTimeModule"
  jackson-modules += "com.fasterxml.jackson.module.scala.DefaultScalaModule"
}

You can amend the configuration pekko.serialization.jackson.jackson-modules to enable other modules.

The ParameterNamesModule requires that the -parameters Java compiler option is enabled.

Compression

JSON can be rather verbose and for large messages it can be beneficial to compress large payloads. For the jackson-json binding the default configuration is:

copysource# Compression settings for the jackson-json binding
pekko.serialization.jackson.jackson-json.compression {
  # Compression algorithm.
  # - off  : no compression
  # - gzip : using common java gzip
  # - lz4 : using lz4-java
  algorithm = gzip

  # If compression is enabled with the `algorithm` setting the payload is compressed
  # when it's larger than this value.
  compress-larger-than = 32 KiB
}

Supported compression algorithms are: gzip, lz4. Use ‘off’ to disable compression. Gzip is generally slower than lz4. Messages larger than the compress-larger-than property are compressed.

Compression can be disabled by setting the algorithm property to off. It will still be able to decompress payloads that were compressed when serialized, e.g. if this configuration is changed.

For the jackson-cbor and custom bindings other than jackson-json compression is by default disabled, but can be enabled in the same way as the configuration shown above but replacing jackson-json with the binding name (for example jackson-cbor).

Using Pekko Serialization for embedded types

For types that already have a Pekko Serializer defined that are embedded in types serialized with Jackson the PekkoSerializationSerializerPekkoSerializationSerializer and PekkoSerializationDeserializerPekkoSerializationDeserializer can be used to Pekko Serialization for individual fields.

The serializer/deserializer are not enabled automatically. The @JsonSerialize and @JsonDeserialize annotation needs to be added to the fields containing the types to be serialized with Pekko Serialization.

The type will be embedded as an object with the fields:

• serId - the serializer id
• serManifest - the manifest for the type
• payload - base64 encoded bytes

Additional configuration

Configuration per binding

By default the configuration for the Jackson serializers and their ObjectMappers is defined in the pekko.serialization.jackson section. It is possible to override that configuration in a more specific pekko.serialization.jackson.<binding name> section.

copysourcepekko.serialization.jackson.jackson-json {
  serialization-features {
    WRITE_DATES_AS_TIMESTAMPS = off
  }
}
pekko.serialization.jackson.jackson-cbor {
  serialization-features {
    WRITE_DATES_AS_TIMESTAMPS = on
  }
}

It’s also possible to define several bindings and use different configuration for them. For example, different settings for remote messages and persisted events.

copysourcepekko.actor {
  serializers {
    jackson-json-message = "org.apache.pekko.serialization.jackson.JacksonJsonSerializer"
    jackson-json-event   = "org.apache.pekko.serialization.jackson.JacksonJsonSerializer"
  }
  serialization-identifiers {
    jackson-json-message = 9001
    jackson-json-event = 9002
  }
  serialization-bindings {
    "com.myservice.MyMessage" = jackson-json-message
    "com.myservice.MyEvent" = jackson-json-event
  }
}
pekko.serialization.jackson {
  jackson-json-message {
    serialization-features {
      WRITE_DATES_AS_TIMESTAMPS = on
    }
  }
  jackson-json-event {
    serialization-features {
      WRITE_DATES_AS_TIMESTAMPS = off
    }
  }
}

Manifest-less serialization

When using the Jackson serializer for persistence, given that the fully qualified class name is stored in the manifest, this can result in a lot of wasted disk and IO used, especially when the events are small. To address this, a type-in-manifest flag can be turned off, which will result in the class name not appearing in the manifest.

When deserializing, the Jackson serializer will use the type defined in deserialization-type, if present, otherwise it will look for exactly one serialization binding class, and use that. For this to be useful, generally that single type must be a Polymorphic type, with all type information necessary to deserialize to the various sub types contained in the JSON message.

When switching serializers, there will be periods of time when you may have no serialization bindings declared for the type. In such circumstances, you must use the deserialization-type configuration attribute to specify which type should be used to deserialize messages.

Since this configuration can only be applied to a single root type, you will usually only want to apply it to a per binding configuration, not to the regular jackson-json or jackson-cbor configurations.

copysourcepekko.actor {
  serializers {
    jackson-json-event = "org.apache.pekko.serialization.jackson.JacksonJsonSerializer"
  }
  serialization-identifiers {
    jackson-json-event = 9001
  }
  serialization-bindings {
    "com.myservice.MyEvent" = jackson-json-event
  }
}
pekko.serialization.jackson {
  jackson-json-event {
    type-in-manifest = off
    # Since there is exactly one serialization binding declared for this
    # serializer above, this is optional, but if there were none or many,
    # this would be mandatory.
    deserialization-type = "com.myservice.MyEvent"
  }
}

Note that Pekko remoting already implements manifest compression, and so this optimization will have no significant impact for messages sent over remoting. It’s only useful for messages serialized for other purposes, such as persistence or distributed data.

Additional features

Additional Jackson serialization features can be enabled/disabled in configuration. The default values from Jackson are used aside from the following that are changed in Pekko’s default configuration.

copysourcepekko.serialization.jackson {
  # Configuration of the ObjectMapper serialization features.
  # See com.fasterxml.jackson.databind.SerializationFeature
  # Enum values corresponding to the SerializationFeature and their boolean value.
  serialization-features {
    # Date/time in ISO-8601 (rfc3339) yyyy-MM-dd'T'HH:mm:ss.SSSZ format
    # as defined by com.fasterxml.jackson.databind.util.StdDateFormat
    # For interoperability it's better to use the ISO format, i.e. WRITE_DATES_AS_TIMESTAMPS=off,
    # but WRITE_DATES_AS_TIMESTAMPS=on has better performance.
    WRITE_DATES_AS_TIMESTAMPS = off
    WRITE_DURATIONS_AS_TIMESTAMPS = off
    FAIL_ON_EMPTY_BEANS = off
  }

  # Configuration of the ObjectMapper deserialization features.
  # See com.fasterxml.jackson.databind.DeserializationFeature
  # Enum values corresponding to the DeserializationFeature and their boolean value.
  deserialization-features {
    FAIL_ON_UNKNOWN_PROPERTIES = off
  }

  # Configuration of the ObjectMapper mapper features.
  # See com.fasterxml.jackson.databind.MapperFeature
  # Enum values corresponding to the MapperFeature and their
  # boolean values, for example:
  #
  # mapper-features {
  #   SORT_PROPERTIES_ALPHABETICALLY = on
  # }
  mapper-features {}

  # Configuration of the ObjectMapper JsonParser features.
  # See com.fasterxml.jackson.core.JsonParser.Feature
  # Enum values corresponding to the JsonParser.Feature and their
  # boolean value, for example:
  #
  # json-parser-features {
  #   ALLOW_SINGLE_QUOTES = on
  # }
  json-parser-features {}

  # Configuration of the ObjectMapper JsonParser features.
  # See com.fasterxml.jackson.core.JsonGenerator.Feature
  # Enum values corresponding to the JsonGenerator.Feature and
  # their boolean value, for example:
  #
  # json-generator-features {
  #   WRITE_NUMBERS_AS_STRINGS = on
  # }
  json-generator-features {}

  # Configuration of the JsonFactory StreamReadFeature.
  # See com.fasterxml.jackson.core.StreamReadFeature
  # Enum values corresponding to the StreamReadFeatures and
  # their boolean value, for example:
  #
  # stream-read-features {
  #   STRICT_DUPLICATE_DETECTION = on
  # }
  stream-read-features {}

  # Configuration of the JsonFactory StreamWriteFeature.
  # See com.fasterxml.jackson.core.StreamWriteFeature
  # Enum values corresponding to the StreamWriteFeatures and
  # their boolean value, for example:
  #
  # stream-write-features {
  #   WRITE_BIGDECIMAL_AS_PLAIN = on
  # }
  stream-write-features {}

  # Configuration of the JsonFactory JsonReadFeature.
  # See com.fasterxml.jackson.core.json.JsonReadFeature
  # Enum values corresponding to the JsonReadFeatures and
  # their boolean value, for example:
  #
  # json-read-features {
  #   ALLOW_SINGLE_QUOTES = on
  # }
  json-read-features {}

  # Configuration of the JsonFactory JsonWriteFeature.
  # See com.fasterxml.jackson.core.json.JsonWriteFeature
  # Enum values corresponding to the JsonWriteFeatures and
  # their boolean value, for example:
  #
  # json-write-features {
  #   WRITE_NUMBERS_AS_STRINGS = on
  # }
  json-write-features {}

  # Configuration of the JsonFactory Visibility.
  # See com.fasterxml.jackson.annotation.PropertyAccessor
  # and com.fasterxml.jackson.annotation.JsonAutoDetect.Visibility
  # Enum values. For example, to serialize only public fields
  # overwrite the default values with:
  #
  # visibility {
  #   FIELD = PUBLIC_ONLY
  # }
  # Default: all fields (including private and protected) are serialized.
  visibility {
    FIELD = ANY
  }

  # Deprecated, use `allowed-class-prefix` instead
  whitelist-class-prefix = []

  # Additional classes that are allowed even if they are not defined in `serialization-bindings`.
  # This is useful when a class is not used for serialization any more and therefore removed
  # from `serialization-bindings`, but should still be possible to deserialize.
  allowed-class-prefix = ${pekko.serialization.jackson.whitelist-class-prefix}


  # settings for compression of the payload
  compression {
    # Compression algorithm.
    # - off  : no compression
    # - gzip : using common java gzip
    algorithm = off

    # If compression is enabled with the `algorithm` setting the payload is compressed
    # when it's larger than this value.
    compress-larger-than = 0 KiB
  }

  # Whether the type should be written to the manifest.
  # If this is off, then either deserialization-type must be defined, or there must be exactly
  # one serialization binding declared for this serializer, and the type in that binding will be
  # used as the deserialization type. This feature will only work if that type either is a
  # concrete class, or if it is a supertype that uses Jackson polymorphism (ie, the
  # @JsonTypeInfo annotation) to store type information in the JSON itself. The intention behind
  # disabling this is to remove extraneous type information (ie, fully qualified class names) when
  # serialized objects are persisted in Pekko persistence or replicated using Pekko distributed
  # data. Note that Pekko remoting already has manifest compression optimizations that address this,
  # so for types that just get sent over remoting, this offers no optimization.
  type-in-manifest = on

  # The type to use for deserialization.
  # This is only used if type-in-manifest is disabled. If set, this type will be used to
  # deserialize all messages. This is useful if the binding configuration you want to use when
  # disabling type in manifest cannot be expressed as a single type. Examples of when you might
  # use this include when changing serializers, so you don't want this serializer used for
  # serialization and you haven't declared any bindings for it, but you still want to be able to
  # deserialize messages that were serialized with this serializer, as well as situations where
  # you only want some sub types of a given Jackson polymorphic type to be serialized using this
  # serializer.
  deserialization-type = ""

  # Specific settings for jackson-json binding can be defined in this section to
  # override the settings in 'pekko.serialization.jackson'
  jackson-json {}

  # Specific settings for jackson-cbor binding can be defined in this section to
  # override the settings in 'pekko.serialization.jackson'
  jackson-cbor {}

  # Issue #28918 for compatibility with data serialized with JacksonCborSerializer in
  # Akka 2.6.4 or earlier, which was plain JSON format.
  jackson-cbor-264 = ${pekko.serialization.jackson.jackson-cbor}

}

Date/time format

WRITE_DATES_AS_TIMESTAMPS and WRITE_DURATIONS_AS_TIMESTAMPS are by default disabled, which means that date/time fields are serialized in ISO-8601 (rfc3339) yyyy-MM-dd'T'HH:mm:ss.SSSZ format instead of numeric arrays. This is better for interoperability but it is slower. If you don’t need the ISO format for interoperability with external systems you can change the following configuration for better performance of date/time fields.

copysourcepekko.serialization.jackson.serialization-features {
  WRITE_DATES_AS_TIMESTAMPS = on
  WRITE_DURATIONS_AS_TIMESTAMPS = on
}

Jackson is still be able to deserialize the other format independent of this setting.