Serialization with Jackson
Dependency
To use Jackson Serialization, you must add the following dependency in your project:
- sbt
val PekkoVersion = "1.0.3" libraryDependencies += "org.apache.pekko" %% "pekko-serialization-jackson" % PekkoVersion
- Maven
<properties> <scala.binary.version>2.13</scala.binary.version> </properties> <dependencyManagement> <dependencies> <dependency> <groupId>org.apache.pekko</groupId> <artifactId>pekko-bom_${scala.binary.version}</artifactId> <version>1.0.3</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <dependencies> <dependency> <groupId>org.apache.pekko</groupId> <artifactId>pekko-serialization-jackson_${scala.binary.version}</artifactId> </dependency> </dependencies>
- Gradle
def versions = [ ScalaBinary: "2.13" ] dependencies { implementation platform("org.apache.pekko:pekko-bom_${versions.ScalaBinary}:1.0.3") implementation "org.apache.pekko:pekko-serialization-jackson_${versions.ScalaBinary}" }
Introduction
You find general concepts for for Pekko serialization in the Serialization section. This section describes how to use the Jackson serializer for application specific messages and persistent events and snapshots.
Jackson has support for both text based JSON and binary formats.
In many cases ordinary classes can be serialized by Jackson without any additional hints, but sometimes annotations are needed to specify how to convert the objects to JSON/bytes.
Usage
To enable Jackson serialization for a class you need to configure it or one of its super classes in serialization-bindings configuration. Typically you will create a marker traitinterface for that purpose and let the messages extendimplement that.
- Scala
-
source
/** * Marker interface for messages, events and snapshots that are serialized with Jackson. */ trait MySerializable final case class Message(name: String, nr: Int) extends MySerializable
- Java
-
source
/** Marker interface for messages, events and snapshots that are serialized with Jackson. */ public interface MySerializable {} class MyMessage implements MySerializable { public final String name; public final int nr; public MyMessage(String name, int nr) { this.name = name; this.nr = nr; } }
Then you configure the class name of the marker traitinterface in serialization-bindings
to one of the supported Jackson formats: jackson-json
or jackson-cbor
sourcepekko.actor {
serialization-bindings {
"com.myservice.MySerializable" = jackson-json
}
}
A good convention would be to name the marker interface CborSerializable
or JsonSerializable
. In this documentation we have used MySerializable
to make it clear that the marker interface itself is not provided by Pekko.
That is all that is needed for basic classes where Jackson understands the structure. A few cases that requires annotations are described below.
Note that it’s only the top level class or its marker traitinterface that must be defined in serialization-bindings
, not nested classes that it references in member fields.
Add the -parameters
Java compiler option for usage by the ParameterNamesModule. It reduces the need for some annotations.
Security
For security reasons it is disallowed to bind the Jackson serializers to open ended types that might be a target for serialization gadgets, such as:
The deny list of possible serialization gadget classes defined by Jackson databind are checked and disallowed for deserialization.
Don’t use @JsonTypeInfo(use = Id.CLASS)
or ObjectMapper.enableDefaultTyping
since that is a security risk when using polymorphic types.
Formats
The following formats are supported, and you select which one to use in the serialization-bindings
configuration as described above.
jackson-json
- ordinary text based JSONjackson-cbor
- binary CBOR data format
The binary format is more compact, with slightly better performance than the JSON format.
Annotations
Constructor with single parameter
You might run into an exception like this:
MismatchedInputException: Cannot construct instance of `...` (although at least one Creator exists): cannot deserialize from Object value (no delegate- or property-based Creator)
That is probably because the class has a constructor with a single parameter, like:
- Java
-
source
public class SimpleCommand implements MySerializable { private final String name; public SimpleCommand(String name) { this.name = name; } }
That can be solved by adding @JsonCreator
or @JsonProperty
annotations:
- Java
-
source
public class SimpleCommand implements MySerializable { private final String name; @JsonCreator public SimpleCommand(String name) { this.name = name; } }
or
- Java
-
source
public class SimpleCommand implements MySerializable { private final String name; public SimpleCommand(@JsonProperty("name") String name) { this.name = name; } }
The ParameterNamesModule
is configured with JsonCreator.Mode.PROPERTIES
as described in the Jackson documentation
Polymorphic types
A polymorphic type is when a certain base type has multiple alternative implementations. When nested fields or collections are of polymorphic type the concrete implementations of the type must be listed with @JsonTypeInfo
and @JsonSubTypes
annotations.
Example:
- Scala
-
source
final case class Zoo(primaryAttraction: Animal) extends MySerializable @JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type") @JsonSubTypes( Array( new JsonSubTypes.Type(value = classOf[Lion], name = "lion"), new JsonSubTypes.Type(value = classOf[Elephant], name = "elephant"))) sealed trait Animal final case class Lion(name: String) extends Animal final case class Elephant(name: String, age: Int) extends Animal
- Java
-
source
public class Zoo implements MySerializable { public final Animal primaryAttraction; @JsonCreator public Zoo(Animal primaryAttraction) { this.primaryAttraction = primaryAttraction; } } @JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type") @JsonSubTypes({ @JsonSubTypes.Type(value = Lion.class, name = "lion"), @JsonSubTypes.Type(value = Elephant.class, name = "elephant") }) interface Animal {} public final class Lion implements Animal { public final String name; @JsonCreator public Lion(String name) { this.name = name; } } public final class Elephant implements Animal { public final String name; public final int age; public Elephant(String name, int age) { this.name = name; this.age = age; } }
If you haven’t defined the annotations you will see an exception like this:
InvalidDefinitionException: Cannot construct instance of `...` (no Creators, like default construct, exist): abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information
Note that this is not needed for a top level class, but for fields inside it. In this example Animal
is used inside of Zoo
, which is sent as a message or persisted. If Animal
was sent or persisted standalone the annotations are not needed because then it is the concrete subclasses Lion
or Elephant
that are serialized.
When specifying allowed subclasses with those annotations the class names will not be included in the serialized representation and that is important for preventing loading of malicious serialization gadgets when deserializing.
Don’t use @JsonTypeInfo(use = Id.CLASS)
or ObjectMapper.enableDefaultTyping
since that is a security risk when using polymorphic types.
ADT with trait and case object
It’s common in Scala to use a sealed trait and case objects to represent enums. If the values are case classes the @JsonSubTypes
annotation as described above works, but if the values are case objects it will not. The annotation requires a Class
and there is no way to define that in an annotation for a case object
.
The easiest workaround is to define the case objects as case class without any field.
Alternatively, you can define an intermediate trait for the case object and a custom deserializer for it. The example below builds on the previous Animal
sample by adding a fictitious, single instance, new animal, an Unicorn
.
- Scala
-
source
final case class Zoo(primaryAttraction: Animal) extends MySerializable @JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type") @JsonSubTypes( Array( new JsonSubTypes.Type(value = classOf[Lion], name = "lion"), new JsonSubTypes.Type(value = classOf[Elephant], name = "elephant"), new JsonSubTypes.Type(value = classOf[Unicorn], name = "unicorn"))) sealed trait Animal final case class Lion(name: String) extends Animal final case class Elephant(name: String, age: Int) extends Animal @JsonDeserialize(`using` = classOf[UnicornDeserializer]) sealed trait Unicorn extends Animal @JsonTypeName("unicorn") case object Unicorn extends Unicorn class UnicornDeserializer extends StdDeserializer[Unicorn](Unicorn.getClass) { // whenever we need to deserialize an instance of Unicorn trait, we return the object Unicorn override def deserialize(p: JsonParser, ctxt: DeserializationContext): Unicorn = Unicorn }
The case object Unicorn
can’t be used in a @JsonSubTypes
annotation, but its trait can. When serializing the case object we need to know which type tag to use, hence the @JsonTypeName
annotation on the object. When deserializing, Jackson will only know about the trait variant therefore we need a custom deserializer that returns the case object.
On the other hand, if the ADT only has case objects, you can solve it by implementing a custom serialization for the enums. Annotate the trait
with @JsonSerialize
and @JsonDeserialize
and implement the serialization with StdSerializer
and StdDeserializer
.
- Scala
-
source
import com.fasterxml.jackson.core.JsonGenerator import com.fasterxml.jackson.core.JsonParser import com.fasterxml.jackson.databind.DeserializationContext import com.fasterxml.jackson.databind.SerializerProvider import com.fasterxml.jackson.databind.annotation.JsonDeserialize import com.fasterxml.jackson.databind.annotation.JsonSerialize import com.fasterxml.jackson.databind.deser.std.StdDeserializer import com.fasterxml.jackson.databind.ser.std.StdSerializer @JsonSerialize(`using` = classOf[DirectionJsonSerializer]) @JsonDeserialize(`using` = classOf[DirectionJsonDeserializer]) sealed trait Direction object Direction { case object North extends Direction case object East extends Direction case object South extends Direction case object West extends Direction } class DirectionJsonSerializer extends StdSerializer[Direction](classOf[Direction]) { import Direction._ override def serialize(value: Direction, gen: JsonGenerator, provider: SerializerProvider): Unit = { val strValue = value match { case North => "N" case East => "E" case South => "S" case West => "W" } gen.writeString(strValue) } } class DirectionJsonDeserializer extends StdDeserializer[Direction](classOf[Direction]) { import Direction._ override def deserialize(p: JsonParser, ctxt: DeserializationContext): Direction = { p.getText match { case "N" => North case "E" => East case "S" => South case "W" => West } } } final case class Compass(currentDirection: Direction) extends MySerializable
Enumerations
Jackson support for Scala Enumerations defaults to serializing a Value
as a JsonObject
that includes a field with the "value"
and a field with the "type"
whose value is the FQCN of the enumeration. Jackson includes the @JsonScalaEnumeration
to statically specify the type information to a field. When using the @JsonScalaEnumeration
annotation the enumeration value is serialized as a JsonString.
- Scala
-
source
object Planet extends Enumeration { type Planet = Value val Mercury, Venus, Earth, Mars, Krypton = Value } // Uses default Jackson serialization format for Scala Enumerations final case class Alien(name: String, planet: Planet.Planet) extends TestMessage // Serializes planet values as a JsonString class PlanetType extends TypeReference[Planet.type] {} final case class Superhero(name: String, @JsonScalaEnumeration(classOf[PlanetType]) planet: Planet.Planet) extends TestMessage
Schema Evolution
When using Event Sourcing, but also for rolling updates, schema evolution becomes an important aspect of developing your application. The requirements as well as our own understanding of the business domain may (and will) change over time.
The Jackson serializer provides a way to perform transformations of the JSON tree model during deserialization. This is working in the same way for the textual and binary formats.
We will look at a few scenarios of how the classes may be evolved.
Remove Field
Removing a field can be done without any migration code. The Jackson serializer will ignore properties that does not exist in the class.
Add Optional Field
Adding an optional field can be done without any migration code. The default value will be NoneOptional.empty
.
Old class:
- Scala
-
source
case class ItemAdded(shoppingCartId: String, productId: String, quantity: Int) extends MySerializable
- Java
-
source
public class ItemAdded implements MySerializable { public final String shoppingCartId; public final String productId; public final int quantity; public ItemAdded(String shoppingCartId, String productId, int quantity) { this.shoppingCartId = shoppingCartId; this.productId = productId; this.quantity = quantity; } }
New class with a new optional discount
property and a new note
field with default value:
- Scala
-
source
case class ItemAdded(shoppingCartId: String, productId: String, quantity: Int, discount: Option[Double], note: String) extends MySerializable { // alternative constructor because `note` should have default value "" when not defined in json @JsonCreator def this(shoppingCartId: String, productId: String, quantity: Int, discount: Option[Double], note: Option[String]) = this(shoppingCartId, productId, quantity, discount, note.getOrElse("")) }
- Java
-
source
public class ItemAdded implements MySerializable { public final String shoppingCartId; public final String productId; public final int quantity; public final Optional<Double> discount; public final String note; @JsonCreator public ItemAdded( String shoppingCartId, String productId, int quantity, Optional<Double> discount, String note) { this.shoppingCartId = shoppingCartId; this.productId = productId; this.quantity = quantity; this.discount = discount; // default for note is "" if not included in json if (note == null) this.note = ""; else this.note = note; } public ItemAdded( String shoppingCartId, String productId, int quantity, Optional<Double> discount) { this(shoppingCartId, productId, quantity, discount, ""); } }
Add Mandatory Field
Let’s say we want to have a mandatory discount
property without default value instead:
- Scala
-
source
case class ItemAdded(shoppingCartId: String, productId: String, quantity: Int, discount: Double) extends MySerializable
- Java
-
source
public class ItemAdded implements MySerializable { public final String shoppingCartId; public final String productId; public final int quantity; public final double discount; public ItemAdded(String shoppingCartId, String productId, int quantity, double discount) { this.shoppingCartId = shoppingCartId; this.productId = productId; this.quantity = quantity; this.discount = discount; } }
To add a new mandatory field we have to use a JacksonMigration
JacksonMigration
class and set the default value in the migration code.
This is how a migration class would look like for adding a discount
field:
- Scala
-
source
import com.fasterxml.jackson.databind.JsonNode import com.fasterxml.jackson.databind.node.DoubleNode import com.fasterxml.jackson.databind.node.ObjectNode import org.apache.pekko.serialization.jackson.JacksonMigration class ItemAddedMigration extends JacksonMigration { override def currentVersion: Int = 2 override def transform(fromVersion: Int, json: JsonNode): JsonNode = { val root = json.asInstanceOf[ObjectNode] if (fromVersion <= 1) { root.set("discount", DoubleNode.valueOf(0.0)) } root } }
- Java
-
source
import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.node.DoubleNode; import com.fasterxml.jackson.databind.node.ObjectNode; import org.apache.pekko.serialization.jackson.JacksonMigration; public class ItemAddedMigration extends JacksonMigration { @Override public int currentVersion() { return 2; } @Override public JsonNode transform(int fromVersion, JsonNode json) { ObjectNode root = (ObjectNode) json; if (fromVersion <= 1) { root.set("discount", DoubleNode.valueOf(0.0)); } return root; } }
Override the currentVersion
currentVersion()
method to define the version number of the current (latest) version. The first version, when no migration was used, is always 1. Increase this version number whenever you perform a change that is not backwards compatible without migration code.
Implement the transformation of the old JSON structure to the new JSON structure in the transform(fromVersion, jsonNode)
transform(fromVersion, jsonNode)
method. The JsonNode
is mutable so you can add and remove fields, or change values. Note that you have to cast to specific sub-classes such as ObjectNode
and ArrayNode
to get access to mutators.
The migration class must be defined in configuration file:
sourcepekko.serialization.jackson.migrations {
"com.myservice.event.ItemAdded" = "com.myservice.event.ItemAddedMigration"
}
The same thing could have been done for the note
field, adding a default value of ""
in the ItemAddedMigration
.
Rename Field
Let’s say that we want to rename the productId
field to itemId
in the previous example.
- Scala
-
source
case class ItemAdded(shoppingCartId: String, itemId: String, quantity: Int) extends MySerializable
- Java
-
source
public class ItemAdded implements MySerializable { public final String shoppingCartId; public final String itemId; public final int quantity; public ItemAdded(String shoppingCartId, String itemId, int quantity) { this.shoppingCartId = shoppingCartId; this.itemId = itemId; this.quantity = quantity; } }
The migration code would look like:
- Scala
-
source
import org.apache.pekko.serialization.jackson.JacksonMigration import com.fasterxml.jackson.databind.JsonNode import com.fasterxml.jackson.databind.node.ObjectNode class ItemAddedMigration extends JacksonMigration { override def currentVersion: Int = 2 override def transform(fromVersion: Int, json: JsonNode): JsonNode = { val root = json.asInstanceOf[ObjectNode] if (fromVersion <= 1) { root.set[JsonNode]("itemId", root.get("productId")) root.remove("productId") } root } }
- Java
-
source
import org.apache.pekko.serialization.jackson.JacksonMigration; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.node.ObjectNode; public class ItemAddedMigration extends JacksonMigration { @Override public int currentVersion() { return 2; } @Override public JsonNode transform(int fromVersion, JsonNode json) { ObjectNode root = (ObjectNode) json; if (fromVersion <= 1) { root.set("itemId", root.get("productId")); root.remove("productId"); } return root; } }
Structural Changes
In a similar way we can do arbitrary structural changes.
Old class:
- Scala
-
source
case class Customer(name: String, street: String, city: String, zipCode: String, country: String) extends MySerializable
- Java
-
source
public class Customer implements MySerializable { public final String name; public final String street; public final String city; public final String zipCode; public final String country; public Customer(String name, String street, String city, String zipCode, String country) { this.name = name; this.street = street; this.city = city; this.zipCode = zipCode; this.country = country; } }
New class:
- Scala
-
source
case class Customer(name: String, shippingAddress: Address, billingAddress: Option[Address]) extends MySerializable
- Java
-
source
public class Customer implements MySerializable { public final String name; public final Address shippingAddress; public final Optional<Address> billingAddress; public Customer(String name, Address shippingAddress, Optional<Address> billingAddress) { this.name = name; this.shippingAddress = shippingAddress; this.billingAddress = billingAddress; } }
with the Address
class:
- Scala
-
source
case class Address(street: String, city: String, zipCode: String, country: String) extends MySerializable
- Java
-
source
public class Address { public final String street; public final String city; public final String zipCode; public final String country; public Address(String street, String city, String zipCode, String country) { this.street = street; this.city = city; this.zipCode = zipCode; this.country = country; } }
The migration code would look like:
- Scala
-
source
import org.apache.pekko.serialization.jackson.JacksonMigration import com.fasterxml.jackson.databind.JsonNode import com.fasterxml.jackson.databind.node.ObjectNode import scala.annotation.nowarn class CustomerMigration extends JacksonMigration { override def currentVersion: Int = 2 @nowarn("msg=deprecated") override def transform(fromVersion: Int, json: JsonNode): JsonNode = { val root = json.asInstanceOf[ObjectNode] if (fromVersion <= 1) { val shippingAddress = root.`with`("shippingAddress") shippingAddress.set[JsonNode]("street", root.get("street")) shippingAddress.set[JsonNode]("city", root.get("city")) shippingAddress.set[JsonNode]("zipCode", root.get("zipCode")) shippingAddress.set[JsonNode]("country", root.get("country")) root.remove("street") root.remove("city") root.remove("zipCode") root.remove("country") } root } }
- Java
-
source
import org.apache.pekko.serialization.jackson.JacksonMigration; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.node.ObjectNode; public class CustomerMigration extends JacksonMigration { @Override public int currentVersion() { return 2; } @Override public JsonNode transform(int fromVersion, JsonNode json) { ObjectNode root = (ObjectNode) json; if (fromVersion <= 1) { ObjectNode shippingAddress = root.with("shippingAddress"); shippingAddress.set("street", root.get("street")); shippingAddress.set("city", root.get("city")); shippingAddress.set("zipCode", root.get("zipCode")); shippingAddress.set("country", root.get("country")); root.remove("street"); root.remove("city"); root.remove("zipCode"); root.remove("country"); } return root; } }
Rename Class
It is also possible to rename the class. For example, let’s rename OrderAdded
to OrderPlaced
.
Old class:
- Scala
-
source
case class OrderAdded(shoppingCartId: String) extends MySerializable
- Java
-
source
public class OrderAdded implements MySerializable { public final String shoppingCartId; @JsonCreator public OrderAdded(String shoppingCartId) { this.shoppingCartId = shoppingCartId; } }
New class:
- Scala
-
source
case class OrderPlaced(shoppingCartId: String) extends MySerializable
- Java
-
source
public class OrderPlaced implements MySerializable { public final String shoppingCartId; @JsonCreator public OrderPlaced(String shoppingCartId) { this.shoppingCartId = shoppingCartId; } }
The migration code would look like:
- Scala
-
source
class OrderPlacedMigration extends JacksonMigration { override def currentVersion: Int = 2 override def transformClassName(fromVersion: Int, className: String): String = classOf[OrderPlaced].getName override def transform(fromVersion: Int, json: JsonNode): JsonNode = json }
- Java
-
source
public class OrderPlacedMigration extends JacksonMigration { @Override public int currentVersion() { return 2; } @Override public String transformClassName(int fromVersion, String className) { return OrderPlaced.class.getName(); } @Override public JsonNode transform(int fromVersion, JsonNode json) { return json; } }
Note the override of the transformClassName(fromVersion, className)
transformClassName(fromVersion, className)
method to define the new class name.
That type of migration must be configured with the old class name as key. The actual class can be removed.
sourcepekko.serialization.jackson.migrations {
"com.myservice.event.OrderAdded" = "com.myservice.event.OrderPlacedMigration"
}
Remove from serialization-bindings
When a class is not used for serialization any more it can be removed from serialization-bindings
but to still allow deserialization it must then be listed in the allowed-class-prefix
configuration. This is useful for example during rolling update with serialization changes, or when reading old stored data. It can also be used when changing from Jackson serializer to another serializer (e.g. Protobuf) and thereby changing the serialization binding, but it should still be possible to deserialize old data with Jackson.
sourcepekko.serialization.jackson.allowed-class-prefix =
["com.myservice.event.OrderAdded", "com.myservice.command"]
It’s a list of class names or prefixes of class names.
Rolling updates
When doing a rolling update, for a period of time there are two different binaries running in production. If the schema has evolved requiring a new schema version, the data serialized by the new binary will be unreadable from the old binary. This situation causes transient errors on the processes running the old binary. This service degradation is usually fine since the rolling update will eventually complete and all old processes will be replaced with the new binary. To avoid this service degradation you can also use forward-one support in your schema evolutions.
To complete a no-degradation rolling update, you need to make two deployments. First, deploy a new binary which can read the new schema but still uses the old schema. Then, deploy a second binary which serializes data using the new schema and drops the downcasting code from the migration.
Let’s take, for example, the case above where we renamed a field.
The starting schema is:
- Scala
-
source
case class ItemAdded(shoppingCartId: String, productId: String, quantity: Int) extends MySerializable
- Java
-
source
public class ItemAdded implements MySerializable { public final String shoppingCartId; public final String productId; public final int quantity; public ItemAdded(String shoppingCartId, String productId, int quantity) { this.shoppingCartId = shoppingCartId; this.productId = productId; this.quantity = quantity; } }
In a first deployment, we still don’t make any change to the event class:
- Scala
-
source
case class ItemAdded(shoppingCartId: String, productId: String, quantity: Int) extends MySerializable
- Java
-
source
public class ItemAdded implements MySerializable { public final String shoppingCartId; public final String productId; public final int quantity; public ItemAdded(String shoppingCartId, String productId, int quantity) { this.shoppingCartId = shoppingCartId; this.productId = productId; this.quantity = quantity; } }
but we introduce a migration that can read the newer schema which is versioned 2
:
- Scala
-
source
import org.apache.pekko.serialization.jackson.JacksonMigration import com.fasterxml.jackson.databind.JsonNode import com.fasterxml.jackson.databind.node.ObjectNode class ItemAddedMigration extends JacksonMigration { // Data produced in this node is still produced using the version 1 of the schema override def currentVersion: Int = 1 override def supportedForwardVersion: Int = 2 override def transform(fromVersion: Int, json: JsonNode): JsonNode = { val root = json.asInstanceOf[ObjectNode] if (fromVersion == 2) { // When receiving an event of version 2 we down-cast it to the version 1 of the schema root.set[JsonNode]("productId", root.get("itemId")) root.remove("itemId") } root } }
- Java
-
source
import org.apache.pekko.serialization.jackson.JacksonMigration; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.node.ObjectNode; public class ItemAddedMigration extends JacksonMigration { // Data produced in this node is still produced using the version 1 of the schema @Override public int currentVersion() { return 1; } @Override public int supportedForwardVersion() { return 2; } @Override public JsonNode transform(int fromVersion, JsonNode json) { ObjectNode root = (ObjectNode) json; if (fromVersion == 2) { // When receiving an event of version 2 we down-cast it to the version 1 of the schema root.set("productId", root.get("itemId")); root.remove("itemId"); } return root; } }
Once all running nodes have the new migration code which can read version 2
of ItemAdded
we can proceed with the second step. So, we deploy the updated event:
- Scala
-
source
case class ItemAdded(shoppingCartId: String, itemId: String, quantity: Int) extends MySerializable
- Java
-
source
public class ItemAdded implements MySerializable { public final String shoppingCartId; public final String itemId; public final int quantity; public ItemAdded(String shoppingCartId, String itemId, int quantity) { this.shoppingCartId = shoppingCartId; this.itemId = itemId; this.quantity = quantity; } }
and the final migration code which no longer needs forward-compatibility code:
- Scala
-
source
import org.apache.pekko.serialization.jackson.JacksonMigration import com.fasterxml.jackson.databind.JsonNode import com.fasterxml.jackson.databind.node.ObjectNode class ItemAddedMigration extends JacksonMigration { override def currentVersion: Int = 2 override def transform(fromVersion: Int, json: JsonNode): JsonNode = { val root = json.asInstanceOf[ObjectNode] if (fromVersion <= 1) { root.set[JsonNode]("itemId", root.get("productId")) root.remove("productId") } root } }
- Java
-
source
import org.apache.pekko.serialization.jackson.JacksonMigration; import com.fasterxml.jackson.databind.JsonNode; import com.fasterxml.jackson.databind.node.ObjectNode; public class ItemAddedMigration extends JacksonMigration { @Override public int currentVersion() { return 2; } @Override public JsonNode transform(int fromVersion, JsonNode json) { ObjectNode root = (ObjectNode) json; if (fromVersion <= 1) { root.set("itemId", root.get("productId")); root.remove("productId"); } return root; } }
Jackson Modules
The following Jackson modules are enabled by default:
sourcepekko.serialization.jackson {
# The Jackson JSON serializer will register these modules.
jackson-modules += "org.apache.pekko.serialization.jackson.PekkoJacksonModule"
# PekkoTypedJacksonModule optionally included if pekko-actor-typed is in classpath
jackson-modules += "org.apache.pekko.serialization.jackson.PekkoTypedJacksonModule"
# PekkoStreamsModule optionally included if pekko-streams is in classpath
jackson-modules += "org.apache.pekko.serialization.jackson.PekkoStreamJacksonModule"
jackson-modules += "com.fasterxml.jackson.module.paramnames.ParameterNamesModule"
jackson-modules += "com.fasterxml.jackson.datatype.jdk8.Jdk8Module"
jackson-modules += "com.fasterxml.jackson.datatype.jsr310.JavaTimeModule"
jackson-modules += "com.fasterxml.jackson.module.scala.DefaultScalaModule"
}
You can amend the configuration pekko.serialization.jackson.jackson-modules
to enable other modules.
The ParameterNamesModule requires that the -parameters
Java compiler option is enabled.
Compression
JSON can be rather verbose and for large messages it can be beneficial to compress large payloads. For the jackson-json
binding the default configuration is:
source# Compression settings for the jackson-json binding
pekko.serialization.jackson.jackson-json.compression {
# Compression algorithm.
# - off : no compression
# - gzip : using common java gzip
# - lz4 : using lz4-java
algorithm = gzip
# If compression is enabled with the `algorithm` setting the payload is compressed
# when it's larger than this value.
compress-larger-than = 32 KiB
}
Supported compression algorithms are: gzip, lz4. Use ‘off’ to disable compression. Gzip is generally slower than lz4. Messages larger than the compress-larger-than
property are compressed.
Compression can be disabled by setting the algorithm
property to off
. It will still be able to decompress payloads that were compressed when serialized, e.g. if this configuration is changed.
For the jackson-cbor
and custom bindings other than jackson-json
compression is by default disabled, but can be enabled in the same way as the configuration shown above but replacing jackson-json
with the binding name (for example jackson-cbor
).
Using Pekko Serialization for embedded types
For types that already have a Pekko Serializer defined that are embedded in types serialized with Jackson the PekkoSerializationSerializer
PekkoSerializationSerializer
and PekkoSerializationDeserializer
PekkoSerializationDeserializer
can be used to Pekko Serialization for individual fields.
The serializer/deserializer are not enabled automatically. The @JsonSerialize
and @JsonDeserialize
annotation needs to be added to the fields containing the types to be serialized with Pekko Serialization.
The type will be embedded as an object with the fields:
- serId - the serializer id
- serManifest - the manifest for the type
- payload - base64 encoded bytes
Additional configuration
Configuration per binding
By default the configuration for the Jackson serializers and their ObjectMapper
s is defined in the pekko.serialization.jackson
section. It is possible to override that configuration in a more specific pekko.serialization.jackson.<binding name>
section.
sourcepekko.serialization.jackson.jackson-json {
serialization-features {
WRITE_DATES_AS_TIMESTAMPS = off
}
}
pekko.serialization.jackson.jackson-cbor {
serialization-features {
WRITE_DATES_AS_TIMESTAMPS = on
}
}
It’s also possible to define several bindings and use different configuration for them. For example, different settings for remote messages and persisted events.
sourcepekko.actor {
serializers {
jackson-json-message = "org.apache.pekko.serialization.jackson.JacksonJsonSerializer"
jackson-json-event = "org.apache.pekko.serialization.jackson.JacksonJsonSerializer"
}
serialization-identifiers {
jackson-json-message = 9001
jackson-json-event = 9002
}
serialization-bindings {
"com.myservice.MyMessage" = jackson-json-message
"com.myservice.MyEvent" = jackson-json-event
}
}
pekko.serialization.jackson {
jackson-json-message {
serialization-features {
WRITE_DATES_AS_TIMESTAMPS = on
}
}
jackson-json-event {
serialization-features {
WRITE_DATES_AS_TIMESTAMPS = off
}
}
}
Manifest-less serialization
When using the Jackson serializer for persistence, given that the fully qualified class name is stored in the manifest, this can result in a lot of wasted disk and IO used, especially when the events are small. To address this, a type-in-manifest
flag can be turned off, which will result in the class name not appearing in the manifest.
When deserializing, the Jackson serializer will use the type defined in deserialization-type
, if present, otherwise it will look for exactly one serialization binding class, and use that. For this to be useful, generally that single type must be a Polymorphic type, with all type information necessary to deserialize to the various sub types contained in the JSON message.
When switching serializers, there will be periods of time when you may have no serialization bindings declared for the type. In such circumstances, you must use the deserialization-type
configuration attribute to specify which type should be used to deserialize messages.
Since this configuration can only be applied to a single root type, you will usually only want to apply it to a per binding configuration, not to the regular jackson-json
or jackson-cbor
configurations.
sourcepekko.actor {
serializers {
jackson-json-event = "org.apache.pekko.serialization.jackson.JacksonJsonSerializer"
}
serialization-identifiers {
jackson-json-event = 9001
}
serialization-bindings {
"com.myservice.MyEvent" = jackson-json-event
}
}
pekko.serialization.jackson {
jackson-json-event {
type-in-manifest = off
# Since there is exactly one serialization binding declared for this
# serializer above, this is optional, but if there were none or many,
# this would be mandatory.
deserialization-type = "com.myservice.MyEvent"
}
}
Note that Pekko remoting already implements manifest compression, and so this optimization will have no significant impact for messages sent over remoting. It’s only useful for messages serialized for other purposes, such as persistence or distributed data.
Additional features
Additional Jackson serialization features can be enabled/disabled in configuration. The default values from Jackson are used aside from the following that are changed in Pekko’s default configuration.
sourcepekko.serialization.jackson {
# Configuration of the ObjectMapper serialization features.
# See com.fasterxml.jackson.databind.SerializationFeature
# Enum values corresponding to the SerializationFeature and their boolean value.
serialization-features {
# Date/time in ISO-8601 (rfc3339) yyyy-MM-dd'T'HH:mm:ss.SSSZ format
# as defined by com.fasterxml.jackson.databind.util.StdDateFormat
# For interoperability it's better to use the ISO format, i.e. WRITE_DATES_AS_TIMESTAMPS=off,
# but WRITE_DATES_AS_TIMESTAMPS=on has better performance.
WRITE_DATES_AS_TIMESTAMPS = off
WRITE_DURATIONS_AS_TIMESTAMPS = off
FAIL_ON_EMPTY_BEANS = off
}
# Configuration of the ObjectMapper deserialization features.
# See com.fasterxml.jackson.databind.DeserializationFeature
# Enum values corresponding to the DeserializationFeature and their boolean value.
deserialization-features {
FAIL_ON_UNKNOWN_PROPERTIES = off
}
# Configuration of the ObjectMapper mapper features.
# See com.fasterxml.jackson.databind.MapperFeature
# Enum values corresponding to the MapperFeature and their
# boolean values, for example:
#
# mapper-features {
# SORT_PROPERTIES_ALPHABETICALLY = on
# }
mapper-features {}
# Configuration of the ObjectMapper JsonParser features.
# See com.fasterxml.jackson.core.JsonParser.Feature
# Enum values corresponding to the JsonParser.Feature and their
# boolean value, for example:
#
# json-parser-features {
# ALLOW_SINGLE_QUOTES = on
# }
json-parser-features {}
# Configuration of the ObjectMapper JsonParser features.
# See com.fasterxml.jackson.core.JsonGenerator.Feature
# Enum values corresponding to the JsonGenerator.Feature and
# their boolean value, for example:
#
# json-generator-features {
# WRITE_NUMBERS_AS_STRINGS = on
# }
json-generator-features {}
# Configuration of the JsonFactory StreamReadFeature.
# See com.fasterxml.jackson.core.StreamReadFeature
# Enum values corresponding to the StreamReadFeatures and
# their boolean value, for example:
#
# stream-read-features {
# STRICT_DUPLICATE_DETECTION = on
# }
stream-read-features {}
# Configuration of the JsonFactory StreamWriteFeature.
# See com.fasterxml.jackson.core.StreamWriteFeature
# Enum values corresponding to the StreamWriteFeatures and
# their boolean value, for example:
#
# stream-write-features {
# WRITE_BIGDECIMAL_AS_PLAIN = on
# }
stream-write-features {}
# Configuration of the JsonFactory JsonReadFeature.
# See com.fasterxml.jackson.core.json.JsonReadFeature
# Enum values corresponding to the JsonReadFeatures and
# their boolean value, for example:
#
# json-read-features {
# ALLOW_SINGLE_QUOTES = on
# }
json-read-features {}
# Configuration of the JsonFactory JsonWriteFeature.
# See com.fasterxml.jackson.core.json.JsonWriteFeature
# Enum values corresponding to the JsonWriteFeatures and
# their boolean value, for example:
#
# json-write-features {
# WRITE_NUMBERS_AS_STRINGS = on
# }
json-write-features {}
# Configuration of the JsonFactory Visibility.
# See com.fasterxml.jackson.annotation.PropertyAccessor
# and com.fasterxml.jackson.annotation.JsonAutoDetect.Visibility
# Enum values. For example, to serialize only public fields
# overwrite the default values with:
#
# visibility {
# FIELD = PUBLIC_ONLY
# }
# Default: all fields (including private and protected) are serialized.
visibility {
FIELD = ANY
}
# Deprecated, use `allowed-class-prefix` instead
whitelist-class-prefix = []
# Additional classes that are allowed even if they are not defined in `serialization-bindings`.
# This is useful when a class is not used for serialization any more and therefore removed
# from `serialization-bindings`, but should still be possible to deserialize.
allowed-class-prefix = ${pekko.serialization.jackson.whitelist-class-prefix}
# settings for compression of the payload
compression {
# Compression algorithm.
# - off : no compression
# - gzip : using common java gzip
algorithm = off
# If compression is enabled with the `algorithm` setting the payload is compressed
# when it's larger than this value.
compress-larger-than = 0 KiB
}
# Whether the type should be written to the manifest.
# If this is off, then either deserialization-type must be defined, or there must be exactly
# one serialization binding declared for this serializer, and the type in that binding will be
# used as the deserialization type. This feature will only work if that type either is a
# concrete class, or if it is a supertype that uses Jackson polymorphism (ie, the
# @JsonTypeInfo annotation) to store type information in the JSON itself. The intention behind
# disabling this is to remove extraneous type information (ie, fully qualified class names) when
# serialized objects are persisted in Pekko persistence or replicated using Pekko distributed
# data. Note that Pekko remoting already has manifest compression optimizations that address this,
# so for types that just get sent over remoting, this offers no optimization.
type-in-manifest = on
# The type to use for deserialization.
# This is only used if type-in-manifest is disabled. If set, this type will be used to
# deserialize all messages. This is useful if the binding configuration you want to use when
# disabling type in manifest cannot be expressed as a single type. Examples of when you might
# use this include when changing serializers, so you don't want this serializer used for
# serialization and you haven't declared any bindings for it, but you still want to be able to
# deserialize messages that were serialized with this serializer, as well as situations where
# you only want some sub types of a given Jackson polymorphic type to be serialized using this
# serializer.
deserialization-type = ""
# Specific settings for jackson-json binding can be defined in this section to
# override the settings in 'pekko.serialization.jackson'
jackson-json {}
# Specific settings for jackson-cbor binding can be defined in this section to
# override the settings in 'pekko.serialization.jackson'
jackson-cbor {}
# Issue #28918 for compatibility with data serialized with JacksonCborSerializer in
# Akka 2.6.4 or earlier, which was plain JSON format.
jackson-cbor-264 = ${pekko.serialization.jackson.jackson-cbor}
}
Date/time format
WRITE_DATES_AS_TIMESTAMPS
and WRITE_DURATIONS_AS_TIMESTAMPS
are by default disabled, which means that date/time fields are serialized in ISO-8601 (rfc3339) yyyy-MM-dd'T'HH:mm:ss.SSSZ
format instead of numeric arrays. This is better for interoperability but it is slower. If you don’t need the ISO format for interoperability with external systems you can change the following configuration for better performance of date/time fields.
sourcepekko.serialization.jackson.serialization-features {
WRITE_DATES_AS_TIMESTAMPS = on
WRITE_DURATIONS_AS_TIMESTAMPS = on
}
Jackson is still be able to deserialize the other format independent of this setting.