Unmarshalling

“Unmarshalling” is the process of converting some kind of a lower-level representation, often a “wire format”, into a higher-level (object) structure. Other popular names for it are “Deserialization” or “Unpickling”.

In Apache Pekko HTTP “Unmarshalling” means the conversion of a lower-level source object, e.g. a MessageEntity (which forms the “entity body” of an HTTP request or response) or a full HttpRequestHttpRequest or HttpResponseHttpResponse, into an instance of type T.

Basic Design

Unmarshalling of instances of type A into instances of type B is performed by an Unmarshaller<A, B>Unmarshaller[A, B].

Apache Pekko HTTP also predefines a number of helpful aliases for the types of unmarshallers that you’ll likely work with most:

sourcetype FromEntityUnmarshaller[T] = Unmarshaller[HttpEntity, T]
type FromMessageUnmarshaller[T] = Unmarshaller[HttpMessage, T]
type FromResponseUnmarshaller[T] = Unmarshaller[HttpResponse, T]
type FromRequestUnmarshaller[T] = Unmarshaller[HttpRequest, T]
type FromByteStringUnmarshaller[T] = Unmarshaller[ByteString, T]
type FromStringUnmarshaller[T] = Unmarshaller[String, T]
type FromStrictFormFieldUnmarshaller[T] = Unmarshaller[StrictForm.Field, T]

At its core an Unmarshaller<A, B>Unmarshaller[A, B] is very similar to a function A => Future[B]Function<A, CompletionStage<B>> and as such quite a bit simpler than its marshalling counterpart. The process of unmarshalling does not have to support content negotiation which saves two additional layers of indirection that are required on the marshalling side.

Using unmarshallers

For an example on how to use an unmarshaller on the server side, see for example the Dynamic Routing Example. For the client side, see Processing Responses

Predefined Unmarshallers

Apache Pekko HTTP already predefines a number of unmarshallers for the most common types. Specifically these are:

Additional unmarshallers are available in separate modules for specific content types, such as JSON and XML.

Implicit Resolution

The unmarshalling infrastructure of Apache Pekko HTTP relies on a type-class based approach, which means that UnmarshallerUnmarshaller instances from a certain type A to a certain type B have to be available implicitly.

The implicits for most of the predefined unmarshallers in Apache Pekko HTTP are provided through the companion object of the UnmarshallerUnmarshaller trait. This means that they are always available and never need to be explicitly imported. Additionally, you can simply “override” them by bringing your own custom version into local scope.

Custom Unmarshallers

Apache Pekko HTTP gives you a few convenience tools for constructing unmarshallers for your own types. Usually you won’t have to “manually” implement the UnmarshallerUnmarshaller traitclass directly. Rather, it should be possible to use one of the convenience construction helpers defined on the UnmarshallerUnmarshaller companionUnmarshallerUnmarshaller:

Scala
source/**
 * Creates an `Unmarshaller` from the given function.
 */
def apply[A, B](f: ExecutionContext => A => Future[B]): Unmarshaller[A, B] =
  withMaterializer(ec => _ => f(ec))

def withMaterializer[A, B](f: ExecutionContext => Materializer => A => Future[B]): Unmarshaller[A, B] =
  new Unmarshaller[A, B] {
    def apply(a: A)(implicit ec: ExecutionContext, materializer: Materializer) =
      try f(ec)(materializer)(a)
      catch { case NonFatal(e) => FastFuture.failed(e) }
  }

/**
 * Helper for creating a synchronous `Unmarshaller` from the given function.
 */
def strict[A, B](f: A => B): Unmarshaller[A, B] = Unmarshaller(_ => a => FastFuture.successful(f(a)))

/**
 * Helper for creating a "super-unmarshaller" from a sequence of "sub-unmarshallers", which are tried
 * in the given order. The first successful unmarshalling of a "sub-unmarshallers" is the one produced by the
 * "super-unmarshaller".
 */
def firstOf[A, B](unmarshallers: Unmarshaller[A, B]*): Unmarshaller[A, B] = //...
Java
source<A, B> Unmarshaller<A, B> async(
    java.util.function.Function<A, java.util.concurrent.CompletionStage<B>> f);
<A, B> Unmarshaller<A, B> sync(java.util.function.Function<A, B> f);
<A, B> Unmarshaller<A, B> firstOf(Unmarshaller<A, B> u1, Unmarshaller<A, B> u2);

<A, B> Unmarshaller<A, B> firstOf(
    Unmarshaller<A, B> u1, Unmarshaller<A, B> u2, Unmarshaller<A, B> u3);

<A, B> Unmarshaller<A, B> firstOf(
    Unmarshaller<A, B> u1, Unmarshaller<A, B> u2, Unmarshaller<A, B> u3, Unmarshaller<A, B> u4);

<A, B> Unmarshaller<A, B> firstOf(
    Unmarshaller<A, B> u1,
    Unmarshaller<A, B> u2,
    Unmarshaller<A, B> u3,
    Unmarshaller<A, B> u4,
    Unmarshaller<A, B> u5);
Note

To avoid unnecessary memory pressure, unmarshallers should make sure to either fully consume the incoming entity data stream, or make sure it is properly cancelled on error. Failure to do so might keep the remaining part of the stream in memory for longer than necessary.

Deriving Unmarshallers

Sometimes you can save yourself some work by reusing existing unmarshallers for your custom ones. The idea is to “wrap” an existing unmarshaller with some logic to “re-target” it to your type.

Usually what you want to do is to transform the output of some existing unmarshaller and convert it to your type. For this type of unmarshaller transformation Apache Pekko HTTP defines these methods:

  • baseUnmarshaller.transform
  • baseUnmarshaller.map
  • baseUnmarshaller.mapWithInput
  • baseUnmarshaller.flatMap
  • baseUnmarshaller.flatMapWithInput
  • baseUnmarshaller.recover
  • baseUnmarshaller.withDefaultValue
  • baseUnmarshaller.mapWithCharset (only available for FromEntityUnmarshallers)
  • baseUnmarshaller.forContentTypes (only available for FromEntityUnmarshallers)
  • baseMarshaller.thenApply
  • baseMarshaller.flatMap
  • Unmarshaller.forMediaType (to derive from a HttpEntityHttpEntity unmarshaller)
  • Unmarshaller.forMediaTypes (to derive from a HttpEntityHttpEntity unmarshaller)

The method signatures should make their semantics relatively clear.

Using Unmarshallers

In many places throughout Apache Pekko HTTP unmarshallers are used implicitly, e.g. when you want to access the entity of a request using the Routing DSL.

However, you can also use the unmarshalling infrastructure directly if you wish, which can be useful for example in tests. The best entry point for this is the org.apache.pekko.http.scaladsl.unmarshalling.Unmarshal objectorg.apache.pekko.http.javadsl.unmarshalling.StringUnmarshallers class, which you can use like this:

Scala
sourceimport org.apache.pekko.http.scaladsl.unmarshalling.Unmarshal
import system.dispatcher // Optional ExecutionContext (default from Materializer)

import scala.concurrent.Await
import scala.concurrent.duration._

val intFuture = Unmarshal("42").to[Int]
val int = Await.result(intFuture, 1.second) // don't block in non-test code!
int shouldEqual 42

val boolFuture = Unmarshal("off").to[Boolean]
val bool = Await.result(boolFuture, 1.second) // don't block in non-test code!
bool shouldBe false
Java
sourceimport org.apache.pekko.http.javadsl.model.*;
import org.apache.pekko.http.javadsl.unmarshalling.StringUnmarshallers;
import org.apache.pekko.http.javadsl.unmarshalling.Unmarshaller;

import java.util.concurrent.CompletionStage;
import java.util.concurrent.TimeUnit;

CompletionStage<Integer> integerStage = StringUnmarshallers.INTEGER.unmarshal("42", system());
int integer =
    integerStage
        .toCompletableFuture()
        .get(1, TimeUnit.SECONDS); // don't block in non-test code!
assertEquals(integer, 42);

CompletionStage<Boolean> boolStage = StringUnmarshallers.BOOLEAN.unmarshal("off", system());
boolean bool =
    boolStage.toCompletableFuture().get(1, TimeUnit.SECONDS); // don't block in non-test code!
assertEquals(bool, false);