Encoding / Decoding

The HTTP spec defines a Content-Encoding header, which signifies whether the entity body of an HTTP message is “encoded” and, if so, by which algorithm. The only commonly used content encodings are compression algorithms.

Currently, Apache Pekko HTTP supports the compression and decompression of HTTP requests and responses with the gzip, deflate or zstd encodings.

zstd support needs the optional com.github.luben:zstd-jni dependency (1.5.7-25 or later) on the classpath, so it is not one of the default encodings and has to be requested explicitly, for example with encodeResponseWith and decodeRequestWith. Compression dictionaries are not supported. As required by RFC 9659, zstd does not compress with a window larger than 8 MB, and by default does not decode messages that need a larger window. The core logic for this lives in the org.apache.pekko.http.scaladsl.coding package.org.apache.pekko.http.javadsl.coding.Coder enum class.

Server side

The support is not enabled automatically, but must be explicitly requested. For enabling message encoding/decoding with Routing DSL see the CodingDirectives.

Usually, it suffices to surround routes that should support response encoding by the encodeResponse directive, and routes that should support request decoding by the decodeRequest directive. Those directives will automatically and transparently enable support for encodings and negotiating which encoding to use out of the default encodings supported. The set of predefined MediaTypesMediaTypes contains a hint whether content of a given media type would benefit from compression.

Client side

Responses are not decoded automatically on the client-side.

Coders.decodeMessageCoder.decodeResponse decodes a response with the coder that matches its Content-Encoding header. A response whose encoding is not supported by any of the given coders is returned unchanged, with its Content-Encoding header kept:

Scala
sourceimport org.apache.pekko
import pekko.actor.ActorSystem
import pekko.http.scaladsl.Http
import pekko.http.scaladsl.coding.Coders
import pekko.http.scaladsl.model._

import scala.concurrent.Future

implicit val system = ActorSystem()
implicit val ec: ExecutionContext = system.dispatcher

val http = Http()

val requests: Seq[HttpRequest] = Seq(
  "https://httpbin.org/gzip", // Content-Encoding: gzip in response
  "https://httpbin.org/deflate", // Content-Encoding: deflate in response
  "https://httpbin.org/get" // no Content-Encoding in response
).map(uri => HttpRequest(uri = uri))

// picks the decoder matching the Content-Encoding of the response, add Coders.Zstd to also decode zstd
def decodeResponse(response: HttpResponse): HttpResponse =
  Coders.decodeMessage(response, Coders.DefaultCoders)

val futureResponses: Future[Seq[HttpResponse]] =
  Future.traverse(requests)(http.singleRequest(_).map(decodeResponse))

futureResponses.futureValue.foreach { resp =>
  system.log.info("response is {}", resp.toStrict(1.second).futureValue)
}

system.terminate()
Java
sourceimport java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionStage;
import java.util.function.Function;
import org.apache.pekko.actor.ActorSystem;
import org.apache.pekko.http.javadsl.Http;
import org.apache.pekko.http.javadsl.coding.Coder;
import org.apache.pekko.http.javadsl.model.HttpRequest;
import org.apache.pekko.http.javadsl.model.HttpResponse;

public class HttpClientDecodingExampleTest {

  public static void main(String[] args) throws Exception {

    final ActorSystem system = ActorSystem.create();

    final List<HttpRequest> httpRequests =
        List.of(
            HttpRequest.create("https://httpbin.org/gzip"), // Content-Encoding: gzip in response
            HttpRequest.create(
                "https://httpbin.org/deflate"), // Content-Encoding: deflate in response
            HttpRequest.create("https://httpbin.org/get") // no Content-Encoding in response
            );

    final Http http = Http.get(system);

    // picks the coder matching the Content-Encoding of the response, add Coder.Zstd to also
    // decode zstd
    final Function<HttpResponse, HttpResponse> decodeResponse =
        response -> Coder.decodeResponse(response, Coder.Gzip, Coder.Deflate, Coder.NoCoding);

    List<CompletableFuture<HttpResponse>> futureResponses =
        httpRequests.stream()
            .map(req -> http.singleRequest(req).thenApply(decodeResponse))
            .map(CompletionStage::toCompletableFuture)
            .toList();

    for (CompletableFuture<HttpResponse> futureResponse : futureResponses) {
      final HttpResponse httpResponse = futureResponse.get();
      system
          .log()
          .info(
              "response is: "
                  + httpResponse.entity().toStrict(1000, system).toCompletableFuture().get());
    }

    system.terminate();
  }
}