Compression.zstd

Creates a flow that zstd-compresses a stream of ByteStrings.

Compression operators

Dependency

The zstd operators use zstd-jni, an optional dependency of Pekko Streams that has to be added explicitly:

sbt

libraryDependencies += "com.github.luben" % "zstd-jni" % "1.5.7-25"

Maven

<dependencies>
  <dependency>
    <groupId>com.github.luben</groupId>
    <artifactId>zstd-jni</artifactId>
    <version>1.5.7-25</version>
  </dependency>
</dependencies>

Gradle

dependencies {
  implementation "com.github.luben:zstd-jni:1.5.7-25"
}

Signature

Compression.zstdCompression.zstd Compression.zstdCompression.zstd Compression.zstdCompression.zstd

Description

Creates a flow that zstd-compresses a stream of ByteStrings. Note that the compressor will SYNC_FLUSH after every ByteStringByteString so that it is guaranteed that every ByteStringByteString coming out of the flow can be fully decompressed without waiting for additional data. This may come at a compression performance cost for very small chunks.

Use the zstd(level, dictionary) overload to control the compression level and, optionally, to compress with a dictionary. Output compressed with a dictionary can only be decompressed with the same dictionary, see zstdDecompress.

The zstd(settings) overload takes a ZstdCompressionSettingsZstdCompressionSettings, which sets the level and dictionary as well as:

• withAutoFlush: if true (the default), the compressor flushes after every single element in the stream. If it is false, the compressor only emits output when the compression algorithm produces it, which can improve the compression ratio for small chunks, but an emitted ByteString may not be fully decompressible until more data arrives.
• withWindowLog: limits the window size to 2^windowLog bytes. The decompressor has to buffer a window of this size, and since the size of a stream is not known up front, a frame declares the default window of its compression level, which is 128 MiB (windowLog 27) for the highest levels. Set windowLog when the output is decompressed by a peer that limits the window, for example to 23 (8 MiB) for the HTTP zstd content-coding, as required by RFC 9659. The default of 0 uses the default of the compression level.

Reactive Streams semantics