Apache Pekko Connectors Kafka Documentation
The Apache Pekko Connectors project is an open source initiative to implement stream-aware and reactive integration pipelines for Java and Scala. It is built on top of Pekko Streams, and has been designed from the ground up to understand streaming natively and provide a DSL for reactive and stream-oriented programming, with built-in support for backpressure. Apache Pekko Streams is a Reactive Streams and JDK 9+ java.util.concurrent.Flow-compliant implementation and therefore fully interoperable with other implementations.
This Apache Pekko Connectors Kafka connector lets you connect Apache Kafka to Apache Pekko Streams. This project is a fork of Alpakka Kafka.
- Overview
- Producer
- Consumer
- Choosing a consumer
- Settings
- Offset Storage external to Kafka
- Offset Storage in Kafka - committing
- Offset Storage in Kafka & external
- Consume “at-most-once”
- Consume “at-least-once”
- Connecting Producer and Consumer
- Source per partition
- Sharing the KafkaConsumer instance
- Accessing KafkaConsumer metrics
- Accessing KafkaConsumer metadata
- Controlled shutdown
- Subscription
- React on Partition Assignment
- Consumer Metadata
- Service discovery
- Apache Pekko Cluster Sharding
- Error handling
- At-Least-Once Delivery
- Transactions
- Serialization
- Debugging
- Testing
- Production considerations
- Snapshots
1.1.0