Distributed Publish Subscribe in Cluster

You are viewing the documentation for the new actor APIs, to view the Pekko Classic documentation, see Classic Distributed Publish Subscribe.

Module info

The distributed publish subscribe topic API is available and usable with the core (/actor-typed module, however it will only be distributed when used in a clustered application:

sbt
val PekkoVersion = "1.0.3"
libraryDependencies += "org.apache.pekko" %% "(/cluster-typed" % PekkoVersion
Maven
<properties>
  <scala.binary.version>2.13</scala.binary.version>
</properties>
<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.apache.pekko</groupId>
      <artifactId>(/bom_${scala.binary.version}</artifactId>
      <version>1.0.3</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>
<dependencies>
  <dependency>
    <groupId>org.apache.pekko</groupId>
    <artifactId>(/cluster-typed_${scala.binary.version}</artifactId>
  </dependency>
</dependencies>
Gradle
def versions = [
  ScalaBinary: "2.13"
]
dependencies {
  implementation platform("org.apache.pekko:(/bom_${versions.ScalaBinary}:1.0.3")

  implementation "org.apache.pekko:(/cluster-typed_${versions.ScalaBinary}"
}

The Topic Actor

Distributed publish subscribe is achieved by representing each pub sub topic with an actor, actor.typed.pubsub.Topicactor.typed.pubsub.Topic.

The topic actor needs to run on each node where subscribers will live or that wants to publish messages to the topic.

The identity of the topic is a tuple of the type of messages that can be published and a string topic name but it is recommended to not define multiple topics with different types and the same topic name.

Scala
@@snip [PubSubExample.scala](/(/actor-typed-tests/src/test/scala/docs/org/apache/pekko/typed/pubsub/PubSubExample.scala) { #start-topic }
Java
@@snip [PubSubExample.java](/(/actor-typed-tests/src/test/java/jdocs/org/apache/pekko/typed/pubsub/PubSubExample.java) { #start-topic }

Local actors can then subscribe to the topic (and unsubscribe from it):

Scala
@@snip [PubSubExample.scala](/(/actor-typed-tests/src/test/scala/docs/org/apache/pekko/typed/pubsub/PubSubExample.scala) { #subscribe }
Java
@@snip [PubSubExample.java](/(/actor-typed-tests/src/test/java/jdocs/org/apache/pekko/typed/pubsub/PubSubExample.java) { #subscribe }

And publish messages to the topic:

Scala
@@snip [PubSubExample.scala](/(/actor-typed-tests/src/test/scala/docs/org/apache/pekko/typed/pubsub/PubSubExample.scala) { #publish }
Java
@@snip [PubSubExample.java](/(/actor-typed-tests/src/test/java/jdocs/org/apache/pekko/typed/pubsub/PubSubExample.java) { #publish }

Pub Sub Scalability

Each topic is represented by one Receptionist service key meaning that the number of topics will scale to thousands or tens of thousands but for higher numbers of topics will require custom solutions. It also means that a very high turnaround of unique topics will not work well and for such use cases a custom solution is advised.

The topic actor acts as a proxy and delegates to the local subscribers handling deduplication so that a published message is only sent once to a node regardless of how many subscribers there are to the topic on that node.

When a topic actor has no subscribers for a topic it will deregister itself from the receptionist meaning published messages for the topic will not be sent to it.

Delivery Guarantee

As in Message Delivery Reliability of Pekko, message delivery guarantee in distributed pub sub modes is at-most-once delivery. In other words, messages can be lost over the wire. In addition to that the registry of nodes which have subscribers is eventually consistent meaning that subscribing an actor on one node will have a short delay before it is known on other nodes and published to.

If you are looking for at-least-once delivery guarantee, we recommend [Pekko Connector Kafka](https://pekko.apache.org/docs/alp(/kafka/current/).

1.0.3