Discovery

The Pekko Discovery API enables service discovery to be provided by different technologies. It allows to delegate endpoint lookup so that services can be configured depending on the environment by other means than configuration files.

Implementations provided by the Pekko Discovery module are

• Configuration (HOCON)
• DNS (SRV records)
• Aggregate multiple discovery methods

In addition, the Pekko Management toolbox contains Pekko Discovery implementations for

• Kubernetes API
• AWS API: EC2 Tag-Based Discovery
• AWS API: ECS Discovery
• Consul
• Marathon API

Module info

sbt

val PekkoVersion = "1.0.2"
libraryDependencies += "org.apache.pekko" %% "pekko-discovery" % PekkoVersion

Gradle

def versions = [
  ScalaBinary: "2.13"
]
dependencies {
  implementation platform("org.apache.pekko:pekko-bom_${versions.ScalaBinary}:1.0.2")

  implementation "org.apache.pekko:pekko-discovery_${versions.ScalaBinary}"
}

Maven

<properties>
  <scala.binary.version>2.13</scala.binary.version>
</properties>
<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.apache.pekko</groupId>
      <artifactId>pekko-bom_${scala.binary.version}</artifactId>
      <version>1.0.2</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>
<dependencies>
  <dependency>
    <groupId>org.apache.pekko</groupId>
    <artifactId>pekko-discovery_${scala.binary.version}</artifactId>
  </dependency>
</dependencies>

Project Info: Pekko Discovery
Artifact	org.apache.pekko pekko-discovery 1.0.2 Snapshots are available
JDK versions	OpenJDK 8 OpenJDK 11 OpenJDK 17 OpenJDK 21
Scala versions	2.13.13, 2.12.19, 3.3.3
JPMS module name	pekko.discovery
License	Apache-2.0
Home page	https://pekko.apache.org/
API documentation	API (Scaladoc) API (Javadoc)
Forums	Apache Pekko User mailing list Apache Pekko Dev mailing list apache/pekko discussion
Release notes	Release Notes
Issues	Github issues
Sources	https://github.com/apache/pekko

How it works

Loading the extension:

Scala

copysourceimport org.apache.pekko.discovery.Discovery

val system = ActorSystem()
val serviceDiscovery = Discovery(system).discovery

Java

copysourceActorSystem as = ActorSystem.create();
ServiceDiscovery serviceDiscovery = Discovery.get(as).discovery();

A Lookup contains a mandatory serviceName and an optional portName and protocol. How these are interpreted is discovery method dependent e.g.DNS does an A/AAAA record query if any of the fields are missing and an SRV query for a full look up:

Scala

copysourceimport org.apache.pekko.discovery.Lookup

serviceDiscovery.lookup(Lookup("pekko.io"), 1.second)
// Convenience for a Lookup with only a serviceName
serviceDiscovery.lookup("pekko.io", 1.second)

Java

copysourceserviceDiscovery.lookup(Lookup.create("pekko.io"), Duration.ofSeconds(1));
// convenience for a Lookup with only a serviceName
serviceDiscovery.lookup("pekko.io", Duration.ofSeconds(1));

portName and protocol are optional and their meaning is interpreted by the method.

Scala

copysourceimport org.apache.pekko
import pekko.discovery.Lookup
import pekko.discovery.ServiceDiscovery.Resolved

val lookup: Future[Resolved] =
  serviceDiscovery.lookup(Lookup("pekko.io").withPortName("remoting").withProtocol("tcp"), 1.second)

Java

copysourceCompletionStage<ServiceDiscovery.Resolved> lookup =
    serviceDiscovery.lookup(
        Lookup.create("pekko.io").withPortName("remoting").withProtocol("tcp"),
        Duration.ofSeconds(1));

Port can be used when a service opens multiple ports e.g. a HTTP port and a Pekko remoting port.

Discovery Method: DNS

DNS discovery maps Lookup queries as follows:

• serviceName, portName and protocol set: SRV query in the form: _port._protocol.name Where the _s are added.
• Any query missing any of the fields is mapped to a A/AAAA query for the serviceName

The mapping between Pekko service discovery terminology and SRV terminology:

• SRV service = port
• SRV name = serviceName
• SRV protocol = protocol

Configure pekko-dns to be used as the discovery implementation in your application.conf:

copysourcepekko {
  discovery {
    method = pekko-dns
  }
}

From there on, you can use the generic API that hides the fact which discovery method is being used by calling:

Scala

copysourceimport org.apache.pekko
import pekko.discovery.Discovery
import pekko.discovery.ServiceDiscovery

val discovery: ServiceDiscovery = Discovery(system).discovery
// ...
val result: Future[ServiceDiscovery.Resolved] = discovery.lookup("pekko.io", resolveTimeout = 3.seconds)

Java

copysourceimport org.apache.pekko.discovery.Discovery;
import org.apache.pekko.discovery.ServiceDiscovery;

ServiceDiscovery discovery = Discovery.get(system).discovery();
// ...
CompletionStage<ServiceDiscovery.Resolved> result =
    discovery.lookup("foo", Duration.ofSeconds(3));

DNS records used

DNS discovery will use either A/AAAA records or SRV records depending on whether a Simple or Full lookup is issued. The advantage of SRV records is that they can include a port.

SRV records

Lookups with all the fields set become SRV queries. For example:

dig srv _service._tcp.pekko.test

; <<>> DiG 9.11.3-RedHat-9.11.3-6.fc28 <<>> srv service.tcp.pekko.test
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 60023
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 1, ADDITIONAL: 5

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 5ab8dd4622e632f6190f54de5b28bb8fb1b930a5333c3862 (good)
;; QUESTION SECTION:
;service.tcp.pekko.test.         IN      SRV

;; ANSWER SECTION:
_service._tcp.pekko.test.  86400   IN      SRV     10 60 5060 a-single.pekko.test.
_service._tcp.pekko.test.  86400   IN      SRV     10 40 5070 a-double.pekko.test.

In this case service.tcp.pekko.test resolves to a-single.pekko.test on port 5060 and a-double.pekko.test on port 5070. Currently discovery does not support the weightings.

A/AAAA records

Lookups with any fields missing become A/AAAA record queries. For example:

dig a-double.pekko.test

; <<>> DiG 9.11.3-RedHat-9.11.3-6.fc28 <<>> a-double.pekko.test
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11983
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 1, ADDITIONAL: 2

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 16e9815d9ca2514d2f3879265b28bad05ff7b4a82721edd0 (good)
;; QUESTION SECTION:
;a-double.pekko.test.            IN      A

;; ANSWER SECTION:
a-double.pekko.test.     86400   IN      A       192.168.1.21
a-double.pekko.test.     86400   IN      A       192.168.1.22

In this case a-double.pekko.test would resolve to 192.168.1.21 and 192.168.1.22.

Discovery Method: Configuration

Configuration currently ignores all fields apart from service name.

For simple use cases configuration can be used for service discovery. The advantage of using Pekko Discovery with configuration rather than your own configuration values is that applications can be migrated to a more sophisticated discovery method without any code changes.

Configure it to be used as discovery method in your application.conf

pekko {
  discovery.method = config
}

By default the services discoverable are defined in pekko.discovery.config.services and have the following format:

pekko.discovery.config.services = {
  service1 = {
    endpoints = [
      {
        host = "cat"
        port = 1233
      },
      {
        host = "dog"
        port = 1234
      }
    ]
  },
  service2 = {
    endpoints = []
  }
}

Where the above block defines two services, service1 and service2. Each service can have multiple endpoints.

Discovery Method: Aggregate multiple discovery methods

Aggregate discovery allows multiple discovery methods to be aggregated e.g. try and resolve via DNS and fall back to configuration.

To use aggregate discovery add its dependency as well as all of the discovery that you want to aggregate.

Configure aggregate as pekko.discovery.method and which discovery methods are tried and in which order.

pekko {
  discovery {
    method = aggregate
    aggregate {
      discovery-methods = ["pekko-dns", "config"]
    }
    config {
      services {
        service1 {
          endpoints = [
            {
              host = "host1"
              port = 1233
            },
            {
              host = "host2"
              port = 1234
            }
          ]
        }
      }
    }
  }
}

The above configuration will result in pekko-dns first being checked and if it fails or returns no targets for the given service name then config is queried which i configured with one service called service1 which two hosts host1 and host2.

Migrating from Pekko Management Discovery (before 1.0.0)

Pekko Discovery started out as a submodule of Pekko Management, before 1.0.0 of Pekko Management. Pekko Discovery is not compatible with those versions of Pekko Management Discovery.

At least version 1.0.0 of any Pekko Management module should be used if also using Pekko Discovery.

Migration steps:

• Any custom discovery method should now implement org.apache.pekko.discovery.ServiceDiscovery
• discovery-method now has to be a configuration location under pekko.discovery with at minimum a property class specifying the fully qualified name of the implementation of org.apache.pekko.discovery.ServiceDiscovery. Previous versions allowed this to be a class name or a fully qualified config location e.g. pekko.discovery.kubernetes-api rather than just kubernetes-api