Persistence Query for LevelDB
The LevelDB journal and query plugin is deprecated and it is not advised to build new applications with it. As a replacement we recommend using Pekko Persistence JDBC.
Dependency
To use Persistence Query, you must add the following dependency in your project:
- sbt
val PekkoVersion = "1.0.3" libraryDependencies += "org.apache.pekko" %% "pekko-persistence-query" % PekkoVersion
- Maven
<properties> <scala.binary.version>2.13</scala.binary.version> </properties> <dependencyManagement> <dependencies> <dependency> <groupId>org.apache.pekko</groupId> <artifactId>pekko-bom_${scala.binary.version}</artifactId> <version>1.0.3</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement> <dependencies> <dependency> <groupId>org.apache.pekko</groupId> <artifactId>pekko-persistence-query_${scala.binary.version}</artifactId> </dependency> </dependencies>
- Gradle
def versions = [ ScalaBinary: "2.13" ] dependencies { implementation platform("org.apache.pekko:pekko-bom_${versions.ScalaBinary}:1.0.3") implementation "org.apache.pekko:pekko-persistence-query_${versions.ScalaBinary}" }
This will also add dependency on the pekko-persistence module.
Introduction
This is documentation for the LevelDB implementation of the Persistence Query API. Note that implementations for other journals may have different semantics.
How to get the ReadJournal
The ReadJournal
is retrieved via the org.apache.pekko.persistence.query.PersistenceQuery
extension:
- Scala
-
source
import org.apache.pekko import pekko.persistence.query.PersistenceQuery import pekko.persistence.query.journal.leveldb.scaladsl.LeveldbReadJournal val queries = PersistenceQuery(system).readJournalFor[LeveldbReadJournal](LeveldbReadJournal.Identifier)
- Java
-
source
LeveldbReadJournal queries = PersistenceQuery.get(system) .getReadJournalFor(LeveldbReadJournal.class, LeveldbReadJournal.Identifier());
Supported Queries
EventsByPersistenceIdQuery and CurrentEventsByPersistenceIdQuery
eventsByPersistenceId
is used for retrieving events for a specific PersistentActor
identified by persistenceId
.
- Scala
-
source
val queries = PersistenceQuery(system).readJournalFor[LeveldbReadJournal](LeveldbReadJournal.Identifier) val src: Source[EventEnvelope, NotUsed] = queries.eventsByPersistenceId("some-persistence-id", 0L, Long.MaxValue) val events: Source[Any, NotUsed] = src.map(_.event)
- Java
-
source
LeveldbReadJournal queries = PersistenceQuery.get(system) .getReadJournalFor(LeveldbReadJournal.class, LeveldbReadJournal.Identifier()); Source<EventEnvelope, NotUsed> source = queries.eventsByPersistenceId("some-persistence-id", 0, Long.MAX_VALUE);
You can retrieve a subset of all events by specifying fromSequenceNr
and toSequenceNr
or use 0L
and Long.MaxValue
Long.MAX_VALUE
respectively to retrieve all events. Note that the corresponding sequence number of each event is provided in the EventEnvelope
, which makes it possible to resume the stream at a later point from a given sequence number.
The returned event stream is ordered by sequence number, i.e. the same order as the PersistentActor
persisted the events. The same prefix of stream elements (in same order) are returned for multiple executions of the query, except for when events have been deleted.
The stream is not completed when it reaches the end of the currently stored events, but it continues to push new events when new events are persisted. Corresponding query that is completed when it reaches the end of the currently stored events is provided by currentEventsByPersistenceId
.
The LevelDB write journal is notifying the query side as soon as events are persisted, but for efficiency reasons the query side retrieves the events in batches that sometimes can be delayed up to the configured refresh-interval
or given RefreshInterval
hint.
The stream is completed with failure if there is a failure in executing the query in the backend journal.
PersistenceIdsQuery and CurrentPersistenceIdsQuery
persistenceIds
is used for retrieving all persistenceIds
of all persistent actors.
- Scala
-
source
val queries = PersistenceQuery(system).readJournalFor[LeveldbReadJournal](LeveldbReadJournal.Identifier) val src: Source[String, NotUsed] = queries.persistenceIds()
- Java
-
source
LeveldbReadJournal queries = PersistenceQuery.get(system) .getReadJournalFor(LeveldbReadJournal.class, LeveldbReadJournal.Identifier()); Source<String, NotUsed> source = queries.persistenceIds();
The returned event stream is unordered and you can expect different order for multiple executions of the query.
The stream is not completed when it reaches the end of the currently used persistenceIds, but it continues to push new persistenceIds when new persistent actors are created. Corresponding query that is completed when it reaches the end of the currently used persistenceIds is provided by currentPersistenceIds
.
The LevelDB write journal is notifying the query side as soon as new persistenceIds
are created and there is no periodic polling or batching involved in this query.
The stream is completed with failure if there is a failure in executing the query in the backend journal.
EventsByTag and CurrentEventsByTag
eventsByTag
is used for retrieving events that were marked with a given tag, e.g. all domain events of an Aggregate Root type.
- Scala
-
source
val queries = PersistenceQuery(system).readJournalFor[LeveldbReadJournal](LeveldbReadJournal.Identifier) val src: Source[EventEnvelope, NotUsed] = queries.eventsByTag(tag = "green", offset = Sequence(0L))
- Java
-
source
LeveldbReadJournal queries = PersistenceQuery.get(system) .getReadJournalFor(LeveldbReadJournal.class, LeveldbReadJournal.Identifier()); Source<EventEnvelope, NotUsed> source = queries.eventsByTag("green", new Sequence(0L));
To tag events you create an Event Adapters that wraps the events in a org.apache.pekko.persistence.journal.Tagged
with the given tags
.
- Scala
-
source
import org.apache.pekko import pekko.persistence.journal.WriteEventAdapter import pekko.persistence.journal.Tagged class MyTaggingEventAdapter extends WriteEventAdapter { val colors = Set("green", "black", "blue") override def toJournal(event: Any): Any = event match { case s: String => val tags = colors.foldLeft(Set.empty[String]) { (acc, c) => if (s.contains(c)) acc + c else acc } if (tags.isEmpty) event else Tagged(event, tags) case _ => event } override def manifest(event: Any): String = "" }
- Java
-
source
static class MyTaggingEventAdapter implements WriteEventAdapter { @Override public Object toJournal(Object event) { if (event instanceof String) { String s = (String) event; Set<String> tags = new HashSet<String>(); if (s.contains("green")) tags.add("green"); if (s.contains("black")) tags.add("black"); if (s.contains("blue")) tags.add("blue"); if (tags.isEmpty()) return event; else return new Tagged(event, tags); } else { return event; } } @Override public String manifest(Object event) { return ""; } }
You can use NoOffset
to retrieve all events with a given tag or retrieve a subset of all events by specifying a Sequence
offset
. The offset
corresponds to an ordered sequence number for the specific tag. Note that the corresponding offset of each event is provided in the EventEnvelope
, which makes it possible to resume the stream at a later point from a given offset.
The offset
is exclusive, i.e. the event with the exact same sequence number will not be included in the returned stream. This means that you can use the offset that is returned in EventEnvelope
as the offset
parameter in a subsequent query.
In addition to the offset
the EventEnvelope
also provides persistenceId
and sequenceNr
for each event. The sequenceNr
is the sequence number for the persistent actor with the persistenceId
that persisted the event. The persistenceId
+ sequenceNr
is an unique identifier for the event.
The returned event stream is ordered by the offset (tag sequence number), which corresponds to the same order as the write journal stored the events. The same stream elements (in same order) are returned for multiple executions of the query. Deleted events are not deleted from the tagged event stream.
Events deleted using deleteMessages(toSequenceNr)
are not deleted from the “tagged stream”.
The stream is not completed when it reaches the end of the currently stored events, but it continues to push new events when new events are persisted. Corresponding query that is completed when it reaches the end of the currently stored events is provided by currentEventsByTag
.
The LevelDB write journal is notifying the query side as soon as tagged events are persisted, but for efficiency reasons the query side retrieves the events in batches that sometimes can be delayed up to the configured refresh-interval
or given RefreshInterval
hint.
The stream is completed with failure if there is a failure in executing the query in the backend journal.
Configuration
Configuration settings can be defined in the configuration section with the absolute path corresponding to the identifier, which is "pekko.persistence.query.journal.leveldb"
for the default LeveldbReadJournal.Identifier
.
It can be configured with the following properties:
source# Configuration for the LeveldbReadJournal
pekko.persistence.query.journal.leveldb {
# Implementation class of the LevelDB ReadJournalProvider
class = "org.apache.pekko.persistence.query.journal.leveldb.LeveldbReadJournalProvider"
# Absolute path to the write journal plugin configuration entry that this
# query journal will connect to. That must be a LeveldbJournal or SharedLeveldbJournal.
# If undefined (or "") it will connect to the default journal as specified by the
# pekko.persistence.journal.plugin property.
write-plugin = ""
# The LevelDB write journal is notifying the query side as soon as things
# are persisted, but for efficiency reasons the query side retrieves the events
# in batches that sometimes can be delayed up to the configured `refresh-interval`.
refresh-interval = 3s
# How many events to fetch in one query (replay) and keep buffered until they
# are delivered downstreams.
max-buffer-size = 100
}