Projections on projections

A projection can build on another projection’s read models, for example to derive a digest from them or to copy a value from them. This page covers how such a chain is declared, why a downstream read model is set and never added to, how a read model can reference another, which restrictions apply, and how to route read-model changes to an external system.

A listener whose first parameter is another projection’s read-model type applies that projection’s changes: whenever the upstream projection changes a read model, the listener receives the read model’s new state, together with the downstream read model stored under the same key.

@JpaProjection(name = "branches")
class BranchProjection {

    @ProjectionListener
    fun on(event: BranchRenamed, branch: Branch) {
        branch.name = event.name
    }
}

@JpaProjection(name = "branch-digests")
class BranchDigestProjection {

    @ProjectionListener
    fun on(branch: Branch, digest: BranchDigest) {
        digest.branchName = branch.name
    }
}

The downstream reads the state the change carries, never the upstream’s store, so the two may live in different stores. Each projection in a chain adds a hop before a change is visible; see Eventual consistency.

A change cascading through projections: MoneyDeposited in the capture topic is applied by AccountSummaryProjection, which writes the account summary’s new state to its read-model topic; AccountRiskProjection reads that state and sets the account risk from it, and a route sends the newest state to an external system; each projection in the chain adds a hop of about two seconds

Set, not add

Set the downstream read model from the upstream’s state; never add to it. A listener receives the upstream read model’s whole state after a change, not the change itself. A new or rebuilt downstream reads older states before the latest. A rebuild of the upstream sends its read models' states again, from the oldest, even to downstreams that already hold the newest. After a crash the same state can arrive again, but only for a projection that does not commit with its checkpoints; see Two delivery paths. When several changes to one read model are read together, only the newest state arrives.

An account summary’s balance passes through 100, 150, 200 and 260. Here is what the account risk holds:

What arrives	`risk.exposure += summary.balance`	`risk.exposure = summary.balance`
state 100	100	100
state 150	250	150
state 150 again, after a crash of a projection that does not commit with its checkpoints	400	150
states 200 and 260 read together: only 260 arrives	660	260

Wrong: adds the upstream’s state

@JpaProjection(name = "account-risks")
class AccountRiskProjection {

    @ProjectionListener
    fun on(summary: AccountSummary, risk: AccountRisk) {
        risk.exposure += summary.balance
    }
}

Right: sets it

@JpaProjection(name = "account-risks")
class AccountRiskProjection {

    @ProjectionListener
    fun on(summary: AccountSummary, risk: AccountRisk) {
        risk.exposure = summary.balance
    }
}

To combine several upstream read models into one, keep each upstream’s latest value in the downstream read model, keyed by the upstream’s id, and compute the total from those values. Setting one entry is safe to repeat; adding to a running total is not.

A read model that references another

Some read models copy a value from another read model: an account summary shows the name of its branch. The declaration lives on the downstream and reads backwards: @ReadModelKey(type = Branch::class, value = "accountIds") on AccountSummaryProjection says which summaries a change to a Branch reaches. They are the summaries whose keys the branch’s accountIds lists.

AccountOpened carries the account’s branchId, and AccountSummary gains branchId and branchName.

@Entity
class Branch(
    @EmbeddedId var branchId: BranchId = BranchId(),
    @Version var version: Long? = null,
    var name: String = "",
    @ElementCollection var accountIds: MutableSet<AccountId> = mutableSetOf(),
)

@JpaProjection(name = "branches")
@ReadModelKey(type = AccountOpened::class, value = "branchId")
class BranchProjection {

    @ProjectionListener
    fun on(event: BranchRenamed, branch: Branch) {
        branch.name = event.name
    }

    @ProjectionListener
    fun on(event: AccountOpened, branch: Branch) {
        branch.accountIds += event.accountId
    }
}

@JpaProjection(name = "account-summary")
@ReadModelKey(type = Branch::class, value = "accountIds")
class AccountSummaryProjection {

    @ProjectionListener
    fun on(event: AccountOpened, summary: AccountSummary) {
        summary.branchId = event.branchId
    }

    @ProjectionListener
    fun on(branch: Branch, summary: AccountSummary) {
        summary.branchName = branch.name
    }
}

The expression is evaluated on the upstream’s state, so the upstream must carry the downstream’s keys. The branchId the summary stores only records the reference; it routes nothing.

Whatever arrives first

No lookup is written by hand: the change names the read models it reaches. A summary gets its branch’s name whatever happened first:

The branch was renamed long before the account was opened: opening the account adds it to the branch’s accountIds, and that change of the branch carries its current name to the new summary.
The branch’s change arrives before the summary’s own AccountOpened: it creates the summary, and AccountOpened sets the rest. A deletion never creates a read model.
account-summary is new or rebuilt: it reads the branch changes from the start and ends on each branch’s latest name.

So no listener reads another projection’s store to fill a new read model. A read model that references several others, such as a branch and an adviser, has one such upstream for each, each listing the read models that reference it.

A summary that drops out of a branch’s list keeps the name it was last given until a branch lists it again; a rebuild of account-summary clears such copies (see When a rebuild is needed).

Restrictions

An @InMemoryProjection cannot be an upstream: its read models live in one instance’s memory, so startup fails when another projection listens to them.
A projection cannot listen to an event that is never captured and that no event-sourced aggregate stores: one without @DomainEvent, one excluded from capture, or any event while capture is off. Startup fails, naming the projection and the type; see Projections on captured events.
A projection may be stored in another datasource than its upstream. It reads the upstream’s changes from the upstream’s datasource and applies them in its own. Its delivery is that of its own store: at least once, and exactly once only when its store commits with its checkpoints; see Two delivery paths.
Projections that listen to each other’s read models under the same key, or one that listens to its own, fail startup, since each change would be applied again without end. So does a read model that is also a @DomainEvent. A chain is bounded by spring.ddd.cqrs.projection.cascade.max-depth (32 by default): a change that would pass it fails and names the cycle.

Routing read-model changes

A route on a read-model type sends a projection’s changes to an external system. The route is an ordinary consumer of the projection’s read-model topic, declared with the routing surface a domain event uses: @EventRoutes on messageType<AccountSummary>(), or a spring.ddd.routing.events entry keyed by the read model’s class name.

@Router("analytics")
class AnalyticsRoutes {

    @EventRoutes(destinations = ["analytics"])
    val summaries = messageType<AccountSummary>()
}

spring.ddd.routing.events:
  "[com.example.bank.AccountSummary]":
    destinations: [analytics]

Each message carries the read model’s current state, not a delta: the external system receives the account summary as it now is, not the deposit that changed it.

Changes to one read model that are read together are collapsed into the newest, so the external system may receive fewer messages than the projection applied changes. Outside a rebuild of the projection, it never receives an older state after a newer one. During a rebuild it receives the re-derived states again, from the oldest, so a balance can go back to an earlier value and then forward again. See Routing read models for deletions, route keys and metadata.