One stream, two consumers, opposite fears
A courier marketplace publishes one event stream of every trip state change, aggregated from all regions. Two systems read it. The pricing view sets surge across the city and must never stop — a gap in it is a map that lies. Courier earnings pays people and must never double-pay and never skip.
The team has been arguing about the consumer configuration for a week. Both sides are right, which is the problem.
- R1The pricing view must keep producing through the loss of an entire region, with no gap in its output. Processing the same event in more than one place is acceptable here and always has been — the view is a calculation over the stream, and calculating it twice gives the same number.
- R2Courier earnings must not pay twice and must not skip. Paying is not a calculation: doing it in two places at once is two payments, and there is no version of that which the second run corrects.
- R3Cross-region replication does not preserve arrival order between clusters, so the position earnings had reached in one region names a different set of processed events in another. The record of where it had got to must therefore be a mapping between the two, it must be durable, and it must still be writable from whichever region survives — a failover reads it at exactly the moment the region it describes is gone.
Boundaries, outermost first: REGION EAST: Payout ledger (what was actually paid), Aggregate stream (every region's events), an empty slot for the surge, and it may not stop, tier 1, an empty slot for the pays people, tier 1, an empty slot for the where earnings had got to, tier 2 REGION WEST: Regional stream (one per region), Trip services (write to the nearest), Replicator (sees both positions) Connections: Trip services calls Regional stream — never cross-region (step 1) Regional stream publishes to Replicator (step 2) Replicator publishes to Aggregate stream — order not preserved (step 3) Aggregate stream publishes to surge, and it may not stop (step 4) Aggregate stream publishes to pays people (step 5) Replicator controls where earnings had got to — both positions (step 6) where earnings had got to controls pays people — resume from here (step 7) pays people calls Payout ledger (step 8)