Loading...
Loading...
Storing state as a sequence of events and separating read/write models
Kafka kept every payments page for 7 days so analytics could rewind the bookmark after a bugfix. Event sourcing asks what changes when the bookmark never expires: the log stops being a transport buffer and becomes the database itself.
Auditors ask. Fraud investigators ask. Your own support team asks, daily. And a normal database cannot answer, because it stores current state, which is the latest value per field, by overwriting the old value on every write. Every update erased the past, so last Tuesday is unanswerable without a separate audit table that someone remembered to maintain.
The naive fix is to add audit columns or history tables beside the real tables, which works until two writers disagree or a migration forgets the history path, and then the audit says one thing while the state says another. The heavier version dumps full state nightly for history: 40 gigabytes times 365 nights is 14.6 terabytes a year, and the dump still cannot answer what happened at 2pm between two midnights. Event sourcing fixes it by flipping the model: never store the current state at all, and instead store every change as an unerasable event such as deposited, withdrew, or refunded, computing now by replaying, which is re-applying events in order, them. Think of a bank ledger as the single analogy here: nobody stores your balance is 500 dollars and deletes how it got there, because the deposits and withdrawals are the record and the balance is just their sum so far.
A million events replay on every cold start unless you snapshot, which is periodically persisting the folded state plus the last event id so the next rebuild starts from the snapshot. Snapshots are a cache, never the truth, and they need versioning alongside the projector code.
A bad event format shipped in 2022 lives in the log forever, because rewriting history breaks every downstream projector offset and destroys the audit trail. You version forward with upcasters, which are functions that convert old formats at read time, and never edit the past.
Reach for event sourcing when the audit trail is the product, such as money, compliance, or collaborative editing. For a blog with comments it is a beautiful cannon aimed at a mosquito, because replay cost and versioning forever outweigh any audit benefit.
Your ledger holds 40 million events for a ten-year-old merchant account at roughly 1 kilobyte each, which is 40 gigabytes to read on every full rebuild, taking minutes to hours while users leave and pods run out of memory. Snapshots fix it: every 1,000 events or every hour, persist the folded state with the last event id, then replay only what came after. The rebuild becomes one snapshot load plus at most 1,000 events, which completes in milliseconds, while the snapshot write cost amortizes across all reads.
replay cost without snapshots: O(all events)
40M events x 1 KB = 40 GB read per rebuild — minutes to hours
with snapshot every 1,000 events:
load snapshot @ event 39,999,000 + replay 1,000 events
rebuild in milliseconds, snapshot write amortized
rules: snapshot is a cache, never the truth
version snapshots with the projector code version
corrupt projection? delete snapshot + projection, rebuild from log
keep snapshot interval adaptive: hot aggregates snapshot oftenOne log feeds many projections such as a search index, dashboard rollups, and CSV export, where CSV is a comma-separated-values file for spreadsheets, each replaying independently at its own pace. Writes never block on a slow read model being behind: the slow projector lags while checkout stays fast, and catches up without coordination.
Forty million 1-kilobyte events is 40 gigabytes per aggregate family before replication, multiplied by partitions and retention years. Archive cold segments to cheap object storage and keep hot replay windows on fast disks, or the audit dream becomes a storage pager at 3am.
Someone shipped an order-paid event with the total as a text string instead of integer cents, and the log kept every copy. You cannot update history, because rewriting the log breaks every downstream projector offset and destroys the audit trail auditors trusted. So you version forward: convert old events at read time with an upcaster, or append compensating events, which are new events that correct the past without erasing it, leaving the original intact for the record.
// v1 (2022, buggy): {"type":"OrderPaid","total":"49.00"}
// v2 (2023, fixed): {"type":"OrderPaid","v":2,"total_cents":4900}
// upcaster at read time:
// if (!e.v) e.total_cents = Math.round(parseFloat(e.total) * 100)
// compensating event (never edit v1):
// {"type":"OrderPaidCorrected","corrects":"evt-9917","total_cents":4900}You rebuild the email projection by replaying a year of order-paid events, and the projector calls send-receipt per event. Ten thousand customers get year-old receipts at 3am, because replay re-executed a side effect, which is any action on the outside world such as sending email or charging a card. Event-sourced systems must separate replay from side effects: projectors that touch the outside world need deduplication on the event id, or a strict rule that replays only touch read models and never call outside services.
Events remember everything, with snapshots keeping replay affordable, versioning keeping old mistakes readable, and idempotency keeping replays safe. But folding forty million events takes minutes, and the customer staring at the order-tracking screen needs this one change in 200 milliseconds, not after the next rebuild. Getting each change to every screen watching right now, without melting the servers, is push. Push has its own protocols.