Loading...
Loading...
Store frequently accessed data for faster retrieval
Simulates a Read-Through cache strategy. Cache Hits serve data directly from memory in ~4ms, bypassing the slower disk database (~115ms).
You already know from the latency module how a millisecond of memory compares with a hundred-millisecond disk seek, and from the database module how one hot row can queue thousands of identical queries behind it. Your homepage answers the same question a thousand times a minute and pays full database price every time. A cache, which is just fast memory that keeps copies of recent answers, is institutional memory: remember the answer, skip the work. Memory answers in about a millisecond while disk takes a hundred times longer.
Think of sticky notes on the fridge, the one comparison we will use here: the first person who cooks the recipe writes down the result, and everyone after just reads the note until it goes stale. In the lab above, run traffic with the cache off and watch every request pay the slow path, then turn it on and watch the same traffic collapse to mostly instant answers, because the note absorbs the repeats.
The naive approach queries the database on every load. We rejected staying with it because it is simple and always correct yet melts at rush hour when the database does the same expensive join a thousand times. Caching pays once and reuses, which is why the two columns below diverge so fast.
Do the napkin math with the sim above: at 95% remembered and 1ms versus 100ms, average answer time is about 20 times faster:(0.95 x 1ms + 0.05 x 100ms = 5.95ms versus 100ms)
Drag the hit-rate slider above from 50% to 99% and watch the average fall steeply at first then flatten, because each extra percent saves a full 99ms trip while costing more memory and more stale-read risk.
A request falls through layers until something remembers: the visitor's own browser (memory on their device, about 0ms), a nearby edge city (a cache server close to users, about 20ms), shared memory every server reaches (a service like Redis, which is a fast in-memory store you run separately, at 1 to 5ms), the database's own buffers (10 to 50ms), and finally disk (100ms or more). Each layer catches what the faster one missed.
Every shelf is finite, so something must go when it is full. Eviction means choosing which note to throw away. The three answers that survived decades of practice are LRU (least recently used, meaning toss whatever sat untouched longest), LFU (least frequently used, meaning toss whatever got asked for fewest times), and TTL (time to live, meaning every note carries a self-destruct timer and simply expires).
Yesterday's viral post leaves while today's obsession stays. Try it above: hammer one key and watch the others fall, because every touch refreshes that key's recency.
Count every touch and evict the lowest score, so loyal regulars survive one-hit wonders. The edge case is a formerly hot key with a huge old count that now blocks fresh keys, which is why real LFU ages counts down over time.
Every note carries a self-destruct timer. No decisions, no bookkeeping, stale data simply ages out. Set it to 300 seconds and the note is gone at second 301 and refetched on next ask.
Reads love memory but writes force a choice, because the note and the truth can disagree. The four standard reconciliations differ only in when they pay the cost of agreement.
Remember answers as questions arrive and rip the note up on writes. Simple and resilient, occasionally stale between the write and the rip. Best when reads outnumber writes a hundred to one.
Update memory and disk together on every write. Never stale and never fast on writes, because the slowest store sets the pace. Use it when a stale read costs real money.
Answer from memory instantly and flush to disk in batches. Blazing until a crash eats the unsynced batch, so size the batch by how much loss you can stomach.
Delete the note on every write and let the next read refetch fresh. Blunt, honest, always correct, and punishing when one hot key gets written constantly.
One famous note expires and a thousand readers stampede the database together. A stampede means correlated misses that arrive faster than the database can refill the note. Fix it by letting one reader fetch while the rest wait on that single fetch, a trick called request coalescing.
In the lab, expire the hot key under load and watch misses spike, then enable coalescing and watch the spike collapse to one.
The database moved on but the note did not, so users decide on lies. Stale means the cached copy no longer matches the truth. Fix it by burning notes on write, or keep TTLs brutally short on fast-changing data like prices and balances.
Requests for ids that do not exist always miss and always punch through to the database, which attackers exploit. Fix it by remembering nothing-here too, briefly: a negative cache entry with a short TTL stops the punch-through without freezing real creates for long.
Redis, an in-memory store you run beside your servers for sessions and query answers, and Memcached, a simpler sibling often used for fragments and sessions, both answer in about a millisecond by design. Timelines live in the first, sessions in either.
Video chunks and images wait in cities near viewers. Distance stops mattering when the note is local, which is why the same video loads instantly in two distant cities after the first viewer in each city paid the slow fetch.
Remembering is easy. Forgetting correctly, what to evict, when to invalidate, and how to survive a stampede, is where caching earns its reputation. Once you can feel those eviction trade-offs above, which of your own hottest reads could tolerate a stale copy for sixty seconds, and which ones never could?
Try this in the playground
Open a template and build it yourself — then take a quiz.