Caching Strategies That Do Not Lie to Your Users

By NTh Hai - 8 min read

Caching is the classic performance lever: keep hot data closer to the request. It is also a classic source of “works on my machine” production bugs — stale permissions, ghost inventory, and mysteriously resurrected deleted entities.

In solution architecture, a cache is not an optimization sprinkle. It is a derived data store with its own failure modes.

First Principle: Name the Source of Truth

Every cached value should have:

  • A primary that remains correct if the cache is wiped
  • A key strategy you can reason about
  • A freshness policy (TTL, explicit invalidation, or both)

If you cannot answer “what happens when Redis restarts empty?”, the design is incomplete — and often that empty restart is exactly what saves you from prolonged corruption.

Common Placement Patterns

Cache-aside (lazy loading)

App reads cache → on miss, reads DB → fills cache. Simple and widely used. Watch for stampedes when many requests miss the same key simultaneously.

Read-through / write-through

A caching layer mediates reads/writes. Write-through keeps cache warmer on writes but couples write latency to cache availability.

Write-behind

Writes hit cache first and flush to DB asynchronously. Higher throughput, higher risk. Use only with clear durability requirements and replay plans.

For most product apps, cache-aside in front of a relational primary is the boring winner.

Invalidation Is the Hard Part

TTL alone is easy and often wrong for user-visible correctness (profile updates, price changes, ACL changes). Prefer:

  • Invalidate on write for keys you know how to enumerate
  • Short TTL as a safety net, not the only mechanism
  • Versioned keys (user:42:v7) when you can bump a version cheaply

Avoid “clear the entire cache” as a routine fix — it turns deploys into stampedes against the database.

Stampede and Thundering Herd

When a popular key expires, thousands of requests may hit the DB. Mitigations:

  • Soft TTL / early refresh by one winner
  • Request coalescing (singleflight)
  • Probabilistic early expiration
  • Serving slightly stale while refresh runs (where product allows)

Architecture reviews should ask: what is our hottest key, and what happens at expiry?

What Not to Cache (Blindly)

  • Authorization decisions without careful invalidation
  • Mutable balances and stock unless you accept bounded inconsistency
  • Large rarely reused blobs that waste memory
  • Personalized pages that explode key cardinality

Cache hit rate is a vanity metric if the hits are for data nobody needed hot.

Observability Checklist

Instrument:

  • Hit/miss ratio by key prefix
  • Latency of cache vs. primary path
  • Eviction and OOM events
  • Error rates when cache is down (does the app fail open correctly?)

A cache that fails closed and takes down the site is worse than no cache. Decide fail-open vs. fail-closed per use case.

Closing Thought

Good caching architecture makes a deliberate trade: speed for controlled staleness. Document the source of truth, invalidation rules, and stampede plan. Then you get the performance win without teaching your users that the UI is a rumor.