Design Reviews Stall on Tools, Not Problems
The platform slows down under load. A release adds a new database, a queue and a cache. Costs rise. The next incident looks a lot like the last one. Teams debate products for weeks while the real constraint goes unnamed.
Why the Usual Approach Fails
Architecture discussions often start with technology choices. That gets the order backwards. When a team adopts a pattern before naming the pressure behind it, the system collects complexity it does not need, and the original bottleneck is still there. Start from the signal you can measure in production. Then pick the smallest fix that relieves it.
Use the cheat sheet below as a one-glance reference in design reviews. Each card shows the signal, a picture of the standard fix, and the cost that fix brings with it.
Too many reads
Scale reads
- Caching: keep hot data in memory so most reads never touch the database.
- Read replicas: copies of the primary database absorb the read traffic.
The cost: stale data. Plan how cache entries are invalidated, and decide which reads can tolerate replica lag.
Too many writes
Scale writes
- Batching: group many small writes into fewer round trips.
- Sharding: split data across databases by a key such as customer or region.
The cost: the shard key is hard to change later, and cross-shard queries and rebalancing get expensive. Choose the key deliberately.
Needs live updates
Real-time
- WebSockets: one open, two-way connection for chat, collaboration and live dashboards.
- Server-sent events: a simpler one-way stream when only the server needs to push.
The cost: long-lived connections need sticky routing, heartbeats and reconnect logic. Use plain polling when seconds of delay are fine.
Slow work
Background jobs
- Message queue: accept the request, park the work, respond immediately.
- Worker pool: workers pull jobs in parallel and scale independently of the API.
The cost: results arrive later, not now. You need job status, dead-letter queues and alerting on queue depth.
Things will fail
Reliability
- Retries with backoff: wait longer between attempts, with a little random jitter.
- Idempotency: running the same request twice has the effect of running it once.
- Circuit breaker: stop calling a failing dependency, fail fast, recover gradually.
The cost: retries without idempotency duplicate work, and retries without jitter turn a blip into a storm. Ship all three together.
Reads and writes compete
Split them (CQRS)
- Commands go to a write model tuned for correctness.
- Queries hit a separate read model shaped for fast answers.
The cost: two models to keep in sync, and reads that lag writes. Worth it only when read and write needs genuinely diverge.
Ask the Six Questions in Order
Before a design review reaches for tools, walk these questions top to bottom. Each one maps to a card above, and the signal next to it is what you will usually see in production first.
- Is it read-heavy?
Database load climbs on repeated lookups of the same records.
- Is it write-heavy?
Insert latency rises, locks pile up, and one database sits at its ceiling.
- Does anyone need to see changes live?
Users refresh or poll to find out whether something happened.
- Is any of the work slow?
Requests time out waiting on reports, imports, emails or AI calls.
- What happens when a dependency fails?
One slow downstream service drags every caller down with it.
- Do reads and writes need different shapes?
Reporting queries fight transactional writes for the same tables.
Deep Dive · Problem 1
Tiered Caching That Cuts Load and Cost
The database is the most expensive place to answer a question. Yet most systems ask it the same few questions all day. Product pages, prices, permissions, settings. Each repeat read adds latency, burns capacity, and pushes the next scaling bill closer.
A single cache helps, until it becomes the new bottleneck or the new single point of failure. Tiered caching answers each request at the closest layer that can safely answer it. Every tier absorbs load so the tier behind it never sees it.
The Five Tiers, Nearest First
How a read flows. Check the nearest tier first. On a miss, fall through to the next. When the answer comes back from the database, write it into each tier on the way out.
Illustrative share of requests that reach each tier, with healthy hit rates:
Why Tiers Beat One Big Cache
- Each tier protects the next. The database is sized for writes and true misses, not for peak read traffic.
- Answers come from the closest safe place. Shared content stops at the edge. Per-user data stops inside the app.
- Failure degrades instead of breaking. Lose L1 and L2 still answers. Lose L2 and the database carries the load for a while.
- Cost follows value. Scarce, fast memory holds only the hottest keys. Cheaper tiers hold the long tail.
Choose a Strategy for Each Kind of Data
| Strategy | How it works | Use it when | Watch for |
|---|---|---|---|
| Cache-aside | The app checks the cache, loads from the database on a miss, then fills the cache. | Read-heavy data. The sensible default. | Stale data until expiry or invalidation. |
| Read-through | The cache layer loads from the database itself on a miss. | You want simpler app code and a library supports it. | The cache must know about the data source. |
| Write-through | Every write updates the cache and the database together. | Data is read right after it is written. | Slower writes. Caches data nobody reads. |
| Write-behind | Writes land in the cache and reach the database later, in batches. | Absorbing short write bursts. | Lost writes if the cache fails. Use with care. |
Keep It Correct: Invalidation and Stampedes
Caching is easy. Knowing when cached data is wrong is the hard part. Decide this per data type, not per system.
- Expire with jitter. Give every entry a lifetime, plus a small random offset so thousands of keys never expire at once.
- Invalidate on write. When a record changes, delete its cache key so the next read reloads fresh data.
- Version your keys. Put a version in the key name to retire a whole set of entries in one step.
- Serve stale while refreshing. Return the slightly old value and refresh it in the background, so users never wait on a reload.
- Stop the stampede. When a hot key expires, let one request rebuild it while the others wait for that result.
Before You Add a Tier, Answer These
- What is the hit ratio today, and what target justifies the tier?
Measure first. A tier that rarely hits only adds a network hop.
- How stale can this data be, and for how long?
Write the staleness budget down for each data type.
- Who invalidates it when the data changes?
Name the owner and the trigger, or rely on short lifetimes.
- What happens when the cache is cold or down?
Confirm the database survives the fall-through load, and plan a warm-up.
- Is any of this data personal or sensitive?
Never store per-user data in a shared tier such as a CDN.
- What does it cost to run?
Count the memory, the monitoring, and the on-call time, not just the license.
What Good Looks Like
Teams that use this discipline reach for one or two patterns, not all six. Each design decision names the signal that justified it and the trade-off that was accepted. The results show up in numbers executives already track:
- Faster delivery: fewer weeks lost to tool debates, because reviews start from a named problem.
- Reduced operating costs: no caches, queues or extra databases running without a measured reason.
- Reduced technical debt: every pattern carries a written rationale the next team can trust or retire.
- Fewer repeat incidents: retries, idempotency and circuit breakers ship together, so one failing service stops taking others down with it.
Our enterprise architecture practice runs design reviews this way: problem first, smallest fix second, trade-offs on the record.
