Design a distributed in-memory cache system like Redis or Memcached. The system stores key-value pairs across a cluster of cache nodes, handling 1 million requests per second with sub-millisecond latency. It supports TTL-based expiration, LRU eviction when memory is full, and must remain available when individual nodes fail. The cluster has 20 nodes, each with 64 GB of RAM, for a total cache capacity of ~1 TB.
In your 90-second answer, cover: - How keys are distributed across cache nodes (sharding strategy) - What happens when a node is added or removed from the cluster - Eviction and expiration: how memory pressure and TTL are handled - High availability: what happens when a cache node crashes
Constraints: adding or removing a node must not invalidate more than 1/N of the cache (where N is the number of nodes). Cache misses should not cause a thundering herd on the backing database.
How to approach it
- Hint 1
Consistent hashing distributes keys across nodes while minimizing redistribution when the cluster changes. Place nodes on a hash ring, and each key maps to the next node clockwise. Adding a node only affects keys in the arc it takes over -- roughly 1/N of the total.
- Hint 2
Distinguish between eviction (removing items because memory is full) and expiration (removing items because their TTL elapsed). LRU eviction uses a doubly-linked list: every access moves the key to the head, and when memory is full, evict from the tail. For TTL, use a combination of lazy deletion (check on access) and active sweeping.
- Hint 3
When a cache node crashes, its keys become misses. All clients simultaneously hit the database for those keys -- a thundering herd. Protect against this with request coalescing: if 100 clients request the same key simultaneously, only 1 request goes to the database, and the other 99 wait for and share the result.
Ready to answer it out loud?
Record your answer in 1:30 and Preptile scores it 1–10 with specifics — what landed, what you skipped, and what to say next time.
Practising needs an invite code. Join the waitlist and we’ll send you one.