The shape
A single logical key is receiving more of something than one machine can handle. That is the whole pattern, and the reason it deserves its own page is that every mechanism for spreading load across machines is silent about it.
Hashing, consistent hashing, virtual nodes, more shards, bigger clusters: all of these divide the key space. A hot key is one key. It has one hash value, so it lands on one machine, and rehashing moves the problem to a different machine rather than solving it. Adding machines divides the key space. It never divides a key.
Uniform traffic is a fiction. In any real system, a handful of keys carry far more size or traffic than the rest, and not as a one-off spike, it's the normal shape of the traffic. That imbalance is already sitting in your system right now, whether or not you've measured it.
Three different problems with one name
The most useful thing on this page is that "hot key" means three unrelated things, and confusing them is why the wrong fix gets applied.
Too much data. One tenant is 1.6 TB when the mean is 200 MB. The partition exceeds its size ceiling and no shard count helps. The fix is a second key component: shard that one oversized tenant, often called a whale, on (tenant, project) while everyone else stays on tenant, and accept two routing schemes in one system.
Too much traffic. One product, one video, one celebrity account absorbing 150,000 requests a second. The partition holding it hits its throughput ceiling while the cluster sits idle. The fixes, coalescing, replicating the value, and demotion, are explained in The moves section below.
Too many edges. One account with 50 million followers. Nothing about the row is large and nothing about its request rate is unusual; the problem is the size of the fan-out it triggers. Placement cannot fix this one at all. Only a different data model can, usually a hybrid where the ordinary case is precomputed and the extreme case is computed on read.
The moves
Four mechanisms, and they compose. Reach for them in this order, because the order is cheapest-first.
Coalesce. If 1,000 concurrent requests want the same key, one of them should fetch it and the rest should wait on that fetch. This is request coalescing, sometimes called single-flight or thundering-herd suppression, and it is usually a dozen lines in front of a cache. It turns those 1,000 concurrent cache misses into a single fetch to the origin server, with everyone else just waiting on that one fetch to finish. For a genuinely hot key, that's the difference between 1,000 simultaneous requests to the origin server, a stampede, and 1, small enough to ignore. This is the cheapest possible fix, since it doesn't involve any data model changes, or any consistency cost for immutable values.
Replicate the value. Store k full copies of the hot key, for example hotkey:0 through hotkey:k-1, each holding the same value. A read picks one of the k copies at random, so each copy only sees 1/k of the read traffic. A write has to update all k copies, which costs k times the writes, and if a reader hits a copy before its update lands, it gets a stale value. That's fine for read-heavy, immutable data, write it once, it never changes again, and bad for a mutable counter, where a stale copy just means a wrong number.
Split the key. Turn one hot key into k physical keys by appending a small salt to the key name, for example hotkey:0 through hotkey:k-1, and send each write to just one of them. This is the only move that helps a hot write key: instead of one key absorbing every write, each of the k keys only takes about 1/k of them. For a sharded counter specifically, a read has to fetch all k keys and sum them to get the true total. The read cost is now k lookups per read, so k should be the smallest number that clears the write ceiling, not the largest number you can administer.
Demote. Serve the hot key from a cheaper, staler source instead: a local in-process cache refreshed every few seconds, a CDN edge, or a precomputed snapshot recomputed on a schedule. This is the one that requires a product decision. You are choosing to answer with data that might be a few seconds or minutes old, and often the cheapest of the four once you've made it.
Step 1: how many requests arrive while the cache is empty? one key, 150,000 requests/s, refill takes 50 ms after expiry 150,000 req/s x 0.05 s refill window = 7,500 requests Step 2: without coalescing, what happens to those 7,500? every one of them sees a cache miss and independently fetches the same bytes from the origin server origin server fetches = 7,500 Step 3: with coalescing, what happens instead? only the first request fetches from the origin server the other 7,499 just wait on that one fetch to finish origin server fetches = 1, done in the same 50 ms Step 4: what did that buy you? 7,500 origin server fetches -> 1, a 7,500x reduction in load and it costs nothing: no schema change, no new placement, no new consistency guarantee Splitting the key would also have worked, but it costs k lookups on every read, forever. Coalescing costs nothing after this one fix. Do the free one first.
How to tell you have this problem
Do not wait for an incident. Two queries answer it:
- Sort your keys by how much data each one holds, in bytes, and look at the top 0.1 percent. This is what finds the too-much-data case, the whale tenant sitting at 1.6 TB when the system's mean is 200 MB.
- Separately, sort your keys by request rate, and look at the top 0.1 percent again. This is what finds the too-much-traffic case, the celebrity key absorbing 150,000 requests a second.
These are two different lists, and they point to two different fixes. If you have only ever looked at averages, you have no information about either. A distribution with a mean of 200 MB and a maximum of 1.6 TB has a perfectly reasonable-looking mean.
The operational tells, roughly in the order they show up:
- A per-key latency metric with a much worse tail than the per-table one.
- Throttling on one partition while the system's overall utilisation stays low.
- One replica or partition consistently hotter than its peers, with no configuration difference to explain it.
- Cache hit rate that collapses briefly and periodically, rather than declining gradually.
What it costs
Every move here buys relief with complexity, and it's worth being explicit about that, because the cheap-looking options aren't always the cheap ones.
Coalesce. It needs a shared data structure tracking which fetches are in progress. A new request finds one already underway and waits on it instead of starting its own. That structure needs careful handling when things fail: if the one in-progress fetch fails, everyone waiting on it has to be told, or they'll wait forever.
Replicate the value. Every write has to land on all k copies before they agree with each other. Until it does, whichever copy a reader happens to pick can hand back a stale value.
Split the key. This permanently turns one read into k reads, for every single key using that scheme, including the 99.9 percent that were never hot to begin with and never needed the extra lookups.
Demote. This is the cheapest of the four to build and the most expensive to agree to. Someone on the team has to sign off on serving an answer that might already be out of date.
Where it shows up in the course
| article | the form it takes there |
|---|---|
| Back-of-Envelope Estimation | why an average tells you nothing, and total capacity does not help a single key |
| Caching Strategies and Invalidation | the stale-set race and the thundering herd, which are both one key being asked for by everybody at once |
| CDNs and Edge Compute | a single popular object at one PoP, where the edge absorbs the read hotspot the origin server could not |
| Consistent Hashing | virtual nodes fix uneven arc lengths and cannot touch one popular key |
| Distributed Rate Limiter | a single principal generating enough traffic to saturate the shard holding its counter |
| a single post's like counter taking six figures a second of increments against one row | |
| Message Queues vs Event Streams | the one case where hashing cannot help at all, because ordering per key is what forbids spreading the key |
| News Feed and Timeline | the celebrity post, where the hot key on the read side is what rescues you from the hot write |
| Partitioning and Sharding | the whale tenant, and the fact that shard count cannot divide a key |
| URL Shortener | one link going viral, which the code scheme does nothing about, fixed by coalescing plus edge replication |
| WhatsApp and Chat Systems | a large group where every message multiplies into one write per member |
| YouTube | a trending video's view counter, taking six figures a second against one key |
Getting it wrong
Reaching for a bigger cluster. Total capacity is not the constraint. The constraint is one partition's ceiling, and it does not move when you add machines.
Splitting when coalescing would have done. Splitting is permanent and costs every read. Try the free move first.
Choosing k too large on a split. For a hot write key, k should be the smallest number that clears the write ceiling. Every increment costs a lookup on every read.
Fixing the traffic problem and ignoring the size problem. They are unrelated. A key can be hot and small, or huge and idle, and the same key can be both, in which case you need both fixes.
Letting a hot key take a node down. In any hash-routed tier, a failure relocates that key's load onto a neighbour, which makes the next failure more likely. A node at its ceiling should shed load or spill the excess to a fallback, not die, or one hot key becomes a cascade.