WhatsApp and Chat Systems

When people are asked to design WhatsApp, most of them start with a database: a messages table, sharded by conversation, indexed by timestamp, with some thought about paging back through history. That instinct is reasonable, but it points at the wrong problem.

WhatsApp published its numbers when it had 465 million users, and one line in that write-up changes the whole question: "messages and multimedia are only stored while they are being delivered." The server isn't an archive. It's a relay with a spool attached, a small queue that holds a message until the recipient's phone has it. After that, the server forgets the message.

That one fact removes most of what candidates spend their 45 minutes on. If the server doesn't keep messages, there's no petabyte data store to shard, no history query to optimize, and no retention policy to design. A chat server's job is to hold a connection, not a message. The real design question isn't "where do I put 50 billion messages a day." It's "given a user id, which of my hundred servers has their socket open right now, and what do I do when none of them does."

Everything below follows from that. Storage shrinks to a spool for messages that haven't been delivered yet. Routing, which means finding the server that holds a user's connection, becomes the main system. Most of the deep dives cover what happens when a connection isn't where you expected it to be.

Understanding the problem

Two people install an app and start messaging each other. Some of those messages go to groups instead. Everyone expects a tick when a message is delivered, another tick when it's read, and for all of it to still work after they spent an hour in a tunnel with no signal.

Before you design anything, split "chat" into the four jobs people usually mean by that one word. Each of them needs a very different system.

1. Real-time relay. Getting a message from one connected device to another, fast. This is the core of the problem, and it's a routing problem.
2. Store and forward. Holding a message for a recipient who isn't connected, and delivering it once they reconnect. A small queue, one per user.
3. Message history. A searchable archive of everything anyone has ever said. WhatsApp doesn't build this. For Slack, it's the whole product. The two are different systems, so don't mix them up.
4. Presence and typing. Who's online, and who's typing. The volume is huge, and each single event is worth almost nothing.

This design covers the first two, plus groups. Message history is a different product, and it stays out.

Functional requirements

Core:

1. Send a message to one other user, and have it arrive on their device quickly when they're connected.
2. Deliver to a user who was offline when the message was sent, once they reconnect, with nothing lost.
3. Send to a group, with every member receiving it.
4. Report delivery state back to the sender: sent, delivered, read.

Out of scope, deliberately:

• Server-side message history and search, for the reasons above.
• Voice and video calls. They use a different protocol and a different transport, and they're mostly a peer-to-peer negotiation problem.
• Media upload and download. It's an object store such as S3 with a signed URL in front of it. Media never travels through the chat path at all.
• Contact discovery, spam and account bans.

Non-functional requirements

1. 100 million live sockets at peak. This number sizes the system, not the message rate.
2. Delivery in under 500 ms at p99 when both parties are connected in the same region.
3. Scale: 2 billion registered users, 500 million daily active, 50 billion messages sent per day, groups up to 1,024 members.
4. Nothing is lost once accepted. If the server acknowledged a message to the sender, it must eventually reach the recipient's device, or the sender has to be told it didn't.
5. Messages within one conversation arrive in the order they were sent. Across conversations, order doesn't matter, because nobody can see it.

Requirement 4 carries the design, so read its wording carefully. It doesn't say a message is delivered exactly once, or quickly, or that the server keeps it. It says the server can't silently drop something it accepted. That promise is much cheaper than an archive, and it's what lets storage be a spool instead of a database.

The set up

Core entities

• User. An account with a stable id, and one or more devices.
• Device. Where a connection ends. If you address messages to the user, multi-device support gets painful later, so make the device the address from the start.
• Connection. A live socket, held by exactly one chat server, with a record of which server holds it. This is the only hot object in the system, the one that changes all the time, and most of the design is about keeping that record correct.
• Conversation. One-to-one or a group. It owns the member list and the sequence numbers for that thread.
• Message. Sender, conversation, a client-generated id, a sequence number, a payload the server doesn't read, and a delivery state per recipient device.

How a message reaches a phone at all

Choose the transport before you design the interface. Interviewers ask about it first, and the right answer only becomes clear once you work out what each option costs.

Most people start with polling, where the client asks. The phone calls GET /messages?since=4411 on a timer, exactly like every other API you've built. Polling needs no new infrastructure and works through every proxy and firewall. Each request is stateless, so any server can answer it.

Then put numbers on it.

Peak is 3x average, assumed. Fanout is 2.1x, from WhatsApp's
real numbers: 19B messages a day coming in and 40B going out.
The 50B a day and the 100M live sockets are both requirements.

  messages a day  50B / 86,400                 ≈ 580,000/s average
  peak hour       × 3                          ≈ 1.7M/s sent
  fanout          × 2.1                        ≈ 3.6M/s delivered
  per socket      100M sockets / 3.6M msg/s    ≈ 28 s between messages

Poll every 3 seconds
  requests        100M sockets / 3 s           = 33M req/s
  useful ones     28s gap / 3s poll            = 1 in 9, 89% empty
  added delay     up to 3s, averaging 1.5s     → blows a 500 ms budget

Poll every 0.5 seconds, which is what 500 ms p99 needs
  requests        100M sockets / 0.5 s         = 200M req/s
  useful ones     28s gap / 0.5s poll          = 1 in 56, 98% empty

  headers, cookies and TLS records, ~1 KB per exchange
  200M × 1 KB                                  = 200 GB/s
                                               ≈ 1,600 Gbps

WhatsApp's measured peak outbound, all traffic including
photos and video                               = 146 Gbps
─────────────────────────────────────────────────────────
The polling overhead alone is eleven times their entire
peak bandwidth, and it carries no messages.

Polling has two problems, and the second is worse. The volume is bad, but more hardware fixes volume. Nothing fixes the latency floor, which is the delay polling adds however fast your servers are. If the phone polls every three seconds, a message can wait three seconds before anyone asks for it. To hit 500 ms, the phone has to poll twice a second, and at that rate 98 percent of requests are a phone asking whether anything happened and hearing no.

Walk up the ladder of options anyway. Each step fixes exactly one problem.

Long polling. The client asks, and the server holds the request open until a message arrives or the request times out. The latency floor disappears, because the server answers the moment it has something. But the connection stays open the whole time, so you pay for a persistent connection while still treating each request as stateless. Every delivered message also costs a full request and response, headers and all, and the client opens a new request right after. You get the latency back, and you keep the overhead.

Server-sent events (SSE). The client opens one connection and holds it open, and the server pushes as many messages down it as it likes. That fixes the overhead: one set of headers for the life of the connection, then a small frame per message. SSE is a good fit for a news feed or a notification stream.

Here SSE falls short, because it only goes one way. Sending a message still needs a separate HTTP request. So you're running two transports, and every send is back to a fresh connection and a kilobyte of headers. Every client in a chat both reads and writes, so a one-way transport solves only half the problem.

WebSocket. One HTTP request upgrades the connection, and after that both sides push frames whenever they want. Framing runs 2 to 14 bytes instead of a kilobyte of headers, and there's no request-response cycle at all: the server writes to the socket when it has something, and the client writes when the user types.

Take 6 bytes of framing as a working figure. At 3.6 million deliveries a second, that's 21.6 MB/s, or 0.17 Gbps. Polling at the same latency target needs 200 million requests a second at 1 KB each, which is 200 GB/s, or 1,600 Gbps. The two are about 10,000 times apart. With a gap that wide, the inputs don't need to be precise.

System interface

The interface follows from the transport we just picked. A device opens one long-lived socket, and both sides push frames over it.

CLIENT → SERVER
{ "op": "send",
  "conversation_id": "c_88f2",
  "client_msg_id": "6f0a-...-91c4",     // idempotency key, from the sender
  "seq": 4412,                           // sender's per-conversation counter
  "payload": "<ciphertext>" }

SERVER → CLIENT
{ "op": "ack",   "client_msg_id": "6f0a-...-91c4", "server_ts": 1754640000123 }
{ "op": "deliver", "conversation_id": "c_88f2", "message_id": "m_71c",
  "from": "u_204", "payload": "<ciphertext>" }
{ "op": "receipt", "conversation_id": "c_88f2", "read_through": 4412, "device": "d_9" }

Five things to note.

One connection, many operations. A device doesn't open a socket per conversation. It opens one and multiplexes everything over it, which means every conversation's frames share that one socket. The connection is the expensive resource, and a conversation costs nothing.

The client generates the message id. The server can't tell a retry from a new message, but the sender can. That id makes the send path safe to retry, and it lets the recipient drop a duplicate.

The ack means accepted, not delivered. These are two different events, so keep them apart. The sender's first tick is an ack from the server. The second tick is a receipt that comes back from the recipient's device, sometimes hours later.

The receipt carries a position, not a message id. read_through says how far a device has read, so one frame covers forty unread messages instead of forty separate frames. And it only travels to one place, the sender, not to everyone in the conversation. Section 5 covers both.

The server doesn't read the payload. Under end-to-end encryption, it can't. If you design as though it can, you'll reach for server-side search, spam filtering and history, and none of those survive that one rule. Treat the payload as opaque from the first minute.

High-level design

1. Which server holds the recipient's socket?

Every message needs an answer to one question: which server holds the recipient's socket?

For the rest of this walkthrough, assume the registry lookup. The hash is a fast path on top of the registry, not a different design, and the hybrid falls back to the registry anyway. So everything below works the same either way.

2. Deliver to someone who is connected

Both devices are connected, and this path should be simple.

The sender's server receives the frame and assigns a server timestamp and a message id. It doesn't deliver the message itself. It routes it to the conversation's owner process.

Every conversation has exactly one owner process: a process running on one of the chat servers, chosen by hashing the conversation id. It isn't a new kind of machine, just one more job for a chat server you already run. To keep it close to the people talking, the hash only picks among the chat servers in the region nearest the conversation's members.

The owner process is usually not on the sender's server. The sender's server is wherever the sender's socket happened to land, and in a group, members send from dozens of different servers at once. If each of those servers numbered messages itself, two of them could give out the same sequence number, and members would see the conversation in different orders. So every message in a conversation passes through its owner process, whoever sent it, and the owner process assigns the sequence number. It picks up more jobs later: it holds the member list, runs group fanout, and collects read receipts.

The owner process looks up the recipient's device in the registry and forwards the message to the chat server that holds it. That server writes it to the socket. The recipient's device acknowledges, and a receipt travels back the same way.

One decision in that path needs a closer look.

Acknowledge the sender before the recipient sees the message. The sender's ack means the server has taken responsibility: the message is now durable enough to satisfy requirement 4. It doesn't mean the recipient has it. If you wait for the recipient before acknowledging, every send becomes as slow as the slowest phone on the other end.

That raises the question requirement 4 depends on: how durable does a message have to be before you may acknowledge it? There are three options, cheapest first.

• Memory on one server. Acknowledge as soon as the message is in the sending server's memory.
• Two machines. Acknowledge once a second copy exists in memory on another server.
• The recipient's spool. Acknowledge only after a durable write to the spool.

Memory on one server promises something you can't keep. That server can die a millisecond later. Nothing else in the system has the message, and the sender is left holding a tick for a message that no longer exists.

The recipient's spool is the safe option. It charges every send a durable write, even when both parties are online and the entry will be gone a moment later. You're paying for durability you're about to throw away.

Two machines is the middle option, and the right one. The server copies the message to the recipient's server, or to a second copy of the owner process, and acknowledges once that copy exists.

So the online path replicates instead of writing to disk. A second copy in memory on another server costs a millisecond, and a disk write costs ten. The risk you accept is that if both servers die at the same instant, the message is lost.

What carries the message between two servers?

Everything above assumes a message can get from the sender's server to the recipient's server, but nothing so far says how. Section 1 ended with a server_id, and you can't send bytes to an id. So what carries the message across, and what does that cost 3.6 million times a second?

What each answer costs at 3.6M deliveries a second, on the
fleet of 100 chat servers sized in deep dive 1.

Redis pub/sub
  shards        3.6M / 100,000 per shard   = 36 shards minimum
                with headroom              ≈ 70 shards
  bytes         3.6M/s × ~1 KB             ≈ 3.6 GB/s
                × 8 for bits               ≈ 29 Gbps
  channels      one per connected device   = 100M subscriptions
  hops          sender → Redis → recipient = 2

Direct connections
  links         100 servers, every pair    = 4,950 links
  per server    100 - 1                    = 99 outbound
                against 1M client sockets  = 0.01% more sockets
  hops          sender → recipient         = 1
  ─────────────────────────────────────────────────────────
Below 100,000 deliveries a second, pub/sub is the answer: one
shard covers everything and a registry buys you nothing. Far
above it, where we are, pub/sub turns into a fleet you now
have to manage, though the direct links still cost only 99
connections per server.

3. Deliver to someone who is not connected

Half the world is asleep at any given moment, so this isn't an edge case. It's half the traffic.

When the registry has no entry for a device, the message goes into that device's spool, a small per-device queue of undelivered messages. When the device reconnects, its server drains the spool. It pushes everything down the socket in order of deliv_seq, a counter that numbers every message the device is owed, and moves the device's cursor past whatever the device acknowledges.

50 billion messages a day, and assume the pessimistic
case that half arrive while the recipient is disconnected.

  undelivered/day     25B
  average life in the spool, before the device reconnects
                      call it 4 hours
  in flight at once   25B × (4 / 24)           ≈ 4.2B messages

  per message         ~1 KB ciphertext + envelope
  ─────────────────────────────────────────────────────────
  spool at rest       4.2B × 1 KB              ≈ 4.2 TB

Compare an archive that keeps everything for a year:
  50B × 365 × 1 KB                             ≈ 18 PB

A spool is four thousand times smaller than a history,
and it shrinks by itself every time somebody opens the app.

4 TB against 18 PB is the thesis as a number. It justifies every simplification that follows.

WhatsApp's reported setup has the same shape. They ran a 2 TB in-memory database, sharded across 16 partitions and holding 18 billion records, with a 98 percent hit rate on their write-back cache. That's the size of a short-lived spool, not an archive.

This is how a drain works.

The spool is ordered by deliv_seq, a number per device that goes up with every message that device is owed. It's a different number from the per-conversation sequence in section 5. A device is in many conversations, so conversation numbers collide: two messages numbered 4,412 in different threads would land on the same row if the spool used that number.

The device reconnects. The server reads everything above the device's cursor and pushes it down in order. The device acknowledges by position, not per message. One "I have everything up to 4,412" moves the cursor past a hundred entries in a single write. That keeps the drain cheap for a phone that's been away for a day.

Two rules keep it correct.

The cursor moves on acknowledgement, not on send. Suppose the server pushes messages 4,411 through 4,510 into the socket and moves the cursor straight to 4,510. If the phone's train then goes into a tunnel, all hundred are lost. They may never have reached the phone, but the cursor says they did, so nothing sends them again. If the cursor stays at 4,410 until the device confirms it has them, the same dropped connection costs a resend instead of a hundred lost messages. The price is duplicates when the acknowledgement itself gets lost, and the client-generated message id is there to absorb exactly those.

The cursor lives somewhere any server can read it, not in the memory of the server running the drain. That server can die halfway through. The device usually reconnects somewhere else anyway, because the spool belongs to the device, not to a machine. Server 88 has to be able to pick up a drain that server 42 started, so the cursor sits next to the spool. Phones on trains drop mid-drain more often than you'd guess, so resuming a drain has to be the normal path.

Who writes to a spool, and who numbers the rows

Several different processes write to the same device's spool, and only one reads it.

• The owner process writes most rows. It decides where every message goes. It appends the message to a device's spool when the registry has no entry for that device, when its cache says a group member is offline, or when its forward to the recipient's chat server fails.
• A chat server writes the rest, as a safety net. Sometimes a message reaches a chat server for a device that has already left it, because the owner process's cache or the registry was a moment behind. That server writes the message to the device's spool instead of dropping it. The same happens when a server has pushed a message down a socket and the socket closes before the device acknowledges it. The server can't tell whether the phone got the message, so it writes it to the spool anyway. If the phone did get it, the phone drops the second copy by its message id.
• The device's current chat server reads it. When the device reconnects, the server it lands on drains the spool and moves the cursor. The cursor is the only thing that server writes.

So one device's spool has many writers. A device in 40 conversations can receive spool rows from 40 different owner processes, on 40 different chat servers, at the same moment. Every row needs a deliv_seq, and two rules apply to those numbers. They have to be unique, or one row lands on top of another and a message disappears. And they have to go up in the order the rows are written, or the drain's "everything above the cursor" misses some.

The obvious answer doesn't work. A writer could read the highest deliv_seq in the spool and add one. Two writers doing that at the same moment both read 4,412 and both write 4,413, and the second row overwrites the first. A message is lost, and nothing reports it.

So one place hands out the numbers: a counter per device, in Redis. Before it writes a row, a writer runs INCR seq:d_9. INCR adds one to the key and returns the new value in a single atomic step, so two writers asking at the same moment get 4,413 and 4,414, never the same number. These are ordinary Redis keys, not pub/sub channels, and ordinary keys spread across a Redis cluster by hash slot, so capacity grows with every shard you add.

Every spool row takes one number. The spool block above
puts undelivered messages at 25B a day. Peak is 3x, assumed
as before, and ~100,000 simple operations a second per
Redis shard is a working figure.

  average     25B / 86,400                     ≈ 290,000/s
  peak        × 3                              ≈ 870,000/s
  shards      870,000 / 100,000                ≈ 9
  ─────────────────────────────────────────────────────────
  A handful of shards, and one extra round trip on each
  spool write, a path where nobody is waiting.

Four failures matter here. The first is harmless. The other three need a rule each.

Two writers ask at the same instant. Nothing goes wrong. Redis runs one INCR after the other, and each writer gets its own number.

A writer takes a number and then dies. An owner process gets 4,413 and crashes before it writes the row. The spool now holds 4,412 and 4,414, with a hole between them. Ordering doesn't mind, because the numbers still go up. But the drain can't tell from the hole alone whether 4,413 is gone for good or still on its way.

A writer takes a number and is slow. This is the dangerous one. A writer gets 4,413 and then stalls for a few seconds, perhaps on a garbage collection pause or a slow disk. Meanwhile another writer gets 4,414 and finishes first. If the device drains at that moment, it receives 4,414 and acknowledges it, and the cursor moves past 4,413. When 4,413 finally lands, it's behind the cursor, and no drain will ever send it.

Two rules close that gap, and they work as a pair.

• Every spool write has a deadline. If a writer can't finish its write within 2 seconds, it gives up on that number, takes a fresh one with another INCR, and writes the row again. If the abandoned write lands late after all, it becomes a duplicate sitting behind the cursor. That costs nothing, because the copy written under the fresh number is the one that gets delivered.
• The cursor only moves across numbers that are all there. If the drain finds 4,412 and then 4,414, it still sends 4,414, but it holds the cursor at 4,412, even after the device acknowledges 4,414. Only once 4,413 has been missing for longer than any writer could still be working on it, 5 seconds against the 2 second deadline, does the drain treat 4,413 as dead and move the cursor past it. Holding the cursor means the next drain may send 4,414 a second time. The phone drops that copy by its message id.

The 5 seconds has to stay longer than the 2 second deadline. If it were shorter, the drain could give up on a number that a writer is still allowed to write.

The counter's Redis fails over. Redis copies writes to its replica asynchronously, so when the primary dies, the replica can be a few increments behind. If the replica took over in that state, INCR seq:d_9 could hand out 4,413 again, a number a row already uses, and the new row would overwrite the old one. So a writer doesn't use a number until the replica has it. After INCR, the writer runs WAIT 1, which blocks until the increment has reached the replica, usually within a millisecond. Only then does it write the row. If the primary dies, the replica already has every number that was handed out, so nothing rewinds. If the replica itself is down, WAIT times out and the writer treats it like any other failed write and tries again. That extra millisecond lands on a path where nobody is waiting.

So the spool has many writers and one counter per device. The counter makes the numbers unique. WAIT keeps them rising across a failover. And the deadline and the cursor rule together make sure that a slow writer can't land a message behind a device that has already moved on.

Which data stores, and the one you do not build

Every interviewer asks this, and here the answer is unusual. Most designs start by choosing a database for the messages. There is no messages table, and saying why is worth more than whichever data store you pick.

The registry doesn't need durability either. Every chat server already knows which sockets it holds, so if the registry is lost, you rebuild it by asking them. That's why 100 million live connections can sit in memory instead of in a database.

Why a wide-column store, and not one of the others

The table above says which requirement forces each data store. It doesn't say why the spool goes in Cassandra rather than the database you already run, and that's the follow-up question. The spool needs four properties, all at once.

The primary key is the access pattern. A partition key of device_id and a clustering key of deliv_seq make one device's spool a single contiguous partition, already in order. A drain is then a range scan inside one partition starting at the cursor. That's the cheapest read a distributed data store can do: one node, no scatter-gather, no secondary index.

Appends stay cheap at 25 billion a day. That's about 290,000 writes a second on average. A log-structured engine collects them in memory and writes them to disk in one long sequential stream. A B-tree engine, like Postgres or MySQL, works differently: every single write has to go to its own place on disk, and the index has to be updated as well. At 290,000 writes a second, that's 290,000 scattered disk writes instead of one steady stream.

Expiry is built in, not a job someone has to run. Rows past default_time_to_live are reclaimed during compaction, which the engine runs anyway. This property forces the choice, because it turns 25 billion deletes a day into zero. If an engine implements expiry as a background sweep that issues real deletes, you're back where the staff tier above started.

No single primary. Writes route by a hash of the partition key to the replicas that own it, so write throughput grows as you add nodes. A single primary accepting every write can't reach 290,000 a second.

Together, these four rule out most engines and leave one kind standing. Each row below gives the verdict first.

So the spool goes in a wide-column store, because its primary key, its write path and its expiry all match the spool at once. Use Cassandra or ScyllaDB if you run it yourself, and DynamoDB if you'd rather not. The choice between them comes down to cost and how much you want to operate, not to fit.

The tables

-- Small, relational, boring on purpose.
users (
  user_id     bigint primary key,
  phone_hash  bytea unique,
  created_at  timestamptz
)

devices (
  device_id   bigint primary key,
  user_id     bigint not null,
  push_token  text,                    -- for waking a disconnected device
  last_seen   timestamptz
)
  index (user_id)                       -- fan out to a user's devices

group_members (
  group_id    bigint,
  user_id     bigint,
  role        smallint,
  primary key (group_id, user_id)
)
  index (user_id, group_id)             -- "which groups am I in"

-- The spool. Wide-column, one partition per device.
spool (
  device_id   bigint,                   -- partition key
  deliv_seq   bigint,                   -- clustering key: from INCR seq:{device_id}
  message_id  uuid,
  sender_id   bigint,
  payload     blob,                     -- opaque, the server cannot read it
  primary key ((device_id), deliv_seq)
) WITH default_time_to_live = 2592000;  -- 30 days, expiry not DELETE

-- Not a table. In memory, rebuildable.
--   key    conn:{device_id}
--   value  { server_id, epoch, connected_at }
--   also   cursor:{device_id} -> last acknowledged deliv_seq
--   also   seq:{device_id}    -> last issued deliv_seq, via INCR

Four things to note.

payload is a blob the server never interprets. Messages are end-to-end encrypted, so the server can't index, search, or validate content. That removes every feature that would have needed a richer schema, so the spool can be a queue instead of a database.

The spool is partitioned by device, not by user or conversation. Delivery is per device: the same message goes to a phone, a laptop and a tablet independently, and each acknowledges separately. If you partitioned by conversation, a drain would have to scatter across every conversation the device is in and gather the results.

The TTL is the deletion strategy; the cursor is the delivery record. A delivered message isn't removed. The cursor moves past it, and expiry collects it later. This turns 25 billion deletes a day into zero.

group_members is indexed both ways. Fanout needs the members of a group, and the client needs the groups a member is in. That's the same link read in two directions, and at this scale one table with two indexes is fine.

The queries that carry the product

-- 1. Where is this device right now? Runs on every message,
--    1.7M times a second at peak. Memory, not SQL.
--    GET conn:{device_id}

-- 2. Append to the spool when the registry says nobody is home.
--    One write, no read-modify-write, no delete later.
INSERT INTO spool (device_id, deliv_seq, message_id, sender_id, payload)
VALUES ($1, $2, $3, $4, $5);

-- 3. Drain on reconnect. Resumable from the cursor, so a
--    connection dropping mid-drain resumes rather than restarts.
SELECT deliv_seq, message_id, sender_id, payload
FROM spool
WHERE device_id = $1 AND deliv_seq > $2
ORDER BY deliv_seq
LIMIT 500;

-- 4. Group fanout. Members, then each member's devices.
SELECT user_id FROM group_members WHERE group_id = $1;
SELECT device_id FROM devices WHERE user_id = ANY($2);

Query 1 is the core of the system. It runs more often than everything else combined, it's why the registry lives in memory, and its p99 uses most of your delivery budget.

Query 3 is the one to watch. Without LIMIT 500, a device that's been offline for two weeks pulls 100,000 rows into one response, and the drain times out before it finishes. So read the spool in pages, have the device acknowledge each page, and move the cursor after each one.

One thing is missing on purpose: no query reads message history, because there's nowhere to read it from. If an interviewer asks for search, or "show me last month's messages," that's not a query you can add. It's a different product with an 18 PB data store attached, and saying so is the right answer.

4. Fan out a group message

One message to a group of 1,024 means 1,024 deliveries, and groups are where the volume comes from. WhatsApp's published figures show it: at 465 million users, they took in 19 billion messages a day and sent out 40 billion. On average, every message sent became about two deliveries, even though most chats are one-to-one.

500M daily active users is requirement 3. About one
connected device each and ~4 reconnects a day are assumed.

  daily active users      500M, ~1 connected device each
  reconnects per device   ~4 a day: a new network, or the
                          app coming back from the background
  reconnects              500M × 4 / 86,400        ≈ 23,000/s
  events per reconnect    one when the old socket goes,
                          one when the new one arrives   × 2
  events                                           ≈ 46,000/s
  ─────────────────────────────────────────────────────────
  About 80 times fewer than the 3.6M deliveries a second,
  and under half of what one Redis shard handles.

5. Order the messages, and report the receipts

Ordering happens per thread, and the sender starts it. Each client keeps a counter per thread and stamps every message with the next value. The server assigns its own sequence when it accepts the message, and recipients order by that server sequence. Ordering by wall-clock timestamp is the classic mistake: two phones with clocks four seconds apart will interleave a conversation wrongly, and nothing the server does can fix that.

The owner process assigns the sequence, which is another reason a conversation needs a home. A one-to-one chat gets an owner process too, placed the same way, so it stays the same whoever is talking.

Three more rules make this work.

The client's own counter keeps the sender's own messages in order. Alice fires three messages in a second, and they can reach the server out of order: after a retry, a reconnect, or two sockets briefly open during a handover. Without her counter, the owner process numbers them in arrival order, and Alice's own messages end up shuffled. With it, the owner process knows 41 came before 42, whichever one landed first.

Knowing the order isn't enough on its own, because once the owner process hands out a number, it can't take it back. So it waits before it numbers. For each sender in a conversation, it remembers the next client counter it expects. If Alice's 42 arrives while it still expects 41, it holds 42 in memory without numbering it. When 41 arrives, it numbers 41 as 4,410, then releases 42 and numbers it 4,411. The owner process acknowledges a message only once it has numbered it, so Alice's phone keeps retrying 41, and that retry usually fills the gap within a moment. The wait has a limit of a second or two. If 41 never turns up, the owner process stops waiting and numbers 42. A 41 that arrives later still gets delivered, just after 42: slightly out of order, but not lost.

Gaps are detected from the server sequence, not the client counter. The server sequence is the only number both devices share. A device holding 4,410 and 4,412 knows 4,411 is missing and can ask for it, instead of showing a chat with a hole in it. Clients notice gaps far more reliably than servers notice that a send failed.

A message that arrives out of order is held back, not shown. If 4,412 arrives before 4,411, the client waits a moment for the missing one before showing either. Without that, the user sees two lines swap places on screen. It looks like a bug, even though every part of the system did its job.

Reads, at peak
  a one-to-one send needs one lookup, to find
  the server holding the recipient's device
    1.7M sends/s                               = 1.7M reads/s

  a group send would need one lookup per member,
  which is the thing that would break this
    a 1,024-member group                       = 1,024 reads

  so the owner process keeps a cache of who
  is connected, refreshed by the same heartbeats
    one cache read per group message           ≈ 1 read

  Reads track sends, not recipients. The cache that makes
  that true has to be kept fresh by the connect/disconnect
  stream below, which is its own cost.

Writes, which are driven by reconnects rather than by traffic
  the registry is written when a device connects and its server
  claims the entry, and again when that entry goes away. Nothing
  else touches it, so message volume does not appear here at all.

  on an ordinary day, one write per reconnect
    reconnects, worked out in section 4        ≈ 23,000/s

  when one server restarts, which is every deploy
    all of its sockets come back inside about a minute
      1M sockets / 60s                         ≈ 17,000/s
  ─────────────────────────────────────────────────────────────
  One server coming back produces nearly as many writes a second as
  the entire fleet generates on an ordinary day. The load that matters
  here is self-inflicted, and it arrives on a schedule you control.

That's the argument for letting entries expire on their own instead
of deleting them. A cleanup job would have its heaviest work to do
right when the fleet is busiest, exactly when it's least likely to
get scheduled. Expiry doesn't need anybody to be available.

Receipts look like messages, but they aren't. A message to a group of 1,024 goes out to 1,024 people. A read receipt goes to exactly one person, the sender, because nobody else is shown who read what. If you treat a receipt like a message and fan it out to the whole group, one message turns into 1,024 × 1,023 delivery events, about a million. And none of them shows the group anything, because the group never sees receipts.

So receipts flow inward instead of fanning out. A receipt is for the sender, but it travels to the conversation's owner process, which already holds that conversation. A thousand people reading one message become a thousand position updates arriving at that one place.

The owner process doesn't forward these one by one. It keeps all 1,024 numbers and sends them on in small batches. Every few seconds, it pushes only the positions that changed since the last batch, such as "Bob and Priya have now read up to 4,412". The batches go to the devices of the people who sent messages in the conversation, never to the whole group. Each of those devices keeps its own table for the conversation: how far every member has read, one sequence number per member. Each batch updates that table. If the sender has the chat open, a tick turns blue as soon as the batch arrives. If the sender is offline, the batch waits in their spool like any other message. Positions only ever move forward, so a batch that arrives late can't undo a newer one.

Read state is a position, not a list of events. A client reports "I have read up to 4,412 in this thread," one number per member per thread. It doesn't report "Bob read message m_71c" once per message. That choice pays off three ways. Restating a position is harmless, because saying it twice gives the same answer, while replaying a stream of events would count things twice. A group of 1,024 members is always 1,024 integers, however many messages the thread holds. And the number is small enough that a client can resend it whenever it reconnects.

The reader's device holds the truth. Bob's phone knows exactly how far Bob has read, and the owner process's tally is only a view of what the devices know. The same shape shows up three times in this design. The connection cache is a view of the registry, and the spool cursor is a view of what a device has confirmed. In each case the truth sits where the fact was created, and the server-side copy exists only to make a read cheap.

So a crash costs only the tally. A replacement owner process starts with an empty view and refills it as members restate their positions. They do that anyway, because a reconnect already carries a position for the spool drain. There are two caveats. A member who never comes back never restates, so the count can stay below the truth forever. And if you'd rather not depend on restatement, store the positions instead: one row per member per thread, written only when someone reads, which is far cheaper than a row per message.

After 30 days the server has nothing, and that's by design. Read state lives where the messages live, on the devices. When you open a chat from six months ago, the ticks come from your own phone, not from a server answering a query. The spool's expiry is a delivery deadline, not a retention policy, and receipts follow the same rule. The owner process holds positions for a live conversation only long enough to batch them to the senders. Nothing survives the expiry, so for an old chat, the devices in it supply the whole answer.

The sender sees two things, both built from the same 1,024 numbers. The chat list shows a tick, which is a yes or no: has every member read past this message? The message info screen shows one line per member, the same check written out instead of summed up. Both read the table on the sender's own phone, so opening message info two months later works the same as two minutes later.

Deep dives

The first three come up in nearly every version of this interview.

1. How many servers, and what actually needs scaling?

Do this arithmetic out loud. The answer settles the shape of the whole design.

Sockets
  100M live at peak
  a tuned server holds ~1M sockets
    (WhatsApp reported 1M/server, down from 2M, on
     FreeBSD with 64 GB and dual 10-core CPUs)
  100M / 1M                                = 100 chat servers
  memory per connection, buffers + state
    ~30 KB × 1M                            ≈ 30 GB per server

Messages, derived in the transport section above
  1.7M/s sent at peak, 3.6M/s delivered

Per chat server at peak
  1.7M / 100                               ≈ 17,000 sends/s
  3.6M / 100                               ≈ 36,000 pushes/s
  ─────────────────────────────────────────────────────────
  A hundred servers. The connections size the fleet,
  and the messages barely register.

The fleet is sized by sockets, not by messages. Say that in the room. 36,000 socket writes a second per server is ordinary. A million open sockets on one server is the hard part, and that's why every serious chat system runs on a runtime with cheap, lightweight processes. WhatsApp ran roughly 550 servers in total for 465 million users: 150 chat servers, and 250 handling multimedia, which never touches the chat path.

Two consequences follow.

Scaling out is cheap, and scaling in is not. Adding a chat server is easy. Removing one drops a million sockets, and they all try to reconnect at once. WhatsApp measured logins peaking at 230,000 a second. A reconnect storm is self-inflicted. It shows up during deploys, when you're already changing things under the fleet.

Overprovision on purpose. Sockets are the limit, and reconnect storms are how the fleet fails, so running it close to full is a bad trade. Keep enough headroom that losing one server, or ten, doesn't push the rest past their socket limit and set off a chain of failures.

2. Where does the 500 ms go?

  phone to nearest edge, mobile network      40 to 150 ms
  edge to chat server, same region            1 to 5 ms
  registry lookup                             0.5 to 2 ms
  server to server forward                    0.5 to 2 ms
  spool write, only when offline              2 to 10 ms
  chat server to recipient phone             40 to 150 ms
  ─────────────────────────────────────────────────────────
  both online, same region                    82 to 310 ms
  cross-region                               +80 to 200 ms

The two mobile legs are about 95% of the budget and
you control neither. Everything you do control adds
up to about 10 milliseconds.

That's the useful result: your servers aren't where the latency is. Past a certain point, making the internal path faster buys nothing a user can feel. What helps is putting the socket close to the user. Terminate connections at regional edges instead of in one datacentre, and when two people are in the same place, keep both of them on servers in the same region.

Keeping the socket alive also saves latency. On a mobile network, a new TLS session costs several round trips before a single byte of a message can move. So chat clients hold one connection open and send small keepalives, instead of reconnecting for each message.

Keepalives are a small design problem of their own. If they're too frequent, they drain the battery and keep waking the radio. If they're too rare, carrier equipment drops the idle socket, and the client believes it's connected while nothing can reach it. Most clients settle on a keepalive every few minutes, tuned per network. When a keepalive gets no response, the client treats the socket as dead and reconnects at once.

When the app is in the background, though, a phone won't keep a socket open at all. The operating system suspends the app, and the connection goes with it. So the message has to take a different route: a small notification through Apple Push Notification service on iOS, or Firebase Cloud Messaging on Android. That service delivers it to the phone when it chooses to. The phone wakes your app, and the app reconnects and drains its spool.

So the spool ends up doing a second job. It holds messages for offline users, and it also holds a message while a backgrounded phone is being woken up. That wake-up runs on Apple's schedule or Google's, not on yours, and both batch, delay and rate-limit notifications to save battery. You can't tune that dependency, so say so, rather than quoting a p99 that only covers your half of the path.

When each socket ends close to its user, a sender on a server in Frankfurt can be talking to a recipient on a server in Mumbai. The forward between them crosses the world, and that cost can't be avoided. One of the two has to wait for it.

You can avoid crossing the world twice for two people sitting next to each other. Every message goes through the conversation's owner process, because that's where the sequence number comes from, so its location is on the path every time.

Suppose every owner process is pinned to one fixed region, usually whichever region the company's infrastructure started in, say Virginia. Two people messaging each other in Mumbai now send every message to Virginia and back. Neither of them is anywhere near Virginia, and nothing about the message needed to leave India.

So don't pin owner processes to a region. Place each one near the people in its conversation.

A thread's home should follow its members. For a one-to-one chat between two people in the same region, everything stays local and the message never leaves. For a group spread across continents there is no good answer, so pick the region holding the most members and accept that the rest pay a hop. Say that out loud rather than drawing a single global box, because "where does the group live" is the follow-up an interviewer will ask.

3. What happens when a chat server dies?

A million devices lose their socket at once, and that happens on every deploy, not only during incidents.

The sockets were only half of what that server was holding

A chat server holds sockets, and it also runs the owner process for every conversation that hashed to it. Both go when the server dies. The sockets fail visibly: a million phones start reconnecting at once. The owner processes fail without any sign. Nothing reconnects to fix them, and the damage shows up later, as messages in the wrong order.

An owner process holds two things. The member list is easy: it lives in the relational database, and whichever server takes over the conversation reads it back.

The sequence number is the hard one. It lived in memory, and it must never repeat. Two messages carrying the same number get ordered differently on different clients, and section 5 exists to prevent exactly that.

This is a different counter from the spool's deliv_seq in section 3. That one numbers rows in one device's spool, on the offline path, where nobody is waiting, so Redis INCR plus WAIT is fine there. This one numbers the messages in a conversation, and it sits on the live path of every message.

There are three ways to make a sequence survive its owner process, and the last one is the answer.

Persist every number. Write the counter durably before handing each number out, either to a database or to Redis with INCR and WAIT, the way the spool does. This is correct, but it puts a round trip on every message, on the one path that section 2 worked to keep free of durable writes.

Recover by asking. After a failover, poll the chat servers or the clients for the highest number they have seen and carry on from there. The answer depends on who replies and how fast, so two recoveries can disagree.

Lease numbers in blocks. The new owner process claims a block of 1,000 from a small durable counter, hands them out from memory, and claims the next block when it runs low. That's one durable write per thousand messages, instead of one per message. A crash loses only the unused end of the current block, and no number is ever handed out twice.

Leasing has a side effect that only shows up next to another part of this design, so follow it through in the room. The sequence now has holes in it. An owner process that died holding 4,413 through 5,000 leaves those numbers unused forever, and its replacement starts at 5,001. Ordering doesn't mind, because it only needs the numbers to increase. Gap detection does mind. A client holding 4,412 and then 5,001 concludes that 588 messages are missing, and asks for them. The server answers that nothing exists in that range, the client stops asking, and the chat shows correctly. That's one wasted request per failover per active client. It's rare and cheap, and far better than a client showing a conversation it believes is incomplete.

One more thing has to hold, and the fencing pattern above handles it. During a network partition, the old owner process can still believe it owns the thread after a replacement has been chosen. Both then hand out numbers, from different blocks, to different halves of the conversation. The epoch on the ownership record makes the system reject the loser's writes, so the thread ends up with one sequence instead of two mixed together.

4. Why media never touches the chat path

Someone sends a 40 MB video to a group of 1,024. If that video travels the way a text message does, you push 40 GB through your chat fleet for a single send, and the fleet is sized for sockets, not bandwidth.

So it doesn't travel that way. The sender's phone asks a chat server for a presigned upload URL, a short-lived link that allows exactly one write to object storage. It uploads the file straight to object storage with that link, then puts a reference in the message. Every recipient gets a small envelope with a pointer. Each one fetches the bytes itself from a content delivery network (CDN) such as Cloudflare or Fastly, with a presigned download URL, over a path that never touches a chat server.

One 40 MB video to a group of 1,024.

  through the chat path
    40 MB × 1,024                             = 41 GB per send
    at 100 chat servers, one send             = 410 MB per server

  upload once, fetch from the edge
    into object storage                       = 40 MB
    the message itself, a pointer             ≈ 200 B × 1,024 = 205 KB
    ────────────────────────────────────────────────────────
    chat fleet carries                        205 KB, not 41 GB

WhatsApp reported 250 of roughly 550 servers doing media, and
only 150 doing chat, plus a peak of 146 Gbps outbound. The
media tier is the bigger fleet, and it's a separate one entirely.

205 KB instead of 41 GB. That gap is why separating media is part of the design, not a later optimisation. The rule applies beyond media: the chat path carries decisions, not payloads. Anything large travels as a pointer.

Three details make it work in practice.

Upload first, then send. The upload finishes before the message frame goes out, so a message is never sitting in a spool referring to bytes that do not exist yet.

The reference is a content hash, not a path. It's a fingerprint of the encrypted file, so forwarding a photo to four more groups reuses the same stored copy and its key, and nothing is uploaded again. Two people sending the same photo independently still store two copies, because each one encrypts it with its own key.

The sender encrypts the file, and the key travels inside the message. Object storage holds ciphertext it cannot read.

Encryption usually makes an architecture harder. Here it makes it simpler. An encrypted file needs no access control to stay private, so the presigned download URL only limits who can pull bytes at your expense. A leaked link exposes nothing, because the key never leaves the chat message.

5. Multiple devices, and why it changes the model

A user with a phone, a laptop, and a tablet isn't one recipient. They're three, and every part of the design has to account for that.

The fix was already made in the entities: address devices, not users. The registry maps device to server. The spool is per device. Fanout resolves a user to their device list first. A message to a user with three devices means three deliveries and three spool entries, and delivery only counts as complete once all three have it.

Three consequences are easy to miss.

Read state is per user, not per device. Reading a message on your phone should clear it on your laptop too, so a read receipt has to go to your own other devices as well as to the sender. This fanout is easy to leave out of a first design. It stays small for the reason section 5 gives: when you open a chat with 40 unread messages, your phone sends one frame, read_through: 4412, not 40 separate receipts. Catching up on forty messages costs the same as catching up on one.

A device that's been off for a month. Its spool has been filling the whole time, unless you capped it and started dropping messages. Cap it, and be clear about what happens past the cap. A device that has been gone long enough is treated as new, and it resynchronizes from the user's other devices rather than from the server.

Encryption multiplies, but only when a group changes. Each device holds its own key, so encrypting a group message separately for every device would mean 2,048 encryptions and 2,048 uploads per message in a group of 1,024 users with 2 devices each. Group messaging avoids that with sender keys, from the Signal protocol. The sender makes one group key and sends it once to each member device, encrypted separately for each. After that, every message is encrypted once with the group key and uploaded once, and the server fans out the single copy. The per-device cost comes back only when someone joins or leaves, because the sender then makes a new key so a departed member can't read what comes next. In a one-to-one chat, the sender still encrypts once per device, for the recipient's devices and its own, which is a handful.

6. Encryption, and what the server can no longer do

This is the deep dive that changes the design the most, and it's the one candidates raise least often.

If the server can't read the payload, many things you might have designed become impossible, not just expensive:

One row changes the design instead of only removing a feature: delivering to a new device from server history. When you buy a new laptop, you want your chats on it. Normally the server sends them, because the server has them. Here it has neither the messages nor the keys. The messages were deleted on delivery, and anything it did hold would be ciphertext it can't read. So only a device that already has your history can give it to a new one. Adding a device becomes a real flow, in which an existing device encrypts your history for the new one and sends it across. It's not a checkbox in settings. That's why WhatsApp Web needed your phone online for years.

7. Hot groups and the largest chats

A very busy group of 1,024 puts all its load on one owner process, which hands out the sequence numbers, runs the fanout, and reads the same member list over and over.

The obvious move is to split the thread across processes, and that breaks something. The sequence number has to come from one place. Otherwise two messages get the same number, and clients order them differently. So the split keeps sequencing in one place and spreads only the fanout. That's exactly the relay layer Discord added between guild processes and session processes.

The other move is to notice that a very large group isn't really a chat. Past some size it becomes a broadcast channel: one person talks, everyone reads, and replies are rare. A broadcast fans out when people read, not when someone writes. So above a few thousand members, the right answer is to switch products instead of stretching the chat path. Saying so beats designing a chat system that is secretly a broadcast system.

8. How do you know it is working?

During an incident, you must be able to answer four questions. How many sockets are up, and where? What fraction of messages is going to the spool instead of being delivered live? How long has the oldest undrained spool entry been waiting? And are receipts coming back?

• Live sockets per server and per region, as the main capacity signal, since sockets are what size the fleet.
• Spool depth and oldest entry age. A rising oldest-entry age means devices are not reconnecting, which usually means you broke something rather than that users went to sleep.
• Delivery latency split by online and offline path, because averaging them together hides everything.
• Reconnect rate, alarmed. A spike is either a deploy or the beginning of a storm, and you want to know which within seconds.
• Don't label metrics by user or by thread. Every distinct label value becomes its own time series, so a user id label means up to 2 billion of them, and the metrics backend falls over when you need it most. Aggregate by region, server and path instead, and trace one person with logs, not metrics. The urge to add that label is strongest during an incident, which is exactly when it would take your monitoring down.

Two of these signals carry more than the rest.

A synthetic pair. In every region, two test accounts send each other a message every few seconds and report how long it took to arrive. Real user traffic tells you about averages across 100 million people. A synthetic pair tells you, right now, whether a message from Frankfurt reaches Mumbai. It's the only signal that covers the whole path end to end, instead of the piece each service can see.

The spool's oldest entry, per region. Depth alone is misleading, because a big spool that is draining fast is healthy, and a small one that has not moved in 10 minutes is not. Age tells them apart. If you alert on age and chart depth, you catch a broken region before your users open the app.

Both measure what the user experiences, not what your servers report about themselves, and that's why they matter. A fleet can show every socket as healthy while nothing gets delivered, and only a signal that crosses the whole path catches that.

Final design

One message, end to end:

1. The sender's device pushes a send frame over its existing socket to its regional edge, and on to the chat server that holds it.
2. That server routes the message to the conversation's owner process, which assigns the server sequence and resolves the member list to devices.
3. The owner process splits recipients by connection state, using its cache of who is connected.
4. For connected devices, it groups them by destination chat server and sends one forward per server. Each destination writes to the sockets it already holds.
5. For everyone else, it appends to per-device spools in one batched write. Nobody is waiting on this, so it is off the latency path.
6. The sender gets an ack as soon as the message is durable, which is the first tick. Receipts arrive later and independently, which is the second.
7. On reconnect, a device authenticates and the chat server holding it writes the registry entry. That server then drains the device's spool in deliv_seq order, moving the cursor past each acknowledged entry, holding it at any gap younger than 5 seconds, and letting the TTL reclaim the rows later.
8. Every stage emits live sockets, spool depth, delivery latency and reconnect rate, never labelled by user.

Four numbers justify the shape. 100 million live sockets sit on roughly a hundred chat servers. 1.7 million messages a second come in and become 3.6 million deliveries going out. The spool holds about 4 TB, where an archive would hold 18 PB. And our side adds around 10 ms of latency, inside a budget the mobile network has already mostly spent.

What is expected at each level

Sources

Every real-world figure here comes from a published source, not an anecdote. The first one is unusually detailed for a company that otherwise said very little publicly.

• High Scalability, How WhatsApp Grew to Nearly 500 Million Users, 11,000 Cores, and 70 Million Messages a Second. The source of nearly every number used above. 465 million users. 19 billion inbound and 40 billion outbound messages a day. 147 million live sockets at peak and 230,000 logins a second. Roughly 550 servers, of which only 150 were chat, at 1 million sockets each. A 2 TB in-memory store across 16 partitions. And the line that reframes the problem: messages are stored only while they are being delivered.
• Discord, How Discord Scaled Elixir to 5,000,000 Concurrent Users. The session-process-per-connection model, the guild process that owns fanout, and the relay layer added between them when single processes became the bottleneck.
• Elixir, Real time communication at scale with Elixir at Discord. Where the figure that about 90 percent of members in a large group are passive comes from, and the roughly 90 percent saving that skipping their fanout produced.
• Redis, Pub/sub. The at-most-once semantics, the absence of any persistence, and sharded pub/sub with SSUBSCRIBE and SPUBLISH in Redis 7, which replaced the cluster-wide broadcast that made shards useless for pub/sub capacity.
• Discord, Manifold. The batch-by-destination-node library behind group fanout, and the roughly 70 microseconds per send call that makes batching worth doing.