Fanout on Write against Fanout on Read

The shape

One thing is produced. Many people need to see it. Somewhere between the producer and the consumers, the item has to be copied into whatever each consumer reads from. The only thing to decide is when that copying happens, and there are two answers.

Fanout on write does the copying when the item is created. A post arrives, and the system immediately appends its id into a per-consumer list for every follower. Reading is then one range read of one list, which is about as cheap as a read gets. You paid for it at write time, once per consumer.

Fanout on read does the copying when somebody looks. Nothing happens at write time beyond storing the item. Reading means going to every producer this consumer follows, pulling their recent items, and merging them. Writes are free. A read now costs one query per producer the consumer follows, so it gets slower the more they follow.

Put generally, you are choosing whether the cost lands on the producer or on the consumer. That is why this turns up in places that look nothing like a social feed. A notification service deciding whether to build a per-user inbox up front is making this choice. A chat system deciding whether a group message writes N copies or one copy read N times is making it. A search index deciding between updating documents eagerly and merging segments at query time is making a version of it. So is any cache that could be filled on write or filled on miss.

It is worth naming as a pattern because the answer is almost never "pick one". The answer is a threshold, and finding it is the work.

The mechanism

Both paths have the same three parts: an item, a set of consumers, and a place each consumer reads from. They differ in who fills that place.

The arithmetic that decides it is a single comparison. For a given producer:

fanout on write cost  =  followers x (writes per item)
fanout on read cost   =  readers   x (reads per session) x (producers followed)

For a normal account these are not close. Someone with 300 followers who posts twice a day costs 600 list appends. Serving those same followers lazily would mean every one of their feed opens querying a few hundred accounts and merging. Write fanout wins by orders of magnitude, because the read side happens far more often than the write side in almost every system where this pattern appears.

Then the ratio inverts. An account with 100 million followers costs 100 million appends for one item. That is a batch job wearing the shape of a write. Meanwhile the number of readers who want it is exactly the same hundred million, and each of them is going to open the app anyway. Doing the work lazily costs one extra list to merge per read.

So the standard answer is the hybrid. Fan out on write for ordinary producers. Skip fanout for the small set of very large ones, and merge their recent items in at read time. Every consumer's read becomes a cheap range read plus a small merge of the handful of large accounts they follow.

The threshold is where the two costs cross. You can compute it for your own system rather than copying a number somebody else uses.

Here is the part that gets told wrong. The hybrid is not adopted because write fanout gets more expensive in total. Per follower it stays cheaper at every scale, and the worksheet below shows it staying cheaper by the same 200x whether the account has 300 followers or 100 million. The ratio never crosses.

What forces the hybrid is that one write turns into 100 million operations that all have to finish before the item is visible. That is a latency problem and a burst-capacity problem, and a cost-per-follower model shows neither of them.

Take a system with:
  posts per producer per day                    2
  feed opens per consumer per day              20
  cost of one timeline append              ~0.1 ms
  cost of one extra list merged at read    ~2.0 ms

Fanout on write, per producer per day:
  followers x 2 x 0.1 ms

Fanout on read for that producer, per day:
  followers x 20 x 2.0 ms   (each follower merges
                             this one extra list,
                             on every feed open)

  → write    0.2 ms per follower per day
  → read    40.0 ms per follower per day
  ──────────────────────────────────────────────
Writing is 200x cheaper per follower, and it stays
200x cheaper at 300 followers and at 100 million.
Cost per follower never picks the hybrid for you.

Three problems, and no per-unit cost model shows any of them.

• Latency. The item is not visible to the last follower until the whole batch drains, which can take minutes.
• Burst capacity. You size the fanout tier for a celebrity posting, not for the average post rate, and that peak is thousands of times the mean.
• Wasted work. Most of those hundred million timelines belong to people who will not open the app today. You wrote into all of them anyway.

How to tell you have this problem

You have it whenever one write has to become visible in many places. Three symptoms, and each one tells you which side you are on:

Writes are slow or spiky, and the spikes correlate with specific users. Draw a graph: follower count along the bottom, post-to-visible latency up the side. Post-to-visible means the time from somebody creating a post to it showing up in the last follower's feed. If the slowest 1 percent of posts (the p99) come almost entirely from a handful of very large accounts, you are fanning out on write past the point where it works.

Reads are slow and the latency scales with how much the user follows. Draw a graph: how many accounts the consumer follows along the bottom, feed load time up the side. If that line slopes upward, you are fanning out on read, and the people who follow the most are getting the slowest feeds, which is backwards.

Your fanout queue depth is bursty rather than steady. Steady depth means throughput sizing. Bursts that take minutes to drain mean single writes are exploding into batches.

One query settles it in most systems. Count followers per account, biggest first: select count(*) from follows group by followee_id order by 1 desc limit 100. If the top of that list is four or five orders of magnitude above the median, you need the hybrid. The shape of that curve tells you where to put the threshold.

What it costs

Four costs, and the write volume is not among them.

Two code paths that must produce identical results. A consumer's feed is now built from a pre-written list plus a live merge. The two can disagree about ordering, about which duplicates to drop, and about what to filter out. Every feature you add later, ranking, blocking, muting, gets built twice and has to agree with itself. That cost never goes away.

The threshold is a config value, and crossing it changes behaviour all at once. An account that crosses it switches strategy from one post to the next. Handled carelessly, that leaves a window where the account's recent items are in some timelines and not others. A follower sees an item vanish and come back. Handle the crossing explicitly rather than letting it happen.

A timeline written in advance is a cache, and it can be wrong. Deletes, blocks, privacy changes and edits all have to reach data you already copied. Most systems handle this by filtering at read time instead of rewriting timelines. That moves work back to the read side, which is the side you were trying to keep cheap.

Storage for something most people will not read. Writing on fanout fills the timelines of people who are not using the app. Capping timeline length is standard, and so is skipping anyone who has not opened the app in N days. Both add a fallback path for when a skipped user returns.

Where it shows up in the course

Getting it wrong

Treating it as a binary choice. Almost every real system uses both. A design that picks one and defends it has usually not looked at the follower distribution.

Putting the item in the timeline instead of the id. Timelines hold references. Copying the item itself multiplies your storage by the average follower count and makes every edit a rewrite of millions of rows.

Sizing the fanout tier for the mean. The mean is irrelevant. Size it for the largest producer you fan out to, which is the threshold you chose, and then the threshold is also a capacity decision.

Forgetting that the read path still merges. Even with a hybrid, reads combine a materialised list with live queries. If that merge is not bounded, a user following two thousand large accounts finds the slow path.

Letting the threshold be implicit. "We handle celebrities differently" is not a design. The number, and what happens to an account the day it crosses, is the design.

Assuming the hybrid is about cost. It is about latency, burst capacity and wasted work. Per follower, eager fanout stays cheaper at every scale, and saying so shows you did the arithmetic rather than repeating what everybody says.