How Meta handles billions of requests, and what your company should copy

When engineers describe how Meta serves billions of people, the conversation can lead a 100-person company to ask whether it needs the same architecture. It does not. What a smaller company can borrow from Meta is the sequence in which large systems add complexity, because that sequence shows why each step exists and when it becomes worth the trouble.

How does Meta handle billions of requests?

Meta spreads the work across layers, so that no single machine sees everything. For its social graph, requests arrive at load balancers, a very large cache (Memcache and TAO) absorbs much of the read load, and a sharded MySQL tier holds the authoritative copy. Meta runs other storage systems for other workloads. A load balancer distributes incoming requests across many servers, and a cache keeps copies of frequently read data in fast memory so the database is asked less often.

The published figures come from different years, so read each one with its date:

These numbers describe a company with over a billion users already in 2013. They set the ceiling of what is possible, and they say little about what a business with 5,000 customers needs.

The network layer sits in front of all this. Meta's 2018 post on Katran describes a layer 4 load balancer that works on network packets rather than application requests, deployed on servers in Facebook's points of presence, which act as proxies for its data centers. Its Proxygen library is, per its README, the core set of C++ HTTP abstractions used at Facebook and the basis for many HTTP servers, proxies and clients.

What should your company copy, and when does each technique pay off?

Copy the order, not the end state. Each technique solves one problem, has a scale at which it starts to pay off, and has a cheaper step that comes first.

What is sharding, in plain English?

Sharding splits one database across several servers, with each server (a shard) holding a subset of the data. MongoDB's documentation defines it as a method for distributing data across multiple machines, used for very large data sets and high throughput. It calls the underlying idea horizontal scaling, which means adding servers instead of buying a bigger one, and states the price: increased complexity in infrastructure and maintenance.

The shard key is the field that decides which shard a record lives on. Pick it well and most requests touch one shard. Notion partitioned by workspace ID because, per its engineering post, each block belongs to exactly one workspace. Meta's TAO embeds a shard ID in each object ID, and stores each association on the shard of its source object, so an association query can be served from a single server.

There are two common ways to map keys to shards:

• Range sharding assigns contiguous ranges of key values to each shard. MongoDB notes that nearby values sit together, so queries for a range can go to specific shards, but a poorly chosen key can produce uneven distribution and hot spots.
• Hash sharding computes a hash of the key. MongoDB says this gives more even distribution, but range queries on the key are more likely to hit many shards.

What does sharding cost?

Sharding trades capacity for complexity, and three costs recur.

Cross-shard queries. When a query does not include the shard key, MongoDB's documentation says the router queries all shards in the cluster, and that these scatter/gather queries can be long running. Notion's post notes that distributed joins are expensive and that transaction guarantees are typically limited to a single host. It gave an example where a block deletion could succeed while a related comment update fails. TAO also has this issue: its paper says an association can involve two shards, because the forward edge sits on one shard and the inverse edge on another.

Resharding. Changing the shard key or rebalancing data later is slow. MongoDB says the balancer's chunk migration process is slow and resource intensive, and that you can reshard a collection from version 5.0 onward but should choose the key carefully to avoid problems. Notion's backfill of existing data took about three days on a large instance.

Operations. More servers mean more backups, upgrades, failure cases and monitoring. Every report that spans the whole company now has to combine results from every shard.

How do caching and read replicas help before sharding?

Caching and replicas address the most common pattern in business software: far more reads than writes. The memcache paper starts from the same pattern. It says users consume an order of magnitude more content than they create, and that of several ways to reduce read traffic on its MySQL databases it chose memcache because it was the best choice given limited engineering resources and time.

The design is a look-aside cache. The application asks the cache first. On a miss it reads the database and stores the result in the cache. On a write it updates the database and deletes the cached copy, since deletes are safe to repeat. The hard part is keeping the cache correct. Meta's 2022 post says a mishandled invalidation can leave inconsistent values in the cache indefinitely. The 2013 paper describes leases to prevent a rush of requests to the database when a heavily read key is repeatedly invalidated. For one set of keys, measured over a week, the peak database query rate fell from 17 thousand per second without leases to 1.3 thousand per second with them.

A read replica is a read-only copy of the database. AWS describes using one to scale beyond a single instance for read-heavy workloads and to run business reporting away from production. Updates arrive asynchronously, so a replica can show slightly old data, which matters for screens that must show a change immediately after it is made. TAO's design accepts the same trade: its paper describes eventual consistency, with reads served from a nearby region.

Partitioning comes after both. The PostgreSQL documentation describes splitting a logical table into smaller physical pieces, gives a rule of thumb that the table should exceed the server's physical memory to benefit, and notes that dropping an old partition is far faster than a bulk delete. A time-based partition on an events table is a common, low-risk use.

What order of operations should a 20 to 200 person company follow?

Follow the sequence below and stop at the first step that solves the problem.

1. Measure. Find the slowest queries and the busiest hours, using database monitoring and application logs.
2. Index and fix queries. Most slowdowns end here.
3. Resize. A larger database instance is cheap compared with an engineering project, and relational databases typically scale up this way.
4. Cache. Cache the data that is read most and changes least, such as price lists, reference data and rendered pages.
5. Add read replicas. Send reports and search-heavy screens to replicas.
6. Partition large tables. Start with time-based tables such as logs, events and history.
7. Shard. Only when write volume or data size passes what the largest practical single server can hold, and when the shard key is clear.

Illustrative example: Consider a 150-person freight broker whose customer portal slows every month-end. The team measures first and finds that three queries account for most of the load, two of them scanning a shipments table without an index. They add the indexes, then cache the rate tables that rarely change, then point finance reports at a replica. The portal recovers. Sharding is never discussed, because the largest single database the company would realistically need is far from its limit. The expensive options were available on day one and unnecessary.

Most companies never reach step 7. Your company should copy the habit of measuring before each move and writing down what problem the move was meant to solve. Our guides to why Postgres became the default database and SQL vs NoSQL cover the database choices underneath this sequence.

How Kastling approaches scaling work

Performance and scaling work belongs to AI & Cloud Infrastructure: cloud architecture and deployment on AWS or Azure, and the data connections and pipelines behind applications. Kastling would begin by measuring how the current system behaves under real load and agreeing what success looks like, then change the least that solves the measured problem. A named person on the client side would sign off on anything consequential, such as a data move or a change to how records are stored, and AWS and Azure specialists would join where needed.