The surprising part is not Postgres

OpenAI's PostgreSQL post sounds surprising at first:

one primary
nearly 50 read replicas
millions of queries per second
800 million ChatGPT users

That is a huge number.

But the useful lesson is not:

Postgres can handle anything.

The useful lesson is:

Postgres can go very far when the workload is mostly reads,
the primary is protected,
and every overload path has a brake.

The primary database is not just a storage box. It is the narrowest coordination point in the system.

OpenAI did not scale PostgreSQL by finding one magic switch. They found each kind of pressure on the primary and gave it somewhere else to go.

That is the practical model.

Do not ask "how do we scale Postgres?"

Ask:

Which pressure is hurting us?

read QPS?
write QPS?
connection count?
CPU from bad queries?
cache miss storms?
replica lag?
schema migrations?
retry storms?

Each one has a different fix.

Why the primary is special

A single-primary PostgreSQL setup has one node that accepts writes.

Read replicas can help with reads, but every write still has to pass through the primary. That matters because writes create more work than they look like.

In PostgreSQL, an update does not usually rewrite a field in place. Because of MVCC, PostgreSQL creates a new row version. Old versions remain until vacuum can clean them up.

So a small update can create:

new row version
index maintenance
WAL records
replication work
future vacuum work
dead tuples that reads may step around

write amplificationone small UPDATE is never just one write

That is why write storms are dangerous.

A product launch, a bad retry loop, or a backfill can turn into more than "extra writes." It can become table bloat, index bloat, replica lag, CPU burn, and slow reads.

Write pressure does not stay in the write path. It leaks into reads, replication, vacuum, and failover risk.

So where did the pressure go?

The architecture itself stays conceptually simple:

clients
  -> application services
  -> PgBouncer
  -> PostgreSQL primary for writes
  -> read replicas for reads

One primary still takes every write. Nearly 50 read replicas absorb reads across regions. Over the past year that load grew more than 10x, and it still runs at low double-digit millisecond p99 latency with five-nines availability.

The interesting part is not the diagram. It is the operational discipline around it: every path that could overload the primary has a brake, and every workload that does not truly belong on the primary has been moved off it.

So rather than list everything they did up front, the rest of this article walks the pressures one at a time. For each one: the problem that shows up first, what OpenAI did about it, and what you can do in a smaller system without an OpenAI-sized team.

Scaling a database is often less about making one database bigger, and more about deciding what no longer belongs in it.

Practical approach 1: reduce writes before scaling writes

Before adding Kafka, sharding, or a new database, look for writes that should not exist.

Common examples:

updating last_seen_at on every request
rewriting the same JSON blob with identical data
saving user preferences even when nothing changed
writing analytics events into the OLTP database
retrying a write without an idempotency key
running backfills with no rate limit

The cheapest write is the one you do not send.

For state that changes frequently but does not need exact per-request freshness, use lazy writes:

user makes 100 requests in 5 minutes
  -> keep recent activity in cache or memory
  -> flush one summarized update later

This is the lazy write, sometimes called write-behind: hold the change in memory or cache and persist a single summarized update later, so the database sees one write instead of a hundred.

This is exactly what OpenAI did. They buffered high-frequency updates in Redis or in-memory application caches and let a background process flush them to Postgres about once per minute. A hundred updates collapse into one, which is roughly a 99% cut in write load, and they repeated the pattern across dozens of hot counters. They also went looking for application bugs that were issuing redundant writes and removed them. A field like last_seen_at does not need a row write on every request.

For fields like counters, timestamps, rankings, or usage metrics, ask whether the product needs every intermediate state in Postgres.

Often the answer is no.

Need exact payment status? Write it synchronously.
Need approximate activity freshness? Buffer it.
Need analytics? Send it to an event pipeline.
Need an audit trail? Append to a log-like table or external stream.

Not all writes have the same correctness requirement. Treating them the same is how the primary gets overloaded.

Practical approach 2: batch writes instead of sending them one by one

OpenAI's playbook leans on doing fewer, cheaper writes, and batching is the most direct version of that. When many small writes do not all need to hit Postgres immediately, collect them and write them together.

A common way to implement this is to put a durable queue such as Kafka between the application and Postgres:

application
  -> durable queue (e.g. Kafka)
  -> consumer group
  -> batches
  -> PostgreSQL

Instead of 10,000 web requests each opening their own small write, consumers group records by time, size, or key.

flush when:
  batch has 1,000 records
  or 50 ms passed
  or the partition is shutting down

Then Postgres receives fewer, larger operations.

batch the writesmany small writes become few large ones

For inserts, that might be:

insert into events (event_id, user_id, action, created_at)
values
  ($1, $2, $3, $4),
  ($5, $6, $7, $8),
  ...
on conflict (event_id) do nothing;

For very large ingest, use COPY instead of row-by-row INSERT. PostgreSQL's own docs recommend COPY for loading many rows because it has much lower overhead than a series of inserts.

The important parts are not the code. They are the contracts:

Can this write be delayed?
What is the maximum acceptable delay?
What key makes retries idempotent?
What happens if the consumer crashes after writing but before committing Kafka offsets?
Can the batch be replayed safely?

Kafka gives you a buffer and replay. It does not automatically give you correct side effects.

A Kafka-backed writer must make the database write idempotent, because replay is part of the design.

That usually means unique keys and ON CONFLICT.

create table user_events (
  event_id text primary key,
  user_id text not null,
  action text not null,
  created_at timestamptz not null
);

Then replay is safe:

insert into user_events (event_id, user_id, action, created_at)
values ($1, $2, $3, $4)
on conflict (event_id) do nothing;

The batch is now fewer, larger, replay-safe writes instead of one fragile write per event.

Practical approach 3: move reads away from the writer

OpenAI leaned heavily on read replicas because the workload was read-heavy. They run nearly 50 replicas across regions while keeping replication lag close to zero, so most reads never touch the primary.

But replicas are not free for the primary. Every replica streams the write-ahead log from the primary, so one primary fanning out to dozens of replicas spends real CPU and network just shipping changes. Past a point, adding a replica adds load to the very node you are trying to protect.

OpenAI's answer is cascading replication. A few replicas read the WAL directly from the primary, and the rest read from those relays instead. The primary streams to a handful of nodes, and the replica tier scales out behind them.

WAL fan-out vs cascadereplicas are not free for the primary

Routing reads to replicas works when the application can tolerate replica lag.

Good candidates:

profile reads
settings reads
history pages
search-ish lookup pages
metadata reads
admin views
recommendation side reads

Bad candidates:

read your own write
payment state transitions
inventory reservations
permission checks immediately after a change
anything inside the same write transaction

A practical routing rule is:

primary:
  writes
  transactions that require fresh reads
  read-after-write paths

replicas:
  normal product reads
  low-priority reads
  batch reads
  admin/reporting reads, if they cannot go to a warehouse

route reads off the writerfresh paths hit the primary, the bulk of reads go to replicas

Then make the application honest about staleness.

If a user updates a setting and immediately reloads the page, you can either read from primary for a short period or write through the updated value to cache.

Read replicas scale reads only when the product has a clear staleness policy.

Practical approach 4: protect the cache boundary

Caching can save Postgres. A broken cache can hurt Postgres.

The failure pattern is:

hot key expires
10,000 requests miss cache
10,000 requests query Postgres
Postgres slows down
requests time out
clients retry
Postgres gets even more traffic

cache stampedeone missing key vs one database incident

OpenAI attacked cache-miss storms from both sides: cache locking or leasing so one request repopulates a missed key while the rest wait, plus moving heavier computation into the application layer and enforcing stricter query timeouts so a miss cannot turn into an unbounded primary query.

The shape is:

cache miss for key K
  -> try lock K
  -> winner queries Postgres and fills cache
  -> others wait, then read cache

This is also called request coalescing or stampede protection.

Useful additions:

jitter TTLs so many keys do not expire together
serve stale data while refreshing
negative-cache missing records briefly
rate-limit cache bypass paths
use small per-key locks, not one global lock

A cache without stampede protection can turn one missing key into a database incident.

Practical approach 5: pool connections

At scale, connection count becomes its own failure mode.

Every application instance wants a pool. Every worker wants a pool. Every deploy can temporarily double the number of clients. If the database has a hard connection limit, a connection storm can break the service before query CPU is the bottleneck.

PgBouncer sits between clients and Postgres:

many client connections
  -> PgBouncer
  -> fewer server connections
  -> PostgreSQL

connection poolingmany clients, few server connections

OpenAI ran PgBouncer in transaction-pooling mode in front of its instances. That absorbed connection spikes and dropped connection setup time from around 50 ms to roughly 5 ms.

The practical caution is that pooling mode matters.

Transaction pooling is efficient, but some session-level features become risky:

session variables
temporary tables
prepared statements, depending on configuration
LISTEN / NOTIFY
advisory locks tied to sessions

So connection pooling is not just an infra toggle. The app must use database sessions in a pool-friendly way.

Practical approach 6: isolate workloads

One bad query should not slow the whole product.

OpenAI separated high-priority and low-priority workloads. You can do the same even at a smaller scale.

primary user path
  -> high-priority pool
  -> high-priority replica

admin exports, backfills, dashboards
  -> low-priority pool
  -> low-priority replica

Also isolate by timeout:

checkout request: 100 ms query budget
profile page: 300 ms query budget
admin export: async job
backfill: strict rows-per-second limit

And isolate by query shape.

If an ORM generates a 12-table join on the hot path, that query needs to be rewritten, moved off the hot path, precomputed, or blocked during an incident.

Workload isolation is how you stop low-value work from spending high-value database capacity.

Practical approach 7: rate-limit writes, backfills, and retries

Many database outages are not caused by normal traffic.

They are caused by bursts:

new feature launch
retry storm
bad deploy
cache miss storm
backfill
schema migration
bulk import

So add brakes where bursts begin.

For API writes:

per-user rate limits
per-endpoint rate limits
idempotency keys
bounded retries with jitter

For workers:

max concurrency
max rows per second
max batch size
sleep between chunks
pause switch

For backfills:

update accounts
set plan_tier = 'free'
where id > $last_id
order by id
limit 1000;

Then commit, sleep, observe metrics, and continue.

A backfill that takes a week without hurting production is better than a backfill that takes an hour and creates an outage.

Practical approach 8: be careful with schema changes

Schema changes look small in code review.

alter table users alter column status type text;

But some changes can rewrite the whole table, block writes, rebuild indexes, or generate huge WAL.

The safe pattern is expand and contract:

1. add new nullable column
2. deploy code that writes both old and new
3. backfill slowly
4. deploy code that reads new
5. stop writing old
6. drop old later

Use short lock timeouts. Create indexes concurrently. Avoid table rewrites on hot tables. Test migrations on production-sized data, not only local data.

OpenAI used strict timeouts and rate limits around schema changes and backfills for the same reason.

Schema migration safety is database scaling work.

When to shard Postgres

Sharding is tempting because it sounds final.

split users by user_id
put each shard on a different primary
now writes scale

That can work.

But sharding changes the application.

You now need answers for:

How do we choose the shard key?
Can one request touch multiple shards?
How do we enforce uniqueness across shards?
How do transactions work across shards?
How do we rebalance hot shards?
How do migrations run across all shards?
How do support tools find a user's data?
How do analytics queries work?

OpenAI did not rush to shard the existing PostgreSQL deployment because changing hundreds of application endpoints would be slow and risky. Instead, they kept the existing single-primary system running and moved shardable, write-heavy workloads to a sharded system, Azure Cosmos DB. They also stopped the problem from growing by disallowing new tables in the existing PostgreSQL deployment, so new write-heavy features had to live somewhere built to shard from the start.

That is a useful lesson.

Shard when the data model has a clean partition key and the operational cost is lower than protecting the primary another way.

For many teams, the right sequence is:

1. remove unnecessary writes
2. add idempotency
3. batch async writes
4. move reads to replicas
5. protect caches
6. pool connections
7. isolate workloads
8. move analytics out
9. move shardable write-heavy features out
10. shard core Postgres only when the model forces it

A practical decision table

Use this as the first-pass map.

Problem: too many reads
Try: caching, read replicas, query indexes, request coalescing

Problem: too many connections
Try: PgBouncer, smaller app pools, transaction pooling, regional poolers

Problem: cache misses overload DB
Try: cache locks, stale-while-revalidate, TTL jitter, negative caching

Problem: too many small writes
Try: Kafka buffer, batch inserts, COPY, idempotency keys

Problem: counters or metrics overload DB
Try: lazy writes, buffer in cache and flush periodically, move raw events to a warehouse

Problem: write-heavy feature does not need relational joins
Try: sharded store, key-value store, event log, document store

Problem: ORM creates expensive joins
Try: inspect SQL, add indexes, split query, precompute, block bad query digest

Problem: backfill hurts production
Try: chunking, rows-per-second limit, pause switch, off-peak scheduling

Problem: replica lag grows
Try: reduce writes, reduce replica count pressure, cascading replication, regional routing

Problem: one bad workload hurts everyone
Try: workload isolation, separate pools, separate replicas, strict timeouts

The real takeaway

OpenAI's story is impressive because it shows how far a boring architecture can go when it is operated with discipline.

But the discipline is the point.

They did not ask one PostgreSQL primary to absorb every workload forever. They protected it. They moved reads away. They smoothed spikes. They blocked expensive queries. They pooled connections. They rate-limited dangerous work. They migrated write-heavy shardable workloads out.

That is the useful pattern for most teams.

Scale Postgres by making the primary do only the work that truly belongs on the primary.

Once you see the system that way, lazy writes, batching, read replicas, cascading replication, PgBouncer, cache locks, workload isolation, and sharded stores stop being random tools.

They become ways to move pressure away from the narrowest point.

Sources