Some systems grow from 100 users to a million with a few upgrades. Others need a rewrite at 10,000.
The difference is rarely clever code. It's a handful of decisions made early, usually without much thought.
Get them right and scaling is a series of small steps. Get them wrong and you pay for it in rewrites.
Here are the decisions that matter, and the order to scale in when growth comes.
The Cost of Early Mistakes
Even very large companies hit these walls:
- Twitter's "Fail Whale" era: its early monolithic application couldn't keep up with user growth, and outages were frequent until it was gradually broken into services.
- Facebook's PHP performance problem: rather than rewrite its huge PHP codebase, Facebook built its own compiler and runtime (HipHop, then HHVM) and later the Hack language to make the existing code fast enough.
Both survived because they had the money and engineers to dig out. Most teams don't. That's why the early decisions matter.
Rule 1: Know Your Numbers
Before you design anything, estimate what the system has to handle. Numbers turn opinions into decisions.
Traffic
- Daily active users: how many people use it each day?
- Requests per second: average, and at peak.
- Peak vs. average: design for the peak, not the average.
- Growth: where will these numbers be in a year?
Data
- Storage growth: how fast does data accumulate?
- Read vs. write ratio: is it mostly reads, mostly writes, or both?
- Data per user: how much does each user add?
- Retention: how long must you keep it?
Performance
- Response time: what's acceptable for users?
- Availability: how much downtime per month can you tolerate?
- Latency budget: how much of that response time can each component use?
- Throughput: how much data must move at peak?
Rule 2: Choose Your Data Foundation Carefully
Your database is the hardest thing to change later. Choose it for how your data behaves.
Relational databases (e.g. PostgreSQL)
Best when consistency and relationships matter:
- Payments and banking
- Bookings and inventory
- Anything where two records must never disagree
NoSQL databases (e.g. MongoDB, Cassandra)
Best when you need flexible schemas or very high write volume spread across many machines:
- Activity feeds
- Logging and event data
- Data whose shape changes often
If you're unsure, start with PostgreSQL. It handles far more scale than most applications ever reach, and moving off it later is easier than moving onto it.
Rule 3: Keep It Simple
A simple system is easier to understand, debug and scale. Complexity added "for later" usually costs you now and doesn't help later.
- Give each service one clear job.
- Keep dependencies between services to a minimum.
- Choose proven technology over new and exciting.
- Cut features that don't earn their keep.
Netflix is a useful example: it ran as a monolith first and moved to microservices only when its growth demanded it.
Scale in Blocks, Not Balloons
Scaling isn't stretching one big system until it bursts. It's adding the next block only when you need it, in roughly this order.
Level 1: Cache
Caching is the cheapest speed-up you'll ever get.
- Keep sessions and hot data in an in-memory cache like Redis.
- Put images and static files on a CDN, close to users.
- Cache expensive queries and computed results, with a clear expiry.
Level 2: Scale the Database
When caching isn't enough:
- Scale up first: a bigger server and better indexes go a long way.
- Add read replicas: send reads to copies, writes to the primary.
- Shard last: split data across servers only when one primary can't keep up with writes.
Level 3: Split Into Services
When one codebase becomes the bottleneck for your teams:
- Define data ownership: decide which part of the system owns each piece of data.
- Extract services along those lines, one at a time.
- Put an API gateway in front to route traffic and handle cross-cutting concerns like auth and rate limits.
Key Takeaways
- Most scaling problems are decided early, not when traffic arrives.
- Estimate traffic, data and performance numbers before you design.
- Pick your database for how your data behaves. When unsure, start with PostgreSQL.
- Keep the system simple until growth forces a change.
- Scale in order: cache, then database, then services.
Final Thoughts
Great systems aren't the ones that never break.
They're the ones built so that when growth comes, the next step is obvious and small.