NGINX: How One Event Loop Handles Thousands of Connections

Your app server is fast.

Then 5,000 users connect at once. Some are on slow mobile networks. Each one holds a connection open while their bytes trickle in.

A server that gives every connection its own thread runs out of memory long before it runs out of CPU.

NGINX was built for exactly this problem.

What a Web Server Actually Does

A web server sits between the browser and your application. It accepts connections, serves files, and passes requests to your app.

Under real traffic, a good one has to:

• Hold many connections at once, most of them idle or slow.
• Serve static files without bothering your application.
• Spread load across several application servers.
• Handle HTTPS, so your app doesn't have to.

What Makes NGINX Different

Event-Driven, Non-Blocking Design

Traditional servers often use a thread or process per connection. That's simple, but every waiting connection costs memory. Thousands of slow clients means thousands of mostly idle threads.

NGINX uses an event loop instead. One worker process watches many connections at once and only does work when a connection is ready: data has arrived, or the socket can accept more. Waiting costs almost nothing.

Master and Worker Processes

• The master process reads the configuration and manages the workers.
• Worker processes handle the actual connections, each running its own event loop.
• You usually run about one worker per CPU core (worker_processes auto;).

This is how NGINX handles thousands of concurrent connections with a small, predictable amount of memory.

NGINX as a Reverse Proxy

A reverse proxy sits in front of your application servers. Clients talk to NGINX, and NGINX talks to your app.

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://backend_server;
    }
}

That gives you one public entry point, hides your app servers from the internet, and lets you add caching, TLS and load balancing in one place.

Caching Responses

When a response is cached, NGINX serves it directly without calling your application.

proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=my_cache:10m max_size=1g inactive=60m use_temp_path=off;

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_cache my_cache;
        proxy_pass http://backend_server;
        proxy_cache_valid 200 302 10m;
        proxy_cache_valid 404 1m;
    }
}

• Lower latency: cached responses skip the application entirely.
• Lower backend load: fewer requests reach your servers.
• Better spike handling: a burst of identical requests is served from the cache.

Only cache responses that are the same for every user. Never cache pages with personal data unless the cache key includes the user.

Terminating TLS

NGINX can handle HTTPS so your application only ever sees plain HTTP on a private network.

server {
    listen 443 ssl;
    server_name example.com;

    ssl_certificate /etc/ssl/certs/example.com.pem;
    ssl_certificate_key /etc/ssl/private/example.com.key;

    location / {
        proxy_pass http://backend_server;
    }
}

Certificates live in one place, and renewing them doesn't touch your app.

Load Balancing

Put several application servers in an upstream block and NGINX spreads requests across them.

upstream backend_servers {
    server server1.example.com;
    server server2.example.com;
}

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://backend_servers;
    }
}

Strategy	How it picks a server	Directive
Round robin	Takes turns. The default.	none
Least connections	Sends to the server with the fewest active connections.	least_conn;
IP hash	Sends the same client IP to the same server.	ip_hash;

Getting Started

1. Install NGINX from your OS package manager or the official packages.
2. Start with a reverse proxy in front of one app server.
3. Add TLS, then caching, then a second upstream server.
4. Test every change with nginx -t before reloading with nginx -s reload.
5. Watch the access and error logs in your monitoring stack.

Key Takeaways

• NGINX uses an event loop, so idle and slow connections cost almost nothing.
• A master process manages workers; run roughly one worker per CPU core.
• As a reverse proxy, it gives you one entry point for caching, TLS and load balancing.
• Cache only responses that are the same for every user.
• Pick a load-balancing strategy on purpose: round robin, least connections, or IP hash.

Final Thoughts

NGINX isn't fast because of a trick.

It's fast because it refuses to spend memory on connections that are just waiting.

That one design decision is why it still sits in front of so many applications today.