Backpressure.

🔧The 5 lines (Node, but language-agnostic)

1 const MAX = 20; let inflight = 0;

2 if (inflight >= MAX) return res.status(429).set("Retry-After", "2").end();

3 inflight++;

4 try { await handle(req); } finally { inflight--; }

5 // caller on 429: sleep(Retry-After + jitter), THEN retry

That's backpressure. Not a library, not a service mesh — a counter and a refusal.

🧠What each line teaches

1️⃣ MAX is a real physical limit, not a vibe. Little's Law gives it to you: L = λ × W. If you serve 200 req/s and each takes 100ms, you have 200 × 0.1 = 20 requests in flight at steady state. Set MAX near that. Above it you're not doing more work, just queueing.

2️⃣ Rejecting fast is a feature. The instinct is "queue it, don't drop it." But an unbounded queue converts an overload problem into a latency problem — and latency problems are worse, because they burn resources on work nobody is waiting for anymore.

3️⃣ The counter must be incremented on the accept path, not inside the handler. Classic bug: you count after some async setup, so 500 requests slip past the check in the same tick.

4️⃣ finally is load-bearing. Miss one decrement path (a throw, an early return, a client disconnect) and inflight leaks upward until your service permanently believes it's full. This is the #1 way a hand-rolled limiter kills a service at 3am.

5️⃣ Backpressure only works if the caller cooperates. A 429 that triggers an instant retry isn't backpressure — it's a retry storm, and you've just amplified your own load. The signal has to propagate up the chain.

📉The number that makes it click

Service does 200 req/s. Traffic spikes to 500 req/s.

• Unbounded queue: after 60s you have ~18,000 requests waiting. Wait time = 18000/200 = 90 seconds. Every client timed out at 30s long ago. You are now doing 100% of your work for nobody. Throughput of useful work: ~0.
• Bounded at 20: you serve 200 req/s perfectly and 429 the rest. Useful throughput: 200 req/s. Same hardware, same spike.

Overload doesn't reduce your capacity. Queueing does.

⚠️Where practitioners get it wrong

• Timeouts are not backpressure. A timeout gives up after paying the full cost. Backpressure refuses before paying it.
• Queue depth > CPU as an alert signal. CPU at 70% with a growing queue is a service already dying. Alert on inflight/queue depth and wait time.
• The limit belongs at every hop. Bounding your API but leaving an unbounded DB pool wait just relocates the queue.
• Kafka/SQS "handles it for you" — no. A consumer that keeps polling faster than it can process has just moved the unbounded queue into memory. Pull-based consumers get backpressure for free only if you actually stop pulling.
• Node streams already model this: writable.write() returning false means "stop, I'm full." Ignoring the return value and writing anyway is the same bug at the byte level. That's why you pipe().

🧪Your exercise

Take one service you own and answer three questions with numbers, not guesses:

1. What is its MAX concurrent in-flight requests today? (If the answer is "however many arrive" — that's your finding.)
2. What does λ × W say it should be? Grab p50 latency and current req/s from your dashboard.
3. When it's saturated, does a client get a fast 429 or a slow 504?

Bonus: hit a local endpoint with hey -c 200 -z 30s http://localhost:3000/job and watch p99 latency. If it climbs without bound instead of flattening while errors appear — you have a queue, not a limit.