CPU Limits & CFS Throttling.

🧠Start with the mental model

Think of a Kubernetes CPU limit as a speed limit on a highway. limits.cpu: 1 means "you may drive at most 1 core fast." Simple, intuitive... and wrong in the one way that costs you your p99.

🔍What actually happens

Linux doesn't meter speed. It meters a quota bucket that refills on a fixed period — the CFS (Completely Fair Scheduler) bandwidth controller.

cpu.cfs_period_us = 100000 (100ms, the default)

cpu.cfs_quota_us = 100000 (from limits.cpu: 1)

Every 100ms your container gets 100ms of CPU-time credit. Spend it and you are frozen — not slowed, FROZEN — until the next period starts.

⚡Where the analogy breaks (the intermediate part)

Quota is summed across ALL threads, but wall-clock is not.

Your JVM/Node/Go process on a 16-core node with limits.cpu: 1:

• 4 threads each doing 25ms of work = 100ms of quota, burned in 25ms of wall-clock.
• Remaining 75ms of that period: every thread stopped dead.

Average CPU usage graph: 30%. Reality: your request sat in a stop-the-world freeze for 75ms. That's your mystery p99 spike. Dashboards show utilization; they hide throttling.

📊The metric that tells the truth

cat /sys/fs/cgroup/cpu.stat

nr_periods 6000

nr_throttled 900 <- throttled in 15% of periods

throttled_usec 41000000 <- 41 seconds frozen

Prometheus equivalent:

rate(container_cpu_cfs_throttled_periods_total[5m])

/ rate(container_cpu_cfs_periods_total[5m])

Above ~5%? You are being throttled, whatever the CPU graph says.

🛠Three fixes that actually work

1. Set requests correctly and consider dropping the limit. requests is what the scheduler guarantees you under contention; limits only caps the ceiling. For latency-sensitive services, request accurately + no limit is a legitimate (and common) production choice — you get burst headroom back.
2. Match your runtime's thread count to the limit. This is the #1 real bug. Go: GOMAXPROCS reads the node's core count (16), not your 1-core limit → 16 threads racing to burn quota in 6ms. Use automaxprocs. Node: UV_THREADPOOL_SIZE. Java: modern JDKs read cgroup limits, old ones don't.
3. Don't go below ~500m for anything multi-threaded. Sub-core limits + parallel runtimes = constant freezing.

⚠️The trap almost everyone hits

"Throttling is high, let me raise the limit from 1 to 2." Often makes it worse: Go now sees more room to parallelize, spawns more concurrent work, and burns 200ms of quota in 12ms of wall-clock instead of 100ms in 6ms. Same freeze, bigger blast radius. Fix the thread count first, then the limit.

Rule of thumb: limits control burst behavior, requests control guaranteed share. Most people set limits carefully and requests carelessly. Do the opposite.

🎯Your exercise

Pick your busiest service. Run:

kubectl exec <pod> -- cat /sys/fs/cgroup/cpu.stat

Compute nr_throttled / nr_periods. Then check what your runtime thinks it has: for Go, log runtime.GOMAXPROCS(0); for Node, os.cpus().length. If that number is bigger than your limits.cpu, you've found a p99 bug you didn't know you had.

Question back to me: your pod has limits.cpu: 500m and one single-threaded process that needs 80ms of CPU to serve a request. What's the worst-case added latency from throttling, and why?