The dashboard looks calm. CPU usage sits near 20%, memory is fine, and yet the p99 latency graph keeps spiking. Many teams hit this puzzle after moving a Java service into Kubernetes, and the culprit is often a CPU limit.
In this article, we explain how a CPU limit really works, why the JVM is especially exposed, and what throttling looks like in a measured experiment. Then we cover practical fixes.
What a CPU Limit Really Is
A quota per period, not a speed cap
Kubernetes enforces a CPU limit with the Linux CFS bandwidth controller. According to the kernel documentation, a group gets a quota of run-time within each period, and the cgroup exports throttling counters such as nr_throttled. In Kubernetes, the kubelet applies this through the CFS quota, with a period that defaults to 100 ms, as the GKE kubelet reference describes.
Consequently, a limit of 500m means 50 ms of CPU time in every 100 ms window. Once the container spends that budget, every thread in it is paused until the next window starts, even if the node has idle cores.
Why averages hide the problem
Monitoring tools usually average CPU over many seconds. Throttling happens in 100 ms slices. Therefore, a service can look 20% busy over a minute and still run out of quota during a burst.
Why the JVM Is Especially Exposed
Many threads, one quota
A JVM is never just one thread. Request threads, garbage collector workers, and JIT compiler threads all run at once. As Microsoft’s guidance for Java on Kubernetes points out, quotas measure CPU time, not the number of CPUs, so a multi-threaded runtime can use several processors at once even with a limit of one vCPU.
For example, with a 50 ms quota, four busy threads can consume the whole budget in about 12 ms of real time. After that, all threads freeze for the rest of the period, including the ones serving requests.
The processor count is a rounded guess
The JVM reads the container limit at startup and divides quota by period, rounding up, as the code in this OpenJDK change shows. The same Microsoft guide notes that the JVM does not adjust later, and that you can set the count yourself with -XX:ActiveProcessorCount.
As a result, a 500m limit still tells the JVM it has one processor, while the quota math allows only half of one. The JVM sizes its thread pools from that number, not from the budget.
What the Numbers Say
To see the effect directly, I ran a real experiment inside a Linux cgroup with a CFS quota. A single-threaded worker handled 282 requests that arrived randomly, about 10 per second, and each needed 20 ms of CPU. I repeated the same traffic under five quota settings.
| CPU limit | Avg CPU used | p50 | p95 | p99 | Periods throttled |
|---|---|---|---|---|---|
| None | 18.9% | 20.5 ms | 36.9 ms | 46.2 ms | no quota |
| 1.0 CPU (100 ms) | 18.9% | 20.6 ms | 39.0 ms | 54.5 ms | 0 of 247 |
| 0.5 CPU (50 ms) | 18.9% | 20.7 ms | 67.9 ms | 92.4 ms | 30 of 256 (12%) |
| 0.3 CPU (30 ms) | 19.0% | 40.9 ms | 239.0 ms | 360.1 ms | 117 of 270 (43%) |
| 0.25 CPU (25 ms) | 18.7% | 100.6 ms | 369.1 ms | 428.3 ms | 180 of 277 (65%) |
Notably, the 0.5 CPU row is the telling one. That limit gives 2.6 times more CPU than the service uses on average, yet p99 doubled and 12% of periods were throttled. At 0.25 CPU the headroom is thin, so some queueing is expected, but the trend is the same: average usage never changed, while tail latency exploded.
How to Fix It
| Fix | What it does | Trade-off |
|---|---|---|
| Raise the limit | Gives bursts room to finish inside one period | Costs more reserved CPU |
| Remove the limit, keep the request | Lets the pod use idle node CPU | Noisy neighbors can compete at busy times |
| Disable CFS quota on the node | Keeps the pod’s QoS class without enforcement | A node-wide policy decision |
| Reduce JVM parallelism | Fewer GC and compiler threads burn the quota | Slower parallel work |
Importantly, opinions differ. Some vendors, such as PerfectScale, advise avoiding CPU limits when performance matters, while others warn that unlimited containers compete during concurrent bursts. Therefore, measure your own workload before choosing.
Quick checklist
- Check the throttling counters, not just average CPU.
- Compare p99 latency with and without the limit.
- Set
-XX:ActiveProcessorCountto match the real budget. - Give the quota headroom for GC and JIT bursts.
- Decide deliberately between limits and requests only.
What We Have Learned
A Kubernetes CPU limit is a quota of CPU time per 100 ms period, not a speed limit, and it pauses every thread in the container once the budget is spent. The JVM makes this worse, because many threads share one quota and the processor count it sees is rounded up from that quota. In our experiment, average CPU stayed near 19% in every run, yet p99 grew from 46 ms to 92 ms at a 0.5 CPU limit and to over 400 ms at 0.25. So, when p99 spikes with idle CPU, check throttling first, then raise or remove the limit, and tune the JVM’s thread counts to match.
