Sizing Thread Pools with Century-Old Telecom Math: Erlang’s Loss and Delay Formulas
When software engineers sit down to configure a database connection pool, an API gateway worker pool, or an asynchronous task queue, they almost always resort to guesswork. Someone suggests setting the maximum thread count to 50 “just to be safe,” or worse, ties it arbitrarily to the physical core count of the machine.
Yet, long before multi-threaded microservices or cloud compute clusters existed, telephone engineers faced this exact resource allocation problem. In 1917, a Danish mathematician named Agner Krarup Erlang, working for the Copenhagen Telephone Company, derived a set of formulas that solved traffic congestion forever.
These equations, known as Erlang B and Erlang C, form the genuine theoretical foundation for modern capacity planning, dictating how many concurrent workers or connections you actually need to handle variable user traffic without blowing up your latency tail.
The Core Metric: Understanding “Erlangs” of Load
Before applying math to software architecture, we have to measure traffic the way teletraffic engineers do: in Erlangs (A).
One Erlang represents continuous utilization of a single resource (a phone line, an agent, or a worker thread) for an entire measurement hour.
To compute the offered load (A), you multiply your average arrival rate (λ) by your average service duration or request handle time (h):
A = λ . h
For example, if your service receives 180 requests per minute (λ = 3 req/sec) and each request takes an average of 4 seconds to process (h = 4/60 minutes, or 0.0667 minutes), your offered load is 12 Erlangs. This means you mathematically require at least 12 active workers just to break even under steady conditions. But because traffic arrives in bursts, you will need more. How many more depends on whether requests get dropped or queued.
Erlang B: The Loss Formula (Drop the Load)
The Erlang B formula models a pure loss system. Imagine calling a customer service line with a limited number of trunk lines. If all lines are busy when your call arrives, you receive a busy signal and your call is dropped immediately. It does not wait; it vanishes.
In software architecture, Erlang B directly maps to rate-limiting or load shedding architectures. If your backend worker pool is completely saturated, and you choose to immediately return a HTTP 503 Service Unavailable or drop packets rather than queue them, Erlang B calculates your exact blocking probability.
When to Use It
Use Erlang B when sizing infrastructure where buffering or queueing is either impossible or explicitly undesirable—such as real-time VoIP trunks, stateless API gateways with aggressive load shedding, or edge proxies dropping excess connections to protect downstream microservices.
Erlang C: The Delay Formula (Queue the Load)
The Erlang C formula models a delay system. Instead of dropping excess traffic when all workers are busy, incoming requests are placed into a queue and forced to wait until a worker frees up.
This is the exact mathematical model behind thread pools, database connection pools, and message queue consumers (like RabbitMQ or Kafka workers). When your application receives a spike in concurrent requests, they do not fail instantly; they enter a task queue.
However, Erlang C reveals a brutal reality: as your worker utilization creeps closer to 100%, queue wait times explode exponentially.
Comparing Capacity Planning Models
To see how these classical telecom models translate into modern software design choices, review the operational differences below:
| Model | Handling Overload | Primary Metric Computed | Software System Analogue |
| Erlang B (Loss) | Drops excess traffic immediately | Probability of Blocking (Pb) | API Gateways, Rate Limiters, Load Shedders |
| Erlang C (Delay) | Queues excess requests | Probability of Waiting (C) & Queue Delay | Thread Pools, Database Connection Pools, Worker Queues |
Visualizing Thread Pool Queue Wait Times
The chart below illustrates how queue wait times surge exponentially when a thread pool is under-provisioned relative to the incoming Erlang load.
What We Have Learned
We explored how century-old queueing mathematics developed by Agner Krarup Erlang provides the actual theoretical basis for sizing software thread pools and connection pools today. By calculating offered load in Erlangs, engineers can choose between Erlang B (for loss-based load shedding) and Erlang C (for queue-based delay systems). Moving away from guesswork and utilizing rigorous queueing theory ensures systems maintain optimal performance even during peak traffic spikes.
