Busy workers against MaxRequestWorkers is the Apache number worth alerting on, and it tells you less than you would think. Our 2-vCPU Debian 13 test server served about 4,370 requests per second to 400 clients with busy workers pinned at or near all 150. Raising the limit to 400 workers added about 4% throughput, while p99 latency got 2.6 to 4.9 times worse, because the CPU was the real ceiling.
This guide shows you how to monitor Apache with Prometheus and Grafana using apache_exporter and mod_status. It covers the scrape config, a dashboard that needs one fix on current Grafana, a hey load test with measured requests per second, and a saturation alert we fired twice. It was tested in October 2026 on Debian 13 with Apache 2.4.69, apache_exporter 1.1.1, Prometheus 3.13.4 LTS (our first load tests ran on 3.15.0), Grafana 13.2.4 and hey 0.1.4.
Monitor Apache with Prometheus: the moving parts
mod_status publishes Apache’s scoreboard at /server-status, and Debian restricts that page to requests from the server itself. That one detail decides the layout: apache_exporter has to run on the web server, read the page over localhost:80 and publish metrics on port 9117. Prometheus can live anywhere that reaches 9117, and Grafana only ever talks to Prometheus:
We ran everything on one VM named mon01 at 10.0.1.50 to keep the lab small. On real servers you would keep only Apache and the exporter on each web host and run Prometheus and Grafana elsewhere, which changes nothing in the steps below except the target address in the scrape config.
Prerequisites
apache_exporter is cheap, so the sizing question is Prometheus and Grafana. The exporter’s systemd cgroup peaked at 15 MB during a 400 client load test, and it publishes fewer than 200 metric lines per Apache server. Prometheus grows with the number of servers times retention, and it held about 65 MB on our single host. Grafana is the heavy one, at roughly 870 MB after a session of dashboard work, so you should plan its RAM first. You need the following:
- Debian 13 with sudo access and Apache 2.4 from the Debian repository
- 2 vCPU and 4 GB RAM if Prometheus and Grafana share the web server
- TCP 9117 open from the Prometheus server to each web server
- TCP 9090 and 3000 open to your admin network only
Our VM had exactly that, 2 vCPU and 4 GB of RAM, and 1.7 GiB of it was in use with everything running. Treat it as a floor for following along, not a production recommendation. If you want the PromQL and alerting theory behind what follows, our roundup of the best Grafana and Prometheus books picks the ones worth your time. Running Ubuntu instead? The Apache install guide for Ubuntu 26.04 gets you to the same starting point, although we only tested these monitoring steps on Debian.
Check mod_status on Apache
On Debian, mod_status is enabled the moment Apache is installed, with ExtendedStatus On and access limited to local requests. So the job here is to confirm it rather than configure it. Install Apache and its utilities, which include ab:
sudo apt install -y apache2 apache2-utilsLook at the status module’s config without the comments. If the file is missing on your server, run sudo a2enmod status followed by sudo systemctl restart apache2, because a2enmod only links the config and the module loads on the restart. On a fresh Debian 13 install it’s already on.
grep -v "^\s*#" /etc/apache2/mods-enabled/status.conf | grep -v "^\s*$"Require local is the line to keep. It means only requests from the machine itself see the page, so the scoreboard never leaks to the internet and the exporter must sit on the same host:
<Location /server-status>
SetHandler server-status
Require local
</Location>
ExtendedStatus On
<IfModule mod_proxy.c>
ProxyStatus On
</IfModule>Now fetch the machine-readable version the exporter uses. The ?auto suffix returns plain key and value lines instead of HTML, which is what the exporter parses on every scrape.
curl -s "http://localhost/server-status?auto" | head -22This excerpt is the top of the page. Total Accesses, Total kBytes and the CPU fields only appear with ExtendedStatus on, which is why the exporter’s request and traffic metrics depend on it:
localhost
ServerVersion: Apache/2.4.69 (Debian)
ServerMPM: event
Server Built: 2026-10-01T16:54:08
CurrentTime: Sunday, 18-Oct-2026 07:52:21 UTC
RestartTime: Sunday, 18-Oct-2026 07:42:00 UTC
ParentServerConfigGeneration: 1
ParentServerMPMGeneration: 0
ServerUptimeSeconds: 621
ServerUptime: 10 minutes 21 seconds
Load1: 0.01
Load5: 5.82
Load15: 25.08
Total Accesses: 44
Total kBytes: 33
Total Duration: 10
CPUUser: .02
CPUSystem: .01
CPUChildrenUser: 0
CPUChildrenSystem: 0
CPULoad: .00483092
Uptime: 621An empty reply or a connection error here means Apache is not running, and the exporter would report apache_up 0 for the same reason. Fix that before you install anything else, because every later step reads this page.
Install apache_exporter
Debian 13 does package the exporter as prometheus-apache-exporter, but its candidate is 1.0.9 from September 2024, while the apache_exporter releases page lists five newer releases, the latest of which rebuilds with a newer Go toolchain to pick up standard library CVE fixes. We use the upstream release. Detect the latest version, download it with its checksum file and verify before you install anything:
APACHE_EXP_VER=$(curl -s https://api.github.com/repos/Lusitaniae/apache_exporter/releases/latest | grep tag_name | cut -d '"' -f 4 | sed 's/^v//')
echo "${APACHE_EXP_VER}"
cd /tmp
curl -sLO "https://github.com/Lusitaniae/apache_exporter/releases/download/v${APACHE_EXP_VER}/apache_exporter-${APACHE_EXP_VER}.linux-amd64.tar.gz"
curl -sLO "https://github.com/Lusitaniae/apache_exporter/releases/download/v${APACHE_EXP_VER}/sha256sums.txt"
sha256sum --ignore-missing -c sha256sums.txtThe checksum line must say OK. Anything else means a truncated or tampered download, so you delete the tarball and fetch it again. The version line above it is what the API detection returned.
1.1.1
apache_exporter-1.1.1.linux-amd64.tar.gz: OKUnpack the binary, install it and give it a system user of its own, so a compromise of the exporter cannot touch Apache’s or Prometheus’s files. The exporter only needs to make HTTP requests to localhost, so the account needs no home directory and no shell.
tar xzf "apache_exporter-${APACHE_EXP_VER}.linux-amd64.tar.gz"
sudo install -m 0755 "apache_exporter-${APACHE_EXP_VER}.linux-amd64/apache_exporter" /usr/local/bin/
sudo useradd --system --no-create-home --shell /usr/sbin/nologin apache_exporter
apache_exporter --versionThe version banner also shows which Go toolchain built it. That is the line you check when a Go standard library CVE makes the news, because the fix arrives as a rebuilt exporter, not as an Apache update.
apache_exporter, version 1.1.1 (branch: HEAD, revision: d9b06987e5e047ae6aac8cc5e4eed80596ff5a4d)
build user: root@a86941370e18
build date: 20260622-11:09:09
go version: go1.25.11
platform: linux/amd64
tags: netgoCreate the systemd unit next. Older guides pass --insecure and --telemetry.endpoint=/metrics, but the first only matters for an HTTPS status URL and the second is already the default, so this unit leaves both out. It does set the listen address to 127.0.0.1, because the exporter republishes the scoreboard and Apache version with no authentication, which would undo the Require local you just checked:
sudo vim /etc/systemd/system/apache_exporter.serviceAdd the service definition. If Prometheus runs on another host, put the web server’s private address in --web.listen-address instead and allow only the Prometheus server to reach 9117 in your firewall. After=apache2.service keeps the exporter from logging scrape errors for the few seconds Apache takes to start at boot. If you put the status page on another port or behind a virtual host, you change --scrape_uri here and nothing else.
[Unit]
Description=Prometheus exporter for Apache mod_status
Wants=network-online.target
After=network-online.target apache2.service
[Service]
User=apache_exporter
Group=apache_exporter
ExecStart=/usr/local/bin/apache_exporter \
--scrape_uri=http://localhost/server-status?auto \
--web.listen-address=127.0.0.1:9117
Restart=on-failure
[Install]
WantedBy=multi-user.targetStart it and pull a few metrics straight from the exporter. The retry flags give the exporter a second to bind its port, since a plain curl run straight after the start printed nothing in our replay of this guide. Checking the exporter on its own port first means that, if Prometheus later shows the target down, you already know the exporter side works and can look at the network or the scrape config instead.
sudo systemctl daemon-reload
sudo systemctl enable --now apache_exporter
curl -s --retry 5 --retry-connrefused --retry-delay 1 http://localhost:9117/metrics | grep -E "^apache_(up|workers|version)"apache_up 1 means the exporter read mod_status. The version is encoded as a number, so 2.04069 is Apache 2.4.69, and the two worker lines are the ones the saturation alert uses later:
apache_up 1
apache_uptime_seconds_total 3
apache_version 2.04069
apache_workers{state="busy"} 1
apache_workers{state="idle"} 49Busy plus idle is 50 here, not 150. Those two series only count slots in running child processes, while apache_scoreboard also lists the 100 unused open_slot entries, and the sum of the scoreboard is MaxRequestWorkers. That difference matters when you write the alert. Confirm the exporter is only reachable locally:
sudo ss -tlnp | grep ":9117 "The listener is bound to 127.0.0.1, and in our lab a request to http://10.0.1.50:9117/metrics got connection refused:
LISTEN 0 4096 127.0.0.1:9117 0.0.0.0:* users:(("apache_exporter",pid=5913,fd=3))With the exporter answering on localhost, the web server side is done.
Install Prometheus and scrape the exporter
Debian 13 packages Prometheus 2.53, a release line that upstream stopped supporting on 31 July 2025, so we use the official tarball from the 3.13 long-term support line, which upstream supports until 31 July 2027. If you already run Prometheus, skip to the config file and add the apache job, plus the rule_files entry and the /etc/prometheus/rules directory if you don’t have them, because the alert rules later go there. The steps match our DNS monitoring guide, because both labs shared this server:
PROM_LTS="3.13" # pin: LTS line, supported until 2027-07-31 https://prometheus.io/docs/introduction/release-cycle/
PROM_VER=$(curl -s "https://api.github.com/repos/prometheus/prometheus/releases?per_page=100" | grep -oP "\"tag_name\": \"v\K${PROM_LTS}\.[0-9]+(?=\")" | head -1)
echo "${PROM_VER}"
cd /tmp
curl -sLO "https://github.com/prometheus/prometheus/releases/download/v${PROM_VER}/prometheus-${PROM_VER}.linux-amd64.tar.gz"
tar xzf "prometheus-${PROM_VER}.linux-amd64.tar.gz"
sudo useradd --system --no-create-home --shell /usr/sbin/nologin prometheus
sudo install -m 0755 "prometheus-${PROM_VER}.linux-amd64/prometheus" "prometheus-${PROM_VER}.linux-amd64/promtool" /usr/local/bin/
sudo mkdir -p /etc/prometheus/rules /var/lib/prometheus
sudo chown prometheus:prometheus /var/lib/prometheusThe echo should print a 3.13 patch release (3.13.4 when we tested). The URL is for x86_64, so use linux-arm64 on an ARM server. The systemd unit keeps 30 days of data, which is the knob that decides disk use. One Apache server adds 88 series (we counted count({job="apache"})), so retention only becomes a real cost once you scrape many servers.
sudo vim /etc/systemd/system/prometheus.serviceAdd the service definition. ExecReload lets you apply rule and target changes with systemctl reload, so you never restart Prometheus just to add a server and you avoid a gap in the graphs.
[Unit]
Description=Prometheus monitoring server
Wants=network-online.target
After=network-online.target
[Service]
User=prometheus
Group=prometheus
ExecStart=/usr/local/bin/prometheus \
--config.file=/etc/prometheus/prometheus.yml \
--storage.tsdb.path=/var/lib/prometheus \
--storage.tsdb.retention.time=30d
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
[Install]
WantedBy=multi-user.targetNow the scrape config. If Prometheus runs on a different host, replace localhost:9117 with the web server’s address, for example 10.0.1.50:9117:
sudo vim /etc/prometheus/prometheus.ymlUse this config. The rule_files glob points at the directory you just created, where the alert rules go later, and an empty directory is fine until then.
global:
scrape_interval: 15s
evaluation_interval: 15s
rule_files:
- /etc/prometheus/rules/*.yml
scrape_configs:
- job_name: prometheus
static_configs:
- targets: ['localhost:9090']
- job_name: apache
static_configs:
- targets: ['localhost:9117']Validate the file and start Prometheus. promtool does not complain about an empty rules directory, so a SUCCESS line is all you need. The curl retries until Prometheus has opened its TSDB, because a plain curl straight after the start gets connection refused:
promtool check config /etc/prometheus/prometheus.yml
sudo systemctl daemon-reload
sudo systemctl enable --now prometheus
curl -s --retry 10 --retry-connrefused --retry-delay 1 http://localhost:9090/-/readyWait one scrape interval (15 seconds), then ask which targets are up. The up query is the quickest health check you have, because Prometheus writes it for every target on every scrape:
promtool query instant http://localhost:9090 'up'Both targets report 1. A 0 for the apache job means Prometheus reached nothing on 9117, so check systemctl status apache_exporter first:
up{instance="localhost:9090", job="prometheus"} => 1 @[1792309941.716]
up{instance="localhost:9117", job="apache"} => 1 @[1792309941.716]Prometheus is now storing the Apache metrics every 15 seconds, which is all a dashboard needs.
Provision the Apache dashboard in Grafana
Install Grafana OSS from its APT repository. TLS and a reverse proxy are out of scope here, and our Grafana install guide with Nginx and SSL covers both if Grafana will face anything beyond your admin network:
sudo apt install -y apt-transport-https wget gnupg
sudo mkdir -p /etc/apt/keyrings
sudo wget -q -O /etc/apt/keyrings/grafana.asc https://apt.grafana.com/gpg-full.key
sudo chmod 644 /etc/apt/keyrings/grafana.asc
echo "deb [signed-by=/etc/apt/keyrings/grafana.asc] https://apt.grafana.com stable main" | sudo tee /etc/apt/sources.list.d/grafana.list
sudo apt update
sudo apt install -y grafanaProvision the Prometheus data source from a file with a fixed uid and name, because the dashboard fix below refers to the data source by name. Clicking it together in the UI works too, but then you have to type the exact same name, and a rebuilt server would need the clicks again.
sudo vim /etc/grafana/provisioning/datasources/prometheus.yamlAdd the data source. editable: false keeps the file as the only place where the URL lives.
apiVersion: 1
datasources:
- name: Prometheus
type: prometheus
uid: prometheus
access: proxy
url: http://localhost:9090
isDefault: true
editable: falseStart Grafana. The first start runs about 720 database migrations, which took 29 to 37 seconds in our runs, so the health endpoint answers after roughly a minute:
sudo chown root:grafana /etc/grafana/provisioning/datasources/prometheus.yaml
sudo systemctl daemon-reload
sudo systemctl enable --now grafana-serverGive it a minute, then check the health endpoint:
curl -s http://localhost:3000/api/healthA JSON reply with "database": "ok" means Grafana is up. A connection refused means it is still migrating, so wait and try again:
{
"database": "ok",
"version": "13.2.4",
"commit": "0e79aef9d91abf3bdadae9af15763029072fb45b"
}Log in at http://10.0.1.50:3000 as admin/admin and set a new password when Grafana asks, because the default login works for anyone who reaches port 3000.
Error: “Datasource ${DS_LOCALHOST} was not found”
The dashboard most guides point to, and the one the apache_exporter FAQ links, is ID 3894, published on grafana.com by rfmoz and last updated in 2022. We imported it from the Grafana UI first and mapped its DS_LOCALHOST input to Prometheus in the import form. Grafana 13 accepted the import, then every panel showed a red warning icon and “No data”, with this tooltip:
The dashboard uses the old rows-and-singlestat format (schema 14). Grafana migrated the panels, but the saved panels still pointed at the literal ${DS_LOCALHOST} instead of the data source chosen in the form, while the template variables did get the right one. The reliable fix is to replace the placeholder in the JSON yourself and load the file through provisioning, which also keeps the dashboard in version control.
Download the latest revision, swap the placeholder for the data source name, and save it where Grafana will look. The name has to match the name: in the data source file exactly, including the capital P. The second sed fixes the Uptime tile, whose singlestat is set to average its value over the time range ("valueName": "avg"), so it showed roughly half the real uptime in our first run. current shows the latest value instead, and it is the only valueName in the file:
sudo mkdir -p /var/lib/grafana/dashboards
curl -s https://grafana.com/api/dashboards/3894/revisions/latest/download | sed "s/\${DS_LOCALHOST}/Prometheus/g" | sudo tee /var/lib/grafana/dashboards/apache-3894.json >/dev/null
grep -c "DS_LOCALHOST" /var/lib/grafana/dashboards/apache-3894.json
sudo sed -i 's/"valueName": "avg"/"valueName": "current"/' /var/lib/grafana/dashboards/apache-3894.json
grep -c '"valueName": "current"' /var/lib/grafana/dashboards/apache-3894.json
sudo chown -R grafana:grafana /var/lib/grafana/dashboardsEach grep prints 1. The DS_LOCALHOST match left over is the name in the dashboard’s __inputs header, which nothing references once the panels point at Prometheus, so you can leave it alone:
1
1Grafana does not scan /var/lib/grafana/dashboards on its own, so you tell it to with a provider file. Without one, the JSON you just saved is ignored and the dashboard never appears.
sudo vim /etc/grafana/provisioning/dashboards/apache.yamlAdd the provider. allowUiUpdates: false means edits in the UI cannot drift from the file, so change the JSON on disk when you want a different panel:
apiVersion: 1
providers:
- name: apache
type: file
allowUiUpdates: false
options:
path: /var/lib/grafana/dashboardsFix the file ownership and restart Grafana so it reads the new provider. In the search that follows, replace StrongPass123 with the admin password you set at first login.
sudo chown root:grafana /etc/grafana/provisioning/dashboards/apache.yaml
sudo systemctl restart grafana-serverA restart is much faster than the first start because the migrations are done, but the API still needs a few seconds (4 in our test), and a plain curl in that window prints nothing at all. The retry flags wait it out, then run the search:
curl -s --retry 10 --retry-connrefused --retry-delay 1 -u admin:StrongPass123 "http://localhost:3000/api/search?query=Apache"The search returns the provisioned dashboard and the URL to open. An empty list ([]) means the provider file was not read, and journalctl -u grafana-server will name the file and the YAML error.
[{"id":4114857783808000,"uid":"1779c42b-993a-4bb2-b00f-c92557d1d92f","orgId":1,"title":"Apache","uri":"db/apache","url":"/d/1779c42b-993a-4bb2-b00f-c92557d1d92f/apache","slug":"","type":"dash-db","tags":[],"isStarred":false,"description":"Apache Prometheus Dashboard","sortMeta":0,"isDeleted":false}]With the placeholder replaced, every panel rendered. The screenshots in the next section were taken before we added the valueName sed, which is why their Uptime tile reads 15.5 minutes when Apache had been running for about 31: with avg, the tile averages a counter that climbs from zero across the time range. After the sed, the tile read 13.2 minutes while Prometheus reported 807 seconds (13.4 minutes) a few seconds later, which is just the lag of one scrape and refresh.
Load test Apache with hey
hey runs for a fixed time instead of a fixed request count, which gives the graphs a clean plateau. ab from apache2-utils works too, but you’d have to guess a request count that lasts long enough for Prometheus to scrape it several times. hey is in the Debian repository.
sudo apt install -y heyWe hit the default Debian welcome page (10,703 bytes) for 30 seconds at four concurrency levels, sampling BusyWorkers from mod_status every 2 seconds. Each run looked like this, with only -c changing:
hey -z 30s -c 200 http://127.0.0.1/hey ran on the same 2 vCPUs as Apache, so these are numbers for this VM, not for Apache in general. A load generator on its own machine would leave Apache more CPU and push every row higher.
| Clients (-c) | Requests/sec | p99 latency | Peak busy workers |
|---|---|---|---|
| 50 | 4,769 | 73 ms | 50 of 150 |
| 100 | 4,825 | 303 ms | 93 of 150 |
| 200 | 4,546 | 771 ms | 145 of 150 |
| 400 | 4,039 | 514 ms | 150 of 150 |
Throughput fell only 15% between 50 and 400 clients (4,769 to 4,039 requests per second), while p99 latency rose from 73 ms to several hundred. That’s the shape of a server whose CPU is already full: extra clients do not get served faster, they wait longer. The busy worker count tracks the queue, which is why it makes a good early warning even though it is not the bottleneck itself. The 400 client row has a lower p99 than the 200 client row, and we would not read much into that difference between two 30-second runs on a box this saturated.
For the alert drill we ran the same test for 4 minutes with 400 clients. hey reported 4,372 requests per second with a p99 of 607 ms, and all the responses it recorded were 200s. This excerpt is hey’s summary and latency distribution:
Summary:
Total: 240.0918 secs
Slowest: 2.6749 secs
Fastest: 0.0003 secs
Average: 0.0960 secs
Requests/sec: 4372.3609
Latency distribution:
10% in 0.0015 secs
25% in 0.0033 secs
50% in 0.0102 secs
75% in 0.1793 secs
90% in 0.2562 secs
95% in 0.2921 secs
99% in 0.6069 secshey’s status line printed [200] 1000000 responses for that run, even though 4,372 requests a second for 240 seconds is about 1.05 million. hey stores at most one million results, so treat its requests per second as the real figure and the status count as a cap. Prometheus agrees with hey, and you can check that yourself with a range query. The window below runs from 8 to 3 minutes before now, so start it about 4 minutes after hey exits and it covers the whole 4-minute plateau. We ran the 4-minute test once more for this check, and hey reported 4,710 requests per second:
promtool query range --start=$(date -d "-8 min" +%s) --end=$(date -d "-3 min" +%s) --step=60s http://localhost:9090 "rate(apache_accesses_total[1m])"The four middle samples sit between 4,669 and 4,747 requests per second, within 1% of hey’s figure. The first sample’s 1-minute window ends just as the test begins, so you should expect a value near zero there. Ours read 2,533 only because it caught the tail of the 30-second hey run we had made shortly before. The last sample falls after the load stopped:
{instance="localhost:9117", job="apache"} =>
2533.087138888889 @[1792327345]
4744.355555555555 @[1792327405]
4746.83853761529 @[1792327465]
4696.0888888888885 @[1792327525]
4668.711111111111 @[1792327585]
0.06666666666666665 @[1792327645]The dashboard shows the same runs. The accesses panel uses a 5-minute rate, so its peak of 3.87K per second is smoothed, which is worth remembering before you compare it with a load tester’s output:
Further down, the worker and scoreboard panels show what the table above describes. Both panels are stacked, so the flat line at 150 is busy plus idle. Read the colours instead: during each test the busy band fills almost all of it while idle shrinks to nearly zero, so the panel you would watch during an incident is this one, not the request rate. A flat request rate with pinned workers is the signature of a full server:
The Apache CPU load panel on the right is mod_status’s CPULoad, a figure averaged over Apache’s whole uptime, so it lags badly and should not be read as current CPU use. Use node_exporter for that.
Alert on worker saturation
Our first version of the saturation rule compared the current busy count with the scoreboard total and waited 2 minutes. It fired on the first 4-minute drill, pending at 15 seconds and firing at 138. On a second, slightly shorter run (210 seconds) it never fired. An instantaneous ratio on a load-dependent gauge is noisy, and we could not pin down from that run alone why the timer never completed, so the rule below averages busy workers over 2 minutes before comparing:
sudo vim /etc/prometheus/rules/apache.ymlAdd both rules. The denominator is the sum of apache_scoreboard, which equals MaxRequestWorkers, so the ratio stays correct when you change that setting:
groups:
- name: apache
rules:
- alert: ApacheDown
expr: apache_up == 0
for: 1m
labels:
severity: critical
annotations:
summary: "apache_exporter on {{ $labels.instance }} cannot read mod_status"
- alert: ApacheWorkersSaturated
expr: |
avg_over_time(apache_workers{state="busy"}[2m])
/ on (instance) sum by (instance) (apache_scoreboard) > 0.8
for: 1m
labels:
severity: warning
annotations:
summary: "{{ $labels.instance }} has used {{ $value | humanizePercentage }} of MaxRequestWorkers, averaged over 2 minutes"Check the rules and reload Prometheus. promtool catches PromQL typos, which matters because a broken rule loads as an error on the Rules page and then never fires.
promtool check rules /etc/prometheus/rules/apache.yml
sudo systemctl reload prometheuspromtool counts both rules. If it reports fewer, a YAML indentation slip has merged two entries, so compare your file with the block above.
Checking /etc/prometheus/rules/apache.yml
SUCCESS: 2 rules foundThen trigger it with the same 4-minute, 400 client run while you watch http://10.0.1.50:9090/alerts. Run this only on a test server or in a quiet window, because it takes every spare CPU cycle the machine has.
hey -z 240s -c 400 http://127.0.0.1/We polled the alerts API every 3 seconds. The 2-minute average takes time to climb, so this rule reacts more slowly than the first version. In exchange it does not flap, and you only get paged when the server has been near the ceiling for about three minutes:
- +108 s: ApacheWorkersSaturated pending at 0.814
- +166 s: firing at 0.944
- +258 s: cleared, 18 seconds after the load stopped
A replay of this guide on a fresh VM gave nearly the same timeline: pending at 114 seconds, firing at 175 and cleared at 249, with hey reporting 4,212 requests per second.
The firing alert carries the averaged ratio as its value, so you can see how close to the ceiling the server sat without opening Grafana.
We also tested ApacheDown by stopping Apache with sudo systemctl stop apache2. It went pending after 9 seconds and fired after 69, and the exporter’s journal showed the error you’ll see whenever mod_status is unreachable:
Oct 18 07:39:33 mon01 apache_exporter[15960]: time=2026-10-18T07:39:33.022Z level=ERROR source=collector.go:567 msg="Error scraping Apache" err="error scraping Apache: Get \"http://localhost/server-status?auto\": dial tcp [::1]:80: connect: connection refused"Start Apache again with sudo systemctl start apache2. Prometheus only evaluates the rules, so to get a page or a Slack message out of them, route them through Prometheus Alertmanager, which we did not configure for this test.
Raise MaxRequestWorkers only when the CPU has room
The obvious reaction to the saturation alert is to raise MaxRequestWorkers, and on this server that bought almost nothing. We tried it twice, and these are the steps from the second run. Back up the event MPM config first, so the revert is one command:
sudo cp /etc/apache2/mods-available/mpm_event.conf /etc/apache2/mods-available/mpm_event.conf.orig
sudo sed -i 's/^MaxRequestWorkers\s*150/ServerLimit 20\nMaxRequestWorkers 400/' /etc/apache2/mods-available/mpm_event.conf
grep -v "^\s*#" /etc/apache2/mods-available/mpm_event.conf | grep -v "^\s*$"With 25 threads per child, 400 workers need 16 processes, which is also the event MPM’s default ServerLimit. We set 20 instead, because on the event MPM the limit also has to cover child processes that are still finishing requests after a graceful restart, and an exact fit leaves no room for them. ServerLimit is only read on a full stop and start, so apply it with systemctl restart; a graceful reload ignores a changed value (see the ServerLimit documentation). The file now reads as follows. If your grep still shows MaxRequestWorkers 150, the directive is indented in your copy of the file and the sed did not match, so edit it by hand:
StartServers 2
MinSpareThreads 25
MaxSpareThreads 75
ThreadLimit 64
ThreadsPerChild 25
ServerLimit 20
MaxRequestWorkers 400
MaxConnectionsPerChild 0Check the syntax, restart Apache and count the scoreboard slots, which should match the new limit. The scoreboard is the denominator of the saturation alert, so the rule needs no edit when you change the limit:
sudo apache2ctl configtest
sudo systemctl restart apache2
curl -s "http://localhost/server-status?auto" | grep ^Scoreboard | awk '{print length($2)}'configtest warns about the missing ServerName, which is harmless here, and the scoreboard now has 400 slots:
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using fe80::1%eth0. Set the 'ServerName' directive globally to suppress this message
Syntax OK
400We then ran the same 30-second, 400 client hey test against both settings, sampling busy workers every 2 seconds:
| Run | MaxRequestWorkers | Requests/sec | p99 latency | Slowest request | Peak busy workers |
|---|---|---|---|---|---|
| First | 150 (default) | 4,039 | 514 ms | not recorded | 150 |
| First | 400 (ServerLimit 16) | 4,213 | 2.51 s | 3.53 s | 396 |
| Second | 150 (default) | 4,532 | 461 ms | 2.25 s | 146 |
| Second | 400 (ServerLimit 20) | 4,709 | 1.19 s | 4.75 s | 394 |
In both runs, 400 workers added about 4% throughput while p99 latency got 2.6 to 4.9 times worse, and the 1-minute load average climbed to 110 and then 146 on two vCPUs. More workers let more requests in at once, and every one of them competed for the same two cores. Put the original file back and restart, then confirm the scoreboard is back to 150:
sudo mv /etc/apache2/mods-available/mpm_event.conf.orig /etc/apache2/mods-available/mpm_event.conf
sudo apache2ctl configtest
sudo systemctl restart apache2
curl -s "http://localhost/server-status?auto" | grep ^Scoreboard | awk '{print length($2)}'The last line printed 150 in our lab, which means the default limits are active again.
So read the alert together with CPU before you touch the MPM settings. If busy workers are saturated and the CPU is idle, the requests are waiting on something slow (a PHP backend, a database, a proxied service) and more workers help. If the CPU is saturated too, add cores or servers instead. Our hey client shared the two vCPUs with Apache, so part of that CPU load was the test itself, which is one more reason to run the load generator on a separate machine before you size anything. To get CPU next to these panels, add node_exporter to the same Prometheus as shown in our Linux server monitoring guide, and if part of your fleet runs Nginx, the Nginx monitoring guide covers its exporter.
