Skip to content

Webstack optimizations: PHP-FPM logging defaults and Apache OCSP stapling - #333

Open
markuslf wants to merge 2 commits into
mainfrom
feat/webstack-optimizations
Open

Webstack optimizations: PHP-FPM logging defaults and Apache OCSP stapling#333
markuslf wants to merge 2 commits into
mainfrom
feat/webstack-optimizations

Conversation

@markuslf

Copy link
Copy Markdown
Member

Two webstack defaults that no longer matched what the software actually does.

roles/php: PHP-FPM had nothing to report

The php-fpm-logfile check could only ever see pool saturation and worker crashes. Request timeouts, slow requests and the emergency reload were all switched off by the role's defaults, and on Debian the applications' PHP errors were discarded entirely.

  • The [global] section is deployed as z00-linuxfabrik-global.conf next to the pools. It pins log_level = notice and enables the emergency reload (emergency_restart_threshold = 10, emergency_restart_interval = 1m). error_log is deliberately not set there: RedHat reads the pool directory before its own [global] and would override it again, Debian and Fedora read it after and would not, so the same drop-in would move the log on some hosts and not on others.
  • request_terminate_timeout 0 → 3900 (breaking). A worker blocked in a syscall held its slot forever. 3900 sits five minutes above the 3600 Nextcloud raises max_execution_time to (lib/OC.php:523), which roles/nextcloud already injects, so PHP's own limit fires first on those requests.
  • pm.start_servers and pm.min_spare_servers 5 → 10. Against pm.max_children = 50 the old values kept so small a warm reserve that an ordinary spike produced a block of seems busy warnings, one per second, on top of the actual saturation line.
  • Debian pool logs move to /var/log/php-fpm/. php_admin_value[error_log] pointed directly into /var/log (root:root 0755), which the pool user cannot write, and PHP reports that failure to a stderr PHP-FPM discards — the application errors were lost without a trace. slowlog resolved to /usr/log/$pool-slow.log. The directory mirrors the RedHat package (<webuser>:root 0770) and gets its own logrotate config under the new php:logrotate tag, since Debian's packaged one covers only PHP-FPM's own error log.
  • request_slowlog_timeout stays at 0. SELinux grants httpd_t no sys_ptrace capability, and master and workers both run in that domain, so on an enforcing RedHat host every slow request logs failed to ptrace(ATTACH) at error level while the slowlog stays empty. The denial is dontaudited, so it does not even show up in the audit log. Documented in the README.
  • vars/Ubuntu.yml added, per the CONTRIBUTING rule that a role with vars/Debian.yml carries an explicit Ubuntu file.

roles/apache_httpd: stapling nothing, loudly

SSLUseStapling defaulted to on. Let's Encrypt no longer publishes an OCSP responder, so the certificates acme_sh obtains carry no OCSP URI. mod_ssl then logs AH02218 and AH02604 at error level per certificate and vhost on every start and reload and staples nothing; httpd starts anyway, the return value is only logged (ssl_util_stapling.c:210, ssl_engine_init.c:2381). The default is now off, matching upstream. The cache and timeout directives stay in the template so switching the variable back on is enough for a CA that does run a responder.

Verification

roles/php run end to end, twice for idempotency (second run changed=0):

OS how result
RHEL 8 (Rocky 8, PHP 7.2) KVM guest ok
RHEL 9 (Rocky 9, 8.0) systemd container ok
RHEL 10 (Rocky 10, 8.4) systemd container ok
Fedora 44 (8.5) KVM guest ok
Debian 12 (8.2) systemd container ok
Debian 13 (8.4) KVM guest ok
Ubuntu 22.04 (8.1) systemd container ok
Ubuntu 24.04 (8.3) systemd container ok
Ubuntu 26.04 (8.5) systemd container fails

Identical effective configuration everywhere: log_level = NOTICE, emergency_restart_threshold = 10, pm.start_servers = 10, request_terminate_timeout = 3900s, log directory <webuser>:root 0770, service active. The Debian fix is confirmed functionally: a request raising a PHP warning now lands in /var/log/php-fpm/www-error.log, where before neither that file nor the FPM log saw anything.

roles/apache_httpd measured with a CA-signed leaf without AIA on httpd 2.4.62 (Rocky 9) and 2.4.68 (Debian 13): four ssl:error lines with stapling on, none with it off, clean start in both cases.

COMPATIBILITY.md updated accordingly: RHEL 10 and Ubuntu 22.04 / 24.04 move from (x) to x.

Two things deliberately left out

  • Ubuntu 26.04 is marked -. It ships PHP 8.5 under unversioned package names only and has no php*-opcache at all, so php__modules__role_var fails with No package matching 'php8.5-opcache' is available. Pre-existing, and a separate fix.
  • roles/php/meta/argument_specs.yml still does not exist, so the three new variables are undeclared like the other ~40. The role has no meta/ at all (as do 117 of 166 roles). A partial spec would reject the ten roles and playbooks that inject php__*__dependent_var at role entry, so this wants its own commit.

Deploy the [global] section as a drop-in next to the pools so log_level is
pinned and PHP-FPM reloads itself after ten worker crashes within a minute.
error_log is deliberately not set there: RedHat reads the pool directory
before its own [global] and would override it again, Debian and Fedora read it
after and would not.

Raise pm.start_servers and pm.min_spare_servers to 10 so an ordinary spike no
longer triggers a block of "seems busy" warnings, and set
request_terminate_timeout to 3900 so a worker stuck in a syscall is reclaimed.
3900 sits above the 3600 Nextcloud raises max_execution_time to (lib/OC.php),
which roles/nextcloud already injects, so PHP's own limit fires first there.

On Debian the pool logs move to /var/log/php-fpm/: the previous
php_admin_value[error_log] under /var/log was not writable by the pool user, so
the applications' errors were discarded without a trace, and slowlog resolved
to /usr/log/. The directory mirrors the RedHat package (<webuser>:root 0770)
and gets its own logrotate config, since Debian's covers only PHP-FPM's own
error log.

request_slowlog_timeout stays at 0: SELinux denies the sys_ptrace capability to
httpd_t, so on an enforcing RedHat host every slow request logs
"failed to ptrace(ATTACH)" at error level while the slowlog stays empty.

BREAKING CHANGE: request_terminate_timeout defaults to 3900 instead of 0.

Verified end to end on Rocky 8 (PHP 7.2), Fedora 44 (8.5) and Debian 13 (8.4),
and in systemd containers on Rocky 9/10, Debian 12 and Ubuntu 22.04/24.04.
Ubuntu 26.04 ships no php*-opcache package and is marked "-".
Let's Encrypt no longer publishes an OCSP responder, so the certificates the
acme_sh role obtains carry no OCSP URI in their AIA extension. mod_ssl then
logs AH02218 and AH02604 at error level per certificate and vhost on every
start and reload and staples nothing; httpd starts anyway, the return value is
only logged (ssl_util_stapling.c:210, ssl_engine_init.c:2381).

Measured with a CA-signed leaf without AIA on httpd 2.4.62 (Rocky 9) and
2.4.68 (Debian 13): four ssl:error lines with stapling on, none with it off,
clean start in both cases. The stapling cache and timeout directives stay in
the template so switching the variable back on is enough for a CA that does
run a responder.
@markuslf
markuslf requested a review from NavidSassan August 28, 2026 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant