Webstack optimizations: PHP-FPM logging defaults and Apache OCSP stapling - #333
Open
markuslf wants to merge 2 commits into
Open
Webstack optimizations: PHP-FPM logging defaults and Apache OCSP stapling#333markuslf wants to merge 2 commits into
markuslf wants to merge 2 commits into
Conversation
Deploy the [global] section as a drop-in next to the pools so log_level is pinned and PHP-FPM reloads itself after ten worker crashes within a minute. error_log is deliberately not set there: RedHat reads the pool directory before its own [global] and would override it again, Debian and Fedora read it after and would not. Raise pm.start_servers and pm.min_spare_servers to 10 so an ordinary spike no longer triggers a block of "seems busy" warnings, and set request_terminate_timeout to 3900 so a worker stuck in a syscall is reclaimed. 3900 sits above the 3600 Nextcloud raises max_execution_time to (lib/OC.php), which roles/nextcloud already injects, so PHP's own limit fires first there. On Debian the pool logs move to /var/log/php-fpm/: the previous php_admin_value[error_log] under /var/log was not writable by the pool user, so the applications' errors were discarded without a trace, and slowlog resolved to /usr/log/. The directory mirrors the RedHat package (<webuser>:root 0770) and gets its own logrotate config, since Debian's covers only PHP-FPM's own error log. request_slowlog_timeout stays at 0: SELinux denies the sys_ptrace capability to httpd_t, so on an enforcing RedHat host every slow request logs "failed to ptrace(ATTACH)" at error level while the slowlog stays empty. BREAKING CHANGE: request_terminate_timeout defaults to 3900 instead of 0. Verified end to end on Rocky 8 (PHP 7.2), Fedora 44 (8.5) and Debian 13 (8.4), and in systemd containers on Rocky 9/10, Debian 12 and Ubuntu 22.04/24.04. Ubuntu 26.04 ships no php*-opcache package and is marked "-".
Let's Encrypt no longer publishes an OCSP responder, so the certificates the acme_sh role obtains carry no OCSP URI in their AIA extension. mod_ssl then logs AH02218 and AH02604 at error level per certificate and vhost on every start and reload and staples nothing; httpd starts anyway, the return value is only logged (ssl_util_stapling.c:210, ssl_engine_init.c:2381). Measured with a CA-signed leaf without AIA on httpd 2.4.62 (Rocky 9) and 2.4.68 (Debian 13): four ssl:error lines with stapling on, none with it off, clean start in both cases. The stapling cache and timeout directives stay in the template so switching the variable back on is enough for a CA that does run a responder.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two webstack defaults that no longer matched what the software actually does.
roles/php: PHP-FPM had nothing to reportThe
php-fpm-logfilecheck could only ever see pool saturation and worker crashes. Request timeouts, slow requests and the emergency reload were all switched off by the role's defaults, and on Debian the applications' PHP errors were discarded entirely.[global]section is deployed asz00-linuxfabrik-global.confnext to the pools. It pinslog_level = noticeand enables the emergency reload (emergency_restart_threshold = 10,emergency_restart_interval = 1m).error_logis deliberately not set there: RedHat reads the pool directory before its own[global]and would override it again, Debian and Fedora read it after and would not, so the same drop-in would move the log on some hosts and not on others.request_terminate_timeout0 → 3900 (breaking). A worker blocked in a syscall held its slot forever. 3900 sits five minutes above the 3600 Nextcloud raisesmax_execution_timeto (lib/OC.php:523), whichroles/nextcloudalready injects, so PHP's own limit fires first on those requests.pm.start_serversandpm.min_spare_servers5 → 10. Againstpm.max_children = 50the old values kept so small a warm reserve that an ordinary spike produced a block ofseems busywarnings, one per second, on top of the actual saturation line./var/log/php-fpm/.php_admin_value[error_log]pointed directly into/var/log(root:root 0755), which the pool user cannot write, and PHP reports that failure to a stderr PHP-FPM discards — the application errors were lost without a trace.slowlogresolved to/usr/log/$pool-slow.log. The directory mirrors the RedHat package (<webuser>:root 0770) and gets its own logrotate config under the newphp:logrotatetag, since Debian's packaged one covers only PHP-FPM's own error log.request_slowlog_timeoutstays at0. SELinux grantshttpd_tnosys_ptracecapability, and master and workers both run in that domain, so on an enforcing RedHat host every slow request logsfailed to ptrace(ATTACH)at error level while the slowlog stays empty. The denial isdontaudited, so it does not even show up in the audit log. Documented in the README.vars/Ubuntu.ymladded, per the CONTRIBUTING rule that a role withvars/Debian.ymlcarries an explicit Ubuntu file.roles/apache_httpd: stapling nothing, loudlySSLUseStaplingdefaulted toon. Let's Encrypt no longer publishes an OCSP responder, so the certificatesacme_shobtains carry no OCSP URI. mod_ssl then logsAH02218andAH02604at error level per certificate and vhost on every start and reload and staples nothing; httpd starts anyway, the return value is only logged (ssl_util_stapling.c:210,ssl_engine_init.c:2381). The default is nowoff, matching upstream. The cache and timeout directives stay in the template so switching the variable back on is enough for a CA that does run a responder.Verification
roles/phprun end to end, twice for idempotency (second runchanged=0):Identical effective configuration everywhere:
log_level = NOTICE,emergency_restart_threshold = 10,pm.start_servers = 10,request_terminate_timeout = 3900s, log directory<webuser>:root 0770, service active. The Debian fix is confirmed functionally: a request raising a PHP warning now lands in/var/log/php-fpm/www-error.log, where before neither that file nor the FPM log saw anything.roles/apache_httpdmeasured with a CA-signed leaf without AIA on httpd 2.4.62 (Rocky 9) and 2.4.68 (Debian 13): fourssl:errorlines with stapling on, none with it off, clean start in both cases.COMPATIBILITY.md updated accordingly: RHEL 10 and Ubuntu 22.04 / 24.04 move from
(x)tox.Two things deliberately left out
-. It ships PHP 8.5 under unversioned package names only and has nophp*-opcacheat all, sophp__modules__role_varfails withNo package matching 'php8.5-opcache' is available. Pre-existing, and a separate fix.roles/php/meta/argument_specs.ymlstill does not exist, so the three new variables are undeclared like the other ~40. The role has nometa/at all (as do 117 of 166 roles). A partial spec would reject the ten roles and playbooks that injectphp__*__dependent_varat role entry, so this wants its own commit.