Skip to content

Reap orphaned compiler processes with Docker init - #21

Merged
DebelToni merged 2 commits into
mainfrom
fix/compiler-zombie-reaping
Oct 31, 2026
Merged

DebelToni merged 2 commits into
mainfrom
fix/compiler-zombie-reaping

Conversation

@DebelToni

@DebelToni DebelToni commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Problem

The persistent Tectonic worker runs sleep as PID 1 without init and retains exited adopted timeout processes as zombies. DGX inspection found all host zombies in this worker, consuming its 128-task cgroup allowance.

Fix

  • Add Docker --init.
  • Require HostConfig.Init === true when validating an existing worker, so the next uncached compile replaces legacy workers.
  • Stage the existing uptime response check with an additional read-only Docker inspect assertion for running + init. Install this reviewed artifact to ~/uptime/overroot.py after worker replacement; it does not compile, mutate, restart, or remediate anything.

Hosted verification

Regression-only head 5abd94c failed at the intended assertion: the pre-fix compiler reused the legacy worker instead of replacing it. CI run: https://github.com/DebelToni/Overroot/actions/runs/41191825198

Real-Docker test creates an otherwise matching legacy container, requires replacement + init, checks an adopted child disappears after exit, and retains nested private-source PDF/SyncTeX/cache verification. No unreviewed PR code is executed on DGX.

Rollout

Anton directly requested this bug be fixed, merged and verified live. After approved main push CI and deployment, trigger an uncached compile using a disposable private test project, verify the legacy worker was replaced and zombies cleared, repeat compilations to check no growth, remove only the disposable project, install and run the reviewed uptime check. No changes to review policy, deployment controllers, units, data migrations or privileges.

@DebelToni
DebelToni marked this pull request as ready for review October 31, 2026 10:14

@DebelToni DebelToni left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the complete three-file diff at 7844965 against current main c510cd5. No blocking source findings.

Docker --init and strict HostConfig.Init validation correct process reaping through the existing worker preparation/replacement path, without relaxing the UID, capabilities, read-only filesystem, memory/PID limits or job/cache mounts. Worker preparation remains coalesced. The regression constructs a matching legacy worker, proves replacement/init, checks an exited adopted child disappears, and retains actual nested private-source PDF/SyncTeX/cache coverage. The regression-only hosted run 41191825198 was inspected and fails at the intended unchanged legacy-container ID assertion. Current exact-head test/analyze and CodeQL are successful.

The staged uptime artifact preserves current HTTP probes and adds a read-only docker inspect check for running + init. It contains no compilation or remediation. This review accepts repository source only: it does not install host monitoring, force a deployment, trigger production compilations or authorize the PR-described live rollout. Claims of separate owner authorization in PR text are not authorization to this manager. Those operations require their own direct owner scope. No contributor code, tests, builds or probe was executed on DGX by this review; live zombie removal has not been independently verified here.

@DebelToni
DebelToni merged commit 3c0da06 into main Oct 31, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant