test(ci): adversarial containment suite per arch/OS - #134
Open
luthermonson wants to merge 13 commits into
Open
Conversation
This was referenced Aug 7, 2026
Closed
luthermonson
added a commit
that referenced
this pull request
Aug 7, 2026
…ners (#136) Windows jobs could reach the fleet's management planes (Incus, Grafana) because installFirewallRules was a no-op on Windows and the per-endpoint HNS/VFP ACLs alone did not contain a Hyper-V-isolated job on the real runner (found by the containment suite, #134). Implement the Windows counterpart of firewall_linux.go: Windows Defender Firewall rules installed on the host, where WinNAT routes every container flow regardless of vSwitch policy. Outbound blocks cover RFC1918 + link-local with the container subnet (gateway, DNS, NAT, container-to- container) subtracted via range arithmetic — Windows Firewall has no rule ordering and Block beats Allow, so the gateway can never appear inside a blocked range. Inbound rules block container→gateway traffic on the ephemerd control ports, mirroring controlPlaneInputRules. Rules are named ephemerd-egress-* and installed delete-before-add, so reinstallation is idempotent and Cleanup removes the exact set. Install failure surfaces as an error that callers already treat as a warning, so a host without firewall privileges degrades to the endpoint ACLs instead of refusing to start. Closes #135
pkg/runtime/hardening_test.go asserts the OCI spec we ASK for; the smoke workflows only print uname. Nothing verified that containment actually holds on real hardware -- which is where an AppArmor profile that failed to load, a knob an older daemon parsed and ignored, or a node missing a release silently differs from the spec. Each step asserts the SAFE outcome and exits non-zero on the unsafe one (probes, not exploits). Linux x64+arm64 cover capabilities via CapEff, seccomp filter mode, AppArmor confinement, host-filesystem and ephemerd-credential reachability, /proc + /sys + cgroup writability, raw devices and mknod, privileged-container refusal, dind bind-mount translation, and firewall isolation from the Proxmox/Incus/Grafana management planes. Windows asserts Hyper-V isolation and host config unreachability; macOS asserts the job is in a VM rather than on the bare mini. Runs on dispatch, on changes to itself, and weekly so drift is caught without a release. Intended to run after every fleet uplift as the evidence a release's security changes reached the metal.
The Proxmox probes used a guessed 192.168.11.10, which resolves to nothing — a probe that always passes is worse than no probe, because it reads as coverage. coyotes is 192.168.5.1 and kings 192.168.5.2; both are now probed on 8006.
The existing linux job checks doors are locked; this adds a breakout job that rattles the handles — each step runs a concrete escape and fails if it WORKS. Aimed at what just shipped: - no_new_privs (#128): a setuid-root helper must not reach euid 0 - AppArmor (#124): mount(2) and sysrq-trigger writes must be DENIED, not merely that a profile is attached - cgroup release_agent: the canonical escape, incl. mounting a fresh hierarchy to get a writable one - core_pattern: |host-binary crash handler must be unwritable - user-namespace cap regain: unshare -Ur must not yield a working mount - /dev/kmsg + dmesg: host kernel log must not leak - containerd socket: must not be reachable from a job Separate job so a regression reads as 'an escape opened' rather than 'a policy check drifted'. x64 + arm64.
luthermonson
force-pushed
the
test/containment-suite
branch
from
August 8, 2026 19:17
ece62bd to
8fe7736
Compare
A private PID namespace (the correct case) makes /proc/1/root the container's OWN rootfs — traversable and harmless. The probe flagged that as an escape. Check for a host marker file VIA the link instead, so it fires only when the PID namespace is actually shared with the host.
Contributor
Author
|
Review: this PR is superseded on main. Commit fc9590e (2026-08-15, "consolidate containment suite") plus follow-ups 9a6c07b/8ce29d9 landed a 1238-line .github/workflows/containment.yml that is a strict superset of this PR's linux, breakout and windows jobs — it adds the Windows NAT/L2Bridge egress-mode handling, the expected-to-fail regression net (#126/#102/#130), userns/cgroup checks, and per-job summaries. Merging this branch would regress that file to the older 459-line version, and it is currently CONFLICTING with main for exactly that reason. The failing Containment checks on this PR ran the old suite and are not meaningful now. One thing here is NOT on main: the macos job (VM-not-bare-host check, host credential reachability, management-plane reachability). If that coverage is still wanted, rebase this PR down to only adding the macos job on top of main's consolidated file (matching its helper/summary conventions); otherwise this can be closed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gap: nothing verifies containment on real hardware. pkg/runtime/hardening_test.go inspects the OCI spec we ask for; the three smoke workflows print \uname -a. The spec and reality diverge exactly when it matters — an AppArmor profile that failed to load, a knob an older daemon parsed and ignored, a node that missed a release.
What it does: every step asserts the SAFE outcome and exits non-zero on the unsafe one. Probes, not exploits — each checks the door is locked rather than walking through it.
Linux (x64 + arm64): dangerous capabilities absent from CapEff (the kernel's effective set, not the request); seccomp in filter mode; AppArmor confined not unconfined; host ephemerd config/PEM/containerd socket unreadable; /proc/1/root not traversable; /proc/sys, sysrq-trigger, uevent_helper, cgroup release_agent not writable; raw devices and mknod denied; \docker run --privileged\ refused; -v /:/hostroot\ cannot read the host config; Proxmox/Incus/Grafana unreachable through the container firewall; /actions-runner read-only.
Windows: Hyper-V isolation active (not process isolation), host ephemerd files unreachable, management planes blocked. macOS: job is inside a per-job VM rather than the bare mini, host credentials unreachable.
When to run: after every fleet uplift, as the evidence that a release's security changes reached the metal — plus weekly, so drift is caught without a release. The highest-value single check is host \config.toml\ readability: it carries the GitHub PAT and the cloudflared tunnel token.