fix(itest): egress verifier survives release-bundle --dir; distinguish "can't run" from "leak"#332
Merged
Merged
Conversation
…h "can't run" from "leak" (#274) assert_egress_posture shelled out to bench-verify-egress.sh via a repo-relative path. When the stack --dir is a release bundle (no tests/ tree, e.g. a deployed /srv/code/pithead-v1.1.0), the script isn't found, so the output lacked the OK marker and the assertion reported a false "clearnet egress leak" — really just script-not-found. This bit the real v1.1.0 release-deploy verification (41/42). - Resolve the verifier by absolute path off run.sh's $HERE in local mode (driver == box), so it runs regardless of what --dir points at. SSH mode keeps the remote-relative path ($HERE is a driver path, meaningless on the remote). - New pure egress_verdict() classifier: ok | leak | inconclusive. Output with neither the OK marker nor a leak line = verifier never produced a verdict → loud INCONCLUSIVE failure with captured stderr, not a silent egress FAIL. A privacy check that can't run must not look identical to a detected leak. - selftest: 4 asserts pinning the three verdicts apart. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…nment) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
tests/integration/run.shassert_egress_posture()shells out tobench-verify-egress.shvia a repo-relative path throughrx. In local moderxruns the snippet from the stack--dir. When--dirpoints at a release bundle (notests/tree — e.g. a deployed/srv/code/pithead-v1.1.0), the script isn't found, so the output lacks the[verify-egress] OKmarker and the assertion reports a false "clearnet egress leak" that's really just script-not-found.This bit the real v1.1.0 release-deploy verification (41/42 — the 1 fail was this artifact; egress was actually clean).
Fix
run.sh's own$HEREin local mode (driver == box), so it runs regardless of what--diris. SSH mode keeps the remote-relative path —$HEREis a driver path, meaningless on the remote (which is a full checkout anyway).egress_verdict()classifier →ok | leak | inconclusive. Output with neither the OK marker nor aLEAK/✗line means the verifier never produced a verdict → loud INCONCLUSIVE failure with captured stderr, not a silent egress FAIL. A privacy check that can't run must not look identical to a detected leak. (Kept as a hardit_fail, not a soft skip — a privacy gate that can't run shouldn't pass green.)Test
Extended
selftest.shwith 4 asserts pinning the three verdicts apart (ok/leak/inconclusivefrom script-not-found and empty output).selftest: 101 passed, 0 failed.🤖 Generated with Claude Code