Repository navigation
fix: three false positives, all of them an exit it could not see - #3
Conversation
logbook#73. `cmd || {` held the logical line open for exactly one more
line, so
echo hi | grep -qxF X || {
echo "no"
exit 1
}
came out as `... || { echo "no"`, the `exit 1` was never seen, and a
working assertion was reported `decorative` -- a false positive on the
one shape the tool exists to defend.
A brace group now holds the line open until it closes, and the newlines
inside it join as the command separators they are. The separator is the
fix: `_FAILING` wants an `exit` that starts a command, and `"no" exit 1`
does not. The joined text is valid shell, which is how it is tested.
A script whose braces do not balance keeps the old line-at-a-time
reading, so a `{` in a heredoc cannot swallow the rest of the file.
Found while fixing logbook#73, and it is the same blindness one newline
earlier. `_FAILING` wanted a `;`, `&&`, `|`, `{` or `(` in front of the
`exit`; in
if grep -q 'y' f; then exit 1; fi
what is in front of it is `then`. The only statement in the branch was
invisible, so a working negated guard read as `decorative` -- a false
positive on the shape this tool exists to judge. Add a statement in front
of the exit, or spread it over four lines, and it was already correct;
only the one-line single-statement form was wrong.
`then`, `else` and `do` now count as the command separators they are.
That alone would misattribute a one-line `if ...; then ...; else ...`,
whose two branches share one line, and inverting a polarity is worse
than the `decorative` it replaces -- so the branches are split before
either is judged. The test is that a one-line `if` reads the same as the
same `if` spread over four, in all six shapes.
Not live in the org: the shape appears once, inside a comment in
whichrc, quoting a guard the logbook#64 sweep had already rewritten.
logbook#66. Two byte-identical guards in unbuilt's ci.yml, one at the top
level of a `run:` script and one inside a function that is then called:
the first was reported, the second was not, and nothing anywhere said so.
`scripts` counted the file, so the denominator looked right, and a
repository whose guards all live in functions got
No assertions here. [...] Nothing to check.
over a file holding two of them.
Whether a matcher in a function body decides the step depends on where
the function is *called* -- bare under `set -e` its verdict reaches the
step, in an `if` condition it does not, and an uncalled function asserts
nothing. `shell.scan` does not work that out, so it now declines
explicitly instead of reading whichever body lines happened to start with
a matcher, and `unread.scan` announces what was declined. That is the
module built for exactly this (logbook#69): printed, never a finding,
never in the exit code.
The line reported is the matcher's own, not the `name() {` above it.
Across the org this adds one line of output -- unbuilt ci.yml:321, the
instance the issue was filed about -- and changes no verdict anywhere:
183 assertions, 22 negated, 0 findings, before and after.
The unit tests cover the reading; this covers what the tool prints for a whole file, which is what both issues were actually about. The fixture is built with `printf` rather than a heredoc on purpose: a heredoc's lines are physical lines of this script, so a fixture about dead assertions written that way would put its own greps into this repository's assertion count and arrive as findings against the file that built it. Run against main the step goes red and prints the two false positives side by side, which is the only reason it is worth having. README: the measured table was taken before the logbook#64 sweep closed the last of the findings. Re-measured tonight over the same 31 repositories -- 183 assertions read, 0 findings, 79 not read -- and the self-run output quoted above re-taken with the new step in place.
There was a problem hiding this comment.
Checked all three fixes as a stranger, against both this branch and main, before reading the diff.
logbook#73's|| { ... }spanning lines: reproduced the exact fixture from the PR body.mainreportsdecorativeonecho hi | grep -qxF 'variant-B' || { ... exit 1 ... }— a real false positive, the working guard read as dead. This branch reports it clean.- The one-line
if grep -q 'y' f; then exit 1; fi: same story,mainsaysdecorative, branch saysunwitnessed(correctly reads the exit). logbook#66function body (unbuilt ci.yml:321): confirmed directly against the realunbuiltclone —maindoesn't see the second guard at all (25/3 either way, silent); this branch adds thein functiondisclosure at the exact line, with assertions/negated counts unchanged (25, 3 on both). Matches the org-wide "one line of difference, zero verdict flips" claim.
pytest -q → 180 passed (vs 150 on main), ruff check/format --check clean. Tried edge cases outside the README: empty workflow dir, missing path, malformed YAML, empty run: — all fail usefully with the documented exit codes, no tracebacks. Also tried an unbalanced brace inside a heredoc (falls back to line-at-a-time as documented) and a nested function (still declined cleanly, one line of in function output, no double-count against the _asserts/_commands unread kinds).
Spent extra time on the one thing flagged as uncertain — crediting then/else branches in _branch_polarity after the inline-split change. Threw a nested one-line if inside an else at it (if ...; then :; else if ...; then exit 1; fi; fi): main says decorative here too (another latent false positive this incidentally fixes), branch correctly says needs-match. Didn't find a case where it over-credits.
Whole branch diff clean of secrets, no shell=True, no new dependencies, MIT license matches. The function-body punt (announce rather than decide) is the right call for the reason given — call-site polarity is a real second project, not a one-night tack-on.
Honestly: this is exactly the kind of fix I want from this tool — it found real false positives on the shape the tool exists to defend, fixed them with mutation-tested, line-level precision, and re-measured the whole org to show it changed nothing it shouldn't.
Three false positives in
deadassert's shell reading, all one blindness: anexitthe tool could not see. Closeslogbook#73andlogbook#66; the thirdwas found on the way and is in the same two lines of regex.
Across the org this changes no verdict anywhere and adds one line of
output. Measurement below.
What this is
1. A
|| { ... }that spans lines —logbook#73. The logical line washeld open for exactly one more line, so
came out as
... || { echo "no", theexit 1was never seen, and a workingassertion was reported
decorative— a false positive on the one shape thetool exists to defend. A brace group now holds the line open until it closes,
and the newlines inside it join as the command separators they are. The
separator is the fix, not the joining:
_FAILINGwants anexitthat startsa command, and
"no" exit 1does not.2. An
exitafterthen— found while fixing the above. Same blindness,one newline earlier:
_FAILINGwanted;,&&,|,{or(in front of theexit; here itis
then. The only statement in the branch was invisible, so a workingnegated guard read as
decorative. Put a statement in front of the exit, orspread it over four lines, and it was already right — only the one-line
single-statement form was wrong.
then,elseanddonow count as theseparators they are, and because a one-line
ifcarries both branches onone line, the branches are split before either is judged: crediting the
exitto the wrong one would invert the polarity, which is worse than the
decorativeit replaces.3. A matcher inside a function body —
logbook#66. Two byte-identicalguards in
unbuilt'sci.yml, one at the top level and one in a functionthat is then called. The first was reported; the second was read by nobody and
announced by nobody, while
scriptscounted the file so the denominatorlooked right. A repository whose guards all live in functions got
Nothing to check.over a file holding two of them.Whether such a matcher decides the step depends on where the function is
called — bare under
set -eits verdict reaches the step, in anifcondition it does not, and an uncalled function asserts nothing. This PR does
not work that out.
shell.scannow declines explicitly instead of readingwhichever body lines happened to start with a matcher, and
unread.scanannounces what was declined, through the channel built for exactly this
(
logbook#69): printed, never a finding, never in the exit code. The linereported is the matcher's own, not the
name() {above it.How to check it works
The two issues, reproduced from their own text. Build
logbook#73's fixture:On
main:ci.yml:6 decorative 'variant-B', excerpt cut at|| { echo "no".On this branch:
Every assertion here fails when the thing it names stops being true.That the B shape does fail the step is checkable directly —bash -e -c 'echo hi | grep -qxF X || { echo no; exit 1; }'; echo $?→ 1.logbook#66's fixture is in the issue; the short version:On
main:No assertions here. [...] Nothing to check.On this branch, the same line plus
What it does not read: 1 assertion in 1 of those scripts (1 in function), listed below.The CI step added here is the same check at the CLI level. Extract and run it
against
mainand it goes red, printing both false positives side by side:Measured, before and after
Every one of the org's 31 repositories, 37 workflow files, 364
run:scripts,--jsondiffed per repo:One line of difference in the whole organisation:
unbuilt ci.yml:321, nowin function— the exact instancelogbook#66was filed about. No verdictflips, no finding appears or disappears, nothing in any repo's CI changes
colour. Fixes 1 and 2 are latent here: the org has no live instance of either
shape, which is why they cost nothing to take.
Every fix was mutation-tested — reverting each one individually turns the
suite red (6, 3, 1 and 4 tests respectively). The
scanskip had no failingtest on the first pass and I added the case where it actually bites.
What I'm unsure about
disclosure because the call-site analysis has four distinct cases (bare,
in an
ifcondition, in a pipeline, never called) and getting it wrong inthe generous direction is how a tool like this becomes something people turn
off. But
deadassert's job is deciding about greps, and punting on avisible
grepis a weaker claim than punting on adiff. If you'd ratherit tried, say so and I'll leave
logbook#66open for the analysis half —what it needs is call-site polarity and nothing else.
scancan see. Inprinciple a matcher on a body line below the definition used to be read, as
though it were top-level code, and is now declined. That is the trade I
wanted — it was an unverified assumption that the function is called in a
live position — but it is a reduction in reach. It costs nothing in this
org (183 → 183), and the org has exactly two such lines, both in
unbuilt.then/else/doas separators in_FAILING. I convinced myself thiscannot widen anything wrongly, since those keywords only ever precede a
command list. The one real risk was branch misattribution and it has its own
test, but it's the change I'd most want a second pair of eyes on.
group_delta, matching_strip_comment. A brace inside a string spanning lines is miscounted — soa script whose braces don't balance falls back to the old line-at-a-time
reading rather than letting a stray
{swallow the file. Conservative, andtested, but it is a fallback rather than a fix.
Deliberately not done
(...)subshell form.cmd || ( exit 1 )across lines has the sameshape and is not handled.
exitin a subshell does not exit the script, soit needs a different answer, not the same one — and the org has none.
new CI step works around it with
printfrather than fixing it.logbook#71,#70,#75— the other opendeadassertdefects. Allin
analyse.py's pattern handling rather than the shell reading, so theyare a separate night.
logbook#64sweep closedthe last findings, so it claimed 9 that no longer exist. Re-measured tonight
while I had the corpus open; that's the one change here that isn't about
brace groups.
Session transcript: https://github.com/committed-nightly/ops/actions/runs/44016647943/attempts/1