Skip to content

fix: three false positives, all of them an exit it could not see - #3

Merged
jennifer-barber[bot] merged 4 commits into
mainfrom
shift/2026-11-17-brace-groups
Nov 17, 2026
Merged

jennifer-barber[bot] merged 4 commits into
mainfrom
shift/2026-11-17-brace-groups

Conversation

@richmond-avenal

Copy link
Copy Markdown
Contributor

Three false positives in deadassert's shell reading, all one blindness: an
exit the tool could not see. Closes logbook#73 and logbook#66; the third
was found on the way and is in the same two lines of regex.

Across the org this changes no verdict anywhere and adds one line of
output. Measurement below.

What this is

1. A || { ... } that spans lines — logbook#73. The logical line was
held open for exactly one more line, so

echo hi | grep -qxF 'variant-B' || {
    echo "no"
    exit 1
  }

came out as ... || { echo "no", the exit 1 was never seen, and a working
assertion was reported decorative — a false positive on the one shape the
tool exists to defend. A brace group now holds the line open until it closes,
and the newlines inside it join as the command separators they are. The
separator is the fix, not the joining: _FAILING wants an exit that starts
a command, and "no" exit 1 does not.

2. An exit after then — found while fixing the above. Same blindness,
one newline earlier:

if grep -q 'y' f; then exit 1; fi

_FAILING wanted ;, &&, |, { or ( in front of the exit; here it
is then. The only statement in the branch was invisible, so a working
negated guard read as decorative. Put a statement in front of the exit, or
spread it over four lines, and it was already right — only the one-line
single-statement form was wrong. then, else and do now count as the
separators they are, and because a one-line if carries both branches on
one line, the branches are split before either is judged: crediting the exit
to the wrong one would invert the polarity, which is worse than the
decorative it replaces.

3. A matcher inside a function body — logbook#66. Two byte-identical
guards in unbuilt's ci.yml, one at the top level and one in a function
that is then called. The first was reported; the second was read by nobody and
announced by nobody, while scripts counted the file so the denominator
looked right. A repository whose guards all live in functions got
Nothing to check. over a file holding two of them.

Whether such a matcher decides the step depends on where the function is
called — bare under set -e its verdict reaches the step, in an if
condition it does not, and an uncalled function asserts nothing. This PR does
not work that out. shell.scan now declines explicitly instead of reading
whichever body lines happened to start with a matcher, and unread.scan
announces what was declined, through the channel built for exactly this
(logbook#69): printed, never a finding, never in the exit code. The line
reported is the matcher's own, not the name() { above it.

How to check it works

git clone -b shift/2026-11-17-brace-groups https://github.com/committed-nightly/deadassert
cd deadassert && python3 -m venv .venv && .venv/bin/pip install -e '.[dev]'
export PATH="$PWD/.venv/bin:$PATH"

pytest -q          # 180 passed (150 on main)
ruff check src tests && ruff format --check src tests

The two issues, reproduced from their own text. Build logbook#73's fixture:

mkdir -p /tmp/r73/.github/workflows && cd /tmp/r73
printf '%s\n' "'on': push" "jobs:" "  j:" "    steps:" "      - run: |" \
  "          echo hi | grep -qxF 'variant-B' || {" "            echo \"no\"" \
  "            exit 1" "          }" > .github/workflows/ci.yml
deadassert .

On main: ci.yml:6 decorative 'variant-B', excerpt cut at || { echo "no".
On this branch: Every assertion here fails when the thing it names stops being true. That the B shape does fail the step is checkable directly —
bash -e -c 'echo hi | grep -qxF X || { echo no; exit 1; }'; echo $? → 1.

logbook#66's fixture is in the issue; the short version:

mkdir -p /tmp/r66/.github/workflows && cd /tmp/r66
printf '%s\n' "'on': push" "jobs:" "  j:" "    steps:" "      - run: |" \
  "          guard() {" "            grep -qF 'never-happens' out.txt" \
  "          }" "          guard" > .github/workflows/ci.yml
deadassert .

On main: No assertions here. [...] Nothing to check.
On this branch, the same line plus What it does not read: 1 assertion in 1 of those scripts (1 in function), listed below.

The CI step added here is the same check at the CLI level. Extract and run it
against main and it goes red, printing both false positives side by side:

ci.yml:7   decorative  'ok'     echo hi | grep -qxF 'ok' out.txt || { echo 'absent'
ci.yml:11  decorative  'ok'     if grep -qxF 'ok' out.txt; then exit 1; fi

Measured, before and after

Every one of the org's 31 repositories, 37 workflow files, 364 run: scripts,
--json diffed per repo:

main this branch
assertions read 183 183
negated 22 22
findings 0 0
not decided 28 28
not read 78 79

One line of difference in the whole organisation: unbuilt ci.yml:321, now
in function — the exact instance logbook#66 was filed about. No verdict
flips, no finding appears or disappears, nothing in any repo's CI changes
colour. Fixes 1 and 2 are latent here: the org has no live instance of either
shape, which is why they cost nothing to take.

Every fix was mutation-tested — reverting each one individually turns the
suite red (6, 3, 1 and 4 tests respectively). The scan skip had no failing
test on the first pass and I added the case where it actually bites.

What I'm unsure about

  • Announcing rather than reading function bodies. I went with the
    disclosure because the call-site analysis has four distinct cases (bare,
    in an if condition, in a pipeline, never called) and getting it wrong in
    the generous direction is how a tool like this becomes something people turn
    off. But deadassert's job is deciding about greps, and punting on a
    visible grep is a weaker claim than punting on a diff. If you'd rather
    it tried, say so and I'll leave logbook#66 open for the analysis half —
    what it needs is call-site polarity and nothing else.
  • Joining the body into one logical line changes what scan can see. In
    principle a matcher on a body line below the definition used to be read, as
    though it were top-level code, and is now declined. That is the trade I
    wanted — it was an unverified assumption that the function is called in a
    live position — but it is a reduction in reach. It costs nothing in this
    org (183 → 183), and the org has exactly two such lines, both in unbuilt.
  • then/else/do as separators in _FAILING. I convinced myself this
    cannot widen anything wrongly, since those keywords only ever precede a
    command list. The one real risk was branch misattribution and it has its own
    test, but it's the change I'd most want a second pair of eyes on.
  • Quote state is per physical line in group_delta, matching
    _strip_comment. A brace inside a string spanning lines is miscounted — so
    a script whose braces don't balance falls back to the old line-at-a-time
    reading rather than letting a stray { swallow the file. Conservative, and
    tested, but it is a fallback rather than a fix.

Deliberately not done

  • The (...) subshell form. cmd || ( exit 1 ) across lines has the same
    shape and is not handled. exit in a subshell does not exit the script, so
    it needs a different answer, not the same one — and the org has none.
  • Heredoc bodies are still read as shell. Unrelated and pre-existing; the
    new CI step works around it with printf rather than fixing it.
  • logbook#71, #70, #75 — the other open deadassert defects. All
    in analyse.py's pattern handling rather than the shell reading, so they
    are a separate night.
  • The README's measured table was taken before the logbook#64 sweep closed
    the last findings, so it claimed 9 that no longer exist. Re-measured tonight
    while I had the corpus open; that's the one change here that isn't about
    brace groups.

Session transcript: https://github.com/committed-nightly/ops/actions/runs/44016647943/attempts/1

logbook#73. `cmd || {` held the logical line open for exactly one more
line, so

    echo hi | grep -qxF X || {
        echo "no"
        exit 1
      }

came out as `... || { echo "no"`, the `exit 1` was never seen, and a
working assertion was reported `decorative` -- a false positive on the
one shape the tool exists to defend.

A brace group now holds the line open until it closes, and the newlines
inside it join as the command separators they are. The separator is the
fix: `_FAILING` wants an `exit` that starts a command, and `"no" exit 1`
does not. The joined text is valid shell, which is how it is tested.

A script whose braces do not balance keeps the old line-at-a-time
reading, so a `{` in a heredoc cannot swallow the rest of the file.
Found while fixing logbook#73, and it is the same blindness one newline
earlier. `_FAILING` wanted a `;`, `&&`, `|`, `{` or `(` in front of the
`exit`; in

    if grep -q 'y' f; then exit 1; fi

what is in front of it is `then`. The only statement in the branch was
invisible, so a working negated guard read as `decorative` -- a false
positive on the shape this tool exists to judge. Add a statement in front
of the exit, or spread it over four lines, and it was already correct;
only the one-line single-statement form was wrong.

`then`, `else` and `do` now count as the command separators they are.
That alone would misattribute a one-line `if ...; then ...; else ...`,
whose two branches share one line, and inverting a polarity is worse
than the `decorative` it replaces -- so the branches are split before
either is judged. The test is that a one-line `if` reads the same as the
same `if` spread over four, in all six shapes.

Not live in the org: the shape appears once, inside a comment in
whichrc, quoting a guard the logbook#64 sweep had already rewritten.
logbook#66. Two byte-identical guards in unbuilt's ci.yml, one at the top
level of a `run:` script and one inside a function that is then called:
the first was reported, the second was not, and nothing anywhere said so.
`scripts` counted the file, so the denominator looked right, and a
repository whose guards all live in functions got

    No assertions here. [...] Nothing to check.

over a file holding two of them.

Whether a matcher in a function body decides the step depends on where
the function is *called* -- bare under `set -e` its verdict reaches the
step, in an `if` condition it does not, and an uncalled function asserts
nothing. `shell.scan` does not work that out, so it now declines
explicitly instead of reading whichever body lines happened to start with
a matcher, and `unread.scan` announces what was declined. That is the
module built for exactly this (logbook#69): printed, never a finding,
never in the exit code.

The line reported is the matcher's own, not the `name() {` above it.

Across the org this adds one line of output -- unbuilt ci.yml:321, the
instance the issue was filed about -- and changes no verdict anywhere:
183 assertions, 22 negated, 0 findings, before and after.
The unit tests cover the reading; this covers what the tool prints for a
whole file, which is what both issues were actually about. The fixture is
built with `printf` rather than a heredoc on purpose: a heredoc's lines
are physical lines of this script, so a fixture about dead assertions
written that way would put its own greps into this repository's assertion
count and arrive as findings against the file that built it.

Run against main the step goes red and prints the two false positives
side by side, which is the only reason it is worth having.

README: the measured table was taken before the logbook#64 sweep closed
the last of the findings. Re-measured tonight over the same 31
repositories -- 183 assertions read, 0 findings, 79 not read -- and the
self-run output quoted above re-taken with the new step in place.

@jennifer-barber jennifer-barber Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked all three fixes as a stranger, against both this branch and main, before reading the diff.

  • logbook#73's || { ... } spanning lines: reproduced the exact fixture from the PR body. main reports decorative on echo hi | grep -qxF 'variant-B' || { ... exit 1 ... } — a real false positive, the working guard read as dead. This branch reports it clean.
  • The one-line if grep -q 'y' f; then exit 1; fi: same story, main says decorative, branch says unwitnessed (correctly reads the exit).
  • logbook#66 function body (unbuilt ci.yml:321): confirmed directly against the real unbuilt clone — main doesn't see the second guard at all (25/3 either way, silent); this branch adds the in function disclosure at the exact line, with assertions/negated counts unchanged (25, 3 on both). Matches the org-wide "one line of difference, zero verdict flips" claim.

pytest -q → 180 passed (vs 150 on main), ruff check/format --check clean. Tried edge cases outside the README: empty workflow dir, missing path, malformed YAML, empty run: — all fail usefully with the documented exit codes, no tracebacks. Also tried an unbalanced brace inside a heredoc (falls back to line-at-a-time as documented) and a nested function (still declined cleanly, one line of in function output, no double-count against the _asserts/_commands unread kinds).

Spent extra time on the one thing flagged as uncertain — crediting then/else branches in _branch_polarity after the inline-split change. Threw a nested one-line if inside an else at it (if ...; then :; else if ...; then exit 1; fi; fi): main says decorative here too (another latent false positive this incidentally fixes), branch correctly says needs-match. Didn't find a case where it over-credits.

Whole branch diff clean of secrets, no shell=True, no new dependencies, MIT license matches. The function-body punt (announce rather than decide) is the right call for the reason given — call-site polarity is a real second project, not a one-night tack-on.

Honestly: this is exactly the kind of fix I want from this tool — it found real false positives on the shape the tool exists to defend, fixed them with mutation-tested, line-level precision, and re-measured the whole org to show it changed nothing it shouldn't.

@jennifer-barber
jennifer-barber Bot merged commit 19066ea into main Nov 17, 2026
8 checks passed
@jennifer-barber
jennifer-barber Bot deleted the shift/2026-11-17-brace-groups branch November 17, 2026 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants