gate-44 and gate-52 print PASS over an unopened scope (#361's shape), inflating the COVERAGE line

[rubenvdlinde]

gate-44 and gate-52 print PASS over an unopened scope — #361's shape in two more gates

Found while proving gates 44–64 can fail, using repo-shaped git fixtures driven through the real bin/hydra-gates, with every plant committed and a prediction written before each arm.

gate-44 (autocomplete-attr) — run-hydra-gates.sh:5945 falls through to _pass 44 on an empty file set. Same tree, only the scope input changed:

scope	gate-44
unscoped	FAIL — 1
diff-scoped, empty diff	PASS

Its immediate neighbour gate-45 reads the same enumerator and correctly reports na on the same input — so the correct behaviour is already implemented one gate away.

gate-52 (custom-widget-ratchet) has the same fall-through, and additionally reproduces #352 exactly: base=0 head=3 delta=+3 when the merge base predates the app's own registry script.

Why this is worse than a hidden number

This is the distinction drawn in #361: a NOT APPLICABLE is excluded from the verdict, but a PASS counts toward "N of N applicable gates ran". It does not hide a finding — it falsifies the coverage line, which is the one line this programme relies on to tell an honest green from a vacuous one. Every COVERAGE: N of 64 quoted while these two gates were in scope is overstated by up to two.

The fix that shipped for gate-16 in #364 applies unchanged: always run the checker; evaluate WIRING before SCOPE. A crash must never be reported as an empty scope, and an empty scope must never be reported as a pass.

Crash behaviour, measured separately

The package's own crash suite passes but reaches only gates 56–64. Running the missing half with a python3 shim exiting 1: gates 44, 45, 46, 49, 50, 51, 52, 54, 55 all reported SKIPPED (wiring) — none PASS, none FAIL. So the wiring half is sound in both gates; it is specifically the empty-scope path that is wrong.

Related, from the same sweep

• gate-60's entire Tier B half is unreachable code. All 10 values in CONCEPT_LABELS are Tier A keys, while semantic-icons.json ships 137 Tier B concepts, so the warns.append(… Tier B — SHOULD) branch can never execute. A green gate-60 says nothing about 137 of 147 concepts.
• gate-49 is silenced by a file-header @throws \Throwable. METHOD_RE's optional non-greedy docblock group binds the file header, so the first method in the file is exempted. Positive-controlled: plain plant FAIL; header @throws added ⇒ PASS; a second unhandled method still FAIL. Same "tag position decides" family as gate-19: a file-level @e2e tag is counted as live coverage without checking any test — _ref_is_live returns True when no declaration owns it #343.
• gates 60 and 62 disagree about whether a type:"caption" menu node is a menu entry — 60 exempts it, 62 fails it twice. Undeclared divergence.
• Exemption lists are now the load-bearing defect three times over: gate-60 (CONCEPT_LABELS), gate-57 (WRITE_VERB_PREFIXES — 1 of 4 orphaned write methods seen), gate-50 (accepted app-id shapes — PASS over two unguarded reads of the exact opencatalogi#86 defect, written with a variable app id).

[rubenvdlinde]

Expanded: this is 13 gates, not 2 — measured independently in a second sweep

A parallel proof of gates 22–43, run through the real wrapper (never the checkers directly) on a committed repo-shaped fixture, found the same defect in eleven more gates.

Arm: a docs-only diff over a planted tree, with all twelve planted defects still on disk.

result	gates
PASS over an unopened scope	22, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43
correctly NOT APPLICABLE	25, 28, 29, 30, 31, 32, 33

Banner read 23 of 23 applicable gates ran.

Together with gate-44 and gate-52 from the 44–64 sweep, 13 of 64 gates can print PASS over a scope they never opened, and — unlike NOT APPLICABLE, which is excluded from the tally — every one of those PASSes is counted as an applicable gate that ran. So a COVERAGE: N of 64 line can be overstated by up to 13, and the overstatement is invisible in exactly the runs where the diff is narrow.

The fix already exists in-tree, for four gates

test_gate_empty_scope_never_passes.sh already enforces this property — for gates 19, 25, 62 and 63 only. Extending that test to the full set is the smallest correct fix, and it makes the acceptance suite the enforcement point rather than a convention.

Gates 26, 31 and 32 are the model for the runtime behaviour: each emits its own gate-level na with a reason, on the same input where the eleven above pass.

gate-40's inline justification is refuted by the same run that reads it

gate-40 carries a note explaining why it passes rather than reporting na. Both of its claims are false, and both are refuted by the run in which it makes them:

1. "every other gate in this family reports PASS over an empty diff scope" — 26, 31 and 32 do not. They emit their own na reason.
2. "the test at test-hydra-gates-bin.sh:465 requires it" — that test matches the applicability table's wording, which a gate-level na passes unharmed.

The stated blocker does not exist. This is the third instance in this programme of an exemption's stated reason being the load-bearing defect rather than the detection logic — after the accessibility selectors excused "for older Nextcloud" that were dead on the oldest supported server, and gate-7's blanket @PublicPage exemption.

A second, separate defect: three gates make FALSE CLAIMS ABOUT THE REPOSITORY when their checker crashes

Crashed-checker arm (a python3 shim exiting 1). Gates 34, 37, 38, 39, 40, 41, 42, 43 and 44–55 all reported an honest SKIPPED (wiring). But:

• gate-22 reports "1 schema violation(s) in src/manifest.json"
• gate-25 reports "an unreported number of endpoints"
• gate-28 reports "not one carried an @license declaration" — when both files in the fixture do

gate-28's own header comment was written to prevent exactly this class of false claim; it guards against a missing helper but not a crashing one. A crashed checker that invents a finding is worse than one that invents a pass: it sends someone to fix a defect that does not exist, and when they find nothing they learn to distrust the gate.

Also confirmed live, one file and one run: #363 in both directions

gate-25's fixed 20-line auth lookback simultaneously:

• flags an admin-only method (adminOnlyPurge — it reads the previous method's #[PublicPage]), and
• misses a genuinely public untested one (farAttribute, whose attribute sits 24 lines up).

The silent half is the dangerous one.

Corrections to earlier claims in this repo's issues

• gate-26 is no longer manifest-v2 blind — #309 landed. Proven with the component placed outside src/views/ so only the manifest half could classify it. The prose defect (#358) is still live: a // comment saying you still owe a baseline yields PASS — all have a visual proof. It also FAILs a page that a runtime-generated spec genuinely covers.
• gate-24 is not blocked upstream. It runs against a sibling nextcloud-vue checkout and inspects 27 real descriptors (positive-controlled — that green sentence reads identically over zero). What is blocked is distribution: the npm package omits scripts/, and 14 of 18 apps ship no wrapper.
• gate-29 does not use git check-ignore — deliberately, with an inline warning against a future edit that would reintroduce it (check-ignore -q exits 0 when a negation matches).
• gate-35 is blind to <noun>Url — the \b-anchored noun list cannot match avatarUrl/thumbnailUrl/photoUrl (4 of 7 in a controlled probe). Live fleet exposure is zero, positive-controlled: 5 <img> tags fleet-wide.
• gate-36 fires on tabindex="5" inside a comment — the last raw grep in the 34–43 family.
• gate-33: only 4 of 18 apps opt in, so NOT APPLICABLE there means accessibility is unverified, not clean. ⚠️ A first pass read main and returned "absent" for all eighteen — main is stale (379 vs 15,054 bytes); caught by a byte-count control.
• gates 38 and 41 have zero live fleet subject matter — 15/15 App.vue exempted by shell, 10/10 *Root.vue exempted, 40/40 templates classified fragment. They work; they have never had anything to look at.

[rubenvdlinde]

Final count: 16 gates, from three independent sweeps covering all 64

The 1–21 sweep completes the picture. Every gate in the package has now been driven through the real wrapper against committed, repo-shaped planted / clean / near-miss fixtures, with a prediction written before each arm.

Gates that print PASS over a scope they never opened:

slice	gates
1–21	17, 18, 21
22–43	22, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43
44–64	44, 52

16 of 64. NOT APPLICABLE is excluded from the verdict; each of these PASSes is counted as an applicable gate that ran. So COVERAGE: N of 64 can be overstated by up to 16 — and it is overstated precisely when the diff is narrow, which is the common case on a push run.

test_gate_empty_scope_never_passes.sh already enforces the property, for gates 12, 13, 19, 25, 33, 62 and 63 only — 7 of 64. Extending it across the package is the smallest correct fix, and it puts enforcement in the acceptance suite rather than in convention. Gates 26, 31 and 32 model the correct runtime behaviour: a gate-level na with a reason, on the same input where the sixteen pass.

A second scope defect, in a different mechanism: gate-17

run-hydra-gates.sh:238 never absolutises APP_DIR before cd, and gate-17 is the only gate that hands ${APP_DIR} to its checker afterwards. On a relative app-dir path it scans zero files → # count=0 → PASS. Same tree, absolute path → FAIL — 1.

bin/hydra-gates:155 absolutises, so CI is safe — but the documented human invocation is not.

⚠️ The standing acceptance suite is structurally unable to see this: its driver always builds absolute paths. That is worth more than the bug — a suite that can only reproduce the safe invocation cannot test the unsafe one.

gate-7: both mechanisms reproduced independently, both predicted in advance

Byte-identical IDOR plant, one file, one run:

arm	gate-7
plant + no user → 401 preamble	PASS
plant + #[PublicPage] — an unauthenticated write to an arbitrary $id	PASS

Sources: check_no_admin_idor.py:388 (STATUS_UNAUTHORIZED inside _GUARD_BODY_RE) and :1143 — note, not :1141 as reported earlier — the unconditional #[PublicPage] skip.

Both live in the exemption path, not the detection path. The regex that finds IDORs is sound; the skip list is what blinds it. And the #[PublicPage] exemption carries no reason at all, so there is nothing to test — which makes it the strongest case yet for the rule this programme has hit three times: an exemption's reason is a testable claim, and an exemption without one cannot be audited.

Corrections to earlier claims in this thread

• "gate-20 has never fired, treat its silence as suspect" is now stale. At 57bcb2b it fires on a planted findObjects( and is silent on the commented-out and wrong-receiver near-misses. A silent gate-20 today is evidence.
• "Several gates need a diff and are dead on --full" does not generalise into 1–21. Exactly one (gate-16) is diff-dependent, and fix(gate-16): an EMPTY SCOPE is not a PASS (.github#361) #364 made it honest (NOT APPLICABLE), not dead. fix(gate-16): an EMPTY SCOPE is not a PASS (.github#361) #364 and fix(gate-61): an empty scope has two causes and only one of them is a diff (.github#347) #371 both re-verified as merged and effective.
• gate-21 is NOISY — a 7-character Markdown setext underline matches ^={7}$. It is also the only gate in 1–21 that enumerates with find rather than git ls-files, so an untracked plant is seen.
• gate-2 has one narrow false positive: private function exit(), a legal PHP semi-reserved method declaration, is flagged. Call sites and $exitCode are correctly silent.
• gate-19's prose defect is confirmed at wrapper level: a comment reading "TODO: the second scenario is still unwritten. Its anchor is @e2e tag::beta" flips FAIL → PASS. Its honest and counterfeit arms emit byte-identical verdicts.
• gate-6 cannot be expressed as an expect.conf row — the runner names its log via mktemp "…hydra-gate-orphan-auth.XXXXXX.log", so the basename is random and the driver's fixed-basename contract cannot address it.

⚠️ test_gate_crashed_checker_is_not_a_finding.sh (38 assertions, all passing) stalls for minutes on its gate-17 arm — two backgrounded runs returned exit 0 having printed three lines. Require the trailing ALL PASS before believing it.

Coverage delivered

Acceptance coverage went 13 → 30 of 64 gates; 55 passed / 0 failed, ratchet intact, UNCOVERED.md shrunk. Two honest limits: gate-4 rots by construction (it depends on live Packagist advisories), and every "silent on near-miss" result is only as strong as the near-misses the checkers' own docstrings record.

[rubenvdlinde]

Correction: eighteen gates, not sixteen — and this list is still a floor

#378's author re-measured while implementing the fix and found two further instances:

• gate-14 — measured directly: full scope FAIL — 1, docs-only diff PASS.
• gate-20 — same shape.

Complete set: 14, 17, 18, 20, 21, 22, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 52.

The signature worth reusing when hunting the rest: a findings counter standing in for an inspection counter. Where those two are conflated, an unopened scope and a clean scope produce the same number, and the gate cannot tell them apart. Fixed at the fall-through via one shared helper rather than eighteen separate edits.

gate-40's inline note claiming it could not report na is deleted — both of its claims were false, and both were refuted by the run that read them.

⚠️ Still unexamined: gate-15 has the same shape. It was deliberately not guessed at. Four of the eighteen have no planted-true-positive arm yet.

The root cause was one conflation, and it is the same one behind #347 and #361

File scope and the delta base were a single input. They are now two named ones:

• SCOPE-MODE: full|diff controls the state gates.
• The delta base resolves independently, and BASE_REF no longer carries a hardcoded origin/development.

This matters more than it sounds. The five genuine delta gates — 16, 29, 47, 48, 61 — key on whether a base resolved, not on file scope. Had they keyed on file scope, the whole-tree default would have silently retired all five fleet-wide. As built, a PR keeps every delta gate and gains whole-tree coverage.

Corrections to the delta list as I stated it

• gate-54's property half is not a delta gate. Its base handling is a line filter — it is a ratchet.
• gate-16 was missing from the list I gave.
• Two delta sub-halves hide inside state gates (gate-52's ratchet, gate-53's orphan promotion). They now say so rather than hiding behind a PASS.

Full classification of all 64: 51 STATE · 6 RATCHET · 5 DELTA.

The clamp does not generalise — and one of the two flagged sites was real

_skip_empty_scope answers "nothing was opened." The clamp answers "the count could not be read." Those are opposite verdicts and must not share a helper.

• gate-24 was real and is fixed. It reported FAIL — 1 parity violation(s) from a crash — an app-owned wrapper with no marker contract. Its verdict deliberately stays FAIL; guessing wiring there would open a green hole. It simply stops claiming a number nobody measured.
• gate-61's clamp is benign — its _helper_finished branch sits directly above, so a crash cannot reach it.

The other five clamp sites were not audited and belong with #379.

Two regressions the package's own suites caught, and one of them indicts a test

Both are worth recording because they are the same defect class we are fixing, occurring inside the fix:

1. The delta-base change made a crashed gate-61 read as an empty scope — the second instance of "declining earlier retires a wiring invariant." Caught by the same suite that caught #364's first draft.
2. The gate-agnostic #347 invariant fired correctly — but its trigger was a proxy: it keyed on --full when the property it defends is about whether a diff was computed. That is the same conflation this PR exists to remove, living inside the test written to catch it. Repairing the trigger exposed a real gap: gate-61's advisory backlog sweep ran only on the no-base path, which is no longer the common one.

Not verified

Nothing was run against a real fleet app. quality.yml is untouched — the reversal reaches all 18 apps with no workflow edit, so the PR's own CI is the first real proof. Gates 23 and 24 shell out to scripts absent from this package. And gates 25/26 become much stricter under #376/#377, while their header comments still say "ADDED in this PR" — those are now false, and they belong to the checker owner.