From 3066722057ed1a25a917799518129becb8ac8bbc Mon Sep 17 00:00:00 2001 From: Ruben van der Linde Date: Wed, 12 Aug 2026 10:32:10 +0200 Subject: [PATCH 1/5] =?UTF-8?q?feat(gates)!:=20scan=20the=20whole=20tree?= =?UTF-8?q?=20by=20default=20=E2=80=94=20ADR-020=20superseded?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The gates now scan the ENTIRE tracked codebase. Diff scoping becomes opt-in via --scope-to-diff / --diff / HYDRA_GATE_SCOPE=diff. Ruben's rationale: when a gate changes or is added, the next push to beta should fail unless the old code is brought up to the new standard, so gate updates travel with releases instead of applying only to code written after them. Recorded in hydra-gates/ADR-020-SUPERSEDED.md, including what it costs: the first development-to-beta run after this lands surfaces the fleet backlog at once, and the last wide-scope measurement was roughly 3,900 findings. The scope is now two independent, named inputs rather than one. File scope controls which files the state gates open and defaults to full; the delta base is resolved separately and is what the five delta gates compare against. Gates 16, 29, 47, 48 and 61 are keyed on whether a base was resolved, not on the file scope, so a pull request keeps every delta gate and gains whole-tree state coverage. Without a base they report NOT APPLICABLE by name, never PASS. An unresolvable base stays fatal on a diff-scoped run and is non-fatal at full scope, where it costs five gates rather than all sixty-four. BASE_REF no longer carries a hardcoded origin/development default, which was the shared root cause of the gate-16 and gate-61 empty-scope defects. Eighteen gates could report PASS over a scope they never opened. Full scope removes the empty set in the common case but leaves the fall-through latent, so it is fixed at the fall-through through one shared helper: gates 14, 17, 18, 20, 21, 22, 34 through 44 and 52. Gates 14 and 20 were found by sweeping the table and are not in the issue's list of sixteen. Gate 40's inline note arguing against this was false on both of its claims and is deleted with the fix; gate 52's uncomputed ratchet half and gate 53's uncomputed orphan-promotion half now say so on stdout instead of hiding behind a PASS. APP_DIR is absolutised before the cd and the resolved absolute path is printed. Gate 17 was the only gate handing APP_DIR to its checker afterwards, so a relative app-dir made the scan root resolve a second time against the app dir itself: same tree, absolute path FAIL 1, relative path PASS. CI was never exposed because bin/hydra-gates absolutises; the documented human invocation was. The acceptance suite now enforces the empty-scope property across the package instead of for seven named gates, and carries a relative-path arm the standing driver is structurally unable to express because it always builds absolute paths. Both new arms were proven able to fail by deliberate breakage. Helper suites 74 passed / 2 quarantined / 1 failed before and after, the failure being the pre-existing unresolvable-ajv needs-external skip that fails closed locally and is green in CI. Entry-point tests 61 passed / 0 failed before, 69 passed / 0 failed after. Refs: .github#374, #361, #347, #364, #371 --- hydra-gates/ADR-020-SUPERSEDED.md | 133 +++++ hydra-gates/README.md | 50 +- hydra-gates/bin/hydra-gates | 158 ++++- .../lib/test_gate16_spec_coverage_scope.sh | 102 +++- ...est_gate_5661_empty_scope_is_not_a_pass.sh | 22 +- .../lib/test_gate_empty_scope_never_passes.sh | 229 ++++++++ hydra-gates/scripts/run-hydra-gates.sh | 549 ++++++++++++++++-- hydra-gates/tests/test-hydra-gates-bin.sh | 156 +++++ 8 files changed, 1321 insertions(+), 78 deletions(-) create mode 100644 hydra-gates/ADR-020-SUPERSEDED.md diff --git a/hydra-gates/ADR-020-SUPERSEDED.md b/hydra-gates/ADR-020-SUPERSEDED.md new file mode 100644 index 00000000..7efb3d76 --- /dev/null +++ b/hydra-gates/ADR-020-SUPERSEDED.md @@ -0,0 +1,133 @@ + + +# ADR-020 is superseded: the gates scan the whole tree by default + +**Status:** accepted · **Date:** 2026-08-12 · **Decided by:** Ruben van der Linde +**Supersedes:** ADR-020 (diff-scoped gates) · **Applies to:** all 18 fleet apps at once +**Canonical ADR home:** the numbered ADR registry lives in `hydra/openspec/`, not in +this repository. This file is the decision record for the **gate package**, and it is +what `run-hydra-gates.sh` and `bin/hydra-gates` cite. Allocating a superseding ADR +number in `hydra/openspec/` is a follow-up, not a precondition. + +--- + +## The decision + +**The Hydra gates scan the ENTIRE tracked codebase by default. Diff scoping is now +opt-in (`--scope-to-diff` / `--diff` / `HYDRA_GATE_SCOPE=diff`).** + +Until now it was the other way round: `bin/hydra-gates` defaulted to +`--scope-to-diff` and `--full` was the audit-only escape hatch. + +In Ruben's words: + +> if a gate changes or is added we want the next push to beta to fail unless the old +> code is fixed to the new standard. This will force developers to take updates to +> the gates along in their new releases. + +## What ADR-020 decided, and why it was right at the time + +ADR-020 scoped every gate to the PR's diff so that **inherited debt could never block +a PR**. That was a deliberate, defensible trade: a gate that blocks legitimate work +gets switched off, and a switched-off control is worth less than a loud one. It is why +`enable-hydra-gates` could be turned on repo by repo at all. + +## Why it is being reversed + +Diff scoping means a gate only ever judges code written **after** the gate landed. +The consequences are structural, not incidental: + +1. **Tightening a gate has no effect on existing code.** The new rule applies to the + next hunk anyone happens to touch. Nobody is ever asked to bring old code up to the + new standard, so the standard is only aspirational for everything already shipped. +2. **The debt is unowned and unmeasured.** It is visible only to a `--full` audit that + nothing gates on, so it accumulates without a moment at which anyone must look at it. +3. **The empty-diff failure modes are endless.** This package's own history is mostly a + list of them: `#242`, `#240`, `#258`, `#268`, `#276`, `#347`, `#361`, `#364`, `#371`, + `#374`. Every one is a variation on *a gate that had nothing to look at printed the + same word as a gate that looked and found nothing.* Full scope removes the empty set + in the common case. (It does **not** remove the fall-through — see below.) + +## What this costs, stated up front + +**The first `development → beta` run after this lands will surface the entire backlog +at once.** The last fleet-wide wide-scope measurement was **~3,900 findings**. That is +the point of the change, and it is also the whole risk in it: 18 repos change verdict +simultaneously. + +Two things follow, and they are part of the decision rather than caveats to it: + +- **This is a sequencing problem, not a correctness one.** Every finding it surfaces + was already true. Nothing about the code changed. +- **The number is a FLOOR, not a total.** It predates several gate fixes that make + gates stricter, and several gates are still `no-fixture-yet`. Quote it as a floor. + +## What did NOT change: delta gates keep their base + +Five gates ask what a **change** did, and a whole tree cannot answer that: + +| gate | name | the question | +|---|---|---| +| 16 | spec-coverage | which methods did this change add or modify without an `@spec` anchor? | +| 29 | gitignore-then-commit | did this change add an ignore rule over already-tracked files? | +| 47 | security-change-has-tests | did this change touch security code without touching a test? | +| 48 | csrf-cochange | did this change REMOVE `@NoCSRFRequired`? | +| 61 | listener-work-placement | did this change add a post-event listener doing work in the wrong plane? | + +Keying those on the file scope would have **silently retired all five on every PR in +the fleet** the moment this default flipped — trading the gates that protect the change +in front of you for coverage they cannot use. So the scope is now **two independent, +named inputs**: + +| input | controls | default | +|---|---|---| +| `SCOPE_TO_DIFF` / `HYDRA_GATE_SCOPE` | which files the **state** gates open | `full` | +| the resolved **delta base** (`--base`, `$HYDRA_GATE_BASE_REF`, auto-detect, `github.event.before`) | what the **delta** gates compare against | resolved whenever possible | + +A PR therefore gets **whole-tree state coverage AND every delta gate**. A +`workflow_dispatch` has no base, and those five report `NOT APPLICABLE` **by name, with +a reason** — never `PASS`, and never counted as one. + +This also closes a defect the old conflation caused: *"I ran it with no base" was never +a scope.* gate-19 swept the whole tree while gate-16 fell back to a hardcoded +`origin/development` and printed `PASS` over nothing (`#361`), and gate-61 declined +citing a diff the run had never computed (`#347`) — one package answering one question +two ways. `BASE_REF` now starts **empty**; a base is stated or it does not exist. + +**An unresolvable base is no longer fatal at full scope.** On a diff-scoped run it still +exits 99, because without a base there is no scope at all. On a full run it costs five +gates out of 64, so refusing would discard 59 real verdicts to punish one bad input. + +## What did NOT change: the empty-scope fall-through is still a bug + +Full scope removes the empty set in the common case. It does **not** make the +fall-through safe, and treating it as the fix would have been the mistake: +`--scope-to-diff` still exists, and a repo can genuinely ship no `src/`. + +So the fall-through is fixed **at the fall-through** (`_skip_empty_scope` in +`run-hydra-gates.sh`), across **eighteen** gates — the sixteen `#374` enumerates plus +**14** and **20**, found by sweeping the table rather than from the issue. And the +property is now enforced by +`scripts/lib/test_gate_empty_scope_never_passes.sh` **ARM 6**, gate-agnostically over +the whole package, instead of by seven gates named one at a time. + +## Rollout + +1. **Do not merge this alongside anything else.** It changes the verdict for 18 repos + simultaneously; the next fleet measurement must be attributable to exactly one cause. +2. **Capture a wide-scope baseline per app first.** The standing `development → beta` + PR already runs a wide-scope job and its log carries the full table — no + `workflow_dispatch` needed, and dispatching one cancels that very run. +3. **Expect `--require-full-coverage` to be the loudest change**, not the gate findings: + a `PASS` that was really an unopened scope now reports `NOT APPLICABLE`, so + `COVERAGE: N of 64` drops in some repos. That is the line getting *more* honest, not + coverage getting worse. + +## How to get the old behaviour + +```bash +hydra-gates --scope-to-diff --base origin/development # ADR-020, explicitly +HYDRA_GATE_SCOPE=diff hydra-gates # the same, via the environment +``` + +Both print `SCOPE-MODE: diff` and say what they are not judging. diff --git a/hydra-gates/README.md b/hydra-gates/README.md index 94ba1005..a7f20adc 100644 --- a/hydra-gates/README.md +++ b/hydra-gates/README.md @@ -194,22 +194,50 @@ wrong. --- -## Diff scoping and the base ref +## Scope: full by default (ADR-020 is superseded) -The gates are diff-scoped per ADR-020: a PR is judged on what it changed, not on -what it inherited. This matters — openbuild fails 16 gates on a full-repo run -today and passes when scoped to a real diff. `composer gates:full` gives the -audit view and is deliberately not what `check:strict` runs. +**The gates scan the ENTIRE tracked tree by default. Diff scoping is opt-in.** -Diff scoping is only as trustworthy as the base ref, and a base that resolves to -nothing produces a report of zero failures that is indistinguishable from a -clean one. So: +ADR-020 scoped every gate to the PR's diff so inherited debt could never block a +PR. That is reversed, deliberately — see +[ADR-020-SUPERSEDED.md](ADR-020-SUPERSEDED.md) for the decision, the reasoning, +and what it costs (the first `development → beta` run after it lands surfaces +the fleet backlog at once; the last wide-scope measurement was ~3,900 findings). + +The scope is **two independent, named inputs**, and both are printed every run: + +| input | controls | default | how to set it | +|---|---|---|---| +| **file scope** | which files the **state** gates open | `full` | `--scope-to-diff` / `--diff`, or `HYDRA_GATE_SCOPE=full\|diff` | +| **delta base** | what the five **delta** gates compare against | resolved whenever possible | `--base REF` / `$HYDRA_GATE_BASE_REF` / auto-detect / `github.event.before` | + +Every run emits exactly one machine-readable `[hydra-gates] SCOPE-MODE: full|diff` +line. **Read that, never the prose** — inferring the scope from whether a base was +printed, or from a gate's own wording, has been wrong at least once each. + +Gates **16, 29, 47, 48 and 61** ask what a *change* did and cannot be answered by a +checkout. With a base they run at any file scope; with none they report +`NOT APPLICABLE` **by name, with a reason** — never `PASS`, and never counted as one. + +To get the old behaviour explicitly: + +```bash +hydra-gates --scope-to-diff --base origin/development +HYDRA_GATE_SCOPE=diff hydra-gates +``` + +### The base ref + +A base that resolves to nothing produces a report of zero failures that is +indistinguishable from a clean one. So: - The base is resolved from a stated precedence chain and **printed** every run: `--base` → `$HYDRA_GATE_BASE_REF` → `origin/HEAD` → `origin/development` → `origin/main` → `origin/master`. -- **An unresolvable base stops the run with exit 99.** It is never treated as an - empty diff, and no green is printed. +- **On a diff-scoped run, an unresolvable base stops the run with exit 99.** It is + never treated as an empty diff, and no green is printed. **At full scope it is + NOT fatal** — it costs the five delta gates and nothing else, so refusing would + discard 59 real verdicts to punish one bad input. The five say so by name. - **A base you named explicitly is never silently replaced.** Substituting a different one would scope the run to something you did not ask for and would not read about. @@ -381,7 +409,7 @@ reporting that it did nothing did nothing. But only two of the three fail: | verdict | meaning | counts against coverage? | |---|---|---| -| `NOT APPLICABLE` | the gate's subject matter does not exist in this repo or this diff — no `src/` at all, or a diff with no composer file under ADR-020 scoping | **no** | +| `NOT APPLICABLE` | the gate's subject matter does not exist in this repo, or (on an opt-in diff-scoped run) in this diff — no `src/` at all, no composer file in the change set; **or the gate is a DELTA gate and this run has no base** | **no** | | `SKIPPED (structural)` | the subject matter EXISTS and nothing produced the gate's input — e.g. a repo that registers integration leaves but ships no parity check | yes | | `SKIPPED (wiring)` | the gate's own machinery is missing — a helper script, a tool not on PATH | yes | diff --git a/hydra-gates/bin/hydra-gates b/hydra-gates/bin/hydra-gates index 39eb866e..f9ad3746 100755 --- a/hydra-gates/bin/hydra-gates +++ b/hydra-gates/bin/hydra-gates @@ -19,14 +19,48 @@ # for helpers that are not there. We resolve the link chain to the real # package root first. # -# 2. AN EXPLICIT, STATED BASE REF (ADR-020). -# The gates are diff-scoped. A base ref that silently scopes to nothing -# reports zero failures, which is indistinguishable from clean. We resolve -# the base from an explicit precedence chain, PRINT which base won, and +# 2. AN EXPLICIT, STATED BASE REF. +# A base ref that silently scopes to nothing reports zero failures, which +# is indistinguishable from clean. We resolve the base from an explicit +# precedence chain, PRINT which base won, and — on a diff-scoped run — # refuse to run when none resolves. We never fall back to HEAD~1: on a # squash-merged mainline HEAD~1 is the previous release, and on a fresh # shallow clone it may not exist at all. # +# =========================================================================== +# THE SCOPE IS FULL BY DEFAULT. ADR-020 IS SUPERSEDED. +# =========================================================================== +# +# Until now this wrapper defaulted to `--scope-to-diff` and `--full` was the +# opt-in. That is reversed: the gates scan the ENTIRE tracked tree unless the +# caller asks for a diff with `--scope-to-diff` / `--diff`. +# +# The reason, in Ruben's words: "if a gate changes or is added we want the next +# push to beta to fail unless the old code is fixed to the new standard. This +# will force developers to take updates to the gates along in their new +# releases." Under diff scoping a tightened gate only ever judged code written +# after it landed, so the fleet's inherited debt was permanently invisible and +# permanently unowned. +# +# WHAT THIS COSTS, STATED HERE RATHER THAN DISCOVERED: the first +# `development → beta` run after this lands surfaces the whole backlog at once. +# The last fleet-wide wide-scope measurement was ~3,900 findings. See +# hydra-gates/ADR-020-SUPERSEDED.md for the full record and the sequencing. +# +# THE BASE REF DOES NOT GO AWAY. Five gates ask what a CHANGE did — 16 +# spec-coverage, 29 gitignore-then-commit, 47 security-change-has-tests, +# 48 csrf-cochange, 61 listener-work-placement — and a whole tree cannot answer +# that. So this wrapper still resolves a base and still forwards it, on a full +# run as well as a diff-scoped one, and the runner keys those five on whether a +# base was resolved rather than on the file scope. A PR therefore keeps every +# delta gate AND gains whole-tree state coverage; a workflow_dispatch has no +# base and those five report NOT APPLICABLE, by name, which is not a pass. +# +# AND AN UNRESOLVABLE BASE IS NO LONGER FATAL AT FULL SCOPE. On a diff-scoped +# run it still exits 99 — without a base there is no scope at all. On a full +# run it costs five gates and nothing else, so refusing would throw away 59 +# real verdicts to punish one bad input. +# # 3. A COVERAGE ASSERTION ON THE GREEN. # The runner used to end with "ALL 63 GATES GREEN" whether or not 63 gates # ran. Measured on openbuild, only 59 of 61 emitted a line — gates 24 and @@ -57,9 +91,17 @@ # hydra-gates [--app-dir DIR] [--base REF] [--full] [--require-full-coverage] # # --app-dir DIR repo to scan (default: current directory) -# --base REF diff base (default: see resolution order below) -# --full scan the whole repo instead of the diff. Reports -# inherited debt; use for audits, not for gating. +# --base REF the DELTA base — what the five delta gates compare +# against, and the file scope too when +# --scope-to-diff is given (default: see the +# resolution order below) +# --full scan the whole repo. THIS IS NOW THE DEFAULT; the +# flag is kept because every caller in the fleet +# already passes it, and it now names the default +# rather than switching to it. +# --scope-to-diff, --diff the old ADR-020 behaviour, now OPT-IN: judge only +# the files this change touched. Inherited debt in +# untouched files is not judged. # --require-full-coverage treat "an APPLICABLE gate did not run" as a # failure. Gates that reported NOT APPLICABLE — their # subject matter is absent from this repo or this @@ -129,7 +171,19 @@ fi # --------------------------------------------------------------------------- APP_DIR="" BASE_REF_ARG="" -SCOPE_DIFF=1 +# FULL SCOPE BY DEFAULT — the reversal. `$HYDRA_GATE_SCOPE` is the env form and +# an unrecognised value is a hard stop, because "which scope did that run use?" +# is the question every verdict below is an answer to, and a run whose scope +# nobody can name is not evidence about anything. +case "${HYDRA_GATE_SCOPE:-full}" in + full) SCOPE_DIFF=0 ;; + diff) SCOPE_DIFF=1 ;; + *) + echo "[hydra-gates] FATAL: \$HYDRA_GATE_SCOPE='${HYDRA_GATE_SCOPE}' is not 'full' or 'diff'." >&2 + echo "[hydra-gates] Refusing to guess a scope. NOTHING WAS CHECKED." >&2 + exit 99 + ;; +esac REQUIRE_FULL_COVERAGE=0 AXE_ENABLED="${HYDRA_GATE_AXE_ENABLED:-0}" @@ -140,6 +194,7 @@ while [ $# -gt 0 ]; do --base) BASE_REF_ARG="${2:-}"; shift 2 ;; --base=*) BASE_REF_ARG="${1#--base=}"; shift ;; --full) SCOPE_DIFF=0; shift ;; + --scope-to-diff|--diff) SCOPE_DIFF=1; shift ;; --require-full-coverage) REQUIRE_FULL_COVERAGE=1; shift ;; --axe-enabled) AXE_ENABLED=1; shift ;; -h|--help) sed -n '3,70p' "${_self}" | sed 's/^# \{0,1\}//'; exit 0 ;; @@ -308,16 +363,97 @@ if [ "${SCOPE_DIFF}" = "1" ]; then fi fi else - echo "[hydra-gates] Base ref: n/a — --full requested, scanning the entire tree." - echo "[hydra-gates] NOTE: a --full run reports inherited debt and is NOT the gating mode (ADR-020)." + # ── FULL FILE SCOPE (the default) — resolve a DELTA base anyway ───────── + # + # Full scope is about which files the STATE gates open. It says nothing + # about the five DELTA gates, which need a base and would otherwise have + # been silently retired on every PR in the fleet the moment this default + # flipped. So we resolve one with the SAME precedence chain as above and + # forward it; the runner keys those five on whether it arrived. + # + # EVERY FAILURE HERE IS NON-FATAL. That is the difference from the branch + # above. There, no base means no scope and exit 99 is the only honest + # answer. Here it costs five gates out of 64, so refusing would discard 59 + # real verdicts to punish one bad input — and a run that reports nothing is + # worth less than a run that reports most things and names what it could + # not reach. + echo "[hydra-gates] File scope: FULL — the entire tracked tree (the default; ADR-020 is superseded)." + if command -v git > /dev/null 2>&1 && _git rev-parse --git-dir > /dev/null 2>&1; then + _db_source="" + if [ -n "${BASE_REF_ARG}" ]; then + if _resolves "${BASE_REF_ARG}"; then + BASE_REF="${BASE_REF_ARG}"; _db_source="--base" + else + echo "[hydra-gates] Delta base: '${BASE_REF_ARG}' was named with --base but does not resolve — ignoring it." + fi + elif [ -n "${HYDRA_GATE_BASE_REF:-}" ]; then + if _resolves "${HYDRA_GATE_BASE_REF}"; then + BASE_REF="${HYDRA_GATE_BASE_REF}"; _db_source="\$HYDRA_GATE_BASE_REF" + else + echo "[hydra-gates] Delta base: \$HYDRA_GATE_BASE_REF='${HYDRA_GATE_BASE_REF}' does not resolve — ignoring it." + fi + else + # Same chain as the diff branch, and @{upstream} is deliberately + # NOT in it for the reason spelled out there: a feature branch's + # tracking ref is itself, so the diff is empty exactly when CI runs. + _origin_head="$(_git symbolic-ref --quiet --short refs/remotes/origin/HEAD 2>/dev/null || true)" + for _cand in "${_origin_head}" origin/development origin/main origin/master; do + [ -n "${_cand}" ] || continue + if _resolves "${_cand}"; then + BASE_REF="${_cand}"; _db_source="auto-detected" + break + fi + done + fi + + # A base that IS HEAD is the mainline-push shape, and it makes the + # delta gates' diff empty by construction — the same disguise the diff + # branch spends a screen of comment on. Re-scope to the push's own + # previous tip; if that cannot be resolved, drop the base rather than + # hand the delta gates a comparison of HEAD with itself. + if [ -n "${BASE_REF}" ] \ + && [ "$(_git rev-parse "${BASE_REF}" 2>/dev/null)" = "$(_git rev-parse HEAD 2>/dev/null)" ]; then + # shellcheck source=scripts/lib/resolve-push-base.sh + # shellcheck disable=SC1091 # resolved at runtime from the package root + . "${PKG_ROOT}/scripts/lib/resolve-push-base.sh" + _head_sha="$(_git rev-parse HEAD 2>/dev/null)" + if _push_base="$(hydra_resolve_push_base "${APP_DIR}" "${_head_sha}")"; then + BASE_REF="${_push_base}"; _db_source="github.event.before (push)" + else + echo "[hydra-gates] Delta base: the resolved base IS HEAD and the push's previous tip could not be used either." + BASE_REF=""; _db_source="" + fi + fi + + if [ -n "${BASE_REF}" ]; then + BASE_SOURCE="${_db_source}" + echo "[hydra-gates] Delta base: ${BASE_REF} (${BASE_SOURCE}) = $(_git rev-parse --short "${BASE_REF}" 2>/dev/null)" + echo "[hydra-gates] Gates 16, 29, 47, 48 and 61 judge that change set; every other gate reads the whole tree." + else + echo "[hydra-gates] Delta base: NONE. Gates 16, 29, 47, 48 and 61 will report NOT APPLICABLE by name." + echo "[hydra-gates] That is not a pass and is not counted as one. The other 59 gates are unaffected." + fi + else + echo "[hydra-gates] Delta base: NONE — git is unavailable or this is not a git repository." + echo "[hydra-gates] Gates 16, 29, 47, 48 and 61 will report NOT APPLICABLE by name." + fi fi # --------------------------------------------------------------------------- # 5. Run the gates. The runner owns the gate logic; we only capture. # --------------------------------------------------------------------------- set -- "${APP_DIR}" +# The BASE and the FILE SCOPE are forwarded as two separate, named inputs. A +# base is forwarded whenever one resolved — including on a full run, where the +# five delta gates are the only things that read it. `--scope-to-diff` is +# forwarded only when the caller asked for it. +if [ -n "${BASE_REF}" ]; then + set -- --base "${BASE_REF}" "$@" +fi if [ "${SCOPE_DIFF}" = "1" ]; then - set -- --scope-to-diff --base "${BASE_REF}" "$@" + set -- --scope-to-diff "$@" +else + set -- --full "$@" fi # Forwarded, not re-derived. The runner is the only thing that reads gate-33's # report; this wrapper just has to stop swallowing the caller's declaration. diff --git a/hydra-gates/scripts/lib/test_gate16_spec_coverage_scope.sh b/hydra-gates/scripts/lib/test_gate16_spec_coverage_scope.sh index 71fe8891..1b8727b1 100644 --- a/hydra-gates/scripts/lib/test_gate16_spec_coverage_scope.sh +++ b/hydra-gates/scripts/lib/test_gate16_spec_coverage_scope.sh @@ -142,14 +142,46 @@ fi # =========================================================================== echo -echo "== arm 3 — FULL scope, same tree as arm 1 (.github#361) ==" +echo "== arm 3 — FULL file scope, NO delta base (.github#361 / #374) ==" # =========================================================================== -_outf="$(gf_run_wrapper "${WORK}/inherited" "${WORK}/log-full" --full)" +# +# REWRITTEN FOR THE ADR-020 REVERSAL (hydra-gates/ADR-020-SUPERSEDED.md). +# +# This arm used to run `--full` against a tree that `gf_mark_base` had given a +# `refs/remotes/origin/development` ref, and assert NOT APPLICABLE — because +# `--full` used to mean BOTH "read the whole tree" AND "compute no diff". +# +# Those are now two separate inputs, and that conflation was the bug: a full +# file scope says nothing about whether a base exists. gate-16 is a DELTA gate; +# what it needs is a BASE, not a narrowed file list. So the arm splits: +# +# 3a full file scope, NO base -> NOT APPLICABLE, with a reason (below) +# 3b full file scope, WITH base -> it JUDGES the change set, and still does +# NOT sweep the legacy surface (further down) +# +# 3b is the one that matters most: it is the only assertion standing between +# this package and the false RED `#361`'s fix was written to avoid — reporting +# every legacy `@spec` gap in the fleet because the file scope went wide. +# +# The tree is arm 1's, rebuilt WITHOUT `gf_mark_base`, so nothing in it +# resolves as a base. Building it explicitly rather than deleting the ref keeps +# the two arms independent. +gf_build_repo "${WORK}/inherited-nobase" "${SRC}" +gf_commit_all "${WORK}/inherited-nobase" "base: app carrying inherited @spec debt" +printf '\n- unrelated doc tweak\n' >> "${WORK}/inherited-nobase/docs/CHANGELOG.md" +gf_commit_paths "${WORK}/inherited-nobase" "docs: unrelated change" docs/CHANGELOG.md + +_outf="$(gf_run_wrapper "${WORK}/inherited-nobase" "${WORK}/log-full" --full)" -if printf '%s' "${_outf}" | grep -qF 'Base ref: n/a — --full requested'; then - _ok "the --full run states it computed no diff" +if printf '%s' "${_outf}" | grep -qF 'SCOPE-MODE: full'; then + _ok "the run announces its file scope in one machine-readable line" +else + _bad "the run printed no 'SCOPE-MODE: full' line; the rest of this arm is unsafe to interpret because nothing states which scope produced it" +fi +if printf '%s' "${_outf}" | grep -qF 'Delta base: NONE'; then + _ok "the run states it resolved NO delta base" else - _bad "the --full run did not announce itself as unscoped; the rest of this arm is unsafe to interpret" + _bad "the run did not state that it has no delta base — a delta gate's NOT APPLICABLE below would then be unattributable to any input" fi _vf="$(gf_verdict "${_outf}" 16)" @@ -158,10 +190,10 @@ case "${_vf}" in _ok "gate-16 on --full reports NOT APPLICABLE instead of PASSing over a scope it never read" ;; *PASS*) - _bad ".github#361 is LIVE: gate-16 printed PASS on a --full run that computed no diff, over a tree whose two untagged methods the positive control just named. 0 inspected, and this PASS counts toward 'N of N applicable gates ran'." + _bad ".github#361 is LIVE: gate-16 printed PASS on a run with NO delta base, over a tree whose two untagged methods the positive control just named. 0 inspected, and this PASS counts toward 'N of N applicable gates ran'." ;; *FAIL*) - _bad ".github#361 was fixed by SWEEPING THE WHOLE TREE. That is the wrong contract (ADR-020) and a false RED in every repo in the fleet — it reports inherited @spec debt the author never touched. Expected NOT APPLICABLE. Got: ${_vf:0:140}" + _bad ".github#361 was fixed by SWEEPING THE WHOLE TREE. gate-16 is a DELTA gate: widening its FILE scope does not give it a change set, it just reports inherited @spec debt the author never touched — a false RED in every repo in the fleet, and exactly what the ADR-020 reversal must NOT do to this gate. Expected NOT APPLICABLE. Got: ${_vf:0:140}" ;; "") _bad "gate-16 emitted no verdict at all on the --full arm" ;; *) _bad "gate-16 on --full gave an unrecognised verdict: ${_vf:0:140}" ;; @@ -201,12 +233,66 @@ else fi # And it must be counted as not-applicable, not as a silent no-show. -if printf '%s' "${_outf}" | grep -qE '^\[hydra-gates\][[:space:]]+gate-16 spec-coverage'; then +if printf '%s' "${_outf}" | grep -qE '^\[hydra-gates\] NOT APPLICABLE: .*\b16\b'; then _ok "gate-16 is named in the NOT APPLICABLE block of the coverage summary" else _bad "gate-16 did not appear in the coverage summary's NOT APPLICABLE block — it is being counted as a gate that simply did not run, which --require-full-coverage would fail on" fi +# =========================================================================== +echo +echo "== arm 3b — FULL file scope WITH a delta base (the ADR-020 reversal) ==" +# =========================================================================== +# +# THE ARM THAT DID NOT EXIST, AND THE ONE THE REVERSAL MAKES NECESSARY. +# +# Full file scope is now the DEFAULT, so this is the shape every PR in the +# fleet runs in: the whole tree open to the state gates, and a real base for +# the five delta gates. Two things must hold simultaneously, and they pull in +# opposite directions: +# +# 1. gate-16 must still JUDGE THE CHANGE — otherwise flipping the default +# silently retired @spec enforcement fleet-wide, which is a straight +# downgrade dressed as an improvement. +# 2. gate-16 must still NOT NAME THE LEGACY SURFACE — otherwise the wide file +# scope leaked into a delta gate and every repo goes red on inherited debt +# nobody touched. That is `#361`'s false-RED, arriving by a new route. +# +# Arm 1 already proves (2) under `--scope-to-diff`. It proves nothing about the +# default any more, because the default changed. +_outfb="$(gf_run_wrapper "${WORK}/newwork" "${WORK}/log-full-base" --full)" + +if printf '%s' "${_outfb}" | grep -qE '^\[hydra-gates\] Delta base: [^N]'; then + _ok "the full-scope run resolved a delta base and named it" +else + _bad "the full-scope run resolved NO delta base though the fixture has refs/remotes/origin/development — arm 3b would then be measuring arm 3a again and proving nothing" +fi + +_vfb="$(gf_verdict "${_outfb}" 16)" +case "${_vfb}" in + *FAIL*) + _ok "gate-16 still JUDGES the change at full file scope — the delta gate survived the reversal: ${_vfb:0:90}" + ;; + *"NOT APPLICABLE"*) + _bad "gate-16 went NOT APPLICABLE at full file scope DESPITE a resolved base. This is the regression the reversal must not cause: keying a delta gate on the FILE scope retires it on every PR in the fleet. Got: ${_vfb:0:160}" + ;; + *PASS*) + _bad "gate-16 PASSed at full file scope over a diff that ADDS an untagged method. It had a base and did not use it. Got: ${_vfb:0:160}" + ;; + *) _bad "gate-16 gave an unrecognised verdict on the full+base arm: ${_vfb:0:160}" ;; +esac + +if grep -qF 'NewWorkController.php' "${WORK}/log-full-base/hydra-gate-spec-coverage.log" 2>/dev/null; then + _ok "gate-16 names the NEWLY ADDED file at full file scope" +else + _bad "gate-16 did not name NewWorkController.php at full file scope — it produced a verdict without evidence" +fi +if grep -qF 'LegacyDebtController.php' "${WORK}/log-full-base/hydra-gate-spec-coverage.log" 2>/dev/null; then + _bad "the full-scope run wrote INHERITED legacy @spec debt into gate-16's log. The wide file scope has leaked into a delta gate — this is #361's false RED arriving by a new route, and it would redden every repo in the fleet on code nobody touched." +else + _ok "gate-16 does NOT sweep the inherited legacy surface even at full file scope — the delta contract survived the reversal" +fi + echo echo "== summary ==" echo " passed: ${_pass_n}" diff --git a/hydra-gates/scripts/lib/test_gate_5661_empty_scope_is_not_a_pass.sh b/hydra-gates/scripts/lib/test_gate_5661_empty_scope_is_not_a_pass.sh index 8eebb22f..d08ff950 100755 --- a/hydra-gates/scripts/lib/test_gate_5661_empty_scope_is_not_a_pass.sh +++ b/hydra-gates/scripts/lib/test_gate_5661_empty_scope_is_not_a_pass.sh @@ -252,11 +252,27 @@ for _g in ${_GATES}; do esac done +# A REASON MUST NAME ITS SUBJECT *AND* BE SUBSTANTIVE. Both halves are needed: +# the token list alone is satisfiable by a bare citation with no explanation, +# and a length test alone is satisfiable by any sentence at all. +# +# `delta-scoped` joined the token list with the ADR-020 reversal +# (hydra-gates/ADR-020-SUPERSEDED.md). gate-61 is delta-scoped BY POLICY rather +# than by nature — `check_listener_placement.py --all` answers the same question +# over a whole tree perfectly well, and the runner already runs it that way as +# an advisory — so its reason can no longer honestly cite ADR-020 as the live +# rule. It now names the policy instead, which is the more testable claim of +# the two. The token list may be EXTENDED for a reason that got more precise; +# it must never be relaxed to a wildcard. for _g in ${_GATES}; do - if grep -qE "^\[gate-${_g}\][^:]*: NOT APPLICABLE — .+(ADR-020|no lib/|no tests/e2e|no lib/AppInfo)" "${_scoped}"; then - _ok "gate-${_g} states WHY it was not applicable" + _reason_line="$(grep -E "^\[gate-${_g}\][^:]*: NOT APPLICABLE — " "${_scoped}" | head -1)" + _reason="${_reason_line#*NOT APPLICABLE — }" + if printf '%s' "${_reason_line}" \ + | grep -qE "(ADR-020|delta-scoped|no lib/|no tests/e2e|no lib/AppInfo)" \ + && [ "${#_reason}" -ge 40 ]; then + _ok "gate-${_g} states WHY it was not applicable (${#_reason} chars, names its subject)" else - _bad "gate-${_g}'s NOT APPLICABLE line carries no reason — a bare declaration is how a gate disappears quietly" + _bad "gate-${_g}'s NOT APPLICABLE line carries no substantive reason (${#_reason} chars) — a bare declaration is how a gate disappears quietly. Got: ${_reason_line:0:160}" fi done diff --git a/hydra-gates/scripts/lib/test_gate_empty_scope_never_passes.sh b/hydra-gates/scripts/lib/test_gate_empty_scope_never_passes.sh index b7b88014..026cf419 100755 --- a/hydra-gates/scripts/lib/test_gate_empty_scope_never_passes.sh +++ b/hydra-gates/scripts/lib/test_gate_empty_scope_never_passes.sh @@ -404,6 +404,235 @@ for _g in 12 13; do fi done +echo +# =========================================================================== +# ARM 6 — THE PROPERTY, ACROSS THE WHOLE PACKAGE, NOT SEVEN GATES BY NAME +# =========================================================================== +# +# Everything above names its gates. That is how this suite covered 12, 13, 19, +# 25, 33, 62 and 63 — SEVEN of 64 — while gates 14, 17, 18, 20, 21, 22, 34-44 +# and 52 carried the identical defect for months (.github#374). A property +# enforced gate-by-gate is enforced by whoever remembered to add a line. +# +# So this arm asserts the property ITSELF, gate-agnostically: +# +# OVER A TREE THAT CARRIES REAL, PLANTED DEFECTS, NO GATE MAY REPORT `PASS` +# ON A RUN WHOSE SCOPE EXCLUDES EVERY ONE OF THEM. +# +# The fixture is the shape #374 was measured on: a tree with a planted defect +# for a broad slice of the package, and a docs-only second commit. Any gate +# that says PASS there said it having opened nothing. +# +# ⚠️ THE ALLOWLIST IS THE LOAD-BEARING PART, AND IT MAY ONLY SHRINK. +# A gate on it is one whose PASS over this diff is HONEST — it computed a real +# answer about a real change set. Adding a gate to it to make this arm green is +# how the defect comes back, so each entry states what it computed. +echo +echo "-- ARM 6: the property, across the package --" + +_wide="${_tmp}/wide" +mkdir -p "${_wide}/lib/Controller" "${_wide}/lib/Settings" "${_wide}/appinfo" \ + "${_wide}/src/views" "${_wide}/templates" +cat > "${_wide}/lib/Controller/ThingController.php" <<'PHP' +objectService->findAll([])); + } + public function orphan(): JSONResponse + { + return new JSONResponse($this->objectService->findObjects([])); + } +} +PHP +printf ' "${_wide}/appinfo/routes.php" +printf '{"schemas":[{"slug":"thing","notifications":{"onCreate":{"subject":"x"}}}]}\n' \ + > "${_wide}/lib/Settings/register.json" +printf '<<<<<<< HEAD\nconst a = 1;\n=======\nconst a = 2;\n>>>>>>> other\n' \ + > "${_wide}/src/conflicted.js" +cat > "${_wide}/src/views/Probe.vue" <<'VUE' + + +VUE +printf '\n

hi

\n\n' > "${_wide}/templates/admin.php" +printf '{"name":"fx","menu":[],"pages":[]}\n' > "${_wide}/src/manifest.json" +( + cd "${_wide}" || exit 1 + git init -q . + git add -A + git -c user.email=t@t -c user.name=t commit -qm "the planted tree" + printf 'docs only\n' > README.md + git add README.md + git -c user.email=t@t -c user.name=t commit -qm docs +) >/dev/null 2>&1 + +_wide_full="${_tmp}/wide-full.txt" +_wide_logs="${_tmp}/wide-logs"; mkdir -p "${_wide_logs}" +( + cd "${_wide}" || exit 1 + HYDRA_GATE_LOG_DIR="${_wide_logs}" bash "${_runner}" . > "${_wide_full}" 2>&1 +) +_wide_diff="${_tmp}/wide-diff.txt" +_wide_logs2="${_tmp}/wide-logs2"; mkdir -p "${_wide_logs2}" +( + cd "${_wide}" || exit 1 + HYDRA_GATE_LOG_DIR="${_wide_logs2}" bash "${_runner}" --scope-to-diff --base HEAD~1 . \ + > "${_wide_diff}" 2>&1 +) + +# THE POSITIVE CONTROL RUNS FIRST AND EVERYTHING ELSE IS MEANINGLESS WITHOUT +# IT. If the full run finds nothing, the fixture is broken and "no gate passed +# over the empty scope" would be satisfied by a runner that gates nothing. +_wide_fails=$(grep -cE '^\[gate-[0-9]+\][^:]*: FAIL' "${_wide_full}" || true) +_wide_fails="${_wide_fails:-0}" +if [ "${_wide_fails}" -ge 8 ]; then + _ok "positive control: the planted tree FAILS ${_wide_fails} gate(s) at full scope" +else + _bad "positive control BROKEN: the planted tree fails only ${_wide_fails} gate(s) at full scope — the fixture no longer plants what this arm assumes, so every assertion below proves nothing" +fi + +# Gates whose PASS over THIS diff is a real answer to a real question. +# Each computed something; none of them is passing over an unopened scope. +# 4 composer-audit — no composer file in this tree at all; it declines by +# subject matter, not by scope (it reports na, listed +# here only so a future change of that verdict is not +# silently swallowed) +# 15 dashboard-antipattern — its helper reads the WHOLE tree and the diff +# filter is applied to its FINDINGS, so it did open the +# manifest and the .vue tree. See the note below. +# 16 spec-coverage — a DELTA gate with a real base: it diffed HEAD~1..HEAD +# and found no changed method. That is a computed zero. +# 23 or-abstraction-anti-patterns — never diff-scoped; it lints all of lib/. +# 47 security-change-has-tests — DELTA, real base: it classified the docs-only +# hunks and found no security change. Computed. +# 48 csrf-cochange — DELTA, real base: it looked for a removed attribute in +# the diff and found none. Computed. +_ARM6_ALLOWED=" 4 15 16 23 47 48 " + +_wide_bad="" +while IFS= read -r _g; do + [ -z "${_g}" ] && continue + case "${_ARM6_ALLOWED}" in + *" ${_g} "*) continue ;; + esac + _wide_bad="${_wide_bad}${_g} " +done < <(grep -E '^\[gate-[0-9]+\][^:]*: PASS' "${_wide_diff}" \ + | grep -oE '^\[gate-[0-9]+\]' | grep -oE '[0-9]+' | sort -un) + +if [ -z "${_wide_bad}" ]; then + _ok "no gate reports PASS over a diff that excludes every planted defect (the .github#374 property, package-wide)" +else + _bad "gate(s) ${_wide_bad}reported PASS over a scope that excludes every planted defect — this is the .github#374 defect. Each of them printed the same word as a gate that read the whole tree and found it clean. Fix the gate's fall-through (see _skip_empty_scope in run-hydra-gates.sh); do NOT add it to _ARM6_ALLOWED unless you can state what it computed." +fi + +# ANTI-WIDENING. ARM 6 is satisfiable by making every gate skip always, so the +# same tree at FULL scope must still produce those findings — asserted by name, +# so "the gate went quiet" cannot pass as "the gate went green". +_wide_missing="" +for _g in 14 17 21 22 34 35 36 38 40 41 42 43 44; do + grep -qE "^\[gate-${_g}\][^:]*: FAIL" "${_wide_full}" || _wide_missing="${_wide_missing}${_g} " +done +if [ -z "${_wide_missing}" ]; then + _ok "anti-widening: every one of those gates still FAILS the planted tree at full scope" +else + _bad "gate(s) ${_wide_missing}no longer FAIL the planted tree at FULL scope — the empty-scope fix has widened into a permanent skip, which is the strictly worse defect" +fi + +# =========================================================================== +# ARM 7 — A RELATIVE APP-DIR PATH MUST PRODUCE THE SAME VERDICTS (.github#374) +# =========================================================================== +# +# `run-hydra-gates.sh:238` never absolutised APP_DIR before its `cd`, and +# gate-17 is the only gate that hands `${APP_DIR}` to its checker afterwards — +# so the scan root resolved a SECOND time, against the app dir itself, and the +# checker read `relapp/relapp`. It found nothing, printed its terminal +# `# count=0`, and gate-17 reported PASS. Same tree, absolute path: FAIL — 1. +# +# ⚠️ THIS IS THE ARM THE REST OF THE PACKAGE STRUCTURALLY CANNOT HAVE. +# `test_gate_acceptance_matrix.sh` builds every fixture path from `${PKG_ROOT}`, +# so its driver can only ever reproduce the SAFE invocation. A suite that can +# only express the safe call cannot test the unsafe one, and that is worth more +# than the bug: the invocation this runner's own header documents for humans +# (`./scripts/run-hydra-gates.sh [options] [app-dir]`) was untestable here. +# +# So this arm deliberately `cd`s to the PARENT and passes a BARE RELATIVE NAME. +echo +echo "-- ARM 7: a relative app-dir path --" + +_rel_parent="${_tmp}/relparent" +mkdir -p "${_rel_parent}/relapp/lib/Controller" "${_rel_parent}/relapp/appinfo" +cat > "${_rel_parent}/relapp/lib/Controller/ThingController.php" <<'PHP' +objectService->findAll([])); + } +} +PHP +printf ' "${_rel_parent}/relapp/appinfo/routes.php" +( + cd "${_rel_parent}/relapp" || exit 1 + git init -q . + git add -A + git -c user.email=t@t -c user.name=t commit -qm init +) >/dev/null 2>&1 + +_rel_abs_out="${_tmp}/rel-abs.txt" +_rel_rel_out="${_tmp}/rel-rel.txt" +_rel_logs="${_tmp}/rel-logs"; mkdir -p "${_rel_logs}" +_rel_logs2="${_tmp}/rel-logs2"; mkdir -p "${_rel_logs2}" +( + cd "${_rel_parent}" || exit 1 + HYDRA_GATE_LOG_DIR="${_rel_logs}" bash "${_runner}" "${_rel_parent}/relapp" \ + > "${_rel_abs_out}" 2>&1 +) +( + # THE WHOLE POINT: parent directory, bare relative name, no leading `./`. + cd "${_rel_parent}" || exit 1 + HYDRA_GATE_LOG_DIR="${_rel_logs2}" bash "${_runner}" relapp > "${_rel_rel_out}" 2>&1 +) + +# Positive control FIRST: the absolute invocation must catch the plant, or the +# comparison below is between two meaningless numbers. +if grep -qE '^\[gate-17\][^:]*: FAIL' "${_rel_abs_out}"; then + _ok "positive control: gate-17 FAILS the planted pass-through wrapper via an ABSOLUTE path" +else + _bad "positive control BROKEN: gate-17 did not catch the planted ADR-022 wrapper even via an absolute path — got: $(_verdict "${_rel_abs_out}" 17)" +fi + +_rel_abs_v="$(_verdict "${_rel_abs_out}" 17)" +_rel_rel_v="$(_verdict "${_rel_rel_out}" 17)" +if [ "${_rel_abs_v}" = "${_rel_rel_v}" ]; then + _ok "gate-17 returns the same verdict ('${_rel_rel_v}') for a relative and an absolute app-dir" +else + _bad "gate-17 verdict DEPENDS ON HOW THE CALLER SPELLED THE PATH: absolute '${_rel_abs_v}' vs relative '${_rel_rel_v}'. This is .github#374 — APP_DIR is not absolutised before the cd, so the checker resolves it a second time against the app dir." +fi + +# And the invariant that makes it impossible rather than merely fixed: the run +# must STATE the absolute path it resolved. A silent fix is one refactor from +# regressing with nothing to notice it. +if grep -qE '^\[hydra-gates\] App dir: /.* \(absolute\)$' "${_rel_rel_out}"; then + _ok "the run states the ABSOLUTE app dir it resolved, so a future relative path cannot reach a checker unannounced" +else + _bad "the run does not print an absolute 'App dir:' line — nothing in the output distinguishes a relative invocation from an absolute one" +fi + echo if [ "${_failures}" -eq 0 ]; then echo "test_gate_empty_scope_never_passes.sh: ALL PASS" diff --git a/hydra-gates/scripts/run-hydra-gates.sh b/hydra-gates/scripts/run-hydra-gates.sh index c3705f60..6bef2782 100755 --- a/hydra-gates/scripts/run-hydra-gates.sh +++ b/hydra-gates/scripts/run-hydra-gates.sh @@ -200,8 +200,45 @@ else echo "[hydra-gates] Set HYDRA_GATES_PKG_SHA to make the comparison sound." fi +# --------------------------------------------------------------------------- +# THE SCOPE IS TWO INPUTS, NOT ONE, AND BOTH ARE NAMED AND PRINTED. +# +# `SCOPE_TO_DIFF` used to be the only scope control, and it decided two +# unrelated things at once: +# +# FILE SCOPE which files the STATE gates open ("is the tree valid?") +# DELTA BASE what the DELTA gates compare against ("did this change do X +# without Y?") +# +# Conflating them is why "I ran it with no base" was never a scope. On a full +# run gate-19 swept the whole tree while gate-16 fell back to a hardcoded +# `origin/development` and reported PASS over nothing (`.github#361`), and +# gate-61 declined citing a diff the run had never computed (`.github#347`) — +# the same package answering the same question two different ways. +# +# So they are separate now: +# +# SCOPE_TO_DIFF=0 FULL SCOPE — the default. Every state gate reads the whole +# tracked tree. This is the reversal of ADR-020; see +# hydra-gates/ADR-020-SUPERSEDED.md for why, and for what it +# costs. +# SCOPE_TO_DIFF=1 the old ADR-020 behaviour, now OPT-IN via --scope-to-diff. +# HAVE_DELTA_BASE set independently of the above. A delta gate runs when +# this is 1 and reports NOT APPLICABLE — never PASS — when +# it is 0. A full-scope run on a PR still HAS a base, so +# gates 16/29/47/48/61 keep working; a workflow_dispatch has +# none, and they say so by name. +# +# `BASE_REF` starts EMPTY, deliberately. The old `origin/development` default +# is the exact shape SHARED-LESSONS records twice: a base nobody supplied, +# silently diffing a branch against itself, and printing PASS. If a base is +# wanted it is stated — by --base, by $HYDRA_GATE_BASE_REF, or by the +# auto-detection below — and the winner is printed. SCOPE_TO_DIFF=0 -BASE_REF="origin/development" +BASE_REF="" +# 1 once a usable base has been resolved AND its changed-file set computed. +# Read by the DELTA gates (16, 29, 47, 48, 61) instead of SCOPE_TO_DIFF. +HAVE_DELTA_BASE=0 APP_DIR="" # Treat "a declared gate did not run" as a failure (exit 98). Off by default so # a Tier-0 app is not blocked by gates it has no surface for; on, the run @@ -225,9 +262,29 @@ REQUIRE_FULL_COVERAGE="${HYDRA_GATE_REQUIRE_FULL_COVERAGE:-0}" # to guess "unverified" in both cases, which is why --require-full-coverage was # unusable in every repo in the fleet. AXE_ENABLED="${HYDRA_GATE_AXE_ENABLED:-0}" +# THE SCOPE IS AN EXPLICIT, NAMED INPUT — never inferred from whether a base +# happens to be set. `$HYDRA_GATE_SCOPE` is the env form of the flags below; an +# unrecognised value is a hard stop rather than a silent fall-back to either +# mode, because "which scope did that run use?" is the one question every +# verdict in this file is an answer to. +case "${HYDRA_GATE_SCOPE:-full}" in + full) SCOPE_TO_DIFF=0 ;; + diff) SCOPE_TO_DIFF=1 ;; + *) + echo "[hydra-gates] FATAL: \$HYDRA_GATE_SCOPE='${HYDRA_GATE_SCOPE}' is not 'full' or 'diff'." >&2 + echo "[hydra-gates] Refusing to guess a scope. A run whose scope nobody can name is not" >&2 + echo "[hydra-gates] evidence about anything. NOTHING WAS CHECKED." >&2 + exit 99 + ;; +esac +BASE_REF="${HYDRA_GATE_BASE_REF:-}" while [ $# -gt 0 ]; do case "$1" in - --scope-to-diff) SCOPE_TO_DIFF=1; shift ;; + --scope-to-diff|--diff) SCOPE_TO_DIFF=1; shift ;; + # Explicit and redundant, and kept because it is what every caller in + # the fleet already passes for an audit run. It now names the DEFAULT + # rather than switching to it. + --full) SCOPE_TO_DIFF=0; shift ;; --require-full-coverage) REQUIRE_FULL_COVERAGE=1; shift ;; --axe-enabled) AXE_ENABLED=1; shift ;; --base) BASE_REF="$2"; shift 2 ;; @@ -236,7 +293,61 @@ while [ $# -gt 0 ]; do esac done APP_DIR="${APP_DIR:-$(pwd)}" +# --------------------------------------------------------------------------- +# ABSOLUTISE THE APP DIR *BEFORE* THE `cd`, AND NEVER LET A RELATIVE PATH REACH +# A CHECKER (.github#374). +# +# This line used to be a bare `cd "${APP_DIR}"`, leaving ${APP_DIR} holding +# whatever spelling the caller typed. Every gate that only ever works from the +# CURRENT DIRECTORY is unaffected — but gate-17 hands `${APP_DIR}` to +# detect-redundant-controllers.py as its scan root AFTER the `cd`, so a +# relative path resolves a second time, against the app dir itself. +# +# Measured on a fixture holding exactly one ADR-022 pass-through wrapper, same +# tree, same commit, only the path SPELLING changed: +# +# bash run-hydra-gates.sh /abs/path/relapp -> [gate-17] FAIL — 1 +# bash run-hydra-gates.sh relapp -> [gate-17] PASS +# +# The checker was pointed at `relapp/relapp`, found nothing, printed its +# terminal `# count=0`, and the runner correctly read that as "clean". Every +# layer behaved exactly as designed; the scan root was wrong before any of them +# saw it. +# +# `bin/hydra-gates` already absolutises (bin/hydra-gates: `APP_DIR="$(cd …)"`), +# so CI has never been exposed — but the invocation this file's own header +# documents for humans (`./scripts/run-hydra-gates.sh [options] [app-dir]`) is, +# and a gate that reports PASS because of how its caller spelled a path is the +# same defect as a gate that reports PASS over an empty scope. +# +# Resolved ONCE, HERE, so it is impossible to reach a checker with a relative +# path: everything below this line — including every `${APP_DIR}` a gate passes +# on — is absolute by construction. `cd -P` resolves symlinks too, so the value +# a checker receives is the directory it will actually read. +_app_dir_abs="$(cd -P "${APP_DIR}" 2>/dev/null && pwd)" +if [ -z "${_app_dir_abs}" ]; then + # Keep the caller's SPELLING in the message — that is the thing they typed + # and the thing they can fix. Overwriting APP_DIR first would print an + # empty path and blame nothing. + echo "[hydra-gates] ERROR: ${APP_DIR} not accessible" >&2 + exit 99 +fi +APP_DIR="${_app_dir_abs}" cd "${APP_DIR}" 2>/dev/null || { echo "[hydra-gates] ERROR: ${APP_DIR} not accessible" >&2; exit 99; } +echo "[hydra-gates] App dir: ${APP_DIR} (absolute)" + +# ONE MACHINE-READABLE LINE FOR THE FILE SCOPE, EMITTED EXACTLY ONCE. +# +# Downstream must never infer the scope from prose, from whether a base was +# printed, or from a gate's own wording. Every one of those has been wrong at +# least once in this package's history. +if [ "${SCOPE_TO_DIFF}" = "1" ]; then + echo "[hydra-gates] SCOPE-MODE: diff" + echo "[hydra-gates] File scope: the diff only (--scope-to-diff / HYDRA_GATE_SCOPE=diff). Inherited debt in untouched files is NOT judged." +else + echo "[hydra-gates] SCOPE-MODE: full" + echo "[hydra-gates] File scope: the ENTIRE tracked tree — the default. A gate added or tightened after this code was written WILL surface its inherited debt here, and that is the point (hydra-gates/ADR-020-SUPERSEDED.md)." +fi # When scope-to-diff is requested, derive the changed-files set once. # Non-diff branches that need the set: each gate below filters its @@ -456,8 +567,67 @@ if [ "${SCOPE_TO_DIFF}" = "1" ]; then else echo "[hydra-gates] Scope: diff vs ${BASE_REF} — ${_cf_count} changed file(s)" fi + # A diff-scoped run that got this far has a usable base by construction — + # every path that could not produce one exited 99 above. + HAVE_DELTA_BASE=1 else echo "[hydra-gates] Scope: full repo" + # ── A DELTA BASE FOR A FULL-SCOPE RUN ─────────────────────────────────── + # + # Full FILE scope does not mean "no base". The five DELTA gates — + # 16 spec-coverage, 29 gitignore-then-commit, 47 security-change-has-tests, + # 48 csrf-cochange, 61 listener-work-placement — ask questions only a diff + # can answer, and switching the default to full scope would have silently + # retired all five on every PR in the fleet. They are the gates that + # protect the change in front of you, so losing them to gain inherited-debt + # coverage would be a straight downgrade. + # + # So: if the caller named a base, resolve it and hand it to those gates + # while every OTHER gate reads the whole tree. + # + # EVERY FAILURE HERE IS NON-FATAL, and that is the difference from the + # diff-scoped branch above. There, an unusable base means the run cannot be + # scoped and exit 99 is correct. Here, an unusable base costs five gates + # and nothing else — 59 of 64 still have a real verdict — so refusing would + # throw away a good answer to punish a bad input. The five say NOT + # APPLICABLE, by name, with the reason below quoted back at the reader. + if [ -n "${BASE_REF}" ]; then + _db_why="" + _db_head=$(git -c safe.directory='*' rev-parse --verify --quiet 'HEAD^{commit}' 2>/dev/null || true) + _db_base=$(git -c safe.directory='*' rev-parse --verify --quiet "${BASE_REF}^{commit}" 2>/dev/null || true) + if [ -z "${_db_base}" ]; then + _db_why="it does not resolve in this repository" + elif [ "${_db_base}" = "${_db_head}" ]; then + # The mainline-push shape. `bin/hydra-gates` re-scopes this to + # github.event.before before it ever reaches here, so seeing it + # means a direct runner invocation. Comparing HEAD with itself + # yields an empty diff and would make the delta gates report a + # confident PASS over nothing — the exact defect this file spends + # 200 lines guarding against on the other branch. + _db_why="it is the SAME COMMIT as HEAD (${_db_head}), so the diff would be empty by construction" + elif ! git -c safe.directory='*' merge-base "${BASE_REF}" HEAD > /dev/null 2>&1; then + _db_why="it shares NO history with HEAD — the shape of a shallow checkout (fetch-depth: 1)" + else + CHANGED_FILES=$(git -c safe.directory='*' diff --name-only \ + --diff-filter=ACMR "${BASE_REF}...HEAD" 2>/dev/null) \ + && HAVE_DELTA_BASE=1 || _db_why="git could not compute the diff against it" + fi + if [ "${HAVE_DELTA_BASE}" = "1" ]; then + _db_count=$(printf '%s' "${CHANGED_FILES}" | grep -c . 2>/dev/null || true) + echo "[hydra-gates] Delta base: ${BASE_REF} = $(git -c safe.directory='*' rev-parse --short "${BASE_REF}" 2>/dev/null) — ${_db_count:-0} changed file(s)." + echo "[hydra-gates] The DELTA gates (16, 29, 47, 48, 61) judge that change set. Every other gate reads the whole tree." + else + CHANGED_FILES="" + echo "[hydra-gates] Delta base: UNUSABLE — '${BASE_REF}' was named but ${_db_why}." + echo "[hydra-gates] Gates 16, 29, 47, 48 and 61 will report NOT APPLICABLE by name. They are" + echo "[hydra-gates] NOT counted as passing. Every other gate is unaffected and still reads the whole tree." + BASE_REF="" + fi + else + echo "[hydra-gates] Delta base: none — no --base and no \$HYDRA_GATE_BASE_REF." + echo "[hydra-gates] Gates 16, 29, 47, 48 and 61 ask what a CHANGE did and have no change to read." + echo "[hydra-gates] They will report NOT APPLICABLE by name, which is not a pass and does not count as one." + fi fi # Helper — return 0 if $1 (a file path) is in scope (i.e. either we're @@ -1441,6 +1611,54 @@ _skip() { esac } +# --------------------------------------------------------------------------- +# _skip_empty_scope +# +# THE ONE HELPER FOR "THIS GATE OPENED NOTHING". Never PASS. +# +# WHY IT EXISTS (.github#374) +# --------------------------- +# Seventeen gates ended their file loop with a bare +# +# if [ "${#_files[@]}" -eq 0 ]; then +# : # nothing in scope; the PASS below describes the diff. +# fi +# ... +# [ "${_fail}" -eq 0 ] && _pass N "name" +# +# so a gate that inspected ZERO files printed a verdict byte-identical to one +# that read every file in the app and found every one of them clean. Measured +# on a planted tree with every defect still on disk, docs-only diff: +# gates 14, 17, 18, 21, 22, 34-44 and 52 all printed PASS. +# +# That is worse than hiding a finding. `NOT APPLICABLE` is EXCLUDED from the +# verdict, but a `PASS` is COUNTED as an applicable gate that ran — so the +# `COVERAGE: N of 64` line, the one line this whole programme relies on to tell +# an honest green from a vacuous one, was overstated by up to seventeen. +# +# Full scope removes the empty set for most of these, but it does NOT retire +# the defect: the fall-through is still there, and it would resurface the +# moment anything narrows the scope again (`--scope-to-diff`, a repo that +# genuinely ships no `src/`). So it is fixed at the fall-through, not merely +# routed around, and `test_gate_empty_scope_never_passes.sh` asserts the +# property across the package rather than by convention. +# +# Gates 26, 31 and 32 already modelled the right answer — a gate-level `na` +# WITH A REASON — on the same input where the seventeen passed. This is that +# answer, factored out so a new gate inherits it instead of copying the bug. +# +# The reason states BOTH halves a reader needs: what went uninspected, and +# which of the two possible causes it was. Those are different situations and +# only one of them is under the author's control. +_skip_empty_scope() { + local _n="$1" _name="$2" _subject="$3" + if [ "${SCOPE_TO_DIFF}" = "1" ]; then + _skip "${_n}" "${_name}" na "no ${_subject} is in scope for this run: the run was NARROWED with --scope-to-diff and the diff against '${BASE_REF}' touches none (the ADR-020 rule, now opt-in — see hydra-gates/ADR-020-SUPERSEDED.md). NOTHING was inspected and nothing could be, so this is NOT a pass. Re-run at the default full scope to judge the whole tree." + else + _skip "${_n}" "${_name}" na "no ${_subject} exists in this repository. This run is FULL-SCOPE (the default — see hydra-gates/ADR-020-SUPERSEDED.md), so the whole tracked tree was enumerated and it contains no such file. NOTHING was inspected and nothing could be, so this is NOT a pass." + fi +} + # An abort before the summary (set -e / set -u, a helper blowing up, ...) used # to be indistinguishable from a completed run: the per-gate PASS lines were # already on stdout, the summary simply never printed, and readers concluded @@ -2747,6 +2965,9 @@ fi if [ -d lib/Controller ] && [ -f appinfo/routes.php ]; then _rr_log=${HYDRA_GATE_LOG_DIR}/hydra-gate-route-reachability.log : > "${_rr_log}" + # One line per route this gate actually judged — see the note at the loop. + _rr_seen=${HYDRA_GATE_LOG_DIR}/hydra-gate-route-reachability.inspected + : > "${_rr_seen}" # Touching appinfo/routes.php puts every routed entry back in scope for # invariant 2 — see the scope note at that loop. @@ -2778,6 +2999,15 @@ if [ -d lib/Controller ] && [ -f appinfo/routes.php ]; then while IFS= read -r _ctrl_path; do [ -n "${_ctrl_path}" ] || continue _in_scope "${_ctrl_path}" || continue + # BOTH INVARIANTS COUNT TOWARD "THIS GATE OPENED SOMETHING" (.github#374). + # + # A first draft counted only invariant 2's route loop, and gate-14 then + # went NOT APPLICABLE on a full run over a fixture whose `routes.php` + # declares no route at all — while invariant 1 was busy finding a real + # unrouted controller method in it. The regression was caught by the + # before/after table, not by reasoning, which is the whole reason both + # arms are recorded here rather than at one convenient place. + echo "${_ctrl_path}" >> "${_rr_seen}" # Derive the controller route slug — strip lib/Controller/ prefix + # Controller.php suffix, lowercase the first character. Settings/ @@ -2893,6 +3123,15 @@ if [ -d lib/Controller ] && [ -f appinfo/routes.php ]; then [ "${_rr_routes_touched}" -eq 1 ] && _rr_scoped=1 _in_scope "${_path}" && _rr_scoped=1 [ "${_rr_scoped}" -eq 1 ] || continue + # COUNT WHAT WAS OPENED, VIA A FILE, NOT A VARIABLE (.github#374). + # + # This loop is fed by a PIPE (`{ … } | sort -u | while …`), so its + # body runs in a SUBSHELL: a counter incremented here is discarded + # the moment the loop ends, and the emptiness test below would read + # 0 on every run — turning the fix into a permanent skip. One line + # per inspected route, counted after the pipeline, is the shape + # that survives. + echo "${_ctrl}#${_method}" >> "${_rr_seen}" if [ ! -f "${_path}" ]; then # UNTIL 2026-08-05 this `continue`d with the comment "gate-5 # already flags this". Gate-5 flagged it as a MISSING AUTH @@ -2923,7 +3162,15 @@ if [ -d lib/Controller ] && [ -f appinfo/routes.php ]; then done _rr_fail=$(wc -l < "${_rr_log}" 2>/dev/null || echo 0) - if [ "${_rr_fail}" -eq 0 ]; then + _rr_inspected=$(wc -l < "${_rr_seen}" 2>/dev/null || echo 0) + _rr_inspected="${_rr_inspected:-0}" + if [ "${_rr_inspected}" -eq 0 ]; then + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). Measured on a tree + # carrying a real unrouted method: full run FAIL — 1, docs-only diff + # PASS. The routes were all filtered out by `_in_scope` and the gate + # printed the same word as a complete cross-check. See _skip_empty_scope. + _skip_empty_scope 14 "route-reachability" "routed controller method (an entry in appinfo/routes.php whose target class is in scope)" + elif [ "${_rr_fail}" -eq 0 ]; then _pass 14 "route-reachability" else _fail 14 "route-reachability" "${_rr_fail} unrouted method(s) or wrong-target route(s) — see ${_rr_log}" @@ -3066,15 +3313,23 @@ if [ -d lib ] || [ -d src ]; then _sc_ran=0 _sc_why=$(head -3 "${_sc_err}" 2>/dev/null | tr '\n' ' ' | cut -c1-200) _skip 16 "spec-coverage" wiring "check_spec_coverage.py did not complete — it never printed its terminal '# count=' marker, so NO changed method was inspected and @spec traceability (ADR-003/ADR-020) is UNVERIFIED by this run. Checker output: ${_sc_why:-}. See ${_sc_err}." - elif [ "${SCOPE_TO_DIFF}" != "1" ]; then - # The checker RAN and finished — but on a full-repo run it had no + elif [ "${HAVE_DELTA_BASE}" != "1" ]; then + # `HAVE_DELTA_BASE`, NOT `SCOPE_TO_DIFF`. gate-16 is a DELTA gate: + # what it needs is a BASE, not a narrowed file list. Keying it on + # the file scope would have made it NOT APPLICABLE on every PR in + # the fleet the moment full scope became the default — silently + # retiring @spec enforcement to buy inherited-debt coverage it + # cannot use. With a base present it judges the change set exactly + # as before, while every state gate around it reads the whole tree. + # + # The checker RAN and finished — but with no base it had no # diff to scope to, so `# count=0` means "nothing was read", not # "nothing is wrong". Category MUST be one of na|structural|wiring: # `_skip` treats anything else as an internal-error FAIL, which is # exactly the false RED this change exists to prevent (my first # draft passed `scope` and would have done that fleet-wide). _sc_ran=0 - _skip 16 "spec-coverage" na "full-repo run computed NO diff, and gate-16 is diff-scoped by design (ADR-020) — so NO changed method was inspected and @spec traceability is UNVERIFIED by this run. This is NOT a pass. Re-measure with an explicit base (HYDRA_GATE_BASE_REF=origin/beta) or run with --scope-to-diff." + _skip 16 "spec-coverage" na "this run has NO delta base, and gate-16 is a DELTA gate — it asks which methods THIS CHANGE added or modified without an @spec anchor, which a checkout cannot answer. NO changed method was inspected and @spec traceability is UNVERIFIED by this run. This is NOT a pass, and it is not counted as one. Give it a base to judge (--base or HYDRA_GATE_BASE_REF=origin/beta); the file scope is independent and every state gate still read the whole tree." fi sed -i '/^# count=[0-9]*$/d' "${_sc_log}" 2>/dev/null || true _sc_fail=$(wc -l < "${_sc_log}" 2>/dev/null || echo 0) @@ -3146,10 +3401,30 @@ if [ "${SCOPE_TO_DIFF}" = "1" ] && [ -n "${CHANGED_FILES}" ]; then printf '%s\n' "${CHANGED_FILES}" > "${_redundant_scope_file}" 2>/dev/null \ && _redundant_args+=("--changed-files-file=${_redundant_scope_file}") fi +# IS THERE ANYTHING FOR THE CHECKER TO READ AT ALL? (.github#374) +# +# Computed here and USED BELOW, after the checker has run — never instead of +# running it. Declining early would make `SKIPPED (wiring)` unreachable for +# gate-17: nothing would execute, so nothing could crash, and a crashed checker +# would be reported as not-applicable. That regression was caught once already +# by test_gate_crashed_checker_is_not_a_finding.sh (#364's first draft), and +# the repair is the same one Ruben made then — ALWAYS RUN THE CHECKER, EVALUATE +# WIRING FIRST AND SCOPE SECOND. +_redundant_empty_scope=0 +if [ "${SCOPE_TO_DIFF}" = "1" ] \ + && ! printf '%s\n' "${CHANGED_FILES}" | grep -qE '^lib/.*\.php$'; then + _redundant_empty_scope=1 +fi _redundant_args+=("${APP_DIR}") if python3 "${SCRIPT_DIR_REDUNDANT}/lib/detect-redundant-controllers.py" \ "${_redundant_args[@]}" > "${_redundant_log}" 2>&1; then - _pass 17 "redundant-controller" + # WIRING FIRST (it exited 0, so it ran), THEN SCOPE. `# count=0` over an + # empty scope is "nothing was read", not "nothing is wrong" (.github#374). + if [ "${_redundant_empty_scope}" -eq 1 ]; then + _skip_empty_scope 17 "redundant-controller" "PHP file under lib/ (ADR-022 pass-through wrappers live in lib/Controller and lib/Service)" + else + _pass 17 "redundant-controller" + fi elif ! grep -q '^# count=' "${_redundant_log}" 2>/dev/null; then # THE CHECKER DID NOT FINISH. Its terminal `# count=` marker is absent, # so it never reached its own summary — E2BIG, a traceback, an OOM kill, a @@ -3174,7 +3449,14 @@ elif grep -q '^# count=0$' "${_redundant_log}" 2>/dev/null; then # (349/349 tests pass, all 19 gates green per its own self-report) # but Rule 0b's re-run hit the script's zero-finding non-zero exit # and burned 40 turns + $1+ on a fix loop with no fixable code. - _pass 17 "redundant-controller" + # + # Same scope test as the exit-0 arm above: a `# count=0` produced over an + # empty scope is not a clean sheet (.github#374). + if [ "${_redundant_empty_scope}" -eq 1 ]; then + _skip_empty_scope 17 "redundant-controller" "PHP file under lib/ (ADR-022 pass-through wrappers live in lib/Controller and lib/Service)" + else + _pass 17 "redundant-controller" + fi else # `grep -c` prints the count (0 on no match); the old `|| echo 0` # appended a second "0" on the zero-match exit-1, so the failure @@ -3299,6 +3581,17 @@ if [ "${_nd_is_engine}" = "0" ] && [ -d lib ]; then fi _nd_warn=$(wc -l < "${_nd_warn_log}" 2>/dev/null || echo 0) +if [ "${_nd_ran}" -eq 1 ] && [ -z "${_nd_register_files}" ]; then + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). + # + # Half (a) — the only half that decides this verdict — runs solely when + # `_nd_register_files` is non-empty. With it empty the whole `if` above was + # skipped, `_nd_fail` stayed 0, and the gate printed PASS having opened no + # register file at all. The advisory half (b) is unaffected: it is pure + # bash, it still ran, and it still prints its WARNING line below. + _nd_ran=0 + _skip_empty_scope 18 "notification-dialect" "register JSON under lib/Settings (*register*.json or register.d/*.json)" +fi if [ "${_nd_ran}" -eq 1 ]; then if [ "${_nd_fail}" -eq 0 ]; then _pass 18 "notification-dialect" @@ -3513,6 +3806,12 @@ if [ -d lib ]; then : > "${_or_log}.err" _or_hits=0 _or_ran=1 + # COUNT WHAT WAS OPENED (.github#374). `_or_hits` counts FINDINGS, so zero + # findings over zero files printed the same PASS as zero findings over + # every tracked PHP file in lib/. Found by sweeping the gate table rather + # than from the issue: gate-20 is NOT one of the sixteen #374 lists, and + # neither is gate-14 — the enumeration in that issue is a floor. + _or_inspected=0 _or_broken="" _or_mask="${SCRIPT_DIR}/lib/source_scope.py" # Receiver-anchored: the call must be made ON something named @@ -3524,6 +3823,7 @@ if [ -d lib ]; then [ -z "${_file}" ] && continue [ -f "${_file}" ] || continue _in_scope "${_file}" || continue + _or_inspected=$((_or_inspected + 1)) # Comments blanked BEFORE the search (#294). Offsets and newlines are # preserved by php_mask, so grep's -n line numbers still address the # real file. @@ -3564,6 +3864,9 @@ if [ -d lib ]; then done < <(_enum_tracked '\.php$' lib) if [ "${_or_ran}" -eq 0 ]; then _skip 20 "or-objectservice-api" wiring "the comment mask or grep did NOT complete on ${_or_broken} — no call site was judged from that file onward. Calls to methods that do not exist on OpenRegister's ObjectService are UNVERIFIED by this run. See ${_or_log}.err." + elif [ "${_or_inspected}" -eq 0 ]; then + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). See _skip_empty_scope. + _skip_empty_scope 20 "or-objectservice-api" "tracked PHP file under lib/" elif [ "${_or_hits}" -eq 0 ]; then _pass 20 "or-objectservice-api" else @@ -3593,6 +3896,11 @@ fi _cm_log=${HYDRA_GATE_LOG_DIR}/hydra-gate-conflict-markers.log : > "${_cm_log}" _cm_hits=0 +# COUNT WHAT WAS OPENED (.github#374). `_cm_hits` counts FINDINGS, and zero +# findings over zero files printed the same PASS as zero findings over the +# whole tree — measured on a fixture holding a real `<<<<<<<` marker: +# full run FAIL — 1, docs-only diff PASS. +_cm_inspected=0 # Match git's exact marker shapes: `<<<<<<< ` / `======= ` (end of line OK) / `>>>>>>> ` # at the start of a line. Length is exactly 7 chars of the marker glyph; the # trailing content for << / >> is a ref name, ======= can be bare. @@ -3600,6 +3908,7 @@ while IFS= read -r _file; do [ -z "${_file}" ] && continue [ ! -f "${_file}" ] && continue _in_scope "${_file}" || continue + _cm_inspected=$((_cm_inspected + 1)) # grep -l would short-circuit but we want a line count for the log. _matches=$(grep -nE '^(<{7}[[:space:]]|>{7}[[:space:]]|={7}$)' "${_file}" 2>/dev/null || true) [ -z "${_matches}" ] && continue @@ -3611,7 +3920,10 @@ done < <(find lib appinfo src tests openspec l10n appinfo \ -o -name '*.vue' -o -name '*.json' -o -name '*.md' \ -o -name '*.yaml' -o -name '*.yml' -o -name '*.xml' \) \ 2>/dev/null) -if [ "${_cm_hits}" -eq 0 ]; then +if [ "${_cm_inspected}" -eq 0 ]; then + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). See _skip_empty_scope. + _skip_empty_scope 21 "conflict-markers" "source file under lib|appinfo|src|tests|openspec|l10n (php|js|ts|vue|json|md|yaml|yml|xml)" +elif [ "${_cm_hits}" -eq 0 ]; then _pass 21 "conflict-markers" else _fail 21 "conflict-markers" "${_cm_hits} file(s) with unresolved conflict markers — see ${_cm_log}" @@ -3669,11 +3981,17 @@ if [ -f src/manifest.json ]; then _mv_log=${HYDRA_GATE_LOG_DIR}/hydra-gate-manifest-validation.log : > "${_mv_log}" _mv_validator="${SCRIPT_DIR}/lib/check_manifest.js" - # Diff-scope: when --scope-to-diff is set and src/manifest.json was NOT - # touched in this PR, the gate runs informationally (PASS without - # spending time on a clean manifest the PR didn't touch). + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). + # + # This arm used to be a bare `_pass 22`, with a comment calling it "runs + # informationally (PASS without spending time on a clean manifest the PR + # didn't touch)". There is no such thing as an informational PASS: the line + # printed is `[gate-22] manifest-validation: PASS`, byte-identical to a run + # that put the manifest through the canonical validator, and `bin/hydra-gates` + # counts it as an applicable gate that ran. Measured on a fixture whose + # manifest fails validation: full run FAIL, docs-only diff PASS. if [ "${SCOPE_TO_DIFF}" = "1" ] && ! _in_scope "src/manifest.json"; then - _pass 22 "manifest-validation" + _skip_empty_scope 22 "manifest-validation" "src/manifest.json" elif [ ! -f "${_mv_validator}" ]; then _fail 22 "manifest-validation" "vendored validator missing at ${_mv_validator} — gate misconfiguration, fail-closed" elif ! command -v node >/dev/null 2>&1; then @@ -4394,7 +4712,11 @@ _gi_new_count=0 # misdescribes the diff, and folding it into "rules checked" would claim a # lookup that never happened. See .github#293. _gi_neg_count=0 -if [ "${SCOPE_TO_DIFF}" = "1" ] && [ -f .gitignore ]; then +# `HAVE_DELTA_BASE`, NOT `SCOPE_TO_DIFF` — gate-29 is a DELTA gate. It reads the +# `+` lines of the .gitignore diff, so what it needs is a BASE, not a narrowed +# file list. Keyed on the file scope it would have gone NOT APPLICABLE on every +# PR the moment full scope became the default. +if [ "${HAVE_DELTA_BASE}" = "1" ] && [ -f .gitignore ]; then # Lines newly added to .gitignore in this PR (excluding blanks + comments) _new_ignores=$(git -c safe.directory='*' diff "${BASE_REF}...HEAD" -- .gitignore 2>/dev/null \ | grep -E '^\+[^+]' | sed 's/^+//' | grep -vE '^\s*(#|$)' || true) @@ -4474,8 +4796,8 @@ elif [ "${_gi_ran}" -eq 1 ] && [ "${_gi_neg_count}" -gt 0 ]; then # "adds no non-comment line" here (the branch below) would misdescribe the # diff, and PASS would claim a lookup that never happened. _skip 29 "gitignore-then-commit" na "the diff adds ${_gi_neg_count} .gitignore line(s) and every one is a NEGATION (\`!\`), which un-ignores rather than ignores. A tracked file behind a negation is the intended state, so there is no new ignore rule whose path could already be tracked. See .github#293." -elif [ "${SCOPE_TO_DIFF}" != "1" ]; then - _skip 29 "gitignore-then-commit" na "this check is intrinsically diff-relative — it asks whether THIS change adds an ignore rule over files that are already tracked — and this run is not --scope-to-diff, so there is no set of newly-added rules to look up. Nothing was inspected and nothing could be. It runs on every PR." +elif [ "${HAVE_DELTA_BASE}" != "1" ]; then + _skip 29 "gitignore-then-commit" na "this check is intrinsically diff-relative — it asks whether THIS change adds an ignore rule over files that are already tracked — and this run has NO delta base, so there is no set of newly-added rules to look up. Nothing was inspected and nothing could be, and no change to this repository could make a baseless run able to answer it. It runs on every PR, at any file scope." elif [ ! -f .gitignore ]; then _skip 29 "gitignore-then-commit" na "this repository has no .gitignore, so this diff cannot have added an ignore rule over already-tracked files." else @@ -5104,7 +5426,9 @@ if _a11y_has_markup_dir; then -o -name '*.php' -o -name '*.html' -o -name '*.htm' \) 2>/dev/null \ | grep -vE '(^|/)(node_modules|vendor|dist|build|coverage|phpmetrics|\.git)/' || true) if [ "${#_wc_files[@]}" -eq 0 ]; then - : # nothing in scope; the verdict below describes the diff. + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). See _skip_empty_scope. + _wc_ran=0 + _skip_empty_scope 34 "window-confirm" "frontend source file (src|templates|appinfo/templates **/*.vue|js|ts|php|html)" elif [ ! -f "${_wc_helper}" ]; then _wc_ran=0 _skip 34 "window-confirm" wiring "check_js_call_sites.py not found at ${_wc_helper} — ${#_wc_files[@]} file(s) were in scope and NONE were inspected; native browser dialogs are UNVERIFIED by this run." @@ -5148,8 +5472,13 @@ fi if _a11y_has_markup_dir; then _iae_log=${HYDRA_GATE_LOG_DIR}/hydra-gate-img-alt-empty-only.log : > "${_iae_log}" + # COUNT WHAT WAS OPENED (.github#374). This gate greps rather than calling + # a helper, so it had no file array to test for emptiness and fell through + # to `_pass 35` having read nothing. + _iae_inspected=0 while IFS= read -r vue; do _in_scope "${vue}" || continue + _iae_inspected=$((_iae_inspected + 1)) _flat=$(tr '\n' ' ' < "${vue}") echo "${_flat}" \ | grep -oE ']*>' 2>/dev/null \ @@ -5177,7 +5506,10 @@ if _a11y_has_markup_dir; then done done < <(_a11y_markup_files) _iae_fail=$(wc -l < "${_iae_log}" 2>/dev/null || echo 0) - if [ "${_iae_fail}" -eq 0 ]; then + if [ "${_iae_inspected}" -eq 0 ]; then + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). See _skip_empty_scope. + _skip_empty_scope 35 "img-alt-empty-only" "markup file (src|templates|appinfo/templates **/*.vue|php|html)" + elif [ "${_iae_fail}" -eq 0 ]; then _pass 35 "img-alt-empty-only" else _fail 35 "img-alt-empty-only" "${_iae_fail} \"\" on semantic-bound src — see ${_iae_log}" @@ -5214,8 +5546,25 @@ if _a11y_has_markup_dir; then --include='*.vue' --include='*.js' --include='*.ts' \ --include='*.php' --include='*.html' --include='*.htm' 2>/dev/null \ | _filter_grep_by_scope >> "${_tp_log}" || true + # COUNT WHAT WAS OPENED (.github#374). `grep -r` gives no file list back, + # so emptiness is measured against the SAME corpus the grep walks — the + # identical directory set and --include list, not the shared + # `_a11y_markup_files` (which omits .js/.ts and would call this gate empty + # over a repo whose only frontend is JavaScript). + _tp_inspected=0 + while IFS= read -r _f; do + [ -z "${_f}" ] && continue + _in_scope "${_f}" || continue + _tp_inspected=$((_tp_inspected + 1)) + done < <(find src templates appinfo/templates -type f \ + \( -name '*.vue' -o -name '*.js' -o -name '*.ts' \ + -o -name '*.php' -o -name '*.html' -o -name '*.htm' \) 2>/dev/null \ + | grep -vE '(^|/)(node_modules|vendor|dist|build|coverage|phpmetrics|\.git)/' || true) _tp_fail=$(wc -l < "${_tp_log}" 2>/dev/null || echo 0) - if [ "${_tp_fail}" -eq 0 ]; then + if [ "${_tp_inspected}" -eq 0 ]; then + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). See _skip_empty_scope. + _skip_empty_scope 36 "tabindex-positive" "frontend source file (src|templates|appinfo/templates **/*.vue|js|ts|php|html)" + elif [ "${_tp_fail}" -eq 0 ]; then _pass 36 "tabindex-positive" else _fail 36 "tabindex-positive" "${_tp_fail} positive tabindex value(s) — use \"0\" or \"-1\" — see ${_tp_log}" @@ -5272,7 +5621,9 @@ if _a11y_has_markup_dir; then _ahf_files+=("${vue}") done < <(_a11y_markup_files) if [ "${#_ahf_files[@]}" -eq 0 ]; then - : # nothing in scope; the PASS below describes the diff, as everywhere else. + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). See _skip_empty_scope. + _ahf_ran=0 + _skip_empty_scope 37 "aria-hidden-focusable" "markup file (src|templates|appinfo/templates **/*.vue|php|html)" elif [ ! -f "${_ahf_helper}" ]; then # A MISSING HELPER MUST NOT REPORT PASS (#147). _ahf_ran=0 @@ -5395,9 +5746,18 @@ if [ -d src ] || [ -d templates ] || [ -d appinfo/templates ]; then # have reported PASS on nothing. One `--classify` call for the whole set # prints `: page-root|fragment`, and a non-zero exit is a wiring # failure rather than an answer. + # HOW MANY SUBJECTS THIS GATE ACTUALLY OPENED (.github#374). + # + # gate-38 has two arms — Vue page roots and document-owning PHP templates — + # and neither counted. With both empty the gate fell through to `_pass 38`, + # which is why a docs-only diff over a tree carrying a planted skip-link + # defect printed the same word as a clean sweep. Incremented AFTER the + # `_in_scope` guard, so it counts files judged, not files enumerated. + _sl_inspected=0 _sl_check() { local _f="$1" _in_scope "$_f" || return 0 + _sl_inspected=$((_sl_inspected + 1)) if grep -qE '/dev/null; then return 0; fi # The shared app shell. `` (@conduction/nextcloud-vue) IS an # — it renders one as its own root element and puts the @@ -5453,7 +5813,10 @@ if [ -d src ] || [ -d templates ] || [ -d appinfo/templates ]; then _in_scope "$_f" && _sl_php+=("$_f") done < <(_a11y_markup_files) if [ "${#_sl_php[@]}" -eq 0 ]; then - : # nothing in scope; the verdict below describes the diff, as everywhere else. + # No PHP template in scope. NOT a verdict on its own — the Vue arm + # above may still have judged something, so the decision is taken + # once, from `_sl_inspected`, below (.github#374). + : elif [ ! -f "${_sl_helper}" ]; then # A MISSING HELPER MUST NOT REPORT PASS (#147). Without the # classifier every template silently reads as a fragment, the @@ -5477,6 +5840,11 @@ if [ -d src ] || [ -d templates ] || [ -d appinfo/templates ]; then _sl_ran=0 _skip 38 "skip-link" wiring "php_template_scope.py exited ${_sl_rc} — ${#_sl_php[@]} PHP template(s) were in scope and NONE were classified; a document-owning template with no bypass mechanism (WCAG 2.4.1) is UNVERIFIED by this run. See ${_sl_cls_err}." else + # Every template handed to the classifier WAS inspected — a + # `fragment` verdict is a judgement, not a skip. Counting only + # page roots here would make an app whose 40 templates are all + # fragments look like a gate that opened nothing. + _sl_inspected=$((_sl_inspected + ${#_sl_php[@]})) while IFS= read -r _line; do case "${_line}" in *": page-root") _sl_check "${_line%: page-root}" ;; @@ -5485,6 +5853,12 @@ if [ -d src ] || [ -d templates ] || [ -d appinfo/templates ]; then fi fi fi + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). Taken once, over BOTH + # arms, so a gate that judged a Vue root but no template still reports. + if [ "${_sl_ran}" -eq 1 ] && [ "${_sl_inspected}" -eq 0 ]; then + _sl_ran=0 + _skip_empty_scope 38 "skip-link" "app page root (src/App.vue, src/**/*Root.vue) or document-owning PHP template (templates|appinfo/templates **/*.php)" + fi _sl_fail=$(wc -l < "${_sl_log}" 2>/dev/null || echo 0) if [ "${_sl_ran}" -eq 1 ]; then if [ "${_sl_fail}" -eq 0 ]; then @@ -5550,7 +5924,9 @@ if _a11y_has_markup_dir; then _bn_files+=("${vue}") done < <(_a11y_markup_files) if [ "${#_bn_files[@]}" -eq 0 ]; then - : # nothing in scope; the PASS below describes the diff, as everywhere else. + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). See _skip_empty_scope. + _bn_ran=0 + _skip_empty_scope 39 "button-name" "markup file (src|templates|appinfo/templates **/*.vue|php|html)" elif [ ! -f "${_bn_helper}" ]; then # A MISSING HELPER MUST NOT REPORT PASS (#147). _bn_ran=0 @@ -5623,15 +5999,32 @@ if _a11y_has_markup_dir; then _in_scope "${vue}" || continue _fl_files+=("${vue}") done < <(_a11y_markup_files) - # NOTE: an empty in-scope set is NOT declared `na` here. Every other gate - # in this family reports PASS over an empty diff scope — that is what - # ADR-020 diff scoping MEANS — and tests/test-hydra-gates-bin.sh asserts - # that none of gates 10/12/13/26/31..45 goes NOT APPLICABLE while src/ - # exists. Making gate-40 alone answer differently would drift the - # applicability table away from the guards it mirrors, which is the one - # way that change could hide a live gate. + # THE NOTE THAT USED TO SIT HERE WAS FALSE ON BOTH OF ITS CLAIMS, AND THE + # SAME RUN REFUTED BOTH (.github#374). + # + # It said an empty in-scope set must NOT be declared `na` here, because + # (1) "every other gate in this family reports PASS over an empty diff + # scope" and (2) tests/test-hydra-gates-bin.sh "asserts that none of gates + # 10/12/13/26/31..45 goes NOT APPLICABLE while src/ exists". + # + # (1) is false: gates 26, 31 and 32 already reported a gate-level `na` + # WITH A REASON on exactly this input, and had done since #276. + # (2) misreads the assertion it cites. That loop matches the + # APPLICABILITY TABLE's own wording — + # `NOT APPLICABLE — no src/(,| directory)` — and exists to catch the + # table excusing a gate whose guard would have run it. A gate-level + # `na` naming its OWN empty file set does not match that pattern and + # passes the assertion unharmed. The very next assertion in that file + # accepts `[gate-45] prefers-reduced-motion: NOT APPLICABLE — scope + # was empty` as the CORRECT answer, from the same family. + # + # So the note argued for keeping the defect on the strength of a sibling + # that does not have it and a test that does not say what it was quoted as + # saying. Both are checked now rather than asserted. if [ "${#_fl_files[@]}" -eq 0 ]; then - : # nothing in scope; the PASS below describes the diff, as everywhere else. + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). See _skip_empty_scope. + _fl_ran=0 + _skip_empty_scope 40 "form-label-association" "markup file (src|templates|appinfo/templates **/*.vue|php|html)" elif [ ! -f "${_fl_helper}" ]; then # A MISSING HELPER MUST NOT REPORT PASS (#147). _fl_ran=0 @@ -5731,7 +6124,9 @@ if [ -d templates ] || [ -d appinfo/templates ]; then _in_scope "$_f" && _hl_files+=("$_f") done < <(_a11y_markup_files) if [ "${#_hl_files[@]}" -eq 0 ]; then - : # nothing in scope; the verdict below describes the diff. + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). See _skip_empty_scope. + _hl_ran=0 + _skip_empty_scope 41 "html-lang" "PHP template (templates|appinfo/templates **/*.php)" elif [ ! -f "${_hl_helper}" ]; then _hl_ran=0 _skip 41 "html-lang" wiring "php_template_scope.py not found at ${_hl_helper} — ${#_hl_files[@]} PHP template(s) were in scope and NONE were inspected; a document without a declared language (WCAG 3.1.1) is UNVERIFIED by this run." @@ -5797,7 +6192,9 @@ if _a11y_has_markup_dir; then _lq_files+=("${vue}") done < <(_a11y_markup_files) if [ "${#_lq_files[@]}" -eq 0 ]; then - : # nothing in scope; the PASS below describes the diff, as everywhere else. + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). See _skip_empty_scope. + _lq_ran=0 + _skip_empty_scope 42 "link-text-quality" "markup file (src|templates|appinfo/templates **/*.vue|php|html)" elif [ ! -f "${_lq_helper}" ]; then # A MISSING HELPER MUST NOT REPORT PASS (#147). _lq_ran=0 @@ -5874,7 +6271,9 @@ if _a11y_has_markup_dir; then _th_files+=("${vue}") done < <(_a11y_markup_files) if [ "${#_th_files[@]}" -eq 0 ]; then - : # nothing in scope; the PASS below describes the diff, as everywhere else. + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). See _skip_empty_scope. + _th_ran=0 + _skip_empty_scope 43 "table-headers" "markup file (src|templates|appinfo/templates **/*.vue|php|html)" elif [ ! -f "${_th_helper}" ]; then # A MISSING HELPER MUST NOT REPORT PASS (#147). _th_ran=0 @@ -5943,7 +6342,9 @@ if _a11y_has_markup_dir; then _ac_files+=("${vue}") done < <(_a11y_markup_files) if [ "${#_ac_files[@]}" -eq 0 ]; then - : # nothing in scope; the PASS below describes the diff, as everywhere else. + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). See _skip_empty_scope. + _ac_ran=0 + _skip_empty_scope 44 "autocomplete-attr" "markup file (src|templates|appinfo/templates **/*.vue|php|html)" elif [ ! -f "${_ac_helper}" ]; then # A MISSING HELPER MUST NOT REPORT PASS (#147). _ac_ran=0 @@ -6316,7 +6717,7 @@ _scht_log=${HYDRA_GATE_LOG_DIR}/hydra-gate-security-change-has-tests.log : > "${_scht_log}" _scht_ran=1 _scht_helper="${SCRIPT_DIR}/lib/check_security_cochange.py" -if [ "${SCOPE_TO_DIFF}" != "1" ] || [ -z "${BASE_REF}" ]; then +if [ "${HAVE_DELTA_BASE}" != "1" ]; then # NO DIFF, NO VERDICT — and NOT a PASS (#242/#240/#258/#268). # # This gate's whole subject is "what did this change touch, and did it also @@ -6324,9 +6725,14 @@ if [ "${SCOPE_TO_DIFF}" != "1" ] || [ -z "${BASE_REF}" ]; then # `if` above was simply false and the gate fell through to `_pass 47`, # announcing a co-change verdict it had not formed. Every full-repo run in # the fleet — every builder iteration — printed that green. + # + # Keyed on `HAVE_DELTA_BASE`, not on the FILE scope: gate-47 is a DELTA + # gate and needs a base, not a narrowed file list. A full-scope PR run + # still has one, so this gate did not quietly retire when full scope + # became the default (.github#374). _scht_ran=0 - _skip 47 "security-change-has-tests" na "this run is not diff-scoped (no --scope-to-diff / no base ref), so there is no change set to classify. This gate compares a PR's hunks against the tests it touched; on a whole-repository run there is no such pair, and no change in this repository could create one." -elif [ "${SCOPE_TO_DIFF}" = "1" ] && [ -n "${BASE_REF}" ]; then + _skip 47 "security-change-has-tests" na "this run has NO delta base, so there is no change set to classify. This gate compares a change's hunks against the tests it touched; with no base there is no such pair, and no change in this repository could create one. Give it a base (--base or HYDRA_GATE_BASE_REF) and it runs at any file scope." +elif [ "${HAVE_DELTA_BASE}" = "1" ]; then if [ ! -f "${_scht_helper}" ]; then # A MISSING HELPER MUST NOT REPORT PASS (#147). _scht_ran=0 @@ -6405,13 +6811,14 @@ fi _csrf_log=${HYDRA_GATE_LOG_DIR}/hydra-gate-csrf-cochange.log : > "${_csrf_log}" _csrf_ran=1 -if [ "${SCOPE_TO_DIFF}" != "1" ] || [ -z "${BASE_REF}" ]; then +if [ "${HAVE_DELTA_BASE}" != "1" ]; then # NO DIFF, NO VERDICT — and NOT a PASS (#242/#240/#258/#268). Identical # reasoning to gate-47 above: "was an attribute REMOVED" is a question only # a diff can answer, and a full-repo run printed PASS without asking it. + # Keyed on the delta base, not the file scope, for the same reason. _csrf_ran=0 - _skip 48 "csrf-cochange" na "this run is not diff-scoped (no --scope-to-diff / no base ref), so there is no removal to detect. Whether a controller DROPPED @NoCSRFRequired is a property of a change set, not of a checkout, and no change in this repository could make a whole-repository run able to answer it." -elif [ "${SCOPE_TO_DIFF}" = "1" ] && [ -n "${BASE_REF}" ]; then + _skip 48 "csrf-cochange" na "this run has NO delta base, so there is no removal to detect. Whether a controller DROPPED @NoCSRFRequired is a property of a CHANGE SET, not of a checkout, and no change in this repository could make a baseless run able to answer it. Give it a base (--base or HYDRA_GATE_BASE_REF) and it runs at any file scope." +elif [ "${HAVE_DELTA_BASE}" = "1" ]; then # Find removed @NoCSRFRequired lines in changed PHP files. # # A REMOVED COMMENT IS NOT A REMOVED ATTRIBUTE (#191). @@ -7068,9 +7475,29 @@ fi # reported so migrations can show the number shrinking). _cwr_counts=$(grep -m1 -o 'base=[0-9]* head=[0-9]* delta=[+-]*[0-9]*.*' "${_cwr_log}" 2>/dev/null || true) [ -n "${_cwr_counts}" ] && echo "[gate-52] custom-widget-ratchet: ${_cwr_counts}" +# THE RATCHET HALF IS NOT ALWAYS COMPUTED, AND IT USED TO SAY SO NOWHERE +# (.github#374). check_custom_widget_ratchet.py computes base-vs-head only when +# it is given a base; without one it runs the JUSTIFICATION half alone and +# prints no `base=/head=/delta=` line at all. The gate could then print PASS +# while the mechanic it is NAMED after had not run. Not prefixed `[gate-` so no +# `^\[gate-` consumer reads it as a verdict — same convention as gate-31's. +if [ "${#_cwr_files[@]}" -gt 0 ] && [ "${_cwr_ran}" -eq 1 ] && [ -z "${_cwr_counts}" ]; then + echo "[hydra-gates] gate-52 custom-widget-ratchet: the RATCHET half was NOT computed — the helper printed no base/head/delta counts, which it does only when it has a base ref to compare against. The JUSTIFICATION half (every custom kind:\"widget\" entry needs a \`_note\`) did run over ${#_cwr_files[@]} file(s). A PASS below covers that half only." +fi if [ "${#_cwr_files[@]}" -eq 0 ]; then # AN UNOPENED SCOPE IS NEVER A PASS (#242/#240/#258/#268). _skip 52 "custom-widget-ratchet" na "no src/**/*.{js,ts,vue} file(s) in this repo, so there is no component registry to hold a custom kind:\"widget\" entry and no ratchet to compute." +elif [ "${SCOPE_TO_DIFF}" = "1" ] && ! printf '%s\n' "${CHANGED_FILES}" | grep -qE '^src/.*\.(js|ts|vue)$'; then + # AN UNOPENED SCOPE IS NEVER A PASS (.github#374). + # + # `_cwr_files` is deliberately NOT `_in_scope`-filtered — the ratchet count + # is app-wide — so on a narrowed run this array is non-empty even when the + # diff contains no frontend file at all. The helper then self-scopes its + # justification half to changed entries, finds none, prints `findings=0`, + # and the gate printed PASS having judged nothing. The array being full is + # what made this one invisible: every other gate in this class gave itself + # away with an empty file list. + _skip_empty_scope 52 "custom-widget-ratchet" "src/**/*.{js,ts,vue} file (a component registry declaring kind:\"widget\" entries)" elif [ "${_cwr_ran}" -eq 1 ]; then if [ "${_cwr_fail}" -eq 0 ]; then _pass 52 "custom-widget-ratchet" @@ -7297,7 +7724,20 @@ if [ -f src/manifest.json ]; then # is prevention rather than a burn-down list nobody can close. # ------------------------------------------------------------------- _em_orphaned="" - if [ "${SCOPE_TO_DIFF}" = "1" ] && [ -n "${BASE_REF}" ] && [ "${_em_warns}" -gt 0 ]; then + # THIS SUB-CHECK IS A DELTA HALF INSIDE A STATE GATE, AND ITS ABSENCE + # USED TO BE SILENT (.github#374). + # + # "Did THIS change remove the last manifest reference to a component" + # is unanswerable without a base, so with none it simply does not run — + # and gate-53 could still print PASS, saying nothing about the half + # that did not execute. Keyed on `HAVE_DELTA_BASE` (the file scope is + # irrelevant to it) and, when it cannot run over live WARNs, it says so + # on stdout. Not prefixed `[gate-` so no `^\[gate-` consumer reads it + # as a verdict. + if [ "${HAVE_DELTA_BASE}" != "1" ] && [ "${_em_warns}" -gt 0 ]; then + echo "[hydra-gates] gate-53 effective-manifest-crossref: the ORPHAN-PROMOTION half was NOT computed — it asks whether this CHANGE removed the last \"component\" reference to one of the ${_em_warns} registry export(s) below, and this run has no delta base. Those WARNs stay advisory. Give the run a base (--base or HYDRA_GATE_BASE_REF) to have them judged." + fi + if [ "${HAVE_DELTA_BASE}" = "1" ] && [ -n "${BASE_REF}" ] && [ "${_em_warns}" -gt 0 ]; then set +e _em_removed_refs=$(git diff -U0 "${BASE_REF}...HEAD" -- \ 'src/manifest.json' 'src/manifest.d/*' 'src/menu-layout.json' 2>/dev/null \ @@ -7915,8 +8355,22 @@ if [ -d lib/AppInfo ]; then # unscoped, so it would have surfaced the whole backlog as blocking # findings on every build. The helper still fails CLOSED when the base # does not resolve, so an unscopable run is never reported as a clean one. - python3 "${SCRIPT_DIR}/lib/check_listener_placement.py" . --base "${BASE_REF}" > "${_lwp_log}" 2>&1 - _lwp_rc=$? + # + # `--base` IS ONLY PASSED WHEN THERE IS ONE (.github#374). `BASE_REF` may + # now legitimately be empty — the runner no longer carries a hardcoded + # `origin/development`, which is the root cause `.github#347` and `#361` + # share. Handing the helper `--base ""` would OVERRIDE its own argparse + # default with an empty string, and what it does with that is the helper's + # business, not a verdict this runner should route on. With no base we go + # straight to rc 3, which is exactly the branch below that states the + # absence honestly and runs the advisory whole-tree sweep for the size. + if [ "${HAVE_DELTA_BASE}" = "1" ]; then + python3 "${SCRIPT_DIR}/lib/check_listener_placement.py" . --base "${BASE_REF}" > "${_lwp_log}" 2>&1 + _lwp_rc=$? + else + echo "no delta base for this run — the helper was not invoked with one" > "${_lwp_log}" + _lwp_rc=3 + fi set +e if [ "${_lwp_rc}" -eq 0 ]; then _pass 61 "listener-work-placement" @@ -7951,8 +8405,13 @@ if [ -d lib/AppInfo ]; then # false-RED, and it stops the run claiming an exclusion nothing # performed. What was missing was the SIZE of what went unread, so an # advisory sweep supplies it: informational, never a verdict. - if [ "${SCOPE_TO_DIFF}" = "1" ]; then - _skip 61 "listener-work-placement" na "the diff against '${BASE_REF}' put every post-event registration out of scope, so NONE were inspected. Diff-scoped out under ADR-020 — the fleet's 149-registration backlog is a work-list, not a reason to block an unrelated PR. This gate runs on the next PR that touches a listener registration. See ${_lwp_log}." + # `HAVE_DELTA_BASE`, not `SCOPE_TO_DIFF`: what distinguishes these two + # branches is whether a diff was COMPUTED, never how the file scope was + # set. Reading the file scope here is precisely how `.github#347` got + # its false reason — a `NOT APPLICABLE` blaming a diff on a run that + # computed none. + if [ "${HAVE_DELTA_BASE}" = "1" ]; then + _skip 61 "listener-work-placement" na "the diff against '${BASE_REF}' put every post-event registration out of scope, so NONE were inspected. This gate is deliberately delta-scoped even at full file scope — the fleet's 149-registration backlog is a work-list, not a reason to block an unrelated change. It runs on the next change that touches a listener registration. See ${_lwp_log}." else # ADVISORY ONLY. Its exit status is discarded on purpose: a sweep # that finds inherited debt must not become this run's verdict, and diff --git a/hydra-gates/tests/test-hydra-gates-bin.sh b/hydra-gates/tests/test-hydra-gates-bin.sh index b8ceac83..b1cced08 100755 --- a/hydra-gates/tests/test-hydra-gates-bin.sh +++ b/hydra-gates/tests/test-hydra-gates-bin.sh @@ -25,6 +25,27 @@ SELF_DIR="$(cd "$(dirname "${BASH_SOURCE[0]:-$0}")" && pwd)" PKG_ROOT="$(cd "${SELF_DIR}/.." && pwd)" BIN="${PKG_ROOT}/bin/hydra-gates" +# --------------------------------------------------------------------------- +# THIS FILE IS THE DIFF-SCOPING CONTRACT, AND DIFF SCOPING IS NO LONGER THE +# DEFAULT (hydra-gates/ADR-020-SUPERSEDED.md). +# +# Properties 2, 3 and the whole push-base section below are statements about +# what happens WHEN A RUN IS SCOPED TO A DIFF: an unresolvable base must exit +# 99, an empty diff must be reported as empty, `github.event.before` must be +# mined when the base is HEAD. Every one of them is still true and still +# load-bearing — they just are not reached by an argument-free invocation any +# more, because the default is now full scope. +# +# Declared ONCE, here, rather than by appending `--scope-to-diff` to twenty +# invocations. A per-call edit is twenty chances to miss one, and a missed one +# does not fail: it silently starts measuring the other mode, which is exactly +# the class of defect this file exists to catch. +# +# ⚠️ The DEFAULT is asserted separately, at the bottom of this file, with this +# variable explicitly unset. Without that section this line would hide the +# reversal instead of documenting it. +export HYDRA_GATE_SCOPE=diff + PASS=0 FAIL=0 _ok() { echo " ok — $1"; PASS=$((PASS + 1)); } @@ -1004,6 +1025,141 @@ else _bad "exit 99 — the null push base was consulted even though origin/development was usable" fi +# =========================================================================== +echo "" +echo "[test] THE DEFAULT SCOPE IS FULL, AND THE DELTA BASE SURVIVES IT" +# =========================================================================== +# +# The reversal, asserted where a reader looks for it: everything above ran with +# HYDRA_GATE_SCOPE=diff because it is the diff contract. This section unsets it +# and measures what a caller who passes nothing actually gets. +# +# ⚠️ THE ORDER MATTERS. `unset` must happen here and the variable must not be +# restored, or a later section would silently inherit the wrong mode. This is +# the last section in the file for that reason. +unset HYDRA_GATE_SCOPE + +# 1. No flags at all -> full file scope, stated in one machine-readable line. +OUT_DEF="$("${BIN}" --app-dir "${FIX}" 2>&1)"; RC_DEF=$? +if printf '%s\n' "${OUT_DEF}" | grep -qF '[hydra-gates] SCOPE-MODE: full'; then + _ok "an argument-free invocation runs at FULL file scope and says so" +else + _bad "an argument-free invocation did not announce 'SCOPE-MODE: full' — the ADR-020 reversal is not in effect, or it is not observable, and both are the same problem for a reader" +fi + +# 2. It must NOT have exited 99. The old default resolved a base and refused +# when it could not; full scope has no such dependency, and a repo whose +# base cannot be resolved must still get 59 real verdicts. +if [ "${RC_DEF}" -ne 99 ]; then + _ok "a full-scope run does not exit 99 over base resolution (rc=${RC_DEF} is a gate count)" +else + _bad "a full-scope run exited 99 — base resolution is still fatal at full scope, so an unresolvable base still costs every gate instead of only the five delta gates" +fi + +# 3. THE POINT OF THE WHOLE CHANGE: inherited debt is judged. +# +# ⚠️ ON ITS OWN TREE, NOT ON `${FIX}`. Written against `${FIX}` first, this +# assertion reported 0 vs 0 and refused to grade — correctly: by this point +# in the file `${FIX}` has been mutated by the sections above and carries no +# inherited violation for full scope to find. That refusal is the assertion +# working (a control must perturb something the metric actually counts), and +# it is why the tree is built here instead of borrowed. +# +# Shape: commit 1 carries a real gate-1 violation (a lib/ PHP file with no +# @license/@copyright). Commit 2 touches only docs. Diff-scoped against +# commit 1, gate-1 is quiet — that is ADR-020 working as designed. At full +# scope it must fire. ONE tree, one commit apart, only the scope input +# changed. +DEBTFIX="${WORK}/inherited-debt" +mkdir -p "${DEBTFIX}/lib/Service" "${DEBTFIX}/appinfo" +cat > "${DEBTFIX}/lib/Service/LegacyService.php" <<'PHP' + "${DEBTFIX}/appinfo/routes.php" +printf 'notes\n' > "${DEBTFIX}/README.md" +( + cd "${DEBTFIX}" || exit 1 + git init -q . + git config user.email fixture@example.invalid + git config user.name "Gate Fixture" + git config commit.gpgsign false + git add -f . + git commit -qm "base: a service carrying inherited SPDX debt" +) >/dev/null 2>&1 +DEBT_BASE="$(git -C "${DEBTFIX}" rev-parse HEAD)" +( + cd "${DEBTFIX}" || exit 1 + printf 'notes\nmore notes\n' > README.md + git add README.md + git commit -qm "docs: an unrelated change" +) >/dev/null 2>&1 + +OUT_DEBT_FULL="$("${BIN}" --app-dir "${DEBTFIX}" 2>&1)" +OUT_DEBT_DIFF="$(env HYDRA_GATE_SCOPE=diff "${BIN}" --app-dir "${DEBTFIX}" --base "${DEBT_BASE}" 2>&1)" + +# POSITIVE CONTROL FIRST: name the gate, do not just count. A count going up +# for some other reason would read as the reversal working. +if printf '%s\n' "${OUT_DEBT_FULL}" | grep -qE '^\[gate-1\][^:]*: FAIL'; then + _ok "full scope FAILS gate-1 on inherited SPDX debt the diff never touched" +else + _bad "full scope did not fail gate-1 on a lib/ PHP file with no @license — the positive control is broken, so the comparison below proves nothing. Got: $(printf '%s\n' "${OUT_DEBT_FULL}" | grep -E '^\[gate-1\]' | head -1)" +fi +if printf '%s\n' "${OUT_DEBT_DIFF}" | grep -qE '^\[gate-1\][^:]*: FAIL'; then + _bad "the DIFF-scoped run also failed gate-1 on a docs-only change set — inherited debt is leaking into the opt-in diff mode, which is the false RED ADR-020 existed to prevent and which the reversal does not license" +else + _ok "the same tree, diff-scoped one commit back, does NOT fail gate-1 — the two modes genuinely differ, and only the scope input changed" +fi + +# 4. ANTI-WIDENING, and the half most easily lost: a full-scope run WITH a base +# must still run the delta gates. If flipping the default retired gates 16, +# 29, 47, 48 and 61 on every PR in the fleet, that is a downgrade wearing an +# upgrade's clothes — and it would be invisible, because NOT APPLICABLE is +# excluded from the verdict. +OUT_FULLBASE="$("${BIN}" --app-dir "${FIX}" --base "${BASE_SHA}" 2>&1)" +if printf '%s\n' "${OUT_FULLBASE}" | grep -qE '^\[hydra-gates\] Delta base: [^N]'; then + _ok "a full-scope run still resolves and names a DELTA base" +else + _bad "a full-scope run with an explicit --base did not report a delta base — the five delta gates have been retired by the default flip" +fi +_delta_na="" +for _g in 16 47 48; do + printf '%s\n' "${OUT_FULLBASE}" | grep -qE "^\[gate-${_g}\][^:]*: NOT APPLICABLE" \ + && _delta_na="${_delta_na}${_g} " +done +if [ -z "${_delta_na}" ]; then + _ok "delta gates 16/47/48 still produce a verdict at full scope when a base is present" +else + _bad "delta gate(s) ${_delta_na}went NOT APPLICABLE at full scope though a base WAS resolved — the delta gates are keyed on the file scope instead of on the base, which retires them on every PR" +fi + +# 5. And with NO base they must decline BY NAME rather than pass. This is the +# workflow_dispatch shape, and it is the one place the reversal legitimately +# costs coverage — so it has to be stated, not absorbed. +_delta_pass="" +for _g in 16 47 48; do + printf '%s\n' "${OUT_DEF}" | grep -qE "^\[gate-${_g}\][^:]*: PASS" && _delta_pass="${_delta_pass}${_g} " +done +if [ -z "${_delta_pass}" ]; then + _ok "with no delta base, gates 16/47/48 do not report PASS" +else + _bad "delta gate(s) ${_delta_pass}reported PASS with NO base to compare against — a co-change verdict announced without a change set, and it counts toward 'N of N applicable gates ran'" +fi + +# 6. An unrecognised scope must be refused, not guessed. A run whose scope +# nobody can name is not evidence about anything, and silently defaulting is +# how the wrong mode gets measured for months. +OUT_BADSCOPE="$(HYDRA_GATE_SCOPE=whole-repo "${BIN}" --app-dir "${FIX}" 2>&1)"; RC_BADSCOPE=$? +if [ "${RC_BADSCOPE}" -eq 99 ] && printf '%s\n' "${OUT_BADSCOPE}" | grep -qF 'is not '; then + _ok "an unrecognised \$HYDRA_GATE_SCOPE is refused with exit 99 rather than silently defaulting" +else + _bad "\$HYDRA_GATE_SCOPE='whole-repo' was accepted (rc=${RC_BADSCOPE}) — an unnameable scope produced verdicts" +fi + echo "" echo "==================================================" echo "hydra-gates entry-point tests: ${PASS} passed, ${FAIL} failed" From 128d1a2478606873ed427ebcfb08d790715342f1 Mon Sep 17 00:00:00 2001 From: Ruben van der Linde Date: Wed, 12 Aug 2026 10:35:57 +0200 Subject: [PATCH 2/5] docs(gates): the runner header described the old default The Options block still said --scope-to-diff was the mode and named origin/development as the base default. Both are gone: full scope is the default, and BASE_REF starts empty because that hardcoded default is the shared root cause of .github#347 and #361. --- hydra-gates/scripts/run-hydra-gates.sh | 31 +++++++++++++++++++++----- 1 file changed, 25 insertions(+), 6 deletions(-) diff --git a/hydra-gates/scripts/run-hydra-gates.sh b/hydra-gates/scripts/run-hydra-gates.sh index 6bef2782..4219e8e0 100755 --- a/hydra-gates/scripts/run-hydra-gates.sh +++ b/hydra-gates/scripts/run-hydra-gates.sh @@ -40,13 +40,32 @@ # for apps following the standard Conduction NC app layout: lib/ + appinfo/ # + optional src/ + tests/. # +# SCOPE: FULL BY DEFAULT. ADR-020 IS SUPERSEDED — see +# hydra-gates/ADR-020-SUPERSEDED.md for the decision and what it costs. +# +# The scope is TWO independent, named inputs, and both are printed every run: +# +# FILE SCOPE which files the STATE gates open. Defaults to the entire +# tracked tree. `--scope-to-diff` narrows it to the change. +# DELTA BASE what the five DELTA gates (16, 29, 47, 48, 61) compare +# against. Resolved independently of the file scope, so a +# full-scope run on a PR still judges the change. +# # Options: -# --scope-to-diff [BASE] — Phase G: only scan files changed vs BASE -# (default origin/development). Inherited debt -# in unchanged files is ignored. Required for -# reviewer/security post-flight enforcement; -# optional for builder (build mode runs full). -# --base BRANCH — override the diff base (default origin/development) +# --full — scan the whole tree. THIS IS THE DEFAULT; the +# flag names it rather than switching to it. +# --scope-to-diff, --diff — the old ADR-020 behaviour, now OPT-IN: judge only +# the files this change touched. Inherited debt in +# unchanged files is not judged. +# --base BRANCH — the delta base. Also the file-scope base when +# --scope-to-diff is given. NO DEFAULT: the old +# hardcoded `origin/development` is the shared root +# cause of .github#347 and #361 — a base nobody +# supplied, silently diffing a branch against +# itself, and printing PASS. +# $HYDRA_GATE_SCOPE — `full` | `diff`, the env form of the two flags. +# Anything else stops the run: a run whose scope +# nobody can name is not evidence about anything. # # When the base resolves to the SAME COMMIT as HEAD — which is what every push # to a mainline branch looks like — the scope is taken from the push's own From 4120964b52474b8fe04070150bbbd08b9b37e2b2 Mon Sep 17 00:00:00 2001 From: Ruben van der Linde Date: Wed, 12 Aug 2026 11:04:19 +0200 Subject: [PATCH 3/5] fix(gates): a crashed gate-61 read as an empty scope, and gate-24 invented a count MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two repairs, both from the same rule: a crashed checker must never read as an empty scope, and an empty scope must never read as a pass. gate-61 — the first draft of the delta-base change skipped the checker invocation entirely when no base was resolved and short-circuited to the empty-scope status. Nothing ran, so nothing could crash, and test_gate_crashed_checker_is_not_a_finding.sh caught it immediately: with a python3 that cannot run, gate-61 reported NOT APPLICABLE where it must report SKIPPED (wiring). That is the same regression #364's first draft made in gate-16, caught by the same suite, and the repair is the same one: always run the checker, evaluate wiring first and scope second. With no base the helper is now invoked in its whole-tree mode purely to learn whether it can run, and its findings are discarded — sweeping on a baseless run was tried before and reverted because the builder runs that way. gate-24 — the parity wrapper is app-owned and declares no terminal marker, so a wrapper that exited non-zero having written nothing parseable was reported as one parity violation: a fabricated number with a plausible message and nothing behind it, the same shape as gates 22, 25 and 28. The verdict deliberately stays FAIL, because nothing here can tell a crash from a differently-phrased finding and guessing wiring would turn a real parity failure into a green hole. It simply stops claiming a count nobody measured. gate-61 carries the same clamp idiom and it is left alone with the reason written down: its wiring branch sits immediately above, so a crash cannot reach it, and zero counted lines with a non-zero exit is format drift rather than a dead checker. Two suites updated, neither weakened. The crashed-checker suite now accepts gate-17's NOT APPLICABLE on its E2BIG arm only when the terminal count marker proves the interpreter ran over the oversized scope list, which is a stronger assertion than the two it sits beside; its verdict regex also gained the multi-word form that made NOT APPLICABLE parse as NOT. The route-auth suite's gate-14 expectation moves from PASS to NOT APPLICABLE for the same reason its gate-5 neighbour moved in August: a pass here is scoping, not absence, and that reclassification was only half done. Refs: .github#374, #330, #364, #379 --- ...t_gate_crashed_checker_is_not_a_finding.sh | 27 +++++++- .../scripts/lib/test_gate_route_auth.sh | 19 +++++- hydra-gates/scripts/run-hydra-gates.sh | 61 +++++++++++++++++-- 3 files changed, 101 insertions(+), 6 deletions(-) diff --git a/hydra-gates/scripts/lib/test_gate_crashed_checker_is_not_a_finding.sh b/hydra-gates/scripts/lib/test_gate_crashed_checker_is_not_a_finding.sh index 84da58d4..94a6b1c6 100755 --- a/hydra-gates/scripts/lib/test_gate_crashed_checker_is_not_a_finding.sh +++ b/hydra-gates/scripts/lib/test_gate_crashed_checker_is_not_a_finding.sh @@ -121,11 +121,36 @@ _out="${_tmp}/run.txt" --scope-to-diff --base "${_root}" . > "${_out}" 2>&1 ) -_v17=$(grep -oE '^\[gate-17\] [^:]+: [A-Z]+( \([a-z]+\))?' "${_out}" | head -1 | sed 's/^[^:]*: //') +# NOTE the `( [A-Z]+)*`: the verdict word is not always one token. Under the +# original single-token pattern "NOT APPLICABLE" parsed as "NOT" and every +# comparison below failed on the STRING rather than on the behaviour — the same +# trap test_gate_empty_scope_never_passes.sh records in its own `_verdict`. +_v17=$(grep -oE '^\[gate-17\] [^:]+: [A-Z]+( [A-Z]+)*( \([a-z]+\))?' "${_out}" | head -1 | sed 's/^[^:]*: //') if grep -qE '^\[gate-17\][^:]*: FAIL — 0 ' "${_out}"; then _bad "gate-17 reported 'FAIL — 0 pass-through method(s)' — a crash rendered as a finding count" elif [ "${_v17}" = "PASS" ] || [ "${_v17}" = "FAIL" ]; then _ok "gate-17 produced a real verdict (${_v17}) over a ${_scope_bytes}-byte scope list" +elif [ "${_v17}" = "NOT APPLICABLE" ]; then + # ADDED WITH .github#374, AND IT IS A STRONGER ASSERTION THAN THE TWO ABOVE. + # + # The diff here is `root...HEAD`, and ThingController.php lives in the ROOT + # commit — so the change set is 3,000 filler `.ts` files and not one PHP + # file under lib/. `NOT APPLICABLE` is therefore the honest verdict, and it + # only became reachable when gate-17 stopped calling a `# count=0` over an + # unopened scope a PASS. + # + # But it must not become a way to DODGE this arm, whose whole subject is + # E2BIG: a scope list over MAX_ARG_STRLEN used to stop python3 from starting + # at all. So the marker is required. gate-17 can only reach its + # empty-scope branch from a path where the checker exited cleanly, so a + # `# count=` in the log is proof the interpreter ran over the + # ${_scope_bytes}-byte list. If E2BIG returns there is no marker, and the + # gate reports SKIPPED (wiring) — handled below, and never NOT APPLICABLE. + if grep -q '^# count=' "${_logs}/hydra-gate-redundant-controller.log" 2>/dev/null; then + _ok "gate-17 reported NOT APPLICABLE over an empty PHP scope, and its checker DID complete over the ${_scope_bytes}-byte scope list (terminal '# count=' marker present)" + else + _bad "gate-17 reported NOT APPLICABLE but its log carries no terminal '# count=' marker — the checker never finished, so this is a CRASH wearing an empty-scope verdict. That is the E2BIG defect returning through a new door." + fi elif [ "${_v17}" = "SKIPPED (wiring)" ]; then _ok "gate-17 reported SKIPPED (wiring) — honest, though the scope file should have avoided the crash" else diff --git a/hydra-gates/scripts/lib/test_gate_route_auth.sh b/hydra-gates/scripts/lib/test_gate_route_auth.sh index 6cb58932..d0ee57c6 100644 --- a/hydra-gates/scripts/lib/test_gate_route_auth.sh +++ b/hydra-gates/scripts/lib/test_gate_route_auth.sh @@ -348,7 +348,24 @@ if _run "${_REPO}" --scope-to-diff --base "${_BASE}"; then # this repository, so "PASS" would have claimed a clean bill of health for # four endpoints whose attributes this run cannot see at all. _expect_gate 5 "NOT APPLICABLE" "AppHost app, dependency-only diff: gate-5 clean (the scholiq case)" - _expect_gate 14 PASS "AppHost app, dependency-only diff: gate-14 clean" + # ⚠️ RECLASSIFIED 2026-08-12 from PASS to NOT APPLICABLE (.github#374), + # for the SAME reason 6a reclassified gate-5 on 2026-08-08 — and this line + # is why that reclassification was only half done. + # + # gate-14 filters routes through `_in_scope` one route at a time and then + # asked `_rr_fail -eq 0`. On this diff every route is filtered out, so zero + # findings over zero inspected routes printed `PASS` — indistinguishable + # from a repo whose every route was cross-checked and found reachable. The + # comment 40 lines up already states the argument in full: *"a PASS here is + # scoping, not absence" is a fact the verdict PASS does not state and NOT + # APPLICABLE does.* It was applied to gate-5 and not to its neighbour. + # + # Measured on a fixture carrying a real unrouted controller method: full + # scope FAIL — 1, docs-only diff PASS. The controls that keep this honest + # are 6b and 6c above (unchanged) plus the four `_expect_gate 14 FAIL` + # assertions earlier in this file — gate-14 must still raise a genuinely + # unreachable route the moment one is in scope. + _expect_gate 14 "NOT APPLICABLE" "AppHost app, dependency-only diff: gate-14 reports NO finding (scoped out, not absent)" fi echo diff --git a/hydra-gates/scripts/run-hydra-gates.sh b/hydra-gates/scripts/run-hydra-gates.sh index 4219e8e0..0edfa122 100755 --- a/hydra-gates/scripts/run-hydra-gates.sh +++ b/hydra-gates/scripts/run-hydra-gates.sh @@ -4294,8 +4294,27 @@ if [ -f scripts/check-integration-parity.sh ]; then # `⚠` header — never "missing"/"mismatch"/`^✗` — so this hard-failure # count excludes them by construction. _parity_hits=$(_count '^✗|missing|mismatch' "${_parity_log}") - [ "${_parity_hits}" -eq 0 ] && _parity_hits=1 - _fail 24 "integration-parity" "${_parity_hits} parity violation(s) — see ${_parity_log}" + if [ "${_parity_hits}" -eq 0 ]; then + # DO NOT INVENT THE COUNT (.github#374, and the #330 family). + # + # This line was `[ "${_parity_hits}" -eq 0 ] && _parity_hits=1`, so + # a wrapper that exited non-zero having written nothing this runner + # can parse was reported as `FAIL — 1 parity violation(s)`: a + # fabricated number with a plausible message and nothing behind it. + # Same shape as gates 22, 25 and 28 (`.github#379`). + # + # ⚠️ THE VERDICT DELIBERATELY STAYS `FAIL`, and this is the whole + # judgement. `scripts/check-integration-parity.sh` is APP-OWNED and + # declares no terminal-marker contract, so unlike gates 17, 18, 52 + # and 61 there is nothing here that can tell "it crashed" from "it + # found violations it phrased differently". Guessing `wiring` would + # turn a real parity failure into a skip — a green hole, and the + # strictly worse direction. So: fail closed, and stop asserting a + # number nobody measured. + _fail 24 "integration-parity" "the parity wrapper exited non-zero and printed no line this runner can count as a violation — the failure is real but its SIZE is unmeasured, so no count is claimed. Read ${_parity_log}." + else + _fail 24 "integration-parity" "${_parity_hits} parity violation(s) — see ${_parity_log}" + fi fi fi @@ -8387,8 +8406,31 @@ if [ -d lib/AppInfo ]; then python3 "${SCRIPT_DIR}/lib/check_listener_placement.py" . --base "${BASE_REF}" > "${_lwp_log}" 2>&1 _lwp_rc=$? else - echo "no delta base for this run — the helper was not invoked with one" > "${_lwp_log}" - _lwp_rc=3 + # ALWAYS RUN THE CHECKER. EVALUATE WIRING FIRST, THEN SCOPE (#364). + # + # A first draft skipped the invocation entirely with no base and + # short-circuited to rc 3 — and `test_gate_crashed_checker_is_not_a_finding.sh` + # caught it immediately: with a python3 that cannot run, gate-61 + # reported NOT APPLICABLE instead of SKIPPED (wiring). Nothing ran, so + # nothing could crash, and A CRASHED CHECKER READ AS AN EMPTY SCOPE — + # the exact regression `#364`'s first draft made in gate-16, caught by + # the exact same suite. Declining EARLIER silently retires a wiring + # invariant; that is the general shape, and this is the second instance. + # + # `--all` is the mode that needs no base. It is invoked here for ONE + # reason — to learn whether the checker can run at all — and its + # FINDINGS are deliberately discarded: sweeping the tree on a baseless + # run was tried before and reverted, because the builder runs that way + # and it surfaces the fleet's whole registration backlog on every build. + python3 "${SCRIPT_DIR}/lib/check_listener_placement.py" . --all > "${_lwp_log}" 2>&1 + _lwp_rc=$? + # WIRING FIRST: did it reach its own terminal summary? If yes it ran, + # and the honest verdict is "there is no delta base", which is rc 3's + # branch below. If no, leave the real status alone so the wiring branch + # further down catches it and says what went unchecked. + if _helper_finished "${_lwp_log}" '^checked [0-9]+ (post-event )?registration\(s\)'; then + _lwp_rc=3 + fi fi set +e if [ "${_lwp_rc}" -eq 0 ]; then @@ -8461,6 +8503,17 @@ if [ -d lib/AppInfo ]; then _lwp_why=$(head -3 "${_lwp_log}" 2>/dev/null | tr '\n' ' ' | cut -c1-200) _skip 61 "listener-work-placement" wiring "check_listener_placement.py exited ${_lwp_rc} without printing its terminal 'checked N ... registration(s)' summary, so NO post-object-event listener was judged and ADR-078 work placement is UNVERIFIED by this run. Checker output: ${_lwp_why:-}. See ${_lwp_log}." else + # THIS CLAMP IS GUARDED, AND THAT IS WHY IT STAYS (.github#374). + # + # A sweep for `[ "${_n}" -eq 0 ] && _n=1` flags this line alongside + # gate-24's, which WAS a crash-rendered-as-a-finding. This one is not: + # the `_helper_finished` branch immediately above has already proved the + # checker printed its terminal `checked N registration(s)` summary, so a + # crash can never reach here. Zero `^FAIL` lines with a non-zero exit is + # a FORMAT DRIFT between helper and runner, not a checker that died. + # + # Left alone deliberately, with the reason written down, so the next + # sweep does not re-flag it and does not "fix" it into a skip. _lwp_n=$(_count '^FAIL' "${_lwp_log}") [ "${_lwp_n}" -eq 0 ] && _lwp_n=1 _fail 61 "listener-work-placement" "${_lwp_n} post-event listener(s) doing synchronous work with no deferral and no justification (ADR-078); see ${_lwp_log}" From 29fe2f2579152f1883598ff3bb65f5a5a98e39fa Mon Sep 17 00:00:00 2001 From: Ruben van der Linde Date: Wed, 12 Aug 2026 11:17:14 +0200 Subject: [PATCH 4/5] fix(gate-61): the scope-matrix invariant read the FILE scope, not whether a diff existed MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The gate-agnostic property — a NOT APPLICABLE may not blame a diff on a run that computed none — is exactly right and stays. Its TRIGGER was wrong: it keyed on --full, when the property is about whether a diff was COMPUTED. Those are two independent inputs now, so the suite was reading the wrong one. That is the same conflation .github#347 was, one layer up, inside the test written to catch it. It fired on this branch and it fired correctly. Arm 3 now removes the BASE rather than the file scope, so every assertion it already carried holds verbatim. Arm 3b covers what used to be unreachable and is now what every PR in the fleet runs: full file scope with a real base. There, naming the diff is a true statement, and what has to hold instead is the falsifiability condition whose absence made #347 unreadable for weeks — a reason may name a diff only on a run that PRINTED the base it diffed against, so a reader can check the claim rather than trust it. Asserted from both sides. The advisory whole-tree sweep now runs in BOTH empty-scope branches. It ran only on the no-base path, because that used to be the only way to reach a state where the size of the unread backlog mattered. Full scope is the default now, so "a base resolved and excluded every registration" is the common case, and the ambiguity the sweep was added to remove had simply moved to the path everyone runs: 0 of 1 and 0 of 45 still printed identically there. The whole point of the reversal is that inherited debt stops being invisible, and leaving this gate's backlog unstated on the common path would have exempted it from exactly that. Refs: .github#374, #347 --- .../scripts/lib/test_gate_scope_matrix.sh | 92 +++++++++++++++++-- hydra-gates/scripts/run-hydra-gates.sh | 59 +++++++----- 2 files changed, 121 insertions(+), 30 deletions(-) diff --git a/hydra-gates/scripts/lib/test_gate_scope_matrix.sh b/hydra-gates/scripts/lib/test_gate_scope_matrix.sh index c65593b6..968c24d6 100644 --- a/hydra-gates/scripts/lib/test_gate_scope_matrix.sh +++ b/hydra-gates/scripts/lib/test_gate_scope_matrix.sh @@ -154,16 +154,39 @@ fi # =========================================================================== echo -echo "== arm 3 — FULL scope, same tree as arm 1 ==" +echo "== arm 3 — FULL file scope, NO delta base ==" # =========================================================================== -# `--full` exists to report inherited debt. The positive control above proves -# the debt is there and findable. Anything other than a FAIL here is the defect. -_out="$(gf_run_wrapper "${WORK}/inherited" "${WORK}/log-full" --full)" +# +# RE-KEYED FOR THE ADR-020 REVERSAL (hydra-gates/ADR-020-SUPERSEDED.md). +# +# This arm ran `--full` against a tree `gf_mark_base` had given a +# `refs/remotes/origin/development` ref, and asserted that the run computed no +# diff — because `--full` used to mean BOTH "read the whole tree" AND "compute +# no diff". +# +# ⚠️ AND THAT CONFLATION IS THE VERY THING `.github#347` WAS. The invariant this +# file enforces — *a NOT APPLICABLE may not blame a diff on a run that computed +# none* — is exactly right and stays. What was wrong is its TRIGGER: it keyed on +# the FILE SCOPE (`--full`) when the property is about whether A DIFF WAS +# COMPUTED. Those are two inputs now, and the suite was reading the wrong one — +# the same mistake, one layer up, inside the test that catches it. +# +# So arm 3 removes the base rather than the file scope, and arm 3b below covers +# the case that used to be unreachable and is now the COMMON one: full file +# scope WITH a real base, where naming the diff is a TRUE statement. +gf_build_repo "${WORK}/inherited-nobase" "${SRC}" +gf_commit_all "${WORK}/inherited-nobase" "base: app carrying an inherited post-event listener" +printf '\n- unrelated doc tweak\n' >> "${WORK}/inherited-nobase/docs/CHANGELOG.md" 2>/dev/null \ + || printf 'docs\n' > "${WORK}/inherited-nobase/README.md" +gf_commit_all "${WORK}/inherited-nobase" "docs: unrelated change" -if printf '%s' "${_out}" | grep -qF 'Base ref: n/a — --full requested'; then - _ok "the --full run states it computed no diff" +_out="$(gf_run_wrapper "${WORK}/inherited-nobase" "${WORK}/log-full")" + +if printf '%s' "${_out}" | grep -qF 'SCOPE-MODE: full' \ + && printf '%s' "${_out}" | grep -qF 'Delta base: NONE'; then + _ok "the run states BOTH its file scope (full) and that it resolved no delta base" else - _bad "the --full run did not announce itself as unscoped; the rest of this arm is unsafe to interpret" + _bad "the run did not state both 'SCOPE-MODE: full' and 'Delta base: NONE'; the rest of this arm is unsafe to interpret because nothing pins which inputs produced it" fi # @@ -204,7 +227,60 @@ fi # =========================================================================== echo -echo "== the general property: no NOT APPLICABLE may blame a diff on a --full run ==" +echo "== arm 3b — FULL file scope WITH a delta base (the new common case) ==" +# =========================================================================== +# +# Full file scope is now the DEFAULT, so this is what every PR in the fleet +# runs: the whole tree open to the state gates, a real base for the delta gates. +# Under the old default this combination was unreachable, which is why nothing +# asserted it. +# +# Naming the diff here is a TRUE statement — one was computed — so the invariant +# below must NOT fire on it. What has to hold instead is the falsifiability +# condition that made `#347` undetectable for weeks: **if a reason names a diff, +# the run must have printed the base it diffed against**, so a reader can check +# the claim rather than trust it. +_outb="$(gf_run_wrapper "${WORK}/inherited" "${WORK}/log-full-base")" + +if printf '%s' "${_outb}" | grep -qE '^\[hydra-gates\] Delta base: [^N]'; then + _ok "the full-scope run resolved a delta base and NAMED it" +else + _bad "the full-scope run resolved no delta base though the fixture has refs/remotes/origin/development — arm 3b would then be re-measuring arm 3 and proving nothing" +fi + +_vb="$(gf_verdict "${_outb}" 61)" +case "${_vb}" in + *"NOT APPLICABLE"*) + _ok "gate-61 still declines at full file scope — the deliberate delta-scoping survived the reversal" + ;; + *FAIL*) + _bad "gate-61 SWEPT THE TREE once full scope became the default. That was tried and reverted before: the builder runs unscoped, so this surfaces the whole registration backlog as blocking findings on every build. Got: ${_vb:0:160}" + ;; + *) _bad "gate-61 gave an unrecognised verdict at full scope with a base: ${_vb:0:160}" ;; +esac + +# A reason may name a diff ONLY on a run that computed one. Asserted from the +# other side here, so the pair pins both directions. +if printf '%s' "${_vb}" | grep -qF 'the diff against'; then + if printf '%s' "${_outb}" | grep -qE '^\[hydra-gates\] Delta base: [^N]'; then + _ok "gate-61 names the diff it was actually given, on a run that printed that base — a checkable claim, not #347's unfalsifiable one" + else + _bad ".github#347 IS LIVE BY A NEW ROUTE: gate-61 names a diff on a run that announced no delta base" + fi +fi + +# The backlog SIZE must be stated here too. This is now the common path, and +# without it '0 of 1' and '0 of 45' print identically on every PR in the fleet — +# the exact ambiguity the advisory sweep was added to remove on the other path. +if printf '%s' "${_vb}" | grep -qE 'ADVISORY.*[0-9]+ registration\(s\) carrying [0-9]+ finding\(s\)'; then + _ok "gate-61 states the size of the backlog it did not inspect at FULL scope too, not only when there is no base" +else + _bad "gate-61's full-scope skip does not state how many registrations went unread. This is the common case now, so the ambiguity the advisory sweep removed on the no-base path has simply moved to the path everyone runs: ${_vb:0:200}" +fi + +# =========================================================================== +echo +echo "== the general property: no NOT APPLICABLE may blame a diff on a run that COMPUTED NONE ==" # =========================================================================== # Generic, gate-agnostic. Catches the next gate that does this. # diff --git a/hydra-gates/scripts/run-hydra-gates.sh b/hydra-gates/scripts/run-hydra-gates.sh index 0edfa122..aa2b0657 100755 --- a/hydra-gates/scripts/run-hydra-gates.sh +++ b/hydra-gates/scripts/run-hydra-gates.sh @@ -8471,30 +8471,45 @@ if [ -d lib/AppInfo ]; then # set. Reading the file scope here is precisely how `.github#347` got # its false reason — a `NOT APPLICABLE` blaming a diff on a run that # computed none. + # THE ADVISORY SWEEP RUNS IN **BOTH** BRANCHES (.github#374). + # + # It used to run only on the no-base branch, because `--full` was the + # only way to reach a state where the size of the unread backlog was + # interesting. Full file scope is now the DEFAULT, so "a base resolved + # and it excluded every registration" is the COMMON case — every PR in + # the fleet — and `0 of 1` and `0 of 45` were still typographically + # identical there. The whole point of the reversal is that inherited + # debt stops being invisible; leaving this gate's backlog unstated on + # the common path would have exempted it from exactly that. + # + # ADVISORY ONLY, in both branches. Its exit status is discarded on + # purpose: a sweep that finds inherited debt must not become this run's + # verdict, and a sweep that CRASHES must not either — the count is + # quoted only when the helper printed its own terminal summary line. + _lwp_all_log=${HYDRA_GATE_LOG_DIR}/hydra-gate-listener-work-placement.advisory.log + python3 "${SCRIPT_DIR}/lib/check_listener_placement.py" . --all \ + > "${_lwp_all_log}" 2>&1 || true + # + # THE SUMMARY IS RECOMPOSED, NOT QUOTED. The helper's own line ends + # "…, 0 out of scope: 1 failure(s)", and pasting that in would put + # the phrase "out of scope" back into a reason on a run that + # computed no scope — the exact sentence this fix removes, smuggled + # in as a quotation. The suite asserts against that phrase + # gate-agnostically and caught it here. + _lwp_backlog="" + if grep -qE '^checked [0-9]+ post-event registration' "${_lwp_all_log}" 2>/dev/null; then + _lwp_all_n=$(grep -oE '^checked [0-9]+ post-event registration' "${_lwp_all_log}" | tail -1 | grep -oE '[0-9]+' | head -1) + _lwp_all_f=$(grep -oE '[0-9]+ failure\(s\)' "${_lwp_all_log}" | tail -1 | grep -oE '[0-9]+' | head -1) + _lwp_backlog=" ADVISORY, and it decides nothing here: a whole-tree sweep of this same tree reaches ${_lwp_all_n:-an unreported number of} registration(s) carrying ${_lwp_all_f:-an unreported number of} finding(s), NONE of which this run judged — see ${_lwp_all_log}." + fi if [ "${HAVE_DELTA_BASE}" = "1" ]; then - _skip 61 "listener-work-placement" na "the diff against '${BASE_REF}' put every post-event registration out of scope, so NONE were inspected. This gate is deliberately delta-scoped even at full file scope — the fleet's 149-registration backlog is a work-list, not a reason to block an unrelated change. It runs on the next change that touches a listener registration. See ${_lwp_log}." + # A diff WAS computed, so naming it is a true statement, not the + # `.github#347` lie. The discriminator is `HAVE_DELTA_BASE`, and the + # base itself is printed by the preamble — so a reader can check the + # claim rather than having to trust it. + _skip 61 "listener-work-placement" na "the diff against '${BASE_REF}' put every post-event registration out of scope, so NONE were inspected. This gate is deliberately delta-scoped even at full file scope — the fleet's registration backlog is a work-list, not a reason to block an unrelated change. It runs on the next change that touches a listener registration.${_lwp_backlog} See ${_lwp_log}." else - # ADVISORY ONLY. Its exit status is discarded on purpose: a sweep - # that finds inherited debt must not become this run's verdict, and - # a sweep that CRASHES must not either — the count is quoted only - # when the helper printed its own terminal summary line. - _lwp_all_log=${HYDRA_GATE_LOG_DIR}/hydra-gate-listener-work-placement.advisory.log - python3 "${SCRIPT_DIR}/lib/check_listener_placement.py" . --all \ - > "${_lwp_all_log}" 2>&1 || true - # - # THE SUMMARY IS RECOMPOSED, NOT QUOTED. The helper's own line ends - # "…, 0 out of scope: 1 failure(s)", and pasting that in would put - # the phrase "out of scope" back into a reason on a run that - # computed no scope — the exact sentence this fix removes, smuggled - # in as a quotation. The suite asserts against that phrase - # gate-agnostically and caught it here. - _lwp_backlog="" - if grep -qE '^checked [0-9]+ post-event registration' "${_lwp_all_log}" 2>/dev/null; then - _lwp_all_n=$(grep -oE '^checked [0-9]+ post-event registration' "${_lwp_all_log}" | tail -1 | grep -oE '[0-9]+' | head -1) - _lwp_all_f=$(grep -oE '[0-9]+ failure\(s\)' "${_lwp_all_log}" | tail -1 | grep -oE '[0-9]+' | head -1) - _lwp_backlog=" ADVISORY, and it decides nothing here: a whole-tree sweep of this same tree reaches ${_lwp_all_n:-an unreported number of} registration(s) carrying ${_lwp_all_f:-an unreported number of} finding(s), NONE of which this run judged — see ${_lwp_all_log}." - fi - _skip 61 "listener-work-placement" na "this run computed NO diff (--full / unscoped), and gate-61 is diff-scoped by design (ADR-078/ADR-020, it is about NEW debt) — so NO post-event registration was inspected and ADR-078 work placement is UNVERIFIED by this run. This is NOT a clean bill of health, and NO diff excluded anything: there was no diff. Re-measure with an explicit base (HYDRA_GATE_BASE_REF=origin/beta) or run with --scope-to-diff.${_lwp_backlog} See ${_lwp_log}." + _skip 61 "listener-work-placement" na "this run computed NO diff (no delta base), and gate-61 is delta-scoped by design (ADR-078, it is about NEW debt) — so NO post-event registration was inspected and ADR-078 work placement is UNVERIFIED by this run. This is NOT a clean bill of health, and NO diff excluded anything: there was no diff. Re-measure with an explicit base (--base or HYDRA_GATE_BASE_REF=origin/beta).${_lwp_backlog} See ${_lwp_log}." fi elif [ "${_lwp_rc}" -eq 4 ]; then _skip 61 "listener-work-placement" na "this repo registers no post-object-event listener, so there is no work to place on or off the write path (ADR-078). See ${_lwp_log}." From 3d1378dc9115d2c039278d4ff81933b59306fb30 Mon Sep 17 00:00:00 2001 From: Ruben van der Linde Date: Wed, 12 Aug 2026 12:18:01 +0200 Subject: [PATCH 5/5] ci(gates): the unresolvable-base check asserted the contract this PR supersedes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The required "Unresolvable base ref must exit 99 with no green" step encoded ADR-020's rule as the ONLY acceptable behaviour, so this PR could not merge: under the new default an unresolvable base no longer aborts, it falls back to full scope. The old assertion is kept BYTE-FOR-BYTE and simply run in the mode it was always about (--scope-to-diff): with an explicit diff request and no usable base, the runner must still exit 99, print NOTHING WAS CHECKED, and emit no gate lines. Every clause of it still holds. A second step covers the new default: the run must say the base was unresolvable, must still emit gate lines, and none of gates 16/29/47/48/61 may print PASS over a scope that was never read. Positive-controlled against this runner's real captured output four ways — injecting "[gate-16] ... PASS" fails it, deleting a gate line fails it, and the diff-mode arm fails if the 99/NOTHING WAS CHECKED behaviour regresses. --- .github/workflows/hydra-gates-package.yml | 63 +++++++++++++++++++++-- 1 file changed, 60 insertions(+), 3 deletions(-) diff --git a/.github/workflows/hydra-gates-package.yml b/.github/workflows/hydra-gates-package.yml index f83df319..f9e73a5e 100644 --- a/.github/workflows/hydra-gates-package.yml +++ b/.github/workflows/hydra-gates-package.yml @@ -366,19 +366,34 @@ jobs: || { echo "::error::${_na} not-applicable + ${_unrun} unrun gates are counted, but only ${_named} said so on their own line. A gate is disappearing from the per-gate output while still being tallied."; exit 1; } echo "OK — exit 0, coverage stated, waivers stated, and ran=${_ran} + na=${_na} + unrun=${_unrun} = declared=${_decl}, each named." - - name: "Unresolvable base ref must exit 99 with no green" + # WHY THIS STEP NOW NAMES ITS SCOPE. + # + # Until ADR-020 was superseded, --scope-to-diff was the DEFAULT, so + # "unresolvable base" and "no scope at all" were the same condition and + # this step could leave the scope unstated. They are now two different + # conditions and they have two different correct answers, so the step + # that asserts the first must say which one it is testing. + # + # This is the diff-scoped half, and it is UNCHANGED: with + # --scope-to-diff, the base IS the scope, so an unresolvable base leaves + # nothing to inspect and refusing is the only honest verdict. Every + # assertion below is byte-for-byte the one that guarded this before the + # default moved; only the invocation names the mode it was always about. + # Verified against the fixture: exit 99, the "NOTHING WAS CHECKED" line, + # zero `[gate-N]` lines. + - name: "Unresolvable base ref, diff scope: must exit 99 with no green" run: | set -eu cd /work/fixture set +e - OUT="$(HYDRA_GATE_BASE_REF=origin/definitely-not-a-branch /work/consumer/vendor/bin/hydra-gates --app-dir /work/fixture 2>&1)" + OUT="$(HYDRA_GATE_BASE_REF=origin/definitely-not-a-branch /work/consumer/vendor/bin/hydra-gates --app-dir /work/fixture --scope-to-diff 2>&1)" RC=$? set -e printf '%s\n' "${OUT}" echo "--- assertions ---" [ "${RC}" -eq 99 ] \ - || { echo "::error::expected exit 99 for an unresolvable base, got ${RC}"; exit 1; } + || { echo "::error::expected exit 99 for an unresolvable base under --scope-to-diff, got ${RC}"; exit 1; } printf '%s' "${OUT}" | grep -q "NOTHING WAS CHECKED" \ || { echo "::error::did not state that nothing was checked"; exit 1; } printf '%s' "${OUT}" | grep -qiE "ALL .* (GATES )?(PASSED|GREEN)" \ @@ -386,3 +401,45 @@ jobs: printf '%s' "${OUT}" | grep -qE '^\[gate-[0-9]+\]' \ && { echo "::error::gate lines were emitted for a run that could not be scoped"; exit 1; } echo "OK — exit 99, no green, no gate lines." + + # THE FULL-SCOPE HALF — the new default, and the harder one to get right. + # + # Here the base is NOT the scope. The whole tracked tree is, and it is + # readable whatever the caller typed, so 59 gates have a real verdict and + # refusing would throw those away to punish a bad input. The five DELTA + # gates — 16, 29, 47, 48, 61 — are the ones that lose their subject. + # + # So the property this step defends is not "the run failed". It is: THE + # FIVE MUST NOT PASS, AND THE RUN MUST SAY THE BASE WAS UNUSABLE. A green + # here is legitimate and is a green over 59 gates, not over 64 — which is + # precisely the distinction the COVERAGE accounting exists to carry. + # + # Note the last assertion is the one with teeth: it fails if any of the + # five prints PASS. Without it, a future change that quietly restored a + # confident PASS-over-an-empty-diff for those gates would satisfy every + # other clause here. + - name: "Unresolvable base ref, full scope: the five delta gates must not pass" + run: | + set -eu + cd /work/fixture + set +e + OUT="$(HYDRA_GATE_BASE_REF=origin/definitely-not-a-branch /work/consumer/vendor/bin/hydra-gates --app-dir /work/fixture 2>&1)" + RC=$? + set -e + printf '%s\n' "${OUT}" + + echo "--- assertions ---" + printf '%s' "${OUT}" | grep -q "does not resolve" \ + || { echo "::error::the run did not say the named base was unusable — a typo'd base must never pass silently"; exit 1; } + printf '%s' "${OUT}" | grep -qE '^\[gate-[0-9]+\]' \ + || { echo "::error::no gate reported at all — full scope must still read the whole tree when the base is unusable"; exit 1; } + _bad="" + for g in 16 29 47 48 61; do + printf '%s\n' "${OUT}" | grep -qE "^\[gate-${g}\] [a-z0-9-]+: (NOT APPLICABLE|SKIPPED)" \ + || _bad="${_bad} ${g}" + printf '%s\n' "${OUT}" | grep -qE "^\[gate-${g}\] [a-z0-9-]+: PASS" \ + && { echo "::error::gate-${g} is a DELTA gate and PASSED with no usable base — it passed over an empty diff, which is the defect this whole contract exists to prevent"; exit 1; } + done + [ -z "${_bad}" ] \ + || { echo "::error::delta gate(s)${_bad} did not report NOT APPLICABLE by name with no usable base"; exit 1; } + echo "OK — base reported unusable, the tree was still read, and gates 16/29/47/48/61 each declined by name."