gate-26's signal is inverted: it credits PROSE and refuses NAVIGATION — 11 of 21 openregister findings are false, 6 of 16 passes are comment-only

[rubenvdlinde]

Summary

gate-26 decides "is this page component covered?" by looking for the component's file stem in the text of tests/e2e/**. A well-written Playwright test navigates by URL route and asserts on accessible names — it has no reason to contain the component's filename. So:

• a page that is driven by a green, executing e2e test is reported uncovered, and
• a page that is merely named in a comment is reported covered.

Measured on openregister: 11 of its 21 findings are false, and 6 of its 16 passes are prose-only. The signal is close to inverted.

Measured

Canonical package @main (3c8da4c), pristine worktree of openregister@origin/development = fdb0d6ca8.

(a) Prose-only passes. Re-running the gate's own is_covered() against a comment-stripped corpus:

page components discovered:        37
UNCOVERED (gate's own verdict):    21
COVERED ONLY BY A COMMENT:          6

The six, with the line that silences each:

src/views/avg/AvgIndex.vue
  tests/e2e/workflows/dsar-cases.spec.ts:6:  * Cases tab added to the AVG view (src/views/avg/AvgIndex.vue): the case
src/views/integration/IntegrationsView.vue
  tests/e2e/leaf-screenshots.spec.ts:5:      * IntegrationsView (/integrations/:register/:schema/:objectId), then
src/views/organisation/OrganisationsIndex.vue
  tests/e2e/spec-coverage/admin-settings-pages.spec.ts:108:  // (OrganisationsIndex.vue), so on a stock instance with exactly one
src/views/Endpoint/EndpointsIndex.vue
  tests/e2e/spec-coverage/admin-settings-pages.spec.ts:170:  // NOTE: EndpointsIndex renders no page <h1>; assert the primary action.
src/views/files/FilesIndex.vue
  tests/e2e/spec-coverage/feature-pages.spec.ts:103:   // FilesIndex exposes a "Toggle search sidebar" action (not a
src/views/object/ObjectsIndex.vue
  tests/e2e/integration-mount.spec.ts:117:  * Open the object detail page (ObjectDetails.vue, served by ObjectsIndex)

Note the first two are silenced by comments in files no Playwright project runs — doubly fictional.

(b) The false positives. Mapping the 21 flagged pages through src/manifest.json against the routes the five CI-executed specs actually navigate to:

pages whose route a CI-RUN spec already navigates to: 11 of 21

  MyAccount(/mijn-account)          ApplicationsIndex(/applications)
  ConfigurationsIndex(/configurations)  SearchTrailIndex(/search-trails)
  ReportsIndex(/reports)            FeaturesRoadmapIndex(/features-roadmap)
  SchemasIndex(/schemas)            SourcesIndex(/sources)
  TemplatesIndex(/templates)        WebhooksIndex(/webhooks)
  WebhookLogsIndex(/webhooks/logs)

These pages are driven, asserted and green in CI today — I ran the suite on a dedicated NC 34 rig: 44 passed, 0 failed.

Mechanism

check_visual_coverage.py:

def is_covered(page, visual_corpus, e2e_corpus):
    stem = Path(page["path"]).stem
    pid  = page["id"]
    needles = {stem, pid}
    ...
    for needle in needles:
        if needle and re.search(rf"\b{re.escape(needle)}\b", e2e_corpus):
            return True

_e2e_corpus() concatenates raw file text, comments included. So the needle is a filename, and the corpus is prose + code undifferentiated.

Both offered remedies are wrong

The finding message reads:

… missing visual-regression baseline (tests/e2e/visual/**) / e2e test / @visual exclude

1. tests/e2e/visual/** — in openregister (and, from the fleet board, elsewhere) that directory is outside the testMatch the CI config uses, so a baseline placed there is never executed. The gate would go green over a file nothing runs.
2. "e2e test" — cannot see a URL-driven test, which is what a good one looks like.
3. @visual exclude — a waiver on a page that is covered.

That leaves exactly one cheap satisfying action: write the component name into a comment. Six components in this repo are green by precisely that route. A gate whose cheapest satisfying action is prose will be satisfied by prose — cf. #358.

Proposed fix

Two halves, and they move the number in opposite directions, which is why they should land together:

1. Resolve component → route from src/manifest.json / the registry, and count a test that navigates to that route (goto with the route literal, or a route constant) as coverage. Removes the 11 false positives.
2. Strip comments from the e2e corpus before needle matching. Reddens the 6 prose-only passes.

Net on openregister: 21 → 16.

⚠️ Not merging. Half of this reddens repos, same hold as #347 / #349 / #356 / #361, and #359's acceptance suite should land first so the change arrives with a fixture. Filing with the measurement so it can be sequenced.

There is a precedent in this same file for a fix landing with its own regression note: check_visual_coverage.py:472 records the exit-status-as-count bug and how it was fixed.

Provenance

Found while closing gate-26 on openregister. I was about to write baselines for 11 pages that already have passing behavioural tests.

[rubenvdlinde]

A third shape of the same inversion: a DATA-DRIVEN e2e suite is invisible to the grep, so it credits nothing

Confirmed on hermiq development @ 468f5f13, canonical package @main 81c8c97,
base origin/beta. This is the same signal inversion this issue describes, arriving by a
route neither the prose half nor the navigation half covers.

_covered() decides coverage by searching the concatenated e2e corpus for the component's
stem or page id. hermiq's tests/e2e/spec-coverage/manifest-pages.spec.ts generates one
test() per page from src/manifest.json at run time, so no component name is ever a
literal in the source. Measured:

component	literal hits in tests/e2e/spec-coverage/	literal hits in all of tests/e2e/	actually mounted+asserted in CI?
AgentMemory	0	0	✅ ✓ [custom] AgentMemory mounts at /memory
McpTools	0	0	✅ ✓ [custom] McpTools mounts at /mcp-tools
GuardrailPolicySettings	0	0	✅ ✓ [custom] GuardrailPolicy mounts at /guardrail-policy
FlowBuilder	0	0	❌ genuinely uncovered

(Passing lines quoted from run 31502914249, job 93818270613.)

So 3 of hermiq's 4 gate-26 findings are false, and they are false in the direction that
matters: the gate reports "no visual coverage" for pages a CI-executed test mounts and
asserts on, while — per #358 — a bare token in a comment would have satisfied it. A repo
whose e2e suite is manifest-driven is scored worse than one that wrote the name in a
comment and tested nothing.

🔴 And the remedy the message names is unreachable on this repo. The preferred remedy is
tests/e2e/visual/**, but hermiq's CI runs playwright-test-path: "tests/e2e/spec-coverage"
(code-quality.yml:163) — nothing under tests/e2e/visual/ would ever execute. Same
end-state as the doriath finding (testIgnore: ['**/visual/**']), different mechanism. So the
gate's own advice, followed literally, produces baselines CI never runs; and the fallback
remedy, followed literally, produces a comment. Both documented remedies on this repo are
dodges.

Suggested direction, additive to what #362 already proposes: coverage evidence should be
able to come from the run, not only from the source text — e.g. accept the Playwright
JSON report's test titles for the scoped project, which on hermiq contains the component
names verbatim precisely because they were generated from the manifest. Grepping test
sources can only ever see coverage an author hard-coded.

⚠️ Filed as a comment rather than a new issue because it is the same inverted signal, not a
separate bug. Not fixed on hermiq: making the token appear would close it with no assertion
added, which is the thing this issue exists to stop. Recorded in ConductionNL/hermiq#184.

[rubenvdlinde]

Re-measured on openregister with the canonical checker — the correction cuts both ways, and the earlier openregister numbers in this issue are wrong

Package: ConductionNL/.github@main = 765d0817. Tree: openregister@origin/development = 9628dc5d0. Gate-26 at full scope: FAIL — 21.

Negative control first, expectation computed in advance. Removing tests/e2e/visual/mdm-merge-ui.visual.spec.ts — the sole credit for MergeOperationsIndex, while DuplicatesIndex keeps two others — should give 22, naming that one page. Observed: 21 → 22, the added line was exactly src/views/quality/MergeOperationsIndex.vue, and restoring returned output byte-identical to the baseline.

Method

For each of the 37 discovered page components I resolved the manifest component → route (via the checker's own _manifest_page_components(), not a reimplementation), then asked whether a spec CI actually executes navigates that route — the five testMatch patterns in tests/e2e/ci/playwright.config.ts — and what it asserts there.

That last part matters, because two answers were previously collapsed into one:

	count
driven by a CI-executed spec with content assertions (heading text, named button, list surface, 0 console errors, 0 5xx)	20
route visited, but the only assertion is main, .app-content is visible — host chrome	4
not driven by any CI-executed spec at all	13

The corrections

The false-positive count is 13, not 11. The earlier reading missed FlowsIndex and FlowDetailPage. Both are driven by tests/e2e/ci/flow-controls.spec.ts, which goes to #/flows, clicks New flow (routing to /flows/new → FlowDetailPage) and asserts the sidebar, a non-empty palette, and the Save / Run now controls. That spec uses a const FLOWS_ROUTE and a template literal, so a goto('...') grep does not see it.

The "6 prose-only passes" overstates the fiction. Five of those six — FilesIndex, ObjectsIndex, OrganisationsIndex, EndpointsIndex, AvgIndex — are genuinely driven by feature-pages / admin-settings-pages / core-list-pages. A comment happens to also mention the stem, but removing it would not make those screens untested. Only IntegrationsView is credited purely by prose. The prose defect is real; its blast radius here is 1, not 6.

So the honest number is 18, not 16 and not 21 — 4 shell-only + 13 undriven, minus the 1 double-counted overlap → the 37 split as 19 really covered / 18 not.

Of the 21 the gate flags: 13 false positives, 2 shell-only (DeletedIndex, AuditTrailIndex), 6 genuinely undriven.
Of the 16 it passes: 6 genuinely covered, 3 shell-only (SearchIndex, EntitiesIndex, RegisterDetail), 7 fictional.

Three findings the route-mapping surfaced that the gate cannot

1. RegisterDetail's only credit is a test that passes whether or not the component renders. manifest-shell.spec.ts:158-172 deep-links /registers/e2e-probe-id and its own comment records that RegisterDetail's mounted() guard bounces back to #/registers on a cold deep-link, so it asserts toHaveURL(/#\/registers(\/e2e-probe-id)?$/) — either outcome passes. Same family as "a widget's TITLE is host chrome": asserting the shell cannot distinguish a rendered page from a redirect.

2. Three of the 37 "pages" are unreachable files. No manifest entry, no src/registry.js entry, no import anywhere in src/:

• src/views/Views.vue — openspec/changes/archive/2026-05-28-openregister-manifest-shell-swap/tasks.md:92 literally records "src/views/Views.vue is now orphaned". It still ships, and gate-26 credits it as covered because the bare English word Views appears in test files.
• src/views/object/MapView.vue — referenced only by a docblock sentence in src/services/geo/mapData.js:7.
• src/views/organisation/OrganisationDetails.vue — referenced by nothing.

Positive control for the absence: the same search returns src/manifest.json + src/registry.js for SchemaDetails, ApplicationDetails, EntityDetail and ReportView. registry.js has exactly 36 page(() => import(...)) entries, matching the 36 manifest pages.

For an unreachable file every remedy the gate offers is wrong — a baseline, an e2e test and a @visual exclude all assert something about a screen that does not exist. Deletion is the fix, and the gate has no vocabulary for it. This is the .github#309 "a directory is not a page" problem in its terminal form: _is_child_component() keeps a file in scope when there is no positive evidence it is somebody's child, and nothing imports it is exactly that state.

3. tests/e2e/visual/** is where openregister's fictional coverage lives. Five of the seven fictional passes (QualityIndex, DuplicatesIndex, MasterEntitiesIndex, QueueHealthIndex, MergeOperationsIndex) are credited only by visual/mdm-*.visual.spec.ts — real, well-written spec files that no Playwright project runs. The gate's preferred remedy is the one directory CI ignores.

On the proposed fix

Resolving component → route and counting a navigation as coverage is right for the 13, but on this tree it should also not credit a route whose only assertion is main, .app-content. Otherwise it converts 13 false positives into 4 new ones and moves RegisterDetail from fictional-pass to confident-pass.

[rubenvdlinde]

Correction to my own numbers, two comments up — the table and the prose disagreed by one

I posted this split:

	count
driven with content assertions	20
route visited, only main, .app-content asserted	4
not driven at all	13

and then said "the 37 split as 19 really covered / 18 not". Both cannot be right. The prose is correct and the table is wrong, by exactly the case the comment itself raises three paragraphs later: RegisterDetail belongs in the shell-only row, not the driven row. Its only test deep-links /registers/e2e-probe-id and asserts toHaveURL(/#\/registers(\/e2e-probe-id)?$/) — satisfied whether the component renders or its mounted() guard bounces to the list. I demoted it in the argument and forgot to demote it in the table.

Corrected:

	of 37 page components
driven with CONTENT assertions by a CI-executed spec	19
route visited, only host chrome asserted	5 — SearchIndex (/tables), EntitiesIndex (/entities), DeletedIndex (/deleted), AuditTrailIndex (/audit-trails), RegisterDetail (/registers/:id)
not driven at all	13

19 + 5 + 13 = 37. Honest gate-26 = 5 + 13 = 18 — unchanged, because the error was a transfer between two rows that both count as "not really covered". The 21/16 breakdown is also unchanged: of the 21 flagged, 13 false / 2 shell-only / 6 undriven; of the 16 passed, 6 genuinely covered / 3 shell-only / 7 fictional.

Recording it rather than quietly editing, because the failure mode is the one this whole exercise is about: a number that survives into a summary table after the argument that produced it has moved on. The headline was right and one of its inputs was not.

[rubenvdlinde]

Independent confirmation from docudesk, and one case where the same file is BOTH failure modes

Measured on a pristine worktree of docudesk@origin/development = 288f587c,
canonical package @main = 765d081, with a positive control whose expected
value was computed before it was run.

Prose-only passes: 6 of 22

Re-running the gate's own is_covered() against a comment-stripped corpus:

arm	uncovered of 22 pages
gate as shipped	0 → PASS
comments stripped from the e2e corpus	6
stripped and restricted to files CI actually executes	6 (the same six)

The six, with the line that silences each:

src/views/correspondence/CorrespondenceIndex.vue
  tests/e2e/spec-coverage/orphaned-surface-restoration.spec.ts:68
    // Real field from CorrespondenceIndex.vue — proves the component
src/views/signing/SigningRequestForm.vue
  tests/e2e/workflows/signing-workflow.spec.ts:13
    *   - There is no UI to CREATE a signing request — `SigningRequestForm.vue`
src/views/versions/VersionsView.vue
  tests/e2e/spec-coverage/versions.spec.ts:72
    // No rows: VersionsView must have said WHY — either the
src/views/templates/TemplateDetail.vue
  tests/e2e/workflows/templates-crud.spec.ts:205
    // TODO: fill name + content fields and save once TemplateDetail.vue is
src/views/templates/TemplateIndex.vue
  tests/e2e/spec-coverage/templates.spec.ts:76
    // page. TemplateIndex.vue is still on disk but is registered by NOTHING
src/components/PrintPreview.vue
  tests/e2e/spec-coverage/print-preview.spec.ts:28
    // Action buttons from PrintPreview.vue. Scope "Close" to the page

Note the shape of two of them: a comment saying the surface does not exist
(SigningRequestForm) and a comment saying the component is registered by
nothing (TemplateIndex) both COVER their page. The gate is satisfied by
prose asserting the opposite of coverage.

Positive control, expectation stated in advance (FAIL — 1, naming
src/components/PrintPreview.vue): rewording one comment so it no longer
contains the token moved the gate PASS(22) → FAIL 1 → PASS. Reverted.

And half of the six are FALSE, which is the other half of #362

Mapping each page to its src/manifest.json route and checking whether a
CI-executed spec navigates there:

page	route	a CI-run spec navigates there?	verdict
PrintPreview.vue	/print-preview/:templateId?	yes — print-preview.spec.ts:22,:41, asserts the heading and three buttons	false pass — real coverage
VersionsView.vue	/versions	yes — versions.spec.ts:29, asserts heading + table/notice	false pass — real coverage
CorrespondenceIndex.vue	/correspondence	yes — orphaned-surface-restoration.spec.ts:65	false pass — real coverage
TemplateDetail.vue	/templates/:id	no	real gap
SigningRequestForm.vue	/signing/new	no	real gap
TemplateIndex.vue	none	n/a (dead component)	real gap

PrintPreview.vue is the cleanest single demonstration of this issue I have
seen. The same file is a false PASS today, and deleting the comment three
lines above the assertions turns it into a false FINDING — while a green,
CI-executed test navigates to its route and asserts its heading and its three
action buttons the whole time. The gate reads the prose and cannot see the
navigation immediately below it.

So docudesk's honest gate-26 number is 3, not the 6 that comment-stripping
alone suggests, and certainly not the 0 the gate reports.

Checked and ruled out: the hermiq run-time-generation shape

hermiq's false findings came from manifest-pages.spec.ts building its cases
from the manifest at run time, so the literal-stem grep finds nothing.
docudesk's page-components.spec.ts contains no loop and no manifest read —
its per-page cases are literal — so that mechanism does not apply here and the
three real gaps above are real.

One more input to the fix

docudesk's tests/e2e/visual/** — gate-26's preferred evidence layer — is
never executed by CI, by a third mechanism worth knowing alongside doriath's
and pipelinq's. The shared workflow runs npx playwright test --config="$CONFIG"
with no --project, and prefers tests/e2e/playwright.config.ts when it
exists. docudesk has one, and it declares only chromium, with
testIgnore: ['**/visual/**']. The root config's three projects (chromium,
docs-capture, visual) are therefore never all run. Measured on run
31516066468: Running 106 tests, and 106 of 106 result lines are
[chromium] — zero [visual], zero [docs-capture]; locally the root config
lists 118 = 106 + 11 + 1.

Whatever replaces the stem grep, a PNG under tests/e2e/visual/ should not
count as proof unless a project that runs it exists. Reading Running N tests
is not enough — the project tag on the result lines is.

[rubenvdlinde]

Sequencing note from the gate-fixes wave (2026-08-11). Landed this session, smallest blast radius first, each with a fixture observed red-before / green-after: #366 (the gate-16 fixture #364 shipped without), #368 (gate-7 hasPermission/canAccess, #360), #370 (gate-24 both registration APIs, #349), #371 (gate-61 honest empty-scope reason, #347).

This one was deliberately NOT landed in that wave. The per-issue reason is in the next comment / in fleet-board/findings/github-gate-fixes.md; the common one is that every remaining item either changes what a gate counts rather than what it says, or lives in quality.yml where run-helper-suites.sh cannot reach it — and four gate changes in one session is already the limit at which the next fleet measurement stays attributable.