Skip to content

gate-19: duplicate scenario titles collapse to one slug — one test credits several scenarios, and dangling anchors are never reported #354

Description

@rubenvdlinde

Summary

admin-settings/spec.md declares the same four scenario titles twice, and gate-19 slugifies a
scenario from its heading text alone. Two structurally different scenarios therefore collapse onto
one <spec>::<slug> ref, and one e2e test marks both covered.

Evidence

openspec/specs/admin-settings/spec.md — the hide-slogan checkbox requirement and the
show-menu-labels checkbox requirement each declare:

line scenario
128 / 156 #### Scenario: Checkbox reflects enabled state
134 / 161 #### Scenario: Checkbox reflects disabled state
140 / 167 #### Scenario: Checkbox label text and accessibility
146 / 172 #### Scenario: Checkbox change triggers API call

check_e2e_coverage.py::_slugify derives the slug from the heading text only — there is no
de-duplicating suffix and no parent-requirement qualifier. Both headings yield
admin-settings::checkbox-reflects-enabled-state.

In run_gate() coverage is tested with if s['ref'] not in covered_refs, so a single reference
satisfies every scenario sharing that ref. The show-menu-labels scenarios have been counted as
covered by the hide-slogan tests.

Two failure modes, both silent

  1. Over-crediting. N duplicate-titled scenarios are discharged by 1 test.
  2. Dangling anchors are not reported. The test author had assumed a -1 suffix and wrote
    ...#checkbox-reflects-enabled-state-1 and ...#checkbox-label-text-and-accessibility-1.
    Neither slug exists. The gate neither credited them nor warned — a typo'd anchor and a correct
    one are indistinguishable in its output. Found by diffing refs collected from tests/e2e/**
    against slugs parsed from openspec/specs/*/spec.md; the gate will never tell you.

Suggested fix

Qualify a Format-A slug with its enclosing ### requirement, as Format B already does
(<parent-req-slug>-scenario-<n>). That makes duplicate titles under different requirements
distinct and is consistent with the existing Format-B convention.

Separately, gate-19 should report anchors that resolve to no scenario rather than ignoring
them. That is a few lines in collect_covered_refs / run_gate and would have surfaced both
defects immediately.

Impact on counts

Fixing the collision will make several apps' gate-19 numbers go up, because scenarios currently
credited by a same-titled sibling will become individually uncovered. That is the honest number.

Found while raising nldesign's gate-19 coverage. The two dangling anchors in
tests/e2e/spec-coverage/admin-settings.spec.ts are corrected in ConductionNL/nldesign (they now
point at the real slug); the collision itself cannot be fixed from the test side.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions