feat(test-plan): add a form-semantics axis, qualify dedup, define the PR set - #37447
feat(test-plan): add a form-semantics axis, qualify dedup, define the PR set#37447erickgonzalez wants to merge 4 commits into
Conversation
… PR set Three gaps found by review of a real run, kept separate from the two findings still under discussion. Form & state semantics — a tenth coverage axis. The existing nine are platform concerns and the UI/UX checklist is purely presentational: it asks whether a control looks right, never whether the data survives. Nothing prompted for every option of a bounded choice, numeric boundaries, derived fields, child CRUD inside a parent form, which gesture persists what, or leaving with unsaved work. On a form-heavy change the matrix simply never asked. A mocked unit spec no longer retires a manual case. The dedup rule dropped any axis a PR-added test covered, with no caveat. PR #37064 added 162 mockReturnValue calls, four of them mocking $hasUnsavedChanges, and zero e2e specs -- so the unsaved-changes guard was retired from the manual plan on the strength of a spec that builds a fake component. A mock proves component logic, not a persistence round-trip, a route change or a status transition. A run now resolves a PR set as well as an issue set. The skill was single-PR by construction: section 5 diffed one <pr>, section 2 resolved issues and never PRs. Spec-driven work lands as several PRs against the same issues, and the skill can be handed issues alone. Coverage now runs over the union of the set's diffs, the Summary names the PRs covered and any deliberately left out, and unmerged PRs are excluded and said to be excluded. Also regenerates CATALOG.md, which was already stale on main by one count line after speckit-docs-converge landed -- unrelated to this change, but skill-lint gates every PR touching .claude/skills/**. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Claude finished @erickgonzalez's task in 1m 26s —— View job ReviewDocumentation-only change to the Note on environment: the working tree in my checkout carries an uncommitted revert of this PR (16 insertions / 98 deletions — the exact inverse), so Claims I verified
New IssuesNo issues found. The change is coherent, the dedup/mocked-spec caveat is now stated identically in both places, and the "does not exist in dotCMS" carve-out is backed by real code references rather than assertion. The self-caught third drift instance ( One forward-looking note you already raised in the description stands on its own merit: at 445 lines · |
Three lines in the file this PR edits still described the state before the
edit:
- the header said "Nine product-surface axes" while the table below it had
ten. This is an instruction file a model reads to decide which axes to
walk, so an undercount risks the new axis being treated as optional --
exactly what adding it was meant to prevent
- the walk instruction said "the union of everything the PR changed",
singular, contradicting SKILL.md section 6's "union of the PR set's diffs"
- the dedup rule said "a test the PR itself added", also singular, and
restated the rule without the mocked-spec caveat added to section 5, so
the two files disagreed on when an axis may be dropped
The first two were raised in review; the third is the same drift one line
further down and would have left the reference file contradicting SKILL.md on
the substantive change in this PR.
Verified consistent across all four files: ten axis rows, ten in the header,
ten in the README, and both the matrix and section 6 walking the PR set.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Both valid — fixed in 2488f01, along with a third instance of the same drift that the review didn't catch. 🟡 "Nine product-surface axes" — fixedRight, and right about why it matters rather than being cosmetic. This is an instruction file a model 🟡 "the union of everything the PR changed" — fixedNow A third one, same drift, one line further down
Root causeAll three are the same mistake: I edited this file to add a row and a checklist and never re-read the Verified consistent across all four files now: ten axis rows, ten in the matrix header, ten in the |
…have
Generated plans have been carrying a "verify light and dark mode" case for a
feature that does not exist. The UI/UX checklist asked for it directly --
"Theme / contrast: light and dark both render correctly" -- so the case was
prompted by the skill, not invented from thin air, and every one of them was a
hallucination a reviewer had to spend attention rejecting.
Verified before encoding the constraint:
- core-web/libs/ui/src/lib/theme/providers.ts pins PrimeNG's
darkModeSelector to `options?.darkModeSelector ?? false`
- no file under core-web/libs or core-web/apps reads prefers-color-scheme
- no element carries a data-theme attribute, and there is no theme switcher
- the repository's only `.dark` selector is in the standalone block-editor
app config, with nothing wired to activate it
Split the checklist item: contrast is real and stays, extended to disabled,
error and selected states. Light/dark is removed and named explicitly in a new
"Does not exist in dotCMS" section, because deleting the prompt is not enough
on its own -- a model generating a UI plan will reach for dark mode from
general web knowledge unless told it is absent, and told why.
The section is scoped so it cannot become stale in the wrong direction: if a
change genuinely introduces one of these, the diff shows it and it is in scope
like any other new behaviour.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Proposed Changes
Three gaps found by reviewing a real run of the skill. Two further findings from the same review are
deliberately not here — see Not in this PR.
1 · A tenth coverage axis: Form & state semantics. The existing nine axes are platform concerns
(permissions, hosts, languages, version state, cache, workflow, push publish, persistence, UI/UX),
and the UI/UX checklist is purely presentational — rendering, keyboard, focus, screen reader, empty
states, responsive, i18n, theme, copy. It asks whether a control looks right; nothing asked whether
the data survives. The new axis prompts for every option of a bounded choice, numeric boundaries,
derived fields, child-entity CRUD inside a parent form, optional-set-vs-unset, which gesture persists
what, leaving with unsaved work, and locked states. On a form-heavy change the matrix previously
never asked.
2 · A mocked unit spec no longer retires a manual case. §5 dropped any axis a PR-added test
covered, with no caveat. PR #37064 added 162
mockReturnValuecalls — four mocking$hasUnsavedChanges— and zero e2e specs, so the unsaved-changes guard was retired from the manualplan on the strength of a spec that builds a fake component. A mock proves component logic, not a
persistence round-trip, a real route change, or a status transition.
3 · A run resolves a PR set as well as an issue set. The skill was single-PR by construction:
§5 diffed one
<pr>and §2 resolved issues but never PRs. Spec-driven work lands as several PRsagainst the same issues, and the skill can legitimately be handed issues alone. §2 now defines the
input contract for all three shapes; coverage runs over the union of the set's diffs; the Summary
names the PRs covered and any deliberately left out; unmerged PRs are excluded and said to be.
Checklist
Additional Info
Not in this PR, from the same review:
proposed trigger was "when the diff contradicts the description". It doesn't fit the case that
prompted it: Experiments Portlet — Screen 2: Create/Update (/experiments/new + /:id/configuration) #37003 specified autosave with no Save button, and feat(experiments): create/update experiment screen (/experiments/new + /:id/configuration) #37064 shipped autosave
(
AUTOSAVE_DEBOUNCE_MS = 500, 158 PATCH lines,replaceUrl) plus Save Draft and a route guard.Nothing was contradicted — it was additive — so a contradiction-keyed rule would never fire. Needs
reframing around unmapped shipped behavior before it's worth encoding.
AC-3:/US1-AS3:citation convention already carries traceability in case names. Open question, not adefect.
Unrelated: this also regenerates
CATALOG.md, which was already stale onmainby one countline after
speckit-docs-convergelanded. Not part of this change, butskill-lintgates every PRtouching
.claude/skills/**.SKILL.mdis now 445 lines — still under the ~500 convention, but close enough that the nextaddition should move into
references/.This PR fixes: #37225