Skip to content

agentic-toolkit: align with the living-doc canon #39

Description

@miroslavpojer

Description of Technical Debt

agentic-toolkit authors the issue bodies, feature-file headers and PageObject headers that the living-doc collectors mine, but its own skills, agent definitions, schemas and docs disagree with the canon living-doc now enforces (verified at merge commit bfcc402ff998085cbf7bb91a7fd55ea8ac12c911):

  • skills/living-doc-update/SKILL.md:139-160 still writes status: descoped, descoped_at, descoped_reason, future_release, and an AC line like (v1.2.0 – descoped) using en-dash bullets; :163-166 puts the AC description on the header line after an em dash instead of on a - Rationale: bullet.
  • skills/gherkin-living-doc-sync/SKILL.md (lines 7, 11, 171, 212), evals/evals.json:154-155 and docs/guides/gherkin-living-doc-sync.md (lines 15, 32) describe "descoped" scenarios with a target-release reference instead of (planned) / (v<target> - planned).
  • skills/living-doc-update/evals/evals.json:108-112 and evals/fixture-map.md:13 still expect the descoped state.
  • .github/agents/living-doc-bdd-copilot.agent.md:278 uses an en-dash in the AC reference format; roughly 81 lines across skills/, .github/, docs/ combine AC: with an en/em dash, plus 8 dash-bulleted lines.
  • skills/shared/references/schemas/project-profile.schema.json:40-44 allows an arbitrary ac_states value (mentions In Review) instead of a subset of the four canonical states, and still carries pageobject_statuses: [planned, candidate, active, deprecated], also present in living-doc-bdd-schemas.md (lines 46, 252, 264, 309, 349, 352).
  • skills/living-doc-create-feature/SKILL.md (lines 79, 174) assigns a Feature a status (including candidate); skills/data-cy-instrument/SKILL.md:207 "promotes" status: candidate to active instead of removing stub-reason:; the create-feature, scenario-creator, pageobject-scan and gap-finder evals all expect a Feature or PageObject status.
  • skills/shared/references/living-doc-glossary.md is an older, unsynced copy of the canon glossary and has drifted.

The canon it should follow now lives in living-doc's docs/guides/living-doc-glossary.md, docs/guides/living-doc-header-types.md and docs/guides/living-doc-document-types.md (a later living-doc move relocates these under docs/reference/; the sync script should be written against the current path and updated when that move lands).

Impact of Technical Debt

  • Content authored by these skills (issue bodies, feature-file headers, PageObject headers) will fail collection or normalization once the collectors move onto the new canon — dashes, AC states, and status placement all disagree.
  • The stale pageobject_statuses / Feature status fields actively mislead authors and tooling into producing a field the canon no longer recognizes, and drop the derived-Feature-state / orphan-Feature reporting the canon expects instead.
  • Evals encode the wrong expected output, so they'll pass against behavior that is itself wrong, hiding regressions instead of catching them.
  • skills/shared/references/living-doc-glossary.md being an unsynced copy means authors following it are working from out-of-date rules with no automated way to detect drift.

Category

Documentation

(Cuts across Code Quality/Refactoring and Testing/Test Coverage too — schemas, skill prompts, and eval fixtures all need updates alongside the docs.)

Priority

High - Actively causing problems

(Blocks the living-doc GitHub-chain milestone gate, which requires this alignment merged before real-project verification.)

Proposed Solution

  1. Add a glossary sync script, scripts/sync-living-doc-glossary.sh <living-doc-ref>, that fetches the canon glossary from living-doc at the given ref, rewrites its relative links to absolute living-doc URLs, and writes the local copy under skills/shared/references/living-doc-glossary.md with a header comment <!-- synced from AbsaOSS/living-doc@<ref> -->. Add a test-scripts.yml check that re-runs the script for the ref recorded in that header and fails the build on any difference.
  2. Switch to one dash. Replace every en/em dash used in generated AC headers, AC bullets, entity names, and # AC: comments with a plain hyphen; move the AC description off the header line and onto a - Rationale:-style bullet wherever a skill currently appends it after a dash.
  3. Rewrite the descope procedure. An AC moved out of scope becomes (v<target> - planned) when a target version is known, otherwise (planned); the reason goes on a - Rationale: bullet. Linked Gherkin scenarios keep their @AC: tag and gain @wip + @review-needed. User-facing trigger phrases such as "AC was descoped" stay as-is in skill descriptions and in evals/trigger-eval.json user queries — only the produced output changes.
  4. Constrain ac_states. In project-profile.schema.json, make ac_states an enum that is a subset of the four canonical states (planned, in_review, active, deprecated); omitting the field stays allowed, but no other spelling or extra state is accepted.
  5. Drop Feature and PageObject status. No skill should write a Feature status or a PageObject status: field, and pageobject_statuses should be removed from the schema and from living-doc-bdd-schemas.md. An uninstrumented surface keeps stub-reason:; instrumenting it means removing stub-reason:, not flipping a status value. A Feature with no linked User Stories or Functionalities should be reported as an orphan (a distinct ORPHAN_FEATURE condition), not tagged candidate.
  6. Limit ## Status / # status: to User Stories and Functionalities in every generated entity file and issue body.
  7. Update every affected eval fixture (living-doc-update, gherkin-living-doc-sync, living-doc-create-feature, data-cy-instrument, scenario-creator, pageobject-scan, gap-finder) so expected outputs match the corrected behavior, and update evals/fixture-map.md accordingly.
  8. Restore the Project Profile / seed.yaml / manifest.json reference sections here if living-doc has removed its own copies in favor of linking out to this repo — add them if missing, and link back to the relevant living-doc reference docs.

Acceptance Criteria

  • skills/shared/references/living-doc-glossary.md matches scripts/sync-living-doc-glossary.sh's output for the ref recorded in its header comment; test-scripts.yml checks this.
  • No line under skills/, .github/, docs/ combines AC: with an en/em dash, opens with a dash bullet (- / / ), or combines a US-/FEAT-/FUNC- id with a middle-dot and an en/em dash.
  • No descoped state, status: descoped, descoped_at, or descoped_reason remains outside the user-query strings in evals/trigger-eval.json; descope output uses (planned) / (v<target> - planned) plus a - Rationale: bullet.
  • No status: candidate or pageobject_statuses remains outside the user-query strings in evals/trigger-eval.json; project-profile.schema.json rejects ac_states: [In Review], ac_states: [done], and any pageobject_statuses key (covered by the repo's own schema tests).
  • living-doc-create-feature produces no Feature status field; data-cy-instrument removes stub-reason: on instrumentation instead of changing a status value.
  • Every eval fixture map and affected expected-output fixture is updated to match; the repo's own checks (including test-scripts.yml) pass.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions