Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
fcd8390
Sync living-doc glossary with canonical definitions, update project p…
miroslavpojer Sep 15, 2026
3351f4f
fix: align living-doc-update description with schema (Features no sta…
miroslavpojer Sep 15, 2026
d3d4d43
fix: clarify stale AC update location in gherkin-living-doc-sync
miroslavpojer Sep 15, 2026
fc9711a
feat: add standard artifact locations for BDD automation in copilot i…
miroslavpojer Sep 15, 2026
c609578
fix: update living-doc BDD schemas and related documentation for clar…
miroslavpojer Sep 16, 2026
c095dff
fix: update expected output formatting in evals and enhance deprecati…
miroslavpojer Sep 16, 2026
429db05
fix: update surface type validation to only allow UI and API, and adj…
miroslavpojer Sep 16, 2026
f525f3f
fix: refine expected output for Notifications Service feature and upd…
miroslavpojer Sep 16, 2026
c6c9d48
fix: update glossary sync script to use environment variable for refe…
miroslavpojer Sep 16, 2026
96c58bd
fix: enhance living-doc BDD schemas and documentation for clarity, ad…
miroslavpojer Sep 16, 2026
7607373
fix: update expected output and validation details in evals for livin…
miroslavpojer Sep 16, 2026
377c154
fix: update descoped scenario handling in documentation and evals, cl…
miroslavpojer Sep 16, 2026
5a02872
fix: enhance validation for ac_states in profile, ensuring schema com…
miroslavpojer Sep 16, 2026
22f3404
fix: add validation to prevent duplicate ac_states in profile and enh…
miroslavpojer Sep 16, 2026
822aad2
fix: enhance validation for profile ac_states to reject non-string it…
miroslavpojer Sep 16, 2026
e7a8e00
fix: add tests to reject duplicate values in profile ac_states and up…
miroslavpojer Sep 16, 2026
602a738
fix: update living doc guidelines to clarify API surface types and en…
miroslavpojer Sep 16, 2026
d5187ca
fix: enhance living doc guidelines for AC versioning and improve vali…
miroslavpojer Sep 16, 2026
2378d51
fix: update living doc guidelines and improve validation for user sto…
miroslavpojer Sep 16, 2026
748b767
fix: enhance error handling in profile loading to catch malformed YAML
miroslavpojer Sep 16, 2026
1ae73b5
fix: update documentation and tests for validation and coverage impro…
miroslavpojer Sep 17, 2026
d1ec392
fix: enforce error handling for unrecognized AC states in validation …
miroslavpojer Sep 17, 2026
0318a13
fix: update living doc guidelines and JSON structure for User Stories…
miroslavpojer Sep 17, 2026
ff2e547
fix: enhance validation for deprecated features to include deprecated…
miroslavpojer Sep 17, 2026
4b46118
fix: update living doc guidelines and JSON structure for Feature and …
miroslavpojer Sep 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 13 additions & 9 deletions .github/agents/evals/living-doc-bdd-copilot/evals.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,12 @@
"id": 3,
"category": "happy-path",
"prompt": "Generate Gherkin scenarios for US-007 — Place an Online Order. ACs: (1) active — happy path: customer places order with saved payment. (2) active — error: order rejected when card declined.",
"expected_output": "Agent generates a .feature file named us-007-place-an-online-order.feature. Feature header uses the As-a/I-can/so-that narrative from US-007. Two scenarios generated one per active AC. Each Scenario: is immediately preceded by a '# AC: US-007-0n (v1.0.0 active) ...' traceability tag. Step text uses domain language (no HTTP calls, selectors, or DB). For steps without a matching step definition, generates stubs: Case A (PageObject method exists) = full stub; Case B (no PageObject method) = stub with NotImplementedError and flag to extend the PageObject.",
"expected_output": "Agent generates a .feature file named us-007-place-an-online-order.feature. Feature header uses the As-a/I-can/so-that narrative from US-007. Two scenarios generated - one per active AC. Each Scenario: is immediately preceded by a '# AC:US-007-0n (v1.0.0 - active) - ...' comment AND carries the matching '@AC:US-007-0n' tag directly above the Scenario: line — the agent contract requires both the human-readable comment and the machine-readable tag on every scenario. Step text uses domain language (no HTTP calls, selectors, or DB). For steps without a matching step definition, generates stubs: Case A (PageObject method exists) = full stub; Case B (no PageObject method) = stub with NotImplementedError and flag to extend the PageObject.",
"files": [],
"expectations": [
"Feature file named us-007-place-an-online-order.feature",
"Each Scenario: immediately preceded by a # AC: traceability comment",
"Each Scenario: also carries the matching @AC: tag — comment alone is not sufficient",
"Skips planned and deprecated ACs — only active ACs drive generation",
"Step text in domain language — no implementation details",
"Case A stubs delegate to PageObject methods",
Expand Down Expand Up @@ -160,13 +161,14 @@
"id": 12,
"category": "output-format",
"prompt": "After scanning /login and generating a LoginPage, show me what the manifest.json entry for /login looks like.",
"expected_output": "The manifest.json entry for /login includes: pageobject_path (path to the generated LoginPage file), feature_id (FEAT-<nnn> or FEAT-UNKNOWN if unlinked), last_scanned (ISO timestamp), elements (list of discovered elements with data_cy and tag), coverage_gaps (empty list initially), and navigation_context with prerequisites, navigation_steps, data_requirements, auth_role, and notes. The feature_id is FEAT-UNKNOWN if no matching Feature entity exists in the living doc — flag this route as 'needs Feature entity' and load `living-doc-create-feature` skill to create it.",
"expected_output": "The manifest.json entry for /login includes: pageobject_path (path to the generated LoginPage file), feature_id (FEAT-<nnn> or FEAT-UNKNOWN if unlinked), last_scanned (ISO timestamp), elements (list of discovered elements, each with the normalized test_id key and tag — not data_cy), coverage_gaps (empty list initially), and navigation_context as a prose STRING describing how to reach the route (e.g. prerequisites, navigation steps, auth role) — not an object. The feature_id is FEAT-UNKNOWN if no matching Feature entity exists in the living doc — flag this route as 'needs Feature entity' and load `living-doc-create-feature` skill to create it.",
"files": [],
"expectations": [
"manifest.json entry has: pageobject_path, feature_id, last_scanned, elements, coverage_gaps, navigation_context",
"feature_id is FEAT-UNKNOWN if no matching Feature entity exists",
"last_scanned is an ISO timestamp",
"navigation_context includes prerequisites, navigation_steps, auth_role",
"elements use the normalized 'test_id' key, not 'data_cy'",
"navigation_context is a prose string (not an object) describing prerequisites, navigation steps, and auth role",
"Missing Feature entity triggers living-doc-create-feature skill load"
]
},
Expand Down Expand Up @@ -215,7 +217,7 @@
"id": 16,
"category": "skill-dispatch",
"prompt": "The @AC: traceability tags in checkout.feature are out of sync with the living doc. Sync them.",
"expected_output": "Agent loads the `gherkin-living-doc-sync` skill. Runs scan_ac_links.py to audit the @AC: tags and # AC: comments across checkout.feature. For each scenario: (1) verifies the @AC: tag matches a live AC in the catalog; (2) checks the # AC: comment format is canonical; (3) flags stale or missing links. Produces a sync diff showing what changed. Does not generate new scenarios routes new-scenario requests to living-doc-scenario-creator.",
"expected_output": "Agent loads the `gherkin-living-doc-sync` skill. Runs scan_ac_links.py to audit the @AC: tags and # AC: comments across checkout.feature. For each scenario: (1) verifies the @AC: tag matches a live AC in the catalog; (2) checks the # AC: comment format is canonical; (3) flags stale or missing links. Produces a sync diff showing what changed. Does not generate new scenarios - routes new-scenario requests to living-doc-scenario-creator.",
"files": [],
"expectations": [
"Loads gherkin-living-doc-sync skill",
Expand Down Expand Up @@ -257,11 +259,11 @@
"id": 19,
"category": "skill-dispatch",
"prompt": "Delete all BDD artifacts linked to the deprecated FEAT-012 feature — clean up feature files, step definitions, and PageObjects.",
"expected_output": "Agent loads the `bdd-maintain` skill in REMOVE mode. Identifies all .feature file scenarios whose # AC: tags reference FEAT-012 User Stories. Runs find_unused_steps.py to find step definitions only used by those scenarios. Runs find_unused_po_methods.py and find_unused_po_components.py to identify exclusively-used PageObject artifacts. Presents the full deletion list to the user for confirmation before touching any file. Also checks fixtures.ts for PageObject imports to remove. After confirmation, removes the identified files.",
"expected_output": "Agent loads the `bdd-maintain` skill in REMOVE mode. Identifies all .feature file scenarios whose @AC: tags reference FEAT-012 User Stories. Runs find_unused_steps.py to find step definitions only used by those scenarios. Runs find_unused_po_methods.py and find_unused_po_components.py to identify exclusively-used PageObject artifacts. Presents the full deletion list to the user for confirmation before touching any file. Also checks fixtures.ts for PageObject imports to remove. After confirmation, removes the identified files.",
"files": [],
"expectations": [
"Loads bdd-maintain skill",
"REMOVE mode identifies scenarios via # AC: tags linked to deprecated Feature",
"REMOVE mode - identifies scenarios via @AC: tags linked to deprecated Feature",
"Runs all three audit scripts to identify exclusively-used artifacts",
"Checks fixtures.ts for PageObject imports",
"Presents deletion list for confirmation before touching any file"
Expand All @@ -271,12 +273,13 @@
"id": 20,
"category": "skill-dispatch",
"prompt": "Document the Notifications Service as a Feature entity in the living doc — it exposes a REST API at /api/notifications.",
"expected_output": "Agent loads the `living-doc-create-feature` skill. Assigns the next sequential FEAT-nnn ID using next_id.py. Creates a Feature entity JSON with: id, name ('Notifications Service'), type ('api'), route ('/api/notifications'), status ('active'), owners (asks user), and empty functionalities and user_stories arrays. Adds the Feature to feature_registry.json. Prompts: 'Do you want to create Functionality entities for specific behaviors of this service?'",
"expected_output": "Agent loads the `living-doc-create-feature` skill. Assigns the next sequential FEAT-nnn ID using next_id.py. Creates a Feature entity JSON with: id, name ('Notifications Service'), surface_type ('API'), purpose, owners (asks user), external_dependencies, and empty `functionalities`/`user_stories` arrays since the prompt supplies no relationships — no `status` field, since a Feature's state is derived from its Functionalities. Repeats the orphan warning outside the JSON (no traceable business value until linked to a User Story; reported as `ORPHAN_FEATURE` by living-doc-gap-finder) rather than fabricating placeholder IDs. Adds the Feature to feature_registry.json. Prompts: 'Do you want to create Functionality entities for specific behaviors of this service?'",
"files": [],
"expectations": [
"Loads living-doc-create-feature skill",
"Assigns next FEAT-nnn ID using next_id.py",
"Creates Feature entity JSON with required fields",
"Creates Feature entity JSON with required fields and empty functionalities/user_stories arrays — no fabricated placeholder IDs",
Comment thread
miroslavpojer marked this conversation as resolved.
"Repeats the ORPHAN_FEATURE warning outside the JSON",
"Adds entry to feature_registry.json",
"Prompts for Functionality creation as next step"
]
Expand Down Expand Up @@ -314,12 +317,13 @@
"id": 23,
"category": "skill-dispatch",
"prompt": "Generate BDD scenarios for US-007 — Place an Online Order. It has 3 active ACs and 1 planned AC.",
"expected_output": "Agent loads the `living-doc-scenario-creator` skill. Generates a .feature file for US-007 with 3 scenarios one per active AC. The planned AC is skipped (not generated until active). Each scenario is preceded by a '# AC: US-007-0n (v1.0.0 active)' traceability comment. Merge policy: if a scenario already exists for an AC, applies the 4-row decision table (skip if intent matches, update if GWT is stale, propose replacement if deprecated, flag if multiple scenarios exist per AC).",
"expected_output": "Agent loads the `living-doc-scenario-creator` skill. Generates a .feature file for US-007 with 3 scenarios - one per active AC. The planned AC is skipped (not generated until active). Each scenario is preceded by a '# AC:US-007-0n (v1.0.0 - active)' comment AND carries the matching '@AC:US-007-0n' tag directly above the Scenario: line — both are required. Merge policy: if a scenario already exists for an AC, applies the 4-row decision table (skip if intent matches, update if GWT is stale, propose replacement if deprecated, flag if multiple scenarios exist per AC).",
"files": [],
"expectations": [
"Loads living-doc-scenario-creator skill",
"Generates scenarios only for active ACs — skips planned",
"Each scenario preceded by # AC: traceability comment",
"Each scenario also carries the matching @AC: tag — comment alone is not sufficient",
"Applies merge policy decision table for existing scenarios",
"Output is a named .feature file: us-007-place-an-online-order.feature"
]
Expand Down
8 changes: 4 additions & 4 deletions .github/agents/living-doc-bdd-copilot.agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,15 +178,15 @@ When a User Story or Feature is deprecated, three skills fire in sequence. Compl

| Step | Skill | Action |
|---|---|---|
| 1 | `living-doc-update` | Set entity `status: deprecated`; add `deprecated_at`, `deprecation_reason`, and optionally `superseded_by` |
| 1 | `living-doc-update` | For a User Story, set `status: deprecated`. A Feature has no `status` field — its state is derived from its Functionalities; deprecate the Feature by deprecating every Functionality it owns instead. Either way, add `deprecated_at`, `deprecation_reason`, and optionally `superseded_by` |
Comment thread
miroslavpojer marked this conversation as resolved.
| 2 | `gherkin-living-doc-sync` | Find all scenarios tagged `@AC:<id>` for the deprecated entity's ACs; add `@deprecated` and `@review-needed` |
| 3 | `bdd-maintain` (REMOVE) | Confirm file deletion list with user; remove confirmed `.feature` files, PageObjects, and step definitions; update `manifest.json` |

Do not skip steps or run them out of order. Complete catalog changes (step 1) before touching any Gherkin or automation files.

**Manifest loading rule:** Use targeted line ranges for the current route(s). Load full manifest only for RE-SCAN. `seed.yaml`: always load in full. When PageObject generation discovers a route with no linked Feature entity, set `feature_id: FEAT-UNKNOWN`, flag the route as needing a Feature entity, and cross-load `living-doc-create-feature` to create it before continuing.

**living-doc-bdd-schemas:** Load [skills/shared/references/living-doc-bdd-schemas.md](skills/shared/references/living-doc-bdd-schemas.md) only when generating or validating feature file headers, PageObject headers, ExplorationFixture entries, seed.yaml form_fixtures, or manifest.json route entries.
**living-doc-bdd-schemas:** Load [skills/shared/references/living-doc-bdd-schemas.md](skills/shared/references/living-doc-bdd-schemas.md) only when generating or validating feature file headers, PageObject headers, seed.yaml form_fixtures, or manifest.json route entries.

---

Expand Down Expand Up @@ -275,12 +275,12 @@ Full model: [living-doc-glossary](skills/shared/references/living-doc-glossary.m

**Entity IDs:** `US-<nnn>` · `FEAT-<nnn>` · `FUNC-<nnn>`

**AC reference format:** `AC:<parent-id>-<nn> (v<version> <state>) <description>`
**AC reference format:** `AC:<parent-id>-<nn> (v<version> - <state>) - <description>`. A backlog AC with no target version yet uses `(planned)` instead of a version. A `deprecated` AC requires a removal note: `(v<version> - deprecated - removal planned v<version>)`.
State: `planned | in_review | active | deprecated`

**Gherkin traceability:** every scenario in the living-doc feature directories (`feature_dirs.user_story` and `feature_dirs.functionality` from the Project Profile, defaults `features/liv_doc_us/` and `features/liv_doc_func/`) requires:
```gherkin
# AC:US-1-01 (v1.0.0 - active) <description>
# AC:US-1-01 (v1.0.0 - active) - <description>
@AC:US-1-01
Scenario: ...
```
Expand Down
16 changes: 16 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,22 @@ See [Token-Saving Skill](../docs/guides/token-saving.md) for details, including

---

## BDD Artifacts

When working with living documentation and BDD automation, use these standard artifact locations:
- **Business Seed:** `.copilot/bdd/seed.yaml` — App configuration, routes, test users, form fixtures
- **Exploration Manifest:** `.copilot/bdd/manifest.json` — Discovered surfaces, PageObject paths, UI elements
- **Project Profile:** `.copilot/bdd/.project-profile.yaml` — BDD conventions for this project
- **Feature Files (User Story):** `features/liv_doc_us/` — E2E scenarios linked to User Stories
- **Feature Files (Functionality):** `features/liv_doc_func/` — System-test scenarios linked to Functionalities
- **PageObjects:** `playwright/pages/` — Locators and page interaction methods
- **Step Definitions:** `playwright/steps/` — Gherkin step implementations
- **Living Doc Catalog:** `docs/living-doc/` — User Stories, Features, Functionalities, and Acceptance Criteria

These paths are configurable in `.copilot/bdd/.project-profile.yaml`.

---

## PR Review

When reviewing a pull request, load and apply:
Expand Down
37 changes: 36 additions & 1 deletion .github/workflows/test-scripts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,47 @@ on:
pull_request:
paths:
- "skills/**/*.py"
- "skills/shared/references/schemas/*.schema.json"
- "scripts/**"
- "skills/shared/references/living-doc-glossary.md"
Comment thread
miroslavpojer marked this conversation as resolved.
- ".github/workflows/test-scripts.yml"

permissions:
contents: read

jobs:
glossary-sync-check:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Extract synced ref from the glossary header
id: ref
run: |
set -euo pipefail
header="$(head -n1 skills/shared/references/living-doc-glossary.md)"
ref="$(echo "$header" | sed -nE 's/.*synced from AbsaOSS\/living-doc@([^ ]+).*/\1/p')"
if [ -z "$ref" ]; then
echo "Error: could not find a 'synced from AbsaOSS/living-doc@<ref>' header comment on line 1 of the glossary." >&2
exit 1
fi
echo "ref=$ref" >> "$GITHUB_OUTPUT"

- name: Re-run the sync script and diff against the committed copy
env:
GLOSSARY_REF: ${{ steps.ref.outputs.ref }}
run: |
set -euo pipefail
cp skills/shared/references/living-doc-glossary.md /tmp/committed-glossary.md
bash scripts/sync-living-doc-glossary.sh "$GLOSSARY_REF"
if ! diff -u /tmp/committed-glossary.md skills/shared/references/living-doc-glossary.md; then
echo
echo "skills/shared/references/living-doc-glossary.md is out of sync with AbsaOSS/living-doc@${GLOSSARY_REF}." >&2
echo "Run: scripts/sync-living-doc-glossary.sh ${GLOSSARY_REF}" >&2
exit 1
fi

run-tests:
runs-on: ubuntu-latest
steps:
Expand All @@ -22,7 +57,7 @@ jobs:
python-version: "3.11"

- name: Install dependencies
run: pip install pyyaml
run: pip install pyyaml jsonschema

- name: Discover and run test_*.py scripts
env:
Expand Down
4 changes: 2 additions & 2 deletions docs/guides/agent-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,12 +234,12 @@ IDs are stable — never change an ID after creation. Bump the `version` field f
### Gherkin traceability tag format

```gherkin
# AC:US-007-01 (v1.0.0 - active) <description>
# AC:US-007-01 (v1.0.0 - active) - <description>
@AC:US-007-01
Scenario: ...
```

One `# AC:` + `@AC:` pair per AC. The `@AC:` tag is the machine-readable traceability anchor never delete or rename it without syncing the catalog entity.
One `# AC:` + `@AC:` pair per AC. The `@AC:` tag is the machine-readable traceability anchor - never delete or rename it without syncing the catalog entity.

### Cooperating agent boundary

Expand Down
2 changes: 1 addition & 1 deletion docs/guides/data-cy-instrument.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ coverage gaps

## Testing Evals

This skill has been validated with **11 test cases** covering:
This skill has been validated with **12 test cases** covering:
- Gap audit workflows
- Angular/React/Vue template instrumentation
- PageObject selector updates
Expand Down
2 changes: 1 addition & 1 deletion docs/guides/gherkin-living-doc-sync.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The skill audits traceability and produces sync reports:
|--------|-------------|
| **AC link audit** | Finds missing `@AC:` tags, stale links, orphaned scenarios |
| **Step text drift** | Flags scenarios where step text diverged from AC descriptions |
| **Deprecation tagging** | Tags scenarios `@deprecated` when ACs are descoped |
| **Deprecation / descope tagging** | Tags scenarios `@deprecated` + `@review-needed` when their AC is deprecated, or `@wip` + `@review-needed` when their AC is descoped back to `planned` |
| **AC change propagation** | Maps AC edits back to affected scenarios |

---
Expand Down
Loading
Loading