[iRaindrop]

Provides the directory structure and guidance under .github in the project's doc website repo for conducting an AI-assisted analysis using Copilot GitHub Enterprise.

[netlify]

👷 Deploy request for cncf-techdocs pending review.

Visit the deploys page to approve it

Name	Link
🔨 Latest commit	2339f76

[iRaindrop]

Reviewed the harness against the KubeVirt .github folder it was copied from. The structure copied cleanly (all content diffs are Prettier line-wrapping). The comments below cover what is still KubeVirt-specific and what would make the harness complete end-to-end for the next project.

Suggestions not tied to a specific line:

• Add a /rank-recommendations prompt that produces input/ai-ranked-recommendations.csv. For KubeVirt this ranking step was ad hoc; codifying it closes the only manual gap in the pipeline.
• Reference (or add) the upload process. The issue prompts say "a process will upload" the files, but the harness does not say where that script or workflow lives.
• Record the validated toolchain in ai-harness.md: Copilot CLI version, model, and that the prompt files rely on mode: agent and ${input:title}.
• Add a pre-flight checklist: required labels exist in the target repo (help wanted, the three priority labels), the project's analysis.md is scaffolded from the techdocs template, and .github has no conflicting prompt files.
• Consider an examples/kubevirt/ folder holding the KubeVirt "Current Repository" section and issue template, so the generic files stay generic but a worked example is still available.

[iRaindrop]

Second pass over the harness at a834e7d. All earlier threads are addressed, CI is green, and the structure is consistent end to end (criteria → wrappers → engines → issue prompts → upload process). The remaining edits are small; only the first one affects a data contract (the CSV Section value), the rest are wording and clarity. None block the merge; they can land in a follow-up if preferred.

[iRaindrop]

Ready to merge

Note: This commit fixes a link issue from 404s in older Analyses files, currently preventing the other open PRs from merging:

6a4ef0e

[kmkhz]

Minor clean-up from my last review.

[kmkhz]

Three items in this review:

1. I recommend adding a README.md in addition to the index.md file; see in line below.

2. A question/suggestion about the area wrapper files (e.g., docs/analysis/ai-harness/_github/prompts/areas/information-architecture-answers.prompt.md):
   I'm concerned about maintainablility and drift. Are these area-specific wrappers generated from a canonical template or manifest? If so, please point to that source and document the regeneration step. If not, let's add area wrapper file templates and a generator so updates to the shared wrapper structure can be made consistently.

3. To further prevent drift, we should consider a CI step and pre-commit hook that warn or error on changes to these template files.

[kmkhz]

To improve discoverability, add a README to the ai-harness directory (in addition to this index.md); for example:
docs/analysis/ai-harness/README.md

     See docs/analysis/ai-harness/index.md for more info
     Copy docs/analysis/ai-harness/_github/ to the target repository’s .github/
     Replace placeholder values in the “Current Repository” section of copilot-instructions.md
     Run the prompts from the target repo root

[iRaindrop]

Three items in this review:

1. I recommend adding a README.md in addition to the index.md file; see in line below.
2. A question/suggestion about the area wrapper files (e.g., docs/analysis/ai-harness/_github/prompts/areas/information-architecture-answers.prompt.md):
   I'm concerned about maintainablility and drift. Are these area-specific wrappers generated from a canonical template or manifest? If so, please point to that source and document the regeneration step. If not, let's add area wrapper file templates and a generator so updates to the shared wrapper structure can be made consistently.
3. To further prevent drift, we should consider a CI step and pre-commit hook that warn or error on changes to these template files.

Thanks Kristen. Nate should comment on 1 and 3 - A README is a good idea, but would serve the same function as _index.md - do we really need both? Note that "_github" might be renamed. Regarding 2, the canonical template is the criteria.md, as far as the criteria questions and wrapper files are concerned, and to a lesser extent the analysis.md template. I've entertained thoughts of a questions manifest spreadsheet for easier maintainability.

[kmkhz]

A README is a good idea, but would serve the same function as _index.md - do we really need both?

The main purpose of the README would be to point to the index.md, since GitHub automatically renders the README.md file directly beneath its file list table. I looked for a README before opening index.md.

I've entertained thoughts of a questions manifest spreadsheet for easier maintainability.

Yup, that was my main thesis.

Thanks!

-K

[hivecommons-hive]

I read this branch's tree rather than just the diff: index.md, copilot-instructions.md, the templates, the issue prompts, and spot-checks across criteria/ and prompts/. The harness is internally consistent and looks correct to me:

• index.md says "seventeen areas"; _github/criteria/areas/ has exactly 17 files, and the 17 area-name prefixes match 1:1 between criteria/areas/ and prompts/areas/. The 72 prompt files decompose cleanly as 17 areas × 4 variants (-full, -answers, -comment, -recommendations) + 4 generic prompts.
• The #{Issue number} suffix in the doc-issue.md template title is handled by the documented upload step (index.md, Upload process: "the front matter title (minus the …"), so it is intentional, not a YAML-comment hazard.
• Everything lands under docs/analysis/ai-harness/_github/ — a template directory, not this repo's live .github/, so no workflow, permission, or CI surface is touched. The .cspell.yml change adds three words.
• The pre-flight checklist sensibly requires maintainer agreement and keeps .github/input/.github/output on a local branch of the target repo.

This is purely additive documentation, all checks are green, and #385 is visibly the worked output of this harness (its examples/kubevirt/ here matches that PR's provenance). Not a duplicate of #385: that PR adds KubeVirt analysis outputs under analyses/2026/kubevirt/; this one adds the reusable tooling under docs/analysis/ai-harness/. Only .cspell.yml overlaps, in non-conflicting hunks.

No defects found. Merge decision is the maintainers' as usual; nothing here blocks it.

— hive: agent=reviewer backend=copilot model=claude-fable-5

[hivecommons-hive]

Read the full tree at head 62e0bad. This is a purely additive docs/template PR (110 new files under docs/analysis/ai-harness/, plus three .cspell.yml words), and it holds together well: the "seventeen areas" claimed in index.md match the 17 files in _github/criteria/areas/, every relative path I checked from index.md and the sample prompts (../howto.md, ../criteria.md, ../templates/analysis.md, criteria/areas/information-architecture.md, prompts/areas/full.prompt.md) resolves in the tree, and all five checks are green. Not a duplicate of #385, which ships KubeVirt analysis outputs under analyses/2026/kubevirt/ while this ships the reusable harness — they only share adjacent .cspell.yml word additions, so whichever merges second may need a trivial rebase.

One small verified issue:

• _github/copilot-instructions.md:3 — applyTo: **/*.md is not valid YAML: a plain scalar cannot start with * (alias indicator), so a strict parser rejects the front matter (confirmed with PyYAML). Quoting it (applyTo: "**/*.md") fixes the parse. Separately worth checking: applyTo is a feature of *.instructions.md files; .github/copilot-instructions.md applies repo-wide and ignores that key, so once copied into a target repo the line is inert either way. Low severity, but since this file is the template everyone copies, fixing it here fixes every downstream analysis.

Otherwise this looks correct and complete to me against its stated goal. Approval is a maintainer's call; no blocking concerns from this review.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.88

[hivecommons-hive]

Read the full tree at head 10534f2, not just the diff. This looks correct to me overall, and I verified the harness is internally consistent:

• All 17 files in _github/criteria/areas/ have matching -answers/-comment/-full/-recommendations wrapper prompts in _github/prompts/areas/, and the four shared engines (answers.prompt.md, comment.prompt.md, full.prompt.md, recommendations.prompt.md) they reference all exist.
• The change is purely additive under docs/analysis/ai-harness/ — the _github/ folder is a template, so nothing in this PR touches the live .github/ of cncf/techdocs.
• index.md's workflow, pre-flight checklist, and folder layout match the actual file layout, and the "seventeen areas" count is accurate.
• The three .cspell.yml words are used by the new content, and all five CI checks are green.

One small verified issue (low severity):

• docs/analysis/ai-harness/_github/copilot-instructions.md:3 — applyTo: **/*.md is not valid YAML: an unquoted scalar cannot start with * (it is parsed as an alias), so the front matter fails to parse in a strict YAML parser. Also, applyTo is the key for path-scoped .github/instructions/*.instructions.md files; copilot-instructions.md applies repo-wide and has no front matter schema. Suggest either quoting it (applyTo: '**/*.md') or dropping the front matter block, since downstream projects will copy this file verbatim.

Not a defect, just a note: the upload step in index.md says the gh issue create script "is not part of the harness yet" — fine to land as-is, but worth a follow-up issue so each analysis doesn't rewrite it.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.88

[hivecommons-hive]

Built the site at this head (832fb81) — one verified issue, otherwise this looks correct to me.

docs/analysis/ai-harness/README.md and index.md collide on the same route (medium). Docusaurus treats both index.md and README.md as category-index candidates, so the build emits:

[WARNING] Duplicate routes found!
- Attempting to create page at /docs/analysis/ai-harness/, but a page already exists at this route.
This could lead to non-deterministic routing behavior.

Consequence: visitors to /docs/analysis/ai-harness/ may get the 15-line README stub instead of the full overview, depending on which page wins. The repo's CI (format/link/spell/lint) doesn't build the site, so this warning only shows up on a local or Netlify build. Fix: drop README.md (its content is a summary of index.md), or rename it _README.md so the underscore exclusion skips it — the same rule that already correctly keeps all of _github/** out of the build, which I verified: the build succeeds and emits no pages for the template tree.

What I checked and found sound:

• docs/analysis/ai-harness/index.md:8-57 — relative links (../howto.md, ../criteria.md, ../templates/analysis.md, ../templates/) all resolve against files present on main; link check agrees.
• References to _github/... paths in index.md are code spans, not links, so the excluded tree causes no broken-link failures.
• No files outside docs/ except a three-word .cspell.yml addition; nothing touches the repo's real .github/.
• Overlap with #385 is only .cspell.yml (different words) — these PRs are complementary (harness vs. its KubeVirt output), not duplicates.

With the README/index collision resolved, this looks mergeable to me. Note the branch is behind main and will need an update before merge. I have not approved or changed anything — flagging for a maintainer.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.88

[iRaindrop]

The main purpose of the README would be to point to the index.md, since GitHub automatically renders the README.md file directly beneath its file list table. I looked for a README before opening index.md.

This repo can't have both (see above) - keeping index.md

[hivecommons-hive]

Read the full tree at head 0163062, not just the diff. This is an additive, docs-only template harness under docs/analysis/ai-harness/ (plus three .cspell.yml words); it touches no repo .github/, CI, or trust surface, and all checks are green.

Verified internal consistency:

• All 17 area criteria files in _github/criteria/areas/ exist, matching the "seventeen areas" claim in index.md (step 3 of the workflow).
• Every one of the 76 *.prompt.md files carries agent: agent front matter, as index.md's "Validated toolchain" section requires.
• No file inside _github/ references a _github/... path — all internal references say .github/..., which is correct since the install step renames the folder.
• The per-area wrappers (e.g. prompts/areas/information-architecture-full.prompt.md) point at criteria and shared-engine paths that exist, and the ${input:title} override is documented consistently.
• examples/kubevirt/current-repository.md fills in exactly the <!-- FILL IN --> fields of copilot-instructions.md (including the Priority label mapping table at lines 504–513), so the example and the template agree.

One small gap, non-blocking:

• docs/analysis/ai-harness/index.md:139 — the folder layout lists README.md # Short pointer to this file, but no README.md is added under ai-harness/ in this PR. Either add the pointer file or drop the line.

This looks correct to me; the structure is coherent and the content matches its own documentation. Approval and the judgment call on landing ~3.6k lines of harness content in docs/analysis/ belong to a maintainer.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.88

[hivecommons-hive]

Read the full tree of this PR (not just the diff) and verified its internal consistency. This looks correct to me; summary of what I checked:

• Structure matches its own docs. docs/analysis/ai-harness/index.md and _github/copilot-instructions.md describe 17 criteria areas, 3 sections, per-area wrapper prompts and shared engines. I verified mechanically that all 17 files in _github/criteria/areas/ have all four wrappers (-answers, -comment, -full, -recommendations) in _github/prompts/areas/, each referencing its correct criteria/areas/<area>.md, and that all 3 section wrappers reference their criteria/sections/<section>.md. No missing or mis-wired files.
• Purely additive. 110 files, +3625/−0, all under docs/analysis/ai-harness/ except three new words in .cspell.yml (afdocs, llms, llmstxt). No existing content is modified; the template lives under _github/ (underscore), so none of it activates in this repo's .github/.
• Body matches the diff. The PR delivers exactly what it says: harness directory structure plus guidance (index.md covers toolchain, pre-flight checklist, workflow, upload process, folder layout).
• CI: FILE FORMAT, LINK checking, SPELLING, MARKDOWN linter, and DCO all pass at head 2339f76.
• Relationship to #385: not a duplicate — #385 is KubeVirt analysis output (issues), this is the reusable harness that produced it. They can land independently.

One minor, non-blocking note: index.md ("Upload process") says the gh issue create upload script "is not part of the harness yet" — a follow-up issue to add it would keep that from being forgotten.

I can't approve as a bot; flagging for a maintainer that this looks ready for a human review pass.

— hive: agent=reviewer backend=copilot model=claude-fable-5 copilot=1.0.88