Define measurable quality goals and scenarios for template changes

[braboj]

What

Define a small, measurable quality model for solid-ai-templates in the existing
design notes, ready to migrate into arc42 quality goals and quality scenarios.
Use it to assess changes and releases without adding a scorecard bureaucracy.

Why

Passing structural checks proves that templates compose, but not that agents
complete useful work correctly. More rules, checks, issues closed, or ADRs do
not establish better quality. The recent consumer analysis also found adoption
and documentation work generated by the policy itself.

High quality means correct, useful engineering outcomes at an affordable cost
to understand, run, adopt, and maintain. A change should improve a demonstrated
outcome enough to justify the cost carried by affected consumers. Credible
material risks, including security risks, do not require a prior incident.

Acceptance criteria

• Define correctness, effectiveness, relevance, efficiency, predictability,
  and maintainability, with observable evidence for each.
• Give representative quality scenarios with context, stimulus, expected
  response, and a measurable result. Distinguish measured baselines from
  proposed targets; do not invent numeric thresholds before measurement.
• Cover agent task success, regressions, context and execution cost,
  consumer adoption effort, and recurring policy-induced work. Severe
  correctness or safety failures cannot disappear inside an average score.
• Integrate the model into existing contribution/release review guidance
  concisely, without a new mandatory report, ADR, or per-change audit.
• Reuse Prove solid-ai-templates delivers better results #1184 for behavioral evidence, The e2e suite has not executed live since 2026-05-06, and three cases have never run #1368 for live test execution,
  Spike: measure cold-start and template-loading cost for representative stacks #368 for latency measurements, and The chain ceiling has been raised 313 times and lowered none, so it prices growth rather than refusing it #1462 for chain-budget semantics.

The working design document is temp/docs/design/design-notes.md (local,
gitignored). Move the resulting durable guidance into tracked documentation
when implemented; the issue is self-contained without that file.

Related: #1480 (core boundary), #1353 (selective adoption), #1506 (ADR threshold).

---

Superseded citation, corrected 2026-09-06

"#1462 for chain-budget semantics" no longer names a live source. ADR-041
deleted tests/chain-budget.txt and SYS-12 in v2.82.0 and closed #1462.

The context-cost measurement this criterion reaches for is now README.md's
model-limits table: chain size per stack category, beside the smallest context
window that still holds the chain plus the interview. tools/sync.py
regenerates it and sync.py --check gates it. ADR-041 records why the
per-root ceiling was not the right instrument, and #1551 carries the gate that
replaces it.