feat(qa): AI 可执行的平台功能标准测试清单 — 15 域 178 项 + 能力覆盖 ratchet + CI 门禁

[baozhoutao]

What this is

A standing, machine-readable platform test checklist executed by AI agents, replacing the per-release one-offs (the #3358 checkbox issue, docs/plans/release-15.1-test-plan.md hand tables) with a durable ledger that accumulates across releases.

docs/qa/platform-checklist/ — 15 areas · 178 items · 995 acceptance clauses, each clause carrying {clause, oracle, verify, evidence}. Enumerable surfaces are variants matrices derived from the spec's own Zod enums (49 field types, 20 chart types, 7 action locations, flow node/trigger types, query operators, 9 auth methods, 6 validation-rule types, …).

How it stays honest (AI-accuracy design)

• Oracle hierarchy (RUNNER.md): server truth (api/network/build/test) outranks screenshot outranks DOM; DOM only after a screenshot confirms render (anti-hydration-false-positive, dogfood skill §3).
• fail is expensive: reproduce ×2 + automation self-check + traps ruled out + issue filed. partial/blocked are first-class verdicts — half-proven never ticks green.
• Both sides of every gate: entitled succeeds AND unentitled is server-side denied.
• 8 blocked items name their missing fixture instead of faking coverage; existing automation is pinned via automated.ref.

How it stays complete (sustainability)

• Coverage ratchet (coverage.json + scripts/check-platform-checklist.mjs, wired into lint.yml, 0.35s): the capability universe is derived from packages/spec/liveness/*.json at check time — a new metadata kind with no checklist item/waiver fails CI, both directions (UNCLASSIFIED/ORPHAN, mirrors the liveness ledger discipline). Now 27 kinds mapped / 2 waived (book/doc, display-only).
• Variants-freshness gate: matrix items pin enumSource; when the spec grows an enum member the checklist's own CI goes red.
• Manual sweep trigger: the internal coverage-sweep skill (.claude/skills/coverage-sweep/ + SWEEP.md) re-runs the five-angle gap audit on demand ("跑一轮 coverage sweep").

Four stale waivers were caught and corrected during authoring (api, datasource, mapping, hook — each had a real stock fixture the waiver claimed didn't exist).

Needs maintainer decisions (not blocking this PR)

FOLLOW-UPS.md carries 8 product findings surfaced while grounding items — notably D1: saved-report schedule routes ignored _context (cross-owner schedule delete/list), now fixed in #6683 (owner-gated, deny-as-404); the dashboards.saved-report-ownership probe flips from expected-fail to a positive assertion once both land (tracked in #6681). Also client-only stubs in AppManagementPage, undo without ifMatch, and docs advertising retired capabilities (enable.trash/enable.mru).

Validation

pnpm check:platform-checklist → OK (15 areas, 178 items; coverage 27 mapped / 2 waived). Negative-tested: injected structural defects and coverage holes each produce named failures (11 structural, 3 coverage classes). Docs/scripts only — no published-package changes, no changeset needed.

---

🤖 Generated with Claude Code

https://claude.ai/code/session_01YD9f6FYyMraUWYeJf53V43

---

Generated by Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
objectstack	Ignored		Aug 8, 2026 1:08pm

[github-actions]

⛔ merge queue 构建失败 — 先分诊，再决定要不要重排

队列构建 31258408522 红了。队列跑的是全量套件（PR 侧 CI 只跑 affected 子集），
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

失败的 job（日志抽取，best effort）：

• Dogfood Regression Gate (2/3) — 失败步骤: Boot example apps and exercise real user flows

  ✗✗ rls_note  [rls-hole] member B cannot read it (GET 404) yet MUTATED it by id (PATCH 200) — by-id write bypassed RLS (#1994 class)
  �[41m�[1m FAIL �[22m�[49m �[30m�[46m isolated �[49m�[39m test/showcase-scope-depth.dogfood.test.ts�[2m > �[22mshowcase: scope-depth read — `unit` (ADR-0057 D1)
  

历史信号：

• 本 PR 过去 24h 无队列失败记录（首次）。
• 过去 24h 队列共有 5 个失败构建（不含本次）。

分诊清单：

1. 失败测试在本 PR 改动的包里 → 真回归，修 PR。
2. 失败测试与本 PR 无关 → 在其他 PR 的同类评论里搜同名测试；出现过 ⇒ flaky 实锤，开 issue 修/隔离那条测试。修好前重排只会再烧一轮全队列。
3. 两者都不是 → 可能与同组 PR 语义冲突；等前面的 PR 落地或失败出队后再重排一次即可，不要连续重排。

---

Generated by Claude Code · merge-queue-triage workflow (#4859)

[claude]

Queue steward — NEW signature: intercepted, NOT requeued.

This PR was dropped from the merge queue by the merge_group build 31258408522 (generation pr-6557-2c2a212a…, 12:57:04Z). The failure signature matches no row of the four ledger tables in #5810, so under the steward's four-branch rule it is not requeued as-is — it goes to the owning lane for diagnosis.

Full signature (read from the complete log archive, not the tail — SKILL note 7)

• Failing job: Dogfood Regression Gate (2/3), failing step Boot example apps and exercise real user flows.

• Failed suite (1 of 29): packages/qa/dogfood/test/showcase-scope-depth.dogfood.test.ts — describe block showcase: scope-depth read — `unit` (ADR-0057 D1).

• Error string, verbatim:

  Error: Hook timed out in 10000ms.
  If this is a long-running hook, pass a timeout value as the last argument or configure it globally with "hookTimeout".
   ❯ test/showcase-scope-depth.dogfood.test.ts:162:3
  

• Line 162 is the afterAll teardown hook, and the source frame the reporter printed makes the asymmetry visible on its own:

  161|   beforeAll(async () => { world = await bootScopeWorld('unit'); }, 120…
  162|   afterAll(async () => { await world?.stack?.stop(); });
  

• Suite totals: Test Files 1 failed | 28 passed (29), Tests 155 passed | 1 skipped (156), Duration 128.47s. Turbo: Failed: @objectstack/dogfood#test, 61 successful, 62 total.

• The second red job (Dogfood Regression Gate, step Verify dogfood shard results) is the consequence, not an independent fault — taken from the aggregator's own declared inputs rather than log adjacency (SKILL note 7).

• ⚠️ check-test-completeness: OK (1 package(s), 156 test(s) declared and all 156 accounted for) — per SKILL note 7 a green completeness check is not a green suite; the workflow's own comment reads "A red suite plus a GREEN completeness check means real test failures".

Preliminary reading (a steward's first pass, not a verdict)

The beforeAll that boots the scope world is given an explicit 120_000ms; the paired afterAll that stops it is given nothing, so it inherits vitest's default 10000ms hookTimeout. Tearing down a booted stack (HTTP server plus drivers) at the tail of a ~128s full-suite run is not obviously a sub-10s operation under merge-queue load, where the queue runs the full suite on a cold cache rather than the affected subset the PR-side CI runs. The shape is a default-timeout coverage hole on the teardown side, not an assertion failure: nothing about the assertion under test disagreed.

Two supporting readings, both against origin/main:

• This asymmetry is pre-existing on main (git show origin/main:packages/qa/dogfood/test/showcase-scope-depth.dogfood.test.ts, lines 161–162 identical to the frame above) — it did not arrive with this PR.
• This PR's diff does not reach the failing file. All 28 changed paths are docs/qa/platform-checklist/**, scripts/check-platform-checklist.mjs, scripts/checklist-select.mjs, .claude/skills/**, .github/workflows/lint.yml and package.json; packages/qa/dogfood/** is untouched.

Suggested action (the lane's call, not the steward's)

1. Decide whether the teardown hook should carry an explicit timeout matching its boot budget, or whether world?.stack?.stop() is genuinely hanging — the two have different fixes and only the second is a real bug.
2. If it is judged a first-instance flake rather than a defect, it still needs a human ledger ruling on 队列管家 Routine（三仓总管）：合并队列健康专责化 —— flaky 签名分诊 / 原样重投 / 新签名拦截 / 跨仓 pin 链停滞观测（座位 Routine 化第二例，维护者 2026-08-06 拍板） #5810 before any future occurrence can be requeued as-is — the steward cannot add ledger rows.

This PR carries no Fixes line, so there is no anchor issue for the steward to file the companion note on; whoever owns this lane please pick it up from here. ⛔ Not requeued, no code touched, no ready/draft switch, no merge, no claim changed.

A ledger-upgrade request covering the class this belongs to has been filed on #5810 for the maintainer.

---

Generated by Claude Code

[os-zhuang]

Queue steward — follow-up to the interception above: signature did NOT reproduce, PR landed.

Recording the outcome so the interception comment (13:27:06Z) does not read as still-pending on a merged PR.

when	what
12:57:04Z	merge_group build 31258408522 red — Hook timed out in 10000ms at showcase-scope-depth.dogfood.test.ts:162 (afterAll)
13:27:06Z	steward intercepted (four ledger tables: zero hits) — ⛔ not requeued
13:42:17Z	lane re-enqueued (qq9340100) — lane's own call, steward did not act
14:04:07Z	retry generation pr-6557-64d764e7…: CI / Lint & Type Check / Spec Liveness Check / Console Pin Freshness all completed: success
14:20:14Z	MERGED

What this settles and what it does not.

• It settles that the failure is intermittent, not a deterministic defect this PR introduced. That was already the steward's initial read for two independent reasons — the timeout asymmetry (beforeAll explicitly 120_000ms, its paired afterAll inheriting the 10000ms default hookTimeout) is pre-existing on origin/main, and this PR's 28 changed paths do not touch packages/qa/dogfood/**. The clean retry is now direct evidence for it.
• It does not settle the ledger question. The teardown-side coverage hole is still there on main, unowned, and will recur on some other PR. The interception itself was correct under the four-branch rule and would be made again: zero ledger hits ⇒ no requeue, regardless of how a retry happens to turn out.

The standing ledger request on #5810 (instance ③ of the timeout-coverage-hole class) is updated with this evidence. No action is required on this PR.

---

Generated by Claude Code