Skip to content

fix(spec): check:liveness 的 stale-evidence 判红,摘要行的数与词对齐 (#5623) - #6057

Open
os-zhuang wants to merge 2 commits into
mainfrom
claude/issue-5623-liveness-stale-evidence-gate
Open

fix(spec): check:liveness 的 stale-evidence 判红,摘要行的数与词对齐 (#5623)#6057
os-zhuang wants to merge 2 commits into
mainfrom
claude/issue-5623-liveness-stale-evidence-gate

Conversation

@os-zhuang

@os-zhuang os-zhuang commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Fixes #5623

前提复现(先证伪,再动手)

origin/main(739f496)上把 packages/spec/liveness/query.json 的 5 条 evidence 路径故意改坏(指回迁移前的 packages/plugins/driver-sql/...),跑真 gate:

evidence paths: 330 resolved against this checkout, 101 attributed to another repo (…)

⚠ 5 'live' entr(ies) cite a missing file:
    query/fields → packages/plugins/driver-sql/src/sql-driver.ts
    query/where → packages/plugins/driver-sql/src/sql-driver.ts
    query/orderBy → packages/plugins/driver-sql/src/sql-driver.ts
    query/limit → packages/plugins/driver-sql/src/sql-driver.ts
    query/offset → packages/plugins/driver-sql/src/sql-driver.ts

EXIT=0。issue 的两条实测逐条成立:五条断链全被点名却不判红,摘要行的 330 在改坏五条之后一动没动。代码位置:check-liveness.mtsfailed 表达式里没有 report.staleEvidence,而摘要行印的是 report.evidenceLocal(local 路径总数),头顶的注释还写着 "actually resolved against this checkout" —— 注释本身就是这个 bug。

main 当前 330 条本仓路径全部解析成功,所以本 PR 落地即绿,不需要修任何 ledger 数据(scope guardrail 里那条例外没有触发)。

「注意的反面」:⚠/✗ 分级的原始意图查证结果

issue 要求先核对分级是不是有意设计。查证结论:这里的 ⚠ 不是对本仓路径的有意宽容,是解析器时代的遗留。四条证据:

  1. evidence.mts 的文件头记录了它自己的来历:fix(spec): liveness stale-evidence check was ~100% false positives — and was burying a real one #3857 之前的检查是 evidence.split(':')[0],227 条里报 48 条、全是误报(prose 解析产物或有意的跨仓指针)。那种噪声比下,它当然不能判红。同一段还写明代价:被那 48 条埋掉的唯一一条真腐烂 object.enable.clone(消费者从 @objectstack/objectql 搬到了 @objectstack/metadata-protocol)一直没人看见。
  2. .changeset/liveness-register-orphan-proofs.md 的收尾句:"the orphan list joins the stale-evidence list at empty, so both mean something again" —— 解析修好之后,这份清单的语义就是「有命中即信号」。
  3. 同族旁证:同目录的 check-empty-state.mtsrotted-evidence 一直是 + exit 1;liveness/README.md 说它的执行点路径 "resolves like evidence above, so a pointer that rots is reported rather than trusted"。同一族里,腐烂指针的既定处置就是判红。
  4. 反向对照:这个 gate 里有意的宽容每一条都在代码里明说了理由 —— verifiedAt 年龄("Age never fails CI — re-verification is a worklist")、undrilled 容器计数("a worklist, not a failure")、PENDING_GOVERNANCE("a worklist, not a merge gate"),连另一条 ⚠(orphan proof tags)都写了 "flag it (warning)"。唯独 stale-evidence 那一段,一个解释都没有。刻意的宽容在这个文件里是会写下来的;这条没写。

所以按 issue 给的边界收紧,没有需要升级到 needs_decision 的发现。

改动

1. live 条目引用本仓缺失文件 → 判红

report.staleEvidence.length > 0 进入 failed,输出块从 改为 并附三条修法(仓内搬家 → 改指针并顺手补 verifiedAt;搬去别的仓 → 加 realm marker;消费者真没了 → 按 ADR-0049 重新判定,而不是随手指向一个看着像的幸存者)。

边界:cross-repo attribution 完全不受影响。 这不是靠约定守住的,是结构性的 —— checkEvidence 只对 local 桶做存在性检查,foreign 桶(realm marker objectui: / cloud: / ee:,以及 packages/services/service-ai/… 前缀)从来不进 missing。当前 101 条跨仓归属一条都不会判红,自测里有三个用例钉死这一点(含一个「同一条 evidence 里 objectui: 子句 + 仓内路径」的混合用例 —— 如果 realm 作用域不在子句边界结束,一个 marker 就能整条豁免 gate)。

2. 摘要行:选「两个数都印」,而不是二选一

issue 给了两个选项(改成真实解析计数 / 把 "resolved" 改成 "declared")。两个都印才是对的,理由是这两个数各自有用途,丢掉任何一个都损失信息:

绿的时候两者相等 —— 这恰恰是当初只印前一个会被读成「通过」的原因:

evidence paths: 330 repo-local path(s) declared by 'live' entries, 330 resolved against this checkout; 101 attributed to another repo (objectui / cloud — not resolvable here).

有断链时行尾追加 , N MISSING,数与词一一对应。JSON 报告同步新增 evidenceMissing,evidenceLocal 的注释从 "actually resolved" 改成 "DECLARED, not yet proven"。

3. 新增 --ledger-root= 路径参数

让 gate 读 packages/spec/liveness 的一份副本。它是为了让「判红」这件事本身可证明:判红的 gate 只值它那份「它确实会红」的证明,而这需要一条真的腐烂指针来红。往 shipped ledger 里提交一条不是选项(那会让其他每一次运行都红),测试中途改动被跟踪文件则会在崩溃时留下脏 worktree。所以自测把真 ledger 拷进临时目录、只坏一条指针、再跑这个脚本本身 —— CI 跑的同一条代码路径,仓库状态零改动;其余全部仍对真 repoRoot 解析,所以那个 exit 1 只有一个成因。

反向验证(方向先判,再跑)

预判:(标准方向)—— 同一批断链在改前 exit 0、改后 exit 1,且摘要行的数应当移动

origin/main 代码 本 PR 代码
5 条本仓路径断链 ⚠ 5 … / EXIT=0 / 330 resolved ✗ 5 … / EXIT=1 / 330 declared, 325 resolved, 5 MISSING
干净 ledger EXIT=0 EXIT=0,330 declared, 330 resolved

改前那一跑就是「新测试若在 main 上运行会红」的证明:测试断言 exit 1,main 给的是 exit 0

测试

新增 packages/spec/scripts/liveness/check-liveness.test.ts(8 例),spawn 真脚本断言退出码,precedent 是 scripts/check-generated-ledger.test.ts。这一层是必需的:这个 bug 从来不是「检查看不见」—— 它把五条全点名了还是 exit 0,所以只测 checkEvidenceevidence.test.ts 全程是绿的,再加一个同层单测也不会红。

✓ is green against a verbatim copy of the shipped ledgers 1341ms      ← 控制组
✓ FAILS when a `live` entry cites a repo-local file that is gone 1274ms
✓ names EVERY rotted pointer, not just the first 1406ms
✓ stays green when the missing path is attributed to ANOTHER repo 1335ms   ← 边界
✓ still fails a local path that shares a string with a foreign clause 1328ms
✓ prints declared and resolved as separate numbers, equal on a green run 1417ms
✓ MOVES the resolved count when a pointer rots — the mis-labelled count of #5623 2862ms
✓ reports foreign attributions separately and never as missing 1378ms
Test Files 1 passed (1) / Tests 8 passed (8)

全量:

pnpm --filter @objectstack/spec test        → Test Files 326 passed (326), Tests 8316 passed (8316)
pnpm --filter @objectstack/spec typecheck   → tsc --noEmit OK; check:test-typecheck OK(债务账本未增)
pnpm --filter @objectstack/spec check:liveness      → EXIT 0(330 declared / 330 resolved / 101 foreign)
pnpm --filter @objectstack/spec check:empty-state   → ✓ all classified
pnpm check:type-check-coverage / check:release-notes / check:objectui-changeset /
  check:published-files / check:nul-bytes / spec check:generated --reconcile-only /
  spec check:spec-changes / scripts/check-changeset-no-major.mjs   → 全部 PASS
npx eslint(改动的两个文件)                        → 0 problems

Changeset:为什么是 @objectstack/spec: patch,不是空 frontmatter,也不是 skip-changeset

派单给的默认是「dev-scripts/CI-only → 空 frontmatter」。实测后改了,因为这个 PR 不满足那个前提:

  • packages/spec/package.jsonfiles 里有 liveness,npm pack --dry-run 实测 29 个文件入包,含 liveness/README.md —— 本 PR 改的那段 README 是发布内容scripts/ 入包 0 个文件,那部分确实是 dev-only。
  • 所以它不是 "releases nothing",skip-changeset(route 2)和空 changeset(route 3)都不适用,pr-automation.yml 的 route 1 才是:it releases something → 具名包
  • 而且 pr-automation.yml 已经把空 frontmatter 明确降级为 LAST RESORT:它是 changesets/action 的真实输入,全空集合会走 "All changesets are empty; not creating PR" 分支,发布静默地绿着卡住 —— 空 changeset 会静默卡死已 version 的发布:Release run 全绿,但 npm 和 Docker 什么都没发(17.0.0-rc.2 现在就卡着) #4898 卡住 17.0.0-rc.2 的正是这个。

因此本 PR 不需要也不应该带 skip-changeset 标签:它写了一个具名包的 changeset,Check Changeset 走的是「有 changeset」那条路。

一个新开始判红的 gate 必须同步它面向作者的文档,否则下一位作者只能靠撞红的 CI 才知道规则变了 —— README 那一段(新的失败语义 + 跨仓边界)因此是这次改动的一部分,而不是搭车。

#5475 的关系(packages/spec/scripts/** 的 tsconfig 覆盖)

实测答案:对错误数 无影响,对文件数轻微增加(而新增文件是干净的)。

tsconfig.test.json 的文件头已经写明 scripts/ "is in no tsconfig at all",实测 16 files / 33 errors。我用一个把 scripts/liveness/** 纳入的探针 program 量了本 PR 之后的这个子目录:

scripts/liveness/check-liveness.mts(686,24): error TS7006: Parameter 's' implicitly has an 'any' type.

唯一 1 条,且在我没碰过的行(v.stale.forEach((s) => …),verifiedAt worklist 那段)。新增的 check-liveness.test.ts 贡献 0 条错误。所以:#5475 的错误清单不因本 PR 变长(不会更难),但它的文件分母 +1(16 → 17)。既没变简单也没变得不必要 —— --ledger-root 那点解析逻辑也没有引入新的类型面。

#5837 的碰撞面

无。本 PR 只碰 scripts/liveness/check-liveness.mts + 新增同目录自测 + liveness/README.md + changeset;不读也不写 authorable-surface.json / json-schema.manifest.json / api-surface.json,不碰 build-schemas.ts / build-docs.ts / .gitattributes

`live` 判定的语义就是它的 evidence 指针。指针指向本仓已不存在的文件时,
这条声明不再可证伪 —— declared 有,enforced 无 —— 而一次目录重组或改名
就足以造成它。实测(origin/main,故意改坏 query.json 的 5 条路径):逐条
点名了,退出码仍是 0;摘要行的 330 也一动没动,因为它数的是 local 路径
总数,不是解析成功数。

- `live` 条目引用本仓缺失文件 → `✗` 判红(退出码 1),并给出三条修法。
  边界不变:cross-repo attribution(objectui / cloud / service-ai,当前
  101 条)只计数不解析,永远不判红 —— checkEvidence 只对 local 桶做存在
  性检查,这是结构性的分界。
- 摘要行改成两个数:declared 与 resolved,绿的时候相等;有断链时追加
  `, N MISSING`。保留 declared 是因为它是 #3857 留下的解析健康度信号。
- 新增 `--ledger-root=<dir>`,自测据此把真 gate 跑在只坏了一条指针的
  ledger 副本上,不改动仓库任何文件。

以前是 ⚠ 不是对本仓路径的有意宽容,是解析器时代的遗留:#3857 之前
`evidence.split(':')[0]` 在 227 条里报 48 条、全是误报。同族的
check-empty-state 对 rotted-evidence 一直是 ✗ + exit 1。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014wsZeReNTqiceBfLb5Pyf5
@vercel

vercel Bot commented Aug 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 6, 2026 3:57pm

Request Review

`packages/spec/liveness/**` 是发布内容(package.json 的 `files` 含
`liveness`;`npm pack --dry-run` 实测 29 个文件入包,含该目录的
README.md),所以本 PR 并非「releases nothing」,route 2 的
`skip-changeset` 与 route 3 的空 changeset 都不适用。

pr-automation.yml 也把空 frontmatter 明确降级为 LAST RESORT:它是
changesets/action 的真实输入,全空集合会走
"All changesets are empty; not creating PR" 分支,发布静默绿着卡住
(#4898 卡住 17.0.0-rc.2)。脚本本身(scripts/)不入包,是 dev-only。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014wsZeReNTqiceBfLb5Pyf5
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/spec.

112 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/agents.mdx (via @objectstack/spec)
  • content/docs/ai/skills-reference.mdx (via @objectstack/spec)
  • content/docs/ai/skills.mdx (via @objectstack/spec)
  • content/docs/api/client-sdk.mdx (via @objectstack/spec)
  • content/docs/api/environment-routing.mdx (via @objectstack/spec)
  • content/docs/api/error-catalog.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-client.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-server.mdx (via @objectstack/spec)
  • content/docs/api/index.mdx (via @objectstack/spec)
  • content/docs/automation/approvals.mdx (via @objectstack/spec)
  • content/docs/automation/connectors.mdx (via @objectstack/spec)
  • content/docs/automation/flows.mdx (via @objectstack/spec)
  • content/docs/automation/hook-bodies.mdx (via packages/spec)
  • content/docs/automation/hooks.mdx (via @objectstack/spec)
  • content/docs/automation/index.mdx (via @objectstack/spec)
  • content/docs/automation/webhooks.mdx (via @objectstack/spec)
  • content/docs/automation/workflows.mdx (via @objectstack/spec)
  • content/docs/concepts/architecture.mdx (via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx (via packages/spec)
  • content/docs/concepts/index.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx (via packages/spec)
  • content/docs/concepts/north-star.mdx (via @objectstack/spec)
  • content/docs/data-modeling/analytics.mdx (via @objectstack/spec)
  • content/docs/data-modeling/drivers.mdx (via @objectstack/spec)
  • content/docs/data-modeling/external-datasources.mdx (via @objectstack/spec)
  • content/docs/data-modeling/field-types.mdx (via @objectstack/spec)
  • content/docs/data-modeling/fields.mdx (via @objectstack/spec)
  • content/docs/data-modeling/formulas.mdx (via @objectstack/spec)
  • content/docs/data-modeling/index.mdx (via @objectstack/spec)
  • content/docs/data-modeling/objects.mdx (via @objectstack/spec)
  • content/docs/data-modeling/queries.mdx (via @objectstack/spec)
  • content/docs/data-modeling/schema-design.mdx (via @objectstack/spec)
  • content/docs/data-modeling/seed-data.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation-rules.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation.mdx (via @objectstack/spec)
  • content/docs/deployment/cli.mdx (via @objectstack/spec)
  • content/docs/deployment/tenancy-modes.mdx (via @objectstack/spec)
  • content/docs/deployment/troubleshooting.mdx (via @objectstack/spec)
  • content/docs/deployment/validating-metadata.mdx (via @objectstack/spec)
  • content/docs/getting-started/build-with-claude-code.mdx (via @objectstack/spec)
  • content/docs/getting-started/common-patterns.mdx (via @objectstack/spec)
  • content/docs/getting-started/examples.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-reference.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx (via @objectstack/spec)
  • content/docs/getting-started/your-first-project.mdx (via @objectstack/spec)
  • content/docs/kernel/cluster.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/auth-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/cache-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/data-engine.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/index.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/metadata-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/storage-service.mdx (via @objectstack/spec)
  • content/docs/kernel/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/data-service.mdx (via @objectstack/spec)
  • content/docs/kernel/runtime-services/email-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/examples.mdx (via @objectstack/spec)
  • content/docs/kernel/runtime-services/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/queue-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/sharing-service.mdx (via @objectstack/spec)
  • content/docs/kernel/runtime-services/sms-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/storage-service.mdx (via @objectstack/spec)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/spec)
  • content/docs/kernel/services.mdx (via @objectstack/spec)
  • content/docs/permissions/authorization.mdx (via @objectstack/spec)
  • content/docs/permissions/permission-sets.mdx (via @objectstack/spec)
  • content/docs/permissions/permissions-matrix.mdx (via @objectstack/spec)
  • content/docs/permissions/positions.mdx (via @objectstack/spec)
  • content/docs/permissions/rls.mdx (via @objectstack/spec)
  • content/docs/permissions/sharing-rules.mdx (via @objectstack/spec)
  • content/docs/plugins/adding-a-metadata-type.mdx (via @objectstack/spec)
  • content/docs/plugins/development.mdx (via @objectstack/spec)
  • content/docs/plugins/index.mdx (via @objectstack/spec)
  • content/docs/plugins/packages.mdx (via @objectstack/spec)
  • content/docs/protocol/backward-compatibility.mdx (via @objectstack/spec)
  • content/docs/protocol/diagram.mdx (via packages/spec)
  • content/docs/protocol/kernel/config-resolution.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/http-protocol.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/i18n-standard.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/lifecycle.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/plugin-spec.mdx (via @objectstack/spec)
  • content/docs/protocol/knowledge.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx (via packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/record-alert.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx (via @objectstack/spec)
  • content/docs/releases/implementation-status.mdx (via @objectstack/spec)
  • content/docs/releases/index.mdx (via @objectstack/spec)
  • content/docs/releases/v12.mdx (via @objectstack/spec)
  • content/docs/releases/v13.mdx (via @objectstack/spec)
  • content/docs/releases/v16.mdx (via @objectstack/spec)
  • content/docs/releases/v17.mdx (via @objectstack/spec)
  • content/docs/releases/v9.mdx (via @objectstack/spec)
  • content/docs/ui/actions.mdx (via @objectstack/spec)
  • content/docs/ui/apps.mdx (via @objectstack/spec)
  • content/docs/ui/create-vs-edit-form.mdx (via @objectstack/spec)
  • content/docs/ui/dashboards.mdx (via @objectstack/spec)
  • content/docs/ui/field-grouping-and-order.mdx (via @objectstack/spec)
  • content/docs/ui/forms.mdx (via @objectstack/spec)
  • content/docs/ui/index.mdx (via @objectstack/spec)
  • content/docs/ui/public-data-collection.mdx (via @objectstack/spec)
  • content/docs/ui/setup-app.mdx (via @objectstack/spec)
  • content/docs/ui/translations.mdx (via @objectstack/spec)
  • content/docs/ui/views.mdx (via @objectstack/spec)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

2 participants