Skip to content

fix(tooling): 文档审计范围改为从目录派生,清单与磁盘不一致时判红 (#4851) - #4921

Merged
xuyushun441-sys merged 2 commits into
mainfrom
claude/issue-4851-docs-audit-dead-roots
Aug 3, 2026
Merged

fix(tooling): 文档审计范围改为从目录派生,清单与磁盘不一致时判红 (#4851)#4921
xuyushun441-sys merged 2 commits into
mainfrom
claude/issue-4851-docs-audit-dead-roots

Conversation

@xuyushun441-sys

Copy link
Copy Markdown
Contributor

Fixes #4851

实情先行:议题说的是「11 条 objectos 路径」,实测烂得更彻底

议题正文的写法本身就是本单要防的东西,所以我没有信它,而是从仓库现状重新测量:

数量 说明
清单条目 146 不是文件头注释里写的 147
指向 objectos/* 的死条目 10 不是 11(#4781 已删掉 runtime-capabilities.mdx)
全部死条目 16 除 objectos 那 10 条,还有 ai/chatbot-integrationdeployment/cloud-artifact-apigetting-started/cligetting-started/validating-metadatapermissions/rolesui/role-based-interfaces
磁盘上存在但根本不在清单里的文档 48 protocol/kernel/** 全部 9 页、整个 content/docs/capabilities/ 目录、releases/v12v17
实际手写文档总数 178

也就是说:一次打印 FULL audit (no args.docs given) 的运行,实际审的是 178 篇里的 130 篇

议题只问了第一个方向(死条目),而 第二个方向(漏列)烂得多三倍,并且没有任何人问起过 —— 这正是「手写清单」的性质:它两个方向同时腐烂,且两个方向都不出声。

为什么这 16 条会烂,以及为什么烂了两个月没人知道

清单头上原本写着「Keep in sync with node scripts/docs-audit/affected-docs.mjs --all」—— 一句没有任何东西检查的承诺。目录改名(protocol/objectos/protocol/kernel/)是一次纯 content/** 的改动,不触发任何与该清单相关的门禁,清单就地失效。

失效之后没有任何信号,是因为审计 agent 拿到一条不存在的路径,读不到文件,返回 fixCount: 0 —— 在运行摘要里,这和「这篇审过了,没问题」长得一模一样#4781(runtime-capabilities.mdx 整页教一个 #3605 已删的 schema)和 #4817(http-protocol.mdx 把 dispatcher 响应挂在 /api/v1/discovery 名下)两个真实失准就是这样在绿色的 full audit 底下活了约两个月。

清单能不能派生?能 —— 但不能在运行时派生

先回答议题里那个更根本的问题。答案分两半,分界线是工作流脚本的执行环境:

  • 内容上完全可派生:审计范围就是 content/docs/**/*.mdx 减去 references/,没有任何人工挑选。这个定义已经存在于 affected-docs.mjs --all
  • 位置上不可派生:工作流脚本跑在一个 node:vm 上下文里,全局只有 log/phase/console/budget/定时器,加上 agent/parallel/pipeline/workflow/args,并且 codeGeneration 关闭 —— 没有 require、没有 import、没有文件系统、没有 eval。所以脚本既无法自己遍历 content/docs/,也无法去读一个 JSON 产物。清单必须内联。

结论:在生成时派生。清单改为一段生成产物,只是恰好住在一个手写文件里:

node scripts/docs-audit/check-audit-scope.mjs --write   # 从文件系统重新生成
pnpm check:docs-audit-scope                             # CI 门禁:两个方向都比

派生源刻意复用 affected-docs.mjs --all(子进程调用)而不是再写一遍目录遍历 —— 「手写文档」只能有一个定义;第二次遍历就是下一个会漂移的东西。

新校验的双向证明

绿(当前分支状态)

$ node scripts/docs-audit/check-audit-scope.mjs
✓ docs-accuracy-audit scope is in sync with content/docs/: 178 hand-written doc(s).
exit=0

#1 —— 把一条路径改回改名前的写法

$ node scripts/docs-audit/check-audit-scope.mjs
✗ .claude/workflows/docs-accuracy-audit.js: ALL_HANDWRITTEN has drifted from content/docs/.

  1 listed path(s) do not exist — an audit agent pointed at one reads
  nothing and reports "0 fixes", which in the run summary is indistinguishable
  from a doc that was checked and found accurate:
    - content/docs/protocol/objectos/http-protocol.mdx

  1 hand-written doc(s) exist but are not listed — a run that calls
  itself a "FULL audit" silently skips every one of them:
    + content/docs/protocol/kernel/http-protocol.mdx

  Fix: node scripts/docs-audit/check-audit-scope.mjs --write
exit=1

#2 —— 在磁盘上真的把目录改名回去(即 #4851 的原始机制本身)

$ git mv content/docs/protocol/kernel content/docs/protocol/objectos
$ node scripts/docs-audit/check-audit-scope.mjs
✗ … 9 listed path(s) do not exist:
    - content/docs/protocol/kernel/config-resolution.mdx
    - content/docs/protocol/kernel/error-handling.mdx
    … (9 条全部点名)
exit=1

#3 —— 把生成标记删掉(门禁找不到自己的审查对象)

✗ .claude/workflows/docs-accuracy-audit.js: could not find the generated scope block.
  Expected a region delimited by "// < generated:docs-audit-scope >" and "// < /generated:docs-audit-scope >".
  … do NOT hand-maintain the list.
exit=1

#4 —— 手工从清单里删掉一条(清单本身没有死条目,只是漏列)

✗ … 1 hand-written doc(s) exist but are not listed:
    + content/docs/index.mdx
exit=1

恢复后重新判绿,git status 干净。四种红都不是「跑完了报个漂亮的绿」,而是点名到具体条目。

CI 门禁只看得见默认清单 —— 所以另外补了两张网

args.docs 是这个工作流最常见的调用方式(affected-docs.mjs 的输出直接传进来),而没有任何 CI 门禁能看见调用方传了什么。一条坏路径在那里失败的方式,和 #4851 里那份烂清单完全一样

  1. 启动前置校验(Scope Preflight):解析出 DOCS 之后、进入昂贵的 pipeline 之前,先派一个 agent 机械地 ls 每一条路径,任何一条解析不到就 throw 并点名。这一步必须走 agent,因为 VM 没有文件系统 —— 也正因如此它是「报告」而不是「测量」,所以它自己的算术要被核对:返回的路径必须与 scope 不重不漏地一一对应,对不上就按「前置校验失败」处理,而不是当成通过。(自检本身没人检 = merge.os-regen.driver 指向「上一个装过依赖的 worktree」的绝对路径 —— 该 worktree 一删,全容器的生成物合并驱动就坏了 #4868。)
  2. 真实读取路径上的 docExists:每个审计 agent 现在必须报告它到底有没有打开成文件,任何一条 false 都让整轮 throw。前置校验和真正读文件的不是同一条调用路径,而 merge.os-regen.driver 指向「上一个装过依赖的 worktree」的绝对路径 —— 该 worktree 一删,全容器的生成物合并驱动就坏了 #4868 的教训正是:跑在别处的自检,证明不了真实路径上的任何事

于是三层各管一段:门禁管默认清单,前置校验管调用方的清单,真实读取路径两个都复核。

扫到 protocol/kernel 之后有没有发现真实失准

按 PM 要求不顺手修,只如实报告。我这边跑不了那个 LLM 工作流(Workflow 工具不在我的工具集里,且 9 页 ≈ 18 个 agent),所以做的是一轮机械抽查,不能替代真正的审计:

  • 9 页里 HOUSE_FACTS 词表(os studio / client.ai.chat / /_studio / AUTH_SECRET / objectstack-ai/spec / …)零命中;
  • 页面里出现的 @objectstack/* 包名逐一对照 workspace,3 处不存在的(@objectstack/service-ai@objectstack/ui@objectstack/testing)都带着显式的 "proposed / target / 不存在" 说明,不是失准;@objectstack/service-ai 那条更是与 packages/spec/src/system/core-services.zod.ts:142 逐字相符;
  • objectos 残留:0。

结论:机械抽查未发现新的失准。 这不等于那 9 页干净 —— 真正的实现级审计还没跑过,建议合并后按正常流程跑一轮 scoped audit。

顺带发现(已按 Prime Directive #10 立单,未在本 PR 修)

#4920 —— 审计工作流的交付物是就地改写 mdx,而它的范围包含 content/docs/releases/**,与 AGENTS.md「releases 是 release-owned,禁止在代码 PR 中编辑」直接冲突。这不是本 PR 引入的(旧清单里已有 3 页 releases),但派生后从 3 页变成 9 页,第一次显眼。修它等于在本仓引入第二套「手写文档」定义,是需要维护者拍板的口径决定,故立单不改。

共享文件

本 PR 触到两个本仓最热的冲突点,均为新增一行/一步,无改写:

验证

$ pnpm check:docs-audit-scope
✓ affected-docs self-test: 32 cases pass.
✓ check-audit-scope self-test: 13 cases pass.
✓ docs-accuracy-audit scope is in sync with content/docs/: 178 hand-written doc(s).

$ pnpm lint      # eslint . --no-inline-config,2990 文件(含 .claude/workflows/*.js)
exit=0

$ pnpm check:doc-authoring
✓ doc authoring guard: 219 files clean.

新增的 --self-test 13 条用例是密封的(不依赖仓库状态),覆盖:标记缺失必须 throw、手写形态的数组必须 throw、渲染/解析往返、以及差异的两个方向各自单独钉住(议题只问了 dead,而 unlisted 才是三倍的洞)。


Generated by Claude Code

claude added 2 commits August 3, 2026 16:15
… of a hand-kept list (#4851)

`.claude/workflows/docs-accuracy-audit.js` carried its default audit scope inline,
as a hand-kept `ALL_HANDWRITTEN` array behind a "keep in sync with
`affected-docs.mjs --all`" comment. Nothing checked that promise, and it had
rotted in BOTH directions:

- 16 listed paths no longer existed — 10 of them the whole
  `content/docs/protocol/objectos/**` directory, renamed to `protocol/kernel/`.
  An audit agent pointed at a non-existent file reads nothing and reports
  `fixCount: 0`, indistinguishable in the run summary from a doc that was
  checked and found accurate. That is how #4781 and #4817 sat in
  `protocol/kernel/` for ~2 months under green "full" audits.
- 48 existing docs were absent from the list — including all 9 of
  `protocol/kernel/**` and the whole `content/docs/capabilities/` directory. A
  run logging `FULL audit` was auditing 130 of 178 hand-written docs.

The list stays inline because it must: a workflow script runs in a `node:vm`
context with no require/import/filesystem, so it can neither walk
`content/docs/` nor read a JSON artifact. So it is GENERATED instead —
`scripts/docs-audit/check-audit-scope.mjs --write` derives it from
`affected-docs.mjs --all` (one definition of "hand-written doc"), and
`pnpm check:docs-audit-scope` fails in lint.yml when the block and
`content/docs/` disagree in either direction, naming every entry. Missing
markers or an unparseable block fail too — a gate that cannot find its subject
must go red, not green.

A CI gate only sees the default list, so two more nets cover the rest:
- the workflow preflights its resolved scope (including a caller-supplied
  `args.docs`) and refuses to start, naming every path that does not exist; the
  preflight's own arithmetic is reconciled against the scope, so a verdict that
  cannot account for every path exactly once is a failed preflight, not a pass;
- every audit agent reports `docExists` from the path that actually opens the
  file, and the run throws if any comes back false — a self-check that runs
  somewhere other than the real read path proves nothing about it (#4868).

Same discipline as #4690 / #4777 / #4804 / #4835 / #4868 / #4890.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018iARDqtrhQgz6fVHDeDkbQ
…s-audit-dead-roots

# Conflicts:
#	package.json
@vercel

vercel Bot commented Aug 3, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 3, 2026 4:18pm

Request Review

@github-actions github-actions Bot added size/l documentation Improvements or additions to documentation ci/cd dependencies Pull requests that update a dependency file tooling labels Aug 3, 2026
@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

No hand-written docs reference the 0 changed package(s). ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation size/l tooling

Projects

None yet

2 participants