Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions .changeset/liveness-stale-evidence-fails-ci.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
"@objectstack/spec": patch
---

fix(spec): `check:liveness` 的 stale-evidence 从 `⚠` 升级为 `✗` 判红,摘要行不再把「声明数」当作「解析成功数」(#5623)

`live` 判定的语义就是它的 evidence 指针 ——「这个属性有运行时消费者,在这里」。指针指向本仓已不存在的文件时,这条声明就不再可证伪:declared 有,enforced 无。而把它变成这样,只需要一次目录重组或一次改名。

实测(`origin/main`,把 `packages/spec/liveness/query.json` 的 5 条 evidence 路径指回迁移前的 `packages/plugins/driver-sql/...`):

```
evidence paths: 330 resolved against this checkout, 101 attributed to another repo (…)

⚠ 5 'live' entr(ies) cite a missing file:
query/fields → packages/plugins/driver-sql/src/sql-driver.ts
```

退出码 **0**。五条断链逐条点名了,CI 一路绿灯 —— 而且摘要行里的 `330` 一动没动,因为它数的是 local 路径**总数**,不是解析成功数,"resolved" 这个词是失真的。

**为什么以前是 `⚠`,以及为什么现在不该是。** 这不是对本仓路径的有意宽容,是解析器时代的遗留:#3857 之前的检查取 `evidence.split(':')[0]` 当文件名,227 条里报 48 条、**全是误报**,那种情况下它当然不能判红。`evidence.mts` 修好了解析(realm marker 归属 + 仓根路径提取),此后这份清单一直是空的 —— 恰恰是这一点让「有命中」重新变成信号。当时被那 48 条噪声埋掉的唯一一条真腐烂(`object.enable.clone`,消费者从 `@objectstack/objectql` 搬到了 `@objectstack/metadata-protocol`),就是宽容的实际代价。

同族的旁证:同一目录下的 `check-empty-state.mts` 对 `rotted-evidence` 一直是 `✗` + exit 1,README 也写着它的执行点路径「resolves like `evidence` above, so a pointer that rots is reported rather than trusted」。真正有意的宽容在这个 gate 里都写明了理由(`verifiedAt` 的年龄、undrilled 容器计数、`PENDING_GOVERNANCE` —— 每一条都在代码里明说 "never fails CI"),stale-evidence 那一段一个字都没有。

**改动**

1. **`live` 条目引用本仓缺失文件 → `✗` 判红**,并给出三条修法(仓内搬家就改指针并补 `verifiedAt`;搬去别的仓就加 realm marker;消费者真没了就按 ADR-0049 重新判定,而不是随手指向一个看着像的幸存者)。
**边界不变**:cross-repo attribution(`objectui:` / `cloud:` / `packages/services/service-ai/…`,当前 101 条)只计数、不解析,永远不判红 —— `checkEvidence` 只对 local 桶做存在性检查,这个分界是结构性的,不是靠约定。
2. **摘要行改成两个数**,而不是二选一:
```
evidence paths: 330 repo-local path(s) declared by 'live' entries, 330 resolved against this checkout; 101 attributed to another repo (…)
```
两个数都留着是有原因的:"declared" 是 #3857 留下的解析健康度信号(有单测防止解析器退化成「什么都提不出来」而假绿),"resolved" 是判定。绿的时候两者相等 —— 这正是当初只印前一个会被读成通过的原因。有断链时行尾追加 `, N MISSING`。
3. 新增 `--ledger-root=<dir>`:让 gate 读 `packages/spec/liveness` 的一份副本。自测据此把真 gate 跑在「只坏了一条指针」的 ledger 副本上,不改动仓库里任何文件。

`check-liveness.test.ts`(新增 8 例)直接 spawn 真脚本断言退出码 —— 这个 bug 从来不是「检查看不见」:它把五条全点名了还是 exit 0,所以只测 `checkEvidence` 的单测全程是绿的。

脚本本身(`scripts/`)不进 npm 包,但 `packages/spec/liveness/**` **是发布内容**(`package.json` 的 `files` 里有 `liveness`,`npm pack --dry-run` 实测 29 个文件入包,含该目录的 `README.md`)。一个新开始判红的 gate 必须同步它面向作者的文档,否则下一位作者只能靠撞红的 CI 才知道规则变了 —— 所以 README 里那段新的失败语义也随包发出,按 patch 记账,而不是空 frontmatter(空 changeset 是 `changesets/action` 的真实输入,全空集合会静默绿着卡住发布 —— #4898 卡住 17.0.0-rc.2 的就是这个)。当前 `main` 上 330 条本仓路径全部解析成功,该 gate 落地即绿。
13 changes: 12 additions & 1 deletion packages/spec/liveness/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,18 @@ checkout. Prose around the paths is fine and encouraged — `packages/spec/src/s
(mergeActionsIntoObjects stable-sorts each group)` resolves the path and ignores
the parenthetical.

**A path in another repo must say so**, or the gate will report it as rot:
**A repo-local path that does not resolve FAILS CI** (`✗`, exit 1 — since #5623;
it was a non-failing `⚠` before, calibrated for the parser bug described in the
note below rather than for the parser we have). A `live` verdict *is* its
evidence pointer, so a pointer into thin air is a claim nothing can falsify — and
a directory move or a rename is the whole cost of getting there. Repair it, or,
if the consumer is genuinely gone, re-classify the property under ADR-0049
enforce-or-remove instead of repointing at a plausible survivor.

**A path in another repo must say so**, or the gate will report it as rot —
attributed paths are counted and never resolved, so they cannot fail this check
(that boundary is deliberate: ~101 entries cite the renderer repo or the closed
cloud runtime, whose files are legitimately absent here):

```jsonc
"evidence": "objectui: packages/app-shell/src/views/RecordDetailView.tsx:573"
Expand Down
85 changes: 77 additions & 8 deletions packages/spec/scripts/liveness/check-liveness.mts
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,17 @@
// (see proof-registry.mts), a `live` classification MUST carry a valid proof —
// the file must exist and declare the `@proof: <id>` tag. CI fails otherwise.
//
// EVIDENCE POINTERS (ADR-0087): a `live` verdict IS its evidence pointer — "this
// property has a runtime consumer, here it is". A cited path that is repo-rooted
// and attributed to THIS repo must resolve against this checkout, or CI fails:
// an unresolvable pointer makes the claim unfalsifiable, and a directory move or
// a rename is all it takes (see evidence.mts). Cross-repo attribution
// (`objectui: …`, `cloud: …`, `packages/services/service-ai/…`) is counted, not
// resolved — those files are legitimately absent here. This was a ⚠ until #5623,
// for one historical reason: the pre-#3857 `evidence.split(':')[0]` parser
// flagged 48 of 227 entries with a ~100% false-positive rate, so failing on it
// would have failed every build. The parse fix is what turned a hit into signal.
//
// RE-VERIFICATION CLOCK (`verifiedAt`): a ledger entry is a claim with a
// timestamp, and code moves under it in BOTH directions — `flow.status` (#3711)
// and `action.undoable` (#3714) were both understated by entries that were
Expand All @@ -56,6 +67,10 @@
// tsx check-liveness.mts --stale-verification # print the re-verification worklist
// tsx check-liveness.mts --stale-verification=90 # ...with a custom staleness threshold
// tsx check-liveness.mts --undrilled # print the undrilled-container worklist
// tsx check-liveness.mts --ledger-root=<dir> # read the ledgers from <dir> instead of
// # packages/spec/liveness — how the self-test
// # runs the REAL gate against a mutated copy
// # of the real ledgers without touching them

process.env.OS_EAGER_SCHEMAS = '1';

Expand Down Expand Up @@ -93,7 +108,22 @@ import {
const here = dirname(fileURLToPath(import.meta.url));
const specRoot = resolve(here, '../..'); // packages/spec
const repoRoot = resolve(specRoot, '../..');
const ledgerRoot = join(specRoot, 'liveness');

const args = process.argv.slice(2);

// `--ledger-root=<dir>` points the walk at a COPY of packages/spec/liveness.
// It exists because the evidence gate below now fails the build, and a gate that
// fails is only worth as much as the proof that it fails — which needs a rotted
// pointer to fail ON. Committing one to a shipped ledger is not an option (it
// would fail every other run), and mutating a tracked file mid-test leaves the
// worktree dirty when the test crashes. So the self-test copies the real ledgers
// to a temp dir, breaks exactly one pointer there, and runs THIS script — same
// code path CI runs, no repo state touched. Everything else still resolves
// against the real repoRoot, so the resulting exit 1 has exactly one cause.
const ledgerRootArg = args.find((a) => a.startsWith('--ledger-root='));
const ledgerRoot = ledgerRootArg
? resolve(ledgerRootArg.slice('--ledger-root='.length))
: join(specRoot, 'liveness');

// Governed metadata types, rolled out highest-frequency / highest-risk first.
// (`query` is not a metadata type — see SPEC_ONLY_SCHEMAS below.)
Expand Down Expand Up @@ -262,7 +292,6 @@ function loadLedger(type: string): any {
}

// ---- dump mode ----
const args = process.argv.slice(2);
const dumpIdx = args.indexOf('--dump');
if (dumpIdx !== -1) {
const type = args[dumpIdx + 1];
Expand Down Expand Up @@ -300,8 +329,15 @@ const report: any = {
deferredContainers: [] as string[], // containers whose subtree IS classified elsewhere — resolved, not believed
deferredChildKeys: 0, // how many child keys those resolved deferrals actually cover
verification: null as VerificationReport | null, // `verifiedAt` ages — the re-verification worklist
evidenceLocal: 0, // repo-rooted evidence paths actually resolved against this checkout
evidenceForeign: 0, // evidence paths attributed to objectui / cloud — not resolvable here
// The three evidence counters, and the distinction between the first two is
// the whole point: `evidenceLocal` is how many repo-rooted paths `live` entries
// DECLARE, `evidenceMissing` is how many of those do not exist here. The
// summary line used to print `evidenceLocal` under the word "resolved", so
// breaking five pointers left the count at 330 and the run still said
// "330 resolved" (#5623). Count and word now agree.
evidenceLocal: 0, // repo-rooted evidence paths attributed to THIS repo — DECLARED, not yet proven
evidenceMissing: 0, // ...of which this many do not exist here (=== staleEvidence.length) — FAILS the gate
evidenceForeign: 0, // evidence paths attributed to objectui / cloud — not resolvable here, never failed
};

// Every classified entry, for the `verifiedAt` fold below. Collected during the
Expand All @@ -324,6 +360,7 @@ function classify(type: string, path: string, status: string, led: any, cat: any
const ev = checkEvidence(led.evidence, (p) => existsSync(join(repoRoot, p)));
report.evidenceForeign += ev.foreign.length;
report.evidenceLocal += ev.local.length;
report.evidenceMissing += ev.missing.length;
for (const miss of ev.missing) report.staleEvidence.push(`${type}/${path} → ${miss}`);
}
// ── ADR-0054 prove-it-runs ──
Expand Down Expand Up @@ -507,6 +544,11 @@ const totalProofFailures = report.proofErrors.length + report.proofMissing.lengt
const failed =
totalUnclassified > 0 ||
totalProofFailures > 0 ||
// A `live` entry whose repo-local evidence path is gone. Red since #5623 — the
// ⚠ it replaces was calibrated for the false-positive era, not for the parser
// that now resolves 330 paths and reports zero. Cross-repo attribution never
// reaches this list: checkEvidence only resolves the LOCAL bucket.
report.staleEvidence.length > 0 ||
report.orphanEntries.length > 0 ||
report.verification.errors.length > 0 ||
report.ungoverned.length > 0 ||
Expand All @@ -524,13 +566,39 @@ if (asJson) {
}
const boundClasses = HIGH_RISK_CLASSES.filter((c) => c.bound).map((c) => c.label);
console.log(`\nprove-it-runs (ADR-0054): proof REQUIRED for bound high-risk classes — ${boundClasses.join(', ') || 'none'}`);
// Two numbers, because one cannot carry both facts. "Declared" is the
// extraction-health signal (#3857's unit test guards it against degrading to
// "extracts nothing"); "resolved" is the verdict. They are equal on a green
// run, which is exactly why printing only the first read as a pass.
console.log(
`\nevidence paths: ${report.evidenceLocal} resolved against this checkout, ` +
`${report.evidenceForeign} attributed to another repo (objectui / cloud — not resolvable here).`,
`\nevidence paths: ${report.evidenceLocal} repo-local path(s) declared by 'live' entries, ` +
`${report.evidenceLocal - report.evidenceMissing} resolved against this checkout` +
(report.evidenceMissing ? `, ${report.evidenceMissing} MISSING` : '') +
`; ${report.evidenceForeign} attributed to another repo (objectui / cloud — not resolvable here).`,
);
if (report.staleEvidence.length) {
console.log(`\n ${report.staleEvidence.length} 'live' entr(ies) cite a missing file:`);
console.log(`\n ${report.staleEvidence.length} 'live' entr(ies) cite a file that is missing from THIS repo:`);
report.staleEvidence.forEach((s: string) => console.log(` ${s}`));
console.log(
'\n A `live` verdict IS its evidence pointer — "this property has a runtime consumer,\n' +
' here it is". When the cited file is gone from this checkout the claim is no longer\n' +
' falsifiable: declared, but nothing enforces that anything still reads the property,\n' +
' and a directory move or a rename is the whole cost of getting there.\n\n' +
' Three repairs, and picking the right one is the work:\n' +
' • the consumer MOVED inside this repo → repoint the path, and stamp `verifiedAt`\n' +
' while you have the call graph open;\n' +
' • the consumer moved to ANOTHER repo → say so with a realm marker\n' +
' (`objectui: packages/app-shell/…`, `cloud: …`). Attributed paths are counted,\n' +
' never resolved, and never fail here — that boundary is deliberate;\n' +
' • the consumer is GONE → the verdict is not `live` any more. Re-classify under\n' +
' ADR-0049 enforce-or-remove instead of repointing at a plausible survivor.\n\n' +
' This was a ⚠ until #5623 for one reason: the pre-#3857 parser took\n' +
' `evidence.split(":")[0]` as the filename, flagged 48 of 227 entries and every one\n' +
' was a false positive — so it could not fail the build, and the one real rot it was\n' +
' burying (`object.enable.clone`, whose consumer had moved packages) sat unread.\n' +
' evidence.mts fixed the parse; the list has been empty since, which is what makes\n' +
' a hit worth stopping for.',
);
}
if (report.orphanProofs.length) {
console.log(`\n⚠ ${report.orphanProofs.length} unregistered dogfood proof tag(s) — add to proof-registry.mts:`);
Expand Down Expand Up @@ -659,7 +727,8 @@ if (asJson) {
console.log(
'\n✓ every governed-type property at the walk\'s one-level granularity is classified, every ' +
'registered type is governed or explicitly pending, no ledger row outlives its property, ' +
'every container inheritance is declared, and all bound high-risk proofs resolve.',
'every container inheritance is declared, every `live` entry\'s repo-local evidence path ' +
'resolves, and all bound high-risk proofs resolve.',
);
if (report.undrilledChildKeys) {
console.log(
Expand Down
Loading
Loading