diff --git a/.changeset/tombstone-age-exact-key.md b/.changeset/tombstone-age-exact-key.md new file mode 100644 index 0000000000..cad59567ac --- /dev/null +++ b/.changeset/tombstone-age-exact-key.md @@ -0,0 +1,44 @@ +--- +"@objectstack/spec": patch +--- + +fix(spec): 墓碑老化时钟改按**确切 key** 起算 —— 无关簇的登记不再替一次退休提前放行 (#5898) + +`scripts/build-schemas.ts` 的检查 (c)(#4650)保证一件事:删掉一条 +`authorable-surface/` 基线行必须自带证明 —— 那个文件正是检查 (a)/(b) 赖以判定的 +证据,行没了证据也就没了。它承认的三种证明里,第一种是「墓碑已老化」:base 里这条 +是 `[RETIRED]`,且它的退休登记比当前 major 至少早 `TOMBSTONE_AGE_MAJORS`(= 2)。 + +#4659 把检查 (b) 收口到了 `RETIRED_KEYS_BY_MAJOR` 的确切 key,检查 (c) 的同一套 +**叶名匹配**原封不动地留了下来:拿 key 的叶名去和**全部 major** 的所有 conversion / +migration `surface` 子句做 `endsWith('.' + prop)`,再取 `Math.min`。两个后果都朝 +「放行」的方向 —— 一条无关登记就能让一个从没被登记过的墓碑通过「有没有登记」这一关, +而 `Math.min` 保证时钟一律从**最早**的那次巧合起算。 + +实测:当前 97 条历史墓碑里有 **2** 条今天就可删,而且**两条都是误判**,机制还不一样: + +- `data/Index:type` 被 protocol 11 的 `flow.node.type` 定了年份 —— + `flow-node-http-callout-rename` 里一个 flow 节点的 `type`,和索引类型毫无关系。 + 它自己那条诚实的登记 `object.indexes[].type` 是 major 17,时钟被提前了六个 major。 +- `api/RestApiConfig:requireAuth` 被 major 12 的 `api.requireAuth` 定了年份 —— + 那是 `rest-requireauth-default-flip`,一次**安全默认值翻转**,该 step 自己写着 + 「No metadata shape changed」。它真正的退休是 protocol 17 的 conversion + `stack.api.requireAuth`(#3963)。同一个 surface,不同**种类**的变更,早了五个 major。 + +现在检查 (c) 读的是检查 (b) 那张表,按确切 `` `${defKey}:${name}` `` 判定, +`build-schemas.ts` 里再没有任何叶名匹配。 + +**历史墓碑不回填,并且因此不可删。** 两条可机械推导的来源都无法诚实定年:叶名匹配 +正是 #4659 拿掉的那种推断(上面两条误判即为实证);而 +`authorable-surface.json` 的 git 历史始于 `17.0.0-rc.0`,把 97 条全部定在 major 17 +—— 那是基线文件的**出生日期**,不是考据。所以这些行保持未登记,检查 (c) 对它们 +fail-closed:没有条目就无法证明年龄,基线行不许删。可删数从 **2 → 0**(按当前 +`authorable-surface/` 的 100 条墓碑口径同样是 2 → 0)。 + +要删其中某一行,是一次有意的、可复核的动作:确定该 key 真正的退休 major,把确切 key +写进 `RETIRED_KEYS_BY_MAJOR`,由检查 (b2) 复核该条目仍指向一个本次构建确实 +tombstone 的 key。⚠ 定不出年份的行不要写 —— 写进这张表的估算,对之后每一道门禁都 +读作事实。 + +导出值本身没有变化;`RETIRED_KEYS_BY_MAJOR` 的文档注释更新为它现在同时被检查 (b)/(b2)/(c) +读取,以及历史墓碑的 fail-closed 口径。 diff --git a/.claude/skills/spec-property-retirement/SKILL.md b/.claude/skills/spec-property-retirement/SKILL.md index 638f8bd1a4..0f87724e55 100644 --- a/.claude/skills/spec-property-retirement/SKILL.md +++ b/.claude/skills/spec-property-retirement/SKILL.md @@ -202,14 +202,24 @@ follows. Write both. ignored), so `dashboard.aria` registered `ui/FormView:aria` and protocol 11's `flow.node.type` registered any `.type` at all (#4658). The guarantee had lapsed for every common leaf. + *The entry also starts the aging clock (#5898):* gate (c) reads the same + table to decide when this tombstone's `authorable-surface/` line may be + deleted (~two majors), so the major you write it under is the release the + clock counts from. Retirements older than the table are **undeclared and + therefore not deletable** — nothing could date them honestly (leaf matching + dated `data/Index:type` from an unrelated `flow.node.type` at major 11, and + the baseline file's own git history starts at 17.0.0-rc.0). To delete one of + those lines, establish its true major, add the entry, and say so in the PR. + ⚠ Never add a row you cannot date: an estimate here reads as a fact to + every later gate. - [ ] **`surface` stays prose — it is no longer matched.** Write it the way an author writes metadata (`flow.nodes[].outputSchema`), which is what the upgrade guide prints. Multi-key conversions still join clauses with exactly `' / '` (house style since the tool sweep). Nothing downstream parses it - for attribution any more — that job moved to the entry above. ⚠ One - consumer still does read the clauses by leaf: gate (c)'s *aged-out - tombstone* proof, which adjudicates retirements older than - `RETIRED_KEYS_BY_MAJOR` and could not be moved with it (#5898). + for attribution any more — that job moved to the entry above. Since #5898 + that is true of **every** consumer: gate (c)'s *aged-out tombstone* proof + was the last leaf matcher and now reads the same exact-key table, so no + rule anywhere parses a `surface` for attribution. - [ ] **`retiredFromLoadPath: true`** — for a retirement, always. Two distinct justifications, and they are not interchangeable: for a *rename* it means "no alias window, deliberately" (the tombstone owns the refusal; the entry diff --git a/packages/spec/scripts/build-schemas-check-mode.test.ts b/packages/spec/scripts/build-schemas-check-mode.test.ts index 16863b5f07..1ed278bd96 100644 --- a/packages/spec/scripts/build-schemas-check-mode.test.ts +++ b/packages/spec/scripts/build-schemas-check-mode.test.ts @@ -389,19 +389,32 @@ const CURRENT_MAJOR = Number.parseInt( const DELETED_LIVE = 'data/Object:zzNeverRetired4650'; /** Reachable def, tombstoned at base but never registered in the ADR-0087 registries. */ const DELETED_UNREGISTERED = 'data/Object:zzRetiredButUnregistered4650 [RETIRED]'; -/** Tombstoned AND registered, but too recently: `skill.triggerPhrases` was - * registered at protocol 17. The guard below fails loudly once that ages out — - * re-pick a clause registered within the last TOMBSTONE_AGE_MAJORS majors. */ -const DELETED_UNAGED_LEAF = 'triggerPhrases'; -const DELETED_UNAGED = `ai/Agent:${DELETED_UNAGED_LEAF} [RETIRED]`; +/** #5898's pin, in the shape the issue measured it. A tombstone whose LEAF is + * spoken for by an old, unrelated ADR-0087 clause — `.type` is registered at + * protocol 11 by `flow-node-http-callout-rename` (`flow.node.type`, a flow + * node's type) — and which nothing declares in RETIRED_KEYS_BY_MAJOR. + * + * Until #5898 check (c) dated the aging clock by exactly this leaf match and + * took the `Math.min`, so this deletion was WAVED THROUGH as an aged-out + * tombstone: `data/Index:type`, the live specimen, was deletable at major 17 + * one major after its own honest clause (`object.indexes[].type`, major 17) + * registered it. The def here is real and root-reachable, the prop is + * synthetic — the real key cannot be used, because check (c) only ever sees a + * key the build STOPPED emitting. */ +const DELETED_LEAF_COLLIDER_LEAF = 'type'; +const DELETED_LEAF_COLLIDER = `data/Object:${DELETED_LEAF_COLLIDER_LEAF} [RETIRED]`; /** Emitted but not reachable from any metadata-type root: a REST response * envelope no metadata document is ever parsed against (the issue's own * over-collection example). */ const DELETED_UNREACHABLE = 'api/SessionResponse:zzOverCollected4650'; /** Def the build no longer emits at all — the literal #4643 cluster. */ const DELETED_GONE_DEF = ['identity/Session:userId', 'identity/Session:token']; -/** Aged-out tombstone: `object.compactLayout` registered at protocol 11 — - * ≥ 2 majors behind any current major, so this fixture never goes stale. */ +/** Aged-out tombstone. Since #5898 the proof is a DECLARATION, not a clause + * match: the key must be named exactly in RETIRED_KEYS_BY_MAJOR under an old + * major. That cannot be exercised in this sandbox — it symlinks `src/`, so the + * registry is not writable here — so the aged-out and not-yet-aged branches + * moved to the #5898 block at the bottom, which copies `src/` and substitutes + * the table. What stays here is the branch that needs no declaration. */ const DELETED_AGED_LEAF = 'compactLayout'; const DELETED_AGED = `data/Object:${DELETED_AGED_LEAF} [RETIRED]`; /** Base key under a def RENAMED_DEFS moved: carried, so never a deletion. @@ -420,7 +433,7 @@ describe('build-schemas.ts — deleted baseline lines must prove themselves (#46 for (const injected of [ DELETED_LIVE, DELETED_UNREGISTERED, - DELETED_UNAGED, + DELETED_LEAF_COLLIDER, DELETED_UNREACHABLE, ...DELETED_GONE_DEF, DELETED_AGED, @@ -435,26 +448,35 @@ describe('build-schemas.ts — deleted baseline lines must prove themselves (#46 keys.some((k) => k.startsWith('identity/Session:')), 'identity/Session is emitted again — the vanished-def fixture needs a new def', ).toBe(false); - const unagedMajor = minRegisteredMajorForLeaf(DELETED_UNAGED_LEAF); + // The #5898 pin is only a pin while its leaf STILL collides with a clause + // old enough that the pre-#5898 matcher would have waved the deletion + // through. If either half stops holding, the test below still passes while + // asserting nothing about leaf-name matching — so both are loud here. + const colliderMajor = minRegisteredMajorForLeaf(DELETED_LEAF_COLLIDER_LEAF); expect( - unagedMajor !== null && CURRENT_MAJOR - unagedMajor < 2, - `'.${DELETED_UNAGED_LEAF}' (registered at major ${unagedMajor}) has aged out at major ` + - `${CURRENT_MAJOR} — re-pick a clause registered within the last 2 majors`, + colliderMajor !== null, + `'.${DELETED_LEAF_COLLIDER_LEAF}' is no longer registered by ANY ADR-0087 clause — the ` + + `#5898 fixture has stopped modelling a leaf collision; re-pick a colliding leaf`, ).toBe(true); - const agedMajor = minRegisteredMajorForLeaf(DELETED_AGED_LEAF); expect( - agedMajor !== null && CURRENT_MAJOR - agedMajor >= 2, - `'.${DELETED_AGED_LEAF}' is no longer an aged-out registration`, + colliderMajor !== null && CURRENT_MAJOR - colliderMajor >= 2, + `'.${DELETED_LEAF_COLLIDER_LEAF}' collides only at major ${colliderMajor}, which has NOT ` + + `aged out at major ${CURRENT_MAJOR} — the pre-#5898 matcher would have rejected this ` + + `deletion anyway, so the pin no longer discriminates; re-pick an older colliding leaf`, ).toBe(true); + expect( + Object.values(RETIRED_KEYS_BY_MAJOR).flat(), + `${DELETED_LEAF_COLLIDER} is now declared for real — the pin needs an UNdeclared key`, + ).not.toContain(DELETED_LEAF_COLLIDER.replace(RETIRED_MARK, '')); // The manifest ratchet runs first; keep it current so every run reaches (c). seedManifest((s) => s); }); it( - 'fails on deletions of reachable keys — live, unregistered, and not-yet-aged tombstones each say why', + 'fails on deletions of reachable keys — a live key and an undeclared tombstone each say why', { timeout: SPAWN_TIMEOUT_MS }, () => { - seedBase((s) => [...s, DELETED_LIVE, DELETED_UNREGISTERED, DELETED_UNAGED].sort()); + seedBase((s) => [...s, DELETED_LIVE, DELETED_UNREGISTERED].sort()); const canonical = seedSurface((s) => s); const { status, output } = run(['--check']); @@ -462,15 +484,51 @@ describe('build-schemas.ts — deleted baseline lines must prove themselves (#46 expect(status).toBe(1); expect(output).toContain('authorable baseline line(s) were deleted without proof (#4650)'); expect(output).toMatch(/data\/Object:zzNeverRetired4650 — def reachable .* LIVE \(never tombstoned\)/); - expect(output).toMatch(/data\/Object:zzRetiredButUnregistered4650 — .*tombstoned, but no conversion\/migration clause/); - expect(output).toMatch(new RegExp(`ai/Agent:${DELETED_UNAGED_LEAF} — .*registered at major \\d+`)); - expect(output).toMatch(/must age ≥ 2 majors/); + expect(output).toMatch( + /data\/Object:zzRetiredButUnregistered4650 — .*tombstoned, but no entry in RETIRED_KEYS_BY_MAJOR/, + ); // The remedy names the retirement route, not a hand-edit. expect(output).toContain('gen:schema'); expect(readSurface()).toBe(canonical); }, ); + it( + "#5898 pin — an undeclared tombstone is NOT dated by an unrelated clause that ends in its leaf", + { timeout: SPAWN_TIMEOUT_MS }, + () => { + // The gate's own vocabulary, stated as the fixture guard above proves it: + // `.type` IS registered — at protocol 11, by a flow-node rename — and this + // key is NOT in RETIRED_KEYS_BY_MAJOR. Before #5898 those two facts + // combined into "registered at major 11 ⇒ aged out ⇒ deletion allowed", + // exit 0. The retirement it actually belongs to (`object.indexes[].type`) + // is major 17; the clock was started six majors early by a coincidence of + // spelling, and `Math.min` guaranteed the earliest such coincidence won. + seedBase((s) => [...s, DELETED_LEAF_COLLIDER].sort()); + const canonical = seedSurface((s) => s); + + const { status, output } = run(['--check']); + + expect(status).toBe(1); + expect(output).toContain('authorable baseline line(s) were deleted without proof (#4650)'); + expect(output).toMatch( + new RegExp( + `data/Object:${DELETED_LEAF_COLLIDER_LEAF} — .*tombstoned, but no entry in RETIRED_KEYS_BY_MAJOR`, + ), + ); + // Specifically NOT the aged-out verdict, and specifically not dated by the + // colliding clause's major. Asserting the absence is the pin: a leaf-name + // matcher restored here makes the run exit 0 with exactly these strings. + expect(output).not.toMatch( + new RegExp(`data/Object:${DELETED_LEAF_COLLIDER_LEAF} — .*tombstone aged out`), + ); + expect(output).not.toMatch( + new RegExp(`data/Object:${DELETED_LEAF_COLLIDER_LEAF} — .*registered at major \\d+`), + ); + expect(readSurface()).toBe(canonical); + }, + ); + it( 'still fails after the deletion is COMMITTED (the CI shape): the anchor is the merge base with origin/main, not HEAD', { timeout: SPAWN_TIMEOUT_MS * 2 }, @@ -498,10 +556,14 @@ describe('build-schemas.ts — deleted baseline lines must prove themselves (#46 ); it( - 'allows deletions that carry their own proof: unreachable def, vanished def, aged-out tombstone — each with its reason printed', + 'allows deletions that carry their own proof: unreachable def, vanished def — each with its reason printed', { timeout: SPAWN_TIMEOUT_MS }, () => { - seedBase((s) => [...s, DELETED_UNREACHABLE, ...DELETED_GONE_DEF, DELETED_AGED].sort()); + // The third proof — an aged-out tombstone — needs a RETIRED_KEYS_BY_MAJOR + // entry since #5898, so it is exercised in the block at the bottom that can + // write the table. `DELETED_AGED` stays out of this fixture deliberately: + // left in, it would now be a violation, and this test asserts on `allowed`. + seedBase((s) => [...s, DELETED_UNREACHABLE, ...DELETED_GONE_DEF].sort()); const canonical = seedSurface((s) => s); const { status, output } = run(['--check']); @@ -516,9 +578,6 @@ describe('build-schemas.ts — deleted baseline lines must prove themselves (#46 // Whole-def removal is the manifest ratchet's jurisdiction. expect(output).toContain('identity/Session:* (2 line(s))'); expect(output).toContain('json-schema.manifest/ (#2978)'); - // Aged-out tombstone names its registration major. - expect(output).toMatch(/data\/Object:compactLayout — \[RETIRED\] at [0-9a-f]+ and registered at major 11/); - expect(output).toContain('tombstone aged out'); expect(readSurface()).toBe(canonical); expect(status).toBe(0); }, @@ -2069,3 +2128,208 @@ describe('build-schemas.ts — a deleted manifest key must prove itself (#4725)' }, ); }); + +// ───────────────────────────────────────────────────────────────────────────── +// #5898 — check (c) dates a tombstone by its EXACT key, not by its leaf. +// +// #4659 moved check (b) off leaf-name matching and left check (c) on it, in the +// open, with a comment saying so: (c) adjudicates HISTORICAL tombstones, all 97 +// of which predate RETIRED_KEYS_BY_MAJOR, so pointing it at that table meant +// first deciding what to do about rows nothing could date. +// +// The two consequences both pointed at "release early", and both were live on +// the real baseline — 2 of the 97 rows were deletable, and BOTH were false +// positives, by different mechanisms: +// +// - `data/Index:type` was dated by `flow.node.type` (protocol 11, +// `flow-node-http-callout-rename`) — an unrelated cluster that merely ends +// in the same leaf. Its own clause, `object.indexes[].type`, is major 17. +// - `api/RestApiConfig:requireAuth` was dated by `api.requireAuth` at major +// 12 — which is `rest-requireauth-default-flip`, a secure-DEFAULT flip whose +// step states "No metadata shape changed". Its retirement is the protocol 17 +// conversion `stack.api.requireAuth` (#3963). Same surface, different KIND +// of change, five majors early. +// +// So the backfill was not attempted: leaf-matching the registry is the very +// inference #4659 removed, and `authorable-surface.json`'s git history starts at +// 17.0.0-rc.0 and therefore dates all 97 rows at major 17 — the file's birth, +// not archaeology. Undated ⇒ undeletable, and the after-count is 0 of 97 by +// design. The gate below is what that costs and what it buys. +// +// This block needs its own sandbox because the aged-out branch is now a claim +// about `src/migrations/registry.ts`, which the main sandbox symlinks. + +/** A tombstone declared at a major far enough back to have aged out. */ +const AGED_DECLARED_MAJOR = 11; +/** …and one declared at the current major, which has not. */ +const UNAGED_DECLARED_KEY = 'data/Object:zzDeclaredThisMajor5898'; +const UNAGED_DECLARED = `${UNAGED_DECLARED_KEY} [RETIRED]`; + +describe('build-schemas.ts — check (c) dates a tombstone by its exact key (#5898)', () => { + let box: string; + let boxScript: string; + let boxSurfaceDir: string; + let boxRegistry: string; + let pristineRegistry: string; + + const boxGit = (...args: string[]): string => { + const r = spawnSync( + 'git', + ['-c', 'user.name=build-schemas-test', '-c', 'user.email=test@example.invalid', ...args], + { cwd: box, encoding: 'utf8' }, + ); + if (r.status !== 0) throw new Error(`git ${args.join(' ')} failed (${r.status}): ${r.stderr}`); + return (r.stdout ?? '').trim(); + }; + + const runBox = (args: string[] = []): { status: number; output: string } => { + const r = spawnSync(TSX, [boxScript, ...args], { + cwd: box, + encoding: 'utf8', + timeout: SPAWN_TIMEOUT_MS, + stdio: ['ignore', 'pipe', 'pipe'], + }); + return { status: r.status ?? -1, output: `${r.stdout ?? ''}${r.stderr ?? ''}` }; + }; + + /** Substitute RETIRED_KEYS_BY_MAJOR in this box's own copy of the registry. */ + const seedRetiredKeys = (table: Record): void => { + const rendered = + `export const RETIRED_KEYS_BY_MAJOR: Readonly> = {\n` + + Object.keys(table) + .map(Number) + .sort((a, b) => a - b) + .map((m) => ` ${m}: [\n${table[m]!.map((k) => ` '${k}',\n`).join('')} ],\n`) + .join('') + + `};\n`; + const anchor = /export const RETIRED_KEYS_BY_MAJOR[\s\S]*?\n\};\n/; + expect( + anchor.test(pristineRegistry), + 'RETIRED_KEYS_BY_MAJOR is no longer a single object literal in src/migrations/registry.ts — ' + + 'this fixture substitutes it textually and can no longer find it', + ).toBe(true); + fs.writeFileSync(boxRegistry, pristineRegistry.replace(anchor, rendered)); + }; + + /** Commit a BASE carrying extra lines; the worktree keeps the pristine set, + * so those lines read as deleted by this build (the main sandbox's shape). */ + const seedBoxBase = (...extra: string[]): void => { + writeSurfaceShards(boxSurfaceDir, [...pristineSurface, ...extra].sort()); + boxGit('add', AUTHORABLE_SURFACE_DIR_NAME); + boxGit('commit', '-q', '--allow-empty', '-m', 'base variant'); + boxGit('update-ref', 'refs/remotes/origin/main', boxGit('rev-parse', 'HEAD')); + writeSurfaceShards(boxSurfaceDir, pristineSurface); + }; + + beforeAll(() => { + // Fixture validity, loud. The aged fixture must NOT owe its verdict to a + // clause match — that is the whole point — so its leaf is checked to be + // registered no earlier than the major we declare it at is irrelevant; what + // matters is that the key is not already in the baseline or the real table. + for (const injected of [DELETED_AGED, UNAGED_DECLARED]) { + expect( + pristineSurface.includes(injected) || + pristineSurface.includes(injected.replace(RETIRED_MARK, '')), + `fixture ${injected} already exists in the committed baseline — pick another`, + ).toBe(false); + } + expect(CURRENT_MAJOR - AGED_DECLARED_MAJOR).toBeGreaterThanOrEqual(2); + + box = fs.mkdtempSync(path.join(os.tmpdir(), 'build-schemas-tombstone-age-')); + fs.cpSync(path.join(PKG, 'scripts'), path.join(box, 'scripts'), { recursive: true }); + fs.cpSync(path.join(PKG, 'src'), path.join(box, 'src'), { recursive: true }); + for (const entry of ['node_modules', 'package.json']) { + fs.symlinkSync(path.join(PKG, entry), path.join(box, entry)); + } + writeManifestShards(path.join(box, SCHEMA_MANIFEST_DIR_NAME), pristine); + boxSurfaceDir = path.join(box, AUTHORABLE_SURFACE_DIR_NAME); + writeSurfaceShards(boxSurfaceDir, pristineSurface); + boxScript = path.join(box, 'scripts', 'build-schemas.ts'); + boxRegistry = path.join(box, 'src', 'migrations', 'registry.ts'); + pristineRegistry = fs.readFileSync(boxRegistry, 'utf8'); + + boxGit('init', '-q', '-b', 'main', '.'); + boxGit('add', AUTHORABLE_SURFACE_DIR_NAME); + boxGit('commit', '-q', '-m', `baseline: committed ${AUTHORABLE_SURFACE_DIR_NAME}/`); + fs.writeFileSync( + path.join(box, 'authorable-surface.base.json'), + JSON.stringify( + { description: surfaceBaseDescription, baseRev: boxGit('rev-parse', 'HEAD'), keys: pristineSurface }, + null, + 2, + ) + '\n', + ); + boxGit('add', 'authorable-surface.base.json'); + boxGit('commit', '-q', '-m', 'baseline anchor'); + boxGit('update-ref', 'refs/remotes/origin/main', 'HEAD'); + }); + + afterAll(() => { + if (box) fs.rmSync(box, { recursive: true, force: true }); + }); + + it( + 'a tombstone DECLARED at an aged major may have its baseline line deleted', + { timeout: SPAWN_TIMEOUT_MS }, + () => { + // The surviving positive route. Note what supplies the major: the table, + // not the leaf — `compactLayout` happens to have an old clause too, so + // this case alone cannot tell the two mechanisms apart. The pin that can + // is in the main sandbox (`data/Object:type`) and in the last test here. + seedRetiredKeys({ [AGED_DECLARED_MAJOR]: [DELETED_AGED.replace(RETIRED_MARK, '')] }); + seedBoxBase(DELETED_AGED); + + const { status, output } = runBox(['--check']); + + expect(output).toContain('carry their own proof (#4650)'); + expect(output).toMatch( + new RegExp( + `data/Object:${DELETED_AGED_LEAF} — \\[RETIRED\\] at [0-9a-f]+ and registered at major ${AGED_DECLARED_MAJOR}`, + ), + ); + expect(output).toContain('tombstone aged out'); + expect(status).toBe(0); + }, + ); + + it( + 'a tombstone DECLARED at the current major is not deletable yet', + { timeout: SPAWN_TIMEOUT_MS }, + () => { + seedRetiredKeys({ [CURRENT_MAJOR]: [UNAGED_DECLARED_KEY] }); + seedBoxBase(UNAGED_DECLARED); + + const { status, output } = runBox(['--check']); + + expect(status).toBe(1); + expect(output).toContain('deleted without proof (#4650)'); + expect(output).toMatch( + new RegExp(`${UNAGED_DECLARED_KEY} — .*tombstone registered at major ${CURRENT_MAJOR}`), + ); + expect(output).toMatch(/must age ≥ 2 majors/); + }, + ); + + it( + 'the pin: with the table EMPTY, a leaf collision proves nothing — the deletion is refused', + { timeout: SPAWN_TIMEOUT_MS }, + () => { + // `compactLayout` is registered at protocol 11 by an ADR-0087 clause, so + // the pre-#5898 matcher dated it there and allowed the deletion — exit 0, + // "tombstone aged out". With the declaration removed and nothing else + // changed, the only thing that can still supply major 11 is leaf matching. + expect(minRegisteredMajorForLeaf(DELETED_AGED_LEAF)).toBe(AGED_DECLARED_MAJOR); + seedRetiredKeys({}); + seedBoxBase(DELETED_AGED); + + const { status, output } = runBox(['--check']); + + expect(status).toBe(1); + expect(output).toContain('deleted without proof (#4650)'); + expect(output).toMatch( + new RegExp(`data/Object:${DELETED_AGED_LEAF} — .*no entry in RETIRED_KEYS_BY_MAJOR`), + ); + expect(output).not.toContain('tombstone aged out'); + }, + ); +}); diff --git a/packages/spec/scripts/build-schemas.ts b/packages/spec/scripts/build-schemas.ts index db642b411f..639b1f8124 100644 --- a/packages/spec/scripts/build-schemas.ts +++ b/packages/spec/scripts/build-schemas.ts @@ -39,12 +39,7 @@ import { writeShards, type GitRun, } from './lib/sharded-artifacts'; -import { CONVERSIONS_BY_MAJOR } from '../src/conversions/registry'; -import { - MIGRATIONS_BY_MAJOR, - RETIRED_DEFS_BY_MAJOR, - RETIRED_KEYS_BY_MAJOR, -} from '../src/migrations/registry'; +import { RETIRED_DEFS_BY_MAJOR, RETIRED_KEYS_BY_MAJOR } from '../src/migrations/registry'; import { getMetadataTypeSchema, listMetadataTypeSchemaTypes, @@ -887,10 +882,15 @@ if (surfaceDoc) { // A deletion is legitimate on exactly one of three proofs, each computed // inside this gate — never argued in a PR description: // -// 1. aged-out tombstone — the base entry carried `[RETIRED]` AND its surface -// is registered in CONVERSIONS_BY_MAJOR / MIGRATIONS_BY_MAJOR at a major +// 1. aged-out tombstone — the base entry carried `[RETIRED]` AND its EXACT +// `${defKey}:${name}` is declared in RETIRED_KEYS_BY_MAJOR at a major // ≥ TOMBSTONE_AGE_MAJORS behind the current one (the "~two majors" this -// file's description has always promised, now enforced); +// file's description has always promised, now enforced). Since #5898 this +// reads the same exact-key table check (b) reads, not a leaf-name match +// against the conversion registry — see TOMBSTONE_AGE_MAJORS below for the +// two false positives that matching produced and why the 97 pre-existing +// tombstones are left undeclared (and therefore undeletable) rather than +// backfilled from a source that cannot date them; // 2. the def is not reachable from the metadata-type roots (2026-08-02 // ruling on #4650): no metadata document is ever parsed by it, so its // entry was over-collection and there is no author to tombstone for. @@ -909,37 +909,40 @@ if (surfaceDoc) { /** A tombstone may be deleted once its registration is this many majors old. */ const TOMBSTONE_AGE_MAJORS = 2; -/** - * Every ' / '-separated surface clause registered across the ADR-0087 - * registries, mapped to the EARLIEST major that registered it — the moment the - * retirement became visible to consumers, which is when its aging clock - * started. - * - * This is now the LAST leaf-name matcher in this file: check (b) above moved to - * exact `${defKey}:${name}` membership in RETIRED_KEYS_BY_MAJOR (#4659). Check - * (c) cannot follow it there, because it adjudicates tombstones that predate - * that table and were deliberately not backfilled into it — so the aging proof - * still reads the conversion clauses, leaf and all, and inherits the same - * coincidence (an unrelated cluster's `.type` can date a tombstone's clock). - * Narrowing it needs the historical mapping #4659 would have had to invent — - * all 97 current tombstones predate that table — so it is tracked in #5898. - */ -function registeredClauseMajors(): Map { - const out = new Map(); - const add = (clause: string, major: number): void => { - const prev = out.get(clause); - if (prev === undefined || major < prev) out.set(clause, major); - }; - for (const [major, list] of Object.entries(CONVERSIONS_BY_MAJOR)) { - for (const c of list) for (const clause of c.surface.split(' / ')) add(clause, Number(major)); - } - for (const [major, step] of Object.entries(MIGRATIONS_BY_MAJOR)) { - for (const sem of step.semantic ?? []) { - for (const clause of sem.surface.split(' / ')) add(clause, Number(major)); - } - } - return out; -} +// The aging clock reads RETIRED_KEYS_BY_MAJOR — `registeredRetiredKeys()` above, +// the same exact-key map check (b) reads. There is no leaf-name matching left in +// this file (#5898). +// +// Until #5898 check (c) dated a tombstone by matching the key's LEAF against +// every ' / ' clause of every major in CONVERSIONS_BY_MAJOR / MIGRATIONS_BY_MAJOR +// and taking `Math.min` — structurally the matcher #4659 had just removed from +// check (b), and permissive in both of its halves: an unrelated clause ending in +// the same leaf proved "this was registered at all", and `Math.min` then started +// the clock at the EARLIEST such coincidence. Both of the two rows it let through +// on the real baseline were false positives, by two different mechanisms: +// +// - `data/Index:type` (#5898's specimen) matched protocol 11's +// `flow.node.type` — a flow node's type, nothing to do with index types — so +// its clock started at 11 while its own honest clause `object.indexes[].type` +// is major 17. At major 17 the row was deletable one major after its +// retirement became visible. +// - `api/RestApiConfig:requireAuth` matched protocol 12's `api.requireAuth`, +// which is `rest-requireauth-default-flip`: a secure-DEFAULT flip whose own +// step says "No metadata shape changed". Its retirement is the protocol 17 +// conversion `stack.api.requireAuth` (#3963). The clock was started from a +// different KIND of change to the same surface. +// +// Why the historical mapping is not backfilled, and what happens instead: a +// tombstone with no entry in RETIRED_KEYS_BY_MAJOR cannot prove its age, so its +// baseline line stays. That is fail-closed by construction rather than by +// estimate — see the table's "Historical tombstones" section for why neither +// available source could date the 97 pre-existing rows honestly (leaf-matching +// the conversion registry is the very inference #4659 removed; this file's own +// git history begins at 17.0.0-rc.0, so it dates every one of them at major 17 +// — an artifact of the baseline's birth, not archaeology). Deleting one of those +// lines is therefore a deliberate, reviewable act: declare its exact key under +// its true major in the table, and check (b2) verifies the entry still names a +// key this build tombstones. interface SurfaceReachability { /** The metadata-type roots the BFS started from. */ @@ -1641,7 +1644,7 @@ function checkManifestRemovals(git: GitRun, baseRev: string | null): void { const deletedKeys = [...baseSnapshot.keys()].filter((k) => !currentKeys.has(k)); if (deletedKeys.length > 0) { const baseRev = base.rev.slice(0, 12); - const clauseMajors = registeredClauseMajors(); + const declaredRetired = registeredRetiredKeys(); const reachability = computeSurfaceReachability(); const allowed: string[] = []; const violations: string[] = []; @@ -1674,16 +1677,17 @@ function checkManifestRemovals(git: GitRun, baseRev: string | null): void { violations.push(`${key} — def ${how}; the entry at ${baseRev} was LIVE (never tombstoned).`); continue; } - const matches = [...clauseMajors.entries()].filter(([clause]) => clause.endsWith('.' + prop)); - if (matches.length === 0) { + const registeredAt = declaredRetired.get(key); + if (registeredAt === undefined) { violations.push( - `${key} — def ${how}; tombstoned, but no conversion/migration clause matching '.${prop}'\n` + - ` is registered in the ADR-0087 registries, so the retirement never reached\n` + - ` spec-changes.json or \`os migrate meta\`.`, + `${key} — def ${how}; tombstoned, but no entry in RETIRED_KEYS_BY_MAJOR names this\n` + + ` EXACT key, so there is nothing that dates the retirement and its aging clock\n` + + ` has no start. Until #5898 a leaf-name match against unrelated ADR-0087\n` + + ` clauses supplied one by coincidence (a flow node's '.type' dated an index\n` + + ` type's tombstone) — always erring early, since it took the Math.min.`, ); continue; } - const registeredAt = Math.min(...matches.map(([, major]) => major)); if (CURRENT_MAJOR - registeredAt < TOMBSTONE_AGE_MAJORS) { violations.push( `${key} — def ${how}; tombstone registered at major ${registeredAt}, current major is\n` + @@ -1718,9 +1722,15 @@ function checkManifestRemovals(git: GitRun, baseRev: string | null): void { ` compared against the baseline at merge base ${baseRev} with origin/main,\n` + ` which this commit cannot rewrite.\n\n` + ` A line may only leave this file when:\n` + - ` 1. its key was tombstoned (\`retiredKey()\` → "[RETIRED]") with a D2 conversion\n` + - ` (src/conversions/registry.ts) or migration step registered for its surface,\n` + - ` AND that registration is ≥ ${TOMBSTONE_AGE_MAJORS} majors old (≤ v${CURRENT_MAJOR - TOMBSTONE_AGE_MAJORS}); or\n` + + ` 1. its key was tombstoned (\`retiredKey()\` → "[RETIRED]") AND that key is\n` + + ` declared — EXACTLY, as '\${defKey}:\${name}' — in RETIRED_KEYS_BY_MAJOR\n` + + ` (src/migrations/registry.ts) under a major ≥ ${TOMBSTONE_AGE_MAJORS} behind this one\n` + + ` (≤ v${CURRENT_MAJOR - TOMBSTONE_AGE_MAJORS}). Tombstones that predate that table are deliberately\n` + + ` undeclared: nothing could date them honestly, so they are NOT deletable\n` + + ` until someone establishes the true major and writes it down (#5898). The\n` + + ` D2 conversion (src/conversions/registry.ts) naming the surface stays\n` + + ` required — it is the prescription consumers follow — but it is no longer\n` + + ` what dates the clock; or\n` + ` 2. its def is not reachable from the metadata-type roots — this gate computes\n` + ` that itself (it would have said so above); or\n` + ` 3. its whole def stopped being emitted — adjudicated by the manifest deletion\n` + diff --git a/packages/spec/src/migrations/registry.ts b/packages/spec/src/migrations/registry.ts index 6a08d25441..4b74dfaf7c 100644 --- a/packages/spec/src/migrations/registry.ts +++ b/packages/spec/src/migrations/registry.ts @@ -2152,10 +2152,18 @@ export const MIGRATION_MAJORS: readonly number[] = Object.keys(MIGRATIONS_BY_MAJ * * ## What reads it * - * Check (b) of `scripts/build-schemas.ts` (`check:authorable-surface`): a key - * that flips live → retired must appear here, by exact set membership, or the - * build fails. Nothing else consumes the table, and nothing infers an entry — - * the author writes the key down or the gate stays red. + * Two gates in `scripts/build-schemas.ts` (`check:authorable-surface`), and + * nothing infers an entry for either — the author writes the key down or the + * gate stays red: + * + * - **Check (b)**, the retirement gate: a key that flips live → retired must + * appear here, by exact set membership, or the build fails. (Check (b2) is its + * inverse: an entry naming a key this build still emits as LIVE is rejected.) + * - **Check (c)**, the baseline-deletion gate (#4650): the major recorded here + * is what starts a tombstone's ~two-major aging clock, so it is also what + * eventually lets its `authorable-surface/` line be deleted. Since #5898 — + * before that, (c) dated the clock by matching the key's LEAF against the + * conversion registry, and took the `Math.min` of the matches. * * ## Why exact keys, and not the conversion `surface` * @@ -2184,26 +2192,45 @@ export const MIGRATION_MAJORS: readonly number[] = Object.keys(MIGRATIONS_BY_MAJ * table. An entry here is the *proof the retirement was declared*; the * conversion is the *prescription a consumer follows*. A retirement needs both. * - * ## Not a backfill of history + * ## Historical tombstones: absent, and therefore not deletable * * Check (b) fires only on a NEW live → retired transition — measured against the * committed `authorable-surface.json` baseline, which already records every - * older tombstone as `[RETIRED]`. Retirements that landed before this table - * existed therefore never re-trigger it and are deliberately absent: this reads - * "retirements registered under the exact-key gate", not "every retirement - * ever". Do not reconstruct the missing history by leaf-matching the conversion - * registry — that is precisely the inference #4659 removed. + * older tombstone as `[RETIRED]`. The 97 retirements that landed before this + * table existed therefore never re-trigger it and are deliberately absent: this + * reads "retirements registered under the exact-key gate", not "every retirement + * ever". + * + * #5898 measured what it would take to backfill them, and neither available + * source can date a row honestly: + * + * - **Leaf-matching the conversion registry** is precisely the inference #4659 + * removed. It is also demonstrably wrong in both directions on this very data: + * of the two rows it dated as aged-out, `data/Index:type` was dated by + * `flow.node.type` (an unrelated cluster) and `api/RestApiConfig:requireAuth` + * by `api.requireAuth` at major 12 — which is the secure-default FLIP, a + * different kind of change to the same surface, not its retirement. + * - **The git history of `authorable-surface.json`** begins at 17.0.0-rc.0, so + * every one of the 97 rows first carries `[RETIRED]` at major 17. That is an + * artifact of the baseline file's birth, not archaeology. + * + * So the rows stay undeclared, and check (c) is fail-closed about it: a + * tombstone with no entry here cannot prove its age and its baseline line may + * not be deleted. Deleting one is a deliberate, reviewable act — establish the + * true major for that key, add it below, and check (b2) verifies the entry still + * names a key this build tombstones. Do NOT add a row you cannot date; an + * estimate written down here reads as a fact to every later gate. * * ## Lifecycle * - * Entries are permanent. A tombstone ages out after ~two majors and its line - * leaves `authorable-surface.json` (check (c)); its entry here stays, and then - * names a key the build no longer emits — the expected steady state, not an - * error. The one state the gate rejects is an entry naming a key that is still - * LIVE: a registration nothing consumed, pre-approving a retirement that has not - * happened. + * Entries are permanent. A declared tombstone ages out after ~two majors and its + * line may then leave `authorable-surface.json` (check (c)); its entry here + * stays, and then names a key the build no longer emits — the expected steady + * state, not an error. The one state the gate rejects is an entry naming a key + * that is still LIVE: a registration nothing consumed, pre-approving a + * retirement that has not happened. * - * @see scripts/build-schemas.ts — checks (b)/(b2), the only consumers + * @see scripts/build-schemas.ts — checks (b)/(b2)/(c), the only consumers */ export const RETIRED_KEYS_BY_MAJOR: Readonly> = { // The first entries since #4659 built this table (#5552). ONE tombstone