Skip to content

Commit d050a8f

Browse files
fix(spec,objectql): installed pages carrying a retired declaration convert at load instead of stopping the kernel build (#23962)
Fixes #23904 Clause-②: yes (widening) An already-installed package or stored page carrying a declaration spec 18 retires no longer stops the environment's kernel build. The mechanism is the ADR-0087 D2 transition pattern, not a tolerance switch in the registry: two retired conversion entries that data at rest replays, plus the versioned artifact window opened at the one door every package registration reaches. The authoring, save and publish doors refuse exactly what they refused before. ## What changes | Door | Before | Now | | :--- | :--- | :--- | | `definePage` / `defineStack` (`os validate`, `os build`), the metadata save door (`PUT /api/v1/meta`), package publish | refused at `pages.N.isDefault` / `pages.N.template` | unchanged, same sentence (pins: `page-default-and-region-placement.test.ts`, `default-record-page-save-door.test.ts`) | | a `sys_metadata` page row read back | served with `_diagnostics.valid: false` | converted on read: `isDefault` deleted off a non-bound page, `full-width` re-spelled `default`, one `OS_METADATA_CONVERTED` notice per site (pin: `page-default-and-region-placement.test.ts` ledger block) | | a compiled artifact inside the window its `engines.protocol` opens | refused at the artifact door's strict parse | converted before the parse, as every retired entry is (pins: the whole-step blocks in `artifact-forward-conversion.test.ts` and `artifact-forward-conversion-package-body.test.ts`, which now carry both entries) | | a package body registered through `manifest.register` / `ObjectQL.registerApp` that declares `engines.protocol` (an installed package, a marketplace install, a hosted kernel's loader) | `MisplacedDefaultPageError`, kernel build stopped | converted before anything registers; one `warn` per (package, conversion) naming the conversion id, the floor and the first site (pin: `registry-installed-package-retired-page-declarations.test.ts`) | | the same body with no declared range (a manifest a plugin builds in code), or declaring a floor newer than every retirement | refused | unchanged, refused (CONTROL pins in the same file and in `registry-default-record-page.test.ts`, untouched) | - `packages/spec/src/conversions/registry.ts`: `page-is-default-off-record-page-removed` (strips `isDefault: true` from exactly the pages `misplacedDefaultPageRefusal` refuses, the parse's one judge) and `page-template-full-width-to-default`; both `toMajor: 18`, `retiredFromLoadPath: true`, `retiredAfter: '17.7.0'`, order 67, inserted at their id-sorted positions with their definitions directly above the entry that follows (the step-18 merge-shape test is green). - `packages/objectql/src/engine.ts`: `ObjectQL.convertPackageBodyForward(body)` replays `applyArtifactForwardConversions` over a body that declares a protocol range; `registerApp` registers the converted copy (record, live manifest, every item). `packages/objectql/src/plugin.ts`: the `manifest` service converts an artifact that carries `packages[]` once, before `resolveArtifactPackageOrder` strict-parses its bodies. Idempotent and copy-on-write: a body an artifact door already converted comes back by reference with no notice. - `packages/metadata-core/src/artifact-forward-conversion.ts`: a definition with no `manifest` key reads its declared range off itself (the flat package-body shape); one with a `manifest` object is read there, as before. - D3 entries `page-is-default-off-record-page-refused` / `page-template-closed-region-unplaced-refused`: prose now names the data-at-rest halves; no new tombstone, no new key. `packages/spec/liveness/page.json`: the `isDefault` and `template` notes carry the load-path reading (state counts unchanged). - Fixture: `packages/objectql/src/__fixtures__/hotcrm-4.0.1-published-pages.artifact.json`. ## Readings (PM mechanism assumptions, measured) 1. **Call chain of the measured failure.** The cloud log names `MisplacedDefaultPageError`, the registry's own throw. The strict body parse (`resolveArtifactPackageOrder` → `ArtifactPackageSchema`, run by the `manifest` service on an artifact carrying `packages[]`, by `createStandaloneStack`, by `loadArtifactBundle`'s readers) would have refused HotCRM 4.0.1 FIRST, with a Zod issue at `packages.1.manifest.pages.5.isDefault` (and at `template` / `regions`), never with that error name. So the kernel that failed registered the package bodies flat, through `manifest.register` / `registerApp` → `SchemaRegistry.installPackage` → `refuseDefaultRecordPageConflicts` (`registry.ts` package door), with no parse and no artifact file door in between. Both throws (`:2717` package door, `:4652` item seam) sit behind `registerApp`; the door now converts ahead of both. `template` and `regions`: the registry never judged them (parse-only refusals, as #23708 recorded), so on that chain they never fired; they fire at the strict-parse doors only. 2. **Mechanism premise, partly false.** On the artifact file doors (`loadArtifactBundle`, `MetadataPlugin`, `AppPlugin`) the registry already sees converted bytes, so the D2 entry alone fixes them. On the flat registration chain above nothing converted before the registry, so the entry alone would not have fixed the measured path; the versioned window is therefore opened at `registerApp`, gated on a declared range so a code-registered manifest keeps today's refusals. HotCRM 4.0.1 declares `engines.protocol: ^17.7.0` (floor 17.7.0 < 18.0.0): verdict `converted-forward`, 152 + 3 + 1 sites across the chain. 3. **Which side a new install of a published version falls on.** A published artifact always declares its range (`os build` stamps it), so the marketplace install of 4.0.1 takes the load posture; publishing a new version goes through `os build`, whose parse refuses the shapes. `POST /api/v1/packages` (`protocol.installPackage`) and the `sys_packages` boot hydration call `registry.installPackage` directly and are NOT covered by this door (see acceptance notes). 4. **Named warning.** The structured `OS_METADATA_CONVERTED` notice of each entry, surfaced at the registration door as one `[ObjectQL] package '…' … ADR-0087 conversion 'CONVERSION-ID' converted N site(s) forward (first at …)` warn per (package, conversion), deduped per process, the wording family `MetadataPlugin`'s artifact door prints; the stored-row seams and artifact doors surface it through their existing channels. 5. **Artifact source.** `objectstack-ai/hotcrm` at tag `v4.0.1` (`1d7148bf2d3a`), cloned read-only and built with its own pinned toolchain (`@objectstack/cli` 17.7.0 → `dist/objectstack.json`, 2.4 MB, option-B, two package bodies). The repository commits no `dist/` and the GitHub release API answered 403 to this session, so the artifact was built, not downloaded. The fixture is that artifact trimmed to its `manifest` and each body's manifest fields plus the FULL `pages` collection (8 pages, byte-identical). Not a hand-made shape. 6. **Ablations.** (a) D2 entry: the `{ conversion: pageIsDefaultOffRecordPageRemoved, order: 67 }` line deleted from step 18 (`scripts/ablation-replace.mjs --delete --hold`, anchor 1 → 0 on disk), spec rebuilt: the boot test 4 of 8 red (the same four cases as (b)), the 4 refusal controls green; restored with `git checkout HEAD -- PATH`, blob `2e91e262` equals `HEAD`, anchor 1 again, spec rebuilt, `ablation-dist-preflight.mjs @objectstack/spec 'page-is-default-off-record-page-removed'` → present in 8 built files, 8 of 8 green. The mutated leg's `--absent` preflight exited 4: the id string is also prose in the D3 entry this PR edits, so `dist/migrations/index.{js,mjs}` still carried it (2 hits, both migrations) while the 6 conversion-dist files had lost it — the dist reading of the mutation is the 8 → 2 drop, read off the two preflight outputs. (b) Registration door (`engine.ts`, the declared-range guard replaced by an unconditional return, source resolved from `src`): 4 of 8 cases red (`× registers both package bodies …`, `× … registers through manifest.register …`, `× … without a second warning …`, `× an artifact declaring ^17.7.0 …`), the 4 refusal controls green; restored, blob `d45112dd` equals `HEAD`, `git diff HEAD` empty. ## The other load-time refusals of the same commit (reviewed, measured) - **`template: 'full-width'`** → D2 `page-template-full-width-to-default`. objectui at the pin `daeb77fc84dd` maps `'default'` and `'full-width'` to the one `FullWidthTemplate` component (`renderers/layout/page.tsx`, `TEMPLATE_REGISTRY`), so the re-spelling draws the page exactly as before. HotCRM 4.0.1 ships three such pages. - **An unregistered template name** (`centered`, `utility-bar` in 4.0.1): the renderer fell back to the page-type layout (`resolveTemplate` → `null` → `HomePageLayout` / `AppPageLayout` / `UtilityPageLayout` = `RegionLayout(regions, pageType)`), which is the same layout `FullWidthTemplate` draws, differing only in the `data-page-layout` attribute's source. Deliberately NOT converted: "any string outside the set" is not a shape a conversion positively recognizes (the module's own rule), so a D2 entry would rewrite shapes it cannot name. It stays with the parse; the registry never judged it, so the measured chain is unaffected. - **A region the template does not place** (`left_sidebar` / `right_sidebar` on 4.0.1's `three-column` home page): the renderer did render them, stacked under `main` (`getRemainingRegions` → `extras`), so dropping them loses content and renaming them (`sidebar` / `aside`) moves them beside `main`: no lossless rewrite exists, and which slot was meant is the author's call (D3 `page-template-closed-region-unplaced-refused`). Not converted; same reach as above. ## Acceptance - a kernel built from HotCRM 4.0.1's published bundle boots, with the named warning: `registry-installed-package-retired-page-declarations.test.ts` › "a kernel built from HotCRM 4.0.1's published bundle boots, with the named warning" (both bodies at the engine door with the artifact's package list; the measured package alone through `manifest.register`; dedupe on re-registration); - the same page saved or published through the authoring door is still refused: `page-default-and-region-placement.test.ts` (parse, `defineStack`), `default-record-page-save-door.test.ts` (save door, untouched), and the CONTROL block of the new file (range-less body, newer-than-every-retirement body, the item seam); - an ablation of the load-time path turns the boot red: §Readings 6. Through the strict-parse artifact doors (`os start --artifact`, `createStandaloneStack`), HotCRM 4.0.1 is refused regardless of this PR, on three `controlled_by_parent` objects with no required `master_detail` (ADR-0055 §1, a D3 semantic refusal older than #23708) and on the two shapes above with no lossless rewrite. Measured by parsing the built artifact after the full chain: 11 issues, 8 on pages, 3 on `sharingModel`. That door was not the measured failure and is not widened here. ## Acceptance notes (observed, not changed here) - `services/service-package` boot hydration (`registry.installPackage(rec.manifest)` from `sys_packages`) and `metadata-protocol`'s `protocol.installPackage` (`POST /api/v1/packages`) call the registry's package door directly, bypassing `registerApp`: a stored package record carrying a misplaced default still meets `MisplacedDefaultPageError` there (the hydration catches it, so the boot continues without that package). Seam: `spec:ADR-0087 addendum 2026-07-31 (every rehydration seam replays the chain)` → `runtime:packages/services/service-package/src/index.ts hydration loop`, consumer: the registry. Reported for the seat, not fixed in this PR. - `cloud-connection`'s marketplace install hands the fetched bundle to `manifest.register` raw (now converted at that door) but its side readers (`bindArtifactHandlers`, `applySideEffects`) read the unconverted ledger entry; pre-existing, not a page concern. - The registry's two throws stay in place as the authority for a body that survives conversion (no declared range, or a floor past every retirement). ## Verification - Build: `pnpm --filter '@objectstack/cloud-connection...' build` (closure, exit 0), `pnpm --filter @objectstack/spec --filter @objectstack/metadata-core build` (exit 0), under `os-verify-lock`. - Tests (all under the lock, exit codes captured before any pipe): spec `src/conversions src/migrations src/ui/page-default-and-region-placement.test.ts src/ui/page.test.ts src/ui/default-record-page.test.ts src/stack-default-record-page.test.ts src/ui/object-refinement-check-exports.test.ts` + `scripts/conversions-major18-merge.test.ts` → green after the repin (1157 tests); metadata-core `pnpm test` → exit 0; objectql `registry-installed-package-retired-page-declarations`, `registry-default-record-page`, `artifact-load-path`, `registry-artifact-co-ownership`, `registry-security-catalog-namespace`, `protocol-install-package`, `manifest-service-residual-objects` → exit 0; metadata `plugin-artifact-forward-conversion` → exit 0; runtime `app-plugin-artifact-forward-conversion`, `standalone-stack-option-b-forward-conversion`, `load-artifact-bundle.granted-permissions` → exit 0; cloud-connection `marketplace-install-local-bundle`, `-hot-loaded` → exit 0. - Typecheck: `@objectstack/spec`, `@objectstack/metadata-core`, `@objectstack/objectql` → exit 0. - `pnpm --filter @objectstack/spec check:generated` → green after `gen:liveness-counts` (0 shards rewritten). `scripts/check-nul-bytes.mjs` → OK. - Gates: `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived 94 families; each run with its exit code captured before any pipe; `--ran` reconciliation: 94 derived, 92 run (exit 0 except `check-engine-split-ratio.mjs --days 90`, exit 2: shallow clone, refuses the 90-day window), 2 NOT-MEASURED on a recorded exit 3 (prerequisite): `check-plugin-teardown-shape.mjs --self-test` (needs an unshallow clone) and `pnpm check:dual-build-cjs-loads` (needs every package's `dist/`; only the cloud-connection closure was built here). Both are CI's on the PR. `check:adr-0087-registration --base origin/main`, `check:changeset-no-major`, `check:empty-changeset`, `check:spec-changes`, `check:upgrade-guide`, `check:liveness`, `check:api-surface` (rerun on the rebuilt dist: surface unchanged), `check:docs`, `check:authorable-surface`: exit 0 Readings taken at `9fad187b` (`git rev-parse --short HEAD` after the last commit; the gate union and the ablations ran on that tree). --- _Generated by [Claude Code](https://claude.ai/code/session_016MPJ3Y2xUTT42wq9ZiK3MR)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent b25d98a commit d050a8f

17 files changed

Lines changed: 2279 additions & 44 deletions
Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
---
2+
'@objectstack/spec': minor
3+
'@objectstack/objectql': minor
4+
'@objectstack/metadata-core': minor
5+
---
6+
7+
fix(spec,objectql)!: an installed or stored page carrying a declaration spec 18 retires converts at load instead of stopping the kernel build — `isDefault` off a bound record page is deleted and `template: 'full-width'` re-spelled `default`, each with a named notice; the authoring, save and publish doors keep refusing
8+
9+
Clause-②: yes (widening)
10+
11+
**What changes.** Protocol 18 refused `isDefault: true` on a page that is not a record page bound to an object, and closed the page `template` set (retiring `full-width`), at every door — the page parse (`definePage`, `defineStack`, so `os validate` / `os build`, and the metadata save door) and the kernel's package registration (`SchemaRegistry.installPackage` / `registerItem`, `MisplacedDefaultPageError`). The registration refusal also fired when a kernel was built from a package that was installed BEFORE the retirement: every hosted environment holding such a package failed its kernel build, with no remedy short of upgrading the package in each one. Data at rest now converts, and only data at rest:
12+
13+
| Door | Before | Now |
14+
| :--- | :--- | :--- |
15+
| `definePage` / `defineStack` (`os validate`, `os build`), the metadata save door, package publish | refused at `isDefault` / `template` | **unchanged** — refused with the same sentence |
16+
| a `sys_metadata` page row read back | served with `_diagnostics.valid: false` | converted on read: the flag deleted, `full-width` → `default`, `OS_METADATA_CONVERTED` notice |
17+
| a compiled artifact inside the window its `engines.protocol` opens | the artifact door refused the artifact | converted before the strict parse, as every retired entry is |
18+
| a package body registered through `manifest.register` / `ObjectQL.registerApp` that declares `engines.protocol` (an installed package, a marketplace install) | `MisplacedDefaultPageError`, kernel build stopped | converted before anything registers, with one `warn` per (package, conversion) naming the conversion id and the first site; the body is left unchanged on disk |
19+
| the same body with NO declared range (a manifest a plugin builds in code), or declaring a floor newer than every retirement | refused | **unchanged** — refused |
20+
21+
**The two ADR-0087 D2 entries**, both `retiredFromLoadPath` from the day they land (the authoring funnel never replays them):
22+
23+
- `page-is-default-off-record-page-removed` (`page.isDefault`): deletes `isDefault: true` from exactly the pages `misplacedDefaultPageRefusal` refuses — a `home`, `app`, `utility` or `list` page, or a record page with no `object`. Lossless: nothing read the flag there (an app opens on its first navigation item). A default record page bound to an object keeps its flag.
24+
- `page-template-full-width-to-default` (`page.template`): `'full-width'` → `'default'`. Lossless: the console drew both names with one template component. An unregistered template name (`centered`, `two-column`) and a region the template does not place have no lossless rewrite and are NOT converted — they stay with the parse, and the D3 entries `page-is-default-off-record-page-refused` / `page-template-closed-region-unplaced-refused` name the hand edits.
25+
26+
**`@objectstack/objectql`** — `ObjectQL.registerApp` (and the `manifest` service, for an artifact carrying `packages[]`) replays the versioned forward conversion (`applyArtifactForwardConversions`, `@objectstack/metadata-core`) over a package body that declares its protocol range, before `installPackage`; the converted copy is what the package record and every registered item carry. Public: `ObjectQL.convertPackageBodyForward(body)`.
27+
28+
**`@objectstack/metadata-core`** — `applyArtifactForwardConversions` reads the declared range off a FLAT package body (the manifest half beside its collections, no `manifest` key) when the definition carries no `manifest`; a definition with a `manifest` object is read there, as before.
29+
30+
Migration: none for authors — the authoring doors refuse exactly what they refused. An installed package converts again at every registration until it is rebuilt (`os build`) and re-published with tooling from a release that ships the retirement; the boot log's warn names the conversion and the site.
31+
32+
<!-- adr-0087: registered page-is-default-off-record-page-removed, page-template-full-width-to-default -->

‎packages/metadata-core/src/artifact-forward-conversion.test.ts‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -223,6 +223,30 @@ describe('applyArtifactForwardConversions — the versioned window (#12772)', ()
223223
expect(grant).not.toHaveProperty('allowRestore');
224224
});
225225

226+
it('reads a FLAT package body\'s range off the body itself — the shape `registerApp` and the marketplace ledger hold', () => {
227+
// The same bytes as `legacyPermissionDefinition`, flattened the way
228+
// `normalizeBundle` and a package body inside `packages[]` spell them: the
229+
// manifest half beside the collections, no `manifest` key.
230+
const nested = legacyPermissionDefinition('^17.1.0');
231+
const { manifest, ...collections } = nested;
232+
const flat = { ...manifest, ...collections };
233+
const fromFlat = applyArtifactForwardConversions(flat, { runtimeSpecVersion: '17.2.0' });
234+
const fromNested = applyArtifactForwardConversions(nested, { runtimeSpecVersion: '17.2.0' });
235+
expect(fromFlat.verdict).toBe('converted-forward');
236+
expect(fromFlat.authoredFloor).toBe('17.1.0');
237+
expect([fromFlat.verdict, fromFlat.authoredFloor]).toEqual([fromNested.verdict, fromNested.authoredFloor]);
238+
expect((fromFlat.definition as typeof flat).permissions[0]!.objects.crm_ticket).not.toHaveProperty('allowPurge');
239+
// A flat body authored at a current-or-newer floor is NOT the undeclared
240+
// amnesty: the strict parse stays its authority.
241+
const current = applyArtifactForwardConversions({ ...flat, engines: { protocol: '>=99.0.0' } }, { runtimeSpecVersion: '17.2.0' });
242+
expect(current.verdict).toBe('authored-current');
243+
expect(current.authoredFloor).toBe('99.0.0');
244+
// A `manifest` object, when present, is still the only place the range is read.
245+
const both = applyArtifactForwardConversions({ ...nested, engines: { protocol: '>=99.0.0' } }, { runtimeSpecVersion: '17.2.0' });
246+
expect(both.verdict).toBe('converted-forward');
247+
expect(both.authoredFloor).toBe('17.1.0');
248+
});
249+
226250
it('closes the window when the runtime spec version cannot be resolved — amnesty needs positive version evidence', () => {
227251
const def = legacyPermissionDefinition('^17.1.0');
228252
const result = applyArtifactForwardConversions(def, { runtimeSpecVersion: null });

‎packages/metadata-core/src/artifact-forward-conversion.ts‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,9 @@
8989
* the next release — so the change that re-legalizes a retired key decides
9090
* that entry's fate at this door, where the default-flip refusal below
9191
* ({@link DEFAULT_FLIPS_NOT_REPLAYED_HERE}) is the existing mechanism.
92+
* - **A flat package body** — the manifest half beside the collections, with
93+
* no `manifest` key (the shape `registerApp` and the marketplace ledger
94+
* hold) — declares its range on itself, and the window is read there.
9295
* - **No declared range** → replay the full chain. Same posture as the
9396
* protocol handshake (which grandfathers range-less packages with a warning,
9497
* ADR-0087 "never false-reject") and as the stored-row pass (whose rows
@@ -498,11 +501,18 @@ export function applyArtifactForwardConversions<T>(
498501
return { definition, verdict: 'not-an-object', authoredFloor: null, runtimeSpecVersion, notices: [], replayedRetirements: [] };
499502
}
500503

504+
// WHERE the declared range is read from. A stack definition carries it on
505+
// its `manifest`; a package BODY carries it on itself — the manifest half of
506+
// `AssembledPackageBodySchema` sits beside the collections, which is the
507+
// shape a flat `manifest.register` / `registerApp` payload and a marketplace
508+
// ledger entry have, and the shape `packages[i].manifest` has inside an
509+
// option-B artifact (ADR-0130 D4). A definition that carries a `manifest`
510+
// object is read there and nowhere else, as before; only a definition with
511+
// no `manifest` is read on its own keys, so a flat body's window is the
512+
// body's own, versioned, instead of the undeclared amnesty.
501513
const manifest = (definition as { manifest?: unknown }).manifest;
502-
const declared =
503-
manifest && typeof manifest === 'object'
504-
? resolveDeclaredRange(manifest as ProtocolHandshakeManifest)
505-
: null;
514+
const rangeCarrier: unknown = manifest && typeof manifest === 'object' && !Array.isArray(manifest) ? manifest : definition;
515+
const declared = resolveDeclaredRange(rangeCarrier as ProtocolHandshakeManifest);
506516
const floor = declared ? parseRangeFloor(declared.range) : null;
507517
const authoredFloor = floor ? floor.join('.') : null;
508518

0 commit comments

Comments
 (0)