diff --git a/.changeset/15206-boot-report-carry-overs.md b/.changeset/15206-boot-report-carry-overs.md new file mode 100644 index 00000000000..a6bbff855b2 --- /dev/null +++ b/.changeset/15206-boot-report-carry-overs.md @@ -0,0 +1,10 @@ +--- +'@objectstack/metadata-protocol': patch +--- + +The legacy-organization boot report reports each of its reads on its own, and a stored row under a code-defined datasource name is named once at boot + +Clause-②: no + +- `[metadata_org_scoped_unserved]` reads `sys_metadata` and counts the legacy rows of `sys_metadata_commit`, `sys_metadata_history` and `sys_metadata_audit` as four separate terms. A read that fails for any reason but a missing table now costs only its own term: the line names it NOT MEASURED, with the reason, and still reports the other terms. Before, one failing read silenced the whole line. +- `[metadata_sealed_overlay_unserved]` no longer lists an environment row stored under the name of a datasource the host defines in code. The datasource restore's boot warning already names that row, with its own remedy, so the row was named twice. No read serves the row either way, and nothing is deleted or rewritten. diff --git a/.changeset/15206-reads-environment-only.md b/.changeset/15206-reads-environment-only.md index 7e409114fa6..bdb377a53f6 100644 --- a/.changeset/15206-reads-environment-only.md +++ b/.changeset/15206-reads-environment-only.md @@ -23,7 +23,7 @@ ADR-0131 D6 retires the per-organization overlay axis. The doors carry no organi - `overlayScope` on the layered read is `'env'` or `null`; `'org'` is gone. - `listDrafts` lists the environment's drafts only, and its rows carry no `organizationId`. - `listCommits` lists the environment's commits only, for every caller: a legacy organization commit is not shown, so an operator on the packages door no longer sees every organization's legacy commits, and a rollback plan never contains one. -- [Triage ruling Q1 → C] An environment row that overlays an item a managed package ships, on a type sealed against overlays (any type whose registry entry admits no overlay — a managed flow, action, hook, object or datasource, for example — written through the `OS_METADATA_WRITABLE` hatch before the seal), is not served: the package's definition is. An `object` such row is not loaded at boot either, so the engine keeps the packaged schema. A stored fork of a code-declared permission set keeps its own ruling (detection reading and Discard Overlay) and is still served. +- [Triage ruling Q1 → C] An environment row that overlays an item a managed package ships, on a type sealed against overlays (any type whose registry entry admits no overlay — a managed flow, action, hook, object or datasource, for example — written through the `OS_METADATA_WRITABLE` hatch before the seal), is not served: the package's definition is. An `object` such row is not loaded at boot either, so the engine keeps the packaged schema. A stored fork of a code-declared permission set keeps its own ruling instead: the reads do not decline it, a cold boot refuses a deployment that holds one (ADR-0048), and the offline `os migrate security-catalog-overlays` removes it. - `@objectstack/objectql` binds the environment's authored hook and action rows only; a legacy organization-scoped hook or action row is no longer bound. - Boot names both populations, once each: `[metadata_org_scoped_unserved]` lists every legacy organization row per type (active and draft), plus the legacy rows of `sys_metadata_commit`, `sys_metadata_history` and `sys_metadata_audit`, and names the v18 migration ceremony (`os migrate`, ADR-0131 D10) that carries them; `[metadata_sealed_overlay_unserved]` lists the sealed overlays per type with the two remedies. Nothing is deleted or rewritten. - The anonymous public-form doors keep reading the Default Organization's legacy `view` layer, fail-closed, until that ceremony (triage ruling Q3 A) — through the protocol-internal `legacyFormOrganizationId` key, which no spec request declares and which the protocol honours for `view` alone. @@ -36,9 +36,11 @@ ADR-0131 D6 retires the per-organization overlay axis. The doors carry no organi | `organizationId` on a `GetMetaItems` / `GetMetaItem` / `GetMetaItemLayered` / `AuditMetaItem` / `HistoryMetaItem` / `GetMetaItemCached` request (`@objectstack/spec`) | drop it: the read is the environment's. The key is stripped at a spec parse (the schemas are not strict) and no longer type-checks | | `organizationId` on a `listDrafts` / `listCommits` / `diffMetaItem` / `getMetaDiagnostics` / `findReferencesToMeta` request (`@objectstack/metadata-protocol`) | drop it | | `ListDraftsResponse` draft `organizationId` | gone: every listed draft is the environment's | +| `SysMetadataRepository.listDrafts` row `organizationId` (`@objectstack/metadata-protocol`) | gone: every listed row is an environment draft, whichever repository asks | | `overlayScope: 'org'` (`GetMetaItemLayeredResponse`) | only `'env'` or `null` | | `declaresOrgOverride`, `organizationIdForMetaRead` (`@objectstack/metadata-core`) | removed — no read takes an organization. Read `allowOrgOverride` off `DEFAULT_METADATA_TYPE_REGISTRY` where a type's overlay channel is the question | | `MetadataAuthoringGateContext.organizationId` (`@objectstack/metadata-protocol`) | removed — no write is organization-scoped, and no gate read it | +| `organizationId` on `projectPermissionMutation`'s `evt` (`@objectstack/plugin-security`) | drop it: the projection reads the environment → code layered read, whatever organization the event names | | a legacy organization-scoped `sys_metadata` row a deployment relied on | re-save the item in Studio to make it environment-wide now, or wait for the v18 migration ceremony, which promotes it | | an environment overlay of a managed item on a sealed type (flow, action, hook, object, datasource …) | re-express the change as a new item under a new name (a linkage-free clone), then delete the stored row; or delete the row | diff --git a/content/docs/concepts/metadata-lifecycle.mdx b/content/docs/concepts/metadata-lifecycle.mdx index f6123e63565..54cf97bbc50 100644 --- a/content/docs/concepts/metadata-lifecycle.mdx +++ b/content/docs/concepts/metadata-lifecycle.mdx @@ -144,7 +144,7 @@ A permission set and a position also have a switch, outside the refusal: `POST / The switches are operator-only where one install serves several organizations. The refusal names the same path whichever layer answers it: `403 NOT_OVERRIDABLE` for a write that names no base, and `403 ITEM_LOCKED` for one that names the read-only package. Which writes are refused, and with which code, is unchanged. -An environment row stored over a managed item of a type sealed against overlays (a flow, action, hook, object or datasource, for example — any type whose registry entry admits no overlay) before the seal — written through the `OS_METADATA_WRITABLE` hatch — is **not served** (ADR-0131 D6): every read, and boot, takes the package's definition, and the row stays at rest, untouched. Boot names each such row, per type (`[metadata_sealed_overlay_unserved]`), with the two remedies: re-express the change as a new item under a new name, or delete the stored row. A stored fork of a code-declared permission set is the one exception: the deployment does not boot while it holds one (ADR-0048), and `os migrate security-catalog-overlays` lists and, with `--apply`, deletes it. +An environment row stored over a managed item of a type sealed against overlays (a flow, action, hook, object or datasource, for example — any type whose registry entry admits no overlay) before the seal — written through the `OS_METADATA_WRITABLE` hatch — is **not served** (ADR-0131 D6): every read, and boot, takes the package's definition, and the row stays at rest, untouched. Boot names each such row, per type (`[metadata_sealed_overlay_unserved]`), with the two remedies: re-express the change as a new item under a new name, or delete the stored row. A row under the name of a datasource the host defines in code is named once, by the datasource restore's boot warning, with that warning's remedy. A stored fork of a code-declared permission set is the one exception: the deployment does not boot while it holds one (ADR-0048), and `os migrate security-catalog-overlays` lists and, with `--apply`, deletes it. --- diff --git a/packages/metadata-protocol/src/protocol.org-scoped-cold-boot-audit.test.ts b/packages/metadata-protocol/src/protocol.org-scoped-cold-boot-audit.test.ts index 54a5249e3cf..0afd0f1b137 100644 --- a/packages/metadata-protocol/src/protocol.org-scoped-cold-boot-audit.test.ts +++ b/packages/metadata-protocol/src/protocol.org-scoped-cold-boot-audit.test.ts @@ -249,6 +249,38 @@ describe('[#6190, ADR-0131 D6] cold boot names every legacy organization-scoped expect(call).toBeGreaterThan(1); expect(result).toMatchObject({ loaded: 1, errors: 0, storeUnavailable: false }); expect(warns.filter((w) => w.includes('DB hydration skipped'))).toEqual([]); - expect(warns.filter((w) => w.includes(AUDIT))).toEqual([]); + // [#15206] …and the failed read is said, not swallowed: its term is + // NOT MEASURED, with the reason. + const audit = warns.filter((w) => w.includes(AUDIT)); + expect(audit).toHaveLength(1); + expect(audit[0]).toContain('NOT MEASURED'); + expect(audit[0]).toContain('sys_metadata (probe exploded)'); + }); + + it('[#15206] a read that fails costs its own term only: the other terms are reported as measured', async () => { + const { engine } = makeEngine([ + row({ type: 'flow', name: 'org_sweep', organization_id: 'org_a' }), + ], { + ledgers: { + sys_metadata_commit: [{ organization_id: 'org_a' }], + sys_metadata_audit: [{ organization_id: 'org_b' }], + }, + }); + const inner = engine.count; + engine.count = async (t: string, q: any) => { + if (t === 'sys_metadata_history') throw new Error('history count refused'); + return inner(t, q); + }; + + const { result, warns } = await bootAndCapture(engine); + + expect(result).toMatchObject({ errors: 0, storeUnavailable: false }); + const line = warns.find((w) => w.includes(AUDIT)); + expect(line, `no ${AUDIT} line in: ${JSON.stringify(warns)}`).toBeDefined(); + expect(line).toContain('flow×1 (org_sweep@org_a)'); + expect(line).toContain('sys_metadata_commit×1, sys_metadata_audit×1'); + expect(line).toContain('NOT MEASURED'); + expect(line).toContain('sys_metadata_history (history count refused)'); + expect(line).not.toContain('sys_metadata_history×'); }); }); diff --git a/packages/metadata-protocol/src/protocol.reads-environment-only.test.ts b/packages/metadata-protocol/src/protocol.reads-environment-only.test.ts index cbd537c2d78..d2780f05fb2 100644 --- a/packages/metadata-protocol/src/protocol.reads-environment-only.test.ts +++ b/packages/metadata-protocol/src/protocol.reads-environment-only.test.ts @@ -352,6 +352,29 @@ describe('§3 the boot report names both populations, per type, with their remed const { warns } = await boot({ sys_metadata: [metaRow('view', 'env_grid', null, { label: 'env' })] }); expect(warns.filter((w) => w.includes('_unserved]'))).toEqual([]); }); + + it('[#15206] a row under a datasource name the host defines in code is left to the datasource restore\'s warning; a package-declared one is still listed', async () => { + const warns: string[] = []; + vi.spyOn(console, 'warn').mockImplementation((...a: unknown[]) => { warns.push(a.map(String).join(' ')); }); + const { protocol } = harness({ + sys_metadata: [ + metaRow('datasource', 'host_ds', null, { label: 'stored' }), + metaRow('datasource', 'pkg_ds', null, { label: 'stored' }), + ], + }); + protocol.getServicesRegistry = () => new Map([['code-datasource-names', new Set(['host_ds'])]]); + protocol.engine.registry.getAllPackages = () => [{ manifest: { datasources: [{ name: 'pkg_ds' }] } }]; + // Both rows are still declined at read; only the boot line's membership moves. + expect(protocol.declinesStoredRow('datasource', 'host_ds')).toBe(true); + expect(protocol.declinesStoredRow('datasource', 'pkg_ds')).toBe(true); + + await protocol.loadMetaFromDb(); + + const sealed = warns.filter((w) => w.includes('[metadata_sealed_overlay_unserved]')); + expect(sealed).toHaveLength(1); + expect(sealed[0]).toContain('datasource×1 (pkg_ds)'); + expect(sealed[0]).not.toContain('host_ds'); + }); }); describe('§4 [triage ruling Q3 A] the anonymous form doors\' legacy layer is the one read left that reaches a legacy organization', () => { diff --git a/packages/metadata-protocol/src/protocol.ts b/packages/metadata-protocol/src/protocol.ts index b40fc873428..8fe8448f01c 100644 --- a/packages/metadata-protocol/src/protocol.ts +++ b/packages/metadata-protocol/src/protocol.ts @@ -16163,7 +16163,8 @@ export class ObjectStackProtocolImplementation implements * environment authoring, governed by `allowRuntimeCreate` and the hatch as * before; * - the removal of a stored overlay row that merges at read (#6960) — the - * removal door's own carve-out, which restores the managed definition. + * removal door's own carve-out, which removes no managed definition + * (and changes what is served for `permission` alone). * * ⛔ The hatch is not consulted: `OS_METADATA_WRITABLE` unlocks a TYPE, and a * type-level unlock says nothing about an item a managed package ships. @@ -16327,8 +16328,7 @@ export class ObjectStackProtocolImplementation implements */ private isDeclaredCodeDatasource(type: string, name: string): boolean { if ((PLURAL_TO_SINGULAR[type] ?? type) !== 'datasource') return false; - const hostCode = this.getServicesRegistry?.().get('code-datasource-names') as { has?: unknown } | undefined; - if (typeof hostCode?.has === 'function' && (hostCode as { has(n: string): boolean }).has(name)) return true; + if (this.isHostCodeDatasource(type, name)) return true; const registry = (this.engine as any)?.registry; if (typeof registry?.getAllPackages !== 'function') return false; for (const record of registry.getAllPackages() as unknown[]) { @@ -16341,6 +16341,19 @@ export class ObjectStackProtocolImplementation implements return false; } + /** + * [#21944] The host's code-datasource set alone — the first half of + * {@link isDeclaredCodeDatasource}, and exactly the names the + * datasource-admin boot restore reads to keep a stored row off a code + * definition and to name it in a warning ({@link loadMetaFromDb} leaves + * those rows to that warning). + */ + private isHostCodeDatasource(type: string, name: string): boolean { + if ((PLURAL_TO_SINGULAR[type] ?? type) !== 'datasource') return false; + const hostCode = this.getServicesRegistry?.().get('code-datasource-names') as { has?: unknown } | undefined; + return typeof hostCode?.has === 'function' && (hostCode as { has(n: string): boolean }).has(name); + } + /** * [#20679, ADR-0126 §2] The `/meta` door's LOCKED-BASE verdict on an item * that already exists, as a VALUE — for a second door that writes or @@ -16625,8 +16638,10 @@ export class ObjectStackProtocolImplementation implements * DELETE removes it as repair. * - [ADR-0131 D6, triage ruling Q1 → C] a SEALED managed item * ({@link isUnservedSealedOverlay}): an environment overlay row on an - * item a managed package ships, of a type whose registry entry opens no - * overlay channel (a managed flow, action, hook or object), is in no + * item a managed package ships, of ANY type whose registry entry opens + * no overlay channel ({@link isSealedManagedItem}: every type but the + * five regime-O ones, so a merge-at-read `page` or `app` as much as a + * flow or an object), `permission` excepted, is in no * regime D6 recognises — the package definition wins. Such rows were * written through the `OS_METADATA_WRITABLE` hatch before the seal; they * stay at rest, untouched, and the boot report names them per type with @@ -17051,8 +17066,11 @@ export class ObjectStackProtocolImplementation implements * [ADR-0131 D6] Managed content is sealed against removal as against edits: * the predicate is {@link isSealedManagedItem}, the save door's, so the * `OS_METADATA_WRITABLE` hatch no longer opens the removal either. The - * #6960 carve-out stands — removing a row that merges at read restores the - * managed definition, it does not remove it. The sentence names the managed + * #6960 carve-out stands — removing a row of a type that merges at read + * removes no managed definition. Only for `permission`, whose stored fork + * is still served ({@link isUnservedSealedOverlay}), does it change what is + * served; every other such row is served by no read already + * ({@link declinesStoredRow}). The sentence names the managed * package ({@link managedItemSealedSentence}). */ private refusePackagedBaseRemoval(request: { type: string; name: string }): void { @@ -26560,7 +26578,13 @@ export class ObjectStackProtocolImplementation implements // Normalize DB type to singular (DB may store legacy plural forms) const normalizedType = PLURAL_TO_SINGULAR[record.type] ?? record.type; if (this.isUnservedSealedOverlay(normalizedType, String(record.name))) { - sealedOverlays.push({ type: normalizedType, name: String(record.name) }); + // [#15206] A row under a datasource name the host + // defines in code is named once, by the + // datasource-admin boot restore's warning, which + // carries its own remedy — not here as well. + if (!this.isHostCodeDatasource(normalizedType, String(record.name))) { + sealedOverlays.push({ type: normalizedType, name: String(record.name) }); + } if (normalizedType === 'object') continue; } const data = this.convertStoredItem( @@ -26853,6 +26877,10 @@ export class ObjectStackProtocolImplementation implements * a false line. * - **One aggregated line.** Counts per type plus a capped sample of * names, so a tenant with a thousand such rows costs one line. + * - **Each read is its own term.** A read that fails for any reason but + * a missing table (an unprovisioned ledger holds no legacy row) costs + * its own term only: the line names it NOT MEASURED, with the reason, + * beside the terms that were measured. * - **Best-effort, and non-fatal by construction.** A diagnostic must * never be the reason a boot fails, so its own catch swallows: the * caller's outer catch classifies REAL hydration outages @@ -26866,7 +26894,7 @@ export class ObjectStackProtocolImplementation implements // organization: neither NULL nor the empty string. const LEGACY = { $and: [{ organization_id: { $null: false } }, { organization_id: { $ne: '' } }] }; // An unprovisioned ledger holds no legacy row; any other failure is - // this diagnostic's own, and stays here. + // this diagnostic's own, and its term reports it (below). const orNoTable = async (object: string, read: () => Promise, empty: T): Promise => { try { return await read(); @@ -26876,20 +26904,35 @@ export class ObjectStackProtocolImplementation implements } }; + // [#15206] Each of the four reads is its own term: one that fails + // costs its own term, never the others, and is printed NOT + // MEASURED with its reason — never a zero, never a silent line. + const unmeasured: string[] = []; + const isolated = async (object: string, read: () => Promise): Promise => { + try { + await read(); + } catch (error) { + unmeasured.push(`${object} (${error instanceof Error ? error.message : String(error)})`); + } + }; + // Only the columns the line prints — never a row's body. The // three ledgers below are counted, never read: this runs on every // boot, and the populations it names can be large. - const rows = (await orNoTable('sys_metadata', async () => ((await this.engine.find('sys_metadata', { - where: { state: { $in: ['active', 'draft'] }, ...LEGACY }, - fields: ['type', 'name', 'organization_id', 'state'], - context: { isSystem: true }, - })) ?? []) as any[], [] as any[])) - // A driver that cannot lower the predicate hands back a - // superset; the names this line prints are re-checked here. - .filter((row) => { - const org = (row as { organization_id?: string | null }).organization_id; - return org !== null && org !== undefined && org !== ''; - }); + let rows: any[] = []; + await isolated('sys_metadata', async () => { + rows = (await orNoTable('sys_metadata', async () => ((await this.engine.find('sys_metadata', { + where: { state: { $in: ['active', 'draft'] }, ...LEGACY }, + fields: ['type', 'name', 'organization_id', 'state'], + context: { isSystem: true }, + })) ?? []) as any[], [] as any[])) + // A driver that cannot lower the predicate hands back a + // superset; the names this line prints are re-checked here. + .filter((row) => { + const org = (row as { organization_id?: string | null }).organization_id; + return org !== null && org !== undefined && org !== ''; + }); + }); const counts = new Map(); const samples = new Map(); for (const row of rows) { @@ -26902,13 +26945,15 @@ export class ObjectStackProtocolImplementation implements } const ledgers: string[] = []; for (const ledger of ['sys_metadata_commit', 'sys_metadata_history', 'sys_metadata_audit']) { - const n = await orNoTable(ledger, () => this.engine.count(ledger, { - where: LEGACY, - context: { isSystem: true }, - }), 0); - if (n > 0) ledgers.push(`${ledger}×${n}`); + await isolated(ledger, async () => { + const n = await orNoTable(ledger, () => this.engine.count(ledger, { + where: LEGACY, + context: { isSystem: true }, + }), 0); + if (n > 0) ledgers.push(`${ledger}×${n}`); + }); } - if (rows.length === 0 && ledgers.length === 0) return; + if (rows.length === 0 && ledgers.length === 0 && unmeasured.length === 0) return; const detail = Array.from(counts.entries()) .map(([type, count]) => { @@ -26929,6 +26974,9 @@ export class ObjectStackProtocolImplementation implements ? `Legacy organization-scoped ledger rows, which the commit timeline, history, diff and audit ` + `reads no longer show: ${ledgers.join(', ')}. ` : '') + + (unmeasured.length > 0 + ? `NOT MEASURED, so this account may be incomplete — the read failed: ${unmeasured.join(', ')}. ` + : '') + `Nothing is deleted or rewritten: the v18 migration ceremony (\`os migrate\`, ADR-0131 D10) ` + `carries these rows and names each one's fate — promoted to the environment layer, or reported ` + `when another organization holds the same name.`, @@ -26942,12 +26990,16 @@ export class ObjectStackProtocolImplementation implements /** * [ADR-0131 D6, triage ruling Q1 → C] Say out loud which stored - * ENVIRONMENT rows boot did not load because they overlay sealed managed + * ENVIRONMENT rows no read serves because they overlay sealed managed * content ({@link isUnservedSealedOverlay}): an overlay of an item a - * managed package ships, on a type whose registry entry opens no overlay - * channel — a managed flow, action, hook or object, written through the - * `OS_METADATA_WRITABLE` hatch before D6 sealed it. No read serves such a - * row: the package's definition wins. + * managed package ships, on ANY type whose registry entry opens no + * overlay channel ({@link isSealedManagedItem}: every type but the five + * regime-O ones, so a merge-at-read `page` or `app` as much as a flow or + * an object), `permission` excepted, written through the + * `OS_METADATA_WRITABLE` hatch before D6 sealed it. The package's + * definition wins; an `object` such row is not even loaded. A row under a + * datasource name the host defines in code is not listed: the + * datasource-admin boot restore names it, once ({@link isHostCodeDatasource}). * * Per type, a capped sample of names, and the two remedies the ruling * names. ⛔ Nothing here deletes or rewrites the row — its fate is the v18