From 0555b80a0a9bf5e75423857896a879edede527c3 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 12 Oct 2026 11:25:55 +0000 Subject: [PATCH 1/5] =?UTF-8?q?feat(cli):=20os=20migrate=20organization-ow?= =?UTF-8?q?nership=20--apply=20=E2=80=94=20the=20ADR-0131=20D10=20ceremony?= =?UTF-8?q?'s=20apply=20step=20and=20post-check?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The read-only plan gains its writing mode. --apply asks about the backup first (--backup-taken, or a yes at the prompt; refused non-interactively), then performs the fates in order, stopping at the first blocker: 1. attribution — the owners the plan's own walk derives (anchors, else the Default Organization under single), to a fixed point; an unattributable row is never assigned and stays listed by id; 2. authored catalog rows — permission sets and positions become environment metadata through the metadata door via core's row-to-definition mappers, with position bindings carried into permissionSets (single only); 3. the verified id-to-name rewrite of sys_user_permission_set.permission_set, then every grant verified to name a set the security catalog resolves (permission_set_id stays); 4. mirror deletion, gated on that verification read again before the first delete; only mirror rows no named category claims; 5. column drops through the schema-drift ceremony's drop_column entries. Every write is conditioned on the state it changes, and each finished step is checkpointed in sys_migration (adr-0131-organization-ownership), so an interrupted run resumes at its step. The post-check re-reads the plan; the ledger row's applied_at / verified_at are the ceremony's marker. The acceptance fixture carries every fate: effective access identical, with a positive control; mirrors gone; attributed rows stamped; the orphan row present and reported; row counts reconciling. Claude-Session: https://claude.ai/code/session_01DjKf52bcyXa3fbGW723phR Co-authored-by: Claude --- .../22937-cli-organization-ownership-apply.md | 25 + ...organization-ownership.apply-flags.test.ts | 68 ++ .../migrate/organization-ownership.ts | 294 +++++- .../organization-ownership-apply.test.ts | 464 +++++++++ .../src/utils/organization-ownership-apply.ts | 942 ++++++++++++++++++ .../src/utils/organization-ownership-plan.ts | 71 +- ...igrate.one-shot-family.integration.test.ts | 7 +- packages/cli/src/utils/schema-migrate.ts | 15 +- .../cli/test/json-stdout-purity.e2e.test.ts | 5 +- 9 files changed, 1863 insertions(+), 28 deletions(-) create mode 100644 .changeset/22937-cli-organization-ownership-apply.md create mode 100644 packages/cli/src/commands/migrate/organization-ownership.apply-flags.test.ts create mode 100644 packages/cli/src/utils/organization-ownership-apply.test.ts create mode 100644 packages/cli/src/utils/organization-ownership-apply.ts diff --git a/.changeset/22937-cli-organization-ownership-apply.md b/.changeset/22937-cli-organization-ownership-apply.md new file mode 100644 index 0000000000..4b2c410f71 --- /dev/null +++ b/.changeset/22937-cli-organization-ownership-apply.md @@ -0,0 +1,25 @@ +--- +'@objectstack/cli': minor +--- + +feat(cli): `os migrate organization-ownership --apply` — the v18 organization-ownership ceremony's apply step and post-check (ADR-0131 D10, ceremony items 2 to 4) + +Clause-②: yes (widening) + +The command that planned the ceremony now performs it. Without `--apply` nothing changes: the read-only plan, written to a file. With `--apply`: + +- **The backup comes first.** It is the operator's act: pass `--backup-taken`, or answer yes at the prompt. A non-interactive run without the flag is refused before the database is opened. +- **The fates run in order**, each step stopping the run where it meets a blocker: + 1. **Attribution** — every NULL `organization_id` row the plan derives an owner for takes it (a parent, subject or holder anchor; under `single`, the Default Organization). A row with no derivable owner is never assigned one: it stays NULL and is listed by id. + 2. **Authored catalog rows** — permission sets and positions an organization authored in Setup become environment metadata through the metadata door (under `single`; under a wall they are reported), with each position's binding rows carried into its `permissionSets`. + 3. **The verified id→name rewrite** — every `sys_user_permission_set` grant names its set in `permission_set`, then every grant is verified to name a set the security catalog resolves. `permission_set_id` stays. + 4. **Mirror deletion** — only after that verification, read again just before the first delete: the seeded catalog copies (per organization and NULL residue), the binding rows, seeded capabilities and seeded templates. Authored, customized and organization-stamped rows stay. + 5. **Column drops** — `organization_id` leaves the deployment-level objects through the schema-drift ceremony (`os migrate apply`'s machinery), never a hand `ALTER`. A table whose rows still need another step keeps its column, and the run says why. +- **Idempotent and resumable.** Every write is conditioned on the state it changes, and each finished step is checkpointed in `sys_migration` (row `adr-0131-organization-ownership`); a run interrupted mid-step resumes at that step and never re-derives an owner already written. +- **The post-check** re-reads the plan and reports, per table, what remains. The ledger row records `applied_at` once every step has finished and `verified_at` when the post-check also reads zero discrepancies. Exit code 1 while the ceremony is unfinished or the post-check finds a discrepancy. +- A JSON report is written beside the plan (`organization-ownership-apply-TIMESTAMP.json`, or `--out`); an existing file is never overwritten. + +```bash +os migrate organization-ownership --out plan.json # read the plan first +os migrate organization-ownership --apply --backup-taken # then perform it +``` diff --git a/packages/cli/src/commands/migrate/organization-ownership.apply-flags.test.ts b/packages/cli/src/commands/migrate/organization-ownership.apply-flags.test.ts new file mode 100644 index 0000000000..19b7de0a2e --- /dev/null +++ b/packages/cli/src/commands/migrate/organization-ownership.apply-flags.test.ts @@ -0,0 +1,68 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * `os migrate organization-ownership --apply` asks about the backup before it + * opens the database (ADR-0131 D10 ceremony item 2): without `--backup-taken` + * a non-interactive run is refused, and the boot never happens. The flag means + * nothing without `--apply`, and is refused there too. + */ + +import { afterEach, describe, expect, it, vi } from 'vitest'; +import { fileURLToPath } from 'node:url'; + +const boot = vi.hoisted(() => vi.fn(async () => { + throw new Error('the database was opened'); +})); +vi.mock('../../utils/schema-migrate.js', () => ({ bootSchemaStack: boot })); + +const { default: MigrateOrganizationOwnership } = await import('./organization-ownership.js'); + +const CLI_ROOT = fileURLToPath(new URL('../../..', import.meta.url)); + +async function runJson(argv: string[]): Promise<{ payload: Record; exitCode: unknown }> { + const chunks: string[] = []; + const savedExit = process.exitCode; + vi.spyOn(process.stdout, 'write').mockImplementation(((chunk: unknown, ...rest: unknown[]) => { + chunks.push(String(chunk)); + const cb = rest.find((a) => typeof a === 'function') as (() => void) | undefined; + if (cb) cb(); + return true; + }) as typeof process.stdout.write); + vi.spyOn(process.stderr, 'write').mockImplementation((() => true) as typeof process.stderr.write); + try { + await MigrateOrganizationOwnership.run(argv, { root: CLI_ROOT }); + } catch (error) { + if ((error as { oclif?: unknown }).oclif === undefined) throw error; + } + const exitCode = process.exitCode; + process.exitCode = savedExit; + vi.restoreAllMocks(); + return { payload: JSON.parse(chunks.join('').trim()) as Record, exitCode }; +} + +afterEach(() => { + boot.mockClear(); +}); + +describe('os migrate organization-ownership --apply — the backup precondition', () => { + it('refuses a non-interactive --apply without --backup-taken, before the boot', async () => { + const { payload, exitCode } = await runJson(['--apply', '--json']); + expect(payload).toMatchObject({ error: 'apply_refused', reason: 'backup-unacknowledged' }); + expect(exitCode).toBe(1); + expect(boot).not.toHaveBeenCalled(); + }); + + it('refuses --backup-taken without --apply, before the boot', async () => { + const { payload, exitCode } = await runJson(['--backup-taken', '--json']); + expect(payload).toMatchObject({ error: 'plan_refused', reason: 'flag-without-apply' }); + expect(exitCode).toBe(1); + expect(boot).not.toHaveBeenCalled(); + }); + + it('with --backup-taken, goes on to the boot', async () => { + const { payload } = await runJson(['--apply', '--backup-taken', '--json']); + expect(payload).toMatchObject({ error: 'boot_failed', reason: 'boot-failed' }); + expect(boot).toHaveBeenCalledTimes(1); + expect(boot.mock.calls[0]).toEqual([expect.objectContaining({ deferSchemaDdl: true, composeHostStack: true })]); + }); +}); diff --git a/packages/cli/src/commands/migrate/organization-ownership.ts b/packages/cli/src/commands/migrate/organization-ownership.ts index 327c136b88..b01f152cf1 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.ts @@ -2,6 +2,7 @@ import { writeFile } from 'node:fs/promises'; import { resolve } from 'node:path'; +import { createInterface } from 'node:readline'; import { Command, Flags } from '@oclif/core'; import chalk from 'chalk'; import { resolveTenancyPosture } from '@objectstack/types'; @@ -24,6 +25,29 @@ import { OrganizationOwnershipPlanRefusal, type OrganizationOwnershipPlan, } from '../../utils/organization-ownership-plan.js'; +import { + applyOrganizationOwnership, + checkpointFromDetails, + ORGANIZATION_OWNERSHIP_MIGRATION_ID, + type ApplyExecutor, + type CeremonyLedgerPort, + type ColumnDropOutcome, + type OrganizationOwnershipApplyReport, +} from '../../utils/organization-ownership-apply.js'; + +/** Who the ceremony's metadata saves name as their actor. */ +const CEREMONY_ACTOR = 'os migrate organization-ownership'; + +async function confirm(question: string): Promise { + if (!process.stdin.isTTY) return false; // non-interactive → require --backup-taken + const rl = createInterface({ input: process.stdin, output: process.stdout }); + try { + const answer: string = await new Promise((resolve) => rl.question(question, resolve)); + return /^y(es)?$/i.test(answer.trim()); + } finally { + rl.close(); + } +} /** * `os migrate organization-ownership` — the v18 organization-ownership @@ -53,20 +77,43 @@ import { * Bare `os migrate` is the schema-drift plan (#2186) and stays exactly that. * Every data step in this family is a named subcommand whose bare run is the * read-only preview and whose `--apply` is the only writing mode; the ADR's - * `--plan` names this mode. C7b adds `--apply` (fate order: attribution, the - * verified id-to-name rewrite, mirror deletion, column drops) and the - * post-check to THIS command; until then it has no writing mode at all. + * `--plan` names this mode. + * + * ## `--apply` (ADR-0131 D10 ceremony items 2 to 4) + * + * The writing mode. It asks about the backup first — the operator's act and + * the precondition (D10 item 2): `--backup-taken`, or a yes at the prompt; a + * non-interactive run without the flag is refused before the boot. It then + * performs the fates in order — attribution, the verified id→name rewrite, + * mirror deletion, column drops — through `../../utils/organization-ownership-apply.ts`, + * checkpointing each finished step in the deployment ledger (`sys_migration`, + * row {@link ORGANIZATION_OWNERSHIP_MIGRATION_ID}) so an interrupted run + * resumes where it stopped, and ends with the post-check. The ledger row is + * the ceremony's marker: `applied_at` once every step has finished, + * `verified_at` when the post-check also reads zero discrepancies. A report + * file is written beside the plan's, never over an existing one. + * + * The apply boot composes the deployment's served object set + * (`composeHostStack`, with the auth-gated security declarations and the + * served platform) so the security catalog it verifies names against is the + * one the deployment serves, and the schema-drift ceremony proposes the + * column drops of exactly the objects this release declares without the + * column. It still holds the boot's own DDL back (`deferSchemaDdl`): the only + * schema change the ceremony makes is the drops it asked for. */ export default class MigrateOrganizationOwnership extends Command { static override description = - 'The v18 organization-ownership ceremony\'s read-only plan (ADR-0131 D10): per table, the fate, the row counts, the ' + - 'rows whose owner cannot be derived (by id), and which tables will receive NOT NULL. Writes the plan to a file; ' + - 'never changes a row or a column. Refuses — naming it — any table it cannot enumerate.'; + 'The v18 organization-ownership ceremony (ADR-0131 D10). Without --apply, the read-only plan: per table, the fate, ' + + 'the row counts, the rows whose owner cannot be derived (by id), and which tables will receive NOT NULL, written ' + + 'to a file — never a row or a column changed; it refuses, naming it, any table it cannot enumerate. With --apply ' + + '(and a backup): attribution, the verified id-to-name rewrite, mirror deletion and column drops, in that order, ' + + 'resumable from a recorded checkpoint, then the post-check.'; static override examples = [ '$ os migrate organization-ownership', '$ os migrate organization-ownership --out plans/v18-ownership.json', '$ os migrate organization-ownership --database-url postgres://… --json', + '$ os migrate organization-ownership --apply --backup-taken', ]; static override flags = { @@ -79,7 +126,19 @@ export default class MigrateOrganizationOwnership extends Command { 'Where to write the plan (default: organization-ownership-plan-TIMESTAMP.json in the current directory). ' + 'An existing file is never overwritten', }), - json: Flags.boolean({ description: 'Also print the plan document on stdout' }), + json: Flags.boolean({ description: 'Also print the plan (or, with --apply, the apply report) on stdout' }), + apply: Flags.boolean({ + description: + 'Perform the ceremony: attribution, the verified id-to-name rewrite, mirror deletion, column drops, then the ' + + 'post-check. Idempotent; an interrupted run resumes from its recorded checkpoint. Requires a backup', + default: false, + }), + 'backup-taken': Flags.boolean({ + description: + 'With --apply: state that a backup of this database has been taken. The platform takes none and assumes ' + + 'none; without this flag the run asks, and a non-interactive run is refused', + default: false, + }), }; async run(): Promise { @@ -88,7 +147,12 @@ export default class MigrateOrganizationOwnership extends Command { const json = Boolean(flags.json); const refuse = async (payload: Record, message: string): Promise => { if (json) { - await emitJson({ error: payload.reason === 'boot-failed' ? 'boot_failed' : 'plan_refused', ...payload }, 1, { compact: true }); + const error = payload.reason === 'boot-failed' + ? 'boot_failed' + : payload.reason === 'apply-failed' + ? 'apply_failed' + : flags.apply ? 'apply_refused' : 'plan_refused'; + await emitJson({ error, ...payload }, 1, { compact: true }); return; } printError(message); @@ -103,6 +167,15 @@ export default class MigrateOrganizationOwnership extends Command { return; } + if (flags.apply) { + await this.runApply(flags, posture, refuse); + return; + } + if (flags['backup-taken']) { + await refuse({ reason: 'flag-without-apply', detail: '--backup-taken is only meaningful with --apply' }, '--backup-taken is only meaningful with --apply.'); + return; + } + if (!json) { printHeader('Migrate · organization-ownership (plan)'); printStep('Booting data stack (read-only)…'); @@ -199,4 +272,209 @@ export default class MigrateOrganizationOwnership extends Command { printSuccess(`Plan written to ${target} — nothing in the database was changed.`); console.log(chalk.dim(` ${timer.display()}`)); } + + /** `--apply`: the backup precondition, the boot, the ceremony, the report file. */ + private async runApply( + flags: { 'database-url'?: string; out?: string; json?: boolean; 'backup-taken'?: boolean }, + posture: string, + refuse: (payload: Record, message: string) => Promise, + ): Promise { + const json = Boolean(flags.json); + const timer = createTimer(); + if (!flags['backup-taken']) { + const target = flags['database-url'] ?? process.env.OS_DATABASE_URL ?? 'the project database'; + const yes = !json && (await confirm( + `${chalk.yellow('Backup')}: this run rewrites ownership, deletes mirror rows and drops columns in ${target}. ` + + 'Has a backup of it been taken? [y/N] ', + )); + if (!yes) { + await refuse( + { reason: 'backup-unacknowledged', detail: 'pass --backup-taken once a backup of this database exists' }, + 'Refused: --apply needs a backup of this database first. Take one, then re-run with --backup-taken ' + + '(or answer yes at the prompt). The platform takes no backup and assumes none.', + ); + return; + } + } + + if (!json) { + printHeader('Migrate · organization-ownership (apply)'); + printStep('Booting the deployment\'s served stack (schema DDL held back)…'); + } + let stack; + try { + stack = await bootSchemaStack({ + jsonOutput: json, + ...(flags['database-url'] ? { databaseUrl: flags['database-url'] } : {}), + deferSchemaDdl: true, + composeHostStack: true, + composeServedPlatform: true, + composeAuthGatedSecurity: true, + }); + } catch (error: unknown) { + await refuse({ reason: 'boot-failed', detail: String((error as Error)?.message ?? error) }, String((error as Error)?.message ?? error)); + return; + } + + let report: OrganizationOwnershipApplyReport; + try { + const exec = resolvePlannedDriverExec(stack.driver); + if (!stack.driver || !exec) { + await refuse( + { reason: 'driver-unsupported', detail: 'no SQL driver with a raw-SQL seam is active' }, + 'No SQL driver with a raw-SQL seam is active. The ceremony supports SQLite, PostgreSQL and MySQL.', + ); + return; + } + if (stack.tableAbsent('sys_migration')) { + await refuse( + { reason: 'ledger-absent', table: 'sys_migration', detail: 'the deployment ledger the ceremony checkpoints into does not exist' }, + 'Refused: this database has no sys_migration table, where the ceremony records its checkpoint. Boot this ' + + 'release against it once (its schema sync creates the table), then re-run.', + ); + return; + } + const { normalizeRows } = await import('@objectstack/metadata-protocol'); + const { createSecurityCatalogReader } = await import('@objectstack/core'); + const { readDataMigrationFlag, recordDataMigrationRun } = await import('@objectstack/platform-objects/system'); + const kernel = stack.kernel as { getService(name: string): unknown }; + const engine = kernel.getService('objectql') as Parameters[0] & { registry?: unknown }; + const protocol = kernel.getService('protocol') as { + saveMetaItem?: (request: Record) => Promise; + } | undefined; + const catalog = createSecurityCatalogReader({ + registry: engine.registry as never, + metadata: kernel.getService('metadata') as never, + }); + const client = (stack.driver.config as { client?: unknown } | undefined)?.client; + const executor: ApplyExecutor = { + client: client === undefined ? undefined : String(client), + query: async (sql, params) => normalizeRows(await exec(sql, params ? [...params] : [])), + execute: async (sql, params) => { + await exec(sql, params ? [...params] : []); + }, + }; + const ledger: CeremonyLedgerPort = { + read: async () => checkpointFromDetails((await readDataMigrationFlag(engine, ORGANIZATION_OWNERSHIP_MIGRATION_ID))?.details), + write: async (checkpoint, verdict) => { + await recordDataMigrationRun(engine, { + migrationId: ORGANIZATION_OWNERSHIP_MIGRATION_ID, + passed: verdict.complete && verdict.blocking === 0, + applied: verdict.complete, + blocking: verdict.blocking, + advisory: verdict.advisory, + details: { checkpoint }, + }); + }, + }; + const liveStack = stack; + report = await applyOrganizationOwnership({ + executor, + posture, + database: stack.dbLabel, + catalog, + door: { + save: async (type, name, body) => { + if (typeof protocol?.saveMetaItem !== 'function') { + throw new Error('no metadata protocol is composed on this stack'); + } + await protocol.saveMetaItem({ type, name, item: body, actor: CEREMONY_ACTOR }); + }, + }, + columnDrops: { + drop: async (tables) => { + const drift = await liveStack.detectManagedDrift(); + const wanted = new Set(tables); + const entries = drift.filter((d) => { + const op = (d as { op?: { type?: string; column?: string } }).op; + return op?.type === 'drop_column' && op.column === 'organization_id' && wanted.has(String(d.table)); + }); + const outcomes: ColumnDropOutcome[] = []; + const proposed = new Set(entries.map((d) => String(d.table))); + for (const table of tables) { + if (!proposed.has(table)) { + outcomes.push({ table, outcome: 'not-proposed', detail: 'the drift differ proposed no drop for it' }); + } + } + if (entries.length === 0) return outcomes; + const { applied, skipped } = await liveStack.applyMigrationEntries(entries, { allowDestructive: true }); + for (const d of applied) outcomes.push({ table: String(d.table), outcome: 'dropped' }); + for (const d of skipped) { + outcomes.push({ table: String(d.table), outcome: 'failed', detail: String((d as { message?: unknown }).message ?? 'skipped by the schema-drift ceremony') }); + } + return outcomes; + }, + }, + ledger, + ...(json ? {} : { onProgress: (message: string) => printStep(message) }), + }); + } catch (error: unknown) { + if (isExitSignal(error)) throw error; + if (error instanceof OrganizationOwnershipPlanRefusal) { + await refuse( + { reason: error.reason, ...(error.table ? { table: error.table } : {}), detail: error.message }, + `Refused${error.table ? ` (${error.table})` : ''}: ${error.message}`, + ); + return; + } + await refuse( + { reason: 'apply-failed', detail: String((error as Error)?.message ?? error), ...errorCodeFields(error) }, + `${String((error as Error)?.message ?? error)} — every step finished before this one is checkpointed; ` + + 're-run --apply to resume.', + ); + return; + } finally { + await stack.shutdown(); + } + + const target = resolve(flags.out ?? `organization-ownership-apply-${report.generatedAt.replace(/[:.]/g, '-')}.json`); + try { + await writeFile(target, `${JSON.stringify(report, null, 2)}\n`, { flag: 'wx' }); + } catch (error) { + await refuse( + { reason: 'report-file-unwritable', file: target, detail: String((error as Error).message) }, + `The ceremony ran, and its report could not be written to ${target}: ${(error as Error).message}. ` + + 'An existing file is never overwritten; re-run --apply (it resumes, writing nothing new) with --out.', + ); + return; + } + const verdictCode = report.complete && report.blocking === 0 ? 0 : 1; + if (json) { + await emitJson({ file: target, report }, verdictCode); + return; + } + console.log(''); + for (const step of report.steps) { + const mark = step.status === 'done' || step.status === 'checkpointed' ? chalk.green('✓') : step.status === 'blocked' ? chalk.red('✗') : chalk.dim('·'); + console.log(` ${mark} ${step.step.padEnd(22)} ${step.status.padEnd(12)} ${chalk.dim(JSON.stringify(step.summary))}`); + for (const blocker of step.blockers) console.log(` ${chalk.red(blocker)}`); + for (const note of step.notes) console.log(` ${chalk.dim(note)}`); + } + console.log(''); + for (const table of report.tables) { + if (!table.before) continue; + if (table.attributed + table.deleted + table.unattributable.length + table.remaining === 0 && !table.columnDropped) continue; + console.log( + ` ${chalk.white(table.object.padEnd(34))} ${table.fate.padEnd(16)} attributed ${String(table.attributed).padStart(6)} ` + + `deleted ${String(table.deleted).padStart(6)} ${table.columnDropped ? chalk.green('column dropped ') : ''}` + + `${table.unattributable.length ? chalk.yellow(`reported ${table.unattributable.length} `) : ''}` + + `${table.remaining ? chalk.red(`remaining ${table.remaining}`) : ''}`, + ); + } + console.log(''); + if (report.advisory > 0) { + printWarning(`${report.advisory} row(s) whose owner cannot be derived stay NULL; they are listed by id in the report file.`); + } + if (verdictCode === 0) { + printSuccess(`The ceremony is complete; recorded as ${ORGANIZATION_OWNERSHIP_MIGRATION_ID}. Report: ${target}`); + } else { + printError( + report.complete + ? `The post-check found ${report.blocking} discrepancy(ies). Report: ${target}` + : `The ceremony stopped at a blocker (above). Resolve it and re-run --apply; finished steps are not repeated. Report: ${target}`, + ); + } + console.log(chalk.dim(` ${timer.display()}`)); + if (verdictCode !== 0) this.exit(1); + } } diff --git a/packages/cli/src/utils/organization-ownership-apply.test.ts b/packages/cli/src/utils/organization-ownership-apply.test.ts new file mode 100644 index 0000000000..6591616de4 --- /dev/null +++ b/packages/cli/src/utils/organization-ownership-apply.test.ts @@ -0,0 +1,464 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The ADR-0131 D10 ceremony's acceptance: a fixture database carrying every + * fate comes through `--apply` with effective access IDENTICAL (and a positive + * control: a grant the rewrite would lose reddens the comparison), mirrors + * gone, attributed rows carrying their organization, the unattributable row + * still present and reported, and row counts reconciling. Plus the ceremony's + * guarantees: idempotent, resumable from the recorded checkpoint, gated + * (nothing deleted while a reference does not resolve by name), and no + * deletion outside fate 2. + * + * The shape is the post-stage-8 one: the catalog objects are no longer + * registered, so every read and write here goes through the raw-SQL seam, and + * `permission_set_id` is a plain text column beside `permission_set`. + */ + +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; +import Database from 'better-sqlite3'; +import { createHash } from 'node:crypto'; +import { mkdtempSync, readFileSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { permissionSetDefinitionFromRow } from '@objectstack/core'; +import { + applyOrganizationOwnership, + type ApplyExecutor, + type CatalogPort, + type CeremonyCheckpoint, + type CeremonyLedgerPort, + type ColumnDropPort, + type MetadataDoorPort, + type OrganizationOwnershipApplyReport, +} from './organization-ownership-apply.js'; + +let dir: string; +let file: string; +let db: Database.Database; +/** Every write statement the ceremony issued, in order. */ +let writes: string[]; + +function executor(opts: { failOnceOn?: RegExp } = {}): ApplyExecutor { + let failed = false; + return { + client: 'better-sqlite3', + async query(sql, params = []) { + return db.prepare(sql).all(...(params as unknown[])) as Record[]; + }, + async execute(sql, params = []) { + if (opts.failOnceOn && !failed && opts.failOnceOn.test(sql)) { + failed = true; + throw new Error('SQLITE_IOERR: the process was killed mid-step'); + } + writes.push(sql); + db.prepare(sql).run(...(params as unknown[])); + }, + }; +} + +/** + * The declared catalog (code) plus the environment metadata the door saved. + * Like the registry, it serves a saved definition only from the next boot on + * ({@link boot}), never the moment it is saved. + */ +function catalog(): CatalogPort & MetadataDoorPort & { saved: Map>; boot(): void } { + const declared = new Map>([ + ['permission|member_default', permissionSetDefinitionFromRow(MEMBER_DEFAULT_ROW)], + ['position|org_admin', { name: 'org_admin', label: 'Org Admin', delegatable: false, permissionSets: ['member_default'] }], + ]); + const saved = new Map>(); + let hydrated = new Map>(); + return { + saved, + boot() { + hydrated = new Map(saved); + }, + async resolve(type, name) { + const definition = hydrated.get(`${type}|${name}`) ?? declared.get(`${type}|${name}`); + return definition ? { definition } : undefined; + }, + async save(type, name, body) { + if (declared.has(`${type}|${name}`)) throw new Error(`NAMESPACE_CONFLICT: ${type} ${name} is held by a package`); + saved.set(`${type}|${name}`, body); + }, + }; +} + +function ledger(): CeremonyLedgerPort & { checkpoint: CeremonyCheckpoint | null; verdicts: Array<{ complete: boolean; blocking: number; advisory: number }> } { + const state = { + checkpoint: null as CeremonyCheckpoint | null, + verdicts: [] as Array<{ complete: boolean; blocking: number; advisory: number }>, + async read() { + return state.checkpoint ? (JSON.parse(JSON.stringify(state.checkpoint)) as CeremonyCheckpoint) : null; + }, + async write(checkpoint: CeremonyCheckpoint, verdict: { complete: boolean; blocking: number; advisory: number }) { + state.checkpoint = JSON.parse(JSON.stringify(checkpoint)) as CeremonyCheckpoint; + state.verdicts.push(verdict); + }, + }; + return state; +} + +/** The schema-drift ceremony, as SQLite answers it: the column goes. */ +const columnDrops: ColumnDropPort = { + async drop(tables) { + return tables.map((table) => { + writes.push(`ALTER TABLE ${table} DROP COLUMN organization_id`); + db.exec(`ALTER TABLE "${table}" DROP COLUMN organization_id`); + return { table, outcome: 'dropped' as const }; + }); + }, +}; + +const MEMBER_DEFAULT_ROW = { + name: 'member_default', + label: 'Member', + object_permissions: '{"crm_deal":{"allowRead":true}}', + system_permissions: '[]', +}; + +/** A 17.x-shaped database carrying every fate, on the post-stage-8 shape. */ +function seed(): void { + db.exec(` + CREATE TABLE sys_organization (id TEXT PRIMARY KEY, slug TEXT); + INSERT INTO sys_organization VALUES ('org_default', 'default'), ('org_b', 'b'); + + -- fate 2: the catalog. Per-organization mirrors and NULL residue of one + -- declared set and one declared position, beside Setup-authored rows. + CREATE TABLE sys_permission_set (id TEXT PRIMARY KEY, organization_id TEXT, name TEXT, label TEXT, managed_by TEXT, + active INTEGER, object_permissions TEXT, system_permissions TEXT); + INSERT INTO sys_permission_set VALUES + ('ps_null', NULL, 'member_default', 'Member', 'platform', 1, '{"crm_deal":{"allowRead":true}}', '[]'), + ('ps_org', 'org_default', 'member_default', 'Member', 'package', 1, '{"crm_deal":{"allowRead":true}}', '[]'), + ('ps_auth', 'org_default', 'field_auditor', 'Field Auditor', 'admin', 1, + '{"crm_deal":{"allowRead":true,"allowEdit":true}}', '["export_reports"]'); + CREATE TABLE sys_position (id TEXT PRIMARY KEY, organization_id TEXT, name TEXT, label TEXT, managed_by TEXT, + delegatable INTEGER, active INTEGER); + INSERT INTO sys_position VALUES + ('pos_null', NULL, 'org_admin', 'Org Admin', 'platform', 0, 1), + ('pos_org', 'org_default', 'org_admin', 'Org Admin', 'platform', 0, 1), + ('pos_auth', 'org_default', 'sales_lead', 'Sales Lead', 'admin', 1, 1); + CREATE TABLE sys_position_permission_set (id TEXT PRIMARY KEY, organization_id TEXT, position_id TEXT, permission_set_id TEXT); + INSERT INTO sys_position_permission_set VALUES + ('bind_1', NULL, 'pos_null', 'ps_null'), + ('bind_2', 'org_default', 'pos_org', 'ps_org'), + ('bind_3', 'org_default', 'pos_auth', 'ps_auth'); + CREATE TABLE sys_capability (id TEXT PRIMARY KEY, organization_id TEXT, name TEXT, managed_by TEXT); + INSERT INTO sys_capability VALUES ('cap_seeded', NULL, 'manage_users', 'platform'), ('cap_authored', 'org_default', 'audit_fields', 'admin'); + -- fate 2: templates — seeded mirrors, one customized, one organization-stamped + CREATE TABLE sys_email_template (id TEXT PRIMARY KEY, organization_id TEXT, name TEXT, customized INTEGER, managed_by TEXT); + INSERT INTO sys_email_template VALUES + ('tpl_seeded', NULL, 'welcome', 0, 'platform'), + ('tpl_customized', NULL, 'reset_password', 1, 'platform'), + ('tpl_org', 'org_b', 'welcome', 0, 'platform'); + + -- the references: grants by id (the rewrite names them) and assignments by name + CREATE TABLE sys_user_permission_set (id TEXT PRIMARY KEY, organization_id TEXT, user_id TEXT, + permission_set_id TEXT, permission_set TEXT); + INSERT INTO sys_user_permission_set VALUES + ('grant_1', 'org_default', 'usr_1', 'ps_org', NULL), + ('grant_2', NULL, 'usr_2', 'ps_null', NULL), + ('grant_3', 'org_default', 'usr_3', 'ps_auth', NULL), + ('grant_4', 'org_default', 'usr_4', 'ps_null', 'member_default'); + CREATE TABLE sys_business_unit (id TEXT PRIMARY KEY, organization_id TEXT, parent_business_unit_id TEXT); + INSERT INTO sys_business_unit VALUES ('bu_b', 'org_b', NULL); + CREATE TABLE sys_user_position (id TEXT PRIMARY KEY, organization_id TEXT, user_id TEXT, position TEXT, business_unit_id TEXT); + INSERT INTO sys_user_position VALUES + ('up_1', 'org_default', 'usr_5', 'sales_lead', NULL), + ('up_2', NULL, 'usr_6', 'org_admin', NULL); + + -- fate 3: NULL sys_file rows whose sys_attachment holder carries the organization + CREATE TABLE crm_deal (id TEXT PRIMARY KEY, organization_id TEXT, name TEXT); + INSERT INTO crm_deal VALUES ('deal_1', 'org_b', 'Acme'), ('deal_2', NULL, 'Orphan'); + CREATE TABLE sys_attachment (id TEXT PRIMARY KEY, organization_id TEXT, file_id TEXT, parent_object TEXT, parent_id TEXT); + INSERT INTO sys_attachment VALUES ('att_1', NULL, 'file_1', 'crm_deal', 'deal_1'); + CREATE TABLE sys_file (id TEXT PRIMARY KEY, organization_id TEXT, ref_object TEXT, ref_id TEXT); + INSERT INTO sys_file VALUES ('file_1', NULL, NULL, NULL), ('file_2', 'org_b', NULL, NULL); + + -- fate 4: one genuinely unattributable row + CREATE TABLE sys_saved_report (id TEXT PRIMARY KEY, organization_id TEXT); + INSERT INTO sys_saved_report VALUES ('report_orphan', NULL); + + -- fate 1: a D7 object whose column is still there + CREATE TABLE sys_job (id TEXT PRIMARY KEY, organization_id TEXT); + INSERT INTO sys_job VALUES ('job_1', NULL), ('job_2', 'org_b'); + `); +} + +const counts = (): Record => { + const tables = db.prepare(`SELECT name FROM sqlite_master WHERE type = 'table' ORDER BY name`).all() as { name: string }[]; + return Object.fromEntries(tables.map(({ name }) => [name, (db.prepare(`SELECT COUNT(*) AS n FROM "${name}"`).get() as { n: number }).n])); +}; +const rows = (sql: string): Record[] => db.prepare(sql).all() as Record[]; +const digestOf = (definition: Record): string => { + const keys = ['name', 'label', 'objects', 'fields', 'systemPermissions', 'rowLevelSecurity', 'tabPermissions']; + return JSON.stringify(keys.map((k) => definition[k] ?? null)); +}; + +/** + * Effective access BEFORE — the 17.x reading: a grant reaches its set row by + * id, a position reaches its sets through its rows' bindings. Under `single` + * an organization-less row is the one organization's. + */ +function accessBefore(): string[] { + const out = new Set(); + const sets = new Map(rows('SELECT * FROM sys_permission_set').map((r) => [String(r.id), r])); + const sees = (row: Record, org: string): boolean => row.organization_id === null || row.organization_id === org; + for (const grant of rows('SELECT * FROM sys_user_permission_set')) { + const org = String(grant.organization_id ?? 'org_default'); + const set = sets.get(String(grant.permission_set_id)); + if (set && sees(set, org)) out.add(`${grant.user_id}|${org}|${set.name}|${digestOf(permissionSetDefinitionFromRow(set))}`); + } + const bindings = rows('SELECT * FROM sys_position_permission_set'); + for (const up of rows('SELECT * FROM sys_user_position')) { + const org = String(up.organization_id ?? 'org_default'); + for (const position of rows('SELECT * FROM sys_position').filter((p) => p.name === up.position && sees(p, org))) { + for (const binding of bindings.filter((b) => b.position_id === position.id)) { + const set = sets.get(String(binding.permission_set_id)); + if (set) out.add(`${up.user_id}|${org}|${set.name}|${digestOf(permissionSetDefinitionFromRow(set))}`); + } + } + } + return [...out].sort(); +} + +/** Effective access AFTER — the v18 reading, on the next boot: every reference is a name, resolved in the catalog. */ +async function accessAfter(c: ReturnType): Promise { + c.boot(); + const out = new Set(); + const add = async (user: unknown, org: string, name: string): Promise => { + const set = await c.resolve('permission', name); + if (set) out.add(`${user}|${org}|${name}|${digestOf(set.definition as Record)}`); + }; + for (const grant of rows('SELECT * FROM sys_user_permission_set')) { + if (grant.permission_set) await add(grant.user_id, String(grant.organization_id), String(grant.permission_set)); + } + for (const up of rows('SELECT * FROM sys_user_position')) { + const position = await c.resolve('position', String(up.position)); + for (const name of (position?.definition.permissionSets as string[] | undefined) ?? []) { + await add(up.user_id, String(up.organization_id), name); + } + } + return [...out].sort(); +} + +const apply = ( + c: ReturnType, + l: ReturnType, + opts: { posture?: string; executor?: ApplyExecutor } = {}, +): Promise => { + c.boot(); + return applyOrganizationOwnership({ + executor: opts.executor ?? executor(), + posture: opts.posture ?? 'single', + database: file, + catalog: c, + door: c, + columnDrops, + ledger: l, + now: () => new Date('2026-10-12T00:00:00Z'), + }); +}; + +const tableOf = (report: OrganizationOwnershipApplyReport, object: string) => { + const found = report.tables.find((t) => t.object === object); + if (!found) throw new Error(`no ${object} in the report`); + return found; +}; + +beforeEach(() => { + dir = mkdtempSync(join(tmpdir(), 'os-c7b-apply-')); + file = join(dir, 'fixture.db'); + db = new Database(file); + writes = []; + seed(); +}); + +afterEach(() => { + db.close(); + rmSync(dir, { recursive: true, force: true }); +}); + +describe('ADR-0131 D10 --apply — the acceptance fixture', () => { + it('comes through with effective access identical, mirrors gone, owners written, the orphan reported, counts reconciling', async () => { + const before = accessBefore(); + const countsBefore = counts(); + const c = catalog(); + const l = ledger(); + + const report = await apply(c, l); + + expect(report.steps.map((s) => [s.step, s.status])).toEqual([ + ['attribution', 'done'], + ['authored-conversion', 'done'], + ['rewrite', 'done'], + ['mirror-deletion', 'done'], + ['column-drops', 'done'], + ]); + expect(report.complete).toBe(true); + expect(report.blocking).toBe(0); + expect(l.verdicts.at(-1)).toEqual({ complete: true, blocking: 0, advisory: 1 }); + + // Effective access: identical, read the 17.x way before and the v18 way after. + expect(before.length).toBeGreaterThan(0); + expect(await accessAfter(c)).toEqual(before); + + // The authored catalog rows became environment metadata, through the door. + expect(c.saved.get('permission|field_auditor')).toMatchObject({ name: 'field_auditor', systemPermissions: ['export_reports'] }); + expect(c.saved.get('position|sales_lead')).toMatchObject({ name: 'sales_lead', delegatable: true, permissionSets: ['field_auditor'] }); + expect(c.saved.has('position|org_admin')).toBe(false); + + // Mirrors gone; authored, customized and organization-stamped rows stay. + expect(rows('SELECT id FROM sys_permission_set').map((r) => r.id)).toEqual(['ps_auth']); + expect(rows('SELECT id FROM sys_position').map((r) => r.id)).toEqual(['pos_auth']); + expect(rows('SELECT id FROM sys_position_permission_set')).toEqual([]); + expect(rows('SELECT id FROM sys_capability').map((r) => r.id)).toEqual(['cap_authored']); + expect(rows('SELECT id FROM sys_email_template ORDER BY id').map((r) => r.id)).toEqual(['tpl_customized', 'tpl_org']); + + // Every grant names its set; the id column is still there (C7b's later drop). + expect(rows('SELECT id, permission_set, permission_set_id FROM sys_user_permission_set ORDER BY id')).toEqual([ + { id: 'grant_1', permission_set: 'member_default', permission_set_id: 'ps_org' }, + { id: 'grant_2', permission_set: 'member_default', permission_set_id: 'ps_null' }, + { id: 'grant_3', permission_set: 'field_auditor', permission_set_id: 'ps_auth' }, + { id: 'grant_4', permission_set: 'member_default', permission_set_id: 'ps_null' }, + ]); + + // Attributed rows carry their organization — the anchor's, else the Default Organization. + expect(rows(`SELECT organization_id FROM sys_file WHERE id = 'file_1'`)).toEqual([{ organization_id: 'org_b' }]); + expect(rows(`SELECT organization_id FROM sys_attachment WHERE id = 'att_1'`)).toEqual([{ organization_id: 'org_b' }]); + expect(rows(`SELECT organization_id FROM crm_deal WHERE id = 'deal_2'`)).toEqual([{ organization_id: 'org_default' }]); + expect(rows(`SELECT organization_id FROM sys_user_permission_set WHERE id = 'grant_2'`)).toEqual([{ organization_id: 'org_default' }]); + + // The unattributable row is still present, still NULL, and reported by id. + expect(rows(`SELECT organization_id FROM sys_saved_report`)).toEqual([{ organization_id: null }]); + expect(tableOf(report, 'sys_saved_report').unattributable.map((u) => u.id)).toEqual(['report_orphan']); + + // Fate 1: the column is gone through the column-drop port. + expect(tableOf(report, 'sys_job').columnDropped).toBe(true); + expect(rows(`SELECT name FROM pragma_table_info('sys_job')`).map((r) => r.name)).toEqual(['id']); + + // Row counts reconcile: only fate 2 deletes, and exactly what it reports. + const countsAfter = counts(); + for (const table of report.tables) { + if (!table.before || !table.after) continue; + expect(table.before.total - table.deleted, table.object).toBe(table.after.total); + expect(countsAfter[table.object], table.object).toBe(table.after.total); + expect(countsBefore[table.object], table.object).toBe(table.before.total); + if (table.fate !== 'mirror-deletion') expect(table.deleted, table.object).toBe(0); + } + expect(writes.filter((w) => /^DELETE/.test(w)).every((w) => + /sys_permission_set|sys_position|sys_position_permission_set|sys_capability|sys_email_template/.test(w))).toBe(true); + }); + + it('positive control: a grant the rewrite would lose reddens the effective-access pin', async () => { + const before = accessBefore(); + const c = catalog(); + await apply(c, ledger()); + expect(await accessAfter(c)).toEqual(before); + + db.exec(`UPDATE sys_user_permission_set SET permission_set = NULL WHERE id = 'grant_3'`); + const after = await accessAfter(c); + expect(after).not.toEqual(before); + expect(before.filter((g) => !after.includes(g))).toEqual([expect.stringMatching(/^usr_3\|org_default\|field_auditor\|/)]); + }); +}); + +describe('ADR-0131 D10 --apply — idempotent, resumable, gated', () => { + it('a second run resumes from the checkpoint and writes nothing', async () => { + const c = catalog(); + const l = ledger(); + await apply(c, l); + const hash = createHash('sha256').update(readFileSync(file)).digest('hex'); + writes = []; + + const again = await apply(c, l); + + expect(again.resumedFrom).toEqual(['attribution', 'authored-conversion', 'rewrite', 'mirror-deletion', 'column-drops']); + expect(again.steps.every((s) => s.status === 'checkpointed')).toBe(true); + expect(again.complete).toBe(true); + expect(writes).toEqual([]); + expect(createHash('sha256').update(readFileSync(file)).digest('hex')).toBe(hash); + }); + + it('a run killed inside mirror deletion resumes there, never re-deriving an owner it already wrote', async () => { + const c = catalog(); + const l = ledger(); + await expect(apply(c, l, { executor: executor({ failOnceOn: /^DELETE/ }) })).rejects.toThrow(/killed mid-step/); + expect(Object.keys(l.checkpoint!.completed)).toEqual(['attribution', 'authored-conversion', 'rewrite']); + expect(rows('SELECT COUNT(*) AS n FROM sys_position_permission_set')).toEqual([{ n: 3 }]); + writes = []; + + const resumed = await apply(c, l); + + expect(resumed.resumedFrom).toEqual(['attribution', 'authored-conversion', 'rewrite']); + expect(resumed.steps.map((s) => s.status)).toEqual(['checkpointed', 'checkpointed', 'checkpointed', 'done', 'done']); + expect(writes.some((w) => /^UPDATE/.test(w))).toBe(false); + expect(resumed.complete).toBe(true); + expect(rows('SELECT COUNT(*) AS n FROM sys_position_permission_set')).toEqual([{ n: 0 }]); + }); + + it('reads the gate again before the first delete: a grant written between runs keeps every mirror', async () => { + const c = catalog(); + const l = ledger(); + await expect(apply(c, l, { executor: executor({ failOnceOn: /^DELETE/ }) })).rejects.toThrow(/killed mid-step/); + db.exec(`INSERT INTO sys_user_permission_set VALUES ('grant_late', 'org_default', 'usr_8', 'ps_gone', NULL)`); + const countsBefore = counts(); + + const resumed = await apply(c, l); + + const deletion = resumed.steps.find((s) => s.step === 'mirror-deletion')!; + expect(deletion.status).toBe('blocked'); + expect(deletion.blockers.join('\n')).toMatch(/grant_late/); + expect(counts()).toEqual(countsBefore); + expect(resumed.complete).toBe(false); + }); + + it('deletes nothing while a reference does not resolve by name — the gate of fate 2', async () => { + // Another organization's set row: a name carries no organization, so it is never written. + db.exec(` + INSERT INTO sys_permission_set VALUES ('ps_b', 'org_b', 'b_only', 'B', 'admin', 1, '{}', '[]'); + INSERT INTO sys_user_permission_set VALUES ('grant_x', 'org_default', 'usr_9', 'ps_b', NULL); + `); + const countsBefore = counts(); + const l = ledger(); + + const report = await apply(catalog(), l); + + const rewrite = report.steps.find((s) => s.step === 'rewrite')!; + expect(rewrite.status).toBe('blocked'); + expect(rewrite.blockers.join('\n')).toMatch(/grant_x/); + expect(report.steps.find((s) => s.step === 'mirror-deletion')!.status).toBe('not-reached'); + expect(writes.filter((w) => /^DELETE/.test(w))).toEqual([]); + expect(counts()).toEqual(countsBefore); + expect(report.complete).toBe(false); + expect(report.blocking).toBeGreaterThan(0); + expect(l.verdicts.at(-1)!.complete).toBe(false); + }); + + it('under a wall: an anchorless row is reported, never assigned, and authored rows are not converted', async () => { + const c = catalog(); + const report = await apply(c, ledger(), { posture: 'isolated' }); + + expect(rows(`SELECT organization_id FROM crm_deal WHERE id = 'deal_2'`)).toEqual([{ organization_id: null }]); + expect(tableOf(report, 'crm_deal').unattributable.map((u) => u.id)).toEqual(['deal_2']); + expect(rows(`SELECT organization_id FROM sys_file WHERE id = 'file_1'`)).toEqual([{ organization_id: 'org_b' }]); + expect(c.saved.size).toBe(0); + expect(report.steps.find((s) => s.step === 'mirror-deletion')!.status).toBe('not-reached'); + expect(rows('SELECT COUNT(*) AS n FROM sys_permission_set')).toEqual([{ n: 3 }]); + }); + + it('holds a column whose rows another stage owns, and reports it', async () => { + db.exec(` + CREATE TABLE sys_metadata (id TEXT PRIMARY KEY, organization_id TEXT, type TEXT, name TEXT, state TEXT, package_id TEXT); + INSERT INTO sys_metadata VALUES ('md_1', 'org_b', 'view', 'deal_list', 'active', NULL); + `); + const report = await apply(catalog(), ledger()); + + const drops = report.steps.find((s) => s.step === 'column-drops')!; + expect(drops.status).toBe('blocked'); + expect(drops.blockers.join('\n')).toMatch(/sys_metadata\.organization_id is not dropped/); + expect(rows(`SELECT name FROM pragma_table_info('sys_metadata') WHERE name = 'organization_id'`)).toHaveLength(1); + expect(rows(`SELECT name FROM pragma_table_info('sys_job') WHERE name = 'organization_id'`)).toHaveLength(0); + }); +}); diff --git a/packages/cli/src/utils/organization-ownership-apply.ts b/packages/cli/src/utils/organization-ownership-apply.ts new file mode 100644 index 0000000000..3c8c9b6f8b --- /dev/null +++ b/packages/cli/src/utils/organization-ownership-apply.ts @@ -0,0 +1,942 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import { + convertPositionBindingRows, + permissionSetDefinitionFromRow, + positionDefinitionFromRow, +} from '@objectstack/core'; +import { + inventoryEntryFor, + type InventoryEntry, + type RowPredicate, +} from './organization-ownership-inventory.js'; +import { + buildOrganizationOwnershipPlan, + columnListSql, + columnNameOf, + dialectOf, + predicateSql, + Sql, + type AttributionSink, + type OrganizationOwnershipPlan, + type PlanReader, + type TablePlan, + type UnattributableRow, +} from './organization-ownership-plan.js'; + +/** + * The ADR-0131 D10 ceremony's apply step — ceremony items 2 to 4: the backup + * precondition is the command's, and this module performs the fates, in fate + * order, then the post-check. + * + * 1. `attribution` (fate 3) — every NULL row the plan derives an owner for + * takes it. The owners come from the plan's own walk + * ({@link AttributionSink}): the apply writes exactly what the plan + * reported, derived once, never a second derivation. Repeated to a fixed + * point, so a child whose parent was itself NULL in the first pass takes + * the parent's owner in the next. + * 2. `authored-conversion` — the catalog rows an organization authored + * (the inventory's `authored` categories) become environment metadata + * through the metadata door, by core's row→definition mappers; the + * position bindings are carried into the position definitions + * (`convertPositionBindingRows`). Only under `single` (ADR-0131 D3: a + * Setup catalog write is an environment metadata write under `single` and + * refused under a wall); under a wall the rows are reported. + * 3. `rewrite` — the verified id→name rewrite (D4): + * `sys_user_permission_set.permission_set` is named from the set row its + * `permission_set_id` points at, then EVERY grant is verified to name a set + * the security catalog resolves. The id column stays (C7b drops it). + * 4. `mirror-deletion` (fate 2) — gated on the rewrite's verification, which + * is read again immediately before the first delete. Only rows of a + * `mirror-deletion` object that match its mirror selector and none of its + * named categories are deleted. + * 5. `column-drops` (fate 1) — `organization_id` leaves the D7 objects + * through the schema-drift ceremony ({@link ColumnDropPort}), never a hand + * `ALTER`. + * + * Then the post-check re-runs the plan and reports, per table, what remains. + * + * ## Idempotent and resumable + * + * Every write is conditioned on the state it changes — an owner is written + * only where `organization_id IS NULL`, a name only where `permission_set` is + * empty, a delete selects only mirrors — so a second run writes nothing the + * first one wrote. On top of that, each step that finishes is recorded in the + * ceremony's checkpoint ({@link CeremonyLedgerPort}) and is not run again: a + * run interrupted inside step N resumes at step N. A step that stops on a + * blocker stops the ceremony there — fate order is the contract — and the + * report names the blocker and its remedy. + * + * ## What it never does + * + * ⛔ Delete a row outside fate 2. ⛔ Give an unattributable row an owner: + * owners come only from the plan's anchors and, under `single`, the Default + * Organization. ⛔ Run at boot: the command is the only caller. + */ + +/** The `sys_migration` row the ceremony checkpoints into and completes. */ +export const ORGANIZATION_OWNERSHIP_MIGRATION_ID = 'adr-0131-organization-ownership'; + +export const CEREMONY_STEPS = ['attribution', 'authored-conversion', 'rewrite', 'mirror-deletion', 'column-drops'] as const; +export type CeremonyStep = (typeof CEREMONY_STEPS)[number]; + +/** The raw-SQL seam: the plan's reader plus the one write verb. */ +export interface ApplyExecutor extends PlanReader { + execute(sql: string, params?: readonly unknown[]): Promise; +} + +/** The security catalog read (core's `SecurityCatalogReader` satisfies it). */ +export interface CatalogPort { + resolve(type: 'position' | 'permission', name: string): Promise<{ definition: Readonly> } | undefined>; +} + +/** The metadata door: an environment metadata save. Throws when the door refuses. */ +export interface MetadataDoorPort { + save(type: 'position' | 'permission', name: string, body: Record): Promise; +} + +/** One table's column drop, as the schema-drift ceremony answered it. */ +export interface ColumnDropOutcome { + table: string; + outcome: 'dropped' | 'not-proposed' | 'failed'; + detail?: string; +} + +/** Drops `organization_id` on the named tables through the schema-drift ceremony (ADR-0120 D4). */ +export interface ColumnDropPort { + drop(tables: readonly string[]): Promise; +} + +/** What the ceremony records after each finished step. */ +export interface CeremonyCheckpoint { + ceremony: 'ADR-0131 D10'; + inventoryDigest: string; + completed: Partial }>>; +} + +/** Where the checkpoint lives (the command: the `sys_migration` row). */ +export interface CeremonyLedgerPort { + read(): Promise; + write(checkpoint: CeremonyCheckpoint, verdict: { complete: boolean; blocking: number; advisory: number }): Promise; +} + +export interface StepReport { + step: CeremonyStep; + /** `checkpointed`: finished by an earlier run; `not-reached`: an earlier step stopped the run. */ + status: 'done' | 'checkpointed' | 'blocked' | 'not-reached'; + summary: Record; + /** Each blocker, with its remedy, in the operator's terms. */ + blockers: string[]; + /** Reported, never blocking. */ + notes: string[]; +} + +export interface TableOutcome { + object: string; + fate: TablePlan['fate']; + before: { total: number; organizationNull: number } | null; + after: { total: number; organizationNull: number } | null; + attributed: number; + deleted: number; + /** Fate 1: the column is gone after this run. */ + columnDropped: boolean; + /** Rows still NULL and listed by id — reported, never guessed. */ + unattributable: UnattributableRow[]; + /** Post-check: what is left for this table's fate, `0` when the fate is done. */ + remaining: number; + notNull: TablePlan['notNull']; +} + +export interface OrganizationOwnershipApplyReport { + kind: 'organization-ownership-apply'; + ceremony: 'ADR-0131 D10'; + migrationId: typeof ORGANIZATION_OWNERSHIP_MIGRATION_ID; + generatedAt: string; + database: string; + posture: string; + inventoryDigest: string; + /** Steps an earlier run finished, read from the checkpoint. */ + resumedFrom: CeremonyStep[]; + steps: StepReport[]; + tables: TableOutcome[]; + /** Every step finished. */ + complete: boolean; + /** Blockers (an unfinished ceremony) or post-check discrepancies (a finished one). Zero is the verdict. */ + blocking: number; + /** Rows reported, never guessed: unattributable rows and rows held for another stage. */ + advisory: number; +} + +export interface ApplyOptions { + executor: ApplyExecutor; + posture: string; + database: string; + catalog: CatalogPort; + door: MetadataDoorPort; + columnDrops: ColumnDropPort; + ledger: CeremonyLedgerPort; + inventory?: readonly InventoryEntry[]; + now?: () => Date; + /** One line per milestone, for the command's human face. */ + onProgress?: (message: string) => void; +} + +const ORG = 'organization_id'; +const IN_CHUNK = 200; +const PAGE = 500; +const MAX_ATTRIBUTION_PASSES = 8; + +const GRANTS = 'sys_user_permission_set'; +const SETS = 'sys_permission_set'; +const POSITIONS = 'sys_position'; +const BINDINGS = 'sys_position_permission_set'; +const CAPABILITIES = 'sys_capability'; + +const text = (value: unknown): string | null => + value === null || value === undefined || String(value) === '' ? null : String(value); + +/** Shared state of one run. */ +interface Run { + opts: ApplyOptions; + sql: Sql; + plan: OrganizationOwnershipPlan; + columns: Map | null>; + attributed: Map; + deleted: Map; + /** + * What the door accepted this run. The registry hydrates environment + * metadata at boot, so a definition saved a moment ago is not in it until + * the next one; the door's acceptance is what makes it resolve now. + */ + saved: Map>>; +} + +/** The catalog read, then what the door accepted this run. */ +async function resolveName(run: Run, type: 'position' | 'permission', name: string): Promise<{ definition: Readonly> } | undefined> { + const found = await run.opts.catalog.resolve(type, name); + if (found) return found; + const saved = run.saved.get(`${type}|${name}`); + return saved ? { definition: saved } : undefined; +} + +async function saveDefinition(run: Run, type: 'position' | 'permission', name: string, body: Record): Promise { + await run.opts.door.save(type, name, body); + run.saved.set(`${type}|${name}`, body); +} + +async function columnsOf(run: Run, table: string): Promise | null> { + if (run.columns.has(table)) return run.columns.get(table)!; + const present = run.plan.tables.some((t) => t.tables.includes(table)); + let cols: Set | null = null; + if (present) { + const rows = await run.sql.read(table, columnListSql(run.sql.dialect), [table]); + cols = new Set(rows.map(columnNameOf).filter((c): c is string => c !== null)); + } + run.columns.set(table, cols); + return cols; +} + +/** Every row of `table` matching `where`, keyset-paged by id. */ +async function readAll(run: Run, table: string, select: readonly string[], where: { text: string; params: unknown[] } = { text: '1 = 1', params: [] }): Promise[]> { + const { sql } = run; + const t = sql.q(table, table); + const id = sql.q('id', table); + const cols = [...new Set(['id', ...select])].map((c) => `${sql.q(c, table)} AS ${sql.q(c, table)}`).join(', '); + const out: Record[] = []; + let after: unknown; + for (;;) { + const rows = await sql.read( + table, + `SELECT ${cols} FROM ${t} WHERE (${where.text})${after === undefined ? '' : ` AND ${id} > ?`} ORDER BY ${id} LIMIT ${PAGE}`, + after === undefined ? where.params : [...where.params, after], + ); + out.push(...rows); + if (rows.length < PAGE) return out; + after = rows[rows.length - 1]!.id; + } +} + +async function execute(run: Run, table: string, statement: string, params: readonly unknown[]): Promise { + try { + await run.opts.executor.execute(statement, params); + } catch (error) { + throw new Error(`The write to ${table} failed: ${error instanceof Error ? error.message : String(error)}`, { cause: error }); + } +} + +const planFor = (run: Run, extra: { onAttribution?: AttributionSink } = {}): Promise => + buildOrganizationOwnershipPlan({ + reader: run.opts.executor, + posture: run.opts.posture, + database: run.opts.database, + ...(run.opts.inventory ? { inventory: run.opts.inventory } : {}), + ...(run.opts.now ? { now: run.opts.now() } : {}), + ...extra, + }); + +const nullsUnderAttribution = (plan: OrganizationOwnershipPlan): number => + plan.tables.filter((t) => t.fate === 'attribution').reduce((n, t) => n + (t.rows?.organizationNull ?? 0), 0); + +// ── Step 1: attribution ────────────────────────────────────────────────────── + +async function runAttribution(run: Run): Promise { + let passes = 0; + let total = 0; + let lastNulls = Number.POSITIVE_INFINITY; + for (;;) { + passes++; + let wrote = 0; + const plan = await planFor(run, { + onAttribution: async ({ object, assignments }) => { + const groups = new Map(); + for (const a of assignments) { + const key = `${a.table}\u0000${a.organizationId}`; + const ids = groups.get(key) ?? []; + ids.push(a.id); + groups.set(key, ids); + } + for (const [key, ids] of groups) { + const [table, organizationId] = key.split('\u0000') as [string, string]; + const t = run.sql.q(table, table); + for (let i = 0; i < ids.length; i += IN_CHUNK) { + const chunk = ids.slice(i, i + IN_CHUNK); + await execute( + run, + table, + `UPDATE ${t} SET ${run.sql.q(ORG, table)} = ? WHERE ${run.sql.q(ORG, table)} IS NULL AND ${run.sql.q('id', table)} IN (${chunk.map(() => '?').join(', ')})`, + [organizationId, ...chunk], + ); + } + } + wrote += assignments.length; + run.attributed.set(object, (run.attributed.get(object) ?? 0) + assignments.length); + }, + }); + const nulls = nullsUnderAttribution(plan); + if (wrote === 0) break; + if (nulls >= lastNulls || passes >= MAX_ATTRIBUTION_PASSES) { + return { + step: 'attribution', + status: 'blocked', + summary: { rowsAttributed: total, passes }, + blockers: [ + `Attribution did not reach a fixed point: pass ${passes} still derived ${wrote} owner(s) while the NULL rows did ` + + `not decrease (${nulls}). The writes are not landing — check that the database accepts the UPDATE, then re-run.`, + ], + notes: [], + }; + } + total += wrote; + lastNulls = nulls; + run.opts.onProgress?.(`attribution pass ${passes}: ${wrote} row(s) took their organization`); + } + return { step: 'attribution', status: 'done', summary: { rowsAttributed: total, passes }, blockers: [], notes: [] }; +} + +// ── Step 2: authored catalog rows → environment metadata ──────────────────── + +/** The inventory's `authored` selector for a catalog object. */ +function authoredWhere(object: string, inventory: readonly InventoryEntry[] | undefined): RowPredicate | null { + const entry = inventory ? inventory.find((e) => e.object === object) : inventoryEntryFor(object); + return entry?.categories?.find((c) => c.id === 'authored')?.where ?? null; +} + +const rowIsOff = (value: unknown): boolean => value === false || value === 0 || value === '0' || value === 'false'; + +/** The JSON of a definition's own keys, for comparing two bodies. */ +const bodyKey = (body: Readonly>, keys: readonly string[]): string => + JSON.stringify(keys.map((k) => [k, body[k] ?? null])); + +interface CatalogRows { + positions: Record[]; + sets: Record[]; + bindings: Record[]; +} + +async function readCatalogRows(run: Run): Promise { + const read = async (table: string, select: string[]): Promise[]> => { + const cols = await columnsOf(run, table); + if (!cols) return []; + return readAll(run, table, select.filter((c) => cols.has(c))); + }; + return { + positions: await read(POSITIONS, ['name', ORG]), + sets: await read(SETS, ['name', ORG]), + bindings: await read(BINDINGS, ['position_id', 'permission_set_id']), + }; +} + +/** The `permissionSets` each catalog position definition names today, by name. */ +async function declaredPositionSets(run: Run, names: Iterable): Promise> { + const declared = new Map(); + for (const name of new Set(names)) { + const entry = await resolveName(run, 'position', name); + const sets = entry?.definition.permissionSets; + if (Array.isArray(sets)) declared.set(name, sets.map(String)); + } + return declared; +} + +async function runAuthoredConversion(run: Run): Promise { + const blockers: string[] = []; + const notes: string[] = []; + const summary: Record = { converted: 0, alreadyConverted: 0, reported: 0 }; + const authoredRows = async (table: string): Promise[]> => { + const cols = await columnsOf(run, table); + const where = authoredWhere(table, run.opts.inventory); + if (!cols || !where) return []; + return readAll(run, table, [...cols], predicateSql(run.sql, table, cols, where)); + }; + const sets = await authoredRows(SETS); + const positions = await authoredRows(POSITIONS); + const capabilities = await authoredRows(CAPABILITIES); + + if (capabilities.length > 0) { + summary.reported += capabilities.length; + notes.push( + `${capabilities.length} authored ${CAPABILITIES} row(s) are reported, not converted: no runtime reader honours a ` + + 'capability row, so nothing is converted for them (fate 4).', + ); + } + if (sets.length + positions.length === 0 && (await columnsOf(run, BINDINGS)) === null) { + return { step: 'authored-conversion', status: 'done', summary, blockers, notes }; + } + if (run.opts.posture !== 'single') { + summary.reported += sets.length + positions.length; + if (sets.length + positions.length > 0) { + notes.push( + `${sets.length + positions.length} organization-authored catalog row(s) are reported, not converted: under the ` + + `${run.opts.posture} posture a catalog definition is not an environment write (ADR-0131 D3). A grant that ` + + 'names one of them does not resolve, and the rewrite step reports it.', + ); + } + return { step: 'authored-conversion', status: 'done', summary, blockers, notes }; + } + + // Bindings first: an authored position's definition carries the sets its rows bind. + const catalogRows = await readCatalogRows(run); + const conversion = convertPositionBindingRows({ + positions: catalogRows.positions, + permissionSets: catalogRows.sets, + bindings: catalogRows.bindings, + declared: await declaredPositionSets(run, catalogRows.positions.map((r) => String(r.name))), + }); + const boundSets = new Map(); + for (const position of conversion.positions) { + if (position.fate === 'conflicting') { + blockers.push( + `Position ${position.name}: organizations bind different permission sets to it ` + + `(${JSON.stringify(position.byOrganization)}). Merging them would grant one organization's members another's ` + + `sets, so it is never guessed: make the ${BINDINGS} rows agree (or remove the extra ones), then re-run.`, + ); + } else if (position.fate === 'converted') { + boundSets.set(position.name, position.permissionSets); + } + } + + const convert = async ( + type: 'permission' | 'position', + rows: Record[], + toBody: (row: Record) => Record, + ): Promise> => { + const handled = new Set(); + const byName = new Map[]>(); + for (const row of rows) { + const name = text(row.name); + if (!name) { + summary.reported++; + notes.push(`An authored ${type} row (${String(row.id)}) carries no name and is reported, never converted.`); + continue; + } + byName.set(name, [...(byName.get(name) ?? []), row]); + } + for (const [name, group] of byName) { + handled.add(name); + if (group.some((r) => 'active' in r && rowIsOff(r.active))) { + summary.reported += group.length; + notes.push( + `The authored ${type} ${name} is deactivated and is not converted: converting it would make it grant. A grant ` + + 'that names it does not resolve, and the rewrite step reports it.', + ); + continue; + } + const bodies = group.map((row) => { + const body = toBody(row); + const extra = type === 'position' ? boundSets.get(name) : undefined; + return extra ? { ...body, permissionSets: [...extra] } : body; + }); + const keys = Object.keys(bodies[0]!); + if (new Set(bodies.map((b) => bodyKey(b, [...new Set(bodies.flatMap(Object.keys))]))).size > 1) { + blockers.push( + `The authored ${type} ${name} is held by ${group.length} rows (${group.map((r) => String(r.id)).join(', ')}) ` + + 'that define it differently. Which one the environment keeps is never guessed: make them agree, or ' + + 'rename all but one, then re-run.', + ); + continue; + } + const body = bodies[0]!; + const existing = await resolveName(run, type, name); + if (existing) { + if (bodyKey(existing.definition, keys) === bodyKey(body, keys)) { + summary.alreadyConverted++; + continue; + } + blockers.push( + `The authored ${type} ${name} differs from the definition the catalog already holds under that name. The ` + + 'catalog\'s definition stays; rename the authored row (and the grants that name it), then re-run.', + ); + continue; + } + try { + await saveDefinition(run, type, name, body); + summary.converted++; + run.opts.onProgress?.(`converted the authored ${type} ${name} to environment metadata`); + } catch (error) { + blockers.push( + `The metadata door refused the authored ${type} ${name}: ${error instanceof Error ? error.message : String(error)}. ` + + 'Correct the row so the definition validates, then re-run.', + ); + } + } + return handled; + }; + + await convert('permission', sets, (row) => permissionSetDefinitionFromRow(row)); + const authoredPositions = await convert('position', positions, (row) => positionDefinitionFromRow(row)); + + // A declared position whose binding rows add sets its definition lacks. + for (const [name, permissionSets] of boundSets) { + if (authoredPositions.has(name)) continue; + const existing = await resolveName(run, 'position', name); + if (!existing) { + blockers.push( + `Binding rows give position ${name} the sets ${permissionSets.join(', ')}, and the catalog holds no position of ` + + 'that name to carry them. Declare the position (or remove the bindings), then re-run.', + ); + continue; + } + try { + await saveDefinition(run, 'position', name, { ...existing.definition, permissionSets: [...permissionSets] }); + summary.converted++; + } catch (error) { + blockers.push( + `Binding rows give the declared position ${name} the sets ${permissionSets.join(', ')}, which its definition ` + + `does not name, and the metadata door refused the change: ${error instanceof Error ? error.message : String(error)}. ` + + `Declare those sets on the position (permissionSets), or remove the ${BINDINGS} rows, then re-run.`, + ); + } + } + if (conversion.dangling.length > 0) { + notes.push( + `${conversion.dangling.length} ${BINDINGS} row(s) name a position or set row that does not exist; they bind nothing ` + + 'and are deleted with the other binding mirrors.', + ); + } + return { step: 'authored-conversion', status: blockers.length ? 'blocked' : 'done', summary, blockers, notes }; +} + +// ── Step 3: the verified id→name rewrite (D4) ─────────────────────────────── + +interface Verification { + /** Blockers: a reference that does not resolve by name, each with its remedy. */ + blockers: string[]; + notes: string[]; + grants: number; + verified: number; +} + +/** Read-only: does every reference resolve by name? The gate of fate 2. */ +async function verifyReferences(run: Run): Promise { + const out: Verification = { blockers: [], notes: [], grants: 0, verified: 0 }; + const cols = await columnsOf(run, GRANTS); + if (cols) { + if (!cols.has('permission_set')) { + out.blockers.push( + `${GRANTS} has no permission_set column, so no grant can name its set. Boot this release once against the ` + + 'database (its schema sync adds the column), then re-run.', + ); + return out; + } + const setCols = await columnsOf(run, SETS); + const setNames = new Map(); + if (setCols) for (const row of await readAll(run, SETS, ['name'])) setNames.set(String(row.id), text(row.name)); + const grants = await readAll(run, GRANTS, ['permission_set', ...(cols.has('permission_set_id') ? ['permission_set_id'] : [])]); + const resolved = new Map(); + const unnamed: string[] = []; + const unresolved = new Map(); + const disagree: string[] = []; + for (const grant of grants) { + out.grants++; + const name = text(grant.permission_set); + if (!name) { + unnamed.push(String(grant.id)); + continue; + } + if (!resolved.has(name)) resolved.set(name, (await resolveName(run, 'permission', name)) !== undefined); + if (!resolved.get(name)) { + unresolved.set(name, [...(unresolved.get(name) ?? []), String(grant.id)]); + continue; + } + const id = text(grant.permission_set_id); + if (id && setNames.has(id) && setNames.get(id) !== name) { + disagree.push(String(grant.id)); + continue; + } + out.verified++; + } + if (unnamed.length) { + out.blockers.push( + `${unnamed.length} grant(s) still name no permission set (${unnamed.slice(0, 20).join(', ')}${unnamed.length > 20 ? ', …' : ''}). ` + + 'The rewrite could not derive their name (see its notes); name or remove each, then re-run.', + ); + } + for (const [name, ids] of unresolved) { + out.blockers.push( + `${ids.length} grant(s) name the permission set ${name}, which the security catalog does not resolve ` + + `(${ids.slice(0, 20).join(', ')}${ids.length > 20 ? ', …' : ''}). Declare or author the set, or remove the grants, then re-run.`, + ); + } + if (disagree.length) { + out.blockers.push( + `${disagree.length} grant(s) name a set other than the one their permission_set_id points at (${disagree.slice(0, 20).join(', ')}). ` + + 'Correct the name, then re-run.', + ); + } + } + if (await columnsOf(run, BINDINGS)) { + const rows = await readCatalogRows(run); + const conversion = convertPositionBindingRows({ + positions: rows.positions, + permissionSets: rows.sets, + bindings: rows.bindings, + declared: await declaredPositionSets(run, rows.positions.map((r) => String(r.name))), + }); + for (const position of conversion.positions) { + if (position.fate === 'declared') continue; + out.blockers.push( + position.fate === 'conflicting' + ? `Position ${position.name}: organizations bind different permission sets to it; the bindings cannot be carried into one definition.` + : `Binding rows give position ${position.name} sets its catalog definition does not name (${position.permissionSets.join(', ')}); ` + + 'deleting them would lose a grant. The authored-conversion step carries them; resolve its blocker, then re-run.', + ); + } + } + return out; +} + +async function runRewrite(run: Run): Promise { + const summary: Record = { named: 0, grants: 0, verified: 0 }; + const notes: string[] = []; + const cols = await columnsOf(run, GRANTS); + if (cols?.has('permission_set') && cols.has('permission_set_id')) { + const sets = new Map(); + if (await columnsOf(run, SETS)) { + const setCols = (await columnsOf(run, SETS))!; + for (const row of await readAll(run, SETS, ['name', ...(setCols.has(ORG) ? [ORG] : [])])) { + sets.set(String(row.id), { name: text(row.name), organization: text(row[ORG]) }); + } + } + const t = run.sql.q(GRANTS, GRANTS); + const nameCol = run.sql.q('permission_set', GRANTS); + const unnamed = await readAll(run, GRANTS, ['permission_set_id', ...(cols.has(ORG) ? [ORG] : [])], { + text: `${nameCol} IS NULL OR ${nameCol} = ''`, + params: [], + }); + const reasons = { noId: [] as string[], dangling: [] as string[], crossOrganization: [] as string[], unresolved: [] as string[] }; + for (const grant of unnamed) { + const gid = String(grant.id); + const setId = text(grant.permission_set_id); + if (!setId) { reasons.noId.push(gid); continue; } + const set = sets.get(setId); + if (!set?.name) { reasons.dangling.push(gid); continue; } + // A name carries no organization: an organization's set row may be named + // only by that organization's grants (the grant rule the resolver applies). + if (set.organization !== null && set.organization !== text(grant[ORG])) { reasons.crossOrganization.push(gid); continue; } + if (!(await resolveName(run, 'permission', set.name))) { reasons.unresolved.push(gid); continue; } + await execute( + run, + GRANTS, + `UPDATE ${t} SET ${nameCol} = ? WHERE ${run.sql.q('id', GRANTS)} = ? AND (${nameCol} IS NULL OR ${nameCol} = '')`, + [set.name, gid], + ); + summary.named++; + } + const describe: Record = { + noId: 'carry neither a name nor a permission_set_id', + dangling: 'point at a set row that does not exist', + crossOrganization: 'point at another organization\'s set row — a name carries no organization, so it is never written', + unresolved: 'point at a set row whose name the security catalog does not resolve', + }; + for (const key of Object.keys(reasons) as (keyof typeof reasons)[]) { + if (reasons[key].length) notes.push(`${reasons[key].length} unnamed grant(s) ${describe[key]}: ${reasons[key].slice(0, 20).join(', ')}.`); + } + if (summary.named) run.opts.onProgress?.(`rewrite: ${summary.named} grant(s) now name their permission set`); + } + const verification = await verifyReferences(run); + summary.grants = verification.grants; + summary.verified = verification.verified; + return { + step: 'rewrite', + status: verification.blockers.length ? 'blocked' : 'done', + summary, + blockers: verification.blockers, + notes: [...notes, ...verification.notes], + }; +} + +// ── Step 4: mirror deletion (fate 2) ───────────────────────────────────────── + +/** + * The rows of a fate-2 object that are deleted: its mirror selector (every + * row when it names none), minus every row a named category of the object + * claims — authored catalog rows, customized templates awaiting promotion. + * `CASE … END = 0` so a category that evaluates NULL on a row still claims it. + */ +function deletableWhere(run: Run, table: string, cols: ReadonlySet, entry: InventoryEntry): { text: string; params: unknown[] } { + const parts: string[] = []; + const params: unknown[] = []; + if (entry.mirror) { + const mirror = predicateSql(run.sql, table, cols, entry.mirror); + parts.push(`(${mirror.text})`); + params.push(...mirror.params); + } + for (const category of entry.categories ?? []) { + if (!category.where || category.conflictBy) continue; + const claimed = predicateSql(run.sql, table, cols, category.where); + parts.push(`(CASE WHEN ${claimed.text} THEN 1 ELSE 0 END) = 0`); + params.push(...claimed.params); + } + return { text: parts.length ? parts.join(' AND ') : '1 = 1', params }; +} + +const mirrorEntry = (run: Run, object: string): InventoryEntry | undefined => + (run.opts.inventory ? run.opts.inventory.find((e) => e.object === object) : inventoryEntryFor(object)); + +async function countWhere(run: Run, table: string, where: { text: string; params: unknown[] }): Promise { + return run.sql.count(table, `SELECT COUNT(*) AS n FROM ${run.sql.q(table, table)} WHERE ${where.text}`, where.params); +} + +async function runMirrorDeletion(run: Run): Promise { + // The gate, read again now: a grant written since the rewrite ran is a reference too. + const gate = await verifyReferences(run); + if (gate.blockers.length) { + return { + step: 'mirror-deletion', + status: 'blocked', + summary: { deleted: 0 }, + blockers: [`Mirror deletion is gated on every reference resolving by name, and it no longer does:`, ...gate.blockers], + notes: [], + }; + } + const notes: string[] = []; + let total = 0; + for (const table of run.plan.tables) { + if (table.fate !== 'mirror-deletion' || table.physical !== 'present') continue; + const entry = mirrorEntry(run, table.object); + if (!entry) continue; + for (const physical of table.tables) { + const cols = (await columnsOf(run, physical))!; + const where = deletableWhere(run, physical, cols, entry); + const before = await countWhere(run, physical, where); + if (before === 0) continue; + await execute(run, physical, `DELETE FROM ${run.sql.q(physical, physical)} WHERE ${where.text}`, where.params); + const left = await countWhere(run, physical, where); + if (left !== 0) { + throw new Error(`The mirror delete on ${physical} left ${left} mirror row(s) behind; nothing past this table was deleted.`); + } + run.deleted.set(table.object, (run.deleted.get(table.object) ?? 0) + before); + total += before; + run.opts.onProgress?.(`mirror deletion: ${before} row(s) of ${physical}`); + } + const kept = Math.max(0, table.rows!.total - (run.deleted.get(table.object) ?? 0)); + if (kept > 0) notes.push(`${table.object}: ${kept} row(s) are not mirrors (a named category claims them) and stay.`); + } + return { step: 'mirror-deletion', status: 'done', summary: { deleted: total }, blockers: [], notes }; +} + +// ── Step 5: column drops (fate 1) ──────────────────────────────────────────── + +async function runColumnDrops(run: Run): Promise { + const plan = await planFor(run); + const blockers: string[] = []; + const due: string[] = []; + for (const table of plan.tables) { + if (table.fate !== 'column-drop' || table.physical !== 'present' || table.organizationColumn !== 'present') continue; + const held = table.categories.filter((c) => c.rows > 0); + if (held.length) { + blockers.push( + `${table.object}.${ORG} is not dropped: ${held.map((c) => `${c.rows} row(s) of "${c.label}"`).join('; ')}. ` + + 'Those rows need their own step before the column can go; resolve them, then re-run.', + ); + continue; + } + due.push(...table.tables); + } + let dropped = 0; + if (due.length) { + for (const outcome of await run.opts.columnDrops.drop(due)) { + if (outcome.outcome === 'dropped') { + dropped++; + continue; + } + blockers.push( + outcome.outcome === 'not-proposed' + ? `${outcome.table}.${ORG} was not proposed for dropping by the schema-drift ceremony: the object this ` + + `runtime declares still carries the column${outcome.detail ? ` (${outcome.detail})` : ''}. Run the ceremony ` + + 'with the release whose declaration drops it.' + : `Dropping ${outcome.table}.${ORG} failed: ${outcome.detail ?? 'no detail'}. Re-run once the cause is fixed.`, + ); + } + } + return { step: 'column-drops', status: blockers.length ? 'blocked' : 'done', summary: { dropped, due: due.length }, blockers, notes: [] }; +} + +// ── The run ────────────────────────────────────────────────────────────────── + +const RUNNERS: Record Promise> = { + attribution: runAttribution, + 'authored-conversion': runAuthoredConversion, + rewrite: runRewrite, + 'mirror-deletion': runMirrorDeletion, + 'column-drops': runColumnDrops, +}; + +/** What a table still owes its fate, read from the post-check plan. */ +async function remainingFor(run: Run, table: TablePlan, complete: boolean): Promise { + if (table.physical !== 'present' || !table.rows) return 0; + switch (table.fate) { + case 'attribution': + return Number(table.fateCounts.rowsToAttribute ?? 0); + case 'mirror-deletion': { + const entry = mirrorEntry(run, table.object); + if (!entry) return 0; + let left = 0; + for (const physical of table.tables) { + left += await countWhere(run, physical, deletableWhere(run, physical, (await columnsOf(run, physical))!, entry)); + } + return left; + } + case 'column-drop': + return complete && table.organizationColumn === 'present' && table.categories.every((c) => c.rows === 0) ? 1 : 0; + default: + return 0; + } +} + +/** + * Run the ceremony's apply: the fates in order, resuming from the checkpoint, + * then the post-check. Throws only for a failure mid-write (the checkpoint + * holds every step finished before it) or a plan refusal; every blocker is in + * the report. + */ +export async function applyOrganizationOwnership(opts: ApplyOptions): Promise { + const dialect = dialectOf(opts.executor.client); + const now = opts.now ?? (() => new Date()); + const run: Run = { + opts, + sql: new Sql(opts.executor, dialect ?? 'sqlite'), + plan: undefined as unknown as OrganizationOwnershipPlan, + columns: new Map(), + attributed: new Map(), + deleted: new Map(), + saved: new Map(), + }; + // Refuses (driver, catalog, an uninventoried table) before anything is written. + run.plan = await planFor(run); + const before = run.plan; + + const stored = await opts.ledger.read(); + const checkpoint: CeremonyCheckpoint = + stored && stored.ceremony === 'ADR-0131 D10' && stored.inventoryDigest === before.inventoryDigest + ? { ceremony: 'ADR-0131 D10', inventoryDigest: before.inventoryDigest, completed: { ...stored.completed } } + : { ceremony: 'ADR-0131 D10', inventoryDigest: before.inventoryDigest, completed: {} }; + const resumedFrom = CEREMONY_STEPS.filter((s) => checkpoint.completed[s]); + const steps: StepReport[] = []; + if (stored && resumedFrom.length === 0 && Object.keys(stored.completed ?? {}).length > 0) { + opts.onProgress?.('the inventory changed since the recorded checkpoint, so every step runs again (each is idempotent)'); + } + + let stopped = false; + for (const step of CEREMONY_STEPS) { + if (stopped) { + steps.push({ step, status: 'not-reached', summary: {}, blockers: [], notes: [] }); + continue; + } + const done = checkpoint.completed[step]; + if (done) { + steps.push({ step, status: 'checkpointed', summary: done.summary, blockers: [], notes: [] }); + continue; + } + opts.onProgress?.(`step ${step}…`); + const report = await RUNNERS[step](run); + steps.push(report); + if (report.status !== 'done') { + stopped = true; + continue; + } + checkpoint.completed[step] = { at: now().toISOString(), summary: report.summary }; + await opts.ledger.write(checkpoint, { complete: false, blocking: 0, advisory: 0 }); + // Later steps read the tables as this one left them. + run.columns.clear(); + } + + // Post-check: the plan again, read-only. + run.columns.clear(); + const after = await planFor(run); + const complete = CEREMONY_STEPS.every((s) => checkpoint.completed[s]); + const tables: TableOutcome[] = []; + const objects = new Set([...before.tables, ...after.tables].map((t) => t.object)); + let discrepancies = 0; + for (const object of [...objects].sort()) { + const b = before.tables.find((t) => t.object === object); + const a = after.tables.find((t) => t.object === object); + const ref = a ?? b!; + const remaining = a ? await remainingFor(run, a, complete) : 0; + if (complete) discrepancies += remaining; + tables.push({ + object, + fate: ref.fate, + before: b?.rows ? { total: b.rows.total, organizationNull: b.rows.organizationNull } : null, + after: a?.rows ? { total: a.rows.total, organizationNull: a.rows.organizationNull } : null, + attributed: run.attributed.get(object) ?? 0, + deleted: run.deleted.get(object) ?? 0, + columnDropped: b?.organizationColumn === 'present' && a?.organizationColumn === 'absent', + unattributable: a?.unattributable ?? [], + remaining, + notNull: ref.notNull, + }); + } + const advisory = after.summary.unattributableRows; + const blocking = complete ? discrepancies : steps.reduce((n, s) => n + s.blockers.length, 0) || 1; + await opts.ledger.write(checkpoint, { complete, blocking, advisory }); + return { + kind: 'organization-ownership-apply', + ceremony: 'ADR-0131 D10', + migrationId: ORGANIZATION_OWNERSHIP_MIGRATION_ID, + generatedAt: now().toISOString(), + database: opts.database, + posture: opts.posture, + inventoryDigest: before.inventoryDigest, + resumedFrom, + steps, + tables, + complete, + blocking, + advisory, + }; +} + +/** Parse a checkpoint from the ledger row's `details`, or `null` when it holds none. */ +export function checkpointFromDetails(details: unknown): CeremonyCheckpoint | null { + let parsed: unknown = details; + if (typeof details === 'string') { + try { + parsed = JSON.parse(details); + } catch { + return null; + } + } + const cp = (parsed as { checkpoint?: unknown } | null)?.checkpoint as CeremonyCheckpoint | undefined; + if (!cp || cp.ceremony !== 'ADR-0131 D10' || typeof cp.inventoryDigest !== 'string' || typeof cp.completed !== 'object') return null; + return { ceremony: cp.ceremony, inventoryDigest: cp.inventoryDigest, completed: { ...cp.completed } }; +} diff --git a/packages/cli/src/utils/organization-ownership-plan.ts b/packages/cli/src/utils/organization-ownership-plan.ts index 697069c86d..f3a384ebd8 100644 --- a/packages/cli/src/utils/organization-ownership-plan.ts +++ b/packages/cli/src/utils/organization-ownership-plan.ts @@ -85,6 +85,27 @@ export interface UnattributableRow { shard?: string; } +/** + * One NULL-organization row the walk derived an owner for. The plan only + * counts these; `--apply` hands a sink to the same walk so the owner it writes + * is the owner the plan reported, derived once ({@link AttributionSink}). + */ +export interface AttributionAssignment { + id: string; + /** The physical table the row lives in (a rotation shard, or the object's own table). */ + table: string; + organizationId: string; + /** The anchor that answered, as the plan's `attribution` counts label it. */ + via: string; +} + +/** + * Receives each page of derived owners, after the page is classified and + * before the walk reads the next. The plan itself never writes; a sink is the + * one place a caller may. + */ +export type AttributionSink = (batch: { object: string; assignments: AttributionAssignment[] }) => Promise; + export interface PlannedCategory { id: string; label: string; @@ -176,7 +197,8 @@ export function inventoryDigest(inventory: readonly InventoryEntry[] = ORGANIZAT return createHash('sha256').update(JSON.stringify(inventory)).digest('hex'); } -class Sql { +/** Identifier quoting, value binding and refusing reads, shared with the apply step. */ +export class Sql { constructor( private readonly reader: PlanReader, readonly dialect: PlanDialect, @@ -231,7 +253,7 @@ function firstValue(row: Record): unknown { return Object.values(row)[0]; } -function columnNameOf(row: Record): string | null { +export function columnNameOf(row: Record): string | null { for (const key of ['column_name', 'COLUMN_NAME', 'name']) { const value = row[key]; if (typeof value === 'string' && value) return value; @@ -247,7 +269,7 @@ function tableNameOf(row: Record): string | null { return null; } -function columnListSql(dialect: PlanDialect): string { +export function columnListSql(dialect: PlanDialect): string { if (dialect === 'sqlite') return 'SELECT name AS column_name FROM pragma_table_info(?)'; if (dialect === 'postgres') { return 'SELECT column_name FROM information_schema.columns WHERE table_schema = ANY (current_schemas(false)) AND table_name = ?'; @@ -256,7 +278,7 @@ function columnListSql(dialect: PlanDialect): string { } /** The SQL of one predicate, its columns required present on `table`. */ -function predicateSql(sql: Sql, table: string, columns: ReadonlySet, predicate: RowPredicate): { text: string; params: unknown[] } { +export function predicateSql(sql: Sql, table: string, columns: ReadonlySet, predicate: RowPredicate): { text: string; params: unknown[] } { const parts: string[] = []; const params: unknown[] = []; for (const condition of predicate) { @@ -281,7 +303,7 @@ function predicateSql(sql: Sql, table: string, columns: ReadonlySet, pre return { text: parts.length ? parts.join(' AND ') : '1 = 1', params }; } -function requireColumn(table: string, columns: ReadonlySet, column: string): void { +export function requireColumn(table: string, columns: ReadonlySet, column: string): void { if (!columns.has(column)) { throw new OrganizationOwnershipPlanRefusal( 'missing-column', @@ -378,6 +400,7 @@ interface PlanContext { catalog: Catalog; posture: string; defaultOrganizationId: string | null; + onAttribution?: AttributionSink; } /** The organization of each id in `ids` on `object`'s own table, or a reason it cannot be read. */ @@ -412,9 +435,19 @@ interface PendingRow { } /** Resolve rows through one anchor; returns the rows it could not derive. */ -async function applyAnchor(ctx: PlanContext, anchor: Anchor, rows: PendingRow[], derived: Record): Promise { +async function applyAnchor( + ctx: PlanContext, + anchor: Anchor, + rows: PendingRow[], + derived: Record, + assigned: AttributionAssignment[], +): Promise { const label = anchorLabel(anchor); const left: PendingRow[] = []; + const derive = (row: PendingRow, organizationId: string): void => { + derived[label] = (derived[label] ?? 0) + 1; + assigned.push({ id: row.id, table: row.shard, organizationId, via: label }); + }; if (anchor.kind === 'parent') { const keys = [...new Set(rows.map((r) => nonEmpty(r.values[anchor.childKey])).filter((k): k is string => k !== null))]; const orgs = keys.length ? await organizationsById(ctx, anchor.parentObject, keys) : new Map(); @@ -424,7 +457,7 @@ async function applyAnchor(ctx: PlanContext, anchor: Anchor, rows: PendingRow[], else if (typeof orgs === 'string') left.push({ ...row, reason: orgs }); else if (!orgs.has(key)) left.push({ ...row, reason: `${anchor.parentObject} row ${key} not found` }); else if (orgs.get(key) === null) left.push({ ...row, reason: `${anchor.parentObject} row ${key} has no organization` }); - else derived[label] = (derived[label] ?? 0) + 1; + else derive(row, orgs.get(key)!); } return left; } @@ -450,7 +483,7 @@ async function applyAnchor(ctx: PlanContext, anchor: Anchor, rows: PendingRow[], if (typeof orgs === 'string') left.push({ ...row, reason: orgs }); else if (!orgs.has(id)) left.push({ ...row, reason: `${object} row ${id} not found` }); else if (orgs.get(id) === null) left.push({ ...row, reason: `${object} row ${id} has no organization` }); - else derived[label] = (derived[label] ?? 0) + 1; + else derive(row, orgs.get(id)!); } } return left; @@ -482,7 +515,7 @@ async function applyAnchor(ctx: PlanContext, anchor: Anchor, rows: PendingRow[], const orgs = holders.get(row.id); if (!orgs) left.push({ ...row, reason: `no stamped ${anchor.holderObject} holder` }); else if (orgs.size > 1) left.push({ ...row, reason: `${anchor.holderObject} holders disagree (${orgs.size} organizations)` }); - else derived[label] = (derived[label] ?? 0) + 1; + else derive(row, [...orgs][0]!); } return left; } @@ -669,16 +702,21 @@ async function walkNullRows( } pending.push({ id: String(row.row_id), values: row, reason: entry.fate === 'report' ? 'fate 4: reported, never guessed' : 'no anchor names the owner', shard: table }); } + const assigned: AttributionAssignment[] = []; if (entry.fate === 'attribution') { for (const anchor of entry.anchors ?? []) { if (pending.length === 0) break; - pending = await applyAnchor(ctx, anchor, pending, derived); + pending = await applyAnchor(ctx, anchor, pending, derived, assigned); } if (pending.length > 0 && ctx.posture === 'single' && ctx.defaultOrganizationId !== null) { derived['default-organization'] = (derived['default-organization'] ?? 0) + pending.length; + for (const row of pending) { + assigned.push({ id: row.id, table: row.shard, organizationId: ctx.defaultOrganizationId, via: 'default-organization' }); + } pending = []; } } + if (ctx.onAttribution && assigned.length > 0) await ctx.onAttribution({ object: entry.object, assignments: assigned }); for (const row of pending) { plan.unattributable.push({ id: row.id, @@ -725,6 +763,11 @@ export async function buildOrganizationOwnershipPlan(opts: { database: string; now?: Date; inventory?: readonly InventoryEntry[]; + /** + * `--apply`'s attribution step: receives every owner the walk derives. The + * plan's own run passes none and stays SELECT-only. + */ + onAttribution?: AttributionSink; }): Promise { const inventory = opts.inventory ?? ORGANIZATION_OWNERSHIP_INVENTORY; const dialect = dialectOf(opts.reader.client); @@ -754,7 +797,13 @@ export async function buildOrganizationOwnershipPlan(opts: { ); defaultOrganizationId = rows.length > 0 ? nonEmpty(rows[0]!.id) : null; } - const ctx: PlanContext = { sql, catalog, posture: opts.posture, defaultOrganizationId }; + const ctx: PlanContext = { + sql, + catalog, + posture: opts.posture, + defaultOrganizationId, + ...(opts.onAttribution ? { onAttribution: opts.onAttribution } : {}), + }; const tables: TablePlan[] = []; let outsideCeremony = 0; diff --git a/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts b/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts index 6dd09036b8..9d794dc173 100644 --- a/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts +++ b/packages/cli/src/utils/schema-migrate.one-shot-family.integration.test.ts @@ -192,8 +192,11 @@ const CALLERS: Record = { label: 'migrate organization-ownership', argv: ['--out', '@DIR@/organization-ownership-plan.json', '--database-url', '@DB@', '--json'], }], - write: [], - note: 'the ADR-0131 D10 preflight: it has no writing mode until the ceremony\'s apply lands', + // `--apply` writes the ceremony's fates and its report file; never a seed. + write: [{ + label: 'migrate organization-ownership --apply', + argv: ['--apply', '--backup-taken', '--out', '@DIR@/organization-ownership-apply.json', '--database-url', '@DB@', '--json'], + }], }, 'commands/migrate/plan.ts': { run: invoke(MigratePlan), diff --git a/packages/cli/src/utils/schema-migrate.ts b/packages/cli/src/utils/schema-migrate.ts index a3b8cb6ef2..8890d00121 100644 --- a/packages/cli/src/utils/schema-migrate.ts +++ b/packages/cli/src/utils/schema-migrate.ts @@ -605,7 +605,10 @@ export async function bootSchemaStack( * Compose the SAME object set this deployment's `os serve` boot registers — * its `objectstack.config.ts` and the platform floor `serve` composes * unconditionally (#12938). Set by the two SCHEMA commands, `os migrate - * plan` and `os migrate apply`, and by nothing else. + * plan` and `os migrate apply`; by the commands whose subject is that same + * set (`unmapped-columns`, `security-catalog-overlays`); and by `os migrate + * organization-ownership --apply`, whose column drops are that set's drift + * and whose security catalog is the one the deployment serves. * * Off by default and opted into at the call site rather than deduced here: * the DATA subcommands declare their own, narrower set through @@ -701,7 +704,8 @@ export async function bootSchemaStack( * * Off by default and opted into at the call site, like `composeHostStack`: * `os migrate plan` / `apply` compose no tier-gated plugin, and this does - * not change that. See `buildSchemaMigrationPlugins`'s `authGatedSecurity`. + * not change that. `os migrate organization-ownership --apply` sets it so + * the platform's shipped permission sets are names its catalog resolves. See `buildSchemaMigrationPlugins`'s `authGatedSecurity`. */ composeAuthGatedSecurity?: boolean; /** @@ -721,9 +725,10 @@ export async function bootSchemaStack( * family behind its auth gate, the provider of every capability its * resolver mounts (the stack's `requires` and the always-on slate), and * the REST API plugin. Set by `os migrate plan` and `os migrate apply`, - * whose subject is the deployment's whole object set, and by `os migrate - * unmapped-columns`, which reads the plan's own findings over that set — - * and by nothing else. + * whose subject is the deployment's whole object set, by `os migrate + * unmapped-columns`, which reads the plan's own findings over that set, + * and by `os migrate organization-ownership --apply`, which drops the + * ceremony's columns through that set's drift — and by nothing else. * See `buildSchemaMigrationPlugins`'s `servedPlatform`. */ composeServedPlatform?: boolean; diff --git a/packages/cli/test/json-stdout-purity.e2e.test.ts b/packages/cli/test/json-stdout-purity.e2e.test.ts index 0bb95fb871..84bb341f83 100644 --- a/packages/cli/test/json-stdout-purity.e2e.test.ts +++ b/packages/cli/test/json-stdout-purity.e2e.test.ts @@ -99,8 +99,9 @@ const FAMILY: Record = { 'migrate files-to-references': [], 'migrate meta': ['--stored'], 'migrate multi-value-columns': [], - // Its bare form is its only mode — a read-only plan (`deferSchemaDdl` + - // `readOnlyProbe`). The fixture's database holds none of the platform tables + // Its bare form — a read-only plan (`deferSchemaDdl` + `readOnlyProbe`) — + // is the one driven here; `--apply` emits through the same two `emitJson` + // paths after the same shutdown. The fixture's database holds none of the platform tables // the ceremony inventories, so the face driven here is its REFUSAL face // (`plan_refused`, `emitJson(…, 1, { compact: true })`), emitted after the // stack is shut down. A refused plan writes no plan file, so the fixture From 691bb957bb222e396fd1e434928af53a2a8461d1 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 13 Oct 2026 06:12:32 +0000 Subject: [PATCH 2/5] =?UTF-8?q?feat(core):=20emailTemplateDefinitionFromRo?= =?UTF-8?q?w=20=E2=80=94=20a=20sys=5Femail=5Ftemplate=20row=20as=20the=20e?= =?UTF-8?q?mail=5Ftemplate=20definition=20it=20carries?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The pure row-to-definition mapper beside permissionSetDefinitionFromRow and positionDefinitionFromRow, for the v18 ceremony's promotion of customized template rows (ADR-0131 D10, ruling C on section 6 Q1). No import, so the lean entry carries it. Pinned against EmailTemplateDefinitionSchema. Claude-Session: https://claude.ai/code/session_01DjKf52bcyXa3fbGW723phR Co-Authored-By: Claude --- .../email-template-row-definition.test.ts | 108 ++++++++++++++++++ .../security/email-template-row-definition.ts | 106 +++++++++++++++++ packages/core/src/security/index.ts | 4 + 3 files changed, 218 insertions(+) create mode 100644 packages/core/src/security/email-template-row-definition.test.ts create mode 100644 packages/core/src/security/email-template-row-definition.ts diff --git a/packages/core/src/security/email-template-row-definition.test.ts b/packages/core/src/security/email-template-row-definition.test.ts new file mode 100644 index 0000000000..02c68afa4f --- /dev/null +++ b/packages/core/src/security/email-template-row-definition.test.ts @@ -0,0 +1,108 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [ADR-0131 D6/D10, ruling C] The pin of `emailTemplateDefinitionFromRow` — the + * mapping the upgrade ceremony applies to promote a customized + * `sys_email_template` row to an environment `email_template` item through the + * metadata door. The expected bodies are the outputs the module doc states, + * written out by hand; the schema is read only to hold the body to it. + */ + +import { describe, expect, it } from 'vitest'; +import { EmailTemplateDefinitionSchema } from '@objectstack/spec/system'; +import { emailTemplateDefinitionFromRow } from './email-template-row-definition.js'; + +/** A row as `sys_email_template` stores it, its state and provenance riding along. */ +const storedRow = { + id: 'tpl_01', + name: 'crm.welcome', + label: 'Welcome', + category: 'notification', + locale: 'zh-CN', + description: 'Sent on sign-up', + subject: 'Welcome, {{user.name}}', + body_html: '

Hello {{user.name}}

', + body_text: 'Hello {{user.name}}', + from_name: 'Acme Support', + from_address: 'support@acme.test', + reply_to: 'help@acme.test', + active: 0, + is_system: 1, + variables_json: '[{"name":"user.name","type":"string","required":true}]', + managed_by: 'package', + customized: 1, + organization_id: 'org_1', + created_at: '2026-01-01T00:00:00.000Z', + updated_at: '2026-01-02T00:00:00.000Z', +}; + +const schemaKeys = (): Set => + new Set(Object.keys((EmailTemplateDefinitionSchema as unknown as { shape: Record }).shape)); + +describe('emailTemplateDefinitionFromRow', () => { + it('answers the definition the row carries, in order, with no state or provenance column', () => { + expect(emailTemplateDefinitionFromRow(storedRow)).toStrictEqual({ + name: 'crm.welcome', + label: 'Welcome', + locale: 'zh-CN', + subject: 'Welcome, {{user.name}}', + bodyHtml: '

Hello {{user.name}}

', + bodyText: 'Hello {{user.name}}', + variables: [{ name: 'user.name', type: 'string', required: true }], + fromOverride: { name: 'Acme Support', address: 'support@acme.test' }, + replyTo: 'help@acme.test', + active: false, + description: 'Sent on sign-up', + }); + }); + + it('answers a body the email_template schema accepts whole, with no key the schema does not declare', () => { + const body = emailTemplateDefinitionFromRow(storedRow); + const keys = schemaKeys(); + expect(Object.keys(body).filter((k) => !keys.has(k))).toEqual([]); + expect(EmailTemplateDefinitionSchema.safeParse(body).success).toBe(true); + }); + + it('falls back for the sparse row: label to name, locale to en-US, bodies to empty, active to true', () => { + const body = emailTemplateDefinitionFromRow({ + name: 'auth.reset', + label: '', + locale: null, + subject: null, + body_html: null, + body_text: '', + from_name: 'Nobody', + from_address: '', + reply_to: null, + variables_json: 'not json', + description: null, + }); + expect(body).toStrictEqual({ + name: 'auth.reset', + label: 'auth.reset', + locale: 'en-US', + subject: '', + bodyHtml: '', + variables: [], + active: true, + }); + expect(EmailTemplateDefinitionSchema.safeParse(body).success).toBe(true); + }); + + it('reads active the way the stored spellings mean it, and variables already parsed', () => { + for (const off of [false, 0, '0', 'false']) { + expect(emailTemplateDefinitionFromRow({ name: 'a', active: off }).active).toBe(false); + } + for (const on of [true, 1, '1', 'true', undefined, null]) { + expect(emailTemplateDefinitionFromRow({ name: 'a', active: on }).active).toBe(true); + } + expect(emailTemplateDefinitionFromRow({ name: 'a', variables_json: [{ name: 'x' }] }).variables).toEqual([{ name: 'x' }]); + expect(emailTemplateDefinitionFromRow({ name: 'a', variables_json: '{"name":"x"}' }).variables).toEqual([]); + }); + + it('is deterministic: the same row answers the same serialization', () => { + expect(JSON.stringify(emailTemplateDefinitionFromRow({ ...storedRow }))).toBe( + JSON.stringify(emailTemplateDefinitionFromRow(storedRow)), + ); + }); +}); diff --git a/packages/core/src/security/email-template-row-definition.ts b/packages/core/src/security/email-template-row-definition.ts new file mode 100644 index 0000000000..b0731d6d71 --- /dev/null +++ b/packages/core/src/security/email-template-row-definition.ts @@ -0,0 +1,106 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * [ADR-0131 D6/D10, ruling C on §6 Q1] A `sys_email_template` row, turned into + * the `email_template` DEFINITION it carries. + * + * Templates resolve from the registry (`email_template` metadata); the rows are + * no longer read on the send path. A row an organization customized before + * that door closed is the one copy of the customer's edit, and ruling C + * promotes it to an environment template before the row is deleted. The + * upgrade ceremony (`os migrate`, C7b) reads the row, calls this function and + * saves the answer through the metadata door. ⛔ Never at boot. + * + * The function is pure: one row in, one body out — no engine, no I/O, and no + * import, so the lean entry carries it as it stands. The mapping is the + * inverse of the row shape the retired projector wrote, which is also the + * render input the email service reads (`EmailTemplateRow`): + * + * - `name` → `name`; `label` → `label`, falling back to the name when the + * column is empty or absent; + * - `locale` → `locale`, falling back to `en-US` (the schema's default and + * the bundle floor) when empty or absent; + * - `subject` → `subject` and `body_html` → `bodyHtml`, each `''` when empty + * or absent (both are required strings); + * - `body_text` → `bodyText`, `reply_to` → `replyTo` and `description` → + * `description`, each only when a non-empty string; + * - `from_address` → `fromOverride.address`, with `from_name` → + * `fromOverride.name` when non-empty; no `fromOverride` without an address; + * - `variables_json` → `variables` when it is JSON text (or an already + * parsed value) holding an array, else `[]`; + * - `active` → `active`: `false` for a stored `false`, `0`, `'0'` or + * `'false'`, `true` for anything else, an absent column included (the + * schema's default). + * + * The retired keys (`category`, `is_system`) and every state or provenance + * column (`managed_by`, `customized`, `organization_id`, the timestamps) are + * not a definition and never travel into one. + * + * ⛔ It is not validation. A name outside the schema's dotted snake_case still + * answers a body; the metadata door refuses it, and reporting that refusal is + * the ceremony's. + */ + +/** + * One `sys_email_template` row, as the mapping reads it. Every other column a + * row carries is accepted and ignored. + */ +export interface EmailTemplateRowForDefinition { + readonly name?: unknown; + readonly label?: unknown; + readonly locale?: unknown; + readonly description?: unknown; + readonly subject?: unknown; + readonly body_html?: unknown; + readonly body_text?: unknown; + readonly from_name?: unknown; + readonly from_address?: unknown; + readonly reply_to?: unknown; + readonly active?: unknown; + readonly variables_json?: unknown; + readonly [column: string]: unknown; +} + +const filled = (value: unknown): string | undefined => + typeof value === 'string' && value !== '' ? value : undefined; + +const variablesOf = (value: unknown): unknown[] => { + let parsed = value; + if (typeof value === 'string') { + try { + parsed = JSON.parse(value); + } catch { + return []; + } + } + return Array.isArray(parsed) ? parsed : []; +}; + +/** + * The `email_template` definition one `sys_email_template` row carries (module + * doc). Deterministic: the same row answers the same body, key for key and in + * the same key order. + */ +export function emailTemplateDefinitionFromRow(row: EmailTemplateRowForDefinition): Record { + const body: Record = { + name: row.name, + label: filled(row.label) ?? row.name, + locale: filled(row.locale) ?? 'en-US', + subject: filled(row.subject) ?? '', + bodyHtml: filled(row.body_html) ?? '', + }; + const bodyText = filled(row.body_text); + if (bodyText !== undefined) body.bodyText = bodyText; + body.variables = variablesOf(row.variables_json); + const address = filled(row.from_address); + if (address !== undefined) { + const name = filled(row.from_name); + body.fromOverride = name === undefined ? { address } : { name, address }; + } + const replyTo = filled(row.reply_to); + if (replyTo !== undefined) body.replyTo = replyTo; + body.active = !(row.active === false || row.active === 0 || row.active === '0' || row.active === 'false'); + const description = filled(row.description); + if (description !== undefined) body.description = description; + return body; +} diff --git a/packages/core/src/security/index.ts b/packages/core/src/security/index.ts index f7963f7216..b4d93e9448 100644 --- a/packages/core/src/security/index.ts +++ b/packages/core/src/security/index.ts @@ -290,6 +290,10 @@ export { positionDefinitionFromRow, type PositionRowForDefinition, } from './position-row-definition.js'; +export { + emailTemplateDefinitionFromRow, + type EmailTemplateRowForDefinition, +} from './email-template-row-definition.js'; // [commit f8eb73601] The measured read surface of the administrator derivation — the // single source `plugin-auth`'s break-glass standing-key lists correspond to. From 49d85fc6538769d536c7e0d4699219ad3f31fa27 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 13 Oct 2026 06:18:58 +0000 Subject: [PATCH 3/5] feat(cli): the ceremony promotes customized email templates and moves the settings global rung MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two steps of `os migrate organization-ownership --apply`, per the maintainer's ruling on #22937 (Q3, Q4 → A), checkpointed and resumable like the others: - template-promotion: customized and admin-authored sys_email_template rows become environment email_template items through the metadata door (core's emailTemplateDefinitionFromRow) before mirror deletion; never over an environment template that differs; reported under a wall. - settings-move: the inventory's global-rung-move departure, executed: sys_setting scope = global rows copied into sys_platform_setting by INSERT ... SELECT, then removed; a key already served with a different value is a blocker. The sys_sharing_rule declared-rule mirrors are measured and not removed: the declared-rule seeder is still live and rule grants key on the row id. Claude-Session: https://claude.ai/code/session_01DjKf52bcyXa3fbGW723phR Co-Authored-By: Claude --- .changeset/22937-cli-ceremony-promotions.md | 21 ++ ...2937-core-email-template-row-definition.md | 13 + ...organization-ownership.apply-flags.test.ts | 19 +- .../migrate/organization-ownership.ts | 32 ++- .../organization-ownership-apply.test.ts | 118 +++++++-- .../src/utils/organization-ownership-apply.ts | 250 +++++++++++++++++- 6 files changed, 424 insertions(+), 29 deletions(-) create mode 100644 .changeset/22937-cli-ceremony-promotions.md create mode 100644 .changeset/22937-core-email-template-row-definition.md diff --git a/.changeset/22937-cli-ceremony-promotions.md b/.changeset/22937-cli-ceremony-promotions.md new file mode 100644 index 0000000000..be2068c0a1 --- /dev/null +++ b/.changeset/22937-cli-ceremony-promotions.md @@ -0,0 +1,21 @@ +--- +'@objectstack/cli': minor +--- + +feat(cli): `os migrate organization-ownership --apply` promotes customized email templates and moves the settings global rung (ADR-0131 D10) + +Clause-②: yes (widening) + +Two steps join the ceremony, after the authored catalog conversion and before the verified id→name rewrite. Each is checkpointed and resumable like the others. + +- **`template-promotion`** (ruling C on ADR-0131 §6 Q1). Every `sys_email_template` row an admin customized (`customized: true`) or authored (`managed_by` other than a seeder) becomes an environment `email_template` item through the metadata door, one per `(name, locale)`, before mirror deletion removes the rows. Seeded, uncustomized copies are not promoted. A promotion that already landed is skipped. The step stops, keeping every row, in three cases: + - two rows of one `(name, locale)` that differ; + - an environment template that already exists and differs, since the environment's edit is never overwritten; + - a door refusal. + + Under a walled posture the rows are reported and kept: promoting them would apply one organization's edit to every organization. +- **`settings-move`** (ADR-0131 D7). Every `sys_setting` row with `scope = 'global'` is copied into `sys_platform_setting` by the database itself (`INSERT … SELECT`), so `value` and the `value_enc` secret handle move verbatim. The row then leaves `sys_setting`. A key `sys_platform_setting` already holds with the same value is only removed from `sys_setting`. The step stops, moving nothing for that key, in two cases: + - `sys_platform_setting` already holds the key with a different value (the one the runtime serves); + - `sys_setting` holds differing rows for the key. + +Mirror deletion now also deletes the promoted template rows: under `single`, once the promotion step has finished, every remaining `sys_email_template` row is a mirror. The apply report gains `movedOut` / `movedIn` per table, and row counts reconcile as before − deleted − moved out + moved in. diff --git a/.changeset/22937-core-email-template-row-definition.md b/.changeset/22937-core-email-template-row-definition.md new file mode 100644 index 0000000000..0a5c862257 --- /dev/null +++ b/.changeset/22937-core-email-template-row-definition.md @@ -0,0 +1,13 @@ +--- +'@objectstack/core': minor +--- + +feat(core): `emailTemplateDefinitionFromRow` — a `sys_email_template` row as the `email_template` definition it carries (ADR-0131 D10, ruling C on §6 Q1) + +Clause-②: yes (widening) + +A new pure export beside `permissionSetDefinitionFromRow` and `positionDefinitionFromRow`: one row in, one body out, with no engine, no I/O and no import. The upgrade ceremony (`os migrate organization-ownership --apply`) uses it to promote a customized or admin-authored template row to an environment `email_template` item through the metadata door before the row is deleted. It runs only in that ceremony, never at boot. + +The mapping: `body_html` → `bodyHtml`, `body_text` → `bodyText`, `from_address` / `from_name` → `fromOverride`, `reply_to` → `replyTo`, `variables_json` → `variables`, and `active` keeps its stored meaning. `label` falls back to the name and `locale` to `en-US`. The retired `category` / `is_system` columns and every provenance or state column are left out. It does not validate; the door does. + +Nothing existing changes. diff --git a/packages/cli/src/commands/migrate/organization-ownership.apply-flags.test.ts b/packages/cli/src/commands/migrate/organization-ownership.apply-flags.test.ts index 19b7de0a2e..beeef57114 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.apply-flags.test.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.apply-flags.test.ts @@ -7,7 +7,7 @@ * nothing without `--apply`, and is refused there too. */ -import { afterEach, describe, expect, it, vi } from 'vitest'; +import { afterEach, beforeAll, describe, expect, it, vi } from 'vitest'; import { fileURLToPath } from 'node:url'; const boot = vi.hoisted(() => vi.fn(async () => { @@ -15,7 +15,16 @@ const boot = vi.hoisted(() => vi.fn(async () => { })); vi.mock('../../utils/schema-migrate.js', () => ({ bootSchemaStack: boot })); -const { default: MigrateOrganizationOwnership } = await import('./organization-ownership.js'); +type Command = (typeof import('./organization-ownership.js'))['default']; +let MigrateOrganizationOwnership: Command; + +// The command's module graph is large, and oclif reads the package's config on +// the first run: both are paid outside the 5 s default (CASE_TIMEOUT_MS). +beforeAll(async () => { + ({ default: MigrateOrganizationOwnership } = await import('./organization-ownership.js')); +}, 60_000); + +const CASE_TIMEOUT_MS = 60_000; const CLI_ROOT = fileURLToPath(new URL('../../..', import.meta.url)); @@ -50,19 +59,19 @@ describe('os migrate organization-ownership --apply — the backup precondition' expect(payload).toMatchObject({ error: 'apply_refused', reason: 'backup-unacknowledged' }); expect(exitCode).toBe(1); expect(boot).not.toHaveBeenCalled(); - }); + }, CASE_TIMEOUT_MS); it('refuses --backup-taken without --apply, before the boot', async () => { const { payload, exitCode } = await runJson(['--backup-taken', '--json']); expect(payload).toMatchObject({ error: 'plan_refused', reason: 'flag-without-apply' }); expect(exitCode).toBe(1); expect(boot).not.toHaveBeenCalled(); - }); + }, CASE_TIMEOUT_MS); it('with --backup-taken, goes on to the boot', async () => { const { payload } = await runJson(['--apply', '--backup-taken', '--json']); expect(payload).toMatchObject({ error: 'boot_failed', reason: 'boot-failed' }); expect(boot).toHaveBeenCalledTimes(1); expect(boot.mock.calls[0]).toEqual([expect.objectContaining({ deferSchemaDdl: true, composeHostStack: true })]); - }); + }, CASE_TIMEOUT_MS); }); diff --git a/packages/cli/src/commands/migrate/organization-ownership.ts b/packages/cli/src/commands/migrate/organization-ownership.ts index b01f152cf1..f0a41dcae5 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.ts @@ -84,8 +84,9 @@ async function confirm(question: string): Promise { * The writing mode. It asks about the backup first — the operator's act and * the precondition (D10 item 2): `--backup-taken`, or a yes at the prompt; a * non-interactive run without the flag is refused before the boot. It then - * performs the fates in order — attribution, the verified id→name rewrite, - * mirror deletion, column drops — through `../../utils/organization-ownership-apply.ts`, + * performs the fates in order — attribution, the authored catalog conversion, + * the template promotion, the settings global-rung move, the verified id→name + * rewrite, mirror deletion, column drops — through `../../utils/organization-ownership-apply.ts`, * checkpointing each finished step in the deployment ledger (`sys_migration`, * row {@link ORGANIZATION_OWNERSHIP_MIGRATION_ID}) so an interrupted run * resumes where it stopped, and ends with the post-check. The ledger row is @@ -106,7 +107,8 @@ export default class MigrateOrganizationOwnership extends Command { 'The v18 organization-ownership ceremony (ADR-0131 D10). Without --apply, the read-only plan: per table, the fate, ' + 'the row counts, the rows whose owner cannot be derived (by id), and which tables will receive NOT NULL, written ' + 'to a file — never a row or a column changed; it refuses, naming it, any table it cannot enumerate. With --apply ' + - '(and a backup): attribution, the verified id-to-name rewrite, mirror deletion and column drops, in that order, ' + + '(and a backup): attribution, the authored catalog conversion, the template promotion, the settings move, the ' + + 'verified id-to-name rewrite, mirror deletion and column drops, in that order, ' + 'resumable from a recorded checkpoint, then the post-check.'; static override examples = [ @@ -341,6 +343,7 @@ export default class MigrateOrganizationOwnership extends Command { const engine = kernel.getService('objectql') as Parameters[0] & { registry?: unknown }; const protocol = kernel.getService('protocol') as { saveMetaItem?: (request: Record) => Promise; + getMetaItemsForExecution?: (request: { type: string }) => Promise<{ items?: unknown[] }>; } | undefined; const catalog = createSecurityCatalogReader({ registry: engine.registry as never, @@ -381,6 +384,29 @@ export default class MigrateOrganizationOwnership extends Command { await protocol.saveMetaItem({ type, name, item: body, actor: CEREMONY_ACTOR }); }, }, + templates: { + // The environment's own row first (it is what a promotion must never + // overwrite), then what the registry serves for the bundle member. + find: async (name, locale) => { + const stored = await executor.query( + 'SELECT metadata FROM sys_metadata WHERE type = ? AND name = ? AND state = ? AND organization_id IS NULL ' + + 'AND (bundle_member = ? OR (bundle_member IS NULL AND ? = ?))', + ['email_template', name, 'active', locale, locale, 'en-US'], + ); + if (stored.length > 0) { + const raw = stored[0]!.metadata; + const definition = (typeof raw === 'string' ? JSON.parse(raw) : raw) as Record; + return { definition, environment: true }; + } + if (typeof protocol?.getMetaItemsForExecution !== 'function') return undefined; + const listed = await protocol.getMetaItemsForExecution({ type: 'email_template' }); + const served = (listed?.items ?? []).find((item) => { + const i = item as Record | null; + return !!i && i.name === name && (typeof i.locale === 'string' && i.locale ? i.locale : 'en-US') === locale; + }) as Record | undefined; + return served ? { definition: served, environment: false } : undefined; + }, + }, columnDrops: { drop: async (tables) => { const drift = await liveStack.detectManagedDrift(); diff --git a/packages/cli/src/utils/organization-ownership-apply.test.ts b/packages/cli/src/utils/organization-ownership-apply.test.ts index 6591616de4..034917b8f0 100644 --- a/packages/cli/src/utils/organization-ownership-apply.test.ts +++ b/packages/cli/src/utils/organization-ownership-apply.test.ts @@ -31,6 +31,7 @@ import { type ColumnDropPort, type MetadataDoorPort, type OrganizationOwnershipApplyReport, + type TemplatePort, } from './organization-ownership-apply.js'; let dir: string; @@ -62,15 +63,31 @@ function executor(opts: { failOnceOn?: RegExp } = {}): ApplyExecutor { * Like the registry, it serves a saved definition only from the next boot on * ({@link boot}), never the moment it is saved. */ -function catalog(): CatalogPort & MetadataDoorPort & { saved: Map>; boot(): void } { +function catalog(): CatalogPort & MetadataDoorPort & TemplatePort & { + saved: Map>; + templates: Map>; + boot(): void; +} { const declared = new Map>([ ['permission|member_default', permissionSetDefinitionFromRow(MEMBER_DEFAULT_ROW)], ['position|org_admin', { name: 'org_admin', label: 'Org Admin', delegatable: false, permissionSets: ['member_default'] }], ]); const saved = new Map>(); + /** Environment `email_template` items, keyed `name|locale`; a package ships `auth.reset_password`. */ + const templates = new Map>(); + const packaged = new Map>([ + ['auth.reset_password|en-US', { name: 'auth.reset_password', label: 'Reset', locale: 'en-US', subject: 'Reset', bodyHtml: '

code

' }], + ]); let hydrated = new Map>(); return { saved, + templates, + async find(name, locale) { + const own = templates.get(`${name}|${locale}`); + if (own) return { definition: own, environment: true }; + const shipped = packaged.get(`${name}|${locale}`); + return shipped ? { definition: shipped, environment: false } : undefined; + }, boot() { hydrated = new Map(saved); }, @@ -79,6 +96,11 @@ function catalog(): CatalogPort & MetadataDoorPort & { saved: Maphi

', 1, '[]', 0, 'platform'), + ('tpl_customized', NULL, 'auth.reset_password', 'Reset', 'en-US', 'Reset your password', '

our code

', 1, '[]', 1, 'platform'), + ('tpl_org', 'org_b', 'auth.welcome', 'Welcome', 'en-US', 'Hi', '

hi

', 1, '[]', 0, 'platform'), + ('tpl_admin', 'org_b', 'crm.renewal', 'Renewal', 'zh-CN', '续约', '

续约

', 1, '[]', 0, 'admin'); + + -- the settings global rung, still in sys_setting, and its new home + CREATE TABLE sys_setting (id TEXT PRIMARY KEY, organization_id TEXT, namespace TEXT, key TEXT, scope TEXT, user_id TEXT, + value TEXT, encrypted INTEGER, locked INTEGER, locked_reason TEXT, value_enc TEXT); + INSERT INTO sys_setting VALUES + ('set_global', NULL, 'mail', 'transport', 'global', NULL, '"smtp"', 0, 1, 'ops', NULL), + ('set_secret', NULL, 'ai', 'api_key', 'global', NULL, NULL, 1, 0, NULL, 'secret:h_1'), + ('set_tenant', 'org_b', 'mail', 'from', 'tenant', NULL, '"b@x.test"', 0, 0, NULL, NULL); + CREATE TABLE sys_platform_setting (id TEXT PRIMARY KEY, namespace TEXT, key TEXT, value TEXT, encrypted INTEGER, + locked INTEGER, locked_reason TEXT, value_enc TEXT); -- the references: grants by id (the rewrite names them) and assignments by name CREATE TABLE sys_user_permission_set (id TEXT PRIMARY KEY, organization_id TEXT, user_id TEXT, @@ -255,6 +290,7 @@ const apply = ( database: file, catalog: c, door: c, + templates: c, columnDrops, ledger: l, now: () => new Date('2026-10-12T00:00:00Z'), @@ -292,6 +328,8 @@ describe('ADR-0131 D10 --apply — the acceptance fixture', () => { expect(report.steps.map((s) => [s.step, s.status])).toEqual([ ['attribution', 'done'], ['authored-conversion', 'done'], + ['template-promotion', 'done'], + ['settings-move', 'done'], ['rewrite', 'done'], ['mirror-deletion', 'done'], ['column-drops', 'done'], @@ -314,7 +352,19 @@ describe('ADR-0131 D10 --apply — the acceptance fixture', () => { expect(rows('SELECT id FROM sys_position').map((r) => r.id)).toEqual(['pos_auth']); expect(rows('SELECT id FROM sys_position_permission_set')).toEqual([]); expect(rows('SELECT id FROM sys_capability').map((r) => r.id)).toEqual(['cap_authored']); - expect(rows('SELECT id FROM sys_email_template ORDER BY id').map((r) => r.id)).toEqual(['tpl_customized', 'tpl_org']); + expect(rows('SELECT id FROM sys_email_template')).toEqual([]); + + // Ruling C: the customized and the authored template became environment + // items before any row went; the seeded copies were not promoted. + expect([...c.templates.keys()].sort()).toEqual(['auth.reset_password|en-US', 'crm.renewal|zh-CN']); + expect(c.templates.get('auth.reset_password|en-US')).toMatchObject({ subject: 'Reset your password', bodyHtml: '

our code

' }); + + // The global rung moved to sys_platform_setting, value and secret handle verbatim; the tenant row stayed. + expect(rows('SELECT id, scope FROM sys_setting')).toEqual([{ id: 'set_tenant', scope: 'tenant' }]); + expect(rows('SELECT * FROM sys_platform_setting ORDER BY id')).toEqual([ + { id: 'set_global', namespace: 'mail', key: 'transport', value: '"smtp"', encrypted: 0, locked: 1, locked_reason: 'ops', value_enc: null }, + { id: 'set_secret', namespace: 'ai', key: 'api_key', value: null, encrypted: 1, locked: 0, locked_reason: null, value_enc: 'secret:h_1' }, + ]); // Every grant names its set; the id column is still there (C7b's later drop). expect(rows('SELECT id, permission_set, permission_set_id FROM sys_user_permission_set ORDER BY id')).toEqual([ @@ -342,13 +392,16 @@ describe('ADR-0131 D10 --apply — the acceptance fixture', () => { const countsAfter = counts(); for (const table of report.tables) { if (!table.before || !table.after) continue; - expect(table.before.total - table.deleted, table.object).toBe(table.after.total); + expect(table.before.total - table.deleted - table.movedOut + table.movedIn, table.object).toBe(table.after.total); expect(countsAfter[table.object], table.object).toBe(table.after.total); expect(countsBefore[table.object], table.object).toBe(table.before.total); if (table.fate !== 'mirror-deletion') expect(table.deleted, table.object).toBe(0); } + expect(tableOf(report, 'sys_setting').movedOut).toBe(2); + expect(tableOf(report, 'sys_platform_setting').movedIn).toBe(2); + // Every delete is fate 2's, or the moved rows leaving sys_setting. expect(writes.filter((w) => /^DELETE/.test(w)).every((w) => - /sys_permission_set|sys_position|sys_position_permission_set|sys_capability|sys_email_template/.test(w))).toBe(true); + /sys_permission_set|sys_position|sys_position_permission_set|sys_capability|sys_email_template|"sys_setting" WHERE \("scope" = \?\)/.test(w))).toBe(true); }); it('positive control: a grant the rewrite would lose reddens the effective-access pin', async () => { @@ -374,7 +427,9 @@ describe('ADR-0131 D10 --apply — idempotent, resumable, gated', () => { const again = await apply(c, l); - expect(again.resumedFrom).toEqual(['attribution', 'authored-conversion', 'rewrite', 'mirror-deletion', 'column-drops']); + expect(again.resumedFrom).toEqual([ + 'attribution', 'authored-conversion', 'template-promotion', 'settings-move', 'rewrite', 'mirror-deletion', 'column-drops', + ]); expect(again.steps.every((s) => s.status === 'checkpointed')).toBe(true); expect(again.complete).toBe(true); expect(writes).toEqual([]); @@ -384,15 +439,15 @@ describe('ADR-0131 D10 --apply — idempotent, resumable, gated', () => { it('a run killed inside mirror deletion resumes there, never re-deriving an owner it already wrote', async () => { const c = catalog(); const l = ledger(); - await expect(apply(c, l, { executor: executor({ failOnceOn: /^DELETE/ }) })).rejects.toThrow(/killed mid-step/); - expect(Object.keys(l.checkpoint!.completed)).toEqual(['attribution', 'authored-conversion', 'rewrite']); + await expect(apply(c, l, { executor: executor({ failOnceOn: /^DELETE FROM "sys_capability"/ }) })).rejects.toThrow(/killed mid-step/); + expect(Object.keys(l.checkpoint!.completed)).toEqual(['attribution', 'authored-conversion', 'template-promotion', 'settings-move', 'rewrite']); expect(rows('SELECT COUNT(*) AS n FROM sys_position_permission_set')).toEqual([{ n: 3 }]); writes = []; const resumed = await apply(c, l); - expect(resumed.resumedFrom).toEqual(['attribution', 'authored-conversion', 'rewrite']); - expect(resumed.steps.map((s) => s.status)).toEqual(['checkpointed', 'checkpointed', 'checkpointed', 'done', 'done']); + expect(resumed.resumedFrom).toEqual(['attribution', 'authored-conversion', 'template-promotion', 'settings-move', 'rewrite']); + expect(resumed.steps.map((s) => s.status)).toEqual(['checkpointed', 'checkpointed', 'checkpointed', 'checkpointed', 'checkpointed', 'done', 'done']); expect(writes.some((w) => /^UPDATE/.test(w))).toBe(false); expect(resumed.complete).toBe(true); expect(rows('SELECT COUNT(*) AS n FROM sys_position_permission_set')).toEqual([{ n: 0 }]); @@ -401,7 +456,7 @@ describe('ADR-0131 D10 --apply — idempotent, resumable, gated', () => { it('reads the gate again before the first delete: a grant written between runs keeps every mirror', async () => { const c = catalog(); const l = ledger(); - await expect(apply(c, l, { executor: executor({ failOnceOn: /^DELETE/ }) })).rejects.toThrow(/killed mid-step/); + await expect(apply(c, l, { executor: executor({ failOnceOn: /^DELETE FROM "sys_capability"/ }) })).rejects.toThrow(/killed mid-step/); db.exec(`INSERT INTO sys_user_permission_set VALUES ('grant_late', 'org_default', 'usr_8', 'ps_gone', NULL)`); const countsBefore = counts(); @@ -414,6 +469,30 @@ describe('ADR-0131 D10 --apply — idempotent, resumable, gated', () => { expect(resumed.complete).toBe(false); }); + it('moves no setting over a different value the new table already serves, and deletes nothing for it', async () => { + db.exec(`INSERT INTO sys_platform_setting VALUES ('ps_1', 'mail', 'transport', '"ses"', 0, 0, NULL, NULL)`); + const report = await apply(catalog(), ledger()); + + const move = report.steps.find((s) => s.step === 'settings-move')!; + expect(move.status).toBe('blocked'); + expect(move.blockers.join('\n')).toMatch(/mail\.transport/); + expect(rows(`SELECT id FROM sys_setting WHERE scope = 'global' ORDER BY id`).map((r) => r.id)).toEqual(['set_global']); + expect(rows(`SELECT value FROM sys_platform_setting WHERE key = 'transport'`)).toEqual([{ value: '"ses"' }]); + expect(report.steps.find((s) => s.step === 'mirror-deletion')!.status).toBe('not-reached'); + }); + + it('never overwrites a template the environment already holds, and keeps every template row', async () => { + const c = catalog(); + c.templates.set('crm.renewal|zh-CN', { name: 'crm.renewal', label: 'Renewal', locale: 'zh-CN', subject: '新的', bodyHtml: '

新的

' }); + const report = await apply(c, ledger()); + + const promotion = report.steps.find((s) => s.step === 'template-promotion')!; + expect(promotion.status).toBe('blocked'); + expect(promotion.blockers.join('\n')).toMatch(/crm\.renewal \(zh-CN\)/); + expect(c.templates.get('crm.renewal|zh-CN')).toMatchObject({ subject: '新的' }); + expect(rows('SELECT COUNT(*) AS n FROM sys_email_template')).toEqual([{ n: 4 }]); + }); + it('deletes nothing while a reference does not resolve by name — the gate of fate 2', async () => { // Another organization's set row: a name carries no organization, so it is never written. db.exec(` @@ -429,8 +508,11 @@ describe('ADR-0131 D10 --apply — idempotent, resumable, gated', () => { expect(rewrite.status).toBe('blocked'); expect(rewrite.blockers.join('\n')).toMatch(/grant_x/); expect(report.steps.find((s) => s.step === 'mirror-deletion')!.status).toBe('not-reached'); - expect(writes.filter((w) => /^DELETE/.test(w))).toEqual([]); - expect(counts()).toEqual(countsBefore); + // The only rows that left are the settings rung, which moved before the rewrite. + expect(writes.filter((w) => /^DELETE/.test(w) && !/^DELETE FROM "sys_setting"/.test(w))).toEqual([]); + const { sys_setting: _s, sys_platform_setting: _p, ...after } = counts(); + const { sys_setting: _sb, sys_platform_setting: _pb, ...before } = countsBefore; + expect(after).toEqual(before); expect(report.complete).toBe(false); expect(report.blocking).toBeGreaterThan(0); expect(l.verdicts.at(-1)!.complete).toBe(false); @@ -446,6 +528,8 @@ describe('ADR-0131 D10 --apply — idempotent, resumable, gated', () => { expect(c.saved.size).toBe(0); expect(report.steps.find((s) => s.step === 'mirror-deletion')!.status).toBe('not-reached'); expect(rows('SELECT COUNT(*) AS n FROM sys_permission_set')).toEqual([{ n: 3 }]); + expect(c.templates.size).toBe(0); + expect(rows('SELECT COUNT(*) AS n FROM sys_email_template')).toEqual([{ n: 4 }]); }); it('holds a column whose rows another stage owns, and reports it', async () => { diff --git a/packages/cli/src/utils/organization-ownership-apply.ts b/packages/cli/src/utils/organization-ownership-apply.ts index 3c8c9b6f8b..bdc1f4bf3c 100644 --- a/packages/cli/src/utils/organization-ownership-apply.ts +++ b/packages/cli/src/utils/organization-ownership-apply.ts @@ -2,6 +2,7 @@ import { convertPositionBindingRows, + emailTemplateDefinitionFromRow, permissionSetDefinitionFromRow, positionDefinitionFromRow, } from '@objectstack/core'; @@ -42,6 +43,17 @@ import { * (`convertPositionBindingRows`). Only under `single` (ADR-0131 D3: a * Setup catalog write is an environment metadata write under `single` and * refused under a wall); under a wall the rows are reported. + * 2b. `template-promotion` — ruling C on ADR-0131 §6 Q1: every + * `sys_email_template` row an organization customized or authored becomes + * an environment `email_template` item through the metadata door (core's + * `emailTemplateDefinitionFromRow`), so the deletion that follows loses no + * customer edit. Under `single` only; under a wall one organization's edit + * would become every organization's, so the rows are reported and kept. + * 2c. `settings-move` — the inventory's `global-rung-move` departure, + * executed (#15207's ruling: "C7's ceremony moving the existing rows"): + * each `sys_setting` row of `scope = 'global'` is copied, by the database + * itself, into `sys_platform_setting` and then removed from `sys_setting`. + * A key the new table already holds with a different value is a blocker. * 3. `rewrite` — the verified id→name rewrite (D4): * `sys_user_permission_set.permission_set` is named from the set row its * `permission_set_id` points at, then EVERY grant is verified to name a set @@ -77,7 +89,15 @@ import { /** The `sys_migration` row the ceremony checkpoints into and completes. */ export const ORGANIZATION_OWNERSHIP_MIGRATION_ID = 'adr-0131-organization-ownership'; -export const CEREMONY_STEPS = ['attribution', 'authored-conversion', 'rewrite', 'mirror-deletion', 'column-drops'] as const; +export const CEREMONY_STEPS = [ + 'attribution', + 'authored-conversion', + 'template-promotion', + 'settings-move', + 'rewrite', + 'mirror-deletion', + 'column-drops', +] as const; export type CeremonyStep = (typeof CEREMONY_STEPS)[number]; /** The raw-SQL seam: the plan's reader plus the one write verb. */ @@ -92,7 +112,12 @@ export interface CatalogPort { /** The metadata door: an environment metadata save. Throws when the door refuses. */ export interface MetadataDoorPort { - save(type: 'position' | 'permission', name: string, body: Record): Promise; + save(type: 'position' | 'permission' | 'email_template', name: string, body: Record): Promise; +} + +/** The `email_template` item the registry serves for one `(name, locale)` bundle member. */ +export interface TemplatePort { + find(name: string, locale: string): Promise<{ definition: Readonly>; environment: boolean } | undefined>; } /** One table's column drop, as the schema-drift ceremony answered it. */ @@ -138,6 +163,9 @@ export interface TableOutcome { after: { total: number; organizationNull: number } | null; attributed: number; deleted: number; + /** Rows a `moved` departure carried out of this table, and into it. */ + movedOut: number; + movedIn: number; /** Fate 1: the column is gone after this run. */ columnDropped: boolean; /** Rows still NULL and listed by id — reported, never guessed. */ @@ -173,6 +201,7 @@ export interface ApplyOptions { database: string; catalog: CatalogPort; door: MetadataDoorPort; + templates: TemplatePort; columnDrops: ColumnDropPort; ledger: CeremonyLedgerPort; inventory?: readonly InventoryEntry[]; @@ -191,6 +220,48 @@ const SETS = 'sys_permission_set'; const POSITIONS = 'sys_position'; const BINDINGS = 'sys_position_permission_set'; const CAPABILITIES = 'sys_capability'; +const TEMPLATES = 'sys_email_template'; +const SETTINGS = 'sys_setting'; +const PLATFORM_SETTINGS = 'sys_platform_setting'; + +/** Provenance the seeders stamped: such a row is a copy of code unless an admin customized it. */ +const SEEDED_BY = ['package', 'platform', 'system']; + +/** + * The `sys_email_template` rows the promotion carries: customized seeded rows + * and every row an admin authored. A seeded, uncustomized row — NULL residue or + * an organization's copy — is a copy of code and is not promoted. + */ +const templatePromotionSql = (sql: Sql, cols: ReadonlySet): { text: string; params: unknown[] } => { + const t = TEMPLATES; + const customized = cols.has('customized') ? `${sql.q('customized', t)} = ?` : '1 = 0'; + const params: unknown[] = cols.has('customized') ? [sql.bind(true)] : []; + if (!cols.has('managed_by')) return { text: '1 = 1', params: [] }; + const managed = sql.q('managed_by', t); + return { + text: `${customized} OR ${managed} IS NULL OR ${managed} NOT IN (${SEEDED_BY.map(() => '?').join(', ')})`, + params: [...params, ...SEEDED_BY], + }; +}; + +/** + * Categories of a fate-2 object that the promotion discharges: once it has + * run (under `single`), the rows they claim have their environment copy and + * are mirrors like the rest. + */ +const DISCHARGED_BY_PROMOTION: Readonly> = { + [TEMPLATES]: ['template-promotion', 'organization-stamped'], +}; + +/** The `moved` departures the ceremony executes, by departure id. */ +const MOVES: Readonly> = { + 'global-rung-move': { + from: SETTINGS, + to: PLATFORM_SETTINGS, + key: ['namespace', 'key'], + compare: ['value', 'value_enc', 'encrypted', 'locked', 'locked_reason'], + }, +}; const text = (value: unknown): string | null => value === null || value === undefined || String(value) === '' ? null : String(value); @@ -203,6 +274,8 @@ interface Run { columns: Map | null>; attributed: Map; deleted: Map; + movedOut: Map; + movedIn: Map; /** * What the door accepted this run. The registry hydrates environment * metadata at boot, so a definition saved a moment ago is not in it until @@ -535,6 +608,159 @@ async function runAuthoredConversion(run: Run): Promise { return { step: 'authored-conversion', status: blockers.length ? 'blocked' : 'done', summary, blockers, notes }; } +// ── Step 2b: template promotion (ruling C) ────────────────────────────────── + +async function runTemplatePromotion(run: Run): Promise { + const summary: Record = { promoted: 0, alreadyPromoted: 0, reported: 0 }; + const blockers: string[] = []; + const notes: string[] = []; + const cols = await columnsOf(run, TEMPLATES); + if (!cols) return { step: 'template-promotion', status: 'done', summary, blockers, notes }; + const rows = await readAll(run, TEMPLATES, [...cols], templatePromotionSql(run.sql, cols)); + if (rows.length === 0) return { step: 'template-promotion', status: 'done', summary, blockers, notes }; + if (run.opts.posture !== 'single') { + summary.reported = rows.length; + notes.push( + `${rows.length} customized or authored ${TEMPLATES} row(s) are reported and kept, not promoted: under the ` + + `${run.opts.posture} posture an environment template would carry one organization's edit to every organization.`, + ); + return { step: 'template-promotion', status: 'done', summary, blockers, notes }; + } + const groups = new Map[]; bodies: Record[] }>(); + for (const row of rows) { + const body = emailTemplateDefinitionFromRow(row); + const name = text(body.name); + if (!name) { + summary.reported++; + notes.push(`A ${TEMPLATES} row (${String(row.id)}) carries no name and is reported, never promoted.`); + continue; + } + const locale = String(body.locale); + const key = `${name}\u0000${locale}`; + const group = groups.get(key) ?? { name, locale, rows: [], bodies: [] }; + group.rows.push(row); + group.bodies.push(body); + groups.set(key, group); + } + for (const { name, locale, rows: group, bodies } of groups.values()) { + const keys = [...new Set(bodies.flatMap(Object.keys))]; + if (new Set(bodies.map((b) => bodyKey(b, keys))).size > 1) { + blockers.push( + `The template ${name} (${locale}) is held by ${group.length} customized or authored rows ` + + `(${group.map((r) => String(r.id)).join(', ')}) that differ. Which edit the environment keeps is never ` + + 'guessed: make them agree, or delete all but one, then re-run.', + ); + continue; + } + const body = bodies[0]!; + const existing = await run.opts.templates.find(name, locale); + const saved = run.saved.get(`email_template|${name}|${locale}`); + const current = saved ?? existing?.definition; + if (current && bodyKey(current, Object.keys(body)) === bodyKey(body, Object.keys(body))) { + summary.alreadyPromoted++; + continue; + } + if (existing?.environment && !saved) { + blockers.push( + `The environment already holds its own template ${name} (${locale}), and it differs from the customized row(s) ` + + `${group.map((r) => String(r.id)).join(', ')}. The environment's edit is never overwritten: carry the ` + + 'row\'s change into it in Studio (or delete the row), then re-run.', + ); + continue; + } + try { + await run.opts.door.save('email_template', name, body); + run.saved.set(`email_template|${name}|${locale}`, body); + summary.promoted++; + run.opts.onProgress?.(`promoted the template ${name} (${locale}) to environment metadata`); + } catch (error) { + blockers.push( + `The metadata door refused the template ${name} (${locale}): ${error instanceof Error ? error.message : String(error)}. ` + + 'Correct the row so the definition validates, then re-run.', + ); + } + } + return { step: 'template-promotion', status: blockers.length ? 'blocked' : 'done', summary, blockers, notes }; +} + +// ── Step 2c: the settings global rung moves (D7) ──────────────────────────── + +async function runSettingsMove(run: Run): Promise { + const summary: Record = { moved: 0, alreadyMoved: 0 }; + const blockers: string[] = []; + for (const entry of run.opts.inventory ?? [inventoryEntryFor(SETTINGS)!]) { + for (const departure of entry?.departures ?? []) { + const move = MOVES[departure.id]; + if (departure.as !== 'moved' || !move || entry.object !== move.from) continue; + const fromCols = await columnsOf(run, move.from); + if (!fromCols) continue; + const where = predicateSql(run.sql, move.from, fromCols, departure.where); + const rows = await readAll(run, move.from, [...move.key, ...move.compare].filter((c) => fromCols.has(c)), where); + if (rows.length === 0) continue; + const toCols = await columnsOf(run, move.to); + if (!toCols) { + blockers.push( + `${rows.length} ${move.from} row(s) move to ${move.to}, which this database does not have. Boot this release ` + + 'against it once (its schema sync creates the table), then re-run.', + ); + continue; + } + const same = (a: Record, b: Record): boolean => + move.compare.every((c) => JSON.stringify(a[c] ?? null) === JSON.stringify(b[c] ?? null)); + const keyOf = (r: Record): string => move.key.map((k) => String(r[k])).join('.'); + const byKey = new Map[]>(); + for (const row of rows) byKey.set(keyOf(row), [...(byKey.get(keyOf(row)) ?? []), row]); + const shared = [...fromCols].filter((c) => toCols.has(c)); + const sql = run.sql; + for (const [key, group] of byKey) { + if (group.some((r) => !same(r, group[0]!))) { + blockers.push( + `${move.from} holds ${group.length} differing ${departure.id} rows for ${key} (${group.map((r) => String(r.id)).join(', ')}). ` + + 'Which value the deployment keeps is never guessed: delete all but one, then re-run.', + ); + continue; + } + const target = (await readAll(run, move.to, move.compare.filter((c) => toCols.has(c)), { + text: move.key.map((k) => `${sql.q(k, move.to)} = ?`).join(' AND '), + params: move.key.map((k) => group[0]![k]), + })); + if (target.length > 0 && !same(target[0]!, group[0]!)) { + blockers.push( + `${move.to} already holds ${key} with a different value — the one the runtime serves — and the ${move.from} ` + + `row(s) ${group.map((r) => String(r.id)).join(', ')} hold another. Delete the stale ${move.from} row(s) or ` + + 'align them, then re-run.', + ); + continue; + } + if (target.length === 0) { + const list = shared.map((c) => sql.q(c, move.to)).join(', '); + await execute( + run, + move.to, + `INSERT INTO ${sql.q(move.to, move.to)} (${list}) SELECT ${shared.map((c) => sql.q(c, move.from)).join(', ')} ` + + `FROM ${sql.q(move.from, move.from)} WHERE ${sql.q('id', move.from)} = ?`, + [group[0]!.id], + ); + summary.moved++; + run.movedIn.set(move.to, (run.movedIn.get(move.to) ?? 0) + 1); + } else { + summary.alreadyMoved++; + } + const ids = group.map((r) => r.id); + await execute( + run, + move.from, + `DELETE FROM ${sql.q(move.from, move.from)} WHERE (${where.text}) AND ${sql.q('id', move.from)} IN (${ids.map(() => '?').join(', ')})`, + [...where.params, ...ids], + ); + run.movedOut.set(move.from, (run.movedOut.get(move.from) ?? 0) + ids.length); + } + if (summary.moved) run.opts.onProgress?.(`settings move: ${summary.moved} ${departure.id} key(s) now in ${move.to}`); + } + } + return { step: 'settings-move', status: blockers.length ? 'blocked' : 'done', summary, blockers, notes: [] }; +} + // ── Step 3: the verified id→name rewrite (D4) ─────────────────────────────── interface Verification { @@ -700,8 +926,17 @@ function deletableWhere(run: Run, table: string, cols: ReadonlySet, entr parts.push(`(${mirror.text})`); params.push(...mirror.params); } + // The promotion runs under `single` only; elsewhere its rows were not + // carried anywhere, so they stay, whatever category claims them or not. + const promoted = run.opts.posture === 'single'; + const discharged = promoted ? DISCHARGED_BY_PROMOTION[entry.object] ?? [] : []; + if (!promoted && entry.object === TEMPLATES) { + const kept = templatePromotionSql(run.sql, cols); + parts.push(`(CASE WHEN ${kept.text} THEN 1 ELSE 0 END) = 0`); + params.push(...kept.params); + } for (const category of entry.categories ?? []) { - if (!category.where || category.conflictBy) continue; + if (!category.where || category.conflictBy || discharged.includes(category.id)) continue; const claimed = predicateSql(run.sql, table, cols, category.where); parts.push(`(CASE WHEN ${claimed.text} THEN 1 ELSE 0 END) = 0`); params.push(...claimed.params); @@ -796,6 +1031,8 @@ async function runColumnDrops(run: Run): Promise { const RUNNERS: Record Promise> = { attribution: runAttribution, 'authored-conversion': runAuthoredConversion, + 'template-promotion': runTemplatePromotion, + 'settings-move': runSettingsMove, rewrite: runRewrite, 'mirror-deletion': runMirrorDeletion, 'column-drops': runColumnDrops, @@ -806,7 +1043,8 @@ async function remainingFor(run: Run, table: TablePlan, complete: boolean): Prom if (table.physical !== 'present' || !table.rows) return 0; switch (table.fate) { case 'attribution': - return Number(table.fateCounts.rowsToAttribute ?? 0); + return Number(table.fateCounts.rowsToAttribute ?? 0) + + table.departures.filter((d) => d.as === 'moved' && MOVES[d.id]).reduce((n, d) => n + d.rows, 0); case 'mirror-deletion': { const entry = mirrorEntry(run, table.object); if (!entry) return 0; @@ -839,6 +1077,8 @@ export async function applyOrganizationOwnership(opts: ApplyOptions): Promise Date: Tue, 13 Oct 2026 18:24:14 +0000 Subject: [PATCH 4/5] docs(changeset): the promotion population is the rows mirror deletion keeps; the executed departure stops counting Claude-Session: https://claude.ai/code/session_01DjKf52bcyXa3fbGW723phR Co-Authored-By: Claude --- .changeset/22937-cli-ceremony-promotions.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/.changeset/22937-cli-ceremony-promotions.md b/.changeset/22937-cli-ceremony-promotions.md index be2068c0a1..ef50aed684 100644 --- a/.changeset/22937-cli-ceremony-promotions.md +++ b/.changeset/22937-cli-ceremony-promotions.md @@ -8,14 +8,16 @@ Clause-②: yes (widening) Two steps join the ceremony, after the authored catalog conversion and before the verified id→name rewrite. Each is checkpointed and resumable like the others. -- **`template-promotion`** (ruling C on ADR-0131 §6 Q1). Every `sys_email_template` row an admin customized (`customized: true`) or authored (`managed_by` other than a seeder) becomes an environment `email_template` item through the metadata door, one per `(name, locale)`, before mirror deletion removes the rows. Seeded, uncustomized copies are not promoted. A promotion that already landed is skipped. The step stops, keeping every row, in three cases: +- **`template-promotion`** (ruling C on ADR-0131 §6 Q1). Every `sys_email_template` row the mirror deletion keeps for it (customized, or with `customized` unknown, and every row no seeder wrote: `managed_by` NULL or other than package / platform / system, with or without an organization) becomes an environment `email_template` item through the metadata door, one per `(name, locale)`, before mirror deletion removes the rows. Seeded, uncustomized copies are not promoted. A promotion that already landed is skipped. The step stops, keeping every row, in three cases: - two rows of one `(name, locale)` that differ; - an environment template that already exists and differs, since the environment's edit is never overwritten; - a door refusal. - Under a walled posture the rows are reported and kept: promoting them would apply one organization's edit to every organization. + Under a walled posture the rows are kept and not promoted: promoting them would apply one organization's edit to every organization. They count as remaining and are named as a `template-promotion` departure, so the ceremony's `verified_at` stays null. - **`settings-move`** (ADR-0131 D7). Every `sys_setting` row with `scope = 'global'` is copied into `sys_platform_setting` by the database itself (`INSERT … SELECT`), so `value` and the `value_enc` secret handle move verbatim. The row then leaves `sys_setting`. A key `sys_platform_setting` already holds with the same value is only removed from `sys_setting`. The step stops, moving nothing for that key, in two cases: - `sys_platform_setting` already holds the key with a different value (the one the runtime serves); - `sys_setting` holds differing rows for the key. +The `global-rung-move` departure is now executed, so it reads zero after the run and no longer holds back `verified_at`. The `declared-rule-mirrors` departure still counts until the declared-rule seeder is retired. + Mirror deletion now also deletes the promoted template rows: under `single`, once the promotion step has finished, every remaining `sys_email_template` row is a mirror. The apply report gains `movedOut` / `movedIn` per table, and row counts reconcile as before − deleted − moved out + moved in. From 1bede353c7505d01db732d43dd480588221fa885 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 13 Oct 2026 20:31:42 +0000 Subject: [PATCH 5/5] fix(cli): the template delete stays inside fate 2 on every run; walled promotion with operator-chosen conflicts (#22937) - Mirror deletion re-runs the idempotent template promotion before its first template delete, then deletes only the seeded, uncustomized mirrors and, by id, the rows whose environment copy that pass verified. A resumed run can no longer delete a row customized or authored between runs. - A customized or authored template row with no name blocks: kept, named. - The promotion runs under every posture. Rows of one (name, locale) that differ are a conflict: listed in the plan and the --apply report with each candidate's row id and organization, counted as remaining, and blocking until the operator chooses one row with the repeatable --template-choice ROW_ID. An unknown or ambiguous choice is refused before anything is written. - The settings move selects organization_id IS NULL and reads each destination row back before deleting its source; the human face prints the moves. - emailTemplateDefinitionFromRow falls back to EMAIL_TEMPLATE_FLOOR_LOCALE. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01DjKf52bcyXa3fbGW723phR --- .changeset/22937-cli-ceremony-promotions.md | 11 +- ...2937-core-email-template-row-definition.md | 4 +- .../migrate/organization-ownership.ts | 41 ++- .../organization-ownership-apply.test.ts | 156 ++++++++-- .../src/utils/organization-ownership-apply.ts | 267 +++++++++++------- .../src/utils/organization-ownership-plan.ts | 100 +++++++ .../email-template-row-definition.test.ts | 4 +- .../security/email-template-row-definition.ts | 12 +- 8 files changed, 455 insertions(+), 140 deletions(-) diff --git a/.changeset/22937-cli-ceremony-promotions.md b/.changeset/22937-cli-ceremony-promotions.md index ef50aed684..7dd2aa630e 100644 --- a/.changeset/22937-cli-ceremony-promotions.md +++ b/.changeset/22937-cli-ceremony-promotions.md @@ -8,16 +8,15 @@ Clause-②: yes (widening) Two steps join the ceremony, after the authored catalog conversion and before the verified id→name rewrite. Each is checkpointed and resumable like the others. -- **`template-promotion`** (ruling C on ADR-0131 §6 Q1). Every `sys_email_template` row the mirror deletion keeps for it (customized, or with `customized` unknown, and every row no seeder wrote: `managed_by` NULL or other than package / platform / system, with or without an organization) becomes an environment `email_template` item through the metadata door, one per `(name, locale)`, before mirror deletion removes the rows. Seeded, uncustomized copies are not promoted. A promotion that already landed is skipped. The step stops, keeping every row, in three cases: - - two rows of one `(name, locale)` that differ; +- **`template-promotion`** (ruling C on ADR-0131 §6 Q1). Every `sys_email_template` row the mirror deletion keeps for it (customized, or with `customized` unknown, and every row no seeder wrote: `managed_by` NULL or other than package / platform / system, with or without an organization) becomes an environment `email_template` item through the metadata door, one per `(name, locale)`, before mirror deletion removes the rows. It runs under every posture. Seeded, uncustomized copies are not promoted. A promotion that already landed is skipped. The step stops, keeping every row of the template, in four cases: + - a conflict: rows of one `(name, locale)` that differ. Each conflict is listed in the plan and in the `--apply` report, with every candidate's row id and organization. It counts as remaining and holds back `verified_at` until the operator chooses one row with the new repeatable flag `--template-choice ROW_ID`. A choice that names no candidate, or two candidates of one conflict, is refused before anything is written. The chosen body is promoted, and the other candidates are deleted only after that promotion is verified; + - a customized or authored row with no name; - an environment template that already exists and differs, since the environment's edit is never overwritten; - a door refusal. - - Under a walled posture the rows are kept and not promoted: promoting them would apply one organization's edit to every organization. They count as remaining and are named as a `template-promotion` departure, so the ceremony's `verified_at` stays null. -- **`settings-move`** (ADR-0131 D7). Every `sys_setting` row with `scope = 'global'` is copied into `sys_platform_setting` by the database itself (`INSERT … SELECT`), so `value` and the `value_enc` secret handle move verbatim. The row then leaves `sys_setting`. A key `sys_platform_setting` already holds with the same value is only removed from `sys_setting`. The step stops, moving nothing for that key, in two cases: +- **`settings-move`** (ADR-0131 D7). Every `sys_setting` row with `scope = 'global'` and no organization is copied into `sys_platform_setting` by the database itself (`INSERT … SELECT`), so `value` and the `value_enc` secret handle move verbatim. The destination row is read back, and only when it equals the source does the row leave `sys_setting`. A key `sys_platform_setting` already holds with the same value is only removed from `sys_setting`. The step stops, moving nothing for that key, in two cases: - `sys_platform_setting` already holds the key with a different value (the one the runtime serves); - `sys_setting` holds differing rows for the key. The `global-rung-move` departure is now executed, so it reads zero after the run and no longer holds back `verified_at`. The `declared-rule-mirrors` departure still counts until the declared-rule seeder is retired. -Mirror deletion now also deletes the promoted template rows: under `single`, once the promotion step has finished, every remaining `sys_email_template` row is a mirror. The apply report gains `movedOut` / `movedIn` per table, and row counts reconcile as before − deleted − moved out + moved in. +Mirror deletion now also deletes the promoted template rows, and only those. Before its first template delete it runs the promotion pass again, so a row customized or authored between runs (on a resumed ceremony) is promoted, not deleted. It then deletes the seeded, uncustomized mirrors and, by id, the rows whose environment copy that pass verified. Any other promotion row stays and blocks. The apply report gains `movedOut` / `movedIn` per table, and row counts reconcile as before − deleted − moved out + moved in. diff --git a/.changeset/22937-core-email-template-row-definition.md b/.changeset/22937-core-email-template-row-definition.md index 0a5c862257..18aba45eb4 100644 --- a/.changeset/22937-core-email-template-row-definition.md +++ b/.changeset/22937-core-email-template-row-definition.md @@ -6,8 +6,8 @@ feat(core): `emailTemplateDefinitionFromRow` — a `sys_email_template` row as t Clause-②: yes (widening) -A new pure export beside `permissionSetDefinitionFromRow` and `positionDefinitionFromRow`: one row in, one body out, with no engine, no I/O and no import. The upgrade ceremony (`os migrate organization-ownership --apply`) uses it to promote a customized or admin-authored template row to an environment `email_template` item through the metadata door before the row is deleted. It runs only in that ceremony, never at boot. +A new pure export beside `permissionSetDefinitionFromRow` and `positionDefinitionFromRow`: one row in, one body out, with no engine and no I/O. Its one import is the spec's floor-locale constant. The upgrade ceremony (`os migrate organization-ownership --apply`) uses it to promote a customized or admin-authored template row to an environment `email_template` item through the metadata door before the row is deleted. It runs only in that ceremony, never at boot. -The mapping: `body_html` → `bodyHtml`, `body_text` → `bodyText`, `from_address` / `from_name` → `fromOverride`, `reply_to` → `replyTo`, `variables_json` → `variables`, and `active` keeps its stored meaning. `label` falls back to the name and `locale` to `en-US`. The retired `category` / `is_system` columns and every provenance or state column are left out. It does not validate; the door does. +The mapping: `body_html` → `bodyHtml`, `body_text` → `bodyText`, `from_address` / `from_name` → `fromOverride`, `reply_to` → `replyTo`, `variables_json` → `variables`, and `active` keeps its stored meaning. `label` falls back to the name and `locale` to `EMAIL_TEMPLATE_FLOOR_LOCALE` (`en-US`). The retired `category` / `is_system` columns and every provenance or state column are left out. It does not validate; the door does. Nothing existing changes. diff --git a/packages/cli/src/commands/migrate/organization-ownership.ts b/packages/cli/src/commands/migrate/organization-ownership.ts index ea6e0f26dd..e2a7b80c05 100644 --- a/packages/cli/src/commands/migrate/organization-ownership.ts +++ b/packages/cli/src/commands/migrate/organization-ownership.ts @@ -29,6 +29,7 @@ import { applyOrganizationOwnership, checkpointFromDetails, ORGANIZATION_OWNERSHIP_MIGRATION_ID, + TemplateChoiceRefusal, type ApplyExecutor, type CeremonyLedgerPort, type ColumnDropOutcome, @@ -142,6 +143,13 @@ export default class MigrateOrganizationOwnership extends Command { 'none; without this flag the run asks, and a non-interactive run is refused', default: false, }), + 'template-choice': Flags.string({ + description: + 'With --apply: the sys_email_template row id the environment keeps for a listed template conflict ' + + '(rows of one name and locale that differ). Repeat once per conflict; the other rows of that conflict are ' + + 'deleted only after the chosen body is promoted and verified', + multiple: true, + }), }; async run(): Promise { @@ -174,6 +182,10 @@ export default class MigrateOrganizationOwnership extends Command { await this.runApply(flags, posture, refuse); return; } + if (flags['template-choice']?.length) { + await refuse({ reason: 'flag-without-apply', detail: '--template-choice is only meaningful with --apply' }, '--template-choice is only meaningful with --apply.'); + return; + } if (flags['backup-taken']) { await refuse({ reason: 'flag-without-apply', detail: '--backup-taken is only meaningful with --apply' }, '--backup-taken is only meaningful with --apply.'); return; @@ -263,6 +275,15 @@ export default class MigrateOrganizationOwnership extends Command { `${table.notNull.willReceive ? chalk.green(' NOT NULL') : ''}`, ); } + for (const table of plan.tables) { + for (const conflict of table.templateConflicts ?? []) { + printWarning( + `Template conflict ${conflict.name} (${conflict.locale}): ` + + conflict.candidates.map((c) => `${c.id} (organization ${c.organizationId ?? 'none'})`).join(', ') + + ' differ. --apply blocks on it until one row is chosen with --template-choice ROW_ID.', + ); + } + } console.log(''); const { summary } = plan; printInfo( @@ -278,7 +299,7 @@ export default class MigrateOrganizationOwnership extends Command { /** `--apply`: the backup precondition, the boot, the ceremony, the report file. */ private async runApply( - flags: { 'database-url'?: string; out?: string; json?: boolean; 'backup-taken'?: boolean }, + flags: { 'database-url'?: string; out?: string; json?: boolean; 'backup-taken'?: boolean; 'template-choice'?: string[] }, posture: string, refuse: (payload: Record, message: string) => Promise, ): Promise { @@ -374,6 +395,7 @@ export default class MigrateOrganizationOwnership extends Command { const liveStack = stack; report = await applyOrganizationOwnership({ executor, + ...(flags['template-choice']?.length ? { templateChoices: flags['template-choice'] } : {}), posture, database: stack.dbLabel, catalog, @@ -437,6 +459,10 @@ export default class MigrateOrganizationOwnership extends Command { }); } catch (error: unknown) { if (isExitSignal(error)) throw error; + if (error instanceof TemplateChoiceRefusal) { + await refuse({ reason: 'template-choice-refused', detail: error.message }, `Refused: ${error.message}`); + return; + } if (error instanceof OrganizationOwnershipPlanRefusal) { await refuse( { reason: error.reason, ...(error.table ? { table: error.table } : {}), detail: error.message }, @@ -480,16 +506,25 @@ export default class MigrateOrganizationOwnership extends Command { console.log(''); for (const table of report.tables) { if (!table.before) continue; - if (table.attributed + table.deleted + table.unattributable.length + table.remaining + table.departures.length === 0 && !table.columnDropped) continue; + if (table.attributed + table.deleted + table.movedOut + table.movedIn + table.unattributable.length + table.remaining + table.departures.length === 0 && !table.columnDropped) continue; console.log( ` ${chalk.white(table.object.padEnd(34))} ${table.fate.padEnd(16)} attributed ${String(table.attributed).padStart(6)} ` + - `deleted ${String(table.deleted).padStart(6)} ${table.columnDropped ? chalk.green('column dropped ') : ''}` + + `deleted ${String(table.deleted).padStart(6)} ` + + `${table.movedOut ? `moved out ${table.movedOut} ` : ''}${table.movedIn ? `moved in ${table.movedIn} ` : ''}` + + `${table.columnDropped ? chalk.green('column dropped ') : ''}` + `${table.unattributable.length ? chalk.yellow(`reported ${table.unattributable.length} `) : ''}` + `${table.remaining ? chalk.red(`remaining ${table.remaining}`) : ''}`, ); for (const departure of table.departures) { console.log(chalk.yellow(` ${departure.rows} row(s) still to leave by a step not yet run — ${departure.label}`)); } + for (const conflict of table.templateConflicts ?? []) { + console.log(chalk.yellow( + ` template conflict ${conflict.name} (${conflict.locale}): ` + + conflict.candidates.map((c) => `${c.id} (organization ${c.organizationId ?? 'none'})`).join(', ') + + ' — choose one with --template-choice ROW_ID', + )); + } } console.log(''); if (report.advisory > 0) { diff --git a/packages/cli/src/utils/organization-ownership-apply.test.ts b/packages/cli/src/utils/organization-ownership-apply.test.ts index cae958c0b1..388fad9ee3 100644 --- a/packages/cli/src/utils/organization-ownership-apply.test.ts +++ b/packages/cli/src/utils/organization-ownership-apply.test.ts @@ -24,6 +24,7 @@ import { join } from 'node:path'; import { permissionSetDefinitionFromRow } from '@objectstack/core'; import { applyOrganizationOwnership, + TemplateChoiceRefusal, type ApplyExecutor, type CatalogPort, type CeremonyCheckpoint, @@ -284,11 +285,12 @@ async function accessAfter(c: ReturnType): Promise { const apply = ( c: ReturnType, l: ReturnType, - opts: { posture?: string; executor?: ApplyExecutor } = {}, + opts: { posture?: string; executor?: ApplyExecutor; templateChoices?: string[] } = {}, ): Promise => { c.boot(); return applyOrganizationOwnership({ executor: opts.executor ?? executor(), + ...(opts.templateChoices ? { templateChoices: opts.templateChoices } : {}), posture: opts.posture ?? 'single', database: file, catalog: c, @@ -300,6 +302,25 @@ const apply = ( }); }; +/** Run `body` against a database holding nothing but the given templates, so every step can finish under any posture. */ +async function onlyTemplates(values: string, body: () => Promise): Promise { + const only = new Database(join(dir, 'templates.db')); + only.exec(` + CREATE TABLE sys_organization (id TEXT PRIMARY KEY, slug TEXT); + CREATE TABLE sys_email_template (id TEXT PRIMARY KEY, organization_id TEXT, name TEXT, locale TEXT, subject TEXT, + customized INTEGER, managed_by TEXT); + INSERT INTO sys_email_template VALUES ${values}; + `); + const saved = db; + db = only; + try { + await body(); + } finally { + db = saved; + only.close(); + } +} + const tableOf = (report: OrganizationOwnershipApplyReport, object: string) => { const found = report.tables.find((t) => t.object === object); if (!found) throw new Error(`no ${object} in the report`); @@ -407,7 +428,7 @@ describe('ADR-0131 D10 --apply — the acceptance fixture', () => { expect(tableOf(report, 'sys_platform_setting').movedIn).toBe(2); // Every delete is fate 2's, or the moved rows leaving sys_setting. expect(writes.filter((w) => /^DELETE/.test(w)).every((w) => - /sys_permission_set|sys_position|sys_position_permission_set|sys_capability|sys_email_template|"sys_setting" WHERE \("scope" = \?\)/.test(w))).toBe(true); + /sys_permission_set|sys_position|sys_position_permission_set|sys_capability|sys_email_template|"sys_setting" WHERE \(\("scope" = \?\) AND "organization_id" IS NULL\)/.test(w))).toBe(true); }); it('positive control: a grant the rewrite would lose reddens the effective-access pin', async () => { @@ -475,6 +496,36 @@ describe('ADR-0131 D10 --apply — idempotent, resumable, gated', () => { expect(resumed.complete).toBe(false); }); + it('promotes again before deleting a template: a row customized between runs is promoted, never deleted unpromoted', async () => { + const c = catalog(); + const l = ledger(); + await expect(apply(c, l, { executor: executor({ failOnceOn: /^DELETE FROM "sys_capability"/ }) })).rejects.toThrow(/killed mid-step/); + expect(l.checkpoint!.completed['template-promotion']).toBeDefined(); + db.exec(`INSERT INTO sys_email_template VALUES + ('tpl_late', NULL, 'crm.late', 'Late', 'en-US', 'Edited late', '

late

', 1, '[]', 1, 'platform')`); + + const resumed = await apply(c, l); + + expect(c.templates.get('crm.late|en-US')).toMatchObject({ subject: 'Edited late' }); + expect(rows(`SELECT id FROM sys_email_template`)).toEqual([]); + expect(resumed.complete).toBe(true); + expect(resumed.blocking).toBe(0); + }); + + it('a customized template with no name blocks: kept, named, and verified_at stays null', async () => { + db.exec(`INSERT INTO sys_email_template VALUES + ('tpl_nameless', NULL, NULL, 'Untitled', 'en-US', 'S', '

s

', 1, '[]', 1, 'platform')`); + const l = ledger(); + const report = await apply(catalog(), l); + + const promotion = report.steps.find((s) => s.step === 'template-promotion')!; + expect(promotion.status).toBe('blocked'); + expect(promotion.blockers.join('\n')).toMatch(/tpl_nameless/); + expect(rows(`SELECT id FROM sys_email_template WHERE id = 'tpl_nameless'`)).toHaveLength(1); + expect(report.complete).toBe(false); + expect(l.verdicts.at(-1)).toMatchObject({ complete: false }); + }); + it('moves no setting over a different value the new table already serves, and deletes nothing for it', async () => { db.exec(`INSERT INTO sys_platform_setting VALUES ('ps_1', 'mail', 'transport', '"ses"', 0, 0, NULL, NULL)`); const report = await apply(catalog(), ledger()); @@ -530,7 +581,7 @@ describe('ADR-0131 D10 --apply — idempotent, resumable, gated', () => { expect(l.verdicts.at(-1)!.complete).toBe(false); }); - it('under a wall: an anchorless row is reported, never assigned, and authored rows are not converted', async () => { + it('under a wall: an anchorless row is reported, never assigned, authored rows are not converted, templates are promoted', async () => { const c = catalog(); const report = await apply(c, ledger(), { posture: 'isolated' }); @@ -540,7 +591,8 @@ describe('ADR-0131 D10 --apply — idempotent, resumable, gated', () => { expect(c.saved.size).toBe(0); expect(report.steps.find((s) => s.step === 'mirror-deletion')!.status).toBe('not-reached'); expect(rows('SELECT COUNT(*) AS n FROM sys_permission_set')).toEqual([{ n: 3 }]); - expect(c.templates.size).toBe(0); + // The promotion runs under every posture; nothing is deleted before mirror deletion. + expect([...c.templates.keys()].sort()).toEqual(['auth.reset_password|en-US', 'crm.memo|en-US', 'crm.note|en-US', 'crm.renewal|zh-CN']); expect(rows('SELECT COUNT(*) AS n FROM sys_email_template')).toEqual([{ n: 6 }]); }); @@ -565,29 +617,79 @@ describe('ADR-0131 D10 --apply — idempotent, resumable, gated', () => { expect(rows(`SELECT id FROM sys_sharing_rule ORDER BY id`).map((r) => r.id)).toEqual(['rule_declared', 'rule_org']); }); - it('under a wall, a template no promotion carried keeps verified_at null and is named', async () => { - // A database holding nothing but templates: every step can finish under a wall. - const only = new Database(join(dir, 'templates.db')); - only.exec(` - CREATE TABLE sys_organization (id TEXT PRIMARY KEY, slug TEXT); - CREATE TABLE sys_email_template (id TEXT PRIMARY KEY, organization_id TEXT, name TEXT, customized INTEGER, managed_by TEXT); - INSERT INTO sys_email_template VALUES ('tpl_seeded', NULL, 'auth.welcome', 0, 'platform'), ('tpl_admin', NULL, 'crm.note', 0, 'admin'); - `); - const saved = db; - db = only; - try { - const l = ledger(); - const report = await apply(catalog(), l, { posture: 'isolated' }); - expect(report.complete).toBe(true); - expect(report.blocking).toBe(1); - expect(l.verdicts.at(-1)).toMatchObject({ complete: true, blocking: 1 }); - const templates = tableOf(report, 'sys_email_template'); - expect(templates.departures).toEqual([expect.objectContaining({ id: 'template-promotion', rows: 1 })]); - expect(rows('SELECT id FROM sys_email_template')).toEqual([{ id: 'tpl_admin' }]); - } finally { - db = saved; - only.close(); - } + it('promotes under a wall too: a template with no conflict ends verified, its seeded mirror and its row gone', async () => { + await onlyTemplates( + `('tpl_seeded', NULL, 'auth.welcome', 'en-US', 'Hi', 0, 'platform'), ('tpl_admin', 'org_b', 'crm.note', 'en-US', 'A note', 0, 'admin')`, + async () => { + const c = catalog(); + const l = ledger(); + const report = await apply(c, l, { posture: 'isolated' }); + expect(report.complete).toBe(true); + expect(report.blocking).toBe(0); + expect(l.verdicts.at(-1)).toMatchObject({ complete: true, blocking: 0 }); + expect(c.templates.get('crm.note|en-US')).toMatchObject({ name: 'crm.note', subject: 'A note' }); + expect(tableOf(report, 'sys_email_template').departures).toEqual([]); + expect(rows('SELECT id FROM sys_email_template')).toEqual([]); + }, + ); + }); + + it('under a wall, rows of one template that differ are a conflict: listed with each candidate, kept, and blocking', async () => { + await onlyTemplates( + `('tpl_a', 'org_a', 'crm.note', 'en-US', 'From A', 1, 'platform'), ('tpl_b', 'org_b', 'crm.note', 'en-US', 'From B', 1, 'platform')`, + async () => { + const c = catalog(); + const l = ledger(); + const report = await apply(c, l, { posture: 'group' }); + const promotion = report.steps.find((s) => s.step === 'template-promotion')!; + expect(promotion.status).toBe('blocked'); + expect(promotion.summary).toMatchObject({ conflicts: 1, promoted: 0 }); + expect(promotion.blockers.join('\n')).toMatch(/crm\.note \(en-US\).*tpl_a \(organization org_a\), tpl_b \(organization org_b\).*--template-choice/); + expect(tableOf(report, 'sys_email_template').templateConflicts).toEqual([ + { name: 'crm.note', locale: 'en-US', candidates: [{ id: 'tpl_a', organizationId: 'org_a' }, { id: 'tpl_b', organizationId: 'org_b' }] }, + ]); + expect(report.complete).toBe(false); + expect(l.verdicts.at(-1)).toMatchObject({ complete: false }); + expect(c.templates.size).toBe(0); + expect(rows('SELECT id FROM sys_email_template ORDER BY id').map((r) => r.id)).toEqual(['tpl_a', 'tpl_b']); + }, + ); + }); + + it('a chosen conflict promotes the chosen body, verifies, and only then deletes every candidate', async () => { + await onlyTemplates( + `('tpl_a', 'org_a', 'crm.note', 'en-US', 'From A', 1, 'platform'), ('tpl_b', 'org_b', 'crm.note', 'en-US', 'From B', 1, 'platform')`, + async () => { + const c = catalog(); + const l = ledger(); + const report = await apply(c, l, { posture: 'isolated', templateChoices: ['tpl_b'] }); + expect(report.complete).toBe(true); + expect(report.blocking).toBe(0); + expect(l.verdicts.at(-1)).toMatchObject({ complete: true, blocking: 0 }); + expect(c.templates.get('crm.note|en-US')).toMatchObject({ subject: 'From B' }); + expect(rows('SELECT id FROM sys_email_template')).toEqual([]); + expect(tableOf(report, 'sys_email_template').deleted).toBe(2); + }, + ); + }); + + it('refuses a template choice that names no candidate, or two candidates of one conflict, before anything is written', async () => { + await onlyTemplates( + `('tpl_a', 'org_a', 'crm.note', 'en-US', 'From A', 1, 'platform'), ('tpl_b', 'org_b', 'crm.note', 'en-US', 'From B', 1, 'platform')`, + async () => { + const c = catalog(); + await expect(apply(c, ledger(), { posture: 'isolated', templateChoices: ['tpl_nope'] })).rejects.toThrow( + new TemplateChoiceRefusal( + '--template-choice tpl_nope names no candidate row of a listed template conflict. The conflicts are: crm.note (en-US): tpl_a, tpl_b.', + ), + ); + await expect(apply(c, ledger(), { posture: 'isolated', templateChoices: ['tpl_a', 'tpl_b'] })).rejects.toThrow( + /tpl_a and tpl_b both choose for the template crm\.note \(en-US\)/, + ); + expect(writes).toEqual([]); + expect(c.templates.size).toBe(0); + }, + ); }); it('holds a column whose rows another stage owns, and reports it', async () => { diff --git a/packages/cli/src/utils/organization-ownership-apply.ts b/packages/cli/src/utils/organization-ownership-apply.ts index 7e290e53b1..5805c2d00e 100644 --- a/packages/cli/src/utils/organization-ownership-apply.ts +++ b/packages/cli/src/utils/organization-ownership-apply.ts @@ -2,7 +2,6 @@ import { convertPositionBindingRows, - emailTemplateDefinitionFromRow, permissionSetDefinitionFromRow, positionDefinitionFromRow, } from '@objectstack/core'; @@ -14,11 +13,17 @@ import { import { buildOrganizationOwnershipPlan, columnListSql, + definitionKey, + groupTemplateRows, + templatePromotionSql, + SEEDED_BY, + TEMPLATE_OBJECT, columnNameOf, dialectOf, predicateSql, Sql, type AttributionSink, + type TemplateConflict, type OrganizationOwnershipPlan, type PlanReader, type TablePlan, @@ -47,12 +52,14 @@ import { * `sys_email_template` row an organization customized or authored becomes * an environment `email_template` item through the metadata door (core's * `emailTemplateDefinitionFromRow`), so the deletion that follows loses no - * customer edit. Under `single` only; under a wall one organization's edit - * would become every organization's, so the rows are reported and kept. + * customer edit, under every posture. One `(name, locale)` held by rows + * that differ is a conflict: listed, kept, and blocking until the operator + * chooses one row with `--template-choice ROW_ID`; a nameless row blocks. * 2c. `settings-move` — the inventory's `global-rung-move` departure, * executed (#15207's ruling: "C7's ceremony moving the existing rows"): - * each `sys_setting` row of `scope = 'global'` is copied, by the database - * itself, into `sys_platform_setting` and then removed from `sys_setting`. + * each `sys_setting` row of `scope = 'global'` with no organization is + * copied, by the database itself, into `sys_platform_setting`, read back, + * and only then removed from `sys_setting`. * A key the new table already holds with a different value is a blocker. * 3. `rewrite` — the verified id→name rewrite (D4): * `sys_user_permission_set.permission_set` is named from the set row its @@ -61,7 +68,9 @@ import { * 4. `mirror-deletion` (fate 2) — gated on the rewrite's verification, which * is read again immediately before the first delete. Only rows of a * `mirror-deletion` object that match its mirror selector and none of its - * named categories are deleted. + * named categories are deleted. Templates: the promotion runs again first, + * and beyond the seeded mirrors only the rows whose environment copy that + * pass verified are deleted, by id. * 5. `column-drops` (fate 1) — `organization_id` leaves the D7 objects * through the schema-drift ceremony ({@link ColumnDropPort}), never a hand * `ALTER`. @@ -176,6 +185,8 @@ export interface TableOutcome { * until it is gone, and its label is the remedy. */ departures: Array<{ id: string; label: string; as: 'mirror-deletion' | 'moved'; rows: number }>; + /** Templates: each `(name, locale)` whose rows still differ, with every candidate row and its organization. */ + templateConflicts?: TemplateConflict[]; /** Post-check: what is left for this table's fate, `0` when the fate is done. */ remaining: number; notNull: TablePlan['notNull']; @@ -211,6 +222,12 @@ export interface ApplyOptions { columnDrops: ColumnDropPort; ledger: CeremonyLedgerPort; inventory?: readonly InventoryEntry[]; + /** + * The operator's choice for each template conflict: one candidate row id per + * `(name, locale)` whose customized or authored rows differ. Validated before + * anything is written ({@link TemplateChoiceRefusal}). + */ + templateChoices?: readonly string[]; now?: () => Date; /** One line per milestone, for the command's human face. */ onProgress?: (message: string) => void; @@ -226,43 +243,10 @@ const SETS = 'sys_permission_set'; const POSITIONS = 'sys_position'; const BINDINGS = 'sys_position_permission_set'; const CAPABILITIES = 'sys_capability'; -const TEMPLATES = 'sys_email_template'; +const TEMPLATES = TEMPLATE_OBJECT; const SETTINGS = 'sys_setting'; const PLATFORM_SETTINGS = 'sys_platform_setting'; -/** Provenance the seeders stamped: only such a row can be a copy of code. */ -const SEEDED_BY = ['package', 'platform', 'system']; - -/** - * The `sys_email_template` rows the promotion carries: exactly the rows mirror - * deletion keeps for it — a customized row (or one whose `customized` is - * unknown), and every row no seeder wrote (`managed_by` NULL or outside - * package / platform / system), with or without an organization. A seeded, - * uncustomized row — NULL residue or an organization's copy — is a copy of - * code: it is not promoted, and it is deleted as a mirror. - */ -const templatePromotionSql = (sql: Sql, cols: ReadonlySet): { text: string; params: unknown[] } => { - const t = TEMPLATES; - const c = sql.q('customized', t); - const customized = cols.has('customized') ? `${c} = ? OR ${c} IS NULL` : '1 = 0'; - const params: unknown[] = cols.has('customized') ? [sql.bind(true)] : []; - if (!cols.has('managed_by')) return { text: '1 = 1', params: [] }; - const managed = sql.q('managed_by', t); - return { - text: `${customized} OR ${managed} IS NULL OR ${managed} NOT IN (${SEEDED_BY.map(() => '?').join(', ')})`, - params: [...params, ...SEEDED_BY], - }; -}; - -/** - * Categories of a fate-2 object that the promotion discharges: once it has - * run (under `single`), the rows they claim have their environment copy and - * are mirrors like the rest. - */ -const DISCHARGED_BY_PROMOTION: Readonly> = { - [TEMPLATES]: ['template-promotion', 'organization-stamped'], -}; - /** The `moved` departures the ceremony executes, by departure id. */ const MOVES: Readonly> = { 'global-rung-move': { @@ -292,6 +276,20 @@ interface Run { * the next one; the door's acceptance is what makes it resolve now. */ saved: Map>>; + /** + * Template rows whose environment copy this run verified (promoted, or found + * equal): only these, and seeded mirrors, may be deleted. Rebuilt by the + * promotion pass mirror deletion re-runs, so a resumed run verifies afresh. + */ + verifiedTemplateIds: Set; +} + +/** A `--template-choice` that names no candidate of a listed conflict, or two of one. Nothing is written. */ +export class TemplateChoiceRefusal extends Error { + constructor(message: string) { + super(message); + this.name = 'TemplateChoiceRefusal'; + } } /** The catalog read, then what the door accepted this run. */ @@ -427,8 +425,7 @@ function authoredWhere(object: string, inventory: readonly InventoryEntry[] | un const rowIsOff = (value: unknown): boolean => value === false || value === 0 || value === '0' || value === 'false'; /** The JSON of a definition's own keys, for comparing two bodies. */ -const bodyKey = (body: Readonly>, keys: readonly string[]): string => - JSON.stringify(keys.map((k) => [k, body[k] ?? null])); +const bodyKey = definitionKey; interface CatalogRows { positions: Record[]; @@ -621,59 +618,49 @@ async function runAuthoredConversion(run: Run): Promise { // ── Step 2b: template promotion (ruling C) ────────────────────────────────── async function runTemplatePromotion(run: Run): Promise { - const summary: Record = { promoted: 0, alreadyPromoted: 0, reported: 0 }; + const summary: Record = { promoted: 0, alreadyPromoted: 0, conflicts: 0 }; const blockers: string[] = []; const notes: string[] = []; + run.verifiedTemplateIds.clear(); const cols = await columnsOf(run, TEMPLATES); if (!cols) return { step: 'template-promotion', status: 'done', summary, blockers, notes }; const rows = await readAll(run, TEMPLATES, [...cols], templatePromotionSql(run.sql, cols)); - if (rows.length === 0) return { step: 'template-promotion', status: 'done', summary, blockers, notes }; - if (run.opts.posture !== 'single') { - summary.reported = rows.length; - notes.push( - `${rows.length} customized or authored ${TEMPLATES} row(s) are reported and kept, not promoted: under the ` + - `${run.opts.posture} posture an environment template would carry one organization's edit to every organization.`, + const { groups, nameless } = groupTemplateRows(rows); + if (nameless.length) { + blockers.push( + `${nameless.length} customized or authored ${TEMPLATES} row(s) carry no name (${nameless.join(', ')}), so no ` + + 'environment template can hold them, and they are never deleted unpromoted. Name each row, or delete it, then re-run.', ); - return { step: 'template-promotion', status: 'done', summary, blockers, notes }; - } - const groups = new Map[]; bodies: Record[] }>(); - for (const row of rows) { - const body = emailTemplateDefinitionFromRow(row); - const name = text(body.name); - if (!name) { - summary.reported++; - notes.push(`A ${TEMPLATES} row (${String(row.id)}) carries no name and is reported, never promoted.`); - continue; - } - const locale = String(body.locale); - const key = `${name}\u0000${locale}`; - const group = groups.get(key) ?? { name, locale, rows: [], bodies: [] }; - group.rows.push(row); - group.bodies.push(body); - groups.set(key, group); } - for (const { name, locale, rows: group, bodies } of groups.values()) { - const keys = [...new Set(bodies.flatMap(Object.keys))]; - if (new Set(bodies.map((b) => bodyKey(b, keys))).size > 1) { - blockers.push( - `The template ${name} (${locale}) is held by ${group.length} customized or authored rows ` + - `(${group.map((r) => String(r.id)).join(', ')}) that differ. Which edit the environment keeps is never ` + - 'guessed: make them agree, or delete all but one, then re-run.', - ); - continue; + const choices = new Set(run.opts.templateChoices ?? []); + for (const group of groups) { + const { name, locale } = group; + let body = group.bodies[0]!; + if (group.conflict) { + summary.conflicts++; + const chosen = group.rows.findIndex((r) => choices.has(String(r.id))); + if (chosen < 0) { + blockers.push( + `The template ${name} (${locale}) is held by ${group.rows.length} customized or authored rows that differ: ` + + group.rows.map((r) => `${String(r.id)} (organization ${text(r.organization_id) ?? 'none'})`).join(', ') + + '. Which one the environment keeps is never guessed: choose one with --template-choice ROW_ID, then re-run.', + ); + continue; + } + body = group.bodies[chosen]!; } - const body = bodies[0]!; const existing = await run.opts.templates.find(name, locale); const saved = run.saved.get(`email_template|${name}|${locale}`); const current = saved ?? existing?.definition; if (current && bodyKey(current, Object.keys(body)) === bodyKey(body, Object.keys(body))) { summary.alreadyPromoted++; + for (const r of group.rows) run.verifiedTemplateIds.add(String(r.id)); continue; } if (existing?.environment && !saved) { blockers.push( `The environment already holds its own template ${name} (${locale}), and it differs from the customized row(s) ` + - `${group.map((r) => String(r.id)).join(', ')}. The environment's edit is never overwritten: carry the ` + + `${group.rows.map((r) => String(r.id)).join(', ')}. The environment's edit is never overwritten: carry the ` + 'row\'s change into it in Studio (or delete the row), then re-run.', ); continue; @@ -681,6 +668,7 @@ async function runTemplatePromotion(run: Run): Promise { try { await run.opts.door.save('email_template', name, body); run.saved.set(`email_template|${name}|${locale}`, body); + for (const r of group.rows) run.verifiedTemplateIds.add(String(r.id)); summary.promoted++; run.opts.onProgress?.(`promoted the template ${name} (${locale}) to environment metadata`); } catch (error) { @@ -693,6 +681,37 @@ async function runTemplatePromotion(run: Run): Promise { return { step: 'template-promotion', status: blockers.length ? 'blocked' : 'done', summary, blockers, notes }; } +/** + * Refuse, before anything is written, a `--template-choice` that names no + * candidate row of a listed conflict, or two candidates of one conflict. + */ +function validateTemplateChoices(run: Run): void { + const choices = run.opts.templateChoices ?? []; + if (choices.length === 0) return; + const conflicts = run.plan.tables.find((t) => t.object === TEMPLATES)?.templateConflicts ?? []; + const groupOf = new Map(); + for (const c of conflicts) for (const cand of c.candidates) groupOf.set(cand.id, `${c.name} (${c.locale})`); + const listed = conflicts.length + ? conflicts.map((c) => `${c.name} (${c.locale}): ${c.candidates.map((cand) => cand.id).join(', ')}`).join('; ') + : 'none'; + const seen = new Map(); + for (const id of choices) { + const group = groupOf.get(id); + if (!group) { + throw new TemplateChoiceRefusal( + `--template-choice ${id} names no candidate row of a listed template conflict. The conflicts are: ${listed}.`, + ); + } + const other = seen.get(group); + if (other !== undefined && other !== id) { + throw new TemplateChoiceRefusal( + `--template-choice ${other} and ${id} both choose for the template ${group}; choose exactly one row per conflict.`, + ); + } + seen.set(group, id); + } +} + // ── Step 2c: the settings global rung moves (D7) ──────────────────────────── async function runSettingsMove(run: Run): Promise { @@ -704,7 +723,11 @@ async function runSettingsMove(run: Run): Promise { if (departure.as !== 'moved' || !move || entry.object !== move.from) continue; const fromCols = await columnsOf(run, move.from); if (!fromCols) continue; - const where = predicateSql(run.sql, move.from, fromCols, departure.where); + const selector = predicateSql(run.sql, move.from, fromCols, departure.where); + // Only the environment's own rung moves: an organization's row is not global. + const where = fromCols.has('organization_id') + ? { text: `(${selector.text}) AND ${run.sql.q('organization_id', move.from)} IS NULL`, params: selector.params } + : selector; const rows = await readAll(run, move.from, [...move.key, ...move.compare].filter((c) => fromCols.has(c)), where); if (rows.length === 0) continue; const toCols = await columnsOf(run, move.to); @@ -730,10 +753,12 @@ async function runSettingsMove(run: Run): Promise { ); continue; } - const target = (await readAll(run, move.to, move.compare.filter((c) => toCols.has(c)), { - text: move.key.map((k) => `${sql.q(k, move.to)} = ?`).join(' AND '), - params: move.key.map((k) => group[0]![k]), - })); + const readTarget = () => + readAll(run, move.to, move.compare.filter((c) => toCols.has(c)), { + text: move.key.map((k) => `${sql.q(k, move.to)} = ?`).join(' AND '), + params: move.key.map((k) => group[0]![k]), + }); + const target = await readTarget(); if (target.length > 0 && !same(target[0]!, group[0]!)) { blockers.push( `${move.to} already holds ${key} with a different value — the one the runtime serves — and the ${move.from} ` + @@ -751,6 +776,14 @@ async function runSettingsMove(run: Run): Promise { `FROM ${sql.q(move.from, move.from)} WHERE ${sql.q('id', move.from)} = ?`, [group[0]!.id], ); + // The source leaves only once its destination row reads back equal. + const landed = await readTarget(); + if (landed.length !== 1 || !same(landed[0]!, group[0]!)) { + throw new Error( + `The move of ${key} into ${move.to} did not read back equal; the ${move.from} row(s) ` + + `${group.map((r) => String(r.id)).join(', ')} were kept.`, + ); + } summary.moved++; run.movedIn.set(move.to, (run.movedIn.get(move.to) ?? 0) + 1); } else { @@ -948,17 +981,18 @@ function deletableWhere(run: Run, table: string, cols: ReadonlySet, entr parts.push(`(CASE WHEN ${claimed.text} THEN 1 WHEN NOT (${claimed.text}) THEN 0 ELSE 1 END) = 0`); params.push(...claimed.params, ...claimed.params); }; - // The template promotion runs under `single` only, and before this step: - // there, the rows it carried are discharged — their environment copy - // exists, so they are mirrors like the rest. Elsewhere nothing carried - // them, and every claim keeps its rows. - const promoted = run.opts.posture === 'single' && entry.object === TEMPLATES; - const discharged = promoted ? DISCHARGED_BY_PROMOTION[entry.object] ?? [] : []; + // Templates: a mirror is a seeded, uncustomized copy (with an organization + // or without); every promotion row is kept here, and is deleted only by id + // once this run has verified its environment copy (`runMirrorDeletion`). + if (entry.object === TEMPLATES) { + keep(templatePromotionSql(run.sql, cols)); + return { text: parts.join(' AND '), params }; + } for (const category of entry.categories ?? []) { - if (!category.where || category.conflictBy || discharged.includes(category.id)) continue; + if (!category.where || category.conflictBy) continue; keep(predicateSql(run.sql, table, cols, category.where)); } - if (cols.has('managed_by') && !promoted) { + if (cols.has('managed_by')) { keep(predicateSql(run.sql, table, cols, [{ column: 'managed_by', notIn: SEEDED_BY }])); } return { text: parts.length ? parts.join(' AND ') : '1 = 1', params }; @@ -983,6 +1017,21 @@ async function runMirrorDeletion(run: Run): Promise { notes: [], }; } + // The promotion, run again now: on a resumed run its checkpoint says nothing + // about a row customized or authored since, and only a row whose copy this + // pass verifies may be deleted. It is idempotent: an equal copy is found. + if (await columnsOf(run, TEMPLATES)) { + const pass = await runTemplatePromotion(run); + if (pass.status !== 'done') { + return { + step: 'mirror-deletion', + status: 'blocked', + summary: { deleted: 0 }, + blockers: ['Mirror deletion re-runs the template promotion first, and it no longer completes:', ...pass.blockers], + notes: [], + }; + } + } const notes: string[] = []; let total = 0; for (const table of run.plan.tables) { @@ -994,6 +1043,7 @@ async function runMirrorDeletion(run: Run): Promise { const where = deletableWhere(run, physical, cols, entry); const before = await countWhere(run, physical, where); if (before === 0) continue; + await execute(run, physical, `DELETE FROM ${run.sql.q(physical, physical)} WHERE ${where.text}`, where.params); const left = await countWhere(run, physical, where); if (left !== 0) { @@ -1003,12 +1053,36 @@ async function runMirrorDeletion(run: Run): Promise { total += before; run.opts.onProgress?.(`mirror deletion: ${before} row(s) of ${physical}`); } + if (table.object === TEMPLATES) total += await deletePromotedTemplates(run, table.tables); const kept = Math.max(0, table.rows!.total - (run.deleted.get(table.object) ?? 0)); if (kept > 0) notes.push(`${table.object}: ${kept} row(s) are not mirrors (a named category or their non-seeded provenance claims them) and stay.`); } return { step: 'mirror-deletion', status: 'done', summary: { deleted: total }, blockers: [], notes }; } +/** Delete, by id, the promotion rows whose environment copy this run verified; nothing else. */ +async function deletePromotedTemplates(run: Run, tables: readonly string[]): Promise { + const ids = [...run.verifiedTemplateIds]; + let deleted = 0; + for (const physical of tables) { + const id = run.sql.q('id', physical); + for (let i = 0; i < ids.length; i += IN_CHUNK) { + const chunk = ids.slice(i, i + IN_CHUNK); + const where = { text: `${id} IN (${chunk.map(() => '?').join(', ')})`, params: chunk as unknown[] }; + const before = await countWhere(run, physical, where); + if (before === 0) continue; + await execute(run, physical, `DELETE FROM ${run.sql.q(physical, physical)} WHERE ${where.text}`, where.params); + if ((await countWhere(run, physical, where)) !== 0) { + throw new Error(`The delete of promoted template rows on ${physical} left rows behind; nothing past this table was deleted.`); + } + deleted += before; + } + } + run.deleted.set(TEMPLATES, (run.deleted.get(TEMPLATES) ?? 0) + deleted); + if (deleted) run.opts.onProgress?.(`mirror deletion: ${deleted} promoted template row(s), each with its environment copy`); + return deleted; +} + // ── Step 5: column drops (fate 1) ──────────────────────────────────────────── async function runColumnDrops(run: Run): Promise { @@ -1076,9 +1150,9 @@ async function remainingFor(run: Run, table: TablePlan, complete: boolean): Prom for (const physical of table.tables) { const cols = (await columnsOf(run, physical))!; left += await countWhere(run, physical, deletableWhere(run, physical, cols, entry)); - // Template rows no promotion carried (none runs under a wall) still - // hold a customer's edit with no environment copy: they remain. - if (table.object === TEMPLATES && run.opts.posture !== 'single') { + // A promotion row leaves only once its environment copy is verified: + // until then it remains, under every posture. + if (table.object === TEMPLATES) { left += await countWhere(run, physical, templatePromotionSql(run.sql, cols)); } } @@ -1110,9 +1184,11 @@ export async function applyOrganizationOwnership(opts: ApplyOptions): Promise d.rows > 0).map((d) => ({ id: d.id, label: d.label, as: d.as, rows: d.rows })); - if (a?.physical === 'present' && object === TEMPLATES && opts.posture !== 'single') { + if (a?.physical === 'present' && object === TEMPLATES) { let unpromoted = 0; for (const physical of a.tables) { unpromoted += await countWhere(run, physical, templatePromotionSql(run.sql, (await columnsOf(run, physical))!)); @@ -1172,7 +1248,7 @@ export async function applyOrganizationOwnership(opts: ApplyOptions): Promise 0) { departures.push({ id: 'template-promotion', - label: `customized or admin-authored templates no promotion carried (none runs under the ${opts.posture} posture)`, + label: 'customized or admin-authored templates not yet promoted (a conflict, a nameless row or a blocker names each)', as: 'moved', rows: unpromoted, }); @@ -1191,6 +1267,7 @@ export async function applyOrganizationOwnership(opts: ApplyOptions): Promise 0 ? { willReceive: false, reason: `${waiting} row(s) still wait for a step not yet run: ${departures.map((d) => d.label).join('; ')}` } diff --git a/packages/cli/src/utils/organization-ownership-plan.ts b/packages/cli/src/utils/organization-ownership-plan.ts index f3a384ebd8..0a3a60ac91 100644 --- a/packages/cli/src/utils/organization-ownership-plan.ts +++ b/packages/cli/src/utils/organization-ownership-plan.ts @@ -1,6 +1,7 @@ // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. import { createHash } from 'node:crypto'; +import { emailTemplateDefinitionFromRow } from '@objectstack/core'; import { hasPlatformObjectPrefix } from '@objectstack/spec/system'; import { APPLICATION_OBJECT_FATE, @@ -147,6 +148,100 @@ export interface TablePlan { cloudCarried?: true; external?: true; note?: string; + /** + * `sys_email_template` only: the `(name, locale)` groups of the promotion + * population whose candidate rows define the template differently. Fate 4 + * until the operator chooses one row per group; nothing is guessed. + */ + templateConflicts?: TemplateConflict[]; +} + +/** One `(name, locale)` the promotion cannot carry without the operator's choice. */ +export interface TemplateConflict { + name: string; + locale: string; + candidates: Array<{ id: string; organizationId: string | null }>; +} + +/** The table whose customized and authored rows the ceremony promotes (ruling C on ADR-0131 §6 Q1). */ +export const TEMPLATE_OBJECT = 'sys_email_template'; + +/** Provenance the seeders stamped: only such a row can be a copy of code. */ +export const SEEDED_BY: readonly string[] = ['package', 'platform', 'system']; + +/** + * The `sys_email_template` rows the promotion carries: a customized row (or one + * whose `customized` is unknown), and every row no seeder wrote (`managed_by` + * NULL or outside package / platform / system), with or without an + * organization. Every other row — seeded and uncustomized, NULL residue or an + * organization's copy — is a copy of code: a mirror, never promoted. + */ +export function templatePromotionSql(sql: Sql, cols: ReadonlySet): { text: string; params: unknown[] } { + const t = TEMPLATE_OBJECT; + const c = sql.q('customized', t); + const customized = cols.has('customized') ? `${c} = ? OR ${c} IS NULL` : '1 = 0'; + const params: unknown[] = cols.has('customized') ? [sql.bind(true)] : []; + if (!cols.has('managed_by')) return { text: '1 = 1', params: [] }; + const managed = sql.q('managed_by', t); + return { + text: `${customized} OR ${managed} IS NULL OR ${managed} NOT IN (${SEEDED_BY.map(() => '?').join(', ')})`, + params: [...params, ...SEEDED_BY], + }; +} + +/** The JSON of a definition's own keys, for comparing two bodies. */ +export const definitionKey = (body: Readonly>, keys: readonly string[]): string => + JSON.stringify(keys.map((k) => [k, body[k] ?? null])); + +/** One `(name, locale)` of the promotion population, its rows and the bodies they define. */ +export interface TemplateGroup { + name: string; + locale: string; + rows: Record[]; + bodies: Record[]; + /** The rows define the template differently. */ + conflict: boolean; +} + +/** Group population rows by `(name, locale)`; a row whose mapped body carries no name is listed apart. */ +export function groupTemplateRows(rows: readonly Record[]): { groups: TemplateGroup[]; nameless: string[] } { + const byKey = new Map(); + const nameless: string[] = []; + for (const row of rows) { + const body = emailTemplateDefinitionFromRow(row); + const name = typeof body.name === 'string' && body.name !== '' ? body.name : null; + if (name === null) { + nameless.push(String(row.id)); + continue; + } + const locale = String(body.locale); + const key = `${name}\u0000${locale}`; + const group = byKey.get(key) ?? { name, locale, rows: [], bodies: [], conflict: false }; + group.rows.push(row); + group.bodies.push(body); + byKey.set(key, group); + } + for (const group of byKey.values()) { + const keys = [...new Set(group.bodies.flatMap(Object.keys))]; + group.conflict = new Set(group.bodies.map((b) => definitionKey(b, keys))).size > 1; + } + return { groups: [...byKey.values()], nameless }; +} + +const nonEmptyId = (value: unknown): string | null => + value === null || value === undefined || String(value) === '' ? null : String(value); + +/** The conflicts among the promotion population of one physical template table. */ +async function readTemplateConflicts(sql: Sql, table: string, columns: ReadonlySet): Promise { + const where = templatePromotionSql(sql, columns); + const rows = await sql.read(table, `SELECT * FROM ${sql.q(table, table)} WHERE ${where.text} ORDER BY ${sql.q('id', table)}`, where.params); + return groupTemplateRows(rows) + .groups.filter((g) => g.conflict) + .map((g) => ({ + name: g.name, + locale: g.locale, + candidates: g.rows.map((r) => ({ id: String(r.id), organizationId: nonEmptyId(r[ORG]) })), + })); } export interface OrganizationOwnershipPlan { @@ -622,6 +717,11 @@ async function planPresentTable(ctx: PlanContext, entry: InventoryEntry, source: } plan.categories.push({ id: category.id, label: category.label, citation: category.citation, rows, ...(groups !== undefined ? { groups } : {}) }); } + if (entry.object === TEMPLATE_OBJECT) { + const conflicts: TemplateConflict[] = []; + for (const table of tables) conflicts.push(...(await readTemplateConflicts(sql, table, ctx.catalog.columns.get(table)!))); + plan.templateConflicts = conflicts; + } const rows = plan.rows!; const departing = plan.departures.reduce((n, d) => n + d.rows, 0); diff --git a/packages/core/src/security/email-template-row-definition.test.ts b/packages/core/src/security/email-template-row-definition.test.ts index 02c68afa4f..ece43a9448 100644 --- a/packages/core/src/security/email-template-row-definition.test.ts +++ b/packages/core/src/security/email-template-row-definition.test.ts @@ -9,7 +9,7 @@ */ import { describe, expect, it } from 'vitest'; -import { EmailTemplateDefinitionSchema } from '@objectstack/spec/system'; +import { EMAIL_TEMPLATE_FLOOR_LOCALE, EmailTemplateDefinitionSchema } from '@objectstack/spec/system'; import { emailTemplateDefinitionFromRow } from './email-template-row-definition.js'; /** A row as `sys_email_template` stores it, its state and provenance riding along. */ @@ -80,7 +80,7 @@ describe('emailTemplateDefinitionFromRow', () => { expect(body).toStrictEqual({ name: 'auth.reset', label: 'auth.reset', - locale: 'en-US', + locale: EMAIL_TEMPLATE_FLOOR_LOCALE, subject: '', bodyHtml: '', variables: [], diff --git a/packages/core/src/security/email-template-row-definition.ts b/packages/core/src/security/email-template-row-definition.ts index b0731d6d71..97a3c3b4aa 100644 --- a/packages/core/src/security/email-template-row-definition.ts +++ b/packages/core/src/security/email-template-row-definition.ts @@ -11,15 +11,15 @@ * upgrade ceremony (`os migrate`, C7b) reads the row, calls this function and * saves the answer through the metadata door. ⛔ Never at boot. * - * The function is pure: one row in, one body out — no engine, no I/O, and no - * import, so the lean entry carries it as it stands. The mapping is the + * The function is pure: one row in, one body out — no engine, no I/O. Its one + * import is the spec's floor-locale constant, a plain string. The mapping is the * inverse of the row shape the retired projector wrote, which is also the * render input the email service reads (`EmailTemplateRow`): * * - `name` → `name`; `label` → `label`, falling back to the name when the * column is empty or absent; - * - `locale` → `locale`, falling back to `en-US` (the schema's default and - * the bundle floor) when empty or absent; + * - `locale` → `locale`, falling back to `EMAIL_TEMPLATE_FLOOR_LOCALE` (the + * schema's default and the bundle floor) when empty or absent; * - `subject` → `subject` and `body_html` → `bodyHtml`, each `''` when empty * or absent (both are required strings); * - `body_text` → `bodyText`, `reply_to` → `replyTo` and `description` → @@ -41,6 +41,8 @@ * the ceremony's. */ +import { EMAIL_TEMPLATE_FLOOR_LOCALE } from '@objectstack/spec/system'; + /** * One `sys_email_template` row, as the mapping reads it. Every other column a * row carries is accepted and ignored. @@ -85,7 +87,7 @@ export function emailTemplateDefinitionFromRow(row: EmailTemplateRowForDefinitio const body: Record = { name: row.name, label: filled(row.label) ?? row.name, - locale: filled(row.locale) ?? 'en-US', + locale: filled(row.locale) ?? EMAIL_TEMPLATE_FLOOR_LOCALE, subject: filled(row.subject) ?? '', bodyHtml: filled(row.body_html) ?? '', };