Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .changeset/15206-sys-view-definition-retired.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
'@objectstack/metadata-core': minor
'@objectstack/metadata-protocol': minor
'@objectstack/metadata': minor
'@objectstack/platform-objects': minor
'@objectstack/spec': minor
---

feat(metadata-core,metadata-protocol,metadata,platform-objects,spec)!: `sys_view_definition` retires as inert (ADR-0131 D13)

Clause-②: no (narrowing)

<!-- adr-0087: registered sys-view-definition-retired -->

**BREAKING**, graded `minor` on the v18 prerelease line: Changesets is in pre mode with the tag `next`, and the fixed group is already majored by the line's opening marker, so this ships in an `18.0.0-next.N` like every other stage.

`sys_view_definition` was declared for runtime-authored shared and personal views (ADR-0017) and never used for them: no framework code writes or reads its rows, and the Studio console's view doors write the ADR-0005 `view` overlay in `sys_metadata` through the metadata API. ADR-0131 D13 retires it. A runtime-authored view is a `view` metadata item, written through `PUT /api/v1/meta/view/<name>` (the client's `meta.saveItem` for type `view`); nothing about that path changes.

**What moves for consumers.**

- **The object.** FROM `import { SysViewDefinitionObject } from '@objectstack/metadata-core'` (or from `@objectstack/platform-objects`, or its `/metadata` subpath) TO nothing: delete the import. Neither `MetadataPlugin` nor the metadata protocol assembly registers the object any more, so the generic data door no longer serves it.
- **The migration exports** of `@objectstack/metadata-protocol`. FROM `ensureViewDefinitionActiveIndex`, `resolveIndexExec`, `buildActiveIndexSql`, `VIEW_DEFINITION_TABLE`, `VIEW_ACTIVE_INDEX_NAME`, `VIEW_ACTIVE_PROBE_INDEX_NAME`, `VIEW_ACTIVE_INDEX_COLUMNS`, `EnsureViewIndexLogger`, `EnsureViewIndexStatus` and `EnsureViewIndexResult` TO nothing: delete the import. `classifyIndexFailure` and the `IndexExec` type are still exported, unchanged, from the shared index-migration module.
- **The name.** `PLATFORM_OBJECTS_BY_PACKAGE['metadata-core']` (`@objectstack/spec/system`) lists four objects, and `isPlatformProvidedObjectName` answers `false` for `sys_view_definition`. A stack that names it (a lookup target, a flow trigger, a permission entry, a platform-global declaration) is now flagged as naming an unknown object: remove the reference.
- **`os migrate duplicates`.** `runtimeIndexPreflight` carries three entries (the two `sys_metadata` overlay indexes and `sys_setting`'s row identity), with no `idx_sys_view_def_active` entry. A serving boot no longer runs the active-row index migration for the table, and no longer logs its `sys_view_definition active-row index` lines.
- **Translations.** The platform translation bundles no longer carry `sys_view_definition` keys.

**Existing databases.** Schema sync is additive and never drops a table, so a database an earlier release provisioned keeps `sys_view_definition`, with any rows a caller wrote through the generic data door; nothing reads them. `os migrate apply --allow-destructive` does not drop it either, because it reconciles declared objects only (measured on one database: it dropped an orphaned column of a declared table and left this table and its row in place). On a project with a host config, `os migrate plan` lists it among the platform-prefixed tables nothing declares. Export any row worth keeping; dropping the table is the operator's call, by hand: `DROP TABLE sys_view_definition`.
17 changes: 8 additions & 9 deletions content/docs/data-modeling/drivers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -376,7 +376,7 @@ log because they are worth knowing *before* you choose MySQL, not after.
| Caveat | What MySQL does not give you | Way out |
| :--- | :--- | :--- |
| [`upsert` cannot honour a conflict target](#upsert-conflict-targets-the-target-mysql-cannot-honour) | `ON DUPLICATE KEY UPDATE` carries no target, so a merge can land on a UNIQUE key you never named. The driver refuses the call rather than let it. | Keep one UNIQUE key per table, or run the object on SQLite/PostgreSQL. |
| [Three runtime uniqueness indexes cannot be built](#uniqueness-indexes-mysql-cannot-build) | Database enforcement of three platform integrity guarantees. The weaker index stays in force and every boot logs an `error`. | None in-dialect. Run the platform on SQLite/PostgreSQL for the guarantee. |
| [Two platform tables' runtime uniqueness indexes cannot be built](#uniqueness-indexes-mysql-cannot-build) | Database enforcement of two platform integrity guarantees. The weaker index stays in force and every boot logs an `error`. | None in-dialect. Run the platform on SQLite/PostgreSQL for the guarantee. |
| [A unique violation does not name the column](#the-conflicting-column-is-not-named) | The conflicting **field** in import errors and form-field conflict messages. The `409 UNIQUE_VIOLATION` itself is unaffected. | None. The message falls back to generic copy. |

### `upsert` conflict targets: the target MySQL cannot honour
Expand Down Expand Up @@ -471,24 +471,23 @@ object on SQLite/PostgreSQL.

### Uniqueness indexes MySQL cannot build

Three platform tables get their uniqueness from an index issued as raw SQL at
Two platform tables get their uniqueness from an index issued as raw SQL at
boot by a `metadata-protocol` runtime migration, because the shape each one needs
cannot be expressed through the declaration surface. Two of the three are
**partial** indexes (`CREATE UNIQUE INDEX … WHERE state = '…'`), and all three
cannot be expressed through the declaration surface. `sys_metadata`'s are
**partial** indexes (`CREATE UNIQUE INDEX … WHERE state = '…'`), and both tables'
use **functional key parts** (`COALESCE(…)`) to fold a nullable column's NULLs
into one bucket that is unique among itself.

MySQL/MariaDB has no partial indexes at any version, and rejects the functional
key parts as these statements spell them. So on MySQL none of the three is built,
and the guarantee each one backs is not enforced by the database:
key parts as these statements spell them. So on MySQL neither is built, and the
guarantee each one backs is not enforced by the database:

| Table | The guarantee that is not enforced on MySQL |
| :--- | :--- |
| `sys_metadata` | ADR-0005 overlay uniqueness. Package-less rows (`package_id` NULL) stay NULL-distinct and can duplicate, and `getMetaItem` then has no defined answer for which row wins. |
| `sys_view_definition` | Active-row view-name uniqueness. An archived view keeps occupying its name slot, and two same-name **active** shared views (`owner` NULL) or environment-level views (`organization_id` NULL) can coexist even though the platform states they cannot. |
| `sys_setting` | NULL-safe row identity. `user_id` is NULL on every row that is not `scope='user'`, so two tenant-scope rows for one `(namespace, key)` in one organization — or two platform defaults on the global layer — can coexist, and `SettingsService` has no defined answer for which one wins. |

What happens instead is the same in all three cases, and it is deliberate. Each
What happens instead is the same in both cases, and it is deliberate. Each
migration proves the new index is possible under a throwaway probe name *before*
dropping anything, so a dialect that cannot take it is left holding **exactly the
index it already had** — never an unconstrained table, and never a failed boot.
Expand All @@ -497,7 +496,7 @@ lists any rows already violating the guarantee.

<Callout type="warn">
**There is no in-dialect fix**, and none is planned — the DDL these migrations
need does not exist in MySQL/MariaDB. If you need these three guarantees enforced
need does not exist in MySQL/MariaDB. If you need these guarantees enforced
by the database, run the platform on SQLite or PostgreSQL. On MySQL, treat the
boot's `[metadata-protocol] this database cannot build …` lines as expected
rather than as a failure, and run the duplicate-listing query each one prints to
Expand Down
5 changes: 2 additions & 3 deletions content/docs/deployment/cli.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1532,14 +1532,13 @@ The report carries a second section, `runtimeIndexPreflight`, answering a
different question: **will the next server start be able to finish tightening
the platform's own unique indexes?**

Three migrations run at `kernel:ready` on a serving boot (`os dev`, `os serve`,
Two migrations run at `kernel:ready` on a serving boot (`os dev`, `os serve`,
`os start`) and replace a declared UNIQUE index with the NULL-safe — and
sometimes active-rows-only — form it was always meant to have:
sometimes state-scoped — form it was always meant to have:

| Table | Index | What the tightening adds |
| :--- | :--- | :--- |
| `sys_metadata` | overlay `active` and `draft` | package-less overlays stop being NULL-distinct |
| `sys_view_definition` | `idx_sys_view_def_active` | shared and environment-level views stop being NULL-distinct, and only active rows are constrained |
| `sys_setting` | the declared row identity | tenant- and global-scope rows stop being NULL-distinct on `user_id` |

Each is a **tightening**, so rows an installation already holds can block it.
Expand Down
2 changes: 1 addition & 1 deletion content/docs/plugins/packages.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ import { InMemoryDriver } from '@objectstack/driver-memory';

- **Purpose**: Production-ready SQL database support with migrations
- **Supports**: PostgreSQL, MySQL, SQLite, and all Knex-compatible databases
- **MySQL caveats**: MySQL is supported, with three dialect caveats — `upsert` cannot honour a conflict target, three runtime uniqueness indexes cannot be built, and a unique violation does not name the conflicting column. See [Drivers → MySQL dialect caveats](/docs/data-modeling/drivers#mysql-dialect-caveats)
- **MySQL caveats**: MySQL is supported, with three dialect caveats — `upsert` cannot honour a conflict target, two platform tables' runtime uniqueness indexes cannot be built, and a unique violation does not name the conflicting column. See [Drivers → MySQL dialect caveats](/docs/data-modeling/drivers#mysql-dialect-caveats)
- **When to use**: Traditional relational database deployments
- **README**: [View README](https://github.com/objectstack-ai/objectstack/blob/main/packages/drivers/driver-sql/README.md)

Expand Down
14 changes: 10 additions & 4 deletions docs/qa/platform-checklist/areas/cli.json
Original file line number Diff line number Diff line change
Expand Up @@ -1102,7 +1102,7 @@
"title": "os migrate duplicates: a read-only JSON inventory of identifiers minted across partitions — within-partition repeats excluded, nothing written, the live two-counter condition reported, and runnable BEFORE the #8686 repair destroys the evidence",
"since": "v17",
"status": "active",
"revision": 5,
"revision": 6,
"priority": "P1",
"surface": "cli",
"personas": ["operator (local shell, pre-repair audit)"],
Expand All @@ -1122,7 +1122,7 @@
"seed via direct SQL per the fixture recipe: the cross-partition duplicate, the within-partition repeat, an organizations row for '<org>', and the paired sequence counters",
"dump the DB's CONTENT before the run — every user table's rows (sqlite3 <db> '.dump' with the CREATE lines aside, or one ordered SELECT * per table), `SELECT * FROM _objectstack_sequences ORDER BY 1`, and `SELECT type, name, tbl_name, sql FROM sqlite_master ORDER BY name` — and checksum each dump; run `os migrate duplicates > report.json; echo $?`; dump and checksum again and byte-compare the three pairs. ⛔ Do not use the whole-file md5 as the oracle: it is not what the read-only contract promises (rows, counters and schema)",
"jq the report: .report/.reportVersion/.generatedAt/.database/.globalPartition/.filter/.counters/.scanned/.skipped/.duplicates/.liveConditions/.runtimeIndexPreflight/.summary",
"seed the kernel:ready blocker too (#8725): two ACTIVE sys_view_definition rows with the SAME name and organization_id/owner both NULL — then jq .runtimeIndexPreflight and .summary.runtimeIndexesBlocked",
"seed the kernel:ready blocker too (#8725): the first boot already TIGHTENED idx_sys_metadata_overlay_active, so first put back the declared NULL-distinct form an untightened install still has (DROP INDEX idx_sys_metadata_overlay_active, then CREATE UNIQUE INDEX idx_sys_metadata_overlay_active ON sys_metadata (type, name, organization_id, package_id)); then insert two ACTIVE sys_metadata rows with the SAME type and name, the SAME non-NULL organization_id and package_id NULL, every other NOT NULL column filled — then jq .runtimeIndexPreflight and .summary.runtimeIndexesBlocked",
"run `os migrate duplicates --object <the-object>` and `--object <an-object-with-no-findings>` — capture .filter in both payloads",
"run `os migrate duplicates --database-url file:/tmp/<run>/dup.db` and confirm it reaches the same DB (the flag also honors OS_DATABASE_URL)",
"negative (no_sql_seam): run `pnpm --filter @objectstack/cli exec vitest run src/commands/migrate/duplicates.null-seam.test.ts` and cite its pass — the seam double that answers no result set, driving the real command; then run `os migrate duplicates --database-url memory://qa; echo $?` and capture the boot_failed payload (the retired engine is refused at boot, naming the replacement)",
Expand All @@ -1148,9 +1148,9 @@
"evidence": "the three before/after dump-checksum pairs (rows, _objectstack_sequences, sqlite_master)"
},
{
"clause": "the kernel:ready pre-flight (#8725) reports the RUNTIME-migration class os migrate plan cannot see: one entry per index the three kernel:ready migrations tighten (four — the overlay migration owns two), each blocked|clear|table-absent|unreadable, a blocked one naming every colliding key group and its row count",
"clause": "the kernel:ready pre-flight (#8725) reports the RUNTIME-migration class os migrate plan cannot see: one entry per index the two kernel:ready migrations tighten (three — the overlay migration owns two; the third migration, sys_view_definition's, retired with its table under ADR-0131 D13), each blocked|clear|table-absent|unreadable, a blocked one naming every colliding key group and its row count",
"oracle": "log",
"verify": ".runtimeIndexPreflight names idx_sys_view_def_active as blocked with the seeded view name, organization_id_key '__global__' and owner_key '' — and the SAME database run through `os migrate plan` mentions neither the index nor the view name (the matched control: the declared-index duplicate above IS reported by plan, this one is not)",
"verify": ".runtimeIndexPreflight names idx_sys_metadata_overlay_active as blocked with the seeded type, name and organization_id and package_id_key '' — and the SAME database run through `os migrate plan` mentions neither the index nor the seeded name (the matched control: the declared-index duplicate above IS reported by plan, this one is not); no entry names sys_view_definition",
"evidence": "the pre-flight section beside the plan output for one database"
},
{
Expand Down Expand Up @@ -1201,6 +1201,12 @@
"date": "2026-10-04",
"change": "A3 and its step re-pointed: the read-only oracle is the DB's content — rows, _objectstack_sequences and sqlite_master byte-identical before and after — not the whole-file md5, which is not what the contract promises (duplicates.ts header: 'a full run leaves both the rows and `_objectstack_sequences` byte-identical'; duplicates.pre-repair.test.ts pins the same). Assertion defect. Source duplicates.ts contract header",
"ref": "#21735"
},
{
"revision": 6,
"date": "2026-10-08",
"change": "the kernel:ready blocker re-seeded on sys_metadata: sys_view_definition retired as inert with its active-row index migration (ADR-0131 D13), so the pre-flight now probes three indexes from two migrations and never names that table. The seeding step and A4 now use two ACTIVE package-less sys_metadata overlays for one type, name and NON-NULL organization — the organization key part stays bare in the tightened index (#6418), so a NULL-organization pair would not block its CREATE — and the step first puts back the declared NULL-distinct index the first boot replaced, without which the insert is refused. Source runtime-index-preflight.ts runtimeIndexProbes",
"ref": "#15206"
}
]
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ const OWN_TABLES: Record<CommandName, readonly string[]> = {
resume: ['sys_migration_journal'],
'recorded-by': ['sys_metadata_history'],
// The objects with a covered field: the scan walks each.
'value-shapes': ['os21529_contact', 'sys_metadata', 'sys_view_definition'],
'value-shapes': ['os21529_contact', 'sys_metadata'],
};

const absentJson = {} as Record<CommandName, Run>;
Expand Down
9 changes: 4 additions & 5 deletions packages/cli/src/commands/migrate/duplicates.contract.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -134,15 +134,15 @@ const collect = async (objectFilter?: string) =>
client: 'better-sqlite3',
now: () => new Date('2026-08-17T12:00:00.000Z'),
// The real pre-flight against the real fixture — never a stand-in. This
// database has none of the four platform tables, so every entry is
// database has none of the platform tables the probes read, so every entry is
// `table-absent`, and the `blocked` shape is pinned in its own test below
// over a database that really carries the damage.
runtimeIndexPreflight: await collectRuntimeIndexPreflight(exec, { client: 'better-sqlite3' }),
...(objectFilter ? { objectFilter } : {}),
});

/**
* The four probes, as `@objectstack/metadata-protocol` declares them.
* The three probes, as `@objectstack/metadata-protocol` declares them.
*
* Read from the producer rather than restated here: the descriptor (table,
* index name, key parts, row scope, the two statements) is the migration's own
Expand Down Expand Up @@ -231,7 +231,7 @@ describe('#8928 os migrate duplicates — the report document', () => {
organizationCounters: [{ organization: 'org_x', lastValue: 4 }],
},
],
// One entry per index the three `kernel:ready` migrations tighten — FOUR,
// One entry per index the two `kernel:ready` migrations tighten — THREE,
// because the overlay migration builds one per state and either can be
// blocked on its own. Present whatever the outcome: an index left out
// would make "nothing blocks it" and "it was never probed" the same
Expand All @@ -253,14 +253,13 @@ describe('#8928 os migrate duplicates — the report document', () => {
});
});

it('names the four kernel:ready indexes, and the migration behind each', async () => {
it('names the three kernel:ready indexes, and the migration behind each — none for the retired sys_view_definition (ADR-0131 D13)', async () => {
const report = await collect();
expect(
report.runtimeIndexPreflight.map((p) => `${p.migration}:${p.table}:${p.index}`),
).toEqual([
'ensureMetadataOverlayIndexes:sys_metadata:idx_sys_metadata_overlay_active',
'ensureMetadataOverlayIndexes:sys_metadata:idx_sys_metadata_overlay_draft',
'ensureViewDefinitionActiveIndex:sys_view_definition:idx_sys_view_def_active',
'ensureSysSettingIdentityIndex:sys_setting:uniq_sys_setting_organization_id_namespace_key_scope_user_id',
]);
});
Expand Down
Loading
Loading