diff --git a/.changeset/25761-seed-column-apply-on.md b/.changeset/25761-seed-column-apply-on.md new file mode 100644 index 00000000000..e7f203f5c5e --- /dev/null +++ b/.changeset/25761-seed-column-apply-on.md @@ -0,0 +1,33 @@ +--- +'@objectstack/spec': minor +'@objectstack/metadata-protocol': minor +'@objectstack/skills': minor +--- + +feat(spec,metadata-protocol): a seed dataset column can be written only when its row is created (`columns.FIELD.applyOn: 'create'`) + +Clause-②: yes +Metadata-protocol: added + +A seed is replayed on every dev-server boot and every package re-publish, and an `upsert` replay re-asserts every column a record declares on the row it matched. That reverts the columns an app's own jobs write after the first seed (a status a payment job advances, a flag a renewal job sets), so a re-run undid the jobs' work with nothing in the log. + +A seed dataset now takes a per-column write policy, keyed by field name: + +```ts +defineSeed(Instalment, { + externalId: 'code', + columns: { + status: { applyOn: 'create' }, + is_expiring: { applyOn: 'create' }, + }, + records: [{ code: 'INS-001', amount: 1200, status: 'pending', is_expiring: false }], +}); +``` + +- **`applyOn: 'create'`** writes the column when the seed creates the row and leaves it out of every write to a row the seed matched by its `externalId`: an `upsert` replay over an existing row, and every row of an `update` dataset. The matched row keeps the value it holds, while the `always` columns are refreshed. A row whose only differences are create-only columns is not written at all. A create-only reference column on a matched row is not resolved and not deferred to pass 2, so nothing later in the load writes it back. A column of the `externalId` is the key the row was matched by and stays in its update. `insert`, `ignore` and `replace` never update a matched row, so nothing changes on those modes. +- **`applyOn: 'always'`** (the default, and every column without a `columns` entry) is the behaviour before this change. +- The values, the default and the meaning are those of the import door's `FieldMappingEntry.applyOn`. Both doors now read one enum, exported as `ColumnApplyOnSchema` from `@objectstack/spec/data`. `SeedColumnSchema` (with `SeedColumn` / `SeedColumnParsed`) is the per-column shape. +- A `columns` key must name a column the object has: a declared field, or a system column the platform injects on it such as `owner_id`. `defineSeed` refuses one that does not when it runs (and at compile time for an object literal), naming the key with a near miss. The seed loader logs an `error` naming the column, once per dataset, whenever it loads a seed that names one, since a stored seed never passes through `defineSeed`. The rows still load. +- `SeedSchema` refuses an unknown key on a column (`unrecognized_keys`), and a dataset-level `applyOn` is pointed at `columns`. + +`@objectstack/skills` only regenerates its schema reference indexes, which now list `SeedColumnSchema` and `ColumnApplyOnSchema`. diff --git a/content/docs/data-modeling/seed-data.mdx b/content/docs/data-modeling/seed-data.mdx index 3561aa8dd00..902950dcd8f 100644 --- a/content/docs/data-modeling/seed-data.mdx +++ b/content/docs/data-modeling/seed-data.mdx @@ -147,6 +147,52 @@ so a typo like `'complete'` for `'completed'` still surfaces before boot. --- +## Columns Your App Owns After the First Seed — `applyOn: 'create'` + +An `upsert` replay re-asserts every column a record declares on the row it +matched. That is right for master data and wrong for a column the app itself +owns once the row exists: a status a scheduled job advances, a flag a job sets +after it sends a notice, a reference a workflow re-points. Replayed, the seed +would put those back, undoing the job's work on every boot. + +Declare such a column `applyOn: 'create'` under `columns`: + +```typescript +defineSeed(Instalment, { + externalId: 'code', + mode: 'upsert', + columns: { + status: { applyOn: 'create' }, // the payment job owns it after the first seed + is_expiring: { applyOn: 'create' }, // the renewal job sets it + }, + records: [ + { code: 'INS-001', amount: 1200, status: 'pending', is_expiring: false }, + ], +}); +``` + +- When the seed **creates** the row, every column is written, `status` and + `is_expiring` included. +- When a replay **matches** an existing row, `status` and `is_expiring` are left + out of the write: the row keeps the values the jobs gave it, while `amount` + (`applyOn: 'always'`, the default) is refreshed. A row whose only differences + are create-only columns is not written at all. +- An `update` dataset never writes a create-only column. `insert`, `ignore` and + `replace` never update a matched row, so the policy changes nothing there. +- A create-only **reference** on a matched row is not resolved at all, so a + replay neither re-points it nor reports it. +- A column of the `externalId` is what the row was matched by, so it stays in + the update whatever it declares. + +`applyOn` takes the same values, with the same default and meaning, as an import +mapping entry's [`applyOn`](/docs/data-modeling/import-mappings). A `columns` key +must name a field of the object (or a system column the platform injects on it, +such as `owner_id`): `defineSeed` refuses one that does not, and the loader logs +an error naming the column whenever it loads a seed that names one, since a +stored seed never passes through `defineSeed`. + +--- + ## Environment Scoping The `env` array controls which deployment environments receive the records. The @@ -587,6 +633,7 @@ function defineSeed< config: { externalId?: string | string[]; // single field, or a composite list (join tables); default: 'name' mode?: 'insert' | 'update' | 'upsert' | 'replace' | 'ignore'; // default: 'upsert' + columns?: { [field: string]: { applyOn?: 'create' | 'always' } }; // per-column write policy; default: 'always' env?: Array<'prod' | 'dev' | 'test'>; // default: ['prod','dev','test'] locale?: string[]; // BCP-47 tags; omitted = every locale records: Array>; // declared fields + injectable system columns @@ -596,7 +643,8 @@ function defineSeed< `SeedRecord` keys are the object's declared fields (a `lookup` / `master_detail` field takes its target's natural-key string) plus the injectable system column -names. Every record key is checked again when the call runs. +names. Every record key, and every `columns` key, is checked again when the call +runs. The returned `Seed` object is a plain serialisable value — pass it to your stack's seed runner or store it in an export array. diff --git a/content/docs/getting-started/quick-reference.mdx b/content/docs/getting-started/quick-reference.mdx index 827f0921da5..85db14ba0af 100644 --- a/content/docs/getting-started/quick-reference.mdx +++ b/content/docs/getting-started/quick-reference.mdx @@ -22,7 +22,7 @@ Categories that have no section here at all are named under [Categories Without a Section](#categories-without-a-section) — that curation is stated, not left implicit. -## Data Protocol (15 of 29 schemas) +## Data Protocol (15 of 30 schemas) Core business logic and data modeling schemas. diff --git a/content/docs/references/api/package-api-assembled.mdx b/content/docs/references/api/package-api-assembled.mdx index a60394e2249..cfa11b1628c 100644 --- a/content/docs/references/api/package-api-assembled.mdx +++ b/content/docs/references/api/package-api-assembled.mdx @@ -99,7 +99,7 @@ Installed package row whose manifest is the assembled package body | **dependencies** | `Record` | optional | Package dependencies | | **configuration** | `never` | optional | [REMOVED] `manifest.configuration` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing ever read the block: no settings UI rendered it and no loader resolved a setting from it, so authoring it configured nothing. Worse, `properties.*.secret` promised "value is encrypted/masked (e.g. API Keys)" while nothing encrypted, masked or even parsed the flag — a false assurance about credential handling. Delete the key. A plugin is configured by the host that composes it: pass options to its constructor in `defineStack({ plugins: [new MyPlugin({ … })] })`, which is the enforced channel. A declarative settings surface must be designed with an enforcing reader first, not revived here. | | **contributes** | `{ kinds?: object[] }` | optional | Platform contributions | -| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; env?: Enum<'prod' \| 'dev' \| 'test'>[]; … }[]` | optional | Seed Data / Fixtures for bootstrapping | +| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; columns?: Record; … }[]` | optional | Seed Data / Fixtures for bootstrapping | | **capabilities** | `{ name: string; label?: string; description?: string; scope?: Enum<'platform' \| 'org'>; … }[]` | optional | [ADR-0066 D1] Authorization capabilities this package defines (seeded with package provenance) | | **extensions** | `never` | optional | [REMOVED] `manifest.extensions` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — an untyped map with zero readers: whatever was parked here was stored and never consulted. Delete the key. Extend the platform through the enforced channels instead: `contributes.kinds` registers metadata kinds, `navigationContributions` injects navigation into other packages' apps, and code-level extension happens in the plugin itself (`init`/`start`). | | **navigationContributions** | `{ app: string; group?: string; priority?: integer; items: (object \| … +9 more)[] }[]` | optional | Navigation items this package contributes into apps owned by other packages | @@ -281,7 +281,7 @@ This schema accepts one of the following structures: | **dependencies** | `Record` | optional | Package dependencies | | **configuration** | `never` | optional | [REMOVED] `manifest.configuration` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing ever read the block: no settings UI rendered it and no loader resolved a setting from it, so authoring it configured nothing. Worse, `properties.*.secret` promised "value is encrypted/masked (e.g. API Keys)" while nothing encrypted, masked or even parsed the flag — a false assurance about credential handling. Delete the key. A plugin is configured by the host that composes it: pass options to its constructor in `defineStack({ plugins: [new MyPlugin({ … })] })`, which is the enforced channel. A declarative settings surface must be designed with an enforcing reader first, not revived here. | | **contributes** | `{ kinds?: object[] }` | optional | Platform contributions | -| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; env?: Enum<'prod' \| 'dev' \| 'test'>[]; … }[]` | optional | Initial seed data (prefer top-level data field) | +| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; columns?: Record; … }[]` | optional | Initial seed data (prefer top-level data field) | | **capabilities** | `never` | optional | [REMOVED] `manifest.capabilities` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no discovery path ever consulted the block: nothing read `implements`, `provides`, `requires`, `extensionPoints` or `extensions`, so the declared "interoperability and automatic discovery" never happened. Delete the key. Real dependency resolution runs off top-level `manifest.dependencies`, which stays. Capability-based discovery must be designed with an enforcing reader first, not revived here. | | **extensions** | `never` | optional | [REMOVED] `manifest.extensions` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — an untyped map with zero readers: whatever was parked here was stored and never consulted. Delete the key. Extend the platform through the enforced channels instead: `contributes.kinds` registers metadata kinds, `navigationContributions` injects navigation into other packages' apps, and code-level extension happens in the plugin itself (`init`/`start`). | | **navigationContributions** | `{ app: string; group?: string; priority?: integer; items: (object \| … +9 more)[] }[]` | optional | Navigation items this package contributes into apps owned by other packages | @@ -345,7 +345,7 @@ This schema accepts one of the following structures: | **dependencies** | `Record` | optional | Package dependencies | | **configuration** | `never` | optional | [REMOVED] `manifest.configuration` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing ever read the block: no settings UI rendered it and no loader resolved a setting from it, so authoring it configured nothing. Worse, `properties.*.secret` promised "value is encrypted/masked (e.g. API Keys)" while nothing encrypted, masked or even parsed the flag — a false assurance about credential handling. Delete the key. A plugin is configured by the host that composes it: pass options to its constructor in `defineStack({ plugins: [new MyPlugin({ … })] })`, which is the enforced channel. A declarative settings surface must be designed with an enforcing reader first, not revived here. | | **contributes** | `{ kinds?: object[] }` | optional | Platform contributions | -| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; env?: Enum<'prod' \| 'dev' \| 'test'>[]; … }[]` | optional | Seed Data / Fixtures for bootstrapping | +| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; columns?: Record; … }[]` | optional | Seed Data / Fixtures for bootstrapping | | **capabilities** | `{ name: string; label?: string; description?: string; scope?: Enum<'platform' \| 'org'>; … }[]` | optional | [ADR-0066 D1] Authorization capabilities this package defines (seeded with package provenance) | | **extensions** | `never` | optional | [REMOVED] `manifest.extensions` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — an untyped map with zero readers: whatever was parked here was stored and never consulted. Delete the key. Extend the platform through the enforced channels instead: `contributes.kinds` registers metadata kinds, `navigationContributions` injects navigation into other packages' apps, and code-level extension happens in the plugin itself (`init`/`start`). | | **navigationContributions** | `{ app: string; group?: string; priority?: integer; items: (object \| … +9 more)[] }[]` | optional | Navigation items this package contributes into apps owned by other packages | diff --git a/content/docs/references/api/package-api.mdx b/content/docs/references/api/package-api.mdx index 3c4db9ead16..903d29313c1 100644 --- a/content/docs/references/api/package-api.mdx +++ b/content/docs/references/api/package-api.mdx @@ -155,7 +155,7 @@ Install package request | **dependencies** | `Record` | optional | Package dependencies | | **configuration** | `never` | optional | [REMOVED] `manifest.configuration` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing ever read the block: no settings UI rendered it and no loader resolved a setting from it, so authoring it configured nothing. Worse, `properties.*.secret` promised "value is encrypted/masked (e.g. API Keys)" while nothing encrypted, masked or even parsed the flag — a false assurance about credential handling. Delete the key. A plugin is configured by the host that composes it: pass options to its constructor in `defineStack({ plugins: [new MyPlugin({ … })] })`, which is the enforced channel. A declarative settings surface must be designed with an enforcing reader first, not revived here. | | **contributes** | `{ kinds?: object[] }` | optional | Platform contributions | -| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; env?: Enum<'prod' \| 'dev' \| 'test'>[]; … }[]` | optional | Initial seed data (prefer top-level data field) | +| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; columns?: Record; … }[]` | optional | Initial seed data (prefer top-level data field) | | **capabilities** | `never` | optional | [REMOVED] `manifest.capabilities` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no discovery path ever consulted the block: nothing read `implements`, `provides`, `requires`, `extensionPoints` or `extensions`, so the declared "interoperability and automatic discovery" never happened. Delete the key. Real dependency resolution runs off top-level `manifest.dependencies`, which stays. Capability-based discovery must be designed with an enforcing reader first, not revived here. | | **extensions** | `never` | optional | [REMOVED] `manifest.extensions` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — an untyped map with zero readers: whatever was parked here was stored and never consulted. Delete the key. Extend the platform through the enforced channels instead: `contributes.kinds` registers metadata kinds, `navigationContributions` injects navigation into other packages' apps, and code-level extension happens in the plugin itself (`init`/`start`). | | **navigationContributions** | `{ app: string; group?: string; priority?: integer; items: (object \| … +9 more)[] }[]` | optional | Navigation items this package contributes into apps owned by other packages | @@ -202,7 +202,7 @@ Package manifest installed through POST /api/v1/packages (AUTHORING stage; scope | **dependencies** | `Record` | optional | Package dependencies | | **configuration** | `never` | optional | [REMOVED] `manifest.configuration` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing ever read the block: no settings UI rendered it and no loader resolved a setting from it, so authoring it configured nothing. Worse, `properties.*.secret` promised "value is encrypted/masked (e.g. API Keys)" while nothing encrypted, masked or even parsed the flag — a false assurance about credential handling. Delete the key. A plugin is configured by the host that composes it: pass options to its constructor in `defineStack({ plugins: [new MyPlugin({ … })] })`, which is the enforced channel. A declarative settings surface must be designed with an enforcing reader first, not revived here. | | **contributes** | `{ kinds?: object[] }` | optional | Platform contributions | -| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; env?: Enum<'prod' \| 'dev' \| 'test'>[]; … }[]` | optional | Initial seed data (prefer top-level data field) | +| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; columns?: Record; … }[]` | optional | Initial seed data (prefer top-level data field) | | **capabilities** | `never` | optional | [REMOVED] `manifest.capabilities` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no discovery path ever consulted the block: nothing read `implements`, `provides`, `requires`, `extensionPoints` or `extensions`, so the declared "interoperability and automatic discovery" never happened. Delete the key. Real dependency resolution runs off top-level `manifest.dependencies`, which stays. Capability-based discovery must be designed with an enforcing reader first, not revived here. | | **extensions** | `never` | optional | [REMOVED] `manifest.extensions` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — an untyped map with zero readers: whatever was parked here was stored and never consulted. Delete the key. Extend the platform through the enforced channels instead: `contributes.kinds` registers metadata kinds, `navigationContributions` injects navigation into other packages' apps, and code-level extension happens in the plugin itself (`init`/`start`). | | **navigationContributions** | `{ app: string; group?: string; priority?: integer; items: (object \| … +9 more)[] }[]` | optional | Navigation items this package contributes into apps owned by other packages | @@ -246,6 +246,7 @@ Package manifest installed through POST /api/v1/packages (AUTHORING stage; scope | **object** | `string` | ✅ | Target Object Name | | **externalId** | `string \| string[]` | optional (default: `"name"`) | Field (or composite list of fields) matched for the uniqueness check | | **mode** | `Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>` | optional (default: `"upsert"`) | Conflict resolution strategy | +| **columns** | `Record }>` | optional | Per-column write policy, keyed by field name: `{ status: { applyOn: 'create' } }` writes `status` only when the seed creates the row. Omitted, every column is written on create and update. | | **env** | `Enum<'prod' \| 'dev' \| 'test'>[]` | optional (default: `["prod","dev","test"]`) | Applicable environments | | **locale** | `string[]` | optional | Applicable locales (BCP-47 tags); omitted applies to every locale. The publish and install paths do not filter by locale — they load every dataset and warn | | **records** | `Record[]` | ✅ | Data records | @@ -309,7 +310,7 @@ Package manifest installed through POST /api/v1/packages (AUTHORING stage; scope | **dependencies** | `Record` | optional | Package dependencies | | **configuration** | `never` | optional | [REMOVED] `manifest.configuration` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing ever read the block: no settings UI rendered it and no loader resolved a setting from it, so authoring it configured nothing. Worse, `properties.*.secret` promised "value is encrypted/masked (e.g. API Keys)" while nothing encrypted, masked or even parsed the flag — a false assurance about credential handling. Delete the key. A plugin is configured by the host that composes it: pass options to its constructor in `defineStack({ plugins: [new MyPlugin({ … })] })`, which is the enforced channel. A declarative settings surface must be designed with an enforcing reader first, not revived here. | | **contributes** | `{ kinds?: object[] }` | optional | Platform contributions | -| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; env?: Enum<'prod' \| 'dev' \| 'test'>[]; … }[]` | optional | Initial seed data (prefer top-level data field) | +| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; columns?: Record; … }[]` | optional | Initial seed data (prefer top-level data field) | | **capabilities** | `never` | optional | [REMOVED] `manifest.capabilities` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no discovery path ever consulted the block: nothing read `implements`, `provides`, `requires`, `extensionPoints` or `extensions`, so the declared "interoperability and automatic discovery" never happened. Delete the key. Real dependency resolution runs off top-level `manifest.dependencies`, which stays. Capability-based discovery must be designed with an enforcing reader first, not revived here. | | **extensions** | `never` | optional | [REMOVED] `manifest.extensions` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — an untyped map with zero readers: whatever was parked here was stored and never consulted. Delete the key. Extend the platform through the enforced channels instead: `contributes.kinds` registers metadata kinds, `navigationContributions` injects navigation into other packages' apps, and code-level extension happens in the plugin itself (`init`/`start`). | | **navigationContributions** | `{ app: string; group?: string; priority?: integer; items: (object \| … +9 more)[] }[]` | optional | Navigation items this package contributes into apps owned by other packages | @@ -353,6 +354,7 @@ Package manifest installed through POST /api/v1/packages (AUTHORING stage; scope | **object** | `string` | ✅ | Target Object Name | | **externalId** | `string \| string[]` | optional (default: `"name"`) | Field (or composite list of fields) matched for the uniqueness check | | **mode** | `Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>` | optional (default: `"upsert"`) | Conflict resolution strategy | +| **columns** | `Record }>` | optional | Per-column write policy, keyed by field name: `{ status: { applyOn: 'create' } }` writes `status` only when the seed creates the row. Omitted, every column is written on create and update. | | **env** | `Enum<'prod' \| 'dev' \| 'test'>[]` | optional (default: `["prod","dev","test"]`) | Applicable environments | | **locale** | `string[]` | optional | Applicable locales (BCP-47 tags); omitted applies to every locale. The publish and install paths do not filter by locale — they load every dataset and warn | | **records** | `Record[]` | ✅ | Data records | @@ -425,7 +427,7 @@ Install package request | **dependencies** | `Record` | optional | Package dependencies | | **configuration** | `never` | optional | [REMOVED] `manifest.configuration` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing ever read the block: no settings UI rendered it and no loader resolved a setting from it, so authoring it configured nothing. Worse, `properties.*.secret` promised "value is encrypted/masked (e.g. API Keys)" while nothing encrypted, masked or even parsed the flag — a false assurance about credential handling. Delete the key. A plugin is configured by the host that composes it: pass options to its constructor in `defineStack({ plugins: [new MyPlugin({ … })] })`, which is the enforced channel. A declarative settings surface must be designed with an enforcing reader first, not revived here. | | **contributes** | `{ kinds?: object[] }` | optional | Platform contributions | -| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; env?: Enum<'prod' \| 'dev' \| 'test'>[]; … }[]` | optional | Initial seed data (prefer top-level data field) | +| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; columns?: Record; … }[]` | optional | Initial seed data (prefer top-level data field) | | **capabilities** | `never` | optional | [REMOVED] `manifest.capabilities` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no discovery path ever consulted the block: nothing read `implements`, `provides`, `requires`, `extensionPoints` or `extensions`, so the declared "interoperability and automatic discovery" never happened. Delete the key. Real dependency resolution runs off top-level `manifest.dependencies`, which stays. Capability-based discovery must be designed with an enforcing reader first, not revived here. | | **extensions** | `never` | optional | [REMOVED] `manifest.extensions` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — an untyped map with zero readers: whatever was parked here was stored and never consulted. Delete the key. Extend the platform through the enforced channels instead: `contributes.kinds` registers metadata kinds, `navigationContributions` injects navigation into other packages' apps, and code-level extension happens in the plugin itself (`init`/`start`). | | **navigationContributions** | `{ app: string; group?: string; priority?: integer; items: (object \| … +9 more)[] }[]` | optional | Navigation items this package contributes into apps owned by other packages | @@ -559,7 +561,7 @@ Upgrade package request | **dependencies** | `Record` | optional | Package dependencies | | **configuration** | `never` | optional | [REMOVED] `manifest.configuration` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing ever read the block: no settings UI rendered it and no loader resolved a setting from it, so authoring it configured nothing. Worse, `properties.*.secret` promised "value is encrypted/masked (e.g. API Keys)" while nothing encrypted, masked or even parsed the flag — a false assurance about credential handling. Delete the key. A plugin is configured by the host that composes it: pass options to its constructor in `defineStack({ plugins: [new MyPlugin({ … })] })`, which is the enforced channel. A declarative settings surface must be designed with an enforcing reader first, not revived here. | | **contributes** | `{ kinds?: object[] }` | optional | Platform contributions | -| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; env?: Enum<'prod' \| 'dev' \| 'test'>[]; … }[]` | optional | Initial seed data (prefer top-level data field) | +| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; columns?: Record; … }[]` | optional | Initial seed data (prefer top-level data field) | | **capabilities** | `never` | optional | [REMOVED] `manifest.capabilities` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no discovery path ever consulted the block: nothing read `implements`, `provides`, `requires`, `extensionPoints` or `extensions`, so the declared "interoperability and automatic discovery" never happened. Delete the key. Real dependency resolution runs off top-level `manifest.dependencies`, which stays. Capability-based discovery must be designed with an enforcing reader first, not revived here. | | **extensions** | `never` | optional | [REMOVED] `manifest.extensions` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — an untyped map with zero readers: whatever was parked here was stored and never consulted. Delete the key. Extend the platform through the enforced channels instead: `contributes.kinds` registers metadata kinds, `navigationContributions` injects navigation into other packages' apps, and code-level extension happens in the plugin itself (`init`/`start`). | | **navigationContributions** | `{ app: string; group?: string; priority?: integer; items: (object \| … +9 more)[] }[]` | optional | Navigation items this package contributes into apps owned by other packages | @@ -654,7 +656,7 @@ Resolve dependencies request | **dependencies** | `Record` | optional | Package dependencies | | **configuration** | `never` | optional | [REMOVED] `manifest.configuration` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing ever read the block: no settings UI rendered it and no loader resolved a setting from it, so authoring it configured nothing. Worse, `properties.*.secret` promised "value is encrypted/masked (e.g. API Keys)" while nothing encrypted, masked or even parsed the flag — a false assurance about credential handling. Delete the key. A plugin is configured by the host that composes it: pass options to its constructor in `defineStack({ plugins: [new MyPlugin({ … })] })`, which is the enforced channel. A declarative settings surface must be designed with an enforcing reader first, not revived here. | | **contributes** | `{ kinds?: object[] }` | optional | Platform contributions | -| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; env?: Enum<'prod' \| 'dev' \| 'test'>[]; … }[]` | optional | Initial seed data (prefer top-level data field) | +| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; columns?: Record; … }[]` | optional | Initial seed data (prefer top-level data field) | | **capabilities** | `never` | optional | [REMOVED] `manifest.capabilities` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no discovery path ever consulted the block: nothing read `implements`, `provides`, `requires`, `extensionPoints` or `extensions`, so the declared "interoperability and automatic discovery" never happened. Delete the key. Real dependency resolution runs off top-level `manifest.dependencies`, which stays. Capability-based discovery must be designed with an enforcing reader first, not revived here. | | **extensions** | `never` | optional | [REMOVED] `manifest.extensions` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — an untyped map with zero readers: whatever was parked here was stored and never consulted. Delete the key. Extend the platform through the enforced channels instead: `contributes.kinds` registers metadata kinds, `navigationContributions` injects navigation into other packages' apps, and code-level extension happens in the plugin itself (`init`/`start`). | | **navigationContributions** | `{ app: string; group?: string; priority?: integer; items: (object \| … +9 more)[] }[]` | optional | Navigation items this package contributes into apps owned by other packages | diff --git a/content/docs/references/api/protocol.mdx b/content/docs/references/api/protocol.mdx index 9d2f17507ec..fd8be4bc56b 100644 --- a/content/docs/references/api/protocol.mdx +++ b/content/docs/references/api/protocol.mdx @@ -2099,7 +2099,7 @@ Install package request | **dependencies** | `Record` | optional | Package dependencies | | **configuration** | `never` | optional | [REMOVED] `manifest.configuration` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing ever read the block: no settings UI rendered it and no loader resolved a setting from it, so authoring it configured nothing. Worse, `properties.*.secret` promised "value is encrypted/masked (e.g. API Keys)" while nothing encrypted, masked or even parsed the flag — a false assurance about credential handling. Delete the key. A plugin is configured by the host that composes it: pass options to its constructor in `defineStack({ plugins: [new MyPlugin({ … })] })`, which is the enforced channel. A declarative settings surface must be designed with an enforcing reader first, not revived here. | | **contributes** | `{ kinds?: object[] }` | optional | Platform contributions | -| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; env?: Enum<'prod' \| 'dev' \| 'test'>[]; … }[]` | optional | Initial seed data (prefer top-level data field) | +| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; columns?: Record; … }[]` | optional | Initial seed data (prefer top-level data field) | | **capabilities** | `never` | optional | [REMOVED] `manifest.capabilities` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no discovery path ever consulted the block: nothing read `implements`, `provides`, `requires`, `extensionPoints` or `extensions`, so the declared "interoperability and automatic discovery" never happened. Delete the key. Real dependency resolution runs off top-level `manifest.dependencies`, which stays. Capability-based discovery must be designed with an enforcing reader first, not revived here. | | **extensions** | `never` | optional | [REMOVED] `manifest.extensions` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — an untyped map with zero readers: whatever was parked here was stored and never consulted. Delete the key. Extend the platform through the enforced channels instead: `contributes.kinds` registers metadata kinds, `navigationContributions` injects navigation into other packages' apps, and code-level extension happens in the plugin itself (`init`/`start`). | | **navigationContributions** | `{ app: string; group?: string; priority?: integer; items: (object \| … +9 more)[] }[]` | optional | Navigation items this package contributes into apps owned by other packages | diff --git a/content/docs/references/data/column-apply-on.mdx b/content/docs/references/data/column-apply-on.mdx new file mode 100644 index 00000000000..48affe5f650 --- /dev/null +++ b/content/docs/references/data/column-apply-on.mdx @@ -0,0 +1,36 @@ +--- +title: Column Apply On — Data Protocol reference +navTitle: Column Apply On +description: "Column Apply On schemas of the ObjectStack Data Protocol: ColumnApplyOn — each property with its type, default and a TypeScript example." +--- + +{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} + + +**Source:** `packages/spec/src/data/column-apply-on.zod.ts` + + +## TypeScript Usage + +```typescript +import { ColumnApplyOnSchema } from '@objectstack/spec/data'; +import type { ColumnApplyOn } from '@objectstack/spec/data'; + +// Validate data +const result = ColumnApplyOnSchema.parse(data); +``` + +--- + +## ColumnApplyOn + +When a matched-row write door writes a column: `always` on create and update, `create` on create only + +### Allowed Values + +* `create` +* `always` + + +--- + diff --git a/content/docs/references/data/index.mdx b/content/docs/references/data/index.mdx index 5f2370560ec..9946692d77e 100644 --- a/content/docs/references/data/index.mdx +++ b/content/docs/references/data/index.mdx @@ -1,7 +1,7 @@ --- title: Data Protocol — complete schema reference navTitle: Data Protocol -description: "The ObjectStack Data Protocol in 29 reference pages: every schema in @objectstack/spec with its properties, types, defaults and a TypeScript example." +description: "The ObjectStack Data Protocol in 30 reference pages: every schema in @objectstack/spec with its properties, types, defaults and a TypeScript example." --- {/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} @@ -10,6 +10,7 @@ This section contains all protocol schemas for the data layer of ObjectStack. + diff --git a/content/docs/references/data/meta.json b/content/docs/references/data/meta.json index 065cbf6dbe4..88c37f782cc 100644 --- a/content/docs/references/data/meta.json +++ b/content/docs/references/data/meta.json @@ -25,6 +25,7 @@ "seed", "seed-loader", "---More---", + "column-apply-on", "context-tokens", "driver-common", "driver-memory", diff --git a/content/docs/references/data/seed-loader.mdx b/content/docs/references/data/seed-loader.mdx index 36ede7b4da4..bc8e7d7fa6c 100644 --- a/content/docs/references/data/seed-loader.mdx +++ b/content/docs/references/data/seed-loader.mdx @@ -220,7 +220,7 @@ Seed loader request with datasets and configuration | Property | Type | Required | Description | | :--- | :--- | :--- | :--- | -| **seeds** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; env?: Enum<'prod' \| 'dev' \| 'test'>[]; … }[]` | ✅ | Seeds to load | +| **seeds** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; columns?: Record; … }[]` | ✅ | Seeds to load | | **config** | `{ dryRun?: boolean; haltOnError?: boolean; multiPass?: boolean; defaultMode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; … }` | optional (has default) | Loader configuration | ### Nested Shape: `SeedLoaderRequest.seeds[number]` @@ -230,6 +230,7 @@ Seed loader request with datasets and configuration | **object** | `string` | ✅ | Target Object Name | | **externalId** | `string \| string[]` | optional (default: `"name"`) | Field (or composite list of fields) matched for the uniqueness check | | **mode** | `Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>` | optional (default: `"upsert"`) | Conflict resolution strategy | +| **columns** | `Record }>` | optional | Per-column write policy, keyed by field name: `{ status: { applyOn: 'create' } }` writes `status` only when the seed creates the row. Omitted, every column is written on create and update. | | **env** | `Enum<'prod' \| 'dev' \| 'test'>[]` | optional (default: `["prod","dev","test"]`) | Applicable environments | | **locale** | `string[]` | optional | Applicable locales (BCP-47 tags); omitted applies to every locale. The publish and install paths do not filter by locale — they load every dataset and warn | | **records** | `Record[]` | ✅ | Data records | diff --git a/content/docs/references/data/seed.mdx b/content/docs/references/data/seed.mdx index 9492a46ad3b..444f62275de 100644 --- a/content/docs/references/data/seed.mdx +++ b/content/docs/references/data/seed.mdx @@ -1,7 +1,7 @@ --- title: Seed schema — Data Protocol property reference navTitle: Seed -description: "Seed schemas of the ObjectStack Data Protocol: Seed, SeedMode — each property with its type, default and a TypeScript example." +description: "Seed schemas of the ObjectStack Data Protocol: Seed, SeedColumn, SeedMode — each property with its type, default and a TypeScript example." --- {/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} @@ -13,8 +13,8 @@ description: "Seed schemas of the ObjectStack Data Protocol: Seed, SeedMode — ## TypeScript Usage ```typescript -import { SeedSchema, SeedMode } from '@objectstack/spec/data'; -import type { Seed } from '@objectstack/spec/data'; +import { SeedSchema, SeedColumnSchema, SeedMode } from '@objectstack/spec/data'; +import type { Seed, SeedColumn } from '@objectstack/spec/data'; // Validate data const result = SeedSchema.parse(data); @@ -31,6 +31,7 @@ const result = SeedSchema.parse(data); | **object** | `string` | ✅ | Target Object Name | | **externalId** | `string \| string[]` | optional (default: `"name"`) | Field (or composite list of fields) matched for the uniqueness check | | **mode** | `Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>` | optional (default: `"upsert"`) | Conflict resolution strategy | +| **columns** | `Record }>` | optional | Per-column write policy, keyed by field name: `{ status: { applyOn: 'create' } }` writes `status` only when the seed creates the row. Omitted, every column is written on create and update. | | **env** | `Enum<'prod' \| 'dev' \| 'test'>[]` | optional (default: `["prod","dev","test"]`) | Applicable environments | | **locale** | `string[]` | optional | Applicable locales (BCP-47 tags); omitted applies to every locale. The publish and install paths do not filter by locale — they load every dataset and warn | | **records** | `Record[]` | ✅ | Data records | @@ -42,6 +43,23 @@ const result = SeedSchema.parse(data); | **_packageVersion** | `string` | optional | Owning package version. | | **_lockDocsUrl** | `string` | optional | Optional documentation link surfaced next to _lockReason. | +### Nested Shape: `Seed.columns[string]` + +| Property | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| **applyOn** | `Enum<'create' \| 'always'>` | optional (default: `"always"`) | When a seed load writes this column. `always` (default) writes it on every row the load creates or updates. `create` writes it only when the load creates the row: an `upsert` replay that matches an existing row leaves the row's value as it is, and an `update` dataset never writes it, so replaying the seed refreshes the `always` columns without reverting state the app wrote since the first load. A column of the dataset's `externalId` is still matched on and stays in the matched row's update. The same values and meaning as an import mapping entry's `applyOn`. | + + +--- + +## SeedColumn + +### Properties + +| Property | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| **applyOn** | `Enum<'create' \| 'always'>` | optional (default: `"always"`) | When a seed load writes this column. `always` (default) writes it on every row the load creates or updates. `create` writes it only when the load creates the row: an `upsert` replay that matches an existing row leaves the row's value as it is, and an `update` dataset never writes it, so replaying the seed refreshes the `always` columns without reverting state the app wrote since the first load. A column of the dataset's `externalId` is still matched on and stays in the matched row's update. The same values and meaning as an import mapping entry's `applyOn`. | + --- diff --git a/content/docs/references/index.mdx b/content/docs/references/index.mdx index 776be9f86a8..743b10b42d3 100644 --- a/content/docs/references/index.mdx +++ b/content/docs/references/index.mdx @@ -1,7 +1,7 @@ --- title: Protocol reference — every schema by module navTitle: Protocol Reference -description: Every schema published by @objectstack/spec — 1540 schemas across 14 protocol modules +description: Every schema published by @objectstack/spec — 1542 schemas across 14 protocol modules --- {/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */} @@ -23,7 +23,7 @@ counts are sums of the rows they head. Regenerate with | [AI Protocol](/docs/references/ai) | 12 | 68 | Agents, tools, skills, RAG and knowledge sources, model registry, conversations. | | [API Protocol](/docs/references/api) | 32 | 438 | REST contracts, endpoints, routing, realtime, batch, discovery. | | [Automation Protocol](/docs/references/automation) | 14 | 73 | Flows and their nodes, approvals, ETL pipelines, webhooks, state machines, execution records. | -| [Data Protocol](/docs/references/data) | 29 | 178 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. | +| [Data Protocol](/docs/references/data) | 30 | 180 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. | | [Identity Protocol](/docs/references/identity) | 5 | 27 | Users and accounts, organizations, positions, SCIM provisioning. | | [Integration Protocol](/docs/references/integration) | 1 | 12 | The single connector protocol (ADR-0097) — catalog descriptors and provider-bound instances. | | [Kernel Protocol](/docs/references/kernel) | 29 | 151 | Plugin lifecycle and manifests, capabilities and security, metadata loading, service registry. | @@ -34,7 +34,7 @@ counts are sums of the rows they head. Regenerate with | [Studio Protocol](/docs/references/studio) | 3 | 35 | Studio designer metadata — the authoring surfaces for the protocols above. | | [System Protocol](/docs/references/system) | 37 | 286 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. | | [UI Protocol](/docs/references/ui) | 16 | 169 | Apps, pages, views, dashboards, reports, actions and themes — the ObjectUI layer. | -| **Total** | **198** | **1540** | 14 protocol modules | +| **Total** | **199** | **1542** | 14 protocol modules | --- @@ -131,13 +131,14 @@ Flows and their nodes, approvals, ETL pipelines, webhooks, state machines, execu ## Data Protocol -**Source:** `packages/spec/src/data/` · **Import:** `@objectstack/spec/data` · **29 pages, 178 schemas** +**Source:** `packages/spec/src/data/` · **Import:** `@objectstack/spec/data` · **30 pages, 180 schemas** Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. | File | Schemas | | :--- | :--- | | [`analytics.zod.ts`](/docs/references/data/analytics) | `AggregationMetricType`, `AnalyticsDateRange`, `AnalyticsDateRangePreset`, `AnalyticsQuery`, `Cube`, `CubeJoin`, `Dimension`, `DimensionType`, `Metric`, `TimeUpdateInterval` | +| [`column-apply-on.zod.ts`](/docs/references/data/column-apply-on) | `ColumnApplyOn` | | [`context-tokens.zod.ts`](/docs/references/data/context-tokens) | `ContextToken`, `ContextTokenPlaceholder` | | [`data-engine.zod.ts`](/docs/references/data/data-engine) | `BaseEngineOptions`, `DataEngineAggregateOptions`, `DataEngineAggregateRequest`, `DataEngineCountOptions`, `DataEngineCountRequest`, `DataEngineDeleteOptions`, `DataEngineDeleteRequest`, `DataEngineExecuteRequest`, `DataEngineFilter`, `DataEngineFindOneRequest`, `DataEngineFindRequest`, `DataEngineInsertOptions`, `DataEngineInsertRequest`, `DataEngineQueryOptions`, `DataEngineRequest`, `DataEngineSort`, `DataEngineUpdateOptions`, `DataEngineUpdateRequest`, `DataEngineVectorFindRequest`, `DroppedFieldsEvent`, `EngineAggregateOptions`, `EngineCountOptions`, `EngineDeleteOptions`, `EngineQueryOptions`, `EngineUpdateOptions`, `QueryPageWindow`, `QueryPageWindowParam`, `QueryTransportParams`, `QueryWithTransport`, `ValidationAdvisoryEvent` | | [`datasource.zod.ts`](/docs/references/data/datasource) | `Datasource`, `DriverDefinition`, `DriverType`, `ExternalDatasourceSettings`, `SchemaMode` | @@ -163,7 +164,7 @@ Objects, fields, queries, filters, datasources and drivers — the ObjectQL laye | [`object.zod.ts`](/docs/references/data/object) | `ApiMethod`, `ApiOperation`, `Index`, `Lifecycle`, `LifecycleClass`, `Object`, `ObjectAccessConfig`, `ObjectCapabilities`, `ObjectExtension`, `ObjectExternalBinding`, `ObjectFieldGroup`, `ObjectOwnershipEnum`, `ObjectRequiredPermissions`, `PerOperationRequiredPermissions`, `RowCrudActionOverride`, `TenancyConfig` | | [`picklist.zod.ts`](/docs/references/data/picklist) | `Picklist`, `PicklistExtension`, `PicklistServedField` | | [`query.zod.ts`](/docs/references/data/query) | `AggregationFunction`, `AggregationNode`, `DateGranularity`, `FieldNode`, `FullTextSearch`, `GroupByNode`, `Query`, `SortNode` | -| [`seed.zod.ts`](/docs/references/data/seed) | `Seed`, `SeedMode` | +| [`seed.zod.ts`](/docs/references/data/seed) | `Seed`, `SeedColumn`, `SeedMode` | | [`seed-loader.zod.ts`](/docs/references/data/seed-loader) | `ObjectDependencyGraph`, `ObjectDependencyNode`, `ReferenceResolution`, `ReferenceResolutionError`, `SeedIdentity`, `SeedLoadResult`, `SeedLoaderConfig`, `SeedLoaderRequest`, `SeedLoaderResult` | | [`validation.zod.ts`](/docs/references/data/validation) | `ConditionalValidation`, `CrossFieldValidation`, `FormatValidation`, `JSONValidation`, `ScriptValidation`, `StateMachineValidation`, `ValidationRule` | diff --git a/content/docs/references/kernel/manifest.mdx b/content/docs/references/kernel/manifest.mdx index f7d1ca21e72..c0fd420abf2 100644 --- a/content/docs/references/kernel/manifest.mdx +++ b/content/docs/references/kernel/manifest.mdx @@ -55,7 +55,7 @@ Install mode: managed (registered as code, upgradeable, sealed) | unmanaged (cop | **dependencies** | `Record` | optional | Package dependencies | | **configuration** | `never` | optional | [REMOVED] `manifest.configuration` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing ever read the block: no settings UI rendered it and no loader resolved a setting from it, so authoring it configured nothing. Worse, `properties.*.secret` promised "value is encrypted/masked (e.g. API Keys)" while nothing encrypted, masked or even parsed the flag — a false assurance about credential handling. Delete the key. A plugin is configured by the host that composes it: pass options to its constructor in `defineStack({ plugins: [new MyPlugin({ … })] })`, which is the enforced channel. A declarative settings surface must be designed with an enforcing reader first, not revived here. | | **contributes** | `{ kinds?: object[] }` | optional | Platform contributions | -| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; env?: Enum<'prod' \| 'dev' \| 'test'>[]; … }[]` | optional | Initial seed data (prefer top-level data field) | +| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; columns?: Record; … }[]` | optional | Initial seed data (prefer top-level data field) | | **capabilities** | `never` | optional | [REMOVED] `manifest.capabilities` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no discovery path ever consulted the block: nothing read `implements`, `provides`, `requires`, `extensionPoints` or `extensions`, so the declared "interoperability and automatic discovery" never happened. Delete the key. Real dependency resolution runs off top-level `manifest.dependencies`, which stays. Capability-based discovery must be designed with an enforcing reader first, not revived here. | | **extensions** | `never` | optional | [REMOVED] `manifest.extensions` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — an untyped map with zero readers: whatever was parked here was stored and never consulted. Delete the key. Extend the platform through the enforced channels instead: `contributes.kinds` registers metadata kinds, `navigationContributions` injects navigation into other packages' apps, and code-level extension happens in the plugin itself (`init`/`start`). | | **navigationContributions** | `{ app: string; group?: string; priority?: integer; items: (object \| … +9 more)[] }[]` | optional | Navigation items this package contributes into apps owned by other packages | @@ -99,6 +99,7 @@ Install mode: managed (registered as code, upgradeable, sealed) | unmanaged (cop | **object** | `string` | ✅ | Target Object Name | | **externalId** | `string \| string[]` | optional (default: `"name"`) | Field (or composite list of fields) matched for the uniqueness check | | **mode** | `Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>` | optional (default: `"upsert"`) | Conflict resolution strategy | +| **columns** | `Record }>` | optional | Per-column write policy, keyed by field name: `{ status: { applyOn: 'create' } }` writes `status` only when the seed creates the row. Omitted, every column is written on create and update. | | **env** | `Enum<'prod' \| 'dev' \| 'test'>[]` | optional (default: `["prod","dev","test"]`) | Applicable environments | | **locale** | `string[]` | optional | Applicable locales (BCP-47 tags); omitted applies to every locale. The publish and install paths do not filter by locale — they load every dataset and warn | | **records** | `Record[]` | ✅ | Data records | diff --git a/content/docs/references/kernel/package-registry.mdx b/content/docs/references/kernel/package-registry.mdx index 866ee3c7834..da0ea3229a6 100644 --- a/content/docs/references/kernel/package-registry.mdx +++ b/content/docs/references/kernel/package-registry.mdx @@ -210,7 +210,7 @@ Install package request | **dependencies** | `Record` | optional | Package dependencies | | **configuration** | `never` | optional | [REMOVED] `manifest.configuration` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing ever read the block: no settings UI rendered it and no loader resolved a setting from it, so authoring it configured nothing. Worse, `properties.*.secret` promised "value is encrypted/masked (e.g. API Keys)" while nothing encrypted, masked or even parsed the flag — a false assurance about credential handling. Delete the key. A plugin is configured by the host that composes it: pass options to its constructor in `defineStack({ plugins: [new MyPlugin({ … })] })`, which is the enforced channel. A declarative settings surface must be designed with an enforcing reader first, not revived here. | | **contributes** | `{ kinds?: object[] }` | optional | Platform contributions | -| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; env?: Enum<'prod' \| 'dev' \| 'test'>[]; … }[]` | optional | Initial seed data (prefer top-level data field) | +| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; columns?: Record; … }[]` | optional | Initial seed data (prefer top-level data field) | | **capabilities** | `never` | optional | [REMOVED] `manifest.capabilities` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no discovery path ever consulted the block: nothing read `implements`, `provides`, `requires`, `extensionPoints` or `extensions`, so the declared "interoperability and automatic discovery" never happened. Delete the key. Real dependency resolution runs off top-level `manifest.dependencies`, which stays. Capability-based discovery must be designed with an enforcing reader first, not revived here. | | **extensions** | `never` | optional | [REMOVED] `manifest.extensions` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — an untyped map with zero readers: whatever was parked here was stored and never consulted. Delete the key. Extend the platform through the enforced channels instead: `contributes.kinds` registers metadata kinds, `navigationContributions` injects navigation into other packages' apps, and code-level extension happens in the plugin itself (`init`/`start`). | | **navigationContributions** | `{ app: string; group?: string; priority?: integer; items: (object \| … +9 more)[] }[]` | optional | Navigation items this package contributes into apps owned by other packages | @@ -309,7 +309,7 @@ Installed package with runtime lifecycle state | **dependencies** | `Record` | optional | Package dependencies | | **configuration** | `never` | optional | [REMOVED] `manifest.configuration` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — nothing ever read the block: no settings UI rendered it and no loader resolved a setting from it, so authoring it configured nothing. Worse, `properties.*.secret` promised "value is encrypted/masked (e.g. API Keys)" while nothing encrypted, masked or even parsed the flag — a false assurance about credential handling. Delete the key. A plugin is configured by the host that composes it: pass options to its constructor in `defineStack({ plugins: [new MyPlugin({ … })] })`, which is the enforced channel. A declarative settings surface must be designed with an enforcing reader first, not revived here. | | **contributes** | `{ kinds?: object[] }` | optional | Platform contributions | -| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; env?: Enum<'prod' \| 'dev' \| 'test'>[]; … }[]` | optional | Initial seed data (prefer top-level data field) | +| **data** | `{ object: string; externalId?: string \| string[]; mode?: Enum<'insert' \| 'update' \| 'upsert' \| 'replace' \| 'ignore'>; columns?: Record; … }[]` | optional | Initial seed data (prefer top-level data field) | | **capabilities** | `never` | optional | [REMOVED] `manifest.capabilities` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — no discovery path ever consulted the block: nothing read `implements`, `provides`, `requires`, `extensionPoints` or `extensions`, so the declared "interoperability and automatic discovery" never happened. Delete the key. Real dependency resolution runs off top-level `manifest.dependencies`, which stays. Capability-based discovery must be designed with an enforcing reader first, not revived here. | | **extensions** | `never` | optional | [REMOVED] `manifest.extensions` was removed in @objectstack/spec 17 (ADR-0049 enforce-or-remove) — an untyped map with zero readers: whatever was parked here was stored and never consulted. Delete the key. Extend the platform through the enforced channels instead: `contributes.kinds` registers metadata kinds, `navigationContributions` injects navigation into other packages' apps, and code-level extension happens in the plugin itself (`init`/`start`). | | **navigationContributions** | `{ app: string; group?: string; priority?: integer; items: (object \| … +9 more)[] }[]` | optional | Navigation items this package contributes into apps owned by other packages | diff --git a/docs/audits/2026-07-unknown-key-strictness-ledger.counts/data.md b/docs/audits/2026-07-unknown-key-strictness-ledger.counts/data.md index d59e672d6e2..afd86d8067f 100644 --- a/docs/audits/2026-07-unknown-key-strictness-ledger.counts/data.md +++ b/docs/audits/2026-07-unknown-key-strictness-ledger.counts/data.md @@ -21,7 +21,7 @@ The `strict` column is the one the campaign schedules against; it counts both th | Dir | Sites | strict | passthrough | catchall | strip | |---|---|---|---|---|---| -| `data/` | 158 | 81 | 2 | 0 | 75 | +| `data/` | 159 | 82 | 2 | 0 | 75 | ## `data/` — sites @@ -54,9 +54,9 @@ classify and is not listed (it becomes reportable the day it grows its first sit | `picklist.zod.ts` | 3 | | `query.zod.ts` | 5 | | `seed-loader.zod.ts` | 12 | -| `seed.zod.ts` | 1 | +| `seed.zod.ts` | 2 | | `validation.zod.ts` | 6 | -| **total** | **158** | +| **total** | **159** | ## `data/` — open @@ -64,7 +64,7 @@ Per file, how many of its sites still silently discard unknown keys. The `Class` column that decides the bucket split is hand-written in the ledger; the arithmetic over it is here. -**75 strip of 158**, in 10 file(s). +**75 strip of 159**, in 10 file(s). | File | Strip | Sites | |---|---|---| @@ -78,7 +78,7 @@ over it is here. | `hook.zod.ts` | 5 | 7 | | `query.zod.ts` | 4 | 5 | | `seed-loader.zod.ts` | 12 | 12 | -| **total** | **75** | **158** | +| **total** | **75** | **159** | | Bucket | Sites | |---|---| diff --git a/packages/metadata-protocol/src/seed-loader-replay.test.ts b/packages/metadata-protocol/src/seed-loader-replay.test.ts index 807ea04e258..b4fcdc3e02c 100644 --- a/packages/metadata-protocol/src/seed-loader-replay.test.ts +++ b/packages/metadata-protocol/src/seed-loader-replay.test.ts @@ -734,3 +734,298 @@ describe('seed replay — a credential the read path masks never blocks the skip expect(credentialNotices(noCredential.logger)).toEqual([]); }); }); + +describe('seed replay — a column declared applyOn: create is written on first seed only (#25761)', () => { + // The hotclm shape: master data the seed owns (`amount`) beside columns the + // app's own jobs own after the first write (`status`, `is_expiring`, and a + // reference a job re-points, `account`). + const INSTALMENT_FIELDS = { + name: { type: 'text' }, + amount: { type: 'number' }, + status: { type: 'text' }, + is_expiring: { type: 'boolean' }, + account: { type: 'lookup', reference: 'apply_on_account' }, + }; + const ACCOUNT = { name: 'apply_on_account', fields: { name: { type: 'text' } } }; + const INSTALMENT = { name: 'apply_on_instalment', fields: INSTALMENT_FIELDS }; + // A self-reference sends the dataset down the sequential `writeRecord` path. + const TREE_INSTALMENT = { + name: 'apply_on_tree_instalment', + fields: { ...INSTALMENT_FIELDS, parent: { type: 'lookup', reference: 'apply_on_tree_instalment' } }, + }; + const PATHS = [ + { path: 'batched decideWriteAction', object: INSTALMENT, sequential: false }, + { path: 'sequential writeRecord', object: TREE_INSTALMENT, sequential: true }, + ]; + const CREATE_ONLY = { status: { applyOn: 'create' }, is_expiring: { applyOn: 'create' }, account: { applyOn: 'create' } }; + const RECORDS = [ + { name: 'inst-1', amount: 100, status: 'pending', is_expiring: false, account: 'Acme' }, + { name: 'inst-2', amount: 200, status: 'pending', is_expiring: false, account: 'Acme' }, + ]; + + function metadataOf(...objects: Array<{ name: string }>): IMetadataService { + const byName = (name: string) => objects.find((o) => o.name === name); + return { + getObject: vi.fn(async (name: string) => byName(name)), + listObjects: vi.fn(async () => objects), + register: vi.fn(async () => {}), + get: vi.fn(async (_t: string, name: string) => byName(name)), + list: vi.fn(async () => []), + unregister: vi.fn(async () => {}), + exists: vi.fn(async () => false), + listNames: vi.fn(async () => []), + } as unknown as IMetadataService; + } + + async function load( + engine: IDataEngine, + object: { name: string }, + options: { mode?: string; columns?: Record; records?: Array>; logger?: ReturnType } = {}, + ) { + const writeRecord = vi.spyOn(SeedLoaderService.prototype as any, 'writeRecord'); + try { + const result = await new SeedLoaderService(engine, metadataOf(ACCOUNT, object), options.logger ?? createLogger()).load({ + seeds: [ + { object: ACCOUNT.name, externalId: 'name', mode: 'upsert', env: ['prod', 'dev', 'test'], records: [{ name: 'Acme' }, { name: 'Globex' }] }, + { + object: object.name, + externalId: 'name', + mode: options.mode ?? 'upsert', + env: ['prod', 'dev', 'test'], + ...(options.columns ? { columns: options.columns } : {}), + records: options.records ?? RECORDS, + }, + ] as any, + config: CONFIG, + }); + return { result, writeRecordCalls: writeRecord.mock.calls.length }; + } finally { + writeRecord.mockRestore(); + } + } + + const rowOf = (store: Record, object: { name: string }, name: string) => + store[object.name].find((r) => r.name === name)!; + const accountId = (store: Record, name: string) => + store[ACCOUNT.name].find((r) => r.name === name)!.id; + /** What the "jobs" did since the first seed: every create-only column moved, and someone edited `amount`. */ + const runJobs = (store: Record, object: { name: string }) => { + for (const row of store[object.name]) { + row.status = 'overdue'; + row.is_expiring = true; + row.account = accountId(store, 'Globex'); + row.amount = 999; + } + }; + const updatesOf = (engine: IDataEngine, object: { name: string }) => + (engine.update as any).mock.calls.filter(([o]: [string]) => o === object.name).map(([, data]: [string, any]) => data); + + it.each(PATHS)('$path: a fresh insert writes the create-only and the always columns alike', async ({ object, sequential }) => { + const { engine, store } = createFaithfulEngine(); + const first = await load(engine, object, { columns: CREATE_ONLY }); + + expect(first.writeRecordCalls > 0).toBe(sequential); + expect(first.result.success).toBe(true); + expect(first.result.summary.totalInserted).toBe(4); + expect(rowOf(store, object, 'inst-1')).toMatchObject({ + amount: 100, status: 'pending', is_expiring: false, account: accountId(store, 'Acme'), + }); + }); + + it.each(PATHS.flatMap((p) => ['upsert', 'update'].map((mode) => ({ ...p, mode }))))( + '$path, mode $mode: a re-seed after the jobs ran keeps the create-only columns and re-asserts the always ones', + async ({ object, sequential, mode }) => { + const { engine, store } = createFaithfulEngine(); + await load(engine, object, { columns: CREATE_ONLY }); + runJobs(store, object); + (engine.update as any).mockClear(); + + const replay = await load(engine, object, { mode, columns: CREATE_ONLY }); + + expect(replay.writeRecordCalls > 0).toBe(sequential); + expect(replay.result.success).toBe(true); + expect(replay.result.summary.totalErrored).toBe(0); + for (const name of ['inst-1', 'inst-2']) { + expect(rowOf(store, object, name)).toMatchObject({ + status: 'overdue', is_expiring: true, account: accountId(store, 'Globex'), + }); + } + expect(rowOf(store, object, 'inst-1').amount).toBe(100); + expect(rowOf(store, object, 'inst-2').amount).toBe(200); + // The merge set itself: each update names the key and the always column, never a create-only one. + const updates = updatesOf(engine, object); + expect(updates).toHaveLength(2); + for (const data of updates) { + expect(Object.keys(data).sort()).toEqual(['amount', 'id', 'name']); + } + }, + ); + + it.each(PATHS)('$path: control — without the policy the same re-seed reverts the jobs\' columns', async ({ object }) => { + const { engine, store } = createFaithfulEngine(); + await load(engine, object); + runJobs(store, object); + + await load(engine, object); + + expect(rowOf(store, object, 'inst-1')).toMatchObject({ + amount: 100, status: 'pending', is_expiring: false, account: accountId(store, 'Acme'), + }); + }); + + it.each(PATHS)('$path: a row whose only differences are create-only columns is a no-op replay — no update at all', async ({ object }) => { + const { engine, store } = createFaithfulEngine(); + await load(engine, object, { columns: CREATE_ONLY }); + for (const row of store[object.name]) { + row.status = 'overdue'; + row.account = accountId(store, 'Globex'); + } + (engine.update as any).mockClear(); + const before = structuredClone(store); + + const replay = await load(engine, object, { columns: CREATE_ONLY }); + + expect(updatesOf(engine, object)).toEqual([]); + expect(replay.result.summary.totalUpdated).toBe(0); + expect(store).toEqual(before); + }); + + it.each(PATHS)( + '$path: every non-key column create-only — the merge set is the key alone, so the row is skipped, never written with an empty patch', + async ({ object }) => { + const { engine, store } = createFaithfulEngine(); + const allCreateOnly = { ...CREATE_ONLY, amount: { applyOn: 'create' } }; + await load(engine, object, { columns: allCreateOnly }); + runJobs(store, object); + (engine.update as any).mockClear(); + const before = structuredClone(store); + + const replay = await load(engine, object, { columns: allCreateOnly }); + + expect(updatesOf(engine, object)).toEqual([]); + expect(replay.result.summary.totalErrored).toBe(0); + expect(store).toEqual(before); + }, + ); + + it.each(PATHS)('$path: a create-only reference on a matched row is not resolved, not deferred and not dropped', async ({ object }) => { + const { engine, store } = createFaithfulEngine(); + await load(engine, object, { columns: CREATE_ONLY }); + (engine.update as any).mockClear(); + const unresolvable = RECORDS.map((r) => ({ ...r, amount: r.amount + 1, account: 'No Such Co' })); + + const replay = await load(engine, object, { columns: CREATE_ONLY, records: unresolvable }); + + expect(replay.result.success).toBe(true); + expect(replay.result.errors).toEqual([]); + const instalments = replay.result.results.find((r) => r.object === object.name)!; + expect(instalments.referencesDeferred).toBe(0); + expect(instalments.referencesDropped).toBe(0); + expect(updatesOf(engine, object).some((data: Record) => 'account' in data)).toBe(false); + expect(rowOf(store, object, 'inst-1')).toMatchObject({ amount: 101, account: accountId(store, 'Acme') }); + }); + + it.each(PATHS)('$path: control — the same unresolvable reference on a row the load creates is deferred as before', async ({ object }) => { + const { engine } = createFaithfulEngine(); + const unresolvable = RECORDS.map((r) => ({ ...r, account: 'No Such Co' })); + + const first = await load(engine, object, { columns: CREATE_ONLY, records: unresolvable }); + + const instalments = first.result.results.find((r) => r.object === object.name)!; + expect(instalments.referencesDeferred).toBe(2); + expect(first.result.success).toBe(false); + }); + + it('a create-only reference declared BEFORE a reference column of a composite externalId is judged after the key resolves', async () => { + // `billed_to` precedes `account` in the field map, so the reference loop + // meets it first. Judged in that order, the row's key would still hold the + // natural key 'Acme', match nothing, and the unresolvable `billed_to` + // would be deferred to pass 2 — whose back-fill writes it onto the + // matched row after all. + const PERIOD = { + name: 'apply_on_period', + fields: { + billed_to: { type: 'lookup', reference: 'apply_on_account' }, + account: { type: 'lookup', reference: 'apply_on_account' }, + period: { type: 'text' }, + amount: { type: 'number' }, + }, + }; + const { engine, store } = createFaithfulEngine(); + const seedOf = (records: Array>) => [ + { object: ACCOUNT.name, externalId: 'name', mode: 'upsert', env: ['prod', 'dev', 'test'], records: [{ name: 'Acme' }, { name: 'Globex' }] }, + { + object: PERIOD.name, externalId: ['account', 'period'], mode: 'upsert', env: ['prod', 'dev', 'test'], + columns: { billed_to: { applyOn: 'create' } }, records, + }, + ] as any; + const loadPeriods = (records: Array>) => + new SeedLoaderService(engine, metadataOf(ACCOUNT, PERIOD), createLogger()).load({ seeds: seedOf(records), config: CONFIG }); + await loadPeriods([{ account: 'Acme', period: '2026-10', billed_to: 'Acme', amount: 1 }]); + (engine.update as any).mockClear(); + + const replay = await loadPeriods([{ account: 'Acme', period: '2026-10', billed_to: 'No Such Co', amount: 2 }]); + + expect(replay.success).toBe(true); + const periods = replay.results.find((r) => r.object === PERIOD.name)!; + expect(periods.referencesDeferred).toBe(0); + expect(store[PERIOD.name]).toHaveLength(1); + expect(store[PERIOD.name][0]).toMatchObject({ amount: 2, billed_to: accountId(store, 'Acme') }); + expect(updatesOf(engine, PERIOD).some((data: Record) => 'billed_to' in data)).toBe(false); + }); + + it('a column of the externalId declared create-only is still matched on, and stays in the update', async () => { + const { engine, store } = createFaithfulEngine(); + const columns = { ...CREATE_ONLY, name: { applyOn: 'create' } }; + await load(engine, INSTALMENT, { columns }); + runJobs(store, INSTALMENT); + (engine.update as any).mockClear(); + + const replay = await load(engine, INSTALMENT, { columns }); + + expect(replay.result.summary.totalInserted).toBe(0); + expect(store[INSTALMENT.name]).toHaveLength(2); + expect(updatesOf(engine, INSTALMENT)).toHaveLength(2); + for (const data of updatesOf(engine, INSTALMENT)) { + expect(Object.keys(data).sort()).toEqual(['amount', 'id', 'name']); + } + }); + + it('a dry run judges every row as a create: nothing is matched, so the policy takes nothing out', async () => { + const { engine } = createFaithfulEngine(); + const result = await new SeedLoaderService(engine, metadataOf(ACCOUNT, INSTALMENT), createLogger()).validate( + [{ object: INSTALMENT.name, externalId: 'name', mode: 'upsert', columns: CREATE_ONLY, records: RECORDS }] as any, + ); + expect(result.summary.totalInserted).toBe(2); + expect(engine.update).not.toHaveBeenCalled(); + expect(engine.insert).not.toHaveBeenCalled(); + }); + + it('a policy naming no column of the object is reported once, by name, at error, and governs nothing', async () => { + const { engine, store } = createFaithfulEngine(); + const logger = createLogger(); + const misspelt = { stauts: { applyOn: 'create' } }; + await load(engine, INSTALMENT, { columns: misspelt, logger }); + runJobs(store, INSTALMENT); + + await load(engine, INSTALMENT, { columns: misspelt, logger }); + + // `error`, not `warn`: the column it was meant for is overwritten on every + // replay while the load reports success, the AGENTS.md degradation class. + const reports = logger.error.mock.calls.filter(([message]: any[]) => String(message).includes('stauts')); + expect(reports).toHaveLength(2); // once per dataset per load + expect(reports[0][2]).toMatchObject({ object: INSTALMENT.name, undeclaredColumns: ['stauts'] }); + expect(logger.warn.mock.calls.filter(([message]: any[]) => String(message).includes('stauts'))).toEqual([]); + expect(rowOf(store, INSTALMENT, 'inst-1').status).toBe('pending'); + }); + + it('a policy on an injected system column is a column of the object: no report', async () => { + const { engine } = createFaithfulEngine(); + const logger = createLogger(); + await load(engine, INSTALMENT, { columns: { owner_id: { applyOn: 'create' } }, logger }); + expect( + [...logger.warn.mock.calls, ...logger.error.mock.calls].filter((call: any[]) => + call.some((arg) => (arg as { undeclaredColumns?: unknown } | undefined)?.undeclaredColumns)), + ).toEqual([]); + }); +}); diff --git a/packages/metadata-protocol/src/seed-loader.ts b/packages/metadata-protocol/src/seed-loader.ts index c1b1195750d..c6d8d47e664 100644 --- a/packages/metadata-protocol/src/seed-loader.ts +++ b/packages/metadata-protocol/src/seed-loader.ts @@ -480,6 +480,22 @@ function seedValuesInStoredForm(record: Record, definition: unk } } +/** + * [#25761] The columns a dataset writes only when it CREATES the row — those + * its `columns` map declares `applyOn: 'create'` — minus the fields of its + * `externalId`. A key field is what a matched row was found by, so it stays in + * that row's write whatever it declares, as an import's match key stays + * (`FieldMappingEntry.applyOn`, the same policy at the import door). + */ +function createOnlySeedColumns(dataset: Seed, externalId: string | string[]): ReadonlySet { + const keyFields = new Set(Array.isArray(externalId) ? externalId : [externalId]); + const createOnly = new Set(); + for (const [column, policy] of Object.entries(dataset.columns ?? {})) { + if (policy?.applyOn === 'create' && !keyFields.has(column)) createOnly.add(column); + } + return createOnly; +} + /** * [#24562] Field name → storage kind, for every field the definition declares * `datetime`, `date` or `time` — the fields a replay compares in their STORED @@ -1080,6 +1096,9 @@ export class SeedLoaderService implements ISeedLoaderService { const objectName = dataset.object; const mode = dataset.mode || config.defaultMode; const externalId = dataset.externalId || 'name'; + // [#25761] Left out of every write to a row this dataset MATCHED — see + // `matchesExistingRow` in the record loop below. + const createOnlyColumns = createOnlySeedColumns(dataset, externalId); let inserted = 0; let updated = 0; @@ -1333,6 +1352,7 @@ export class SeedLoaderService implements ISeedLoaderService { // judge, and nothing here refuses a row for an owner it cannot prove. const objectDefinition = await this.resolveObjectDefinition(objectName); const rowsCarryOrganization = objectDefinition == null || seedRowNeedsOrganization(objectDefinition); + this.warnOnUndeclaredSeedColumns(dataset, objectDefinition); // How a replay compares each field (`isNoOpReplay`): [#24562] a temporal // field in its stored form, [#24624] a field the read path masks never by // its set value. @@ -1572,14 +1592,43 @@ export class SeedLoaderService implements ISeedLoaderService { }); } - const recordRefs: Array< + /** + * [#25761] Is this row one the load MATCHED — an existing row this + * dataset writes over? The create-only policy is the only thing that + * asks. The same lookup {@link SeedLoaderService.decideWriteAction} + * makes: the record's natural key, as resolved so far, against the rows + * pre-loaded for the modes that match one (`upsert`, `update`, + * `ignore`). A dry run and the `insert` / `replace` modes pre-load + * nothing, so every row they see is one they create. + */ + const matchesExistingRow = (): boolean => + existingRecords?.has(this.externalIdKey(record, externalId)) === true; + + const allRecordRefs: Array< Pick & { multiple?: boolean } > = derivedPointerRefs.length === 0 ? objectRefs : [...objectRefs, ...derivedPointerRefs]; + // [#25761] Create-only references resolve LAST, so a reference column of + // the externalId has been resolved by the time one asks + // `matchesExistingRow`. Every other dataset keeps its order. + const recordRefs = createOnlyColumns.size === 0 + ? allRecordRefs + : [ + ...allRecordRefs.filter((ref) => !createOnlyColumns.has(ref.field)), + ...allRecordRefs.filter((ref) => createOnlyColumns.has(ref.field)), + ]; for (const ref of recordRefs) { // Never re-resolve the tenant stamp we just wrote (see above). A seed // that authors `organization_id` ITSELF still goes through resolution, // so naming an org by its natural key keeps working. if (stampedTenantOrg && ref.field === 'organization_id') continue; + // [#25761] A create-only reference on a row the load matched is no + // part of that row's write: not resolved, not dropped for failing to + // resolve, and not deferred — pass 2's back-fill would write it after + // all, the one place the policy could leak. + if (createOnlyColumns.has(ref.field) && matchesExistingRow()) { + delete record[ref.field]; + continue; + } const fieldValue = record[ref.field]; // A blank reference is NO reference: the engine's blank rule // (`isMissing` — `undefined`, `null`, `''`, whitespace-only) is the one @@ -1809,6 +1858,15 @@ export class SeedLoaderService implements ISeedLoaderService { continue; } + // [#25761] The create-only columns leave a matched row's write here, so + // every write path below sees the same record: the no-op replay check + // compares without them (a value a job has changed since is not a + // difference the seed means to undo), and the update writes without + // them. A row the load creates keeps them all. + if (createOnlyColumns.size > 0 && matchesExistingRow()) { + for (const column of createOnlyColumns) delete record[column]; + } + // [commit 1cba33f16] EARLY SIGNAL — emitted here, before this row reaches the // engine, for the deferrals it took on columns the write contract // requires on INSERT. @@ -3162,6 +3220,51 @@ export class SeedLoaderService implements ISeedLoaderService { ); } + /** + * [#25761] Say so, once per dataset, when its `columns` map names a column + * the object does not have. + * + * Such a policy governs nothing, and nothing else reports it: no row writes + * the named column, so the engine never refuses it the way it refuses an + * unknown record key. The column it was meant for (a misspelt `status`) + * keeps `always`, and every replay re-asserts it. `defineSeed` refuses the + * key when it runs; this is the same check at load, where a stored `seed` + * or a package body arrives without passing through `defineSeed`. Judged + * against the set `defineSeed` uses — the declared fields plus the system + * columns the platform injects on this object — and only when the + * definition resolved with a field map: no opinion without one. + * + * `error`, not `warn` (AGENTS.md "Degradation log levels"): the load looks + * normal and reports success while a job's write to the column the policy + * was meant for is overwritten on every replay. The rows themselves are + * valid, so they still load: refusing them would turn a policy typo into + * missing seed data. + * + * The log is the one channel, on a dry run too. `result.errors` holds + * `ReferenceResolutionError`s, each about one record (`recordIndex`), and a + * dataset-level finding has no shape there. Unlike a dry run's possible + * reference miss (#4997), nothing about it is simulated: the policy is + * wrong whether or not the load writes. + */ + private warnOnUndeclaredSeedColumns(dataset: Seed, definition: unknown): void { + const columns = Object.keys(dataset.columns ?? {}); + if (columns.length === 0) return; + const fields = (definition as { fields?: unknown } | null | undefined)?.fields; + if (!fields || typeof fields !== 'object' || Array.isArray(fields)) return; + const known = new Set([...Object.keys(fields), ...resolveInjectedSystemColumns(definition).names]); + const undeclared = columns.filter((column) => !known.has(column)); + if (undeclared.length === 0) return; + this.logger.error( + `[SeedLoader] The ${dataset.object} seed declares a write policy for ${undeclared.map((c) => `\`${c}\``).join(', ')}, ` + + `which ${undeclared.length === 1 ? 'is not a column' : 'are not columns'} of ${dataset.object}: the policy governs ` + + `nothing, and a column it was meant for keeps \`applyOn: 'always'\`, so every replay overwrites what a job ` + + `wrote to it while this load still reports success. Name a field ${dataset.object} declares, or a system ` + + `column the platform injects on it, as the \`columns\` key.`, + undefined, + { object: dataset.object, undeclaredColumns: undeclared }, + ); + } + /** * Drop datasets that do not apply to this load, on every scope axis. * diff --git a/packages/spec/api-surface/data.json b/packages/spec/api-surface/data.json index 672059599d3..9449990146a 100644 --- a/packages/spec/api-surface/data.json +++ b/packages/spec/api-surface/data.json @@ -102,6 +102,8 @@ "CalendarDateValueSchema (const)", "ClockTimeValue (type)", "ClockTimeValueSchema (const)", + "ColumnApplyOn (type)", + "ColumnApplyOnSchema (const)", "ComparandTypeCase (type)", "ComparandTypeCompilesCase (interface)", "ComparandTypeMatchesCase (interface)", @@ -702,6 +704,9 @@ "SearchFieldMeta (interface)", "SearchFieldResolutionOptions (interface)", "Seed (type)", + "SeedColumn (type)", + "SeedColumnParsed (type)", + "SeedColumnSchema (const)", "SeedIdentity (type)", "SeedIdentitySchema (const)", "SeedImportMode (type)", diff --git a/packages/spec/authorable-defaults/data.json b/packages/spec/authorable-defaults/data.json index d389ef57f0f..a47a7d1fea5 100644 --- a/packages/spec/authorable-defaults/data.json +++ b/packages/spec/authorable-defaults/data.json @@ -100,6 +100,7 @@ "data/Seed:env = [\"prod\",\"dev\",\"test\"]", "data/Seed:externalId = \"name\"", "data/Seed:mode = \"upsert\"", + "data/SeedColumn:applyOn = \"always\"", "data/SeedLoadResult:errors = []", "data/SeedLoadResult:landedIds = []", "data/SeedLoadResult:referencesDropped = 0", diff --git a/packages/spec/authorable-surface/data.json b/packages/spec/authorable-surface/data.json index a10b2f8492b..5fc0f149aed 100644 --- a/packages/spec/authorable-surface/data.json +++ b/packages/spec/authorable-surface/data.json @@ -901,12 +901,14 @@ "data/Seed:_packageId", "data/Seed:_packageVersion", "data/Seed:_provenance", + "data/Seed:columns", "data/Seed:env", "data/Seed:externalId", "data/Seed:locale", "data/Seed:mode", "data/Seed:object", "data/Seed:records", + "data/SeedColumn:applyOn", "data/SeedIdentity:org", "data/SeedIdentity:user", "data/SeedLoadResult:errored", diff --git a/packages/spec/declaration-map/data.json b/packages/spec/declaration-map/data.json index a1bd9cd274e..506e1c65832 100644 --- a/packages/spec/declaration-map/data.json +++ b/packages/spec/declaration-map/data.json @@ -32,6 +32,8 @@ "CalendarDateValueSchema": "data/CalendarDateValue", "ClockTimeValue": "data/ClockTimeValue", "ClockTimeValueSchema": "data/ClockTimeValue", + "ColumnApplyOn": "data/ColumnApplyOn", + "ColumnApplyOnSchema": "data/ColumnApplyOn", "ComparisonOperatorSchema": "data/ComparisonOperator", "ConditionalValidation": "data/ConditionalValidation", "ConditionalValidationSchema": "data/ConditionalValidation", @@ -284,6 +286,8 @@ "ScriptValidation": "data/ScriptValidation", "ScriptValidationSchema": "data/ScriptValidation", "Seed": "data/Seed", + "SeedColumn": "data/SeedColumn", + "SeedColumnSchema": "data/SeedColumn", "SeedIdentity": "data/SeedIdentity", "SeedIdentitySchema": "data/SeedIdentity", "SeedLoadResult": "data/SeedLoadResult", diff --git a/packages/spec/export-origins/data.json b/packages/spec/export-origins/data.json index aa3e5350c10..457d665180b 100644 --- a/packages/spec/export-origins/data.json +++ b/packages/spec/export-origins/data.json @@ -99,6 +99,8 @@ "CalendarDateValueSchema": "src/data/field-value.zod.ts#CalendarDateValueSchema (const)", "ClockTimeValue": "src/data/field-value.zod.ts#ClockTimeValue (type)", "ClockTimeValueSchema": "src/data/field-value.zod.ts#ClockTimeValueSchema (const)", + "ColumnApplyOn": "src/data/column-apply-on.zod.ts#ColumnApplyOn (type)", + "ColumnApplyOnSchema": "src/data/column-apply-on.zod.ts#ColumnApplyOnSchema (const)", "ComparandTypeCase": "src/data/filter-comparand-type-conformance.ts#ComparandTypeCase (type)", "ComparandTypeCompilesCase": "src/data/filter-comparand-type-conformance.ts#ComparandTypeCompilesCase (interface)", "ComparandTypeMatchesCase": "src/data/filter-comparand-type-conformance.ts#ComparandTypeMatchesCase (interface)", @@ -690,6 +692,9 @@ "SearchFieldMeta": "src/data/search-fields.ts#SearchFieldMeta (interface)", "SearchFieldResolutionOptions": "src/data/search-fields.ts#SearchFieldResolutionOptions (interface)", "Seed": "src/data/seed.zod.ts#Seed (type)", + "SeedColumn": "src/data/seed.zod.ts#SeedColumn (type)", + "SeedColumnParsed": "src/data/seed.zod.ts#SeedColumnParsed (type)", + "SeedColumnSchema": "src/data/seed.zod.ts#SeedColumnSchema (const)", "SeedIdentity": "src/data/seed-loader.zod.ts#SeedIdentity (type)", "SeedIdentitySchema": "src/data/seed-loader.zod.ts#SeedIdentitySchema (const)", "SeedImportMode": "src/data/seed.zod.ts#SeedImportMode (type)", diff --git a/packages/spec/json-schema.manifest/data.json b/packages/spec/json-schema.manifest/data.json index 008b39787c0..6391af09b01 100644 --- a/packages/spec/json-schema.manifest/data.json +++ b/packages/spec/json-schema.manifest/data.json @@ -18,6 +18,7 @@ "data/BaseEngineOptions", "data/CalendarDateValue", "data/ClockTimeValue", + "data/ColumnApplyOn", "data/ComparisonOperator", "data/ConditionalValidation", "data/ConsistencyLevel", @@ -155,6 +156,7 @@ "data/ScriptBody", "data/ScriptValidation", "data/Seed", + "data/SeedColumn", "data/SeedIdentity", "data/SeedLoadResult", "data/SeedLoaderConfig", diff --git a/packages/spec/liveness/seed.json b/packages/spec/liveness/seed.json index b3dfac08582..705164500c5 100644 --- a/packages/spec/liveness/seed.json +++ b/packages/spec/liveness/seed.json @@ -20,6 +20,22 @@ "evidence": "packages/metadata-protocol/src/seed-loader.ts#loadDataset (`const mode = dataset.mode || config.defaultMode`); packages/metadata-protocol/src/seed-loader.ts#decideWriteAction (insert/update/upsert/replace/ignore, decided against the existing row)", "note": "insert/update/upsert/replace/ignore — drives decideWriteAction/writeRecord. 2026-08-28: RE-ANCHORED (commit 8f10a79f7) and REPOINTED — `:245` had rotted onto `export class SeedLoaderService`, the class declaration rather than any read of the key. Re-closed by hand against 8cb96ec41." }, + "columns": { + "status": "live", + "verifiedAt": "2026-10-25", + "evidenceScope": "in-repo", + "evidence": "packages/metadata-protocol/src/seed-loader.ts#createOnlySeedColumns (`Object.entries(dataset.columns ?? {})` — the one read of the map, once per dataset in `loadDataset`); packages/metadata-protocol/src/seed-loader.ts#warnOnUndeclaredSeedColumns (a key naming no column of the object is reported once per dataset)", + "note": "Seeded 2026-10-25 (#25761, ruled on hotclm#41 letter A). The per-column write policy, keyed by the target object's field names: the keys are the object's columns (the `records` walk boundary's rule), so the ledger drills only the policy's own keys. `defineSeed` holds the keys to the object's field set when it runs.", + "children": { + "applyOn": { + "status": "live", + "verifiedAt": "2026-10-25", + "evidenceScope": "in-repo", + "evidence": "packages/metadata-protocol/src/seed-loader.ts#createOnlySeedColumns (`policy?.applyOn === 'create'` — the create-only set, minus the externalId's fields); packages/metadata-protocol/src/seed-loader.ts#loadDataset (`matchesExistingRow` — on a matched row the set's references are neither resolved nor deferred, and its columns leave the record before the no-op check and the update)", + "note": "`create` leaves the column out of every write to a row an `upsert` / `update` dataset matched; `always` (the default) writes it as before. Same vocabulary as the import door's `FieldMappingEntry.applyOn` (`ColumnApplyOnSchema`)." + } + } + }, "env": { "status": "live", "verifiedAt": "2026-08-28", diff --git a/packages/spec/liveness/state-counts/seed.md b/packages/spec/liveness/state-counts/seed.md index abd5a20e44e..4b437cea0f8 100644 --- a/packages/spec/liveness/state-counts/seed.md +++ b/packages/spec/liveness/state-counts/seed.md @@ -12,4 +12,4 @@ committed anywhere: `check:liveness` sums the shards when it reads them. | Type | live | exp | elsewhere | dead | planned | classified | |---|---|---|---|---|---|---| -| `seed` | 13 | 0 | 0 | 0 | 0 | 13 | +| `seed` | 14 | 0 | 0 | 0 | 0 | 14 | diff --git a/packages/spec/llms.txt b/packages/spec/llms.txt index 36f87f9a3db..75bfb5c00b7 100644 --- a/packages/spec/llms.txt +++ b/packages/spec/llms.txt @@ -77,7 +77,7 @@ const query = { --- -## 3. Schema Inventory by Domain (206 schemas) +## 3. Schema Inventory by Domain (207 schemas) Counted as `*.zod.ts` modules under `packages/spec/src//` — the sources that ship in this tarball (`files` includes `src/**/*.zod.ts`), so every number @@ -87,7 +87,7 @@ here is verifiable from the installed package. |--------|-------|-------------| | system | 37 | Auth, Cache, Demo Staff, Dev Login, Email / SMS Config, Encryption, Fate Supplement (organization ownership), HTTP Server, License, Logging, Metrics | | kernel | 30 | Plugin, Manifest, Events (6 sub-modules), Feature, Context, Package Registry | -| data | 30 | Object, Field, Picklist, Query, Filter, Driver (SQL/NoSQL/Memory/Mongo/Postgres), Cube | +| data | 31 | Object, Field, Picklist, Query, Filter, Driver (SQL/NoSQL/Memory/Mongo/Postgres), Cube | | api | 31 | Endpoint, REST Server, Discovery, OData, Batch, WebSocket, Response Envelope, Package Lifecycle, Package API (assembled stage) | | ui | 18 | View, App, Action, Dashboard, Page, Chart, Component, Animation | | automation | 14 | Flow, Approval, BPMN Interop, Control Flow, Webhook, Schedule Organization | diff --git a/packages/spec/src/api/export.zod.ts b/packages/spec/src/api/export.zod.ts index e86924ddb38..c2e36816b4c 100644 --- a/packages/spec/src/api/export.zod.ts +++ b/packages/spec/src/api/export.zod.ts @@ -4,6 +4,7 @@ import { z } from 'zod'; import { BaseResponseSchema } from './contract.zod'; import { ValidateDataIssueSchema } from './protocol.zod'; import { DroppedFieldsEventSchema } from '../data/data-engine.zod'; +import { ColumnApplyOnSchema } from '../data/column-apply-on.zod'; /** * Data Export & Import Protocol @@ -170,7 +171,7 @@ export const FieldMappingEntrySchema = lazySchema(() => z.object({ .describe('Default value if source field is null/empty'), required: z.boolean().default(false) .describe('Whether this field is required (import validation)'), - applyOn: z.enum(['create', 'always']).default('always') + applyOn: ColumnApplyOnSchema.default('always') .describe( 'When an import writes this column. `always` (default) writes it on every row the import ' + 'creates or updates. `create` writes it only when the row is created: an `upsert` that ' + diff --git a/packages/spec/src/data/column-apply-on.zod.ts b/packages/spec/src/data/column-apply-on.zod.ts new file mode 100644 index 00000000000..78ce5c9bb1c --- /dev/null +++ b/packages/spec/src/data/column-apply-on.zod.ts @@ -0,0 +1,23 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import { z } from 'zod'; + +/** + * Column Apply-On Policy — when a write door that matches rows by a natural + * key writes a column. + * + * - `always` writes the column on every row the door creates or updates. + * - `create` writes it only when the door creates the row. On a row the door + * matched, the column is left out of the update, so the value the row holds + * (one a job or a person wrote since) stays. + * + * One vocabulary for both doors that upsert by a natural key, so a column means + * the same thing whichever door loads it: an import mapping entry + * (`FieldMappingEntry.applyOn`, #25577) and a seed dataset column + * (`SeedColumn.applyOn`, #25761). Each door declares its own default on its own + * key (`always` on both), and neither spells the values a second time. + */ +export const ColumnApplyOnSchema = z.enum(['create', 'always']) + .describe('When a matched-row write door writes a column: `always` on create and update, `create` on create only'); + +export type ColumnApplyOn = z.input; diff --git a/packages/spec/src/data/define-seed-record-keys.test.ts b/packages/spec/src/data/define-seed-record-keys.test.ts index dcbb75b83a2..c3b0c194c2c 100644 --- a/packages/spec/src/data/define-seed-record-keys.test.ts +++ b/packages/spec/src/data/define-seed-record-keys.test.ts @@ -147,3 +147,49 @@ describe('defineSeed accepts every key that names a column of the target object' .toHaveLength(1); }); }); + +describe('defineSeed holds a `columns` key to the same column set (#25761)', () => { + it('a declared field and an injected system column pass tsc and the call', () => { + const seed = defineSeed(Lead, { + externalId: 'first_name', + columns: { lead_source: { applyOn: 'create' }, owner_id: { applyOn: 'create' } }, + records: [{ first_name: 'Alice', lead_source: 'web' }], + }); + expect(seed.columns).toEqual({ lead_source: { applyOn: 'create' }, owner_id: { applyOn: 'create' } }); + }); + + it('a misspelt column fails tsc, and is refused when it runs, with a near miss', () => { + const error = refusalOf(() => + defineSeed(Lead, { + externalId: 'first_name', + // @ts-expect-error `lead_sorce` is not a field of crm_lead + columns: { lead_sorce: { applyOn: 'create' } }, + records: [{ first_name: 'Alice', lead_source: 'web' }], + }), + ); + expect(error.message.split('\n')[0]).toContain("defineSeed('crm_lead'): unknown field(s) in columns"); + expect(error.message).toContain('columns: `lead_sorce` is not a field of `crm_lead`'); + expect(error.message).toContain("Did you mean 'lead_source'?"); + }); + + it('a record key and a column key are refused together, both surfaces named', () => { + const error = refusalOf(() => + defineSeed(Lead, { + externalId: 'first_name', + // @ts-expect-error `stage` is not a field of crm_lead + columns: { stage: { applyOn: 'create' } }, + // @ts-expect-error `source` is not a field of crm_lead + records: [{ first_name: 'Alice', source: 'web' }], + }), + ); + expect(error.message.split('\n')[0]).toContain('in records and columns'); + expect(error.message).toContain('records[0]: `source`'); + expect(error.message).toContain('columns: `stage`'); + }); + + it('a column the object cannot type-check is still judged when the call runs', () => { + const columns: Record = { stauts: { applyOn: 'create' } }; + const error = refusalOf(() => defineSeed(Lead, { externalId: 'first_name', columns, records: [] })); + expect(error.message).toContain('columns: `stauts` is not a field of `crm_lead`'); + }); +}); diff --git a/packages/spec/src/data/index.ts b/packages/spec/src/data/index.ts index 2e4b99050c1..ddbfa841ef6 100644 --- a/packages/spec/src/data/index.ts +++ b/packages/spec/src/data/index.ts @@ -231,6 +231,10 @@ export * from './driver.zod'; export * from './driver-sql.zod'; export * from './driver-nosql.zod'; +// [#25761] The one `create` / `always` vocabulary of a column's apply-on +// policy, shared by the seed dataset column (`seed.zod`) and the import +// mapping entry (`api/export.zod`), so neither door spells it twice. +export * from './column-apply-on.zod'; export * from './seed.zod'; // Form Layouts diff --git a/packages/spec/src/data/seed.test.ts b/packages/spec/src/data/seed.test.ts index 0e25396c70d..bbc8885e18e 100644 --- a/packages/spec/src/data/seed.test.ts +++ b/packages/spec/src/data/seed.test.ts @@ -1,5 +1,7 @@ import { describe, it, expect } from 'vitest'; -import { SeedSchema, SeedMode, type Seed } from './seed.zod'; +import { SeedSchema, SeedMode, SeedColumnSchema, type Seed } from './seed.zod'; +import { ColumnApplyOnSchema } from './column-apply-on.zod'; +import { FieldMappingEntrySchema } from '../api/export.zod'; describe('SeedMode', () => { it('should accept valid dataset modes', () => { @@ -299,6 +301,58 @@ describe('SeedSchema', () => { }); }); + // ── Column policies (#25761) ───────────────────────────────────────────── + // + // `columns..applyOn: 'create'` is the import door's create-only column + // (`FieldMappingEntry.applyOn`) on a seed dataset: one vocabulary, one schema. + + describe('column policies (columns..applyOn)', () => { + const withColumns = (columns: unknown) => + SeedSchema.safeParse({ object: 'clm_instalment', columns, records: [] }); + + it.each(['create', 'always'] as const)('accepts applyOn %s, the whole parse green', (applyOn) => { + const parsed = withColumns({ status: { applyOn } }); + expect(parsed.success).toBe(true); + expect(parsed.success && parsed.data.columns).toEqual({ status: { applyOn } }); + }); + + it('defaults a declared column to always, and leaves the map ABSENT when it is not written', () => { + const declared = withColumns({ status: {} }); + expect(declared.success && declared.data.columns).toEqual({ status: { applyOn: 'always' } }); + const parsed = SeedSchema.parse({ object: 'clm_instalment', records: [] }); + expect('columns' in parsed).toBe(false); + }); + + it('refuses a value outside create / always at the applyOn path', () => { + const parsed = withColumns({ status: { applyOn: 'insert' } }); + expect(parsed.success).toBe(false); + expect(parsed.error?.issues[0]).toMatchObject({ code: 'invalid_value', path: ['columns', 'status', 'applyOn'] }); + }); + + it('refuses an unknown key on a column — the policy shape is closed', () => { + const parsed = withColumns({ status: { applyOn: 'create', writeOnce: true } }); + expect(parsed.success).toBe(false); + expect(parsed.error?.issues[0]).toMatchObject({ code: 'unrecognized_keys', path: ['columns', 'status'] }); + }); + + it('points a dataset-level applyOn at columns', () => { + const parsed = SeedSchema.safeParse({ object: 'clm_instalment', applyOn: 'create', records: [] }); + expect(parsed.success).toBe(false); + expect(parsed.error?.issues[0]).toMatchObject({ code: 'unrecognized_keys' }); + expect(parsed.error?.issues[0]?.message).toContain('`columns`'); + }); + + it('is the import door\'s vocabulary: one enum, the same values and default on both doors', () => { + expect(ColumnApplyOnSchema.options).toEqual(['create', 'always']); + for (const value of [...ColumnApplyOnSchema.options, 'insert', 'never', '']) { + const seedColumn = SeedColumnSchema.safeParse({ applyOn: value }).success; + const importEntry = FieldMappingEntrySchema.safeParse({ sourceField: 'a', targetField: 'b', applyOn: value }).success; + expect(seedColumn, `seed column, applyOn ${JSON.stringify(value)}`).toBe(importEntry); + } + expect(SeedColumnSchema.parse({}).applyOn).toBe(FieldMappingEntrySchema.parse({ sourceField: 'a', targetField: 'b' }).applyOn); + }); + }); + it('should handle large datasets', () => { const largeDataset = SeedSchema.parse({ object: 'bulk_data', diff --git a/packages/spec/src/data/seed.zod.ts b/packages/spec/src/data/seed.zod.ts index 67a8454d8ae..15b68653583 100644 --- a/packages/spec/src/data/seed.zod.ts +++ b/packages/spec/src/data/seed.zod.ts @@ -12,6 +12,7 @@ import { MetadataProtectionFields } from '../kernel/metadata-protection.zod'; import { LocaleSchema } from '../system/translation.zod'; import { findClosestMatches, formatSuggestion } from '../shared/suggestions.zod'; import { resolveInjectedSystemColumns, type InjectedSystemColumnName } from './injected-system-columns'; +import { ColumnApplyOnSchema } from './column-apply-on.zod'; export const SeedMode = z.enum([ 'insert', // Try to insert, fail on duplicate 'update', // Only update found records, ignore new @@ -20,6 +21,50 @@ export const SeedMode = z.enum([ 'ignore' // Try to insert, silently skip duplicates ]); +/** + * Seed Column — the write policy a seed dataset declares for one of its + * columns, under {@link SeedSchema.shape.columns}. + */ +export const SeedColumnSchema = lazySchema(() => strictObject({ + surface: 'this seed column', + history: 'Declared closed from its first day: an unrecognized key is refused, never dropped.', +}, { + /** + * When a load writes this column (#25761, ruled on hotclm#41: "job-written + * columns are written on first seed only — via the platform"). + * + * `create` takes the column out of every write to a row the load MATCHED by + * the dataset's `externalId`: an `upsert` replay over an existing row, and + * every row of an `update` dataset. The column is not compared for the + * replay's no-op check, not written, and — a reference column — not + * resolved and not deferred to pass 2, so nothing the load does afterwards + * writes it back. A row the load creates gets it like any other column. + * `insert`, `ignore` and `replace` never update a matched row, so the policy + * changes nothing on those modes. + * + * A column of the dataset's `externalId` is the key the row was matched by: + * it stays in the matched row's update whatever it declares, as an import's + * match key does. + * + * Same values, same default and same meaning as the import door's + * `FieldMappingEntry.applyOn` ({@link ColumnApplyOnSchema}). + */ + applyOn: ColumnApplyOnSchema.default('always') + .describe( + 'When a seed load writes this column. `always` (default) writes it on every row the load ' + + 'creates or updates. `create` writes it only when the load creates the row: an `upsert` ' + + 'replay that matches an existing row leaves the row\'s value as it is, and an `update` ' + + 'dataset never writes it, so replaying the seed refreshes the `always` columns without ' + + 'reverting state the app wrote since the first load. A column of the dataset\'s ' + + '`externalId` is still matched on and stays in the matched row\'s update. The same values ' + + 'and meaning as an import mapping entry\'s `applyOn`.', + ), +})); +/** Author-time shape of {@link SeedColumnSchema}. */ +export type SeedColumn = z.input; +/** Post-parse shape of {@link SeedColumn} — defaults applied, transforms run (ADR-0122). */ +export type SeedColumnParsed = z.infer; + /** * Seed Schema (Seed Data / Fixtures) * @@ -55,6 +100,11 @@ export const SeedSchema = lazySchema(() => strictObject({ language: 'locale', languages: 'locale', }, + guidance: { + applyOn: + '`applyOn` is a per-column policy, not a dataset key — declare it under `columns`, keyed by the ' + + "field it governs: `columns: { status: { applyOn: 'create' } }`.", + }, }, { /** * Target Object @@ -87,6 +137,33 @@ export const SeedSchema = lazySchema(() => strictObject({ */ mode: SeedMode.default('upsert').describe('Conflict resolution strategy'), + /** + * Column Policies + * + * Per-column write policy, keyed by the field a record writes; a column the + * map does not name keeps every default. Today one policy, + * {@link SeedColumnSchema.shape.applyOn}. + * + * A seed is replayed — every dev-server boot, every package re-publish — and + * an `upsert` replay re-asserts every column a record declares on the row it + * matched. That is right for master data and wrong for a column the app owns + * after the first write: a status a scheduled job advances, a flag a job sets + * once its notice is sent. Declared `applyOn: 'create'`, such a column is + * written when the seed creates the row and left as the row holds it on every + * replay after. + * + * Each key must name a column the target object has — a declared field or a + * system column the platform injects on it. `defineSeed` refuses one that + * does not when it runs, and the loader logs an error naming the column + * whenever it loads a dataset whose map names one (a stored seed never + * passes through `defineSeed`). + */ + columns: z.record(z.string(), SeedColumnSchema).optional() + .describe( + 'Per-column write policy, keyed by field name: `{ status: { applyOn: \'create\' } }` writes ' + + '`status` only when the seed creates the row. Omitted, every column is written on create and update.', + ), + /** * Environment Scope * - 'all': Always load @@ -204,9 +281,19 @@ type SeedRecord = { [K in Exclude]?: unknown; }; +/** + * Shape of a seed's {@link SeedSchema.shape.columns} map for one object: the + * same key set as {@link SeedRecord} (the object's declared `fields` plus the + * injectable system columns), each taking a {@link SeedColumn}. + */ +type SeedColumns = { + [K in keyof TFields | InjectedSystemColumnName]?: SeedColumn; +}; + /** * The call-time half of {@link defineSeed}'s key check: every key of every - * record must name a column the target object has, which is a field it + * record, and every key of the `columns` map, must name a column the target + * object has, which is a field it * declares or a system column the platform injects on THIS object. The injected * set is {@link resolveInjectedSystemColumns}, the per-object answer the * registry's own injection consumes, so a seed may write `created_at` (the @@ -226,9 +313,10 @@ type SeedRecord = { * No opinion without a field map: a caller outside the type that passes no * `fields` object is not judged here. */ -function assertSeedRecordKeysDeclared( +function assertSeedKeysDeclared( objectDef: { name: string; fields: unknown }, records: ReadonlyArray>, + columns: Readonly> | undefined, ): void { const { fields } = objectDef; if (!fields || typeof fields !== 'object' || Array.isArray(fields)) return; @@ -236,26 +324,32 @@ function assertSeedRecordKeysDeclared( const knownSet = new Set(known); const lines: string[] = []; const unknownKeys = new Set(); + const surfaces = new Set<'records' | 'columns'>(); + const judge = (key: string, where: string, surface: 'records' | 'columns'): void => { + if (knownSet.has(key)) return; + unknownKeys.add(key); + surfaces.add(surface); + const suggestion = formatSuggestion( + findClosestMatches(key, known, Math.max(2, Math.floor(key.length / 3)), 1), + ); + lines.push( + ` • ${where}: \`${key}\` is not a field of \`${objectDef.name}\`.` + + (suggestion ? ` ${suggestion}` : ''), + ); + }; records.forEach((record, index) => { - for (const key of Object.keys(record)) { - if (knownSet.has(key)) continue; - unknownKeys.add(key); - const suggestion = formatSuggestion( - findClosestMatches(key, known, Math.max(2, Math.floor(key.length / 3)), 1), - ); - lines.push( - ` • records[${index}]: \`${key}\` is not a field of \`${objectDef.name}\`.` - + (suggestion ? ` ${suggestion}` : ''), - ); - } + for (const key of Object.keys(record)) judge(key, `records[${index}]`, 'records'); }); + // [#25761] A policy on a column the object does not have governs nothing: + // the column it was meant for (a misspelt `status`) stays `always`. + for (const key of Object.keys(columns ?? {})) judge(key, 'columns', 'columns'); if (lines.length === 0) return; throw new Error( - `defineSeed('${objectDef.name}'): unknown field(s) in records — ${[...unknownKeys].join(', ')}.\n` - + `A seed record key must name a field \`${objectDef.name}\` declares or a system column the platform ` - + 'injects on it (`created_at`, `owner_id` and the rest, unless the object opts out of them).\n\n' + `defineSeed('${objectDef.name}'): unknown field(s) in ${[...surfaces].join(' and ')} — ${[...unknownKeys].join(', ')}.\n` + + `A seed record key, and a \`columns\` key, must name a field \`${objectDef.name}\` declares or a system ` + + 'column the platform injects on it (`created_at`, `owner_id` and the rest, unless the object opts out of them).\n\n' + `${lines.join('\n')}\n\n` - + `Fix the key's spelling, declare the field on \`${objectDef.name}\`, or remove the key from the record.`, + + `Fix the key's spelling, declare the field on \`${objectDef.name}\`, or remove the key.`, ); } @@ -278,10 +372,14 @@ function assertSeedRecordKeysDeclared( * ({@link resolveInjectedSystemColumns}). A key that is neither is refused, * naming the object, the record and the key, with a near-miss suggestion. * + * The keys of {@link SeedSchema.shape.columns} are held to the same set, both + * times ({@link SeedColumns}). + * * @example * ```ts * export const leadSeed = defineSeed(Lead, { * externalId: 'email', + * columns: { lead_source: { applyOn: 'create' } }, // ✅ written when the row is created, kept on replay * records: [ * { first_name: 'Alice', lead_source: 'web' }, // ✅ type-checked * { source: 'web' }, // ❌ compile error (unknown field), and refused when it runs @@ -296,11 +394,12 @@ export function defineSeed< const TObj extends { name: string; fields: Record } >( objectDef: TObj, - config: Omit & { + config: Omit & { records: Array>; + columns?: SeedColumns; } ): Seed { const seed = SeedSchema.parse({ ...config, object: objectDef.name }); - assertSeedRecordKeysDeclared(objectDef, seed.records); + assertSeedKeysDeclared(objectDef, seed.records, seed.columns); return seed; } diff --git a/packages/spec/src/type-alias-convention.pin.test.ts b/packages/spec/src/type-alias-convention.pin.test.ts index a81d569bda7..273f20d0de5 100644 --- a/packages/spec/src/type-alias-convention.pin.test.ts +++ b/packages/spec/src/type-alias-convention.pin.test.ts @@ -276,9 +276,11 @@ import type * as M189 from './system/fate-supplement.zod.js'; import type * as M190 from './kernel/execution-context.zod.js'; // An api flow's hook secret named by reference (`config.secretRef`) -- new module, next free index. import type * as M191 from './automation/api-trigger-secret-ref.zod.js'; +// [#25761] The one column apply-on vocabulary both write doors read -- new module, next free index. +import type * as M192 from './data/column-apply-on.zod.js'; // --------------------------------------------------------------------------- -// 786 isomorphic aliases: `z.input` === `z.infer`, so no `XParsed` is declared. +// 787 isomorphic aliases: `z.input` === `z.infer`, so no `XParsed` is declared. // // That number is machine-checked, not hand-kept. The runtime companion at the // bottom of this file recomputes the pin count from the source and asserts that @@ -780,6 +782,9 @@ export type Iso_data_analytics__DimensionType = Assert, z.infer< typeof M55.MetricSchema > >>; export type Iso_data_analytics__TimeUpdateInterval = Assert, z.infer< typeof M55.TimeUpdateInterval > >>; +// data/column-apply-on.zod.ts +export type Iso_data_columnApplyOn__ColumnApplyOnSchema = Assert, z.infer< typeof M192.ColumnApplyOnSchema > >>; + // data/context-tokens.zod.ts export type Iso_data_contextTokens__ContextTokenPlaceholderSchema = Assert, z.infer< typeof M176.ContextTokenPlaceholderSchema > >>; @@ -1701,7 +1706,7 @@ describe('ADR-0122 type-alias convention', () => { // this title and the section header above the pin list — are now asserted // against the recomputed count below, so neither can go stale without a red // test naming it. - it('still declares all 786 isomorphic pins', () => { + it('still declares all 787 isomorphic pins', () => { // The truth of each pin is proved by tsc, not here — an `Assert>` // that stops holding is a compile error with the alias named. What tsc // cannot notice is a pin that was DELETED: removing the assertion removes @@ -2577,7 +2582,14 @@ describe('ADR-0122 type-alias convention', () => { // module, its `M92` import with them (the slot stays empty; the numbering // is a label, not a count). The module's other defs each carried an // `XParsed` alias and were never on this list. -4 removed. - expect(pins).toHaveLength(786); + // + // 786 -> 787 is #25761: `ColumnApplyOnSchema` (`data/column-apply-on.zod.ts`), + // the one `'create' | 'always'` vocabulary an import mapping entry and a seed + // dataset column both read — a plain string enum extracted from + // `FieldMappingEntrySchema.applyOn`, so it is pinned here + // (Iso_data_columnApplyOn__ColumnApplyOnSchema) rather than given a + // `…Parsed` synonym. +1 added. + expect(pins).toHaveLength(787); // The count is stated in PROSE twice as well — this case's title and the // section header above the pin list — and until commit c6b05c76a nothing read either diff --git a/skills/objectstack-data/references/_index.md b/skills/objectstack-data/references/_index.md index 5214ab6627f..911c10e155b 100644 --- a/skills/objectstack-data/references/_index.md +++ b/skills/objectstack-data/references/_index.md @@ -13,7 +13,7 @@ from `node_modules` — there is no local copy in the skill bundle. - `node_modules/@objectstack/spec/src/data/field.zod.ts` — Exports: FieldType, SelectOptionSchema, LocationCoordinatesSchema, CurrencyConfigSchema, CurrencyValueSchema - `node_modules/@objectstack/spec/src/data/hook.zod.ts` — Exports: HookEvent, HookSchema, HookContextSchema - `node_modules/@objectstack/spec/src/data/object.zod.ts` — Exports: ApiMethod, ApiOperationSchema, ObjectCapabilities, IndexSchema, TenancyConfigSchema -- `node_modules/@objectstack/spec/src/data/seed.zod.ts` — Exports: SeedMode, SeedSchema, leadSeed +- `node_modules/@objectstack/spec/src/data/seed.zod.ts` — Exports: SeedMode, SeedColumnSchema, SeedSchema, leadSeed - `node_modules/@objectstack/spec/src/data/validation.zod.ts` — ObjectStack Validation Protocol - `node_modules/@objectstack/spec/src/security/permission.zod.ts` — Exports: ObjectAccessScopeSchema, ObjectPermissionSchema, EffectiveObjectPermissionSchema, AdminScopeSchema, FieldPermissionSchema @@ -21,6 +21,7 @@ from `node_modules` — there is no local copy in the skill bundle. - `node_modules/@objectstack/spec/src/api/errors.zod.ts` — Standardized Error Codes Protocol - `node_modules/@objectstack/spec/src/automation/flow-function.zod.ts` — The contract for a **named handler function a `script` node invokes** — +- `node_modules/@objectstack/spec/src/data/column-apply-on.zod.ts` — Exports: ColumnApplyOnSchema - `node_modules/@objectstack/spec/src/data/data-engine.zod.ts` — Data Engine Protocol - `node_modules/@objectstack/spec/src/data/driver-sql.zod.ts` — Exports: SQLDialectSchema, DataTypeMappingSchema, SSLConfigSchema, SQLDriverConfigSchema, SQLiteDataTypeMappingDefaults - `node_modules/@objectstack/spec/src/data/driver.zod.ts` — Exports: DriverOptionsSchema, DriverCapabilitiesSchema, DriverInterfaceSchema, PoolConfigSchema, DriverConfigSchema diff --git a/skills/objectstack-platform/references/_index.md b/skills/objectstack-platform/references/_index.md index 5db87a4239d..fd8975e38c7 100644 --- a/skills/objectstack-platform/references/_index.md +++ b/skills/objectstack-platform/references/_index.md @@ -10,7 +10,7 @@ from `node_modules` — there is no local copy in the skill bundle. ## Core schemas - `node_modules/@objectstack/spec/src/data/datasource.zod.ts` — Exports: DriverType, DriverDefinitionSchema, SchemaModeSchema, ExternalDatasourceSettingsSchema, DatasourceSchema -- `node_modules/@objectstack/spec/src/data/seed.zod.ts` — Exports: SeedMode, SeedSchema, leadSeed +- `node_modules/@objectstack/spec/src/data/seed.zod.ts` — Exports: SeedMode, SeedColumnSchema, SeedSchema, leadSeed - `node_modules/@objectstack/spec/src/kernel/context.zod.ts` — Exports: RuntimeMode, KernelContextSchema, TenantRuntimeContextSchema - `node_modules/@objectstack/spec/src/kernel/manifest.zod.ts` — Exports: PluginPermissionsSchema, ManifestPermissionsSchema, PluginEnginesSchema, PluginRuntimeSchema, PluginPackagingSchema - `node_modules/@objectstack/spec/src/kernel/metadata-plugin.zod.ts` — Metadata Plugin Protocol