Repository navigation
Commit 572ecfb
fix(spec,service-automation,metadata-protocol): the v18 stored preview lists every row protocol 18 refuses and no conversion rewrites (#23315)
Fixes #23269
Fixes #23270
Fixes #23271
Clause-②: yes (widening)
## What this does
`os migrate meta --stored` now lists every stored row that protocol 18
refuses and that no conversion rewrites. Before, the preview counted
each such row as `canonical` ("already on protocol"), while the parse or
the boot refused it after the upgrade. All three cards get the same
fate, **refused and listed**, through the stored preview's existing TODO
lines. Each line names the step-18 D3 entry that refuses the shape, the
site, and the choice the owner has to make. Nothing is rewritten, on
preview or on `--apply`, and `storedMigrationClean` does not move.
| card | shape | fate | listed by | TODO id |
|:--|:--|:--|:--|:--|
| #23269 | a flow value slot carrying a `{…}` token, including a
`get_record` / `update_record` / `delete_record` filter operand | refuse
and list | flow, node, path | `flow-value-slot-template-dialect-refused`
|
| #23270 | `controlled_by_parent` with two masters (both required, or
neither), no master, a required `lookup` only, an explicit `required:
false` | refuse and list, each with its own fate sentence | object |
`cbp-master-relation-tiers-retired` |
| #23271 | a view whose `data` binds the retired view-level API source,
in every stored view body spelling and in an object's `listViews` |
refuse and list; request headers withheld | view |
`view-data-api-provider-retired` |
The one lossless shape in this family, a single `master_detail` with
`required` omitted, is still converted by
`cbp-master-detail-required-defaulted` and is not listed.
## Public surface: Clause ②, yes (widening)
No door accepts or refuses anything it did not before. The published
flow-canonicalization result gains one output member, declared in the
contract and read through it:
- `IAutomationService.canonicalizeStoredFlow(...).todos?`
(`@objectstack/spec`, `contracts/automation-service.ts`): optional, the
sibling of `notices` and `conflicts`, so an implementation that reports
none still conforms.
- `AutomationEngine.canonicalizeStoredFlow(...).todos`
(`@objectstack/service-automation`): the engine always returns it.
- `StoredFlowCanonicalization.todos?`
(`@objectstack/metadata-protocol`): restates the contract's optional
member, so `flowResult.todos ?? []` reads a member the contract
declares.
Also widened: the published `ConversionTodoNotice.conversionId` may now
name a D3 entry as well as a conversion, and an `onTodo` sink now also
receives the refused-shape TODOs. All three packages ship `minor`
(`.changeset/23269-v18-stored-row-fates.md`).
## The #23269 choice, measured: (b) refuse and list
The lane's ruling allowed (a) only if the chain already declares an
operator-applied rewrite of stored rows that is separate from the
load-path D2. It does not:
- `migrateStoredMetadata` (the `--stored` pass) writes back the read
path's own pass: `applyConversionsToStoredItem` for most types, and
`AutomationEngine.canonicalizeStoredFlow` for a flow, which is the
`registerFlow` seam itself.
- `--from 17` (`applyMetaMigrations`) replays step conversions over
authored source files, never over rows.
- The decision-mode precedent (`flow-decision-mode-inclusive-explicit`)
shows the same boundary. It replays over authored sources only and is
refused at rehydration by `CONVERSIONS_NOT_REPLAYED_AT_REHYDRATION`. For
stored rows it ships a report-only list, not a rewrite.
So (b) applies, and the changeset names it.
## Zone 2 assumptions, each measured
1. **The report channel: partly disproved.**
- The D3 entries carry no stored-row detector, and `--stored` never
reads D3 entries. What it does read is the conversion TODO sink
(`onTodo`, `CONVERSION_TODO_CODE`). For a flow row,
`migrateStoredMetadata` hardcoded `todos: []` and ran only
`canonicalizeStoredFlow`.
- Why the cloud seat's preview named no refused flow:
`FlowSchema.safeParse` accepts a `get_record` filter of `{ id:
'{record.id}' }`, which I measured at `success: true` on this tree. The
refusal is `flowNodeValueTemplateRefusals`, which only `registerFlow`'s
`validateFlowExpressions` runs, after canonicalization. Objects and
views had the same gap: the preview never parses a row it does not
rewrite, so a refused row read as `canonical`.
- A TODO-only D2 entry would have been wrong: `spec-changes.json`
projects every D2 entry as "A lossless conversion applied at load". The
detectors therefore live in their own in-package list
(`packages/spec/src/conversions/refused-shapes.ts`). They are keyed by
their D3 entry ids, they run at the end of `applyConversions` only when
a caller passes `onTodo`, and they never rewrite.
2. **#23270's shapes: holds.** No shape converts losslessly. Two masters
need a pick. A required `lookup` turned into a `master_detail` changes
delete behaviour. An explicit `required: false` is a contradiction the
builder refuses: writing `true` would change what the field accepts and
override what the author wrote. No master leaves nothing to convert.
3. **#23271's move: holds.** No mechanical move exists. A page must
exist to carry the `dataSource`, and deleting `data` makes the view
query its object instead. The population is not provably empty: the save
door accepted the arm before #23160, so I list instead of treating the
population as empty.
4. **The predicate: holds, and is shared.** The detector reuses
ObjectSchema's one-required-master predicate, extracted into the
internal module `packages/spec/src/data/controlled-by-parent-master.ts`.
It is not re-exported, so #23272 stays its own card. The refinement's
messages are byte-identical. The view detector asks
`ViewDataSchema.safeParse` itself, and the flow detector calls
`flowNodeValueTemplateRefusals`, so each listing is exactly what its
door refuses.
5. **The boot behaviour: holds.** It is out of scope, reported below,
and the runtime is not changed.
6. **Hot files.** No generated file is committed. `registry.ts` is
gitignored and regenerated at install, build and test. I merged `main`
three times, never with a conflict.
## Where it lands, and why outside the claim's file surface
The claim's surface is `packages/spec`. For flows, measurement put the
producer elsewhere, and the dispatch's file-surface clause says to fix
at the producer and name it. Changing
`packages/cli/src/commands/migrate/meta*` (the dispatch's STOP path) was
not needed: the CLI renders `formatStoredMigrationReport`, which lives
in metadata-protocol.
- `packages/services/service-automation/src/engine.ts`:
`canonicalizeStoredFlow` passes `onTodo` and returns `todos` beside
`notices` and `conflicts`. `registerFlow` ignores them, because its own
refusal names the same sites.
- `packages/metadata-protocol/src/stored-migration.ts`:
`StoredFlowCanonicalization` gains an optional `todos`, so a
caller-supplied canonicalizer that reports none still fits.
- `packages/metadata-protocol/src/protocol.ts`: the flow branch reads
`flowResult.todos ?? []`. The skip reason of a row that only has TODOs
no longer says "The row keeps loading unchanged", which this change
would have made false for a refused flow. It now says the row is read as
stored and that each TODO says what that leaves it doing.
Inside `packages/spec`:
- `conversions/refused-shapes.ts` (new), `conversions/apply.ts` (the
call), and doc-only edits to `conversions/types.ts` (a TODO's
`conversionId` may name a D3 entry) and `conversions/stored.ts`;
- `contracts/automation-service.ts`: `canonicalizeStoredFlow`'s return
declares `todos?` (the public-surface section above);
- `data/controlled-by-parent-master.ts` (new) and `data/object.zod.ts`
(the refinement delegates to it);
- one sentence on each of the three D3 entries, saying what `--stored`
does with these rows.
Docs: one row in `content/docs/deployment/cli.mdx`'s `--stored` decline
table. Changeset: `.changeset/23269-v18-stored-row-fates.md` (`minor`
for spec, service-automation and metadata-protocol).
Both new spec modules (`refused-shapes.ts`,
`controlled-by-parent-master.ts`) are internal: neither is re-exported,
so no export name changes. The one contract member they need is declared
above.
## Reach decision: all value slots, not only CRUD filters
The flow detector lists every refusal of
`flowNodeValueTemplateRefusals`, not only the CRUD filter operands. That
is the D3 entry `flow-value-slot-template-dialect-refused` whole, and
`registerFlow` refuses every one of those sites the same way. Limiting
it to filters would have taken extra code that filters the one judge,
and the preview would have stayed silent about flows the boot refuses
for the same retirement. The four conditions for a bounded in-place fix
hold: same defect class, mechanical, no other claim on these files, same
gate family.
## Pins
- **#23269:**
- a braced operand on `get_record`, `update_record` and `delete_record`
(field value, `$gt` comparand, `$in` element), and one inside a loop
body: one TODO each, naming the flow, the node and the operand, with its
CEL envelope;
- CEL-envelope, query-placeholder and literal flows: untouched and not
listed;
- the engine returns the TODO, and `registerFlow` refuses the same site;
- the preview lists the flow, `--apply` writes nothing, and the CEL form
is `canonical` and registers.
- "A converted row boots and runs" does not apply under (b): no row
converts. The CEL-form row booting is pinned instead.
- **#23270:**
- one pin per shape: two required masters, two unrequired masters,
explicit `required: false`, required lookup only, no master;
- three conforming objects untouched;
- the Studio shape still converts and is not listed;
- another sharing model is not listed;
- the preview lists four objects, counts the conforming one `canonical`,
writes nothing on `--apply`, and leaves `storedMigrationClean` true.
- **#23271:**
- all five stored view spellings, and an object's `listViews`: listed;
- no header value reaches `from`, `reason` or `message`;
- `object`, `value` and `schema` providers: untouched.
## Verification
### Patch round, at `HEAD` `086d6f1ad` (merge base with `origin/main`:
`404fc477e`)
The contract-tier review (record 6152721854) failed this PR on the
semver and Clause-② section only. This round makes its three edits, with
no change to any door, detector or predicate:
- the contract declares `todos?`;
- the changeset is `minor` for all three packages, with `Clause-②: yes
(widening)` naming the three members;
- this body's `Clause-②` line matches.
`main` was merged first: `331fa99bb`, no conflict.
- Build: `turbo run build` over the closures of
`@objectstack/service-automation^...` and
`@objectstack/metadata-protocol^...`, plus the `@objectstack/client...`
and `@objectstack/client-react...` closures: 35 of 35 tasks, exit 0.
- `pnpm --filter @objectstack/spec check:generated`: exit 0, "All 14
generated artifacts are up to date", `check:api-surface` and
`check:export-origins` among them. Nothing was regenerated, because the
api-surface snapshot records export names, not interface members.
- Reverse verification of the contract change:
`AutomationEngine.canonicalizeStoredFlow`'s `todos` widened to
`ConversionTodoNotice[] | number` through
`scripts/ablation-replace.mjs`. `tsc --noEmit` in service-automation
went red as predicted: `src/engine.ts(4364,5): error TS2416: Property
'canonicalizeStoredFlow' in type 'AutomationEngine' is not assignable to
the same property in base type 'IAutomationService'`. A contract without
`todos` would accept that member, so the red proves `tsc` read the
rebuilt `dist` declarations. Restore proven: blob equal to `HEAD`, empty
`git diff HEAD`.
- `typecheck` for `@objectstack/spec`, `@objectstack/service-automation`
and `@objectstack/metadata-protocol`: exit 0 each.
- Pins, all exit 0:
- spec: `refused-shapes`, `stored` and
`cbp-master-detail-required-defaulted`, 61 tests;
- service-automation: `canonicalize-stored-flow` and `engine`, 175
tests;
- metadata-protocol: `protocol.stored-migration` and
`protocol.flow-canonicalizer`, 60 tests.
- `pnpm --filter @objectstack/spec test:repo`: measured in full, exit 0,
58 files, 1007 tests.
- Gates: `node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` derived 115 commands at `086d6f1ad`, with no
stale-tree warning. All exited 0, the Clause-② gates
(`check-changeset-no-major`, `check-adr-0087-registration`) and
`check:api-surface` included.
- `check:dual-build-cjs-loads` first answered PREREQUISITE NOT MET (exit
3), because 33 packages had no `dist` in the recreated worktree. Re-run
after the battery, whose later gates built them, it gave exit 0: "108
published require entry point(s) across 67 package(s) load".
- Reconcile: "✓ dispatch-gates --ran: 115 derived famil(ies) accounted
for — 115 run, 0 NOT-MEASURED (a DERIVED zero — all 115 recorded an exit
code and none of them is 3)."
- `origin/main` has since gained `87e902dcb`, `4d4040ccb` and
`37510aa0f`, none under the three changed packages. Not merged.
### First round, at `HEAD` `af10b608d` (merge base with `origin/main`:
`9c7cfbf0a`) unless a line names another commit
- Build: `turbo run build` over the closures of
`@objectstack/service-automation^...` and
`@objectstack/metadata-protocol^...`, plus the `@objectstack/client...`
and `@objectstack/client-react...` closures: 35 of 35 tasks, exit 0.
- `pnpm --filter @objectstack/spec test`: exit 0, 670 files, 20307 tests
(1 todo).
- `pnpm --filter @objectstack/metadata-protocol test`: exit 0, 235
passed and 3 skipped files, 28336 tests.
- `pnpm --filter @objectstack/service-automation test`: exit 0, 185
files, 2451 tests, at `e43c67948`. The merge after it brought no change
under that package; the pins re-ran on `af10b608d`:
`canonicalize-stored-flow.test.ts`, 11 of 11.
- `typecheck` for `@objectstack/spec`, `@objectstack/metadata-protocol`
and `@objectstack/service-automation`: exit 0 each.
- `pnpm --filter @objectstack/spec test:repo`: **NOT MEASURED** as a
whole. I stopped my own run at 22 minutes of lock hold while another
agent was queued (exit 143). Declared narrowing, at `e43c67948`: the 34
`src/` tree-scan pins plus `scripts/conversions-major18-merge.test.ts`
ran, exit 0, 35 files, 482 tests. The other 23 `scripts/` tests exercise
`packages/spec/scripts/*`, which this diff does not touch. CI runs the
whole project.
- Gates: `node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` derived 115 commands at `af10b608d`, and
every one exited 0. Reconcile: "✓ dispatch-gates --ran: 115 derived
famil(ies) accounted for — 115 run, 0 NOT-MEASURED (a DERIVED zero — all
115 recorded an exit code and none of them is 3)."
- An earlier battery caught one real finding, now fixed:
`check:dispatcher-error-vocabulary` saw `OS_METADATA_CONVERSION_TODO`
stamped in `refused-shapes.ts`, an unclassified site. That code is now
stamped only in `apply.ts`, the classified one.
- After the merge, `origin/main` moved four commits: `280806912`, the
CLI organization-ownership ceremony; `4c54486dd`, a fix to the
derivation script; `5c9bfb0d3`, plugin-security; `404fc477e`,
plugin-auth. None touches `packages/spec`, `packages/metadata-protocol`
or `packages/services/service-automation`, so I did not merge them then;
the patch round merged them (`331fa99bb`).
- Lint, a proven narrowing, not `pnpm lint`. `eslint --no-inline-config
--format json` over the 16 changed source and docs files: 16 entries, 0
errors, 0 rule warnings.
- `content/docs/deployment/cli.mdx` is outside the population: eslint
itself answers "File ignored because no matching configuration was
supplied".
- Invariance: `eslint.config.mjs` never enables type-aware linting (no
`parserOptions.project`, no typed rules; the note above
`QUERY_OPTIONS_TEST_GLOBS`). So this diff cannot move an untouched
file's verdict through type information.
- Spec's built CJS entries, `dist/index.js` and `dist/browser/index.js`,
both load and list a refused object and a refused view through
`applyConversionsToStoredItem`.
- Ablations, one-shot, through `scripts/ablation-replace.mjs`, run
before the stamping move. Each restore was proven by blob equal to
`HEAD` and an empty `git diff HEAD`:
- A, the `reportRefusedStoredShapes` call removed from
`applyConversions`: `refused-shapes.test.ts` 18 failed and 13 passed,
predicted red. The 13 are the not-listed and registry pins.
- B, `flowResult.todos ?? []` replaced by `[]` in `protocol.ts`: only
the #23269 preview pin went red (1 of 46).
- C, the engine's `todos.push` removed: only the braced-operand engine
pin went red (1 of 11).
## Acceptance notes
- **Real producers of the #23269 shape.** hotcrm at `0f84c9c` has 19
flow files (35 lines) whose `filter:` carries a dotted-path braced
token. Cloud's AI build wrote it while the skill taught `filter: { id:
'{record.id}' }`, per #23269.
- **Out of scope here:** a page component whose `data` (`object-grid`,
and the other component whose `data` is `ViewDataSchema`) binds the
retired API source is refused by the same schema but is not listed.
#23160's D3 entry scopes the retirement to views and object list views,
and this listing follows that scope. Carrier: none.
- **Out of scope here:** the other `registerFlow` refusals (text-slot
`{{ }}` holes, unknown config keys, decision modes) are still not
previewed. The general shape would be the stored pass running
`registerFlow`'s whole validation. Carrier: none.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01BRQ9Mp1yH7R1az6MaUM18H)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent cc14657 commit 572ecfb
18 files changed
Lines changed: 1097 additions & 61 deletions
File tree
- .changeset
- content/docs/deployment
- packages
- metadata-protocol/src
- services/service-automation/src
- spec/src
- contracts
- conversions
- data
- migrations/entries/semantic
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1473 | 1473 | | |
1474 | 1474 | | |
1475 | 1475 | | |
| 1476 | + | |
1476 | 1477 | | |
1477 | 1478 | | |
1478 | 1479 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| 29 | + | |
29 | 30 | | |
30 | 31 | | |
31 | 32 | | |
| |||
1083 | 1084 | | |
1084 | 1085 | | |
1085 | 1086 | | |
| 1087 | + | |
| 1088 | + | |
| 1089 | + | |
| 1090 | + | |
| 1091 | + | |
| 1092 | + | |
| 1093 | + | |
| 1094 | + | |
| 1095 | + | |
| 1096 | + | |
| 1097 | + | |
| 1098 | + | |
| 1099 | + | |
| 1100 | + | |
| 1101 | + | |
| 1102 | + | |
| 1103 | + | |
| 1104 | + | |
| 1105 | + | |
| 1106 | + | |
| 1107 | + | |
| 1108 | + | |
| 1109 | + | |
| 1110 | + | |
| 1111 | + | |
| 1112 | + | |
| 1113 | + | |
| 1114 | + | |
| 1115 | + | |
| 1116 | + | |
| 1117 | + | |
| 1118 | + | |
| 1119 | + | |
| 1120 | + | |
| 1121 | + | |
| 1122 | + | |
| 1123 | + | |
| 1124 | + | |
| 1125 | + | |
| 1126 | + | |
| 1127 | + | |
| 1128 | + | |
| 1129 | + | |
| 1130 | + | |
| 1131 | + | |
| 1132 | + | |
| 1133 | + | |
| 1134 | + | |
| 1135 | + | |
| 1136 | + | |
| 1137 | + | |
| 1138 | + | |
| 1139 | + | |
| 1140 | + | |
| 1141 | + | |
| 1142 | + | |
| 1143 | + | |
| 1144 | + | |
| 1145 | + | |
| 1146 | + | |
| 1147 | + | |
| 1148 | + | |
| 1149 | + | |
| 1150 | + | |
| 1151 | + | |
| 1152 | + | |
| 1153 | + | |
| 1154 | + | |
| 1155 | + | |
| 1156 | + | |
| 1157 | + | |
| 1158 | + | |
| 1159 | + | |
| 1160 | + | |
| 1161 | + | |
| 1162 | + | |
| 1163 | + | |
| 1164 | + | |
| 1165 | + | |
| 1166 | + | |
| 1167 | + | |
| 1168 | + | |
| 1169 | + | |
| 1170 | + | |
| 1171 | + | |
| 1172 | + | |
| 1173 | + | |
| 1174 | + | |
| 1175 | + | |
| 1176 | + | |
| 1177 | + | |
| 1178 | + | |
| 1179 | + | |
| 1180 | + | |
| 1181 | + | |
| 1182 | + | |
| 1183 | + | |
| 1184 | + | |
| 1185 | + | |
| 1186 | + | |
| 1187 | + | |
| 1188 | + | |
| 1189 | + | |
| 1190 | + | |
| 1191 | + | |
| 1192 | + | |
| 1193 | + | |
| 1194 | + | |
| 1195 | + | |
| 1196 | + | |
| 1197 | + | |
| 1198 | + | |
| 1199 | + | |
| 1200 | + | |
| 1201 | + | |
| 1202 | + | |
| 1203 | + | |
| 1204 | + | |
| 1205 | + | |
| 1206 | + | |
| 1207 | + | |
| 1208 | + | |
| 1209 | + | |
| 1210 | + | |
| 1211 | + | |
| 1212 | + | |
| 1213 | + | |
| 1214 | + | |
| 1215 | + | |
| 1216 | + | |
| 1217 | + | |
| 1218 | + | |
| 1219 | + | |
| 1220 | + | |
| 1221 | + | |
| 1222 | + | |
| 1223 | + | |
| 1224 | + | |
| 1225 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
22536 | 22536 | | |
22537 | 22537 | | |
22538 | 22538 | | |
| 22539 | + | |
| 22540 | + | |
| 22541 | + | |
| 22542 | + | |
| 22543 | + | |
| 22544 | + | |
22539 | 22545 | | |
22540 | 22546 | | |
22541 | 22547 | | |
| |||
22822 | 22828 | | |
22823 | 22829 | | |
22824 | 22830 | | |
| 22831 | + | |
| 22832 | + | |
| 22833 | + | |
| 22834 | + | |
| 22835 | + | |
| 22836 | + | |
22825 | 22837 | | |
22826 | | - | |
| 22838 | + | |
22827 | 22839 | | |
22828 | 22840 | | |
22829 | 22841 | | |
| |||
22856 | 22868 | | |
22857 | 22869 | | |
22858 | 22870 | | |
22859 | | - | |
22860 | | - | |
| 22871 | + | |
| 22872 | + | |
22861 | 22873 | | |
22862 | 22874 | | |
22863 | 22875 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
58 | 58 | | |
59 | 59 | | |
60 | 60 | | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
61 | 78 | | |
62 | 79 | | |
63 | 80 | | |
| |||
0 commit comments