fix(spec,service-automation): the v18 stored preview lists page components on the retired API source and every refusal registerFlow makes after the parse

.changeset/23317-stored-preview-gaps.md

@@ -0,0 +1,14 @@
+---
+'@objectstack/spec': minor
+'@objectstack/service-automation': minor
+---
+
+fix: `os migrate meta --stored` lists the two families of stored rows it still read as on protocol: a page component whose `data` prop binds the retired view-level API source, and every other site `registerFlow` refuses in a stored flow the parse accepts.
+
+Clause-②: yes (widening: `ConversionTodoNotice.conversionId` in `@objectstack/spec` may now name `registerFlow`, the refusing door; `IAutomationService.canonicalizeStoredFlow(...).todos?` in `@objectstack/spec` and `AutomationEngine.canonicalizeStoredFlow(...).todos` in `@objectstack/service-automation` now also carry each site flow registration refuses; the `onTodo` sink of `applyConversions` now also receives a page component bound to the retired API source; nothing narrows, and no door accepts or refuses anything it did not before)
+
+**A page component bound to the retired view-level API source: refuse and list, by page and component (`view-data-api-provider-retired`).** `object-grid`, `object-map`, `object-gantt` and `object-tree` take a view data source in their `data` prop, so the step-18 retirement of the `provider: 'api'` arm reaches them through the same schema. The listing judges each stored page component with its own props row, the one the component-props gate parses, and lists it only when that row refuses the binding with the retirement's prescription; a component whose `data` is a row array, a custom type and a component's own `dataSource` binding are not listed. The line names the page, the component and its position, shows the request URL and method, and withholds every other request key. Nothing is rewritten: deleting `data` would make the component query its object instead. Left as stored, the page still loads and saves, because the page parse does not judge a component's properties, while `os validate`, `os build` and `os lint` refuse the binding in source. The D3 entry `view-data-api-provider-retired` now names these components in its scope.
+
+**A stored flow refused by `registerFlow` after the parse: refuse and list, by site (`registerFlow`).** The stored pass ran only one of registration's judges, the value-slot one. `canonicalizeStoredFlow` now runs registration's whole validation without registering: every check `registerFlow` runs once the parse has accepted the flow, through the one list `registerFlow` itself throws from. Each refused site is a `TODO registerFlow` line under its row, in the check's own words: a text slot still on the single-brace dialect or with a hole that does not compile, a decision's `mode`, a malformed node or edge predicate, a plugin node's undeclared config key, an `api` flow with no secret. A check added to registration later is listed the same way, with no second change. A site the chain already lists under a step-18 entry is not repeated. What the parse itself refuses, the builtin config contracts' value arm among it, is not a TODO: `canonicalizeStoredFlow` throws it, and the preview reports the row among those that could not be rewritten, in the parse's words.
+
+`registerFlow` refuses exactly what it refused before, with the same message: its checks now return their refusal instead of throwing it, and `registerFlow` throws the first one. The doors that canonicalize a flow without registering it (the metadata save door, a package duplication) read no `todos`, and the stored pass's run of the checks logs nothing, so they behave as before.

content/docs/deployment/cli.mdx

@@ -1473,7 +1473,8 @@ can produce, because both supply a live one.
 | Rows that still fail the current schema after conversion | That is a genuine contract violation, not chain-owned history. The write path's rejection is correct; fix the row in Studio |
 | A flow whose rename the conflict guard refused | The old node-type token is a live name something else owns here. Rewriting would clobber that owner, so the row fails loudly naming the token — never a silent skip |
 | A site the conversion chain leaves as stored because no lossless rewrite exists — above all a page filter carrying `$and` / `$or` / `$not` | Flattening a combinator changes which rows the page selects, so it is never done. Each site is printed as a `TODO` line under its row — path, block, and what blocks the rewrite — whatever the row's outcome; a row with nothing but TODOs is reported `skipped`. It does not fail the run, since no run of this pass can clear it: rewrite each site by hand |
-| A stored shape protocol 18 refuses and no conversion rewrites — a `controlled_by_parent` object without exactly one required `master_detail` (two masters, none, a required `lookup` only, an explicit `required: false`), a view bound to the retired view-level API data source, a flow value slot carrying a `{…}` token (a `get_record` / `update_record` / `delete_record` filter operand among them) | Each would need a choice only its owner can make — which master, which page, what an absent value should do — so it is never rewritten. Each site is printed as a `TODO` line under its row, naming the step-18 entry that refuses it, the object, view or flow node, and the owner's choices; a view's request headers are withheld. Until it is rewritten by hand, the parse or the boot refuses the row: an object registers but derives no master, a view registers under `[metadata_spec_invalid]`, and a flow is not registered |
+| A stored shape protocol 18 refuses and no conversion rewrites — a `controlled_by_parent` object without exactly one required `master_detail` (two masters, none, a required `lookup` only, an explicit `required: false`), a view bound to the retired view-level API data source or a page component whose `data` prop binds it (`object-grid`, `object-map`, `object-gantt`, `object-tree`), a flow value slot carrying a `{…}` token (a `get_record` / `update_record` / `delete_record` filter operand among them) | Each would need a choice only its owner can make — which master, which page, what an absent value should do — so it is never rewritten. Each site is printed as a `TODO` line under its row, naming the step-18 entry that refuses it, the object, view, page component or flow node, and the owner's choices; a binding's request headers are withheld. Until it is rewritten by hand, the row is refused: an object registers but derives no master, a view registers under `[metadata_spec_invalid]`, a page still loads and saves while `os validate`, `os build` and `os lint` refuse the binding in source, and a flow is not registered |
+| Any other site `registerFlow` refuses in a stored flow the parse accepts — a text slot's template hole, a decision's `mode`, a malformed predicate, a plugin node's undeclared config key, an `api` flow with no secret | The flow's own definition is wrong, and no conversion rewrites it. Each site is printed as a `TODO registerFlow` line under its row, in the words `registerFlow` refuses it with; a site already listed under a step-18 entry is not repeated. Until it is fixed, the flow is not registered at boot and never runs. A flow the parse itself refuses is reported among the rows that could not be rewritten, in the parse's words |
 
 **One thing it lists and never writes: decision nodes that changed meaning at
 protocol 18.** A stored `decision` with no `config.conditions`, no `mode` and two

packages/metadata-protocol/src/protocol.stored-migration.test.ts

@@ -1184,6 +1184,37 @@ describe('migrateStoredMetadata — the v18 stored shapes refused with no lossle
         expect(text).not.toContain('s3cr3t');
     });
 
+    it('#23317 — a page whose component binds the retired API source in its data prop is listed by page and component, without its headers', async () => {
+        const block = (type: string, data: unknown) => ({ id: 'cases', type, properties: { objectName: 'crm_case', data } });
+        const page = (name: string, data: unknown) => ({
+            type: 'page',
+            name,
+            metadata: { name, label: name, type: 'app', regions: [{ name: 'main', components: [block('object-grid', data)] }] },
+        });
+        const { engine, tables } = makeStubEngine([
+            page('partner_page', { provider: API, read: { url: 'https://partner.example.com/cases', headers: { Authorization: 'Bearer s3cr3t' } } }),
+            page('case_page', { provider: 'object', object: 'crm_case' }),
+        ]);
+        const before = JSON.stringify(metaRows(tables));
+        const protocol = new ObjectStackProtocolImplementation(engine);
+
+        const report = await protocol.migrateStoredMetadata();
+
+        expect(report.canonical).toBe(1);
+        expect(report.rows.map((r) => [r.type, r.name, r.outcome, r.todos.map((t) => [t.conversionId, t.path])])).toEqual([
+            ['page', 'partner_page', 'skipped', [['view-data-api-provider-retired', 'pages[0].regions[0].components[0].properties.data']]],
+        ]);
+        const text = formatStoredMigrationReport(report).join('\n');
+        expect(text).toContain('page/partner_page [env-wide]');
+        expect(text).toContain('The `object-grid` component `cases` of page `partner_page`');
+        expect(text).not.toContain('s3cr3t');
+        // Listed, never written.
+        expect(storedMigrationClean(report)).toBe(true);
+        const applied = await protocol.migrateStoredMetadata({ apply: true });
+        expect(applied.rewritten).toBe(0);
+        expect(JSON.stringify(metaRows(tables))).toBe(before);
+    });
+
     it('#23269 — a flow whose CRUD filter operand is a {…} token is listed by flow and node; the CEL form is canonical', async () => {
         const { engine, tables } = makeStubEngine([
             crudFlow('fd_refused_filter', '{record.id}'),

packages/metadata-protocol/src/stored-migration.ts

@@ -61,8 +61,10 @@ export interface StoredFlowCanonicalization {
   /**
    * Sites the chain left as stored because no lossless rewrite exists (#23269)
    * — above all a value slot carrying a `{…}` template token, which
-   * `registerFlow` refuses although the parse accepts the flow. Listed under
-   * the row exactly as a non-flow row's TODOs are ({@link StoredMigrationTodo}).
+   * `registerFlow` refuses although the parse accepts the flow — and every
+   * other site `registerFlow` refuses once the parse has accepted it (#23317),
+   * each under `conversionId: 'registerFlow'`. Listed under the row exactly as
+   * a non-flow row's TODOs are ({@link StoredMigrationTodo}).
    * It restates the contract's own optional member,
    * `IAutomationService.canonicalizeStoredFlow(...).todos?` (`@objectstack/spec`):
    * a canonicalizer that reports none still conforms, and absent reads as none.
@@ -119,7 +121,11 @@ export interface StoredMigrationNotice {
  * this list the row would read as already on protocol.
  */
 export interface StoredMigrationTodo {
-  /** The `MetadataConversion.id` whose surface the site is on. */
+  /**
+   * The `MetadataConversion.id` whose surface the site is on — or the D3 entry
+   * that refuses a stored shape no conversion covers, or `registerFlow` for a
+   * site flow registration refuses.
+   */
   conversionId: string;
   /** Dotted surface that conversion governs. */
   surface: string;

packages/runtime/src/dispatcher-error-vocabulary.ts

@@ -617,6 +617,20 @@ export const UNREGISTERED_CODE_SITES: readonly UnregisteredCodeSite[] = [
             'pass flattens it onto its report rows, where `os migrate meta --stored` prints it; it ' +
             'carries no `code` there. A callback payload, not a thrown error and not a response body.',
     },
+    {
+        code: 'OS_METADATA_CONVERSION_TODO',
+        file: 'packages/services/service-automation/src/engine.ts',
+        shape: 'objlitconst',
+        door: 'none',
+        verdict: 'foreign-vocabulary',
+        why:
+            'The same conversion-TODO vocabulary as the apply.ts row above, stamped by the automation ' +
+            'engine for a site its flow registration refuses once the parse has accepted the flow: ' +
+            "`canonicalizeStoredFlow` returns it in `todos` beside the chain's own TODOs, and the " +
+            'stored-metadata migration pass flattens it onto its report rows exactly as it does those, ' +
+            'with no `code`. A returned record, not a thrown error and not a response body: ' +
+            '`registerFlow` throws its own plain refusal and never this.',
+    },
     {
         code: 'ERR_BULK_PER_ROW_HOOK_LIMIT',
         file: 'packages/spec/src/data/bulk-write-hook-conformance.ts',

packages/services/service-automation/src/builtin/config-unknown-keys.test.ts

@@ -159,7 +159,9 @@ describe('a builtin\'s undeclared key is the spec\'s, met at registration (#4277
 
   it('the descriptor walk stands aside for every type the spec judges, and keeps the rest', () => {
     const engine = engineWith(recordingLogger().logger);
-    const walk = (type: string, config: Record<string, unknown>) => () =>
+    // [#23317] The walk returns its refusal (`registerFlow` throws it), so the
+    // stored pass can list the same sites without registering.
+    const walk = (type: string, config: Record<string, unknown>): { message: string } | undefined =>
       (engine as any).validateNodeConfigKeys('f', flowWith(type, config));
     // One judge per type: the spec already refused these in `FlowSchema.parse`.
     for (const [type, config] of [
@@ -169,11 +171,11 @@ describe('a builtin\'s undeclared key is the spec\'s, met at registration (#4277
       ['try_catch', { try: { nodes: [], edges: [] }, bogusKey: 1, retry: { maxRetry: 2 } }],
     ] as const) {
       expect(builtinNodeConfigKeysJudged(type), type).toBe(true);
-      expect(walk(type, config), type).not.toThrow();
+      expect(walk(type, config), type).toBeUndefined();
     }
     // The walk keeps plugin types.
     expect(builtinNodeConfigKeysJudged(STAMP)).toBe(false);
-    expect(walk(STAMP, { count: 1, bogusKey: 1 })).toThrow(/undeclared config key/);
+    expect(walk(STAMP, { count: 1, bogusKey: 1 })?.message).toMatch(/undeclared config key/);
   });
 });