Skip to content

Commit ef7067f

Browse files
fix(core)!: the import dry run previews create rows in the commit's batches (#26684) (#26778)
<!-- claude-projects-attribution: {"github_login":"os-project-manager"} --> _Requested by **Leehom** · [project thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1XMNo82yTJxw28uqj2PYAVvH)_ Fixes #26684 Clause-②: no (narrowing) Metadata-protocol: none Skills-docs: none — no `skills/**` text describes the import dry run or the insert preview (`git grep -i -e 'dry run' -e dryRun skills/` hits only the i18n CLI `--dry-run` flag and `os migrate plan`) ## 维护者速读 - **改了什么**:导入的试运行(dryRun)不再逐行单独问引擎,而是把新建行按正式导入写入时的同一批次(每批 200 行)一次性交给引擎预览;引擎的批量预览也在规则之后跑一遍写入时的引用检查,通过才计入后面行看到的父汇总。 - **为什么改**:#25579 之后正式导入会把同批前面已通过的行计入父汇总("一个供应商只能有一个主联系人"),试运行却仍逐行判断,两行主联系人都说"有效",正式导入却拒掉第二行,操作员被告知文件没问题。 - **风险与回滚**:试运行现在会多报出正式导入本来就会拒的行(收窄,BREAKING 标注);单行的插入预览会报出悬空引用。回滚即 revert 本 PR。已知未覆盖:试运行不写库,跨 200 行批次边界的两行冲突仍都显示有效(已写进文档,待你决定是否要补)。 - **先看**:`packages/core/src/utils/import-runner.ts`(`previewCreateRun` / `flushPendingPreviews`);`packages/objectql/src/engine.ts` 的 `validate()` 新增引用检查那一段;`packages/rest/src/import-dry-run-batch-parent-summary.test.ts`。 - **两分钟验证**:`pnpm --filter @objectstack/rest exec vitest run src/import-dry-run-batch-parent-summary.test.ts`,预期 9 个用例全绿,其中"两行主联系人"一例里试运行第二行失败、信息与正式导入相同。 - **需要你做的**:无,席位落地;跨批次边界那一条另作决策。 ## What changes **`@objectstack/core` — `packages/core/src/utils/import-runner.ts`.** A dry run's CREATE rows are no longer asked about one at a time. They are buffered as the commit buffers its creates (`pendingPreviews`, beside `pendingCreates`) and flushed at the commit's checkpoints (every `progressEvery` rows, and the end) by `flushPendingPreviews`, which splits them into the commit's runs (`Math.min(progressEvery, MAX_CREATE_BATCH_SIZE)`, the `bulkWrite` batch size) and asks the engine about each run in ONE `validateData` call with `data` as an array, in file order (`previewCreateRun`). The engine's multi-row `validate()` (#26680) does the accumulation: ⛔ the runner neither reads nor judges a rule. Each row still gets its own report, built by the same code as before (`reportPreview`, lifted out of the row loop unchanged): same `row`, `ok`, `action`, `field`, `code`, `error`, `warnings`, `droppedFields`, at the row's own position. The run's call carries the union of its rows' kept option values, as the commit's batch call does (`writeCtxFor`). UPDATE rows keep their per-row preview (`previewVerdict`, one row object, `mode: 'update'`), unchanged. A run whose call throws, or answers other than one verdict per row, is degraded as the commit degrades a failed batch: each row is asked alone (so a row the engine refuses outright, such as an undeclared column, carries its own refusal, as `insertManyData` culls it per row), then the rows that survived alone are asked together again so they still see each other. **`@objectstack/objectql` — `ObjectQL.validate()` (the seat's addendum 6442401477).** The insert preview's admit step now matches `insert()`'s: after the rules, the write's reference check (`assertReferencesResolve`, the same method, with the caller's pre-default row `rawRows[i]` as `supplied`, the write's `suppliedPerRow`). A row it refuses is answered `valid: false` with the write's own `reference_not_found` finding, and is not counted toward a later row's parent summary. ⛔ It runs only behind the same write-gate probe (`mayWrite`) as the preview's related read: the probe on a `lookup` target is elevated, so a caller the probe refuses learns nothing about which ids exist. The rows are now judged in a sequential loop (`judgeRow`) instead of `rows.map`, because the admit step is awaited before the next row reads what the earlier rows added. Update mode is untouched. **Docs.** `content/docs/data-modeling/fields.mdx` (the 批量等于逐条 paragraph #26680 added): the dry run previews create rows in the import's batches, and the one case it cannot see (rows in different batches, below). ## Why #25579 made a bulk create count the earlier admitted rows of the same batch when a child rule reads a parent `summary`. The import commit does; the dry run asked the engine about each row alone, so on #25579's reproduction (two primary contacts for one new supplier) it answered both rows valid while the commit created one and refused the other. The engine's multi-row preview already accumulates (#26680); the dry run just never handed it more than one row. ## Tests - **`packages/rest/src/import-dry-run-batch-parent-summary.test.ts`** (new, 9 cases) — through the real `POST /api/v1/data/:object/import`, a real ObjectQL engine and a sqlite `:memory:` driver, #25579's schema. Each agreement case runs the same file twice on two fresh stacks, `dryRun: true` and committed, and pins the per-row verdicts equal (`row`, `ok`, `action`, `error`) plus `created` / `updated` / `errors`: - two primary contacts for a new supplier: the dry run refuses row 2 with the commit's message, `A supplier has at most one primary contact.`; - the dry run writes nothing (no contact, summary still `0`); - two non-primary rows: both valid in the dry run, as both land; - a row the rules refuse is not counted (placeholder name, then a primary): both halves admit the second; - a file larger than one batch (203 rows): two primary rows inside the second batch, the dry run refuses row 202 as the commit does; - an update-mode dry run keeps its per-row preview (an update against a stored primary is refused, a rename is not) and agrees with its commit; - the seat's addendum: a multi-row insert preview refuses the dangling-reference row with `region` / `reference_not_found` and admits the next primary row, as `insertMany` does; a single-row insert preview refuses a dangling reference as `insert()` does; a caller the write-gate probe refuses gets no `reference_not_found` (no existence oracle). - **`packages/core/src/utils/import-runner-dry-run-batch.test.ts`** (new, 6 cases) — against a protocol double that answers one verdict per submitted row: the create rows of a small file are one insert-mode call in file order with each row's own report; 450 rows are previewed as 200 / 200 / 50; a caller `progressEvery` of 3 sets the checkpoints and run length (`[r1..r3] [r4..r6] [r7]`, progress `3, 6, 7`); an upsert's matched row is asked alone in `update` mode with its id while the creates around it keep one batch; a batch the engine refuses outright is asked row by row, then the survivors together again; no `validateData` falls back to the coercion verdict. - **Updated, stale doubles** — four core tests modelled `validateData` as answering one verdict for any call, so a batch of rows read as one row: `import-runner-dropped-fields.test.ts` (two cases) and `import-runner-kept-options.test.ts` now answer one verdict per submitted row (the `ValidateDataResponse` contract, "one entry per submitted row"); the kept-options dry-run case now pins the ONE batch call and its union context, the context the commit's batch write carries (plus a case: a batch that kept nothing carries the import's context unchanged); `import-runner-update-preview-address.test.ts`'s create control now reads `data` as a one-row array. - **Updated, `packages/objectql/src/validate-preview-parent.test.ts` (c)** — "insert and update previews of a plain line issue no read of the header object" split in two: the update preview still issues none; the insert preview now issues exactly the reads the insert itself makes (its reference check asks the header's read door whether `h_sent` exists), and the #22474 header binding adds none. <details><summary>Red first on main (production sources restored to 1685a06, core and objectql rebuilt)</summary> Restore proved by `git hash-object` equal to the base blobs (`import-runner.ts` 4983d0e, `engine.ts` 1f7cd15714), then `turbo run build --filter=@objectstack/objectql --filter=@objectstack/core` exit 0. - rest `import-dry-run-batch-parent-summary.test.ts`: `Tests 4 failed | 5 passed (9)` — red: the two-primary case, the second-batch case, both dangling-reference cases. The two-primary case's diff: row 2 expected `action: "failed"`, `error: "A supplier has at most one primary contact."`, received `ok: true`, `action: "created"`. - core `import-runner-dry-run-batch.test.ts`: `Tests 5 failed | 1 passed (6)` (only the no-`validateData` fallback is green on `main`). - Restore leg: `git checkout HEAD --`, `git diff --quiet HEAD` clean, both packages rebuilt; the same files then `Tests 9 passed (9)` and `Tests 6 passed (6)`. </details> <details><summary>Ablations (both through scripts/ablation-replace.mjs, package rebuilt, ablation-dist-preflight proving the marker in dist/, restore proved by blob == HEAD)</summary> 1. **Send the create rows one by one again** — `import-runner.ts` `const runLength = Math.min(progressEvery, MAX_CREATE_BATCH_SIZE);` → `const runLength = 1;`. Predicted: the two-primary and second-batch agreement cases red, everything else in the rest file green; the core batching cases red. Observed: rest `Tests 2 failed | 7 passed (9)` (exactly those two), core `Tests 5 failed | 1 passed (6)`. A first run of this ablation read a stale objectql `dist/` left by the red-first leg (the two reference cases were also red); that reading is void and is not counted: objectql was rebuilt at HEAD and the ablation re-run. 2. **Drop the preview's reference check** — `engine.ts` the `await this.assertReferencesResolve(schemaForValidation, row, rawRows[i], options?.context, messages);` line deleted. Predicted: the two dangling-reference cases red. Observed: rest `Tests 2 failed | 7 passed (9)`, exactly those two. After both: tree == HEAD, core and objectql rebuilt, preflight `--absent` / present confirms the HEAD build, rest `9 passed (9)`, core `6 passed (6)`. </details> ## Acceptance notes - **Not covered: two rows that conflict across a batch boundary.** Measured on this head with a throwaway probe (not committed): 199 non-primary rows, then primary rows at 200 and 201. The commit refuses row 201 (batch 1's recompute writes the stored count before batch 2 reads it); the dry run answers both valid, because a dry run writes nothing between batches and batch 2's preview reads the stored `0`. Inside one batch (the card's reproduction, and every file of up to 200 create rows) the two agree. The docs paragraph and the changeset's "Not covered" bullet say so. Closing it needs one of: one `validate()` call over all of a file's create rows (up to 5,000 sync / 50,000 job rows in one engine call), or an accumulation carried across preview calls (a new `validateData` request key in `packages/spec`, outside this lane). Left to the seat as an open question in the os-dev-report. - **Update rows keep their per-row preview** (the ruling). Two update rows that each move a parent summary still preview against the stored value; their commit writes them one by one, so the second can be refused. Unchanged by this PR. - **The reference check in the preview is insert-only.** `update()` also runs it on the write; the update-mode preview still does not. Out of the addendum's scope (the admit step exists only on insert). - **The insert preview now reads one more thing**: the reference check's probe of each id the caller supplied, behind the write-gate probe. A caller the probe refuses gets no such read and no `reference_not_found`. - **Degraded run**: when a run's call throws, each row is asked alone and the survivors together again. This mirrors the commit's `insertManyData`, which culls a refused row and accumulates the rest; it costs extra calls only on a failing run. - `packages/spec` untouched (lane red line). The import door's `dryRun` description in `export.zod.ts` ("The verdict is the engine's own write-path validation") stays true and is not edited. - Open PRs touching these files: #26337 and #26755 (`engine.ts`), #26598 (`fields.mdx`); `git merge-tree` of each against this head is clean. ## Verification All on head `80b0a99e`, in this card thread's container. - **Package suites** (`pnpm --filter PKG test`, the `local` project): `@objectstack/core` 2803 passed (2803); `@objectstack/objectql` 8807 passed (8807); `@objectstack/rest` exit 0. **`repo` projects** (`vitest run --project repo`): core 83 passed, objectql 5 passed, rest 280 passed / 18 skipped. **Type check** (`pnpm --filter PKG typecheck`): core, objectql, rest exit 0. - **Downstream readers of the dry run and of `validate()`**: plugin-auth `admin-import-users*.test.ts` 62 passed; runtime `external-validation-*.test.ts` 25 passed; `@objectstack/service-datasource` test 975 passed; service-automation `connector-pull` 22 passed. - **Derived gates**: `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` on this diff: 96 families, every one run with its exit code captured before any pipe, all exit 0; `--ran` reconciliation: `96 derived famil(ies) accounted for — 96 run, 0 NOT-MEASURED`. Three first answered exit 3 (prerequisite not met: the teardown self-test's positive control commit not fetched, no full `dist/` for `check:skill-examples` and `check:dual-build-cjs-loads`) and were re-run green after `git fetch origin 621a487` and the full build `check:type-check-debt` makes. - **Scoped lint**: `pnpm exec eslint --no-inline-config` on the 8 touched source and test files: exit 0. <details><summary>Dogfood battery (narrowing arm)</summary> On `80b0a99e`, `pnpm --filter @objectstack/dogfood test` under the verify lock, no shard, after `turbo run build --filter='@objectstack/dogfood^...'`: `Test Files 302 passed | 3 skipped (305)`, `Tests 2505 passed | 20 skipped (2525)`, `os-verify-lock: VERDICT command-exit 0`, held 799s. No failures; the skips are declared skips. </details> ## Human Minimum - [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic. (No `packages/spec` change.) - [x] 2. Every name I added uses the casing for its kind. - [x] 3. No formatter ran; `pnpm lint` passes (scoped to the touched files locally; the repo-wide run is CI's). - [x] 4. One issue, one branch off an up-to-date `main`, one pull request. - [x] 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset. - [x] 6. `packages/spec` touched: not touched. - [x] 7. No hand edits to `content/docs/references/`, `packages/*/CHANGELOG.md` or `content/docs/releases/`. - [x] 8. Title, body and comments are in English (the 维护者速读 block is the lane's maintainer summary). - [x] The *Before Submitting* list in CONTRIBUTING.md is done. --- _Generated by [Claude Code](https://claude.ai/code/session_01FEkQ5krDpBpMjEHKqjJ6u8)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 665059b commit ef7067f

10 files changed

Lines changed: 647 additions & 46 deletions
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
'@objectstack/core': patch
3+
'@objectstack/objectql': patch
4+
---
5+
6+
fix(core)!: the import dry run previews create rows in the commit's batches, so a batch-aware parent-summary rule refuses in the dry run the rows the commit refuses (#26684)
7+
8+
Clause-②: no (narrowing) — an import dry run (`POST /api/v1/data/:object/import` and the async import job with `dryRun: true`) answers invalid for a create row that a child rule reading a parent `summary` refuses once the earlier admitted rows of its batch are counted, which it answered valid before; and the insert preview (`validate` / `validateData`, mode `insert`) refuses a row whose caller-supplied reference names no row, which it answered valid before. Nothing is widened.
9+
Metadata-protocol: none
10+
11+
<!-- adr-0087: not-required (no-migration-prescription) No key of any schema is added, removed, renamed or re-shaped, and no export or type changes: the dry run now reports the verdicts the import's own write already gives on the same file. The refusals are the rule's own VALIDATION_FAILED, worded by its author, and the write's reference_not_found, so there is nothing for a migration to rewrite and no tombstone. -->
12+
13+
**BREAKING** accept-set narrowing at the import dry run (`@objectstack/core`) and the insert preview (`@objectstack/objectql`).
14+
15+
- **Before.** The dry run asked the engine about each create row alone, so after #25579 it answered both rows valid when a file held two primary contacts for one new supplier, while the import created one and refused the other. The insert preview judged a row's rules but not the write's reference check, so a row with a dangling reference previewed valid and was counted toward a later row's parent summary.
16+
- **Now.** The dry run buffers its create rows as the import does and previews them through one multi-row `validate()` per import batch (200 rows, or a smaller `progressEvery`), in file order, so the engine counts the earlier rows of the batch exactly as on the write. Each row keeps its own verdict, message and position in the report. The insert preview runs the write's reference check after the rules, behind the same write-gate probe as its related reads, and counts a row only after it passes.
17+
- **Unchanged.** Update rows are previewed one at a time, as before. A dry run writes nothing.
18+
- **Not covered.** A dry run writes nothing between batches, so two rows that only conflict across a batch boundary both preview valid while the import refuses the second.
19+
- **Who sees it.** An operator dry-running an import into an object whose child rule reads a parent summary: rows the import would refuse are now reported in the dry run.
20+
21+
**The remedy.** None needed: fix the rows the dry run now reports, as the import would have required.

‎content/docs/data-modeling/fields.mdx‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -322,6 +322,11 @@ refuse is not counted. This holds for `count`, and for `sum`, `min` and `max`
322322
over number values. An `avg` summary, a `sum`/`min`/`max` meeting a value that
323323
is not a number, and a summary whose `filter` reads a dotted path through
324324
another reference keep reading the stored value, as they did before.
325+
An import's **dry run** previews its create rows in the batches the import
326+
writes them in (200 rows each) and refuses the rows the import refuses, with
327+
the same message. It writes nothing, so a row in a later batch reads the
328+
summary as stored before the import: two such rows that fall in different
329+
batches of a dry run both preview valid, while the import refuses the second.
325330
**Concurrent writes are not guaranteed (并发不保证):** two writes in flight at
326331
the same time each read the summary they started from, so a guard like the one
327332
above can still admit one row from each.

‎packages/core/src/utils/import-runner-dropped-fields.test.ts‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,10 @@ const withDrops = (row: Record<string, unknown>) => {
7171
return d.length > 0 ? { droppedFields: d } : {};
7272
};
7373

74+
/** The submitted rows of a `validateData` call: one verdict is owed per row (#26684 previews a batch). */
75+
const rowsOf = (args: ValidateArgs) =>
76+
(Array.isArray(args.data) ? args.data : [args.data]) as Array<Record<string, unknown>>;
77+
7478
/** The row's key set — a clean row must not grow an empty `droppedFields`. */
7579
const keysOf = (r: ImportRowResult | undefined) => Object.keys(r ?? {}).sort();
7680

@@ -85,7 +89,7 @@ describe('[#20701] the dry run copies validateData\'s per-row drops onto the row
8589
it('a create-resolved row carries the verdict\'s droppedFields verbatim; a clean row carries no key', async () => {
8690
const validateData = vi.fn(async (args: ValidateArgs) => ({
8791
object: OBJECT, mode: args.mode ?? 'insert', valid: true,
88-
results: [{ valid: true, errors: [], warnings: [], ...withDrops(args.data as Record<string, unknown>) }],
92+
results: rowsOf(args).map((row) => ({ valid: true, errors: [], warnings: [], ...withDrops(row) })),
8993
posture: { valueShapeStrict: false, mediaValueShapeStrict: false },
9094
}));
9195
const p: ImportProtocolLike = { findData: vi.fn(async () => []), createData: vi.fn(), updateData: vi.fn(), validateData };
@@ -116,12 +120,12 @@ describe('[#20701] the dry run copies validateData\'s per-row drops onto the row
116120
it('a row the verdict refuses carries no droppedFields, and an admitted warning rides beside the drops', async () => {
117121
const warning = { field: 'name', code: 'invalid_format', message: 'admitted' };
118122
const validateData = vi.fn(async (args: ValidateArgs) => {
119-
const refused = (args.data as Record<string, unknown>).name === 'bad';
123+
const results = rowsOf(args).map((row) => (row.name === 'bad'
124+
? { valid: false, errors: [{ field: 'name', code: 'required', message: 'name is required' }], warnings: [], droppedFields: [COMPUTED] }
125+
: { valid: true, errors: [], warnings: [warning], droppedFields: [COMPUTED] }));
120126
return {
121-
object: OBJECT, mode: 'insert' as const, valid: !refused,
122-
results: [refused
123-
? { valid: false, errors: [{ field: 'name', code: 'required', message: 'name is required' }], warnings: [], droppedFields: [COMPUTED] }
124-
: { valid: true, errors: [], warnings: [warning], droppedFields: [COMPUTED] }],
127+
object: OBJECT, mode: 'insert' as const, valid: results.every((r) => r.valid),
128+
results,
125129
posture: { valueShapeStrict: false, mediaValueShapeStrict: false },
126130
};
127131
});
Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* [#26684] How the import dry run asks the engine for its verdicts: a create
5+
* row is previewed with the rows of the batch the commit would write it in,
6+
* in ONE `validateData` call, and an update row alone, as before.
7+
*
8+
* Driven against an `ImportProtocolLike` double that answers one verdict per
9+
* submitted row (the `ValidateDataResponse` contract) and records each call.
10+
* What the batch buys — a child rule reading a parent roll-up counting the
11+
* earlier rows — is pinned end to end over a real engine in
12+
* `packages/rest/src/import-dry-run-batch-parent-summary.test.ts`.
13+
*
14+
* Pinned here:
15+
* - the batches are the commit's: file order, the commit's checkpoints
16+
* (every `progressEvery` rows) and run length
17+
* (`Math.min(progressEvery, MAX_CREATE_BATCH_SIZE)`, 200);
18+
* - each row keeps its own report, at its own position, between update rows;
19+
* - an update row is asked alone, in `update` mode, as before;
20+
* - a batch the engine refuses outright is degraded as the commit degrades
21+
* one: each row alone, then the survivors together again.
22+
*/
23+
24+
import { describe, it, expect, vi } from 'vitest';
25+
import { runImport, type ImportProtocolLike } from './import-runner';
26+
import type { ExportFieldMeta } from './import-field-meta.js';
27+
28+
type ValidateArgs = Parameters<NonNullable<ImportProtocolLike['validateData']>>[0];
29+
30+
const OBJECT = 'contact';
31+
32+
const baseOpts = {
33+
objectName: OBJECT,
34+
metaMap: new Map<string, ExportFieldMeta>([['name', { name: 'name', type: 'text' }]]),
35+
writeMode: 'insert' as const,
36+
matchFields: [] as string[],
37+
dryRun: true,
38+
runAutomations: false,
39+
trimWhitespace: true,
40+
createMissingOptions: false,
41+
skipBlankMatchKey: false,
42+
};
43+
44+
const POSTURE = { valueShapeStrict: false, mediaValueShapeStrict: false };
45+
46+
/**
47+
* One verdict per submitted row; a row named `bad*` is refused. A batch that
48+
* holds a row named `boom` is refused outright, as the engine throws on an
49+
* undeclared field.
50+
*/
51+
function validatingProtocol(findData: ImportProtocolLike['findData'] = vi.fn(async () => [])) {
52+
const validateData = vi.fn(async (args: ValidateArgs) => {
53+
const rows = (Array.isArray(args.data) ? args.data : [args.data]) as Array<Record<string, unknown>>;
54+
if (rows.some((r) => r.name === 'boom')) throw Object.assign(new Error('boom is not a field'), { code: 'INVALID_FIELD' });
55+
const results = rows.map((r) => String(r.name).startsWith('bad')
56+
? { valid: false, errors: [{ field: 'name', code: 'rule_violation', message: `refused ${String(r.name)}` }], warnings: [] }
57+
: { valid: true, errors: [], warnings: [] });
58+
return { object: OBJECT, mode: args.mode ?? 'insert', valid: results.every((r) => r.valid), results, posture: POSTURE };
59+
});
60+
const p: ImportProtocolLike = { findData, createData: vi.fn(), updateData: vi.fn(), validateData };
61+
return { p, validateData };
62+
}
63+
64+
const names = (call: [ValidateArgs]) => {
65+
const data = call[0].data;
66+
return Array.isArray(data) ? data.map((r) => (r as Record<string, unknown>).name) : (data as Record<string, unknown>).name;
67+
};
68+
69+
describe('[#26684] the dry run previews create rows in the commit\'s batches', () => {
70+
it('the create rows of a small file are ONE insert-mode call, in file order, each row keeping its own verdict', async () => {
71+
const { p, validateData } = validatingProtocol();
72+
const s = await runImport({ ...baseOpts, p, rows: [{ name: 'a' }, { name: 'bad-b' }, { name: 'c' }] });
73+
74+
expect(validateData).toHaveBeenCalledTimes(1);
75+
expect(validateData.mock.calls[0]![0]).toMatchObject({ object: OBJECT, mode: 'insert' });
76+
expect(names(validateData.mock.calls[0] as [ValidateArgs])).toEqual(['a', 'bad-b', 'c']);
77+
expect(s).toMatchObject({ ok: 2, errors: 1, created: 2 });
78+
expect(s.results).toEqual([
79+
{ row: 1, ok: true, action: 'created' },
80+
{ row: 2, ok: false, action: 'failed', field: 'name', code: 'rule_violation', error: 'refused bad-b' },
81+
{ row: 3, ok: true, action: 'created' },
82+
]);
83+
});
84+
85+
it('a file of 450 rows is previewed in the commit\'s runs: 200, 200, 50', async () => {
86+
const { p, validateData } = validatingProtocol();
87+
const rows = Array.from({ length: 450 }, (_, n) => ({ name: `r${n + 1}` }));
88+
const s = await runImport({ ...baseOpts, p, rows });
89+
90+
expect(validateData.mock.calls.map((c) => (c[0].data as unknown[]).length)).toEqual([200, 200, 50]);
91+
expect(names(validateData.mock.calls[1] as [ValidateArgs])).toEqual(rows.slice(200, 400).map((r) => r.name));
92+
expect(s).toMatchObject({ ok: 450, created: 450 });
93+
expect(s.results.map((r) => r.row)).toEqual(rows.map((_, n) => n + 1));
94+
});
95+
96+
it('a caller progressEvery sets the checkpoints, and the run length follows it, as on the commit', async () => {
97+
const { p, validateData } = validatingProtocol();
98+
const rows = Array.from({ length: 7 }, (_, n) => ({ name: `r${n + 1}` }));
99+
const progress: number[] = [];
100+
await runImport({ ...baseOpts, p, rows, progressEvery: 3, onProgress: (pr) => { progress.push(pr.created); } });
101+
102+
expect(validateData.mock.calls.map((c) => names(c as [ValidateArgs]))).toEqual([
103+
['r1', 'r2', 'r3'], ['r4', 'r5', 'r6'], ['r7'],
104+
]);
105+
// Each checkpoint reports the rows previewed so far, as the commit reports the rows written.
106+
expect(progress).toEqual([3, 6, 7]);
107+
});
108+
109+
it('an update row is asked alone, in update mode, and the create rows around it keep their batch', async () => {
110+
const findData = vi.fn(async (args: any) => {
111+
const where = JSON.stringify(args ?? {});
112+
return where.includes('"u1"') ? [{ id: 'e1', name: 'u1' }] : [];
113+
});
114+
const { p, validateData } = validatingProtocol(findData as any);
115+
const s = await runImport({
116+
...baseOpts, p, writeMode: 'upsert', matchFields: ['name'],
117+
rows: [{ name: 'a' }, { name: 'u1' }, { name: 'bad-c' }],
118+
});
119+
120+
const update = validateData.mock.calls.find((c) => c[0].mode === 'update')!;
121+
expect(update[0].data).toEqual({ name: 'u1', id: 'e1' });
122+
const insert = validateData.mock.calls.filter((c) => c[0].mode !== 'update');
123+
expect(insert.map((c) => names(c as [ValidateArgs]))).toEqual([['a', 'bad-c']]);
124+
expect(s.results).toEqual([
125+
{ row: 1, ok: true, action: 'created' },
126+
{ row: 2, ok: true, action: 'updated', id: 'e1' },
127+
{ row: 3, ok: false, action: 'failed', field: 'name', code: 'rule_violation', error: 'refused bad-c' },
128+
]);
129+
});
130+
131+
it('a batch the engine refuses outright is asked row by row, then the survivors together again', async () => {
132+
const { p, validateData } = validatingProtocol();
133+
const s = await runImport({ ...baseOpts, p, rows: [{ name: 'a' }, { name: 'boom' }, { name: 'c' }] });
134+
135+
expect(validateData.mock.calls.map((c) => names(c as [ValidateArgs]))).toEqual([
136+
['a', 'boom', 'c'], ['a'], ['boom'], ['c'], ['a', 'c'],
137+
]);
138+
expect(s.results[0]).toEqual({ row: 1, ok: true, action: 'created' });
139+
expect(s.results[1]).toMatchObject({ row: 2, ok: false, action: 'failed', code: 'INVALID_FIELD' });
140+
expect(s.results[2]).toEqual({ row: 3, ok: true, action: 'created' });
141+
});
142+
143+
it('no validate-only operation: every create row is reported on its coercion verdict, as before', async () => {
144+
const p: ImportProtocolLike = { findData: vi.fn(async () => []), createData: vi.fn(), updateData: vi.fn() };
145+
const s = await runImport({ ...baseOpts, p, rows: [{ name: 'a' }, { name: 'b' }] });
146+
expect(s).toMatchObject({ ok: 2, created: 2, errors: 0 });
147+
expect(p.createData).not.toHaveBeenCalled();
148+
});
149+
});

‎packages/core/src/utils/import-runner-kept-options.test.ts‎

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -77,8 +77,9 @@ function makeProvider(shape: 'inline' | 'createMany' | 'insertMany' | 'update' |
7777
}
7878
if (shape === 'validate') {
7979
p.validateData = vi.fn(async (args: ValidateArgs) => {
80-
calls.push({ kind: 'validateData', rows: [args.data as Record<string, unknown>], context: args.context });
81-
return { results: [{ valid: true, errors: [] }] } as any;
80+
const rows = (Array.isArray(args.data) ? args.data : [args.data]) as Array<Record<string, unknown>>;
81+
calls.push({ kind: 'validateData', rows, context: args.context });
82+
return { results: rows.map(() => ({ valid: true, errors: [] })) } as any;
8283
});
8384
}
8485
return { p, calls };
@@ -135,14 +136,22 @@ describe('runImport — createMissingOptions hands the kept values to the write
135136
expect(calls[0].context.keptOptionValues).toEqual({ priority: ['Bogus'] });
136137
});
137138

138-
it('the dry run asks validateData with the same per-row context the write would carry', async () => {
139+
it('the dry run asks validateData with the context the batch write would carry: the union of its rows\' kept values', async () => {
140+
// [#26684] The dry run previews the create rows as the batch the commit
141+
// writes them in, so its one call carries that batch call's context.
139142
const { p, calls } = makeProvider('validate');
140143
const summary = await runImport({ ...baseOpts, p, dryRun: true, rows: ROWS });
141144
expect(summary.created).toBe(3);
142-
expect(calls.map((c) => c.kind)).toEqual(['validateData', 'validateData', 'validateData']);
143-
expect(calls[0].context.keptOptionValues).toEqual({ priority: ['Bogus'] });
144-
expect(calls[1].context).not.toHaveProperty('keptOptionValues');
145-
expect(calls[2].context.keptOptionValues).toEqual({ tags: ['Other'] });
145+
expect(calls.map((c) => c.kind)).toEqual(['validateData']);
146+
expect(calls[0].rows).toHaveLength(3);
147+
expect(calls[0].context.keptOptionValues).toEqual({ priority: ['Bogus'], tags: ['Other'] });
148+
});
149+
150+
it('a dry run whose rows kept nothing carries the import\'s write context unchanged', async () => {
151+
const { p, calls } = makeProvider('validate');
152+
await runImport({ ...baseOpts, p, dryRun: true, rows: [{ code: 'b', priority: 'High' }] });
153+
expect(calls).toHaveLength(1);
154+
expect(calls[0].context).not.toHaveProperty('keptOptionValues');
146155
});
147156

148157
it('with the option off, an unmatched cell fails at coercion and no write carries the key', async () => {

‎packages/core/src/utils/import-runner-update-preview-address.test.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,7 @@ describe('[#22445] the dry run of a matched row names the stored row it would up
7878
await runImport({ ...baseOpts, p, writeMode: 'upsert', dryRun: true, rows: [{ code: 'c8', title: 'fresh' }] });
7979

8080
expect(validateData.mock.calls[0]![0]).toMatchObject({ mode: 'insert' });
81-
expect(validateData.mock.calls[0]![0].data).toEqual({ code: 'c8', title: 'fresh' });
81+
// [#26684] A create row is previewed with the rows of its batch: an array.
82+
expect(validateData.mock.calls[0]![0].data).toEqual([{ code: 'c8', title: 'fresh' }]);
8283
});
8384
});

0 commit comments

Comments
 (0)