fix(objectql): resync the engine fallback autonumber counter — adopt exempt writers' numbers, re-seed on collision (#6806) - #6907
Merged
Conversation
…ion re-seed) (#6806) The engine fallback autonumber path seeded `object.field.<scope>` once and then incremented purely in memory, so it drifted below the store's real max in two ways it could never recover from. - Adopt an exempt writer's supplied record number into the counter (isSystem seed replay / preserveAudit import / beforeInsert hook stamp). Free: one string parse, no extra query. Read by #6468's anchoring rules, now shared with the seeding scan as `readAutonumberCounter` so the two cannot drift. - Re-seed and re-issue on a unique violation attributable to an autonumber the engine issued, bounded to 3 attempts, then refuse with `code: 'ERR_AUTONUMBER_COLLISION'` carrying the driver error as `cause`. The predicates come from `@objectstack/types` (#6250 / #6544), never a dialect word-list of the engine's own. Batch inserts drop the stale counter but are never re-issued (bulkCreate may be partially applied). #6114's read-failure discrimination is unchanged and now also covers the re-seed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015QW3F6hGmFkf1RpwGBthDo
…the batch outcome (#6806) driver-memory enforces no uniqueness at all (its create is a table.push(), #4065), so the collision branch is unreachable there and a duplicate lands silently. Named and pinned rather than left implied (PD #10); adoption is the half that covers that driver. Also pins what an author gets on a batch collision: the driver's own error, never ERR_AUTONUMBER_COLLISION, with the counter dropped so the caller's retry converges. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015QW3F6hGmFkf1RpwGBthDo
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
📓 Docs Drift CheckThis PR changes 1 package(s): 15 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
…on half (#6806) Measured `supports.autonumber` across all five in-repo drivers rather than asserting "the storage layer": only driver-memory (`supports = {}`) and driver-mongodb (bit absent) take the engine fallback path; driver-sql declares `autonumber: true` and driver-sqlite-wasm / driver-turso inherit it via `extends SqlDriver`. Of the two, only driver-mongodb can raise a unique violation, so the collision retry protects essentially one backend. Anchored to the reading the repo already ruled and gates — scripts/driver-memory-census.ledger.json's `ruled-permanent` disposition for autonumber-seed-cross-side-parity.integration.test.ts ("InMemoryDriver declares `supports = {}`, so the ENGINE's autonumber seeding owns the counter") — rather than authoring a second answer to who owns the counter (#6832's shape). The test rig is a hand-rolled fake driver and imports no driver package, so check:driver-memory-census sees no unledgered arrival; re-run to confirm. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015QW3F6hGmFkf1RpwGBthDo
os-zhuang
marked this pull request as ready for review
August 9, 2026 04:13
os-zhuang
enabled auto-merge
August 9, 2026 04:13
This was referenced Aug 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #6806
Engine-side only.
packages/objectql/src/engine.ts(theapplyAutonumbers/seedAutonumberregion), its tests, one changeset. No spec change;packages/drivers/**untouched (#5499 freeze; maintainer ruling 2026-08-06 = A).The counter stays global — tenant-partitioning remains parked per #5495's disposition and is not relitigated here.
The card's real question: which resync shape is correct?
Candidates were (a) re-seed on collision, (b) re-seed on the first write after an exempt-writer insert is detectable, (c) both. Measured answer: (c) — but (b) is not "re-seed", it is "adopt", and that distinction is the whole measurement.
What the fallback path can observe, and what each candidate costs per insert
The decisive fact is that an exempt writer's insert passes through
applyAutonumbersitself.isSystemseed replay, apreserveAuditimport and abeforeInserthook stamp are exactly the three writers #5503's strip exempts, and all three land on theif (current != null && current !== '') continue;branch. The value is in hand at that moment. So candidate (b) has two possible implementations with wildly different costs:AUTONUMBER_SEED_PAGE_SIZErows/page)String.match, no querycreate, bounded to 3 attemptsA seed replay of N rows interleaved with normal inserts costs N full scope scans under invalidate-and-reseed and zero queries under adopt. That measurement is what rules out the re-seed spelling of (b).
Why adoption is correct, not merely cheap — judged against #6468
Adoption reads the supplied value with
readAutonumberCounter, which is the anchoring logic lifted verbatim out ofseedAutonumber's scan loop and now shared by both callers (PR #6553 / #6468 landed it as the inlineanchoredbranch; this PR only moves it). Same prefix check, same anchored-vs-unanchored rule, same suffix-stripped-when-it-matches-never-required, same linear/\d+/forms (the CodeQL polynomial-ReDoS constraint). Two copies of that reading would be two different answers for one row, i.e. a duplicate record number — the harm the whole family exists to prevent.The invariant this buys: a warm counter now answers what a cold restart would answer. Adoption applies exactly the reading the seeding scan would have applied to that row once it is persisted.
The two are disjoint, which is why both ship
Neither subsumes the other, and neither costs anything on the happy path.
The dependency question: #6250 is satisfied, not blocking
No
Blocked-by:line, deliberately. #6250 is closed completed and its predicate landed in@objectstack/typesasisUniqueViolationError, withuniqueViolationColumnadded by #6544.@objectstack/objectqlalready depends on@objectstack/types, so adopting them adds no edge. No dialect word-list is written in the engine (PD #12, precedent #5841).One measurement that changed the retry gate. #6544's doc anticipates this caller ("#5495's autonumber-retry branch asks a yes/no question of the answer"), but its contract is deliberately narrow: a value comes back only when the driver printed a determinable column, never an index name. MongoDB prints
E11000 ... index: doc_no_1 dup key: { doc_no: "D-0005" }— an index, souniqueViolationColumnanswersundefined. Gating the retry oncolumn === <autonumber field>would therefore make the resync unreachable on driver-mongodb — the only in-repo driver that both takes this path and can raise a collision. The gate landed as:A conflict the dialect attributes to a different column is rethrown untouched — #5495's «非本字段的冲突原样上抛», honoured exactly.
⚠ The collision half is STORAGE-DEPENDENT — measured across all five drivers, by name
The collision branch is triggered by the store rejecting the duplicate, so it reaches only drivers that take this fallback path and enforce uniqueness. Rather than gesturing at "the storage layer",
supports.autonumberwas read from each of the five in-repo drivers:supports.autonumbersupports = {}memory-driver.ts:174{ batchSchemaSync: true }mongodb-driver.ts:94idx_<f>_unique) when the field declaresuniqueE11000 duplicate key→ re-seed + re-issueautonumber: truesql-driver.ts:2256extends SqlDriver, nosupportsoverridesqlite-wasm-driver.ts:67...super.supportsturso-driver.ts:180This confirms the PM's sketch: the violation-triggered retry protects essentially ONE backend — driver-mongodb with a
uniqueautonumber field. Two of five drivers take the path; one of those two can raise anything.InMemoryDriver.createis atable.push()storing no constraints of any kind — its own docstring has said so since #4065, where it also stopped calling itself production-ready and started calling itself a WEAK oracle, for this exact reason.Anchored to the existing ruled answer, not a second one
This is not a fresh prose claim. It is the reading the repo already ruled and gates, in
scripts/driver-memory-census.ledger.json's disposition forpackages/runtime/src/autonumber-seed-cross-side-parity.integration.test.ts— axisruled-permanent, ruling «#6664 A, maintainer 2026-08-08 — inherits #5704 Q2 = B»:The engine doc comment, the test header and the changeset all cite that ruling rather than restating it. Authoring a second answer to "who owns the autonumber counter" is verbatim the one-contract-two-numbers shape graded on #6832, and this change does not create it while fixing a different one.
check:driver-memory-census— no unledgered arrivalThe new test rig is a hand-rolled fake driver object; the file imports no driver package at all (
driver-memoryappears only in prose). So it adds no consumer, the ledger needs no entry, and no disposition was invented to silence a gate.pnpm check:driver-memory-censusre-run after these edits: PASS.The outcome, pinned rather than implied
test §(3b)pins the silent duplicate directly (two rows carryingD-0005, onecreatecall, no error), so "collisions are handled" cannot read as true where it is not (PD #10 — never advertise a capability the runtime does not deliver). A companion pin shows adoption does hold on that driver, so the coverage statement is not empty there.Follow-up finding for the drivers lane, not implemented here: the remaining silent-duplicate row can only be closed by uniqueness enforcement in
driver-memory, which is out of scope under the #5499 freeze. It is explicitly not an engine-side pre-issue existence probe: that costs a query on every insert (the cost this design avoids) and is still TOCTOU-racy, so it would trade a silent duplicate for a rarer silent duplicate at double the read cost.What an author gets on a batch collision
The batch path is re-seeded but never re-issued:
bulkCreatemay be partially applied by a driver without a transaction, so re-writing the batch could duplicate the rows that did land — strictly worse than the collision.So, plainly:
insert(object, rows[])andinsertManyreject with the driver's own duplicate-key error — neverERR_AUTONUMBER_COLLISION, which is the single-row identity for "re-issued and still refused". Whether any row was written is the driver's answer, not the engine's. The engine's guarantee is narrower and is the one that matters: the stale counter is dropped, so the caller's retry re-seeds instead of walking into the same collision one number at a time. Pinned for both entry points, including the negative assertion that the code is notERR_AUTONUMBER_COLLISION.Acceptance
1. Exempt-writer inserts no longer leave the counter stale. §(1), 4 tests —
isSystemseed replay,preserveAuditimport,beforeInserthook stamp, and a batch of exempt rows (max wins, not last-seen). Each asserts the next generated number follows the supplied one (D-0009→D-0010, notD-0005) and that it did so with one seeding scan for the whole sequence, i.e. by adoption rather than re-reading. Both fallback drivers' shapes are used as fixtures: driver-memory's (no uniqueness, §3b) and driver-mongodb's (E11000, §3).§(2) pins adoption's four refusals — never lowers a counter; never seeds an unseeded one (that would skip the scan and answer from a single row — the defect arrived at from the fix); never adopts across scope (a past-date import cannot burn today's band); never throws (an exempt write was accepted before and still is), with the generating branch's
{field}-empty refusal kept as a control.2. A collision does not silently burn numbers. §(3), 9 tests. Observable outcome asserted, never a bare
toThrow:D-0008after colliding onD-0005, with exactly 2createcalls and 2 seeding scans;D-0009— the pre-fix storm was that the stale counter survived, so0006and0007collided in turn, one burned number and one raw driver error per insert;AUTONUMBER_COLLISION_ATTEMPTS(3) the error carriescode: 'ERR_AUTONUMBER_COLLISION', names the field, says "No record was written", and keeps the driver's error ascause— the raw driver error is not the contract;D-0004, i.e. the store decides, not the three advanced in-memory increments;createcall.3. #6114's read-failure discrimination survives. §(4), 3 tests: missing table still seeds from 0; an outage on the first seed still propagates and writes nothing; and — the new call site — an outage during the RE-seed propagates, is never swallowed into 0 or a stale value, attempts exactly one further
create, and once the store is readable comes back from the real max (D-0008), not from 1.4. Reverse verification — below.
Reverse verification (predicted BEFORE running)
Removal applied:
applyAutonumbers' adopt branch reverted to a barecontinue; the single-row path callsdriver.createdirectly; the batch catch dropped. Tests unchanged, 24 total.Predicted: 13 red / 11 green. Actual: 13 red / 11 green — same 13 tests, same 11 tests.
expected 'D-0005' to be 'D-0010'expected 'D-0005' to be 'D-0022'expected 'D-0005' to be 'D-0031'expected 'D-0005' to be 'D-0015'E11000 … D-0005E11000 … D-0005duplicate key value violates unique constraint "doc_doc_no_key"expected 11000 to be 'ERR_AUTONUMBER_COLLISION'toMatchObject({code:'ERR_AUTONUMBER_COLLISION'})vs code11000expected 'D-0006' to be 'D-0010'E11000 … D-0006expected 'D-0006' to be 'D-0010'E11000 … D-0006expected 'D-0005' to be 'D-0010'expected … /Connection terminated/ but got 'E11000 …'Green, as predicted: all five of §(2)'s refusals, the three §(3) pass-through cases, §(3b)'s silent-duplicate pin, and both unchanged §(4) read-failure controls.
The one deviation, reported honestly: #10 and #11 failed harder than predicted. I predicted an assertion diff on the follow-up insert; the follow-up insert instead threw, because without the counter drop it collides on
D-0006— a number the unobservable writer already holds. Same test, same cause, more severe signature. That deviation is itself a finding: while writing the prediction I noticed the original fixture (a single out-of-band row) made "drop the counter" and "merely advance past the collision" indistinguishable — both would have producedD-0006and the test would have passed against the removal. The fixture was widened to a whole band (D-0005…D-0009) before running, so the pin now discriminates. Commit28aaa1b.Test placement
New sibling
packages/objectql/src/engine-autonumber-resync.test.ts, not a lone new file elsewhere and not folded into an existing one. The objectql autonumber coverage is already a one-file-per-concern family —engine-autonumber-{batch,defer,runtime-owned,seed-outage,seed-scan,seed-suffix}.test.ts— each named for the issue it pins.-resyncis that family's next member and is where a future divergence in this behaviour gets caught. It reuses the #6468 suffix file's fake-driver rig deliberately, so the two files must keep agreeing about what the seeding scan is allowed to send.Fake driver, real engine — so no engine write-verb dispatch contract is involved, no
assertEngineDeleteDispatchfake engine is introduced, and no driver package is imported.Gates
Enumerated fresh from
.github/workflows/lint.ymland re-enumerated after eachorigin/mainmerge — it grew twice during this task (check:agent-model-declared, thencheck:tenant-chokepoint; 57check:*scripts now). All run individually, all green:pnpm lint✅check:*✅ — includingcheck:driver-memory-census(re-run after the doc changes),check:durability-log-level,check:engine-double-contract,check:type-check-coverage,check:driver-conformance,check:agent-model-declared,check:tenant-chokepoint, and the@objectstack/spec-filtered setturbo run build --filter='!@objectstack/docs' --concurrency=2) ✅turbo run typecheckoverpackages/*,packages/*/*,apps/*— 120/120 ✅;tsc --noEmit -p packages/objectql✅check:type-check-debtafter that build ✅ — "none above its recorded number"; no ledger entry raised (the-N can be loweredlines are informational and pre-existing)check:platform-checklistis maintainer-only on-demand per its own step comment and was not run.packages/objectql: 159 files / 2752 tests green, no regressions; the new file is 24/24.origin/mainmerged three times during the task (it kept moving under me); none of the merges touchedpackages/objectql.