Skip to content

test(driver-turso): the cross-process autonumber pin starts its writers once and stops waiting on fsync - #23218

Merged
objectstack-fleet[bot] merged 4 commits into
mainfrom
claude/issue-23182-turso-autonumber-test-budget
Oct 13, 2026
Merged

objectstack-fleet[bot] merged 4 commits into
mainfrom
claude/issue-23182-turso-autonumber-test-budget

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Part of #23182 (row 2)

Clause-②: no

Row 2 of #23182 only: @objectstack/driver-turso, src/turso-remote-autonumber-concurrency.test.ts, read at 29.3 s in run 38228297391. Refresh PR #23141 lists it over 10 s in 6 of the 7 hourly full-battery runs that executed it. #23182 remains open for rows 1 and 3 to 6.

What changed (one file, test-only)

The file pins cross-process autonumber atomicity on the remote face: two writer processes and one libSQL database file must give N distinct numbers, monotonic per writer, with evidence that the writers overlapped. Most of its time went to work it does not assert on. Measured at d4aaeaf0e on this 4-vCPU container, one round took about 4.4 s:

phase per round, before what it was
writer start-up ~2.2 s node --import tsx, then the driver's dependencies (@objectstack/driver-sql and what it imports, @objectstack/spec, @libsql/client) loaded through tsx's hook: ~2 s, against ~1.2 s under plain Node. The package's own source compiled in ~40 ms.
writes (2 x 200 creates) ~2.2 s fsync-bound. A create is 3 statements (reserve, insert, read back). A single writer took 5.9 ms per create with synchronous = FULL and 1.2 ms with synchronous = OFF.

A round whose writers did not overlap also re-spawned both writers.

Three changes, none to what is asserted:

  1. Writers start once. They start once per test and serve every round over stdin/stdout, one JSON line per command and per answer. A retry round costs only its writes.
  2. Dependencies load before tsx. A writer requires the driver's dependencies with plain Node first, then calls require('tsx/cjs/api').register() and requires src/index.ts. The source is still compiled by tsx, and the dependencies are the same modules, already in the require cache. tsx is already a declared devDependency, and tsx/cjs/api is one of its public exports.
  3. No fsync. Each writer runs PRAGMA synchronous = OFF on its own connection. That drops only the disk flush at commit, on a throwaway temp file. The journal mode, the locks and the busy timeout are unchanged, so the cross-process serialisation the pin depends on is unchanged.

The round's ready and write times now join the existing measurement line, so a slow CI run can be attributed from its log.

Kept as they were: N = 200 per writer, at most 3 rounds, every pin, and the NOT MEASURED skip. N was not reduced: writes are no longer the cost, and a smaller N would only lower how often a broken counter is caught (see the reverse verification below).

Readings

The CI per-file number is vitest's module line (✓ path (1 test) Nms). All readings below are from this container under scripts/pm/os-verify-lock.sh, on a shared box (other seats' unlocked work was running), with vitest's default reporter (agent-detection variables unset, as in CI).

density BASE d4aaeaf0e this PR
the file alone 4575, 5256, 4406 ms 1836, 1840, 1853 ms (83ece05da)
whole driver-turso package (88 files) 5735, 5914 ms 1912 ms (83ece05da)
package plus driver-sql's suite at once 10658, 8741, 8973 ms 4126 ms (794b74409, same run as BASE's 8973)

Writer changes per round: 2 to 40 before, 141 to 221 after (unmutated runs).

CI reading for the after-state: NOT MEASURED here. Job logs and artefacts redirect to an Azure blob host that this container's egress proxy refuses (403). This PR's own Test Core shard carries the file's module line and the measurement line in its log.

Reverse verification (ablation)

Mutation: the warm reservation (UPDATE … SET last_value = last_value + 1 … RETURNING) split into a SELECT last_value and then an UPDATE … SET last_value = ? (read-then-write), once with no gap and once with a 1 ms gap.

  • Applied through scripts/ablation-replace.mjs: the anchor hit went 1 → 0 and the blob changed. The restore was proven after every leg: blob equal to HEAD and git diff HEAD empty.
  • Reach: a tsx child spawned from the package root sees the mutated method, because nextRemoteSequenceValue.toString() carries the marker. No dist is involved, since both the parent and the writers run src.
  • Predicted: red on the "no write failed" pin, with UNIQUE constraint failed in a writer's error list, as the file's header recorded. Observed: that direction in every red run.
shape no gap 1 ms gap
this PR (794b74409 and 83ece05da) red 8 of 11 red 7 of 7
this PR's first cut 7e0c6cc50 (before the dependency preload) red 7 of 10 red 5 of 5
BASE, same box and hour, interleaved with the first cut red 1 of 5 not run

A non-atomic counter is caught more often than before, not less: without the fsync the writers interleave far more finely. A zero-gap read-then-write remains a probabilistic catch, as it was at BASE.

Gates and tests (at 83ece05da unless named)

  • Build: pnpm --filter '@objectstack/driver-turso...' build green, after merging origin/main.
  • Typecheck and tests: pnpm --filter @objectstack/driver-turso typecheck (tsc --noEmit) is green, and its tsconfig includes src/**/*, tests included. The package suite ran 88/88 files: 2373 passed, 33 skipped, the same counts as BASE.
  • Driver conformance: pnpm check:driver-conformance before the first edit and after the last commit read 50 covered cell(s), 0 in the DEBT ledger, 0 exempt both times.
  • Derived gates: node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack gave 51 commands, identical to the dispatch-time set, and all ran. The --ran verdict reads 51 derived, 51 run, 0 NOT-MEASURED, 0 UNRUN. Three first answered exit 3 (unmet prerequisites) and were re-run green after their remedy: check-plugin-teardown-shape --self-test needed git fetch --unshallow, and check:lean-entry-closure and check:dual-build-cjs-loads needed package dists, restored from the turbo cache.
  • Lint, narrowed and proven: eslint with --no-inline-config on the one changed file linted 1 file with 0 errors and 0 warnings. The file is in eslint's population (--print-config returns a config for it). The config enables no type-aware linting (the resolved parserOptions are ecmaVersion and sourceType), so this diff cannot move any other file's verdict.
  • Bytes: check:nul-bytes is green, and a control-byte self-scan of the file found 0 lines.

Changeset

None. The diff is test-only: the changed file is not in driver-turso's files[] (dist, README.md, CHANGELOG.md). Measured on a dist built after the merge, four symbols unique to the test (PRELOADED_DEPENDENCIES, WORKER_SOURCE, REPLY_DEADLINE_MS, turso-autonumber-concurrency-) have 0 hits, and the positive control nextRemoteSequenceValue has 6. Label: skip-changeset.

Acceptance notes

  • Import time is shared, not this file's. The whole vitest process for this one file still takes about 16 to 19 s of wall here, and about 14 to 15 s of that is vitest's import of the driver's source graph. Every file in this package pays that import, and the per-file reading does not count it.
  • @objectstack/core import cost. It alone takes about 0.9 s to import under plain Node and dominates a writer's remaining start-up of about 1.2 s. That is an observation about another package's import cost, not a defect; it is not filed.
  • The old ablation reading was optimistic. The file's header recorded the zero-gap ablation as red in 2 of 3 runs at BASE. On this box the same BASE shape was red in 1 of 5. The new shape records its own reading beside the old one.
  • The card's pin can only be read after landing: "the next refresh PR's list no longer names the file".

Generated by Claude Code

…sync

The cross-process autonumber pin spent its wall on two things it does
not assert: a tsx start-up per writer per round, and an fsync per
commit on a throwaway temp database. The writers now start once and
serve every round over stdin/stdout, and each sets synchronous = OFF
on its own connection; the lock protocol and every pin are unchanged.

Claude-Session: https://claude.ai/code/session_01P88ehrZ6hM19hBzdpsQTTo
Co-authored-by: Claude <noreply@anthropic.com>
…stered

A writer's start-up was about 40 ms of compiling the package's source
and about 2 s of loading its dependencies' built code through tsx's
require hook, which plain Node loads in about 1.2 s. The writer now
requires those dependencies first, then registers tsx through its
public CJS API and requires the source, whose requires hit the cache.
The phase times join the measurement line.

Claude-Session: https://claude.ai/code/session_01P88ehrZ6hM19hBzdpsQTTo
Co-authored-by: Claude <noreply@anthropic.com>
The read-then-write mutation, re-run on the new shape, goes red in 6 of
8 runs with no gap and 5 of 5 with a 1 ms gap; the earlier shape went
red in 1 of 5 with no gap on the same box. A command written to a
writer that already died now fails with that writer's stderr.

Claude-Session: https://claude.ai/code/session_01P88ehrZ6hM19hBzdpsQTTo
Co-authored-by: Claude <noreply@anthropic.com>
…rso-autonumber-test-budget

Claude-Session: https://claude.ai/code/session_01P88ehrZ6hM19hBzdpsQTTo
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 13, 2026
@github-actions github-actions Bot added the tests label Oct 13, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 75cada0ea6859b735635688e07e5fcba324e90e8 → packageMentionDocs.

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 13, 2026 09:47
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 13, 2026 09:47
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 13, 2026
Merged via the queue into main with commit 6e6881e Oct 13, 2026
45 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-23182-turso-autonumber-test-budget branch October 13, 2026 10:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants