Skip to content

fix(oracle-gaps): copies resolve when the vitest config sets root - #103

Merged
KyleAMathews merged 17 commits into
mainfrom
fix/oracle-gaps-root-copies
Nov 16, 2026
Merged

KyleAMathews merged 17 commits into
mainfrom
fix/oracle-gaps-root-copies

Conversation

@KyleAMathews

@KyleAMathews KyleAMathews commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #101.

Problem

The gap CLI keeps each copy of a test file in memory and serves it to Vitest. Before this change, the wrapper plugin's resolveId resolved a copy to its own path. Vite then named the copy from its root (/.oracle-gaps-…-a.test.ts under test.root: './tests') and imported it again by that name. The resolver tried that name only from the package, so the copy did not load. The owned mutant there reported setup-error and was not judged. The repository's own oracle-gaps/vitest.config.mjs has this shape.

Fix

  • No plugin resolves a copy any more. Vitest imports a copy by its absolute path, which names no file on disk, so Vite keeps that path as the copy's id under every test root.
  • The wrapper's enforce: 'pre' load serves the copy by that absolute id, before the config's own plugins. A config plugin whose load reads test files with a sync read never reads a copy's path (merge-gate review, F1).
  • Removed: the per-context roots (globalThis.__oracleGapsRoots and the line that records it), the source, package and root candidates in resolveId, and the wrapper's own spelling rule (spelled). The wrapper imports posix from the paths.mjs beside it.
  • serveCopies, the promises.readFile patch that serves Vitest's pragma read, looks a copy up with posix(path) (first review, F3).
  • A patch changeset.

How the design got smaller (first review, F2): with the root candidate in place, the hostile mutants "resolve from the package only" and "realpath the root" survived. A trace showed why: a copy reached resolveId by its absolute path, missed, and Vite served it through load. So the resolver branches were dead code, and this PR removes them.

6fa802f also removed the load lookup, because a mutant that drops it survived. That survival was a gap in the histories, not dead code: no history had a config plugin with a load hook (merge-gate F1). That case now has a history (below), and the lookup is back.

Oracle changes

  • The constructed root dimension testRoot is now a set of shapes (testRoots in gap-report-model-oracle.mjs):

    • none: the drawn constant and its neutral value;
    • tests: the subdirectory that holds the tests, the existing history;
    • parent: the directory above the package. The driver places this package in pkg of its scratch directory, so the root holds only the package;
    • outside: ./src, a root that holds no test file. Its globs reach ../tests;
    • link: the package's tests is a symbolic link to tests-target, and the root is ./tests.
  • Each shape gives the include glob and the setup file relative to the root. Vitest reads both from the root.

  • A new field loadPlugin adds a config plugin whose load reads each .test.ts id with readFileSync.

  • Both fields are fc.constant at the end of the drawn record. The take-back mapping sets them back to their constants, as for tornRows. history-fingerprint.json records both lines in both lists, and the canary passes. Generated histories are unchanged.

  • New constructed histories are appended after all of main's. Main has 51 constructed histories (indices 0 to 50). The new ones:

    • 51: a config whose test root is a subdirectory that holds the tests
    • 52: a config whose test root is above the package
    • 53: a config whose test root holds no test file
    • 54: a config whose test root is a symbolic link
    • 55: a config whose plugin loads test files with a sync read

    Each history has its own declared state, in required and separate. Histories 52 to 54 also set setupGlobal, so the setup-file path from the root is reached (first review, F4).

  • F4, unreached fixture branches:

    • The CommonJS root expression is deleted. A root is written only for an ES module config.
    • Under a test root, setupFiles now comes from the shape's row. The tests and link shapes share one row, which the link history reaches.
  • A random base of the parent history found a fixture error (seed 837819645, pragma: true). Vitest resolves vitest-environment-<name> from the test root (Vitest 4.1.12, init chunk). The environment package in the package's node_modules was not found from the root above the package, and the baseline failed in Vitest itself. Under parent, the fixture now installs the environment in the root's node_modules. RED at 7e08435: the replay fails with "baseline 1 of 2 oracle files failed". GREEN at 6fa802f: the same replay passes.

  • The symbolic-link shape is required on every platform. The driver already creates dir links on win32 in CI, so the platform-scoped reach rule does not apply (coordinator ruling).

  • The literate comments now describe the mechanism: no copy resolver, and absolute ids (merge-gate F3).

  • Shards:

    • Windows shard 6 names the 5 histories of this PR: the subdirectory one, the three new root shapes, and the load-plugin history;
    • e2e-3 names their units;
    • the Linux fixed-seed partition takes them in its rest shard.
  • Main (eec7f77, test(oracle-gaps): judge stdout gap lines against results.json, by id #115) is merged in with a normal merge. One conflict, in the Limits string, was resolved by keeping both texts.

Evidence

  • Law: an owned mutant is judged:

    • wherever Vitest's test root sits: inside the package, above it, in a directory that holds no test file, or through a symbolic link;
    • and whatever the config's own plugins do in load.

    A copy that fails to load is no verdict on an owned mutant. Authority: issue oracle-gaps: a mutant of an owner under a config whose root is a subdirectory fails with setup-error #101; the merge-gate review of fix(oracle-gaps): copies resolve when the vitest config sets root #103 (F1); the register's P11 ("a report says what it couldn't judge") and P5 ("every survivor is a gap"); the Windows-paths entry (2026-11-13, revised) for the spelling; the generator rule and its extension (2026-11-13, 2026-11-14) for the fields and the history order.

  • Reach: each of the 5 states is reached 1 time by its constructed history in the fixed-seed campaign. Generated histories reach none of them, because both fields are constants. The random-seed and determinism units of all 5 histories pass (Arca r103-green-randdet: 10 passed). Six earlier local campaigns of the 4 root-shape random units passed (24 random bases).

  • RED:

    • oracle-gaps: a mutant of an owner under a config whose root is a subdirectory fails with setup-error #101, on main's check.mjs (4d3c8a9) with the root-shape oracle: tests, parent and link fail at cli with the gap setup-error … M22 line. outside passes on main.
    • F1, at 17fceb9: the new oracle with 6fa802f's check.mjs (Arca r103-red-loadplugin): -t "fixed seed 20261105: a config whose plugin loads test files with a sync read" fails at checkpoint cli (law: classification) with "src/sut.ts:f0:31:if-true#0: setup-error, owed killed".
  • GREEN (head fc02a63, on Arca):

    • Linux fixed-seed shard 1: 3 passed (r103-green-s1);
    • shard 2: 5 passed (r103-green-s2);
    • shard 3, which holds all 5 histories: 49 passed (r103-green-s3);
    • random and determinism units of the 5 histories: 10 passed (r103-green-randdet).

    Locally: pr-shards check ("5 partitions split their tests exactly"), e2e-shards check ("5 shards partition the e2e tests"), the unit suite (115 tests, including the fingerprint canary), typecheck and eslint all pass.

  • Mutant: scratch branches of fc02a63, not pushed. Each ran the 5 histories on Arca:

    Mutant Histories that kill it
    No load lookup (the code at 6fa802f) the load-plugin history: the RED above
    resolveId resolves a copy by its source or from the package (main's resolver) tests, parent, link (classification, setup-error, owed killed)
    resolveId resolves a copy from Vite's root, but takes the root only when it lies in the package parent only
    resolveId resolves a copy from the real path of Vite's root link only
    The runner takes a test file only when it lies under the config's root (globSpecs) outside only (the run refuses with exit 2, law completion)
  • Limits:

    • F3 of the first review, no legal history reaches it: on POSIX, Vitest 4.1.12 rewrites \ to / in every path (through pathe). Measured in /tmp/bsexp.*:

      • a package directory named pk\g: Vitest does not find its config ("No test files found");
      • a test file tests/a\b.test.ts: Vitest lists it as tests/a/b.test.ts and fails with "Cannot find module".

      Copy keys come from spec.moduleId, so serveCopies never sees a \ on POSIX. pathe also uppercases a win32 drive letter. The posix(path) change is a refactor with no observable change, and there is no RED for it.

    • The histories run copies only in Vite's server environments: node, and the pragma's ssr environment. jsdom and happy-dom (Vite's client environment) are not generated. The merge-gate review ran jsdom by hand under all five root shapes, and the copies loaded at the head in each: 1 killed and 3 gaps in every shape.

    • Config plugins are covered only as the pre transform (prePlugin) and the sync-read load (loadPlugin).

    • The Vitest 5 path of the copy read is not exercised. The Vitest 5 CI job runs no gap oracle.

    • Each test-root shape has one constructed history: an ESM discovered config, one owner, one context. Not covered: --config with test.root, CommonJS configs, test.projects, several contexts with different roots, and a root at the filesystem's root (which would give //… paths).

    • A shrink of a failing constructed history takes both fields back to their constants, as with tornRows.

    • docs/contributing/oracle-coverage.md is not in this repository, so no coverage map is updated here.

CI

Run 43804315582 on fc02a63 concludes success: 44 of 46 jobs pass. The longest job is Windows 2/10 at 9.5 min, and every job is under 10 min (P12).

  • Windows shard 6, which holds all 5 histories of this PR (the symbolic link and the load plugin among them), passes in 2.8 min.
  • The Linux fixed-seed shards take 5.2, 5.5 and 6.1 min. Shard 3, the rest, holds the new histories.

The two failures are the non-blocking Windows end-to-end jobs (C entry (c)). Main fails them too: run 43801009034 on eec7f77, and earlier run 43693786590 on 4d3c8a9. This PR does not change either shard:

Register entries touched

  • P2: the model changes only fixture text. It imports nothing from production.

  • P5: each surviving mutant got a measured cause.

    • The resolver mutants survived because the resolver branches were dead code, which this PR removes.
    • The load mutant survived because no history had a plugin load. A history for it is added, and the lookup is restored.
  • P7: five states and histories are added. No law, bound or exclusion is weakened. The testRoot constant changes from false to none only in this PR's own new field.

  • P9: oracle commits come before each fix. 12a2981 and 7bdef53 come before 7e08435; 17fceb9 comes before fc02a63. RED, GREEN and the mutants are above.

  • P10, P11: no CLI line is added or reworded.

  • Generator rule and its extension: both fields are drawn constants at the end, set through the take-back, and the histories are appended last.

  • Platform-scoped reach (fix(oracle-gaps): Windows static and e2e failures, by class (follow-up to #58) #73): not used. The symbolic-link state is required on win32.

  • Windows paths: serveCopies and the wrapper both use paths.mjs's posix.

  • Zero shipped bytes: check.mjs is test-time code. Its weight:

    Raw gzip -9 gzip -9 without comment lines Diff against main
    main (4d3c8a9) 100,038 B 29,688 B 17,092 B
    ff1705d 100,856 B 29,980 B 17,196 B +18/−6 lines
    6fa802f 99,899 B 29,671 B 16,990 B +17/−21 lines
    this head (fc02a63) 100,220 B 29,746 B 17,015 B +15/−13 lines

    The head's code is 77 B gzip smaller than main's. With comments it is 58 B larger: the growth is the comments that explain why no plugin resolves a copy, plus the one posix import.

  • P12: each new history adds about 10 s. See the shard times in the CI section.

Process

  • Normal commits only. No force-push, no amend, no hook bypass. Main is merged in with a merge commit.
  • Vitest ran with at most 2 workers, through /tmp/e2e-lock.sh locally and through the Arca pool for the shard runs.

This pull request and its description were written by Isaac.

KyleAMathews and others added 6 commits November 15, 2026 00:40
…ubdirectory

Vitest names a copy of a test file by its path from the config's root. A
config that sets root to the directory holding the tests makes every copy's
id root-relative, so the wrapper's resolver must find the copy there too.
Adds one constructed history (appended last, so existing histories keep
their positions) with a config whose root is the tests directory, and
declares the state it reaches. The owned mutant must be judged, never a
setup error. Refs #101.

Co-authored-by: Isaac <no-reply@databricks.com>
…enerator

The constructed history for a config whose root is a subdirectory sets
configRoot, which the histories arbitrary could not take back, so the
campaign failed before running. Declare it as a drawn constant (it draws
nothing, so generated histories keep their values) and take a constructed
history back as false, as tornRows is. Refs #101.

Co-authored-by: Isaac <no-reply@databricks.com>
Vitest names a copy by its path from the test root (test.root), not from
the Vite root the CLI forces to the package. The constructed history must
set test.root to the directory holding the tests, as the repository's own
oracle-gaps config does, and the fixture must reach that state. The field
is renamed testRoot to match. Refs #101.

Co-authored-by: Isaac <no-reply@databricks.com>
The constructed history copied tornRows from its template. Its witness needs
a batch hold this history does not produce, so the campaign failed on the
torn-row witness before the law that the history exists for. Its state is
the test root, so it sets tornRows false. Refs #101.

Co-authored-by: Isaac <no-reply@databricks.com>
A copy of a test file is named beside its file, but Vitest names a module
by its path from the test root (test.root), so a config that sets the test
root to a subdirectory gave every copy an id the resolver did not map back,
and each mutant failed with setup-error. The wrapper's resolver now also
tries the copy from the root Vitest names modules from, recorded per
context after it resolves, and spells paths as the copies are keyed (on
Windows, forward slashes and an uppercase drive; on POSIX, unchanged).
Closes #101.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@changeset-bot

changeset-bot Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: fc02a63

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@tanstack/oracles Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 3d5f038a-4e40-4b9f-9e28-7f5dede4bd9c
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

KyleAMathews and others added 3 commits November 15, 2026 01:20
Takes main's constructed histories (indices 27-29, from #73) first, so they
keep their positions, and appends this branch's test-root history after
them. The grammar's new field testRoot goes at the end of the drawn record,
and its fingerprint is recorded in history-fingerprint.json. Refs #101.

Co-authored-by: Isaac <no-reply@databricks.com>
…shard

The Windows fixed-seed partition lists every test by name, so the new
history must be named in one shard; shard 6 is the lightest. Refs #101.

Co-authored-by: Isaac <no-reply@databricks.com>
The e2e partition lists every unit by name: the fixed-seed, random, and
determinism units of the new history go in shard e2e-3, beside the other
constructed histories of #73. Refs #101.

Co-authored-by: Isaac <no-reply@databricks.com>
@pkg-pr-new

pkg-pr-new Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/TanStack/oracles/@tanstack/eslint-plugin-oracles@103
npm i https://pkg.pr.new/TanStack/oracles/@tanstack/oracles@103

commit: fc02a63

KyleAMathews and others added 8 commits November 15, 2026 18:45
… test root

The test root dimension (testRoot) becomes a set of shapes: the
directory that holds the tests, the directory above the package, a
directory that holds no test file, and a tests directory that is a
symbolic link. Each new shape has its own constructed history, appended
after the existing ones, and runs a setup file named from the root.

Co-authored-by: Isaac <no-reply@databricks.com>
…, and state the copy read's limit

Co-authored-by: Isaac <no-reply@databricks.com>
… and e2e-3

Co-authored-by: Isaac <no-reply@databricks.com>
… their path

No plugin resolves a copy any more, so Vite keeps the absolute path that
Vitest imports it by and reads it through serveCopies, under every test
root. This removes the per-context roots, the package and root
candidates, the load hook's copy lookup, and the wrapper's second copy
of the path rule. serveCopies uses paths.mjs's posix.

Co-authored-by: Isaac <no-reply@databricks.com>
…the package resolves it

Vitest resolves vitest-environment-<name> from the test root. A random
base of the root-above-the-package history drew a pragma, and the
environment in the package's node_modules was not found from the root,
so the fixture's baseline failed in Vitest itself (seed 837819645).

Co-authored-by: Isaac <no-reply@databricks.com>
# Conflicts:
#	packages/oracles/oracle-gaps/gap-report-oracle.property.test.mjs
…st files with a sync read

The merge-gate review of #103 (F1) found that a config plugin whose own
load hook reads each test file with readFileSync gets ENOENT on the
runner's in-memory copy, once the runner's load no longer answers for
it. No history had a plugin with a load hook. The new field loadPlugin
is a drawn constant, set only by the new constructed history, appended
after the test-root histories. Also corrects the test-root comments,
which still described a copy resolver, and the Limits text on Vite's
client environment.

Co-authored-by: Isaac <no-reply@databricks.com>
…s own plugins

The pre plugin's load answers for an in-memory copy of a test file
again, by its absolute id, so a config plugin whose load reads test
files with a sync read never reads a copy's path (review of #103, F1).
No plugin resolves a copy, so #101 stays fixed.

Co-authored-by: Isaac <no-reply@databricks.com>
@KyleAMathews
KyleAMathews merged commit 9aed105 into main Nov 16, 2026
48 of 50 checks passed
@KyleAMathews
KyleAMathews deleted the fix/oracle-gaps-root-copies branch November 16, 2026 05:22
KyleAMathews added a commit that referenced this pull request Nov 16, 2026
Takes #103's testRoot and loadPlugin fields and its five constructed histories
first. #57's fields follow loadPlugin, and its 13 constructed histories follow
main's 35. The take-back uses one table of constructed-only neutral values,
which now covers tornRows, testRoot and loadPlugin too. Shard lists keep both
sides' names.

Co-authored-by: Isaac <no-reply@databricks.com>
KyleAMathews added a commit that referenced this pull request Nov 16, 2026
#103's constructed histories (51 to 55) come first; #99's follow them, its
deterministic-cuts histories and then the two I/O-owner histories. #103's
history fields (testRoot, loadPlugin) come before #99's hangMs, in the draw
order and in the fingerprint, and the take-back reads all of them as their
drawn constants.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

oracle-gaps: a mutant of an owner under a config whose root is a subdirectory fails with setup-error

1 participant