Repository navigation
fix(oracle-gaps): copies resolve when the vitest config sets root - #103
Merged
Merged
Conversation
…ubdirectory Vitest names a copy of a test file by its path from the config's root. A config that sets root to the directory holding the tests makes every copy's id root-relative, so the wrapper's resolver must find the copy there too. Adds one constructed history (appended last, so existing histories keep their positions) with a config whose root is the tests directory, and declares the state it reaches. The owned mutant must be judged, never a setup error. Refs #101. Co-authored-by: Isaac <no-reply@databricks.com>
…enerator The constructed history for a config whose root is a subdirectory sets configRoot, which the histories arbitrary could not take back, so the campaign failed before running. Declare it as a drawn constant (it draws nothing, so generated histories keep their values) and take a constructed history back as false, as tornRows is. Refs #101. Co-authored-by: Isaac <no-reply@databricks.com>
Vitest names a copy by its path from the test root (test.root), not from the Vite root the CLI forces to the package. The constructed history must set test.root to the directory holding the tests, as the repository's own oracle-gaps config does, and the fixture must reach that state. The field is renamed testRoot to match. Refs #101. Co-authored-by: Isaac <no-reply@databricks.com>
The constructed history copied tornRows from its template. Its witness needs a batch hold this history does not produce, so the campaign failed on the torn-row witness before the law that the history exists for. Its state is the test root, so it sets tornRows false. Refs #101. Co-authored-by: Isaac <no-reply@databricks.com>
A copy of a test file is named beside its file, but Vitest names a module by its path from the test root (test.root), so a config that sets the test root to a subdirectory gave every copy an id the resolver did not map back, and each mutant failed with setup-error. The wrapper's resolver now also tries the copy from the root Vitest names modules from, recorded per context after it resolves, and spells paths as the copies are keyed (on Windows, forward slashes and an uppercase drive; on POSIX, unchanged). Closes #101. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
🦋 Changeset detectedLatest commit: fc02a63 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
Takes main's constructed histories (indices 27-29, from #73) first, so they keep their positions, and appends this branch's test-root history after them. The grammar's new field testRoot goes at the end of the drawn record, and its fingerprint is recorded in history-fingerprint.json. Refs #101. Co-authored-by: Isaac <no-reply@databricks.com>
…shard The Windows fixed-seed partition lists every test by name, so the new history must be named in one shard; shard 6 is the lightest. Refs #101. Co-authored-by: Isaac <no-reply@databricks.com>
commit: |
… test root The test root dimension (testRoot) becomes a set of shapes: the directory that holds the tests, the directory above the package, a directory that holds no test file, and a tests directory that is a symbolic link. Each new shape has its own constructed history, appended after the existing ones, and runs a setup file named from the root. Co-authored-by: Isaac <no-reply@databricks.com>
…, and state the copy read's limit Co-authored-by: Isaac <no-reply@databricks.com>
… and e2e-3 Co-authored-by: Isaac <no-reply@databricks.com>
… their path No plugin resolves a copy any more, so Vite keeps the absolute path that Vitest imports it by and reads it through serveCopies, under every test root. This removes the per-context roots, the package and root candidates, the load hook's copy lookup, and the wrapper's second copy of the path rule. serveCopies uses paths.mjs's posix. Co-authored-by: Isaac <no-reply@databricks.com>
…the package resolves it Vitest resolves vitest-environment-<name> from the test root. A random base of the root-above-the-package history drew a pragma, and the environment in the package's node_modules was not found from the root, so the fixture's baseline failed in Vitest itself (seed 837819645). Co-authored-by: Isaac <no-reply@databricks.com>
# Conflicts: # packages/oracles/oracle-gaps/gap-report-oracle.property.test.mjs
…st files with a sync read The merge-gate review of #103 (F1) found that a config plugin whose own load hook reads each test file with readFileSync gets ENOENT on the runner's in-memory copy, once the runner's load no longer answers for it. No history had a plugin with a load hook. The new field loadPlugin is a drawn constant, set only by the new constructed history, appended after the test-root histories. Also corrects the test-root comments, which still described a copy resolver, and the Limits text on Vite's client environment. Co-authored-by: Isaac <no-reply@databricks.com>
…s own plugins The pre plugin's load answers for an in-memory copy of a test file again, by its absolute id, so a config plugin whose load reads test files with a sync read never reads a copy's path (review of #103, F1). No plugin resolves a copy, so #101 stays fixed. Co-authored-by: Isaac <no-reply@databricks.com>
KyleAMathews
added a commit
that referenced
this pull request
Nov 16, 2026
Takes #103's testRoot and loadPlugin fields and its five constructed histories first. #57's fields follow loadPlugin, and its 13 constructed histories follow main's 35. The take-back uses one table of constructed-only neutral values, which now covers tornRows, testRoot and loadPlugin too. Shard lists keep both sides' names. Co-authored-by: Isaac <no-reply@databricks.com>
KyleAMathews
added a commit
that referenced
this pull request
Nov 16, 2026
#103's constructed histories (51 to 55) come first; #99's follow them, its deterministic-cuts histories and then the two I/O-owner histories. #103's history fields (testRoot, loadPlugin) come before #99's hangMs, in the draw order and in the fingerprint, and the take-back reads all of them as their drawn constants. Co-authored-by: Isaac <no-reply@databricks.com>
This was referenced Nov 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #101.
Problem
The gap CLI keeps each copy of a test file in memory and serves it to Vitest. Before this change, the wrapper plugin's
resolveIdresolved a copy to its own path. Vite then named the copy from its root (/.oracle-gaps-…-a.test.tsundertest.root: './tests') and imported it again by that name. The resolver tried that name only from the package, so the copy did not load. The owned mutant there reportedsetup-errorand was not judged. The repository's ownoracle-gaps/vitest.config.mjshas this shape.Fix
enforce: 'pre'loadserves the copy by that absolute id, before the config's own plugins. A config plugin whoseloadreads test files with a sync read never reads a copy's path (merge-gate review, F1).globalThis.__oracleGapsRootsand the line that records it), the source, package and root candidates inresolveId, and the wrapper's own spelling rule (spelled). The wrapper importsposixfrom thepaths.mjsbeside it.serveCopies, thepromises.readFilepatch that serves Vitest's pragma read, looks a copy up withposix(path)(first review, F3).How the design got smaller (first review, F2): with the root candidate in place, the hostile mutants "resolve from the package only" and "realpath the root" survived. A trace showed why: a copy reached
resolveIdby its absolute path, missed, and Vite served it throughload. So the resolver branches were dead code, and this PR removes them.6fa802falso removed theloadlookup, because a mutant that drops it survived. That survival was a gap in the histories, not dead code: no history had a config plugin with aloadhook (merge-gate F1). That case now has a history (below), and the lookup is back.Oracle changes
The constructed root dimension
testRootis now a set of shapes (testRootsingap-report-model-oracle.mjs):none: the drawn constant and its neutral value;tests: the subdirectory that holds the tests, the existing history;parent: the directory above the package. The driver places this package inpkgof its scratch directory, so the root holds only the package;outside:./src, a root that holds no test file. Its globs reach../tests;link: the package'stestsis a symbolic link totests-target, and the root is./tests.Each shape gives the include glob and the setup file relative to the root. Vitest reads both from the root.
A new field
loadPluginadds a config plugin whoseloadreads each.test.tsid withreadFileSync.Both fields are
fc.constantat the end of the drawn record. The take-back mapping sets them back to their constants, as fortornRows.history-fingerprint.jsonrecords both lines in both lists, and the canary passes. Generated histories are unchanged.New constructed histories are appended after all of main's. Main has 51 constructed histories (indices 0 to 50). The new ones:
a config whose test root is a subdirectory that holds the testsa config whose test root is above the packagea config whose test root holds no test filea config whose test root is a symbolic linka config whose plugin loads test files with a sync readEach history has its own declared state, in
requiredandseparate. Histories 52 to 54 also setsetupGlobal, so the setup-file path from the root is reached (first review, F4).F4, unreached fixture branches:
setupFilesnow comes from the shape's row. Thetestsandlinkshapes share one row, which thelinkhistory reaches.A random base of the
parenthistory found a fixture error (seed 837819645,pragma: true). Vitest resolvesvitest-environment-<name>from the test root (Vitest 4.1.12,initchunk). The environment package in the package'snode_moduleswas not found from the root above the package, and the baseline failed in Vitest itself. Underparent, the fixture now installs the environment in the root'snode_modules. RED at7e08435: the replay fails with "baseline 1 of 2 oracle files failed". GREEN at6fa802f: the same replay passes.The symbolic-link shape is required on every platform. The driver already creates
dirlinks on win32 in CI, so the platform-scoped reach rule does not apply (coordinator ruling).The literate comments now describe the mechanism: no copy resolver, and absolute ids (merge-gate F3).
Shards:
restshard.Main (
eec7f77, test(oracle-gaps): judge stdout gap lines against results.json, by id #115) is merged in with a normal merge. One conflict, in the Limits string, was resolved by keeping both texts.Evidence
Law: an owned mutant is judged:
load.A copy that fails to load is no verdict on an owned mutant. Authority: issue oracle-gaps: a mutant of an owner under a config whose root is a subdirectory fails with setup-error #101; the merge-gate review of fix(oracle-gaps): copies resolve when the vitest config sets root #103 (F1); the register's P11 ("a report says what it couldn't judge") and P5 ("every survivor is a gap"); the Windows-paths entry (2026-11-13, revised) for the spelling; the generator rule and its extension (2026-11-13, 2026-11-14) for the fields and the history order.
Reach: each of the 5 states is reached 1 time by its constructed history in the fixed-seed campaign. Generated histories reach none of them, because both fields are constants. The random-seed and determinism units of all 5 histories pass (Arca
r103-green-randdet: 10 passed). Six earlier local campaigns of the 4 root-shape random units passed (24 random bases).RED:
check.mjs(4d3c8a9) with the root-shape oracle:tests,parentandlinkfail atcliwith thegap setup-error … M22line.outsidepasses on main.17fceb9: the new oracle with6fa802f'scheck.mjs(Arcar103-red-loadplugin):-t "fixed seed 20261105: a config whose plugin loads test files with a sync read"fails at checkpointcli(law: classification) with "src/sut.ts:f0:31:if-true#0: setup-error, owed killed".GREEN (head
fc02a63, on Arca):r103-green-s1);r103-green-s2);r103-green-s3);r103-green-randdet).Locally:
pr-shards check("5 partitions split their tests exactly"),e2e-shards check("5 shards partition the e2e tests"), the unit suite (115 tests, including the fingerprint canary), typecheck and eslint all pass.Mutant: scratch branches of
fc02a63, not pushed. Each ran the 5 histories on Arca:loadlookup (the code at6fa802f)resolveIdresolves a copy by its source or from the package (main's resolver)tests,parent,link(classification, setup-error, owed killed)resolveIdresolves a copy from Vite's root, but takes the root only when it lies in the packageparentonlyresolveIdresolves a copy from the real path of Vite's rootlinkonlyglobSpecs)outsideonly (the run refuses with exit 2, law completion)Limits:
F3 of the first review, no legal history reaches it: on POSIX, Vitest 4.1.12 rewrites
\to/in every path (through pathe). Measured in/tmp/bsexp.*:pk\g: Vitest does not find its config ("No test files found");tests/a\b.test.ts: Vitest lists it astests/a/b.test.tsand fails with "Cannot find module".Copy keys come from
spec.moduleId, soserveCopiesnever sees a\on POSIX. pathe also uppercases a win32 drive letter. Theposix(path)change is a refactor with no observable change, and there is no RED for it.The histories run copies only in Vite's server environments: node, and the pragma's
ssrenvironment. jsdom and happy-dom (Vite's client environment) are not generated. The merge-gate review ran jsdom by hand under all five root shapes, and the copies loaded at the head in each: 1 killed and 3 gaps in every shape.Config plugins are covered only as the
pretransform (prePlugin) and the sync-readload(loadPlugin).The Vitest 5 path of the copy read is not exercised. The Vitest 5 CI job runs no gap oracle.
Each test-root shape has one constructed history: an ESM discovered config, one owner, one context. Not covered:
--configwithtest.root, CommonJS configs,test.projects, several contexts with different roots, and a root at the filesystem's root (which would give//…paths).A shrink of a failing constructed history takes both fields back to their constants, as with
tornRows.docs/contributing/oracle-coverage.mdis not in this repository, so no coverage map is updated here.CI
Run 43804315582 on
fc02a63concludessuccess: 44 of 46 jobs pass. The longest job is Windows 2/10 at 9.5 min, and every job is under 10 min (P12).rest, holds the new histories.The two failures are the non-blocking Windows end-to-end jobs (C entry (c)). Main fails them too: run 43801009034 on
eec7f77, and earlier run 43693786590 on4d3c8a9. This PR does not change either shard:Windows 2/10 (
every outcome class): "constructed historyevery outcome classdid not enter a hang cut off for idleness", the known Windows idle gap. On4d3c8a9, main failed the same history atcliwith "122 gap lines for 121 gaps", the gap-line count issue Windows e2e: stdout has one more gap line than the expected count (fixed seed 4/10, noTypescript) #87/random seed: generated finds a greppable-stream counterexample on main too: stdout has 189 gap lines for 190 gaps (seed 333199054) #102. Times:6fa802fff1705deec7f774d3c8a9The shard holds only
every outcome class, so its time over 9 min is main's cost.Windows 4/10 (
every outcome class, the warm and abort stops): "the stopped run's last line isoracle-gaps: done completed, 4 gaps (provisional)…, notoracle-gaps: done error, the run's process exited with SIGABRT". This is fix(oracle-gaps): supervise the check on win32; log each CLI run in the e2e driver #79's win32 abort witness. Times: 9.1 min here, 10.8 min on maineec7f77, and 8.9 min on main4d3c8a9.Register entries touched
P2: the model changes only fixture text. It imports nothing from production.
P5: each surviving mutant got a measured cause.
loadmutant survived because no history had a pluginload. A history for it is added, and the lookup is restored.P7: five states and histories are added. No law, bound or exclusion is weakened. The
testRootconstant changes fromfalsetononeonly in this PR's own new field.P9: oracle commits come before each fix.
12a2981and7bdef53come before7e08435;17fceb9comes beforefc02a63. RED, GREEN and the mutants are above.P10, P11: no CLI line is added or reworded.
Generator rule and its extension: both fields are drawn constants at the end, set through the take-back, and the histories are appended last.
Platform-scoped reach (fix(oracle-gaps): Windows static and e2e failures, by class (follow-up to #58) #73): not used. The symbolic-link state is required on win32.
Windows paths:
serveCopiesand the wrapper both usepaths.mjs'sposix.Zero shipped bytes:
check.mjsis test-time code. Its weight:4d3c8a9)ff1705d6fa802ffc02a63)The head's code is 77 B gzip smaller than main's. With comments it is 58 B larger: the growth is the comments that explain why no plugin resolves a copy, plus the one
posiximport.P12: each new history adds about 10 s. See the shard times in the CI section.
Process
/tmp/e2e-lock.shlocally and through the Arca pool for the shard runs.This pull request and its description were written by Isaac.