Repository navigation
fix(oracle-gaps): close the merged-PR conformance gaps (drawn fields, F-D, cold fallback) - #131
Open
KyleAMathews wants to merge 25 commits into
Open
KyleAMathews wants to merge 25 commits into
KyleAMathews wants to merge 25 commits into
Conversation
…ness wins) Under the maintainer's 2026-11-16 decision, a history field that changes what runs is drawn, unless it switches a test-only hook on. warmReplaced (a scenario), testRoot and loadPlugin (fixture shapes) are now drawn, after every earlier field. tornRows, pauseEngages and rejectBatch switch hooks, so they stay constants that only constructed histories set. A drawn root applies only to an ES module config that Vitest discovers in the package; a drawn load plugin only to a config in ES module syntax. Co-authored-by: Isaac <no-reply@databricks.com>
…allback The witness for R39 that #73 left open. A constructed history's daemon holds its reply at the test-only daemon-reply pause, and nothing ends it while the CLI runs. The CLI owes a completed check that equals a cold run. The driver releases the daemon only after the CLI has ended, by killing it; a CLI that never ends is ended by the call's 120 s backstop, which the incremental check reports as a cold-fallback violation. replyHeld switches a test-only hook on, so it is a constant that only the constructed history sets, appended last. Co-authored-by: Isaac <no-reply@databricks.com>
The held check now reaches a daemon whose session is open: a warm check starts the daemon with the reply pause disarmed (ORACLE_GAPS_TEST_PAUSE_ARM), then the driver arms it. So the CLI's own bound, not a slow session start, decides when it stops waiting. dieMidCheck's reply pause takes no arm. Co-authored-by: Isaac <no-reply@databricks.com>
A check through the daemon had no bound: a daemon that accepted the request and never answered left the CLI with no done line and no fallback (R39, found by the merged-PR conformance check of #73). The daemon now sends each waiting CLI a beat (an empty line) whenever one of its checks makes progress: a streamed line, or a poll of a running copy. The CLI's watchdog counts its polls in which nothing came; after 60 s of them the daemon has stopped answering, and the CLI notes it and runs the check cold, which reports what the daemon would have (incremental equals cold). ORACLE_GAPS_TEST_WAIT_MARK now marks that bound and holds the request there, and the daemon-reply pause waits for ORACLE_GAPS_TEST_PAUSE_ARM when it is set, so the oracle can warm a daemon before it holds a reply. Co-authored-by: Isaac <no-reply@databricks.com>
…(F-D) F-D of #55 (a spun run is stored under the inputs it read) landed with no RED and no hostile mutant (D50): no history made a cut run read a file that its file's unmutated run does not read. spinData, a drawn field, makes file b's verdict tests that read the data spin on the function they test where its output differs and the data is open. Each mutated copy that changes the output then reads a file b's unmutated run never reads, and its call budget cuts it. A constructed history (appended last) edits the data, closing it: the cold run reports those mutants unjudged, so a daemon that stored the spun runs under b's known inputs alone reuses them stale. Co-authored-by: Isaac <no-reply@databricks.com>
…er a link root With warmReplaced drawn, the fixed seed's base of "an edit sequence through the daemon" drew a replaced warm daemon and lost "a warm daemon's connection was reset", a state that history gates; it now keeps the warm daemon. With testRoot drawn, a mid-run edit under the link root looked for its target under tests/, where the fixture keeps no file: the driver now names the file where the fixture keeps it (fixturePathOf). Co-authored-by: Isaac <no-reply@databricks.com>
…t read F-D (#55) stored a run that spun alone under the inputs its closing row names, but a copy that spun in a batch kept no inputs of its own, so the daemon stored it under its file's known inputs only. A copy whose mutant reads a file that the file's unmutated run never reads was then reused after an edit to that file, while a cold run judged it again (I8). The batch's spun unit now carries its closing row's inputs, as every other unit does. Co-authored-by: Isaac <no-reply@databricks.com>
…stories Co-authored-by: Isaac <no-reply@databricks.com>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
|
…r in a batch The alone path and the batch path built a spun unit separately, so the batch path lost its inputs (fixed in the previous commit) while the alone path kept them. Both now build it with spunRun, so the batch history that witnesses F-D judges the one rule both paths take. Co-authored-by: Isaac <no-reply@databricks.com>
…uctor Co-authored-by: Isaac <no-reply@databricks.com>
…he lazy state From #57's findings: a drawn stop after the completed line holds the run's exit (ORACLE_GAPS_TEST_HOLD_EXIT_MS), a drawn stop while the config warms waits at the warm probe (ORACLE_GAPS_TEST_WARM_PROBE), and on win32 every drawn stop goes through the test stop file (ORACLE_GAPS_TEST_STOP). Under "Randomness wins" no drawn field switches a hook on, so the histories map reads those stops as stops while the witness spins, and drops every stop on win32; only the halves of "every outcome class" stop with a hook. The map reads the same draws, so no field's draws move. The canary gains a test: at three seeds, 500 histories each, every hook field holds its neutral value and no stop switches a hook. Two fixed-seed failures from spinData: a flaky test of file b spun while the model leaves flaky tests out (the spin now skips flaky tests, as a hang does), and "an edit to a module that only mutated runs load" drew spinData and lost "edit lazy changed the cold result", which it gates; it now pins spinData off. Co-authored-by: Isaac <no-reply@databricks.com>
On win32 the histories the oracle takes stop nowhere, so their stopRuns fingerprint is no draw; the canary keeps POSIX's line and its hook test checks win32's. Co-authored-by: Isaac <no-reply@databricks.com>
commit: |
# Conflicts: # packages/oracles/oracle-gaps/gap-report-oracle.property.test.mjs
…name that extends it #73's fix class 2 (isWithin in place of a prefix test in the driver) had no hostile mutant (D58): the driver's CLI path and package directory were always spelled alike, so a prefix test agreed with isWithin. A drawn field, siblingDir, places an in-repository package in a directory beside the CLI's whose name extends it (oracle-gaps-sibling-*), and a constructed history, appended last, refuses a run there. The driver now judges the inside label against the package's path from the CLI's directory, so a prefix test that attests it for the sibling fails loudly. A drive-letter case or separator difference is win32-only and is not generated. Co-authored-by: Isaac <no-reply@databricks.com>
KyleAMathews
marked this pull request as ready for review
November 16, 2026 16:52
The merge gate's F1: drawn steps still took dieMidCheck and resetBeforeRead, which arm the daemon pause hooks (13, 6 and 9 of 500 histories at the canary's seeds), and the canary counted pauseEngages: true as neutral. The canary now names the hook-arming steps and stops by value, and a win32 stop's hook through a hookStops field. It fails on this commit's model (RED), as the next commit's fix makes it pass. Co-authored-by: Isaac <no-reply@databricks.com>
…every platform The merge gate's F1 and F8. A drawn history no longer takes dieMidCheck or resetBeforeRead, which arm the daemon pause hooks (v21.2 I16): the histories map reads them out of the drawn steps, as it reads the hook stops, and the constructed histories that gate their states keep them. The win32 stop mapping moves out of the model: a new constant, hookStops, which only the halves of every outcome class set, tells the driver to run a history's stops on win32, where each goes through the test stop file. The same seed and path now give the same history on every platform (v21.2 R4). Co-authored-by: Isaac <no-reply@databricks.com>
…re it runs ORACLE_GAPS_TEST_PAUSE_AT=daemon-check holds the first armed check before it runs, until the file ORACLE_GAPS_TEST_PAUSE_RELEASE names exists: the driver releases it, never the clock. Inert unless set. The next commit's witness uses it to hold a daemon before it writes a run's --out (merge gate F2, F7). Co-authored-by: Isaac <no-reply@databricks.com>
…t runs The merge gate's F2 and F7. The held check now waits at the daemon-check pause, before the daemon writes anything into the run's --out, so only a real cold check gives the run its report. After the CLI has fallen back, the driver poisons the run's results.json, releases the daemon, and runs one more check through it. A daemon that then runs the abandoned check writes over the poison, which the incremental check reports under cold fallback: the run's printed summary would no longer match its --out. Co-authored-by: Isaac <no-reply@databricks.com>
… check The merge gate's F2: after a fallback the daemon still ran the abandoned check and wrote into that run's --out, so the run's printed summary no longer matched its artifacts. On the bound, the CLI now ends the daemon that holds the package's lock (the one that listens on the socket it reached) before it checks cold, so nothing else writes into the run's --out, and no later check waits for that daemon again. F5: the bound is the socket's own idle timeout (setTimeout), a wall-clock bound that changes the run's path and never its outcome (v21.2 I15, R39), in place of a counting interval. F3 is recorded in the Limits with issue #135. Co-authored-by: Isaac <no-reply@databricks.com>
…side its test The merge gate's F4: a batch with an error outside any test runs each copy alone, and a spun copy's unit then comes from runUnit's spunRun, which no history reached. A drawn field, spinThrows, makes a copy that spins on open data first schedule an error that fires after its test, and a constructed history (appended last) edits the data under it. The batch-path history pins spinThrows off, so both paths stay witnessed. Co-authored-by: Isaac <no-reply@databricks.com>
Found by the F-D alone-path history (merge gate F4): Vitest's state keeps a run's errors outside any test for every later run (its own CLI clears them per run), so after one such error every later batch in that context read it as its own and ran its copies alone, and each batch retained the errors (the bounded-retention law, errorsSet, failed at 08e967a). Each run now starts with none, and a batch drops its errors once its units are plain data. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
…ore a rewritten --out Co-authored-by: Isaac <no-reply@databricks.com>
# Conflicts: # packages/oracles/oracle-gaps/gap-report-oracle.property.test.mjs # packages/oracles/scripts/e2e-shards.mjs
…able The merge gate's F6 sweep: random seed 2039724679 gave "a lazy module loaded before a hang in the same batch" a base that drew spinData, whose spin pre-empts the lazy-gated verdict, so it lost "a lazy edit changed the cold result after a batch was cut off", which it gates. It now pins spinData off, as the other lazy history does. Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR closes the conformance gaps that merged PRs #55, #73 and #103 left on main: v21.2 D50, D58, D59 and D61. It also closes one item from #57's findings (D68). It was revised for the v21.2 merge gate (
reviews/pr131-merge-gate.md, F1 to F8). It is oracle-first throughout and merges origin/main at 2552d6a (#121, #128).What changes
1. Randomness wins (R54, I16, R4; D59, D68; gate F1, F8). Each field that changes what a history runs is drawn, unless it switches a test-only hook on.
warmReplacedresetBeforeRead, which drawn histories no longer take (see below), so only constructed histories exercise it.testRoot,loadPluginspinData(new)spinThrows(new)siblingDir(new)oracle-gaps-sibling-*(see 4).tornRows,rejectBatchORACLE_GAPS_TEST_TORN_ROWSand_REJECT_BATCH.replyHeld(new)daemon-checkpause.hookStops(new)ORACLE_GAPS_TEST_STOP.dieMidCheckandresetBeforeReadORACLE_GAPS_TEST_PAUSE_ATand its marks. The histories map reads them out of drawn steps.afterandwarm_HOLD_EXIT_MSand_WARM_PROBE, so a drawn one becomesrun.pauseEngagesprocess.platform, so one seed and path give the same history on every platform (F8, R4).c58783d's model ("expected [dieMidCheck, resetBeforeRead] to not include dieMidCheck", after "hookStops undefined").2. F-D (D50; R9, I8, I12; gate F4).
spinData: a mutated copy reads a data file that its file's unmutated run never reads, and its call budget cuts it.971f22e: a spun copy in a batch kept no inputs of its own. One constructor,spunRun, now serves both paths.spinThrowsadds an error outside any test, so the batch runs its copies alone and the alone path stores the spun run (F4).f22ef48:errorsSet).3. A daemon request with no bound (D61; R39, P10/I7, I8, I15; gate F2, F5, F7).
socket.setTimeout(WAIT_MS), 60 s; F5). It is a wall-clock bound: it changes the run's path, never its outcome. It is not a count. I15 does not cover it, because it decides no kill, survival or stop.oracle-gaps: note the package's daemon sent nothing for 60 s; this check runs cold, and runs the check cold.--out, and no later check waits for that daemon again.ORACLE_GAPS_TEST_WAIT_MARKmarks the bound and pauses there.daemon-checkpause holds a daemon's first armed check before it runs, until the driver's release file exists.daemon-replyhonoursORACLE_GAPS_TEST_PAUSE_ARM.4. #73 class 2 (D58).
siblingDirplaces an in-repository package in a directory beside the CLI's, whose name extends it.relative(how.cli, dir).Evidence
Law:
--outagree. The held witness follows C-30: it is released by the driver, never by time.Reach (fixed seed 20261105, all three Linux shards):
52f9cdf(before the merge of fix(oracle-gaps): prefix an internal error's lines, and judge the stream law for them #128): all three fixed-seed shards pass, in 204, 283 and 324 s.a check whose daemon stopped answering ran coldan edit to data that only a spun copy read changed the cold resultan edit to data that only a copy spun alone read changed the cold resulta refused run whose package lies beside the CLI's directory, under a name that extends itan edit sequence through the daemon, the disarmed pause, the reset after a replaced warm daemon, and the held check).a warm daemon's connection was reset, in the daemon edit sequence (warmReplaced: false);edit lazy changed the cold result, in the lazy-module history (spinData: false);a lazy edit changed the cold result after a batch was cut off, in the lazy-hang history (spinData: false; found by the F6 sweep);siblingDir: false);spinThrows: false).F6, random sweep (every
random seed:unit at52f9cdf, Arcamc-f6-rand-aandmc-f6-rand-b):addd378, and its replay now passes (mc-attr-lazy-head).random seed: a declaration fault: setupThrowsfailed on an abort stop under a throwing setup file. The same seed fails the same way on main 2552d6a (mc-attr-setup-main), so it is main's (R42). It is filed as oracle-gaps e2e: an abort stop owes SIGABRT under a setup file that throws (main and #131) #137 with its full history.RED (parents; local, under
/tmp/e2e-lock.sh):0f10264: lawcold fallback, "exited 143 when the driver's backstop stopped it, and the cold run exited 0".4fbe231: lawcold fallback, "the daemon ran the check whose CLI had fallen back to a cold check, and wrote into that run's --out".d0e32ec: lawincremental equals cold, "after fsData, the daemon's report differs from the cold run's". The daemon gavetimeout, and the cold runsurvived.08e967a: lawbounded retention, "retention lines break the law: retention (errorsSet)".c0f034b: fails on its own commit's model.GREEN:
21d37bc, after the merge of fix(oracle-gaps): prefix an internal error's lines, and judge the stream law for them #128 (local).d1e233b.addd378:Mutants (each killed unless stated):
send(…, false), no bound (atdaa8d02): cold fallback, "exited 143 …".d1e233b): cold fallback, "the daemon ran the check … and wrote into that run's --out".spunRunwith no inputs: incremental equals cold.spunRun(…, {}), with no closing row (atdaa8d02): incremental equals cold, "after fsData, the daemon's report differs".invalidateFilewithoutposix(): authored mutants on win32 (run 43881063085).posix()without its win32 gate: path-spelling, lawsegments.isWithin: "recorded a refused run whose package lies inside the CLI's directory, but ../oracle-gaps-sibling-epxoBI lies outside it".replyHelddrawn: rejected.Code weight (gzip without line comments; test-time package, C-12, K14 open):
spunRun(which removes a duplicate constructor),alive, andclearErrors.Limits:
--stop-daemonagainst a hung daemon has no bound.w_fastturncounterexample under a drawn root and load plugin, seen once.loadPluginas a hook field. Here it is drawn, because it is fixture text (the coordinator's ruling). Whichever of feat(oracle-gaps): write the stream to <out>/stream.log and name it in the first line #57 and fix(oracle-gaps): close the merged-PR conformance gaps (drawn fields, F-D, cold fallback) #131 lands second reconciles the two lists.cc6af3dand21d37bccarry git's default message, without the Isaac trailer; they were not amended.This pull request and its description were written by Isaac.