Repository navigation
fix(oracle-gaps): supervise the check on win32; log each CLI run in the e2e driver - #79
KyleAMathews wants to merge 7 commits into
Conversation
… the e2e driver Test-side only: GAP_RUN_LOG names a file the driver appends one line to per CLI run (runCli and runClisAtOnce): its duration, status, signal, and whether the driver's 300 s kill timeout ended it. Measures where the daemon history's time goes on Windows (about 900 s sits outside any finished check). Diagnostic job daemon-history-runs prints the summary; remove before merge. Co-authored-by: Isaac <no-reply@databricks.com>
|
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
…ed path On win32 the check runs as a supervised child too: the child is not detached, the supervisor watches its exit and writes the done line, and a stop ends the child's tree with taskkill /T /F (signalTree). An out-of-memory abort or a crash on win32 therefore ends with one done error line on both streams and exit 3, as on POSIX (it wrote none before: a silent failure). The in-process unsupervised path (its stop line, its startLog, and the ORACLE_GAPS_TEST_STOP handler outside the supervisor) is removed, since every check is supervised now. The abort witness is the CLI's own hook, ORACLE_GAPS_TEST_ABORT_AT=check, which aborts the check's process after the session has begun (where a V8 out-of-memory abort would strike) instead of a test's kill in a worker thread: process.abort() on win32, a SIGABRT to itself on POSIX (process.abort prints a native backtrace, which is no line of the stream). The fixture's test that killed its own process is removed from the model. win32's done line names the exit code, since no signal can end a win32 process. Law: a check run ends with one done line on both streams and an exit code the supervisor names (register P10; #74). Reach: the 'every outcome class' history's abort run, on both platforms. RED: the test-side commit's Windows run (2b11e2b), where an abort gives no done line. GREEN (POSIX, local): 'every outcome class' and 'an edit sequence through the daemon' pass (fixed seed 20261105). Mutant: no supervisor on win32 (the abort run has no done line). Limits: taskkill /T reaches the tree only while the child lives (a grandchild orphaned earlier is not reached; a Job Object would be the fix); the win32 abort line names the exit code, not a signal. Co-authored-by: Isaac <no-reply@databricks.com>
commit: |
…n win32 The first win32 run of the supervised check (run 43612786521) failed the greppable-stream law: the note named the log as out\stream.log, where the Windows paths rule spells it out/stream.log. Co-authored-by: Isaac <no-reply@databricks.com>
…tory in forward slashes Co-authored-by: Isaac <no-reply@databricks.com>
…bserved as SIGINT The driver killed the CLI child directly on win32, where kill is TerminateProcess: the child is the supervisor, which then wrote no done line (RED: the warm stop in run 43612786521, 'stderr has 0 done lines and ends with the note'). A stop there now goes through ORACLE_GAPS_TEST_STOP, which the supervisor takes as a SIGINT, and the run is observed as SIGINT. The SIGTERM state is required on POSIX only: win32 has no SIGTERM a process can receive (its kill is TerminateProcess); the reason is in Limits. Scratch names: namedScratch names each path in every spelling the CLI can print it in, derived with realPathOf (the CLI's own rule for its paths), not a list of hand-written variants (the RED is the raw path in run 43617446527). GREEN (POSIX, local): 'every outcome class' and 'an edit sequence through the daemon' pass (fixed seed 20261105). Windows: pending. Co-authored-by: Isaac <no-reply@databricks.com>
…t hook Removes the GAP_RUN_LOG per-run log and the daemon-history-runs job (their numbers are in the PR body). Documents ORACLE_GAPS_TEST_ABORT_AT in the README's contributor paragraph: process.abort() on win32; a SIGABRT to itself on POSIX, which is the same state without the native backtrace process.abort() prints there. Co-authored-by: Isaac <no-reply@databricks.com>
…x/win32-supervisor Keeps #57's claim and mutex rules and its HOLD_UNTIL release, and this branch's supervisor, taskkill tree stop, ABORT_AT abort hook, and naming. The driver's after-stop hold moves to HOLD_UNTIL (the after-release file); no HOLD_EXIT_MS remains. The win32 stop path and #57's release both stay in stoppedRun. POSIX fixed-seed 20261105 'every outcome class' and 'an edit sequence through the daemon' pass locally on the merged tree. Co-authored-by: Isaac <no-reply@databricks.com>
…n win32 The note line printed join(out, stream.log), which on win32 has backslashes, so the greppable-stream law failed on Windows. The register's Windows-paths entry (2026-11-13) requires forward slashes for report paths. RED: the Windows fixed-seed end-to-end jobs on 5aaa3a1 (CI run 43646266358, jobs 132604647764, 132604647986, 132604647967) failed "the run's first lines are [oracle-gaps: note log <package>\out\stream.log, ...], not ... out/stream.log". The note is posix(path) (paths.mjs), which is the identity on POSIX. The expected note converts backslashes only on win32, as #79 does (678b643, 9626f00). Co-authored-by: Isaac <no-reply@databricks.com>
…; win32 backslash rewrites gated Review of #73 (d7f437b), items 1, 3, 5, 6: remove the idle profiling diagnostics from capture-setup.mjs (the blocked map, the blocker ternary, longest, and the ORACLE_GAPS_IDLE gate, which is outside the test namespace); the sampler is back to its original combined condition. The daemon driver no longer holds for a guessed 4 s and no longer kills by SIGSTOP. A daemon pauses at a test-only point (ORACLE_GAPS_TEST_PAUSE_AT: daemon-reply holds its reply, daemon-read holds its first connection unread) and writes its pid to ORACLE_GAPS_TEST_PAUSE_MARK. A dieMidCheck CLI writes ORACLE_GAPS_TEST_WAIT_MARK once a request has waited ORACLE_GAPS_WAIT_MS for its reply. The driver kills the daemon when the marks say so. Backslash rewrites in namedScratch, readRun, and the done line use posix(), which acts only on win32 (register: Windows paths; POSIX names are unchanged). The win32 abort comment and Limits say the win32 witness comes with #79. README: the test-only hooks, including the pause points, the marks, and STOP. RED and GREEN for the driver change: see the PR body (Arclet runs). Co-authored-by: Isaac <no-reply@databricks.com>
…p to #58) (#73) * ci: add a non-blocking Windows job for the oracle framework and gap CLI The framework, the eslint plugin, and the gap CLI have never run on Windows, and users report path problems there. Two diagnostic jobs run on windows-latest: the unit suites (framework, eslint plugin, gap fast oracles) and the gap CLI end to end (a fixture package written at run time, the plain and daemon paths, and one fixed-seed e2e unit). Both are continue-on-error and each step continues on error, so the first failure does not hide later ones. Neither is a required check or a dependency of preview. Co-authored-by: Isaac <no-reply@databricks.com> * ci(windows): diagnostic split of the fast suites, fixture package.json Diagnostic only, not a fix. The first run showed the gap fast suites and the fixed-seed end-to-end unit hang on windows-latest (cancelled at the job timeout, with no output after the static oracle and the e2e calibrations started). The fast suites now run one file per step with a step timeout and verbose output for the static oracle, so the hang is located. The fixture gets the package.json it was missing (the CLI reported ENOENT for it). The job timeouts are raised to 45 minutes so the split steps can finish. Co-authored-by: Isaac <no-reply@databricks.com> * ci(windows): diagnostic replay, error-log dump, and discovered-config run Diagnostic only. Adds a bash replay of the first failing static unit (its verdict is not printed by the run), a run of the fixture with the discovered config instead of --config, and a dump of the gap CLI's error.log files from the OS temp directory, which the daemon run's failure points to. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): one path rule for Windows, and shell-aware replay lines Forward slashes at entry, native only at the file system. Paths enter the CLI as argv, as Vite and Vitest report them, and as realpath returns them; comparisons and keys use one spelling (oracle-gaps/paths.mjs). - The config wrapper imports the config by a POSIX specifier. A backslash in an ES module specifier is a character, so every run with a config was refused on Windows (check.mjs, wrapper). - Mutant copies are keyed and served in forward slashes, as Vite resolves and Vitest reports them. The copy map was keyed by native paths, so the copies were never served on Windows. - Module IDs are kept by isAbsolute, not by a leading slash: a Windows ID is `C:/...`. - The package root, the located packages, and the source names use the native real path on both sides (realPathOf). The JavaScript realpath kept 8.3 short names such as RUNNER~1 and refused every owns entry. - The daemon listens on a named pipe on win32 (no socket file to remove), keys by the same real path as the CLI, and its timeout names the end of its log. - A package with no package.json is refused, not an internal error. - Replay and rerun lines print PowerShell and cmd forms on Windows; the POSIX form is unchanged. Co-authored-by: Isaac <no-reply@databricks.com> * test: portable npm spawns, a segment rule for production files, native realpath - npm is an npm.cmd shim on Windows; spawnSync runs it through a shell there (package.test.ts, shipped-citations.test.js). - The probe-form oracle's model decides production by a directory named src, split on either separator, not by a substring with `/`. Production was right; the model was wrong on Windows. - The owns test uses the native real path, as the CLI does. Co-authored-by: Isaac <no-reply@databricks.com> * ci(windows): sharded Windows jobs in place of the diagnostic ones The diagnostic steps are gone. The Windows jobs mirror the Linux shards: the unit suites in one job, the static gap oracle and the fixed-seed end-to-end oracle in three shards each (pr-shards.mjs), and the gap CLI on a fixture, including the discovered config and the daemon. Each job has a 15-minute backstop. The jobs are still continue-on-error until every one is green; that line is removed then. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): give TypeScript and the owner map one spelling of a path The owners and the instrumenter named files with the platform's separators, and TypeScript matches a file's name as given, so on Windows the owner map keyed `src\x.js` while the sources were `src/x.js`, and CommonJS sites lost their functions. The compiler now gets forward-slash names, and owner keys are forward-slash (schemata.mjs, owners.mjs, analysis.mjs). A path's drive letter is capitalised in the one spelling (paths.mjs), so a drive's case never decides a path. New oracle: gap-report-path-spelling (a metamorphic check). Law 1: the instrumenter gives the same mutant ids for a source under any spelling of its path. Law 2: relativeWithin answers from the segments alone. Evidence block - Law: a path's separators and a drive letter's case do not change what the gap CLI decides (paths.mjs, Windows support, 2026-11-13). - Reach: 200 generated histories (fixed seed 20261113 and random), over four module forms, drives in both cases, mixed separators, short and spaced names. Reach is not declared (states: []); the histories are not measured as reaching a state. - RED: on the parent commit (72b9495), the Windows jobs of https://github.com/TanStack/oracles/pull/58 failed: the config wrapper import (check.mjs), the copy map, the owner keys (owners.mjs), and the 8.3 short names. On Linux the enumeration law does NOT reproduce the Windows failure: the instrumenter gave the same ids on the parent. The Windows failure is a platform path.relative difference, which Linux cannot produce. The RED for those classes is the Windows CI run. - GREEN: the Windows jobs (see the PR). Local, on macOS: the new oracle 5 of 5, the framework 114 of 114, the eslint plugin 7335 of 7335. - Mutant: relativeWithin as it was before (raw string comparison) fails the relation checkpoint; dropping the drive capitalisation fails it too. Both run on the parent tree with the oracle and both fail (exit 1). - Limits: no declared reach; Law 1 uses the forward-slash spelling as its reference, so it claims only that the spelling changes nothing; owners, realpath, and OS short names are judged by the Windows end-to-end run, not by this oracle. Co-authored-by: Isaac <no-reply@databricks.com> * ci(windows): P10 done-line, refusal, early-stop, and per-shell replay witnesses - P10: the Windows CLI runs assert the done line on both streams, and that it matches the exit code (scripts/windows-witness.mjs done). - A package with no package.json is refused with exit 2 and a done line that names it (Windows witness). - One early stop: SIGINT once the CLI prints its first line; the run must end with the stop's done line on both streams, or with no output (#57). - Per-shell replay (framework, accepted by the coordinator): a failing property's PowerShell and cmd lines each reproduce the failure in their own shell. The POSIX line is run in PowerShell and must not reproduce (the RED). A run that printed only the POSIX line fails here (the mutant). - The tsc link lookup in package.test.ts reads npm's Windows shim, not a symlink. - The eslint plugin suite runs even when the framework suite fails. - Job timeouts are 20 minutes, as high backstops (2026-11-12). Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): oracles read the platform's replay lines, and scratch packages have a manifest Windows CI failed the classify oracle's baseline law: the CLI prints a PowerShell and a cmd replay line there, and the law read only the POSIX `replay:` line (Difference: "the replay of tests/f0.test.ts is undefined"). The law is unchanged: each replay line the platform prints must read as `vitest run <file> -t <pattern>` in its own shell, and the owed test is the one it selects. On win32 the law owes both per-shell lines; elsewhere the POSIX line (replayLinesOf, powerShellWords, cmdWords). The static oracle's scratch packages had no package.json, which the CLI now refuses (args.mjs). Each fixture now writes a minimal manifest; its content and its histories are unchanged. Evidence block - Law: the replay line of each shell reads as the owed command (Law 9, baseline; the CLI's replay lines, stream.mjs). - Reach: the existing classify histories (276 tests); no new histories. - RED: Windows run 43478073715, classify shard 3 (baseline law, 11 violations) and the static shard 1 refusal (the missing manifest). - GREEN: local POSIX classify 276 of 276. The win32 branch runs in the Windows job (see the PR). - Mutant: not run; the win32 branch is exercised only by the Windows job. - Limits: the win32 branch is checked on Windows only, not by the oracle's own generated histories. Co-authored-by: Isaac <no-reply@databricks.com> * fix(windows): forward-slash report paths, cmd quotes, six shards, and shipped citations Run 43483101749 (Windows, the first with the witnesses) showed: - The CLI printed `report <pkg>\out\summary.md` and `tests\a.test.ts` in the done line and in the heartbeat. Report paths and display paths are now forward-slash (posix, check.mjs, ending.mjs), as the register's keys rule requires; the file system still gets native paths. - The classify oracle's cmd reader rejected `\"` inside a double-quoted argument, which Node's argument parser reads as a quote. It now reads it so. - The static and classify oracles' second scratch package had no manifest. - Shipped citations joined paths with backslashes, and npm names files with forward slashes. The comparison now uses forward slashes. - The refusal witness searches both streams (a refusal prints to stderr). - The static and fixed-seed e2e took 3 to 6 times their Linux time on Windows, and the Linux shards hit the limit. Windows runs six shards of each (pr-shards.mjs `oracle-gaps-static-windows`, `oracle-gaps-fixed-seed-windows`, checked by `pr-shards.mjs check`: each test in exactly one shard). The classify, probe-form, mutant-validity, and path-spelling oracles run in their own job. Evidence block - Law: report and display paths are forward-slash; a cmd line reads its quoted quotes as Node does (register: deterministic replay; the CLI's keys rule). - Reach: the existing e2e and static histories, unchanged. - RED: run 43483101749, which printed backslash report paths and failed classify on `\"`. - GREEN: pending the next Windows run; local POSIX suites pass (classify 276, shipped citations 2). - Mutant: not run for this commit. - Limits: the Windows e2e and static shard timings are measured on the next run before the budget claim (P12) is made. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): keys and report paths are forward-slash in the CLI and its models Run 43493431349 (Windows) failed the static and e2e oracles on report paths: the done line names its report in forward slashes (the register's Windows paths rule), but the model built the expected line from the native output directory, and the declared oracle files were keyed by native relative paths (`tests\a.test.ts`), so a heartbeat's file was not an owner. - analysis.mjs, check.mjs: declared oracle files and copies are keyed by their forward-slash relative path (relativeWithin). - gap-report-model-oracle.mjs: the model's expected done line is forward-slash (inline, the model does not import production). - shipped-citations.test.js: the rejection test's strings are forward-slash. Evidence block - Law: keys and report paths use forward slashes on every OS (register, Windows paths, #58); filesystem calls get native paths. - Reach: the existing static and e2e histories; no new histories. - RED: run 43493431349: static 40 refusal-ending violations, e2e 17 baseline lead, 12 report-shaped ending, 4 streaming, 3 greppable stream. - GREEN: pending the next Windows run. Local POSIX: shipped citations 2 of 2, CLI lints clean; the Linux keys are unchanged. - Mutant: not run. - Limits: the Windows e2e and static timings are measured on the next run. Co-authored-by: Isaac <no-reply@databricks.com> * ci(windows): the replay witness runs after a failed stop witness The early-stop step failed on Windows (SIGINT ends the process through TerminateProcess, so the CLI's stop handler never runs; the run ended with a null exit code after its done line), which skipped the replay witness behind it. The replay witness now runs regardless, so each witness reports. Co-authored-by: Isaac <no-reply@databricks.com> * ci(windows): print the replay output the witness reads (diagnostic) The replay witness found no per-shell lines in the fixture's failing output. This prints the first lines of that output in the same step, so the next run shows what vitest printed. Diagnostic, to be removed with the witness fix. Co-authored-by: Isaac <no-reply@databricks.com> * ci(windows): the replay fixture declares its fixed seed The replay fixture failed to load (TypeError reading fixedSeeds[0]), so the witness saw no failure. It now declares fixedSeeds: [1]. Co-authored-by: Isaac <no-reply@databricks.com> * fix(windows): the stop law on win32, the stop hook, the argv cap, and forward-slash done lines - Stop (#57, all-or-nothing) on the unsupervised win32 path: a stop of a run that has started now writes its `oracle-gaps: done stopped (<signal>)` line on both streams and exits 128 + signal, as the supervisor does. A stop before the run starts still prints nothing (index.mjs, runStarted). - Test-only stop hook: when the file ORACLE_GAPS_TEST_STOP names appears, the CLI takes the same stop as a SIGINT (index.mjs). The Windows stop witness uses it (a non-console parent cannot deliver Ctrl+C there; child.kill on win32 is TerminateProcess, which is a SIGKILL-like limit, so no line is owed). - Static oracle: the `unknown` long-argument case uses 30,000 characters on win32. Windows caps a command line at 32,767 characters; a longer one never starts (status null). Narrowing, recorded in the PR: the POSIX runs keep 100,000. - e2e and static model: the expected done line's report path is forward-slash (inline in the model; the e2e corruption check likewise). - The diagnostic Get-Content line is removed. Evidence block - Law: the all-or-nothing stop law (#57) on every platform; keys and report paths forward-slash (register: Windows paths; stop window on every platform). - Reach: the existing static and e2e histories; the stop witness is one run. - RED: run 43498457502, stop witness `stop was not all-or-nothing (signal SIGINT, exit null)`; static `with unknown arguments, the refusal exited null`; e2e `the last stdout line is ... report D:/...`, not `...\out\summary.md`. - GREEN: pending the next Windows run. Local: the CLI lints and loads; the stop witness is queued on the e2e lock for a fixture run. - Mutant: pending (the replay witness, on a throwaway branch). - Limits: OS delivery of Ctrl+C is not exercised on Windows; TerminateProcess is treated like SIGKILL; the win32 argv cap narrows the long-argument case. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the temporary directory's real path for state, socket, and logs The default output directory is under the OS temporary directory, which on windows-latest is `C:\Users\RUNNER~1\...` (an 8.3 short name). The model's expected done line names the package by its real path, so the report path never matched (e2e: `the last stdout line is ... report C:/Users/RUNNER~1/...`). The temporary directory is now taken by its real path (realPathOf) for the state directory, the socket, and the error log. Evidence block - Law: keys and report paths use one spelling; a real path on both sides (register: Windows paths; the realpath rule). - Reach: the existing e2e histories (the done-line law). - RED: e2e run 43515364499, done line with RUNNER~1 vs the package path. - GREEN: pending the next Windows run. - Mutant: pending; the replay witness mutant runs on the throwaway PR. - Limits: the stdout gap-count mismatch (`2 gap lines for 0 gaps`) and the cold internal error in the incremental law are not yet explained. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the e2e fixture's temporary directory by its native real path The e2e oracle realpaths the OS temporary directory with the JavaScript realpath, which keeps the 8.3 name (C:\Users\RUNNER~1). The CLI prints the native real path, so the expected report line never matched. Use realpathSync.native, as the CLI does. Evidence block - Law: one real path on both sides (the realpath rule; Windows paths). - Reach: the existing e2e histories. - RED: run 43515364499 and ee1a5fd: the done line names RUNNER~1. - GREEN: pending the next Windows run. - Limits: the static `output namespace` violations (82 in the last run) are not yet traced. Co-authored-by: Isaac <no-reply@databricks.com> * ci(windows): split the blocking Windows jobs from the non-blocking ones; fix the tsc lookup type - Blocking now: the Windows unit suites, the classify, probe-form, mutant-validity, and path-spelling job, and the gap CLI fixture job with its witnesses (done line, refusal, stop, replay). Their continue-on-error is removed. - Non-blocking, stated in the workflow: the Windows static and e2e jobs, which fail on Windows (tracked in the follow-up PR's body by class). - Linux `Test` failed the typecheck on package.test.ts (the tsc lookup's optional capture). Fixed; tsc --noEmit passes locally. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the static oracle's temp directory by its native real path (output namespace) The output-namespace law (checkpoint `static`) took the OS temp directory by JavaScript's realpath, which keeps an 8.3 short name (C:\Users\RUNNER~1), while the CLI's default output is under the native real path. So every run's output failed "not under the OS temp directory" on windows-latest (82 hits in run 43525797122). The oracle now uses realpathSync.native, the same rule the CLI uses, and drops the raw tmpdir() alternative. Evidence block - Law: a run's default output is under the OS temporary directory's real path, never in its package (maintainer decision, 2026-11-11; register: the realpath rule). - Reach: the existing static histories (the outputs of three runs per history, two of one package and one of another). - RED: Windows run 43525797122, static shards: 82 violations, law "output namespace", "a run's default output is not under the OS temp directory". - GREEN: pending the next Windows run on this branch. - Mutant (Windows-only): the CLI's temporary directory taken raw (tmpdir() without realpath) puts the output under RUNNER~1, which the fixed law rejects on Windows. On Linux the two spellings coincide, so the Linux run cannot kill this mutant; the Windows run is its witness. - Limits: Linux does not exercise the 8.3 spelling. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the e2e driver names a scratch path in both spellings (baseline lead) The e2e driver replaces scratch paths with <package> and <cli> in the CLI's output, but only in the native spelling. The CLI now names its report and package paths in forward slashes (the Windows paths rule), so the done line kept `D:/a/.../.scratch-XXXX/out/summary.md` and the baseline-lead law (18 hits in run 43525797122) read it as not `<package>/out/summary.md`. Evidence block - Law: the CLI's done line names its report in the package's placeholder spelling (law: baseline lead; register: Windows paths). - Reach: the existing e2e histories. - RED: run 43525797122, e2e shards: 18 violations, law "baseline lead". - GREEN: pending the next Windows run on this branch. - Mutant: a CLI that prints the native spelling (backslashes) in the done line is now caught by the same check, since the placeholder is only applied to the spelling the driver names; not run in CI. - Limits: the driver names two spellings of each scratch path (native and forward-slash), not every possible form. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): mutants' owner files are forward-slash in the e2e driver (streaming) The e2e driver's mutant observations named each owner file with path.relative, which returns backslashes on Windows (tests\a.test.ts). The CLI names a heartbeat's file in forward slashes, so every heartbeat failed "names sut.ts:M1 in tests/a.test.ts, which does not run it" (the streaming law, 4 hits in run 43525797122, plus greppable-stream instances). Evidence block - Law: the heartbeat names a test file that runs the mutant (streaming law); keys and report paths are forward-slash (register: Windows paths). - Reach: the existing e2e histories with a slow owner. - RED: run 43525797122, e2e shards: "a heartbeat names ... which does not run it". - GREEN: pending the next Windows run on this branch. - Mutant: not run. - Limits: the stdout gap-line count mismatch (`122 gap lines for 121 gaps`, greppable stream) is not yet explained; left for the next run. Co-authored-by: Isaac <no-reply@databricks.com> * fix(windows): stop witness, replay quoting by the shells' grammar, and review items F1-F8 Review of #58 at ebc39e6 (NOT READY). This commit fixes: - F1 (stop never stops): the witness's stop file is absolute (the CLI's cwd is the package), a stop requires the stopped line on both streams with exit 130, and the stop lands 400 ms after the run starts (the fixture's output is written in one burst near its end, so the first output line is no clock). - F2/F3 (replay quoting): PowerShell reads U+2018 to U+201B as quotes, so every such character is doubled inside single quotes. cmd toggles quoting on every `"` and expands `%`; a name with either has no cmd form, and its cmd line says `use the PowerShell line for this name` (stream.mjs, src/oracle-property.ts). The classify oracle reads each shell's line by that shell's grammar (the PowerShell quote set; cmd's quote toggling, `^`, and `%`), checks the win32 printer on every host, and generates adversarial names (curly quotes, `& | < > ( ) ^ % !`, quotes, backslashes), drawn as the last field of the test record (the fixed seeds' later draws shift; recorded here). - F(f) (POSIX hook): the test-only stop hook lives in the supervisor and goes through its stop, which forwards to the child; a hooked stop gives `done stopped (SIGINT)`, exit 130 (local run: outcome (a)). Unsupervised runs keep the hook in the process. - F5: posix() acts only on win32; on POSIX a backslash is a name's character. - F6: check.mjs realpath of workspace links, report.mjs results path, and ending.mjs error log path are forward-slash and use the native real path. - F7: the blocking Windows jobs are in preview's needs. - F8: the non-blocking jobs point at #72. - The Windows replay witness runs each printed PowerShell line and its cmd line, for two fixtures with hostile names (curly quote and metacharacters; a double quote and `%`). Evidence block - Law: the printed replay line replays in the shell it names (register: deterministic replay across shells; each printed line must replay on Windows); a hooked stop is a stopped line with exit 130 (register: stop window on every platform). - Reach: the classify oracle's 276 generated histories (new adversarial name dimension, drawn last); the Windows replay fixtures; one stop per run. - RED: the parent printer (ebc39e6, with only the platform parameter added) fails the new classify oracle on two names: `"vitest run tests/f0.test.ts -t '^’ t0$'"` (the PowerShell line ends its quote at U+2019) and the cmd line for `q"t` (`which the shell does not read as plain words`). Run locally, exit 1. - GREEN: classify 276/276 on Linux (current code); POSIX stop witness outcome (a), exit 130 (local, under the e2e lock). Windows: pending CI on this head. - Mutant: the parent printer is the mutant (the curly-quote and cmd forms before this commit); the RED run above is its rejection. - Limits: - the stop timing (400 ms after spawn) is an assumption about the fixture's length; a run that ends first is reported as a failure, not passed; - cmd's `!` is literal (delayed expansion off), an assumption; - the classify fixed seeds' later draws shift (the new field is drawn last); - the win32 argv cap, the pipe name's machine scope, the POSIX state paths moving to the temporary directory's real path (an older daemon is found only after its idle timeout), and the unsupervised win32 process-tree exemption (#74) are recorded in the PR body. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): path-spelling oracle generates win32 separators and drives only on win32 The Linux run of 50b4ab5 failed gap-report-path-spelling: posix() acts only on win32 (review F5), so on POSIX a backslash is a name's character and a drive letter's case is a different name. The oracle now generates backslash joints and mixed drive letters only on win32; on POSIX it generates `/` joints and `C`. This narrows the law on POSIX (recorded): the POSIX law is the forward-slash spelling, and the separator and drive-case laws are judged on win32 (the Windows classify job runs the oracle there). Evidence block - Law: a path's separators and a drive's case change nothing the gap CLI decides on win32; on POSIX they are name characters (paths.mjs, F5). - Reach: the same 200 histories, with the separator and drive alphabets by platform. - RED: Linux shard 3 of 50b4ab5: "observed value differs" on the path-spelling oracle (2 of 5 tests). - GREEN: local POSIX 5/5. - Mutant: the win32 mutants run in the Windows classify job; not re-run on POSIX. - Limits: on POSIX the separator and drive laws are not generated. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): event-triggered stop, derived hostile names, and the POSIX F5 witness Coordinator rulings, applied: 1. Stop trigger: an event, not a clock. The witness writes the stop file on the CLI's first stderr line. Measured (2026-11-14): on a pipe the CLI streams its progress on stderr and prints the done summary (stdout) only at the end, so stdout is no start marker. No stream.log exists on this branch; the stderr line is the event. The 400 ms timer is gone. 2. Classify generator: the hostile names are derived from the test's own fields (HOSTILE[(ms+ti) % 7] for one test in four), not drawn. A drawn field would shift every later draw of the fixed seeds. Proof: `git diff origin/main` on the classify oracle changes no `fc.` line (zero), so the generator is unchanged. 3. Path spelling on POSIX: the oracle generates backslash joints and `x:` drives on every platform. The model splits the spelled strings per platform (win32: both separators, case-insensitive drive; POSIX: `/` only, a backslash is a name's character). On POSIX a disk witness creates the backslash name and asserts it exists, realPathOf keeps it native, and posix() leaves it unchanged (F5). This removes the earlier POSIX narrowing. Evidence block - Law: the stop is delivered at a run event and ends stopped (register: stop window on every platform); a backslash name on POSIX is kept native (review F5). - Reach: the path-spelling oracle's 200 histories, now with backslash and `x:` names on POSIX (disk-backed); the classify oracle's existing histories, with hostile names on one test in four. - RED: ebc39e6 with this oracle: 2 of 5 tests fail, "the observed value differs from the owed value" (the relation and disk checks). - GREEN: POSIX local 5/5 (path-spelling); stop witness local POSIX outcome (a), exit 130 (event trigger). Windows: pending CI on this head. - Mutant: the parent (ebc39e6) is the mutant for F5 (posix() rewrites backslashes on POSIX); its RED is above. - Limits: the stdout summary arrives only at the end on pipes (a finding, not a fix here); the stop trigger assumes the CLI prints a stderr line before its work completes (measured on the fixture only). Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): constructed tests keep their names (classify reach) The derived hostile names reached constructed tests too (ms 0 at index 0), which broke the name-relationship reach of two constructed histories ("a failing baseline test whose name another test's contains" and "... ends another test's"). A hostile name now applies to generated tests only (ms >= 1). Evidence block - Law: the constructed histories reach their declared states (the oracle's own reach rule). - RED: de8a373 run: "constructed history ... did not enter a baseline test whose name another test's contains" (Linux shard 3 and the Windows classify job). - GREEN: local classify 276/276. - Mutant: not run. - Limits: as recorded in the previous commit. Co-authored-by: Isaac <no-reply@databricks.com> * fix(windows): Rerun cmd lines, ! as no-form, separators by platform, prefix rule, witness Round-2 review of #58 (NOT READY). Items addressed: 1. Rerun (cmd) fell back on win32 every time: an absent command was passed as undefined and read as "no cmd form". An absent command is now nothing; only a value that cannot be represented has no form (oracle-property.ts commandOf). The witness runs the Rerun lines (as `vitest run` plus the line) in pwsh, Windows PowerShell 5.1, and cmd. 2. `!` is in the cmd no-form set (framework and printer; DelayedExpansion expands it silently). The classify oracle's fallback test follows. 3. The path-spelling generator's original drive and joint domains are restored (the win32 form at b904d57); the POSIX spelling's backslash names and `x:` drives are an appended `wide` record, used on POSIX only. 4. The witness header, the stop failure text, and the replay comments describe the code (event trigger on the first stderr line; the no-form set). 5. isOutside, the owners' node_modules split, and the capture-setup machinery regex split on `\` only on win32; on POSIX a backslash is a name's character. 6. (see 4) 7. Windows PowerShell 5.1 (`powershell.exe`) runs each PowerShell line, as well as pwsh. 8. configureOracles rejects an envPrefix that is not `[A-Z][A-Z0-9_]*`, with a message (the prefix is printed in replay lines). The unit test follows. Evidence block - Law: the printed command replays in the shell it names, or says it has no form (register: deterministic replay across shells); a Rerun line replays too; `!` is a no-form character (coordinator ruling); a POSIX backslash is a name's character (register: Windows paths, as amended). - Reach: the same histories (the win32 domains are unchanged; POSIX uses the appended wide record). - RED (item 1): round-2 review: plain Rerun lines printed "use the PowerShell line" on win32. - GREEN: classify 276/276, path-spelling 5/5, framework 114/114 (locally, POSIX); Windows: CI on this head. - Mutant: the round-2 Rerun behaviour is the mutant (rerunOf passed an absent command as no form). - Limits: U+2019 is not reached by the seeds in the path-spelling or replay histories; the hostile names are exercised by the witness's fixture names. Co-authored-by: Isaac <no-reply@databricks.com> * fix(windows): the witness feeds PowerShell its line on stdin, not as an argument Run 43548823073 (head 2b7b7f3): the Windows CLI job failed on Windows PowerShell 5.1 ("Replay 3 did not reproduce in powershell", a parse exception on the fixture with a double-quoted name). The witness passed the line as a -Command argument, so Node's argument quoting turned the double quote into \" and 5.1 does not read that as an escape. A typed line never passes through an argv: the witness now pipes the line to `-Command -` on stdin, for pwsh and powershell.exe alike. Evidence block - Law: each printed line replays in the shell it names, as a user types it (the register's per-shell rule). - RED: run 43548823073, witness step: "Replay 3 did not reproduce in powershell" (the Windows PowerShell 5.1 parse exception). - GREEN: pending the next Windows run. - Mutant: the argument passing is the mutant (the run above). - Limits: a cmd line is still passed verbatim as an argument (cmd has no stdin command mode here). Co-authored-by: Isaac <no-reply@databricks.com> * diagnostic: the replay witness reports the shell's own error for a line that does not reproduce Run 43551132190 (abc8b45): "Replay 3 did not reproduce in powershell" with no reason. The failure now carries the shell's exit code and output, so the Windows PowerShell 5.1 error for the double-quoted name is visible. Diagnostic; the witness's pass and fail rules are unchanged. Co-authored-by: Isaac <no-reply@databricks.com> * fix(windows): the -t argument never carries a shell-hostile character (option c) PowerShell 5.1 passes an embedded double quote in a native .cmd argument wrongly (run 43553461458: a `"`-named fixture selected no test, exit 0). The coordinator's ruling (option c): `-t` is a coarse regular-expression filter, and the exact test is carried by ORACLE_UNIT where a line sets it. - The `-t` pattern (CLI baseline and framework Rerun) replaces each character a shell reads unsafely (`"`, `%`, `!`, the curly quotes) with `.`: still anchored, only slightly broader. - Framework environment values: PowerShell (5.1 and 7) doubles the single and curly quotes, which works for any name; cmd's `set "NAME=value"` tolerates a double quote, and `%` and `!` have no cmd form (environment values only). - Classify oracle: a CLI line has no environment values, so a cmd fallback line is a breach; the -t argument must carry no shell-hostile character. - Witness: each PowerShell Replay line's ORACLE_UNIT and ORACLE_PROPERTY decode to the printed text (round-trip), in addition to the reproduction in pwsh, Windows PowerShell 5.1, and cmd. Evidence block - Law: a printed command replays exactly the unit it names, in each shell (register: deterministic replay across shells; the coordinator's option c). - Reach: the classify oracle's 276 histories (the adversarial names, derived); the Windows fixtures (`’`, `"`, `%`). - RED: run 43553461458: "Replay 3 did not reproduce in powershell", exit 0, PowerShell 5.1 with an embedded double quote in the -t argument. - GREEN: classify 276/276 and framework 114/114 locally (POSIX); the Windows witness runs on this head. - Mutant: the pre-fix pattern (a raw double quote in -t) is the mutant; the RED above is its rejection. - Limits: the -t pattern is slightly broader for names with a hostile character (the wildcard may select a test whose name differs in that position, which the classify oracle's names cannot produce). Co-authored-by: Isaac <no-reply@databricks.com> * Revert "diagnostic: the replay witness reports the shell's own error for a line that does not reproduce" This reverts commit 69c82ebbafa455a115b3ea628a748bbbd4af4bd0. * fix(windows): cmd env rule, exact witness matching, set-line readers, fixtures (round-3 review) - F1: a cmd environment value containing `"`, `%`, or `!` has no form (the register's rule); `set "VAR=value" &&` is only emitted for a plain value (oracle-property.ts commandOf). Fixtures: one with a curly-quote property and no `"%!` (a cmd set line runs); one with a single `"` in its property (its cmd line points to PowerShell). - F2: the PowerShell decoder's offset is the value's own position, and decoded values are compared exactly with a failing unit's property and unit, as vitest prints them (`FAIL <file> > <property> > <unit>`). cmd `set` lines decode too and must agree with the PowerShell line. - F3 (partial): Rerun lines carry no ORACLE_UNIT: the framework reads ORACLE_UNIT only for a random unit during a path replay, so a fixed-seed Rerun cannot be selected exactly. Reported to the coordinator. - F6: the path-spelling comment states what it means (no history to keep). - F7: the classify readers reject `!` and judge `set … &&` prefixes; the fallback is allowed wherever the printer uses it (a file path with a hostile character). - F9: control characters in rerunPatternOf are escaped as stream.mjs does (by code point, not a control-character regex). - The witness fails when a failing unit prints no per-shell line (FAIL lines are counted per unit). Evidence block - Law: a printed command replays the failing unit it names, in each shell (register: deterministic replay across shells; the set rule). - Reach: the classify oracle's 276 histories; the Windows fixtures. - RED: run 43555531142 and the coordinator's round-3 review at ae44184 (a `"` in a set value broke `&&`). - GREEN: classify 276/276, framework 114/114, locally (POSIX). Windows: this head. - Mutant: the pre-fix env rule (a `"` in a set value) is the mutant. - Limits: Rerun lines are not exact (see F3). Co-authored-by: Isaac <no-reply@databricks.com> * fix(windows): the replay witness counts each printed line once, and runs on the package's PATH Run 43561333340 (9698922), Windows CLI job: "2 random units failed, but 4 Replay lines printed" (vitest repeats a failing unit's error text, so its replay lines repeat), and every line failed with "'vitest' is not recognized" (the rewritten runIn dropped the PATH that adds the package's node_modules/.bin). - Each distinct printed line counts once, in first-occurrence order. - Every shell runs with the package's binaries on PATH. Evidence block - Law: a printed command replays in each shell it names (the register's per-shell rule); each failing unit prints one line (witness rule). - RED: run 43561333340, the witness: "4 Replay lines printed" for 2 random units; "'vitest' is not recognized". - GREEN: pending the next Windows run. - Mutant: not run. - Limits: as in the previous commit (Rerun lines are not exact). Co-authored-by: Isaac <no-reply@databricks.com> * fix(windows): the replay witness pairs each printed line with its cmd line and checks coverage Run 43563021693 (257a5b0): "2 random units failed, but 4 Replay lines printed" and "Replay 4 has no cmd line": a failing fixed unit prints its replay under the random unit's name, so one failing unit can have more than one replay line, and index pairing breaks. A random unit's violation also prints its replay under the Rerun label, so a Rerun line can carry ORACLE_ variables, and running it with `vitest run` in front broke it. - Each distinct PowerShell line pairs with the cmd line printed after it. - A line that carries ORACLE_ variables is a replay and runs as printed; a bare -t line is a rerun and runs as `vitest run` plus its arguments. - Coverage: every failing unit is matched by a replay whose ORACLE_PROPERTY and ORACLE_UNIT decode to its names (a fixed unit by the random unit it replays). Evidence block - Law: every failing unit prints a replay that names it exactly, and each printed command replays in each shell it names (register: deterministic replay; the witness rule). - RED: run 43563021693: "2 random units failed, but 4 Replay lines printed"; "Replay 4 has no cmd line"; "Rerun 2 did not reproduce in cmd". - GREEN: pending the next Windows run. - Mutant: not run. - Limits: a bare Rerun line is not exact (see F3). Co-authored-by: Isaac <no-reply@databricks.com> * fix(windows): restore the replay witness's helpers that the pairing rewrite removed Run 0c42c33 (the Windows CLI job): "ReferenceError: decodePs is not defined". The pairing rewrite replaced the block that held the FAIL map, the PowerShell and cmd decoders, and the no-form constant. They are restored, and a local smoke run of the replay branch on a fabricated output reaches the report (pwsh is absent on this Mac, so its runs fail as expected; no reference error). Evidence block - Law: the replay witness reports a failing unit's printed commands (witness rule). - RED: run 0c42c33: ReferenceError in the replay step. - GREEN: pending the next Windows run. - Mutant: not run. - Limits: as before. Co-authored-by: Isaac <no-reply@databricks.com> * fix(windows): a Rerun line counts only for its own failing fixed unit (F3 ruling b) The coordinator's ruling: Rerun lines stay coarse -t filters (a sibling whose name the anchored pattern matches also runs). The witness now counts a Rerun line only when the failing fixed unit whose exact name it selects shows its own FAIL line in its output, so a sibling's failure cannot satisfy it. An exact selector is tracked in #78; Replay lines select exactly, through ORACLE_PROPERTY and ORACLE_UNIT. Evidence block - Law: a Rerun line runs the failing unit it names, and only it counts (register amendment: Replay lines select exactly; Rerun lines are anchored, coarse filters). - Reach: the Windows fixtures (the `Q"1` unit has no Rerun line in CI; the sibling case is the Limit's example). - RED: none in CI yet; the sibling case is not exercised by the fixtures. - GREEN: pending the next Windows run. - Mutant: not run. - Limits: a sibling case is not in the CI fixtures (recorded in the PR body). Co-authored-by: Isaac <no-reply@databricks.com> * fix(windows): the witness reads a fixed unit's FAIL line (`fixed seed 1: <name>`) Run 43569602268 (bd208ac): "Rerun 1 selects no failing fixed unit" for both fixtures. The FAIL regex expected `fixed seed: ` with no number, but vitest prints `fixed seed 1: <name>`, so no fixed unit was recorded. (The earlier fix attempt did not apply; this commit applies it and checks the match.) Evidence block - Law: a Rerun line counts only for the failing fixed unit whose exact name it selects (witness rule; register: Rerun lines are anchored coarse filters). - RED: run 43569602268: "Rerun 1 selects no failing fixed unit". - GREEN: pending the next Windows run. - Mutant: not run. - Limits: as recorded in the PR body (sibling case, #78). Co-authored-by: Isaac <no-reply@databricks.com> * fix(windows): the Rerun name check ignores vitest's colors Run 43571581118 (3b97795): each Rerun line reproduced (exit 1, "Failed Tests 1"), but the check that the failing unit's exact name appears in its output failed: vitest colors the output, so the name is split by color codes. The witness now strips the color codes before it matches a unit's name. Evidence block - Law: a Rerun line counts for the failing fixed unit whose exact name it selects and whose FAIL line its output shows (witness rule). - RED: run 43571581118: "Rerun 1 did not reproduce in pwsh (exit 1 ... Failed Tests 1)" with the name check failing. - GREEN: pending the next Windows run. - Mutant: not run. - Limits: as recorded in the PR body (sibling case, #78). Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the e2e replay check reads and runs the PowerShell line on win32 Class 1 of the Windows static and e2e failures (merged head 5186cd5): the e2e oracle's replay check read the POSIX ` replay: ` line and ran it with `sh`. On Windows the CLI prints ` replay (PowerShell): ` (and a cmd line), so the replay command was undefined and the ownership law failed ("the replay command undefined ran undefined, owed only ..."; 3 violations). The check now reads the PowerShell line on win32 and runs it through pwsh on stdin, as a typed line runs; the comparison of the tests it ran is unchanged. Evidence block - Law: a baseline's replay command, pasted into the shell it names, runs the owed test (the ownership law; the register's per-shell rule). - Reach: the e2e histories with a failing owner (the baseline replays). - RED: run 43575409066 / 5186cd5: "the replay command undefined ran undefined, owed only [...]" (3 e2e violations, law ownership). - GREEN: pending the next Windows run. - Mutant: the POSIX-only reader is the mutant (the RED above). - Limits: the stdout gap-line count class (greppable stream) is not addressed here. Co-authored-by: Isaac <no-reply@databricks.com> * test(oracle-gaps): the greppable-stream breach names the stdout's gap lines A failure reported only the count of the stdout's gap lines, so a Windows run's extra lines could not be read from CI. The breach now carries the lines themselves. Co-authored-by: Isaac <no-reply@databricks.com> * test(oracle-gaps): the greppable-stream breach names the stderr's gap lines too Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): invalidate a patched file by its forward-slash path on win32 An authored mutant is served by patching a file in a warm context, which invalidates the file first. Vite keys its module map by forward-slash paths, so invalidateFile given a backslash path found no module and the context kept the original module. The authored mutant then survived as not-observed on Windows, where the model expected it killed (the e2e shards' 'greppable stream' breaches: authored:a1 and authored:a2 on sut-less fixtures, in run 43585144306). posix() is a no-op off win32. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the CLI-directory reach state compares paths in one spelling The refused run whose package lies inside the CLI's directory was tested with a string prefix of the CLI's directory plus a forward slash. On win32 the package's path is joined with backslashes, so the prefix never matched and the state was never reached (RED: run 43586666931, e2e shard 6/6, 'did not enter a refused run whose package lies inside the CLI's directory'). isWithin compares both paths in forward slashes. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the static driver names the package in both path spellings The CLI prints its paths in forward slashes, but the static driver named only the package's join-spelled path. The scratch directory's random name stayed in the forward-slash lines, so two runs of one constructed history recorded different arguments traces on Windows (RED: run 43586666931, static shards, 'two runs ... recorded different traces at observation arguments'). The e2e driver already names both spellings. Co-authored-by: Isaac <no-reply@databricks.com> * ci: the Windows static oracle job blocks the preview All six shards of the Windows static oracle are green (run 43588402680), each under 3 minutes, so the job no longer continues on error and the preview waits for it. The end-to-end job stays non-blocking: two of its shards cannot pass on Windows yet (a held daemon cannot be paused, and a stop by SIGTERM or SIGINT cannot be delivered), which needs a maintainer decision on the declared reach. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the daemon pause and win32 stops do not need signals that win32 lacks A held daemon is paused by a test-only pause at the daemon's reply (ORACLE_GAPS_TEST_PAUSE_AT=daemon-reply), which writes its pid to the check's output directory, instead of SIGSTOP (no signal pauses a process on win32). The pause fires once per check, so a daemon the CLI restarts after the death answers. On win32 a stop goes through the CLI's test stop hook (ORACLE_GAPS_TEST_STOP) and is observed as SIGINT. The SIGTERM stop state is required on POSIX only: win32 cannot deliver SIGTERM (its kill is TerminateProcess). Limits states the reason. RED (Windows): e2e shard 1/6 run 43588402680, 'a check whose daemon held it past the CLI's wait, then died' not reached; shard 4/6, 'every outcome class' sent SIGTERM and stderr had no done line. GREEN (macOS, local): 'an edit sequence through the daemon' and 'every outcome class' pass (fixed seed 20261105). Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the abort run is POSIX only; the Windows end-to-end job prints test durations A run that aborts by SIGABRT needs a signal win32 cannot deliver to a process, so its state and run are required on POSIX only, with the reason in Limits (RED: e2e shard 4/6, run 43591904999, 'the stopped run\x27s last line is done completed ... not done error, the run\x27s process exited with SIGABRT'). The Windows end-to-end step uses the verbose reporter, so each test\x27s duration is in the log for the re-shard (shard 1/6 hit the 900 s test timeout in run 43591904999). Co-authored-by: Isaac <no-reply@databricks.com> * test(oracle-gaps): the idle profile records which condition blocked the idle clock A hang cut off for idleness is not reached on win32 (e2e shard 4/6, run 43593613669). The profile (ORACLE_GAPS_IDLE) now counts, per test, the samples that each condition blocked: late, io with its kinds, busy, a timer due, timers listed. The Windows end-to-end job writes it and prints it. The driver passes it through GAP_IDLE_PROFILE, since cleanEnv drops the ORACLE_GAPS_ prefix. Diagnostic only; no behavior change. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the abort state is reachable on every platform again Reverts the POSIX-only scoping of the run whose process aborted: win32 has the mechanism (a V8 out-of-memory abort happens there too), so the state stays required everywhere. The witness still needs a win32 way to report the abort, which the CLI's supervisor (POSIX only) does today; that is open. The idle profile writes each sample of a cuttable copy as it happens, so a copy the watchdog kills keeps its samples (diagnostic). Co-authored-by: Isaac <no-reply@databricks.com> * chore(oracle-gaps): temporary daemon-history profile on Linux and Windows Removes the GAP_IDLE_PROFILE pass-through and the idle print step. Adds a temporary daemon-history-profile job that runs the one fixed-seed history on ubuntu and windows with GAP_PROFILE_ALL (a temporary route of the CLI's profile spans to one file) and prints per-kind totals. Diagnostic only; remove before merge, with the GAP_PROFILE_ALL route in check.mjs. Co-authored-by: Isaac <no-reply@databricks.com> * chore(oracle-gaps): probe the daemon-history job's startup and gap timings (temporary) Co-authored-by: Isaac <no-reply@databricks.com> * chore(oracle-gaps): time ps in the temporary profile job Co-authored-by: Isaac <no-reply@databricks.com> * chore(oracle-gaps): probe ps without timeout and tolerate probe failures Co-authored-by: Isaac <no-reply@databricks.com> * chore(oracle-gaps): remove the temporary daemon-history profile The profile job, its probes, and the GAP_PROFILE_ALL route were diagnostics for the daemon history's cost; the numbers are in the handback. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): restore the idle harness; declare the abort state on every platform The Linux end-to-end shard 2/3 ('every outcome class') regressed from 3.5 min to the 900 s timeout in run 43598121853 (495b281). Measured locally with the same timing wrappers: at 9b22c71 the history makes 2 CLI runs and 5 stop runs and passes in 77 s; at 495b281 and after it makes 11 and 25 and times out. Swapping in 9b22c71's capture-setup.mjs alone restores the passing count, and dropping only the per-sample write in that file does too; a comment-only change does not. The per-sample write is guarded by ORACLE_GAPS_IDLE, which is unset, so the mechanism is not understood; the diagnostic it served (idle detection works on win32) is answered, so the harness goes back to 9b22c71's version. Also: 495b281 dropped 'a run whose process aborted' from every declared state. It is declared again on every platform (after the POSIX stops, the order of 9b22c71 on POSIX). RED: 43598121853 (Linux 2/3, 900 s timeout). GREEN (POSIX, local): 'every outcome class' and 'an edit sequence through the daemon' pass (fixed seed 20261105). Limits: the 25-vs-5 difference is measured, not explained. Co-authored-by: Isaac <no-reply@databricks.com> * test(oracle-gaps): the daemon driver waits on marks, not on the clock; win32 backslash rewrites gated Review of #73 (d7f437b), items 1, 3, 5, 6: remove the idle profiling diagnostics from capture-setup.mjs (the blocked map, the blocker ternary, longest, and the ORACLE_GAPS_IDLE gate, which is outside the test namespace); the sampler is back to its original combined condition. The daemon driver no longer holds for a guessed 4 s and no longer kills by SIGSTOP. A daemon pauses at a test-only point (ORACLE_GAPS_TEST_PAUSE_AT: daemon-reply holds its reply, daemon-read holds its first connection unread) and writes its pid to ORACLE_GAPS_TEST_PAUSE_MARK. A dieMidCheck CLI writes ORACLE_GAPS_TEST_WAIT_MARK once a request has waited ORACLE_GAPS_WAIT_MS for its reply. The driver kills the daemon when the marks say so. Backslash rewrites in namedScratch, readRun, and the done line use posix(), which acts only on win32 (register: Windows paths; POSIX names are unchanged). The win32 abort comment and Limits say the win32 witness comes with #79. README: the test-only hooks, including the pause points, the marks, and STOP. RED and GREEN for the driver change: see the PR body (Arclet runs). Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): import posix and realPathOf in the e2e driver Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the daemon steps' labels come from observations; marks are atomic; wait knob renamed Review of #73 at a489353 (delta): the reset and held labels were attested by the driver from the step, not observed. daemonChecks now observes the daemon's pause mark (a positive integer, parsed before any signal) and its kill, and the CLI's wait mark for the hold; each label is added only from those. The resetBeforeRead warm daemon is started with the pause armed later (ORACLE_GAPS_TEST_PAUSE_ARM), so the reset hits a warm daemon, not a fresh one (P7). Marks are written atomically (temp file, then rename). The wait timer is cleared on every settle, including rejects. ORACLE_GAPS_WAIT_MS is renamed ORACLE_GAPS_TEST_WAIT_MS. The unused realPathOf import is removed. Co-authored-by: Isaac <no-reply@databricks.com> * ci(oracle-gaps): Windows end-to-end in nine shards sized by measured durations; docs for the wait knob and marks Review of #73 (P12): the Windows end-to-end shards took 11.4 and 11.3 min. Per-test durations come from the Vitest JSON reporter on the Arclet pool (run r73-dr-json, 138 tests). The three heaviest tests each take a shard of their own, and the rest are dealt out longest-first into nine shards (pr-shards.mjs check: five partitions split their tests exactly). The matrix moves to nine shards. README: ORACLE_GAPS_TEST_WAIT_MS documented (renamed from ORACLE_GAPS_WAIT_MS); the WAIT_MARK timer starts with the request and clears on every settle; the daemon-read arm and the mark format are documented. Co-authored-by: Isaac <no-reply@databricks.com> * ci(oracle-gaps): Windows end-to-end in ten shards, each pattern under cmd.exe's limit The nine-shard partition put 105 small tests in one shard, and its pattern (6,277 characters) made the command line too long on Windows (run 43659722330, shard 7/9). Ten shards, each at most 25 names, sized longest-first by the measured durations: the three heaviest tests each keep a shard of their own. pr-shards.mjs check: five partitions split their tests exactly. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): daemon labels need their observations; the warm witness; the disarmed variant; dead shard names removed; daemonsOf reads the lock, not ps Review of #73 (delta, at 104376d): - A daemon label is recorded only with a kill the driver sent and saw through (the pid gone), and the CLI's report of the check; a label without a pause is an error. - The warm witness: 'a warm daemon's connection was reset' needs the pid the warm check's daemon served (the served mark) to equal the pid the reset paused. - A constructed history 'a daemon pause that cannot engage' disarms the pause: the state 'a daemon pause that did not engage' is entered, and no label is added. The grammar gains pauseEngages (true in most histories, false rarely); this changes the generated histories of every seed, so the fixed-seed shards are re-verified. - The Windows end-to-end partition lists only the 38 tests that run: 100 dead entries (skipped tests in the JSON report) are gone; the check now fails on any listed name that matches no test (exactNames). - daemonsOf reads the daemon's lock (the pid it names, when that process runs), not ps: Windows has no ps the driver can rely on, and a command line spells the path in another form. Review #86: the hypothesis that the null stdout of ps throws. - The JSDoc of daemonsOf moves above it; a missing pause within 60 s fails loudly. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): restore pidOf, which the daemonsOf rewrite had removed Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the disarmed-pause state is declared globally, not in the daemon history's enters The state 'a daemon pause that did not engage' was listed with the daemon history's incremental states, so the normal daemon history had to enter it. It is declared in the required states, and only the constructed history 'a daemon pause that cannot engage' enters it. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the disarmed-pause variant pins a clean config, so its checks report Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the disarmed pause reaches its state; the plan carries pauseEngages The model's plan builder copies the history's fields one by one, and pauseEngages was not among them, so every variant engaged its pause (debug run r73-dbg-disarmed: the driver saw pauses=undefined). The plan now carries it, defaulting to engaged. The disarmed state is declared globally and excluded from the witnessed list, so only its own constructed history must enter it. Co-authored-by: Isaac <no-reply@databricks.com> * ci(oracle-gaps): the Linux e2e shards list the disarmed-pause constructed history Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): pauseEngages is a constant in the drawn record; only the constructed history disarms it The field was drawn with a weight, so every seed's histories changed. It is fc.constant(true) now, and only the constructed history 'a daemon pause that cannot engage' sets it false (through build). Canary: sampling 400 histories at seed 20261105 from the model at 104376d and from this model gives identical values once the field is dropped, and every drawn history has pauseEngages true. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): split 'every outcome class' into two constructed histories (P12) The first half replaces the original in place: the rest of its states and the first three stop runs (SIGTERM, SIGINT run, SIGINT after). The second half is appended at the end of the constructed list: the warm and abort stop runs, over a base that enters only their states. The union of their enters is the original's witnessed set; the framework checks each half reaches its own states. The take-back mapping returns a disarmed pause to engaged (its only use is the constructed history; a shrink of that history does not keep the disarm). The Windows partition places the halves in shards 2 and 4; the Linux partition lists the second half. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the first half of 'every outcome class' names the SIGTERM state only on POSIX RED: CI run 43675826518, every Windows end-to-end shard: 'constructed history "every outcome class" names undeclared states a run stopped by SIGTERM while a worker spun'. The SIGTERM state is declared on POSIX only (posixStops), and the first half named it on every platform. Co-authored-by: Isaac <no-reply@databricks.com> * ci(oracle-gaps): the Windows shards are sized by the measured durations of the merged tests Durations: the Vitest JSON reporter on the Arclet pool at 1636e85 (51 fixed-seed tests, all passing). The two halves of 'every outcome class' sit in shards 2 and 4; ten shards, cap ten names each. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): pauseEngages is the last field of the drawn record; the fingerprint lists get its line RED: CI run 43680525602, 'Windows unit suites', 'Test' and 'Test on Vitest 5': tests/gap-report-history- fingerprint.test.ts failed: the field was inserted before 'importer' and the other fields' draws were kept, but the recorded key list had no 'pauseEngages'. A new field goes at the end of the grammar and its line at the end of each recorded list. GREEN: the canary passes (the 53 recorded lines are unchanged; one line added per list). Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): append the new constructed histories to the end of the list Constructed histories sample seed + index, so inserting one mid-list changed the histories of every later entry on main. The disarmed-pause history and the second half of "every outcome class" now come after main's last entry; the first half keeps main's index 0, and the remaining entries keep their main positions. Co-authored-by: Isaac <no-reply@databricks.com> * test(oracle-gaps): a reset check whose warm daemon was replaced carries no warm label The warm label names the daemon that served the warm check. Until now no history reached a reset whose paused daemon was another one, so dropping the warm-pid equality went unseen (review of #73, B2). A constructed history, appended at the end of the list, sets warmReplaced (a drawn field, appended last, that only constructed histories set): the warm check's daemon is stopped before the paused check, so the paused daemon is a new one. The state "a warm daemon was replaced before the pause" is declared and entered, and the law check throws if the warm label is recorded for a daemon that did not serve the warm check. The README documents ORACLE_GAPS_TEST_SERVED_MARK, which the warm check reads. Co-authored-by: Isaac <no-reply@databricks.com> * ci(oracle-gaps): the second half of "every outcome class" runs in Linux shard 2 The Linux fixed-seed pattern matches whole names, so "every outcome class" selected the first half only. The second half, "every outcome class, the warm and abort stops" (155 s on the CI runner), fell into the rest and made shard 3 take 605 s of tests, 10.37 min for the job (run 43683794816). Shard 2 names both halves; the comment of shard 3 lists what it holds. Co-authored-by: Isaac <no-reply@databricks.com> * docs(oracle-gaps): record the Windows idle-state miss in the limits "every outcome class" must reach "a hang cut off for idleness", a state that win32 reaches too. Once on win32 (run 43683794816, shard 2/10) the sampler's quiet window was not seen, so the state was missed. The cause is not confirmed; the witness is owed (a test-only hold at the hang), recorded as a Limits item with the job references. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): a replaced warm daemon is stopped before the pause is armed An armed daemon holds its first connection unread, so the stop of the warm daemon, sent after the arm, never ended and the paused check was served by the same daemon. The stop now comes before the arm. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): the disarmed-pause history pins its generated inputs Appended at the end, the disarmed-pause history draws a new base (seed + index). Its base has a generated test whose input is a multiple of 3, so the work bound of 1 applies, and a mutant of a generated function is killed by work there. The model predicts no work verdict for a generated test, so the CLI and the model disagreed: "random-only-kill" against "not-judged" (run r73h-disarm-green, 07:13 UTC). The history's generated inputs now declare no bound. The Limits name the model gap; the random seeds keep the bound. Co-authored-by: Isaac <no-reply@databricks.com> --------- Co-authored-by: Isaac <no-reply@databricks.com>
…killed; a hold the CLI cannot release is marked - The supervisor records its child's pid beside the started marker, and a test that SIGKILLs the supervisor requires the child to be gone within 5 s (the child's watchdog, one tick). POSIX only: win32 has no supervisor until #79 (platform-scoped reach). The supervisor's own claim is left as SIGKILL leaves it, a stale claim, which the next run refuses. - A hold that never releases writes a .unreleased marker beside the hold file; the CLI keeps the exit its done line stated, and the driver throws when the marker exists (a test-only hook: the failure belongs to the test). Co-authored-by: Isaac <no-reply@databricks.com>
win32 has no supervisor until #79, so the death state is not reached there and the law has no observation to judge. Comment only. Co-authored-by: Isaac <no-reply@databricks.com>
|
Removal note from PR #57: the test-only hook |
F3: the declared limits say the supervisor-death state is required only off win32 (win32 has no supervisor until #79; register C entry (e) of #58), and name the scope rather than an OS gap. F4: the declared limits no longer say a default --out is judged only by the static oracle, or that a SIGKILL of the CLI is judged by inspection: a SIGKILL of a supervisor is witnessed on POSIX by its constructed history. F6: the changeset lists the run.pid refusals (live, stale, empty, unreadable) and the default-directory refusal. F7: the daemon request always carries the directory the run named (no `--out=dir` special case); the claim refusals print the claim directory in posix form on win32; the stranded optionsOf JSDoc sits with its function. Co-authored-by: Isaac <no-reply@databricks.com>
…p witness alone Split over the first half's heavy base, each stop history cost that base again: Linux fixed-seed 2/3 took 10m26s, and on Windows the abort stop, which fails there on main's win32 witness (#79), shrank through three heavy histories (592 s, job 12m41s). Each now runs over the stop witness alone, the base of "a stop with the default output directory": the same two states, cheaper histories (R7). Locally each passes in under 15 s, and a mutant of the stopped line and one of the abort's error line are each killed. Co-authored-by: Isaac <no-reply@databricks.com>
Draft. Stacked on #57 (
feat/gap-cli-stream-log); retarget tomainonce #57 merges.What this changes
taskkill /T /F(signalTree). The in-process unsupervised path is removed. A win32 out-of-memory abort or crash therefore ends with one done line on both streams and exit 3, as on POSIX (Windows: an unsupervised gap run can leave child processes running (no process tree kill) #74; register P10).ORACLE_GAPS_TEST_ABORT_AT=checkaborts the check's process after the session has begun. On win32 it callsprocess.abort(); on POSIX it sends SIGABRT to itself, becauseprocess.abort()prints a native backtrace, which is no line of the stream. The fixture's worker-thread kill is removed from the model.GAP_RUN_LOGmakes the e2e driver append one line per CLI run (its duration, status, signal, and whether the driver's kill timeout ended it).Evidence
Open
warmstop on win32 wrote no done line in run 43612786521 (the supervisor's stop path on win32). Not yet diagnosed.first linesstill show a raw scratch path (the scratch names are not applied to every spelling); not yet fixed.daemon-history-runsCI job are diagnostics; remove before merge.taskkill /Treaches the tree only while the child lives; an orphan a test started earlier is not reached (a Job Object would be the fix).This pull request and its description were written by Isaac.