You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
determinism: an edit sequence through the daemon records a test-output note in one run and not the other #92
The determinism unit determinism: constructed history: an edit sequence through the daemon fails on main (35422e3). Two runs of the same history differ in one daemon stream note: one run's stream includes oracle-gaps: note test output in <package>/<out>/test-output.log, and the other's does not. Found by the determinism units that CI-1 moves onto pull requests (PR #90).
Evidence
Structural diff of the two full traces (a scratch patch printed the untruncated texts), at checkpoint incremental:
$.daemon.stream.notes.1 exists in one run only. Both runs share --hang-limit 20000 ms is above --mutant-limit 15000 ms ... as note 0.
The cause is not determined. The note appears when a test's output is captured, so it probably depends on a timing race between the daemon's test output and the check's end. That has not been confirmed.
Replay
cd packages/oracles && pnpm exec vitest --run --config oracle-gaps/vitest.config.mjs gap-report-oracle -t "determinism: constructed history: an edit sequence through the daemon"
Notes
The 2,000-character prefix in the determinism message (#81) hid this difference. A diff needed the full texts. The check's law is that two runs of one history record the same trace, so either the note must be deterministic, or the comparison must leave it out with a stated reason in the coverage map.
Same symptom on PR #57's new history determinism: constructed history: a daemon check with the default output directory. Two runs of one history differ in one daemon stream note, oracle-gaps: note test output in <out>/test-output.log, present in one run and absent in the other (structural diff at checkpoint incremental, trace length 37,952 vs 38,009, the extra note is the difference).
Mechanism confirmed in code: the note is printed only when test-output.log is non-empty at the end of the check (check.mjs), and its only writer is a synchronous appendFileSync of a test's own stdout/stderr writes (capture-setup.mjs). So its presence means a test wrote to the process's streams during that check. That part is established. What is not established is why a daemon check sometimes has such a write and sometimes not; the cause in #92 remains open.
Per the register, this is main's defect appearing in a new history, so it does not block #57. Not excluded from the determinism unit in code; it stays a failing determinism unit until #92 is decided.
The edit-sequence instance, and how #126 resolves it. This applies only to determinism: constructed history: an edit sequence through the daemon.
Cause. The history starts with two checks at once (concurrentStart). The daemon runs one check at a time, in the order the requests arrive, so whichever check it runs first computes every unit and the other reuses them. Only the check that ran the tests captured console output, so the note test output line follows that check. The driver recorded the pair by the process it started first, so the trace showed which process won the race.
Evidence.
Natural RED: 8 of 10 runs at fac63df fail, and 4 of 10 at 9aed105. Each differs only in that note.
Forced-order RED: a scratch driver runs the two checks one after the other, in opposite orders on alternate runs. It fails 2 of 2.
Still open. The other instance in the comment above is PR #57's a daemon check with the default output directory. That history has concurrentStart: false and steps: ['test'], so it has no pair and no race, and #126's cause cannot explain it. This issue stays open for that instance. #126 says "refs #92", not "Fixes".
Activity
ci(oracles): exclude the six determinism units that fail on main, eac…
ci(oracles): run the determinism units, excluding only the daemon note (
KyleAMathews commentedon Nov 15, 2026
Same symptom on PR #57's new history
determinism: constructed history: a daemon check with the default output directory. Two runs of one history differ in one daemon stream note,oracle-gaps: note test output in <out>/test-output.log, present in one run and absent in the other (structural diff at checkpointincremental, trace length 37,952 vs 38,009, the extra note is the difference).Mechanism confirmed in code: the note is printed only when
test-output.logis non-empty at the end of the check (check.mjs), and its only writer is a synchronousappendFileSyncof a test's own stdout/stderr writes (capture-setup.mjs). So its presence means a test wrote to the process's streams during that check. That part is established. What is not established is why a daemon check sometimes has such a write and sometimes not; the cause in #92 remains open.Per the register, this is main's defect appearing in a new history, so it does not block #57. Not excluded from the determinism unit in code; it stays a failing determinism unit until #92 is decided.
KyleAMathews commentedon Nov 16, 2026
The edit-sequence instance, and how #126 resolves it. This applies only to
determinism: constructed history: an edit sequence through the daemon.concurrentStart). The daemon runs one check at a time, in the order the requests arrive, so whichever check it runs first computes every unit and the other reuses them. Only the check that ran the tests captured console output, so thenote test outputline follows that check. The driver recorded the pair by the process it started first, so the trace showed which process won the race.unchanged reuse. The pair is also held together in the daemon by a test-only hold released by a count (R57).Still open. The other instance in the comment above is PR #57's
a daemon check with the default output directory. That history hasconcurrentStart: falseandsteps: ['test'], so it has no pair and no race, and #126's cause cannot explain it. This issue stays open for that instance. #126 says "refs #92", not "Fixes".