You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
random seed: generated finds a greppable-stream counterexample on main too: stdout has 189 gap lines for 190 gaps (seed 333199054) #102
The end-to-end gap oracle's random unit random seed: generated found a counterexample in CI: at checkpoint cli, the greppable stream law fails with "the stdout has 189 gap lines for 190 gaps". The base fails the same way, so this is not caused by PR #90. Filed as an issue under P4, not rerun away.
CI: run 43688703700 (head 082991d), job Gap report end-to-end oracle (random seed, 3/3). The seed was 333199054, path 1:1:1:1.
Replays, each run twice with the same seed and path: on origin/main (5726197), 1 gap line short both times; on PR head 7df4a50, the same.
The counterexample history is printed in the failure message (History: {...}).
Two defects in the report
The printed Replay line does not reproduce this unit. The framework prints ORACLE_SEED=... ORACLE_PATH=... ORACLE_PROPERTY=... ORACLE_UNIT=... vitest run -t '...'. In this package, a bare vitest run uses the default config and finds no file: all 173 tests are skipped. The command needs --config oracle-gaps/vitest.config.mjs gap-report-oracle. The framework should print the project's test command, or this package's replay should say so.
The unit spent 817 s in this failing run, mostly in shrinking, and the job took 14 minutes. That is over the 10-minute budget (P12). A counterexample shrinks through up to six histories, and each one runs the real CLI.
Authority
P4 (register): a counterexample is filed as an issue with its replay line, and a rerun never hides it. Under the M phase-1 decision, a random counterexample that the base also fails is filed, not blocking. The determinism-as-main reading in PR #90 applies the same rule to determinism units. The greppable-stream law (maintainer request 2026-11-12) is the one that fails.
The printed Replay line also runs no test as printed (it omits the project's config). The framework bug is filed separately: 108. Use the command in this issue's Replay section until then.
Replay by history (register M, 2026-11-16, "Replay by history"; review of #122, F1). This records the full counterexample history, because #122 changes which history a seed and path select for this property's generated unit.
Replayed at:5726197 (the head of the run that found it), on an Arclet, with --maxWorkers=2. A scratch, test-only line printed each history the unit judged; the first one is the replayed counterexample.
The seed-and-path line selects this history only on commits before #122, and only while the property's constructed list is the one at 5726197; appending a constructed history before #122 also moved it. To replay it on a later commit, add the history above as a fixed history of the property, or as a constructed history whose build returns it, and run that test.
Activity
KyleAMathews commentedon Nov 15, 2026
The printed Replay line also runs no test as printed (it omits the project's config). The framework bug is filed separately: 108. Use the command in this issue's Replay section until then.
test(oracle-gaps): calibrate each stdout gap-line rejection, and judg…
test(oracle-gaps): judge stdout gap lines against results.json, by id (…
fix(oracle-gaps): probe code is never work: the capture counts no cal…
fix(oracle-gaps): probe code is never work (#102) (#121)
KyleAMathews commentedon Nov 16, 2026
Replay by history (register M, 2026-11-16, "Replay by history"; review of #122, F1). This records the full counterexample history, because #122 changes which history a seed and path select for this property's
generatedunit.--maxWorkers=2. A scratch, test-only line printed each history the unit judged; the first one is the replayed counterexample.cd packages/oracles && ORACLE_SEED=333199054 ORACLE_PATH=1:1:1:1 ORACLE_PROPERTY=oracle-gaps.report ORACLE_UNIT='random seed: generated' pnpm exec vitest --run --config oracle-gaps/vitest.config.mjs gap-report-oracle -t '^oracle-gaps\.report(?: | > )random seed: generated$'greppable streamatcli,the stdout has 189 gap lines for 190 gaps. The printed counterexample equals the history below.History:
{"functions": [[{"kind": "elseIf", "op": "<", "k": 0, "quote": "'"}, {"kind": "if", "op": "<", "k": 6, "not": true, "quote": "'"}]], "containers": ["C0"], "overloads": [true], "owners": ["both", "a"], "library": [[{"kind": "dollar"}, {"kind": "push", "quote": "'"}], [{"kind": "dollar"}, {"kind": "dollar"}]], "libraryOwners": [], "runes": [[{"kind": "runeState", "op": ">=", "k": 1}, {"kind": "runeStateRaw"}, {"kind": "runeStateRaw"}, {"kind": "runeStateRaw"}]], "derived": {"params": "plain", "field": false, "before": [{"kind": "early", "op": ">", "k": 8}], "form": "arrow", "after": [{"kind": "if", "op": "!==", "k": 4}]}, "sourceSpelling": "plain", "dotGen": false, "ownsSpelling": "plain", "wholeFile": "none", "template": true, "tests": [{"file": "b", "route": "case", "registered": true, "name": "plain", "fn": 0, "input": 3, "judges": false, "law": "L", "channel": "observe", "flaky": false, "judgeBy": "assertion", "hangs": false, "slow": false, "shrinks": false, "draws": false, "ioWait": false, "stops": false, "rawImport": false, "cleanupThrows": false, "asProp": false, "then": {"route": "fixed history", "input": 9}, "work": null, "timed": false}, {"file": "c", "route": "enumerated", "registered": true, "name": "plain", "fn": 1, "input": 4, "judges": false, "law": "L", "channel": "observe", "flaky": false, "judgeBy": "soft", "hangs": false, "slow": false, "shrinks": false, "draws": false, "ioWait": false, "stops": false, "rawImport": false, "cleanupThrows": false, "asProp": false, "then": null, "work": null, "timed": false}, {"file": "b", "route": "random seed", "registered": true, "name": "route", "fn": 2, "input": 4, "judges": true, "law": "reach agreement", "channel": "observe", "flaky": false, "judgeBy": "soft", "hangs": false, "slow": false, "shrinks": false, "draws": false, "ioWait": false, "stops": false, "rawImport": true, "cleanupThrows": false, "asProp": true, "then": {"route": "determinism", "input": 2}, "work": null, "timed": false}, {"file": "c", "route": "determinism", "registered": true, "name": "plain", "fn": 3, "input": 5, "judges": false, "law": "L", "channel": "observe", "flaky": false, "judgeBy": "assertion", "hangs": false, "slow": false, "shrinks": false, "draws": false, "ioWait": false, "stops": false, "rawImport": false, "cleanupThrows": true, "asProp": true, "then": {"route": "probe calibration", "input": 1}, "work": null, "timed": false}], "witness": true, "authored": {"stmt": 19, "kind": "apply"}, "declaration": "oracleProperty", "ownerless": false, "fault": "none", "stability": "none", "ordinary": true, "lazyGate": true, "secondAuthored": true, "consoleOutput": true, "config": "object", "configEnv": false, "setupGlobal": false, "native": "none", "depSubpath": false, "unbuiltDep": false, "boot": false, "budget": "one", "stopRuns": [], "onlyRun": false, "pragma": true, "cliPath": "plain", "concurrentStart": true, "configFile": "vite.config.mjs", "configPlace": "named", "packageType": "commonjs", "configAwait": false, "configRequire": true, "contexts": 4, "steps": null, "dataOwners": ["none", "a", "b"], "exportOwners": ["a", "none", "b", "b", "a", "b"], "importer": "a", "midrunBy": "a", "midrunTarget": "helper", "prePlugin": true, "fsRead": "none", "laterSteps": ["midrun", "midrun"], "tornRows": false}The seed-and-path line selects this history only on commits before #122, and only while the property's constructed list is the one at 5726197; appending a constructed history before #122 also moved it. To replay it on a later commit, add the history above as a
fixedhistory of the property, or as a constructed history whosebuildreturns it, and run that test.This comment was written by Isaac.