Repository navigation
Conversation
With --strategy per-mutant, mutants whose covering tests take least time now run first (--order cost, the default), so most results arrive early and an interrupted run has covered the most ground. --order source keeps the order of files and offsets. The new strain/estimate.py estimates the time left. A mutant costs a rebuild of its file plus its covering tests' baseline durations. As mutants finish, the estimate learns each file's rebuild time (everything but the tests, so mutating and restoring count too) and what fraction of the baseline durations the tests really take, since a kill stops them early. - Each progress line ends with `· ETA 2h10m`. - events.jsonl gains a `progress` event with done, total and remaining_seconds. - The TUI's header counts the runner's estimate down, through the new progress_estimated console hook, instead of multiplying the mean time per mutant. The estimate covers the whole run with per-mutant, and the current test with per-test. Fixes #42. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
lefticus
force-pushed
the
fix/issue-42-cheapest-first-eta
branch
from
November 18, 2026 03:56
d2b3b3c to
59d42b1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #42. Stacked on #50 (#40), which is stacked on #48 (#41). Merge those first; GitHub then retargets this PR, and the diff shrinks to this commit.
Order
With
--strategy per-mutant, mutants now run cheapest first: by the sum of their covering tests' baseline durations, then by file and offset. That's--order cost, the default.--order sourcekeeps the old order by file and offset. Withper-test, every mutant of a test runs the same test, so the order doesn't change there.Time left
strain/estimate.pyestimates the time left. A mutant costs a rebuild of its file plus its covering tests' baseline durations. The baseline durations are an upper bound, since a kill stops the tests early. As mutants finish, the estimate learns:Mutants that finish without a build don't teach it anything: ignored mutants, mutants known to break the build, and fully resumed ones. A mutant that fails to build teaches only the build time.
The estimate shows up in three places:
events.jsonl: aprogressevent after each mutant,{"done", "total", "remaining_seconds"}.progress_estimatedconsole hook. Before, it multiplied the mean time per mutant by the mutants left. The mean is still the fallback until the first estimate arrives.The estimate covers the whole run with
per-mutant, and the current test withper-test, since later tests' mutants aren't known until their baselines run. The README has a "Time left" section and an--orderrow.In the TUI
From a per-mutant run of the fixture, one frame every half second:
The run finished at 0:06. My first version counted only the build and test time, and showed
eta 0:01at 0:01 for the same run. Counting everything but the tests as "build" fixed that.Tests
tests/test_estimate.py): a seeded synthetic run of 500 mutants, cheapest first. Rebuild times differ by file, with ±20% noise. Covering tests take between 0.5 s and 85 s, and really use 40–80% of their bound. After 10% of the mutants, the estimate is within 20% of the real remaining time.--order costand--order sourcetest the same mutants with the same results. Mutantaddhas two covering tests; it runs first in source order and last by cost.format_eta(ETA 45s,ETA 12m30s,ETA 2h10m).LiveState: the runner's estimate counts down, stops at 0 and is dropped with a new queue.--orderand its default.progressevent, and the per-mutant test expectsaddlast, with an ETA in the output.python -W error -m pytest: 280 passed, 9 skipped.ruff format --check,ruff checkand lizard are clean.🤖 Generated with Claude Code