The framework's first mutation run (morph#405) scored 64.06% — 999 mutants,
640 killed, 359 survived — against a library measuring 95.69% lines and
91.19% branches. 321 of the survivors are classified unasserted: code
morph_tests executes, changes to which no test notices.
255 of those 321 are in include/morph/core. This ticket is to kill them.
It is deliberately not a coverage-percentage ticket. A percentage target is what
produced a suite that executes these lines without checking them; the survivor
list names the missing assertions directly, which is a better instrument and a
shorter argument.
Where they are
| survivors |
file |
| 121 |
include/morph/core/remote.hpp |
| 79 |
include/morph/core/bridge.hpp |
| 13 |
include/morph/core/registry.hpp |
| 10 |
include/morph/core/completion.hpp |
| 10 |
include/morph/core/payload_schema.hpp |
| 9 |
include/morph/core/backend.hpp |
| 8 |
include/morph/core/wire.hpp |
| 5 |
include/morph/core/executor.hpp |
| 4 each |
model.hpp, observability.hpp, strand.hpp |
Full triage in scripts/mutation_survivors.json, which also carries the 11
equivalent survivors (each with a reason — do not chase those) and 27
side-channel survivors in logging and metrics, kept separate because they are
absences of assertion rather than equivalences.
Start here: the server's reply is deleted and the suite passes
cxx_remove_void_call on reply(wire::encode(makeOk(env.callId, …))) survives
at eight sites in remote.hpp, and on makeErr replies for unauthorized,
connection closed and too many … at twelve more. The dispatch paths run;
what they put on the wire is not asserted at those sites.
That is the framework's response path — the thing every out-of-process client
depends on — and twenty places in it can be deleted without a red test.
Worth being precise about the scope of the claim: the scenario corpus under
scripts/scenario/ does assert wire replies for the ladder rungs, so this is
not "untested everywhere". It is untested in morph_tests, which is the suite
morph#405 measures and the one that gates every PR.
The other two shapes
A boundary comparison exercised only away from its boundary. cxx_gt_to_ge
and family survive at payload_schema.hpp:169 (kPayloadShapeMaxDepth),
wire.hpp:514's cap and a dozen more. wire.hpp:514 is the one already turned
into a test by morph#405 — and its neighbour was already named "decode accepts
an envelope at the size limit boundary" while asserting a 1 KiB envelope. That is
the shape to look for: a test whose name claims the boundary and whose body tests
the middle.
A completion is never delivered and nothing waits on it.
cxx_remove_void_call on onDone(failure), onDone(nullptr),
onDone(std::current_exception()) and resultState->setException(…) survives in
backend.hpp and completion.hpp. A test that drives a path without awaiting
its completion cannot distinguish a completion that never arrives from one that
arrives late.
The constraint
Do not chase the number. A mutant killed by an assertion written to kill that
mutant, on a path nobody cares about, is worse than a survivor — it is a test
with no reason to exist that will still be there in three years. Each new test
should be justifiable from the behaviour, with the survivor as the evidence that
it was missing.
The 11 equivalent entries are the worked example of what not to touch: the
golden-ratio arithmetic in hash_combine reaches only an unordered_map bucket
index, so no assertion can distinguish the mutant.
Verification status: measured
Mull 0.34.0 / mull-runner-22 / LLVM 22.1.2 / clang 22.1.8, cxx_all, 46m24s on
12 workers. Reproduce with bash scripts/mutation.sh (added by morph#405).
Not verified on CI's compiler. CI pins clang 20; Mull ships a -20 package
but the mutant set is generated from IR and will differ, so the score has not
been re-measured there.
The score can only be too high, not too low. tests/test_outbox.cpp names a
scratch file after the address of a stack object, which is stable across
processes, so concurrent copies of morph_tests can collide; a collision fails a
case, and a failed case reads to Mull as a killed mutant. It cannot manufacture
a survivor — the 359 are real, the 640 kills are an upper bound.
What would close this
bash scripts/mutation.sh over include/morph/core reports zero
surviving mutants at the twenty reply(...) sites named above, and the
completion.hpp/backend.hpp completion-delivery shape.
- Every remaining core survivor is in exactly one of three states: killed,
moved into mutation_survivors.json's equivalent class with a reason,
or moved into a side-channel class. No survivor is left unclassified.
- The re-run score for core is recorded next to the 64.06% baseline, so the
movement is legible.
- Each new test would fail on the unmutated code if its assertion were removed —
i.e. it asserts behaviour, not merely reaches it.
Not closable by raising line or branch coverage. Those are already 95.69% and
91.19% over this code; they are the measurements this ticket exists because of.
The framework's first mutation run (morph#405) scored 64.06% — 999 mutants,
640 killed, 359 survived — against a library measuring 95.69% lines and
91.19% branches. 321 of the survivors are classified
unasserted: codemorph_testsexecutes, changes to which no test notices.255 of those 321 are in
include/morph/core. This ticket is to kill them.It is deliberately not a coverage-percentage ticket. A percentage target is what
produced a suite that executes these lines without checking them; the survivor
list names the missing assertions directly, which is a better instrument and a
shorter argument.
Where they are
include/morph/core/remote.hppinclude/morph/core/bridge.hppinclude/morph/core/registry.hppinclude/morph/core/completion.hppinclude/morph/core/payload_schema.hppinclude/morph/core/backend.hppinclude/morph/core/wire.hppinclude/morph/core/executor.hppmodel.hpp,observability.hpp,strand.hppFull triage in
scripts/mutation_survivors.json, which also carries the 11equivalent survivors (each with a reason — do not chase those) and 27
side-channel survivors in logging and metrics, kept separate because they are
absences of assertion rather than equivalences.
Start here: the server's reply is deleted and the suite passes
cxx_remove_void_callonreply(wire::encode(makeOk(env.callId, …)))survivesat eight sites in
remote.hpp, and onmakeErrreplies forunauthorized,connection closedandtoo many …at twelve more. The dispatch paths run;what they put on the wire is not asserted at those sites.
That is the framework's response path — the thing every out-of-process client
depends on — and twenty places in it can be deleted without a red test.
Worth being precise about the scope of the claim: the scenario corpus under
scripts/scenario/does assert wire replies for the ladder rungs, so this isnot "untested everywhere". It is untested in
morph_tests, which is the suitemorph#405 measures and the one that gates every PR.
The other two shapes
A boundary comparison exercised only away from its boundary.
cxx_gt_to_geand family survive at
payload_schema.hpp:169(kPayloadShapeMaxDepth),wire.hpp:514's cap and a dozen more.wire.hpp:514is the one already turnedinto a test by morph#405 — and its neighbour was already named "decode accepts
an envelope at the size limit boundary" while asserting a 1 KiB envelope. That is
the shape to look for: a test whose name claims the boundary and whose body tests
the middle.
A completion is never delivered and nothing waits on it.
cxx_remove_void_callononDone(failure),onDone(nullptr),onDone(std::current_exception())andresultState->setException(…)survives inbackend.hppandcompletion.hpp. A test that drives a path without awaitingits completion cannot distinguish a completion that never arrives from one that
arrives late.
The constraint
Do not chase the number. A mutant killed by an assertion written to kill that
mutant, on a path nobody cares about, is worse than a survivor — it is a test
with no reason to exist that will still be there in three years. Each new test
should be justifiable from the behaviour, with the survivor as the evidence that
it was missing.
The 11
equivalententries are the worked example of what not to touch: thegolden-ratio arithmetic in
hash_combinereaches only anunordered_mapbucketindex, so no assertion can distinguish the mutant.
Verification status: measured
Mull 0.34.0 /
mull-runner-22/ LLVM 22.1.2 / clang 22.1.8,cxx_all, 46m24s on12 workers. Reproduce with
bash scripts/mutation.sh(added by morph#405).Not verified on CI's compiler. CI pins clang 20; Mull ships a
-20packagebut the mutant set is generated from IR and will differ, so the score has not
been re-measured there.
The score can only be too high, not too low.
tests/test_outbox.cppnames ascratch file after the address of a stack object, which is stable across
processes, so concurrent copies of
morph_testscan collide; a collision fails acase, and a failed case reads to Mull as a killed mutant. It cannot manufacture
a survivor — the 359 are real, the 640 kills are an upper bound.
What would close this
bash scripts/mutation.shoverinclude/morph/corereports zerosurviving mutants at the twenty
reply(...)sites named above, and thecompletion.hpp/backend.hppcompletion-delivery shape.moved into
mutation_survivors.json'sequivalentclass with a reason,or moved into a side-channel class. No survivor is left unclassified.
movement is legible.
i.e. it asserts behaviour, not merely reaches it.
Not closable by raising line or branch coverage. Those are already 95.69% and
91.19% over this code; they are the measurements this ticket exists because of.