Skip to content

core: 255 of the framework's 321 unasserted mutants are in include/morph/core — twenty of them delete the server's reply #431

Description

@Yaraslaut

The framework's first mutation run (morph#405) scored 64.06% — 999 mutants,
640 killed, 359 survived — against a library measuring 95.69% lines and
91.19% branches. 321 of the survivors are classified unasserted: code
morph_tests executes, changes to which no test notices.

255 of those 321 are in include/morph/core. This ticket is to kill them.

It is deliberately not a coverage-percentage ticket. A percentage target is what
produced a suite that executes these lines without checking them; the survivor
list names the missing assertions directly, which is a better instrument and a
shorter argument.

Where they are

survivors file
121 include/morph/core/remote.hpp
79 include/morph/core/bridge.hpp
13 include/morph/core/registry.hpp
10 include/morph/core/completion.hpp
10 include/morph/core/payload_schema.hpp
9 include/morph/core/backend.hpp
8 include/morph/core/wire.hpp
5 include/morph/core/executor.hpp
4 each model.hpp, observability.hpp, strand.hpp

Full triage in scripts/mutation_survivors.json, which also carries the 11
equivalent survivors (each with a reason — do not chase those) and 27
side-channel survivors in logging and metrics, kept separate because they are
absences of assertion rather than equivalences.

Start here: the server's reply is deleted and the suite passes

cxx_remove_void_call on reply(wire::encode(makeOk(env.callId, …))) survives
at eight sites in remote.hpp, and on makeErr replies for unauthorized,
connection closed and too many … at twelve more. The dispatch paths run;
what they put on the wire is not asserted at those sites.

That is the framework's response path — the thing every out-of-process client
depends on — and twenty places in it can be deleted without a red test.

Worth being precise about the scope of the claim: the scenario corpus under
scripts/scenario/ does assert wire replies for the ladder rungs, so this is
not "untested everywhere". It is untested in morph_tests, which is the suite
morph#405 measures and the one that gates every PR.

The other two shapes

A boundary comparison exercised only away from its boundary. cxx_gt_to_ge
and family survive at payload_schema.hpp:169 (kPayloadShapeMaxDepth),
wire.hpp:514's cap and a dozen more. wire.hpp:514 is the one already turned
into a test by morph#405 — and its neighbour was already named "decode accepts
an envelope at the size limit boundary" while asserting a 1 KiB envelope. That is
the shape to look for: a test whose name claims the boundary and whose body tests
the middle.

A completion is never delivered and nothing waits on it.
cxx_remove_void_call on onDone(failure), onDone(nullptr),
onDone(std::current_exception()) and resultState->setException(…) survives in
backend.hpp and completion.hpp. A test that drives a path without awaiting
its completion cannot distinguish a completion that never arrives from one that
arrives late.

The constraint

Do not chase the number. A mutant killed by an assertion written to kill that
mutant, on a path nobody cares about, is worse than a survivor — it is a test
with no reason to exist that will still be there in three years. Each new test
should be justifiable from the behaviour, with the survivor as the evidence that
it was missing.

The 11 equivalent entries are the worked example of what not to touch: the
golden-ratio arithmetic in hash_combine reaches only an unordered_map bucket
index, so no assertion can distinguish the mutant.

Verification status: measured

Mull 0.34.0 / mull-runner-22 / LLVM 22.1.2 / clang 22.1.8, cxx_all, 46m24s on
12 workers. Reproduce with bash scripts/mutation.sh (added by morph#405).

Not verified on CI's compiler. CI pins clang 20; Mull ships a -20 package
but the mutant set is generated from IR and will differ, so the score has not
been re-measured there.

The score can only be too high, not too low. tests/test_outbox.cpp names a
scratch file after the address of a stack object, which is stable across
processes, so concurrent copies of morph_tests can collide; a collision fails a
case, and a failed case reads to Mull as a killed mutant. It cannot manufacture
a survivor — the 359 are real, the 640 kills are an upper bound.

What would close this

  1. bash scripts/mutation.sh over include/morph/core reports zero
    surviving mutants
    at the twenty reply(...) sites named above, and the
    completion.hpp/backend.hpp completion-delivery shape.
  2. Every remaining core survivor is in exactly one of three states: killed,
    moved into mutation_survivors.json's equivalent class with a reason,
    or moved into a side-channel class. No survivor is left unclassified.
  3. The re-run score for core is recorded next to the 64.06% baseline, so the
    movement is legible.
  4. Each new test would fail on the unmutated code if its assertion were removed —
    i.e. it asserts behaviour, not merely reaches it.

Not closable by raising line or branch coverage. Those are already 95.69% and
91.19% over this code; they are the measurements this ticket exists because of.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: coreSubsystem: corebugSomething isn't workingtriage: validWell-framed; implement as written

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions