Skip to content

Fix #40: limit each mutant's memory, not just its run time - #50

Open
lefticus wants to merge 2 commits into
fix/issue-41-fastest-tests-firstfrom
fix/issue-40-memory-limit
Open

lefticus wants to merge 2 commits into
fix/issue-41-fastest-tests-firstfrom
fix/issue-40-memory-limit

Conversation

@lefticus

Copy link
Copy Markdown
Member

Fixes #40. Stacked on #48 (#41): both change run_baseline. Merge #48 first; GitHub then retargets this PR to main, and the diff shrinks to this commit.

What changes

Each test now runs its mutants under a memory limit, so a mutant that allocates forever is stopped before it takes the machine down. Example: i -= 1 → i += 1 in a loop that appends to a vector.

  • The limit: by default, four times the peak memory the test used in its baseline, and at least 512 MB. --memory-limit MB sets it for every test, and --memory-limit 0 turns it off.

  • Measuring the peak: ctest.run_test waits for CTest with os.wait4 on a thread. That reports the peak resident set size of CTest and the test it runs, which subprocess can't, and keeps the timeout. I checked that the peak includes a grandchild. Output goes to a temporary file, with stdout and stderr interleaved as they were written.

  • ASan programs get the limit through ASAN_OPTIONS=hard_rss_limit_mb=N:allocator_may_return_null=0. The user's own ASAN_OPTIONS come after it, so they win. A program counts as ASan when its binary contains __asan_init. I checked this for both dynamic and static libasan; a plain binary doesn't contain it. When strain can't find a test's program, it uses ASAN_OPTIONS alone, which is harmless without ASan.

  • Other programs run under ulimit -d (RLIMIT_DATA) through /bin/sh -c 'ulimit -d …; exec ctest …', which CTest and the test inherit. I used this instead of preexec_fn, which isn't safe once threads exist; the TUI runs the runner on a thread. The data segment limit makes an ASan program fail at startup, so it can't apply to ASan programs:

    ==912315==ERROR: AddressSanitizer failed to allocate 0xdfff0001000 (15392894357504) bytes
    
  • New state, killed_by_resource_limit: counted as detected. A failing run is classified this way when any of these holds:

    • ASan printed AddressSanitizer: hard rss limit exhausted. This message has no ERROR: prefix, so the sanitizer pattern wouldn't catch it.
    • The output mentions std::bad_alloc.
    • The measured peak reached the limit, as with SIGKILL.

    The check runs before the sanitizer check. Without a limit, bad_alloc stays an ordinary kill.

  • Display: the baseline line now reads baseline passed in 0.02s using 45 MB; mutants time out after 10s or stop at 512 MB, and the baseline event records peak_memory. A mutant stopped by the limit says reached the 512 MB memory limit. The per-test summary line adds memory N only when N > 0, so existing output is unchanged. The TUI shows the state with the short label memory.

  • README: a Memory limit section, a row in the result states table, the score formula and the CLI table.

The console's state column widens from 19 to 24 characters, the length of the new state's name. test_console is updated for that.

TUI

 score 100.0% ██   ✔ killed 1   ✔ killed_by_resource_limit 1                               elapsed 0:04  ·  1.6s/mutant
 ...
│ ✔ killed     sample.cpp:6:24  ror  != → ==  1││ state      ✔ killed_by_resource_limit                                │
│ ✔ memory     sample.cpp:6:32  asr  -= → +=  2││ operator   asr  -= → +=                                              │

(The clipped t column in the Mutants table predates this PR; I'll fix it separately.)

Platforms

Both limits are enforced on Linux. On macOS, ulimit -d is accepted but not enforced. As far as I know, ASan checks hard_rss_limit_mb from a background thread that only Linux and NetBSD start; I haven't verified that on macOS. A runaway mutant there still stops at its timeout, as before, and the Linux-only tests skip.

Tests

  • End to end (the acceptance in the issue): a fixture whose -= → += mutant appends forever finishes as killed_by_resource_limit. This is checked with the default ASan/UBSan build and with a build without sanitizers. With the limit disabled in the code, the same mutant runs to its 10 s timeout and the test fails.
  • tests/test_ctest.py: builds a test that allocates 1 MB at a time forever, with and without ASan. Under a 256 MB limit, each is classified killed_by_resource_limit, and the peak stays under the limit plus a 256 MB margin. ASan samples RSS about ten times a second, so it can overshoot. The tests also check that the baseline measures a 200 MB peak, that ASAN_OPTIONS are merged, that the ulimit wrapper is built correctly, and that ASan detection handles a missing file and a directory.
  • tests/test_results.py: the three ways a run is classified as reaching the limit, a passing run near the limit, and bad_alloc without a limit.
  • tests/test_cli.py: --memory-limit, 0, and rejection of negative values.
  • python -W error -m pytest: 271 passed, 9 skipped. ruff format --check, ruff check and lizard are clean.

🤖 Generated with Claude Code

A mutant such as `i -= 1` -> `i += 1` in a loop that appends to a vector
can use up the machine's memory long before the timeout stops it. Each
test now runs its mutants under a memory limit: by default four times the
peak memory of its baseline, at least 512 MB, or --memory-limit MB (0 for
none).

- ctest.run_test waits with os.wait4 on a thread, which gives the peak
  resident set size of CTest and the test while keeping the timeout. The
  output goes to a temporary file.
- Programs that call __asan_init get the limit through ASAN_OPTIONS
  (hard_rss_limit_mb, with the user's own options after it), because ASan
  fails to start under a data segment limit. Other programs run under
  `ulimit -d`, which CTest and the test inherit.
- A run that fails after reaching the limit, by ASan's "hard rss limit
  exhausted", std::bad_alloc or its measured peak, is the new
  killed_by_resource_limit state. It counts as detected, and the console,
  reports and TUI show it.
- The baseline line shows the memory used and the limit. The baseline
  event records the peak.

Fixes #40.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he machine

ASan checks the resident set size about ten times a second. The test
program filled 1 MiB vectors in a tight loop, which a fast runner does at
gigabytes a second, so the peak overshot the 256 MiB margin on Ubuntu CI.
It now sleeps 5 ms per MiB, at most 200 MiB a second, and the margin is
128 MiB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant