Skip to content

javascript: a test that never loads the change is not selected; a mocha load-time throw selects the whole run - #1894

Draft
swapnilpaliwal-sd wants to merge 2 commits into
apps/integration-0.1.9from
fix/js-impact-f1
Draft

swapnilpaliwal-sd wants to merge 2 commits into
apps/integration-0.1.9from
fix/js-impact-f1

Conversation

@swapnilpaliwal-sd

Copy link
Copy Markdown
Contributor

JavaScript test selection: a test that never loads the change is not selected, a mocha load-time throw selects the whole run, and the import facts behind both read require('..') and a package's own name.

What changes

impact

  • imports_fact resolves a directory import (require('..'), require('../'), a folder with a package.json main/exports), the repository's own package name, a side-effect import './x' and a literal import('./x'); a package whose entry is a build output the checkout lacks names its src/ (or lib/) entry. Before, the root normalised to . and was tried as ./index.js, so most CommonJS test suites imported nothing here and loads_change was false for over a thousand failing test files on one subject.
  • loads_change asks whether the test loads the asked declaration's file, not the end of the shortest route (that route can run through another test file's subclass). It says nothing where the target file has no importer, the test imports nothing here, or a computed-name loader (require(path.join(dir, f))) sits in its import closure (imports_dynamic).
  • A JavaScript test whose route is in-process (sound, one of a set, dispatch, registered, defines, fixture) and whose file never loads the changed file goes to not_loading_tests and a not counted: line instead of the selection. Key joins, remote hops and spawned scripts are unaffected (the existing key-join case keeps its test).
  • load_run (impact.dl): test code that reaches the change through calls alone (a resolved call or one of a resolved set, no defines, no hand-over). A test credited through its module body or describe() block that is one of these carries at_load.

path — from a JavaScript test file that never loads the target's file: no route: X never loads Y, exit 1.

test-impact

  • A selected JavaScript file no runner collects is replaced by the collected test files that require it, or moved to not_collected when none does (kept when a test file loads modules by a computed name).
  • A mocha run whose specs reach the change at load time lists every spec it collects: mocha loads them in one process and a throw there fails all of them. jest and vitest are per-file and unchanged.
  • A function constructor whose header line changes is a TYPE to changed; when impact finds no type at that line it is asked again as the function (before: "could not be resolved to a graph symbol", 0 tests).

Measured

8 JavaScript libraries (5 tuning, 3 held-out) from the 18-repo mutation corpus, cached truth (test files that newly fail when a function throws on entry), the archived no-library protocol; base reproduces the archived numbers exactly. Micro recall / precision:

verb split base R base P this R this P
impact --tests tuning 0.487 0.273 0.511 0.271
impact --tests held-out 0.735 0.156 0.735 0.478
tests (mutant applied) tuning 0.455 0.264 0.531 0.284
tests (mutant applied) held-out 0.735 0.264 0.735 0.518
path test -> target tuning 0.660 ~0.245 0.660 ~0.245
path test -> target held-out 0.658 ~0.164 0.658 ~0.578

No failing (test file, target) pair is lost by impact, tests or path on any subject; every change in recall is a gain. Path precision is estimated on a fixed sample of the base engine's non-failing selections.

With the runtime dependencies staged on the web-framework subject: tests R 0.460 -> 0.841, P 0.607 -> 0.706.

Cost: cold-solve impact median unchanged on the three largest subjects (0.46/0.46 s, 0.50/0.48 s, 0.31/0.31 s); tests within ±10%.

Tests

  • tests/cases/javascript/a-test-that-never-loads-the-change: 3 checks fail on base, pass here; 4 controls (computed-name loader, require('..') / '../' / '../index.js', self-name import, the route of the test that does load the plugin) pass on both.
  • tests/cases/javascript/mocha-load-time-fails-the-whole-run: 3 checks fail on base, pass here; controls: a test-body-only change selects its spec alone, and a jest package is not widened.
  • python3 tests/run.py (all five languages): 1607 of 1607 checks pass.

Not addressed

  • Precision on hubs every test drives (a CLI's parse() reaching help and error paths behind flags): per-test-file coverage shows the false pairs never execute the target; no static signal without path conditions.
  • A test helper that calls its own module through module.exports(...), and a parameter defaulted as app = app || new App(), leave one held-out subject's request/response tests unreached (one repo; not tuned on held-out).

swapnilpaliwal-sd and others added 2 commits October 10, 2026 01:17
…ha load-time throw selects the whole run

impact
- imports_fact resolves a directory import (`require('..')`, `require('../')`, a folder with a
  package.json main/exports), the repository's own package name, a side-effect `import './x'` and a
  literal `import('./x')`; a package whose entry is a build output the checkout lacks names its
  src/ (or lib/) entry. imports_dynamic records the files that load a module by a computed name.
- loads_change asks whether the test loads the asked declaration's FILE, not the end of the
  shortest route (which can run through another test file's subclass or callback); it says nothing
  where the target file has no importer, the test file imports nothing here, or the test's import
  closure holds a computed-name loader.
- a JavaScript test whose route is in-process (sound, one of a set, dispatch, registered, defines,
  fixture) and whose file never loads the changed file is kept apart as `not_loading_tests` and
  named on a `not counted:` line, never selected: a plugin's prototype override is credited only to
  the tests that load the plugin, and a callback one test file hands to a shared function no longer
  pulls in every other caller's tests. Key joins, remote hops and spawns are unaffected.
- load_run (impact.dl): test-code declarations that reach the change through calls alone; a test
  credited through its module body or describe() block that is one of them carries `at_load`.

path
- from a JavaScript test file that never loads the target's file: "no route: X never loads Y".

test-impact
- a selected JavaScript file no runner collects is replaced by the collected test files that
  require it, or moved to not_collected when none does (unless a test file loads modules by a
  computed name).
- a mocha run whose specs reach the change at load time (`at_load`) lists every spec that run
  collects: mocha loads them in one process and a throw there fails all of them. jest/vitest are
  per-file and unchanged.
- a changed function-constructor (`function Foo(opts) {` header edit is a TYPE to `changed`) is asked
  again as the function when impact finds no type at that line.

Measured on 8 JavaScript subjects against cached mutation truth (no-library protocol), base ->
this: impact R 0.522 -> 0.542, P 0.238 -> 0.295; tests R 0.494 -> 0.559, P 0.264 -> 0.310; held-out
impact P 0.156 -> 0.478 at equal recall; no failing (test file, target) pair lost in impact, tests
or path on any subject.

Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
IMPACT_VERSION 70 carries both 69s (the library-callback receiver sites and the JavaScript import facts).

Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant