Skip to content

fix(js): test selection — imports past .d.ts, test-hot parameters, wrapped calls - #1877

Merged
swapnilpaliwal-sd merged 3 commits into
apps/integration-0.1.9from
fix/js-impact-loop
Oct 10, 2026
Merged

swapnilpaliwal-sd merged 3 commits into
apps/integration-0.1.9from
fix/js-impact-loop

Conversation

@swapnilpaliwal-sd

@swapnilpaliwal-sd swapnilpaliwal-sd commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

JavaScript test selection / impact / path, measured against behaviour on 18 pinned pure-JavaScript repositories (12 used for tuning, 6 held out and scored only at the end). Truth: 60 functions per repository broken one at a time (throw on entry, AST-sampled without consulting the graph), the test files that newly fail against a per-test double baseline; plus a precise runtime caller trace (every source function instrumented on its own line) for callers and runtime path pairs.

Three patterns, each seen in at least two repositories:

  1. An import resolved to the declaration beside the code. require('../') in a package whose package.json names types (or an exports types condition), and import './x.js' beside a hand-written x.d.ts, resolved to the .d.ts: the value was external and every call through it an untyped receiver, so a test of the package's own API reached nothing. The parser now re-resolves with declarations hidden; a declaration-only module keeps its first answer. (4 of 18 repositories; up to 137 imports each.)
  2. Test sites made the API under test a "generic utility". The parameter fan cap counted test files' call sites, so a constructor called 13 times by the library and 21 by tests lost its parameters, and a callback stored from its options was never followed. Now the program's arguments are decided by the program's own sites; tests' arguments as before, so nothing that flowed stops flowing. In the name count (the overcount for module-level functions) a test's bare call or new of the name leaves the count; a test's member call that only shares the name stays. Two rejected versions, both measured: dropping every test argument broke a held-out plugin pattern (lib.extend(plugin) from a test; path on that repo 0.833 → 0.137 — the case guards that shape), and dropping every test site from the name count made one curried library's graph 3x larger and its index 5x slower for no change in any answer.
  3. A call of what a project wrapper returned never reached what it wrapped. export default wrapper(impl, n): the closure's call of its parameter is context-insensitive and over the fan cap. Calls through the per-site ("wrap", site) value now also run that site's argument (one of a set, under the dispatch cap).

Results (micro unless noted; base = apps/integration-0.1.9 at a811ccf)

tuning (12 repos) base fix
impact --tests recall 0.441 0.553
… macro recall 0.305 0.435
… precision 0.388 0.336
mutants with failing tests but an empty selection 310 236
every failing file selected 92 142
path (failing test file → broken function) found 0.298 0.516
runtime callers recall 0.551 0.569
runtime path pairs 180/240 181/240
held out (6 repos) base fix
impact --tests recall 0.734 0.748
… macro recall 0.702 0.726
… precision 0.242 0.241
empty selections 81 68
path found 0.707 0.720
runtime callers / path pairs 0.681 / 78 of 90 0.681 / 78 of 90

No repository loses recall, path or callers on either set; precision dips only where recall rose (more files selected). Context answers are unchanged.

Cost (warm index, base → fix): unchanged on all 18 repositories within noise (largest: 96 → 71 s, 394 → 395 MB; the curried library 14 → 12 s, 112 → 117 MB; the CLI library's graph 69 → 76 MB from the edges fix 1 now resolves). An uncapped first version of fix 3 added 53k edges to the curried library; the cap removed them with identical answers.

Checks

  • tests/cases: javascript 319/319 (3 new cases, each red on the base engine and green here, with a control), typescript 269/269, python 306/306, java 333/333
  • graph/test/javascript/run-tests.sh: 88/88; two goldens (64-memoize-wrapper-result, 65-wrapper-param-reassigned) gain only the wrapped-call edges, reviewed and blessed with --oracle
  • parser src/test/javascript-tests.ts: 49/49

swapnilpaliwal-sd and others added 3 commits October 9, 2026 10:07
…tion beside it

The compiler's resolver prefers a declaration file: require('../') in a package whose
package.json names `types` (or an exports "types" condition), and import './x.js' beside
a hand-written x.d.ts, both resolved to the .d.ts. JavaScript never loads a declaration,
so every value imported that way was external and every call through it an untyped
receiver: a test of the package's own API reached nothing in it.

When the first answer is a declaration outside node_modules, the import is resolved again
with declarations hidden from the resolver (types/typings and the .d.ts substitution fall
through to main, the exports code conditions and the .js itself). A module that exists only
as a declaration keeps its first answer.

Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
…ments flow

The parameter fan cap refuses argument flow into a function called from more than
--dispatch-cap places, and it counted every site, tests included. A suite calls the API it
tests over and over, so exactly the functions under test went "hot": a constructor called
from 21 test files beside 13 library sites lost its parameters, a callback the library
handed it and stored on the instance (this.decode = options['decode']) was never followed,
and the library's own call through the field resolved only to the fallback beside it.

Two verdicts now. Whether the PROGRAM's arguments flow is decided by the program's own
sites; whether a TEST's arguments flow is decided by every site, as before, so nothing that
flowed before stops flowing: a plugin a test hands the program, which the program calls back
with its own class, still types what the test then calls (a version that dropped every test
argument lost exactly that; the case guards it). For the name count, the overcount used for
module-level functions, a test's bare call or `new` of the name leaves it, while a test's
member call that only shares the name (`lib.add(1, 2)` on an export that is a different
function, such as a curried closure) stays in it: dropping those too made a curried library's
graph three times larger and its index five times slower, for no change in any answer.

Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
…e was handed

`export default promisable(eachItem, 3)`, `const add = curry2(function add(a, b) {...})`:
the wrapper returns a closure that calls its parameter, and the module exports that
closure. The closure's call of the parameter is read context-insensitively, so it held
every function any of the wrapper's sites passed (past the fan cap: nothing), and a test
calling the export reached the closure and stopped there.

The wrapping site is already a value of its own (("wrap", site), value-flow.dl), used for
what a call through it returns. A call through it now also runs the function that site was
handed at the forwarded position, what the wrapper writes into that parameter, or what a
wrapped argument itself runs: one of a set, beside the edge to the closure the call keeps,
and under the dispatch cap (a parameter every wrapped export is passed to runs one of them,
and naming all of them says nothing). A wrapper whose closure never calls its parameter is
not a wrapper and runs nothing it was handed. Two engine goldens gain these edges (memoize
/ once / lazy results and the param-reassigned shapes), reviewed and blessed with --oracle.

Co-authored-by: axiomcode-bot[bot] <334110751+axiomcode-bot[bot]@users.noreply.github.com>
@swapnilpaliwal-sd
swapnilpaliwal-sd merged commit d08b4a5 into apps/integration-0.1.9 Oct 10, 2026
12 checks passed
@swapnilpaliwal-sd
swapnilpaliwal-sd deleted the fix/js-impact-loop branch October 10, 2026 03:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant