Automate JS-to-JS library dependencies using Acorn by sbc100 · Pull Request #27995 · emscripten-core/emscripten

sbc100 · 2026-10-16T18:18:38Z

Replace addImplicitDeps and checkDependencies in src/jsifier.mjs with calculateDeps, which uses acorn.tokenizer to automatically detect references to JS library symbols ($foo and _bar).

Keep the __deps mechanism in place for dependencies that static JS token scanning cannot detect (such as native Wasm/C symbols, function callbacks in __deps, side-effect-only dependencies, and __postset dependencies).

For system libraries we emit a warning when a __deps entry is redundant with an auto-detected dependency. For user JS libraries we only show this warning in -sSTRICT mode.

Performance cost (median of 15 runs on test/hello_world.c):

tools/compiler.mjs (INCLUDE_FULL_LIBRARY=0): 178.8 ms -> 185.1 ms (+6.3 ms, +3.5% slower)
tools/compiler.mjs (INCLUDE_FULL_LIBRARY=1): 268.2 ms -> 343.1 ms (+74.9 ms, +27.9% slower)
tools/compiler.mjs --symbols-only: 181.4 ms -> 254.2 ms (+72.8 ms, +40.1% slower)
Full emcc link (hello.o -> out.js, cold JS cache):
- INCLUDE_FULL_LIBRARY=0: 661.7 ms -> 740.8 ms (+79.1 ms, +12.0%)
- INCLUDE_FULL_LIBRARY=1: 758.3 ms -> 906.0 ms (+147.7 ms, +19.5%)
Full emcc link (hello.o -> out.js, warm JS cache):
- INCLUDE_FULL_LIBRARY=0: 271.8 ms (+0.0 ms, 0% slower)
- INCLUDE_FULL_LIBRARY=1: 282.4 ms (+0.0 ms, 0% slower)

Because I'm landing a few cleanup first (#27999, #28059, #28060) there are zero codesize changes acossiated with this PR.

Fixes: #19098

juj · 2026-10-16T19:34:44Z

This is really great work, and can simplify development a lot.

One thing here is that we really need to think about forwards and backwards compatibility here. The behavior

emit a warning when a system library __deps entry is redundant with an auto-detected dependency

makes it read like every codebase will gain a massive number of new warnings if this lands? That prompts the users to have to remove all the __deps directives in their codebases, which they will always have.

So that means that old Emscripten code will not compile without issuing tons of warnings (I'd argue emitting warnings is like a breakage here), and new code will not compile on old Emscripten versions without breaking. I.e. both directions break.

Emscripten does not have good mechanisms for authors to write JS library code that would conditionally be able to target multiple Emscripten versions. So third-party/middleware vendors are in particularly bad situation (e.g. https://github.com/juj/wasm_webgpu) which cannot ship one lib_foo.js that would be able to target multiple Emscripten versions? Developers would have to fork their JS libraries to two before/after the new rule?

Another concern here is the false positive rates. It is great to see that in the Emscripten repo, there are no code size regressions. But it's tricky to estimate that for all code.

Could we do an opt-in setting first, and find a way to get forwards compatibility?

- Skip scanning `PRE_JS_FILES`, `POST_JS_FILES`, and `EM_JS_SNIPPETS` in `--symbols-only` mode, and use `getIncludeFile` so preprocessed files are handled consistently. - Avoid mutating `LibraryManager.library[symbol]` when creating stub functions and remove the unused `original` variable. - Add `timer.start`/`timer.stop` around the `symbolHandler` loop. Split out from #27995

sbc100 · 2026-10-16T20:15:51Z

This is really great work, and can simplify development a lot.

One thing here is that we really need to think about forwards and backwards compatibility here. The behavior

emit a warning when a system library __deps entry is redundant with an auto-detected dependency

makes it read like every codebase will gain a massive number of new warnings if this lands? That prompts the users to have to remove all the __deps directives in their codebases, which they will always have.

So that means that old Emscripten code will not compile without issuing tons of warnings (I'd argue emitting warnings is like a breakage here), and new code will not compile on old Emscripten versions without breaking. I.e. both directions break.

Emscripten does not have good mechanisms for authors to write JS library code that would conditionally be able to target multiple Emscripten versions. So third-party/middleware vendors are in particularly bad situation (e.g. https://github.com/juj/wasm_webgpu) which cannot ship one lib_foo.js that would be able to target multiple Emscripten versions? Developers would have to fork their JS libraries to two before/after the new rule?

Another concern here is the false positive rates. It is great to see that in the Emscripten repo, there are no code size regressions. But it's tricky to estimate that for all code.

Could we do an opt-in setting first, and find a way to get forwards compatibility?

I actually already did this. User libraries now do not generated this working except it STRICT mode.

- In `src/lib/libcore.js`, check `t.includes('SYSCALLS.')` instead of `t.includes('SYSCALLS')` in `wrapSyscallFunction` so comments and strings mentioning `SYSCALLS_REQUIRE_FILESYSTEM` (such as in `fd_write`) do not pull in `$SYSCALLS` and `$UTF8ToString` when `SYSCALLS_REQUIRE_FILESYSTEM=0`. - In `src/lib/libpthread.js`, guard `'$addOnPreRun'` in `$PThread__deps` with `#if !MINIMAL_RUNTIME && PTHREAD_POOL_SIZE`. - Guard or remove other unused and overly-broad `__deps` entries across `src/lib/*.js` (`libasync.js`, `libbrowser.js`, `libdylink.js`, `libemval.js`, `libfetch.js`, `libfs.js`, `libglemu.js`, `libglew.js`, `libglut.js`, `libhtml5.js`, `libnoderawfs.js`, `libsdl.js`, `libsockfs_node.js`, `libwasmfs.js`, `libwebgl.js`, `libwebgl2.js`). Split out from emscripten-core#27995

- In `src/lib/libcore.js`, check `t.includes('SYSCALLS.')` instead of `t.includes('SYSCALLS')` in `wrapSyscallFunction` so comments and strings mentioning `SYSCALLS_REQUIRE_FILESYSTEM` (such as in `fd_write`) do not pull in `$SYSCALLS` and `$UTF8ToString` when `SYSCALLS_REQUIRE_FILESYSTEM=0`. - In `src/lib/libpthread.js`, guard `'$addOnPreRun'` in `$PThread__deps` with `#if !MINIMAL_RUNTIME && PTHREAD_POOL_SIZE`. - Guard or remove other unused and overly-broad `__deps` entries across `src/lib/*.js` (`libasync.js`, `libbrowser.js`, `libdylink.js`, `libemval.js`, `libfetch.js`, `libfs.js`, `libglew.js`, `libglut.js`, `libhtml5.js`, `libnoderawfs.js`, `libsdl.js`, `libsockfs_node.js`, `libwasmfs.js`, `libwebgl.js`, `libwebgl2.js`). Split out from emscripten-core#27995

- In `src/lib/libpthread.js`, guard `'$addOnPreRun'` in `$PThread__deps` with `#if !MINIMAL_RUNTIME && PTHREAD_POOL_SIZE`. - Guard or remove other unused and overly-broad `__deps` entries across `src/lib/*.js` (`libasync.js`, `libbrowser.js`, `libdylink.js`, `libemval.js`, `libfetch.js`, `libfs.js`, `libglew.js`, `libglut.js`, `libhtml5.js`, `libnoderawfs.js`, `libsdl.js`, `libsockfs_node.js`, `libwasmfs.js`, `libwebgl.js`, `libwebgl2.js`). Split out from emscripten-core#27995

When a JS library alias target (e.g. `foo`) is visited first and transitively depends on its own alias (e.g. `foo_alias: 'foo'`), `addedLibraryItems['foo']` is marked as visited before `var _foo = ...` is emitted. Emitting `var _foo_alias = _foo;` inline during dependency traversal causes it to appear before `var _foo = ...`, leaving `_foo_alias` initialized to `undefined` for arrow functions and non-hoisted values. Collect JS library alias assignments in `libraryAliases` and emit them after `libraryItems` (and before `postSets`) so all target symbols are defined before any alias assignment runs. Split out from #27995 where this issue was first spotted.

- In `src/lib/libpthread.js`, guard `'$addOnPreRun'` in `$PThread__deps` with `#if !MINIMAL_RUNTIME && PTHREAD_POOL_SIZE`. - Guard or remove other unused and overly-broad `__deps` entries across `src/lib/*.js` (`libasync.js`, `libbrowser.js`, `libdylink.js`, `libemval.js`, `libfetch.js`, `libfs.js`, `libglew.js`, `libglut.js`, `libhtml5.js`, `libnoderawfs.js`, `libsdl.js`, `libsockfs_node.js`, `libwasmfs.js`, `libwebgl.js`, `libwebgl2.js`). Split out from emscripten-core#27995

- In `src/lib/libpthread.js`, guard `'$addOnPreRun'` in `$PThread__deps` with `#if !MINIMAL_RUNTIME && PTHREAD_POOL_SIZE`. - Guard or remove other unused and overly-broad `__deps` entries across `src/lib/*.js` (`libasync.js`, `libbrowser.js`, `libdylink.js`, `libemval.js`, `libfetch.js`, `libfs.js`, `libglew.js`, `libglut.js`, `libhtml5.js`, `libnoderawfs.js`, `libsdl.js`, `libsockfs_node.js`, `libwasmfs.js`). Split out from emscripten-core#27995

…7999) - In `src/lib/libcore.js`, check `t.includes('SYSCALLS.')` instead of `t.includes('SYSCALLS')` in `wrapSyscallFunction` so comments and strings mentioning `SYSCALLS_REQUIRE_FILESYSTEM` (such as in `fd_write`) do not pull in `$SYSCALLS` and `$UTF8ToString` when `SYSCALLS_REQUIRE_FILESYSTEM=0`. - In `src/lib/libpthread.js`, guard `'$addOnPreRun'` in `$PThread__deps` with `#if !MINIMAL_RUNTIME && PTHREAD_POOL_SIZE`. - Guard or remove other unused and overly-broad `__deps` entries across `src/lib/*.js` (`libasync.js`, `libbrowser.js`, `libdylink.js`, `libemval.js`, `libfetch.js`, `libfs.js`, `libglemu.js`, `libglew.js`, `libglut.js`, `libhtml5.js`, `libnoderawfs.js`, `libsdl.js`, `libsockfs_node.js`, `libwasmfs.js`, `libwebgl.js`, `libwebgl2.js`). Split out from #27995

sbc100 · 2026-10-21T21:05:18Z

I think this change is ready to go now. We did an emscripten release prior, and all remaining codesize changes are purely the result of the order in which deps are found.

sbc100 · 2026-10-21T21:17:28Z

CC @kainino0x who also maintains out-of-tree JS library code.

kainino0x · 2026-10-21T21:49:20Z

That's very nice. We won't drop our deps yet because I have no idea how old versions of Emscripten people are using, but nice that we can (and also it might fix some latent missing deps bugs for us :) )

…ols-only` mode. NFC Previously, `getTransitiveDeps` was called inside `addFromLibrary` while `symbolHandler` was still iterating through `symbolsNeeded` and mutating `__deps` (such as adding `setTempRet0` for i64-returning functions without `WASM_BIGINT`). Compute `symbolDeps` in the `if (symbolsOnly)` block after `symbolHandler` completes so that all direct `__deps` are populated before traversing the dependency graph. Split out from emscripten-core#27995.

kripken · 2026-10-23T17:31:32Z

  $createContext: 'Browser.createContext',

-  emscripten_run_preload_plugins__deps: ['$PATH'],
+  emscripten_run_preload_plugins__deps: ['$Browser'],


Why is Browser added?

All function ins this file used to depend on $Browser due to the audoAddDeps at the end (that this PR removes).

The reason emscripten_run_preload_plugins need to keep this indirect dependency is because the Browser.init function is what registers the preloadPlugins. See preloadPlugins.push(imagePlugin); for example above.

…ols-only` mode. NFC (#28052) Previously, `getTransitiveDeps` was called inside `addFromLibrary` while `symbolHandler` was still iterating through `symbolsNeeded` and mutating `__deps` (such as adding `setTempRet0` for i64-returning functions without `WASM_BIGINT`). Compute `symbolDeps` in the `if (symbolsOnly)` block after `symbolHandler` completes so that all direct `__deps` are populated before traversing the dependency graph. Split out from #27995.

Add missing direct `__deps` declarations for JS library symbols that previously relied on other callers pulling in their dependencies later in the dependency graph. With `__ps` sorted alphabetically (emscripten-core#28059), adding these missing direct edges ensures that automatically computing JS-to-JS dependencies (emscripten-core#27995) produces identical declaration ordering and zero diffs across `test/codesize/`. See: emscripten-core#19098

sbc100 · 2026-10-23T22:06:12Z

This change is now made up of 4 commits. The first two are uploaded as separate PRs:

Sort deps (Sort JS library dependencies alphabetically in jsifier.mjs. NFC #28059)
Add missing deps (Add missing direct __deps entries across JS libraries. NFC #28060)
Automate deps
Remove hand-crafted deps

The reason I'm landing (1) and (2) first (or at least uploading them here) is so that we can show that (4) is no-op. No codesize tests changes with (4).

Add missing direct `__deps` declarations for JS library symbols that previously relied on other callers pulling in their dependencies later in the dependency graph. With `__ps` sorted alphabetically (emscripten-core#28059), adding these missing direct edges ensures that automatically computing JS-to-JS dependencies (emscripten-core#27995) produces identical declaration ordering and zero diffs across `test/codesize/`. See: emscripten-core#19098

Add missing direct `__deps` declarations for JS library symbols that previously relied on other callers pulling in their dependencies later in the dependency graph. With `__deps` sorted alphabetically (#28059), adding these missing direct edges ensures that automatically computing JS-to-JS dependencies (#27995) produces identical declaration ordering and zero diffs across `test/codesize/`. See: #19098

Replace `addImplicitDeps` and `checkDependencies` in `src/jsifier.mjs` with `calculateDeps`, which uses `acorn.tokenizer` to automatically detect references to JS library symbols (`$foo` and `_bar`). Because `calculateDeps` uses a flat token scan rather than a full AST and scope analysis pass (both for speed and to support partial JS fragments in `--pre-js`/`--post-js`), this scanning is conservative and can include false positives if local variables or parameters shadow a library symbol name. Keep the `__deps` mechanism in place for dependencies that static JS token scanning cannot detect (such as native Wasm/C symbols, function callbacks in `__deps`, side-effect-only dependencies, and `__postset` dependencies). Also log auto-detected dependencies in debug/verbose mode (`-sVERBOSE`). Performance cost (median of 15 runs on `test/hello_world.c`): - `tools/compiler.mjs` (`INCLUDE_FULL_LIBRARY=0`): 178.8 ms -> 185.1 ms (+6.3 ms, +3.5% slower) - `tools/compiler.mjs` (`INCLUDE_FULL_LIBRARY=1`): 268.2 ms -> 343.1 ms (+74.9 ms, +27.9% slower) - `tools/compiler.mjs --symbols-only`: 181.4 ms -> 254.2 ms (+72.8 ms, +40.1% slower) - Full `emcc` link (`hello.o` -> `out.js`, cold JS cache): - `INCLUDE_FULL_LIBRARY=0`: 661.7 ms -> 740.8 ms (+79.1 ms, +12.0%) - `INCLUDE_FULL_LIBRARY=1`: 758.3 ms -> 906.0 ms (+147.7 ms, +19.5%) - Full `emcc` link (`hello.o` -> `out.js`, warm JS cache): - `INCLUDE_FULL_LIBRARY=0`: 271.8 ms (+0.0 ms, 0% slower) - `INCLUDE_FULL_LIBRARY=1`: 282.4 ms (+0.0 ms, 0% slower) See: emscripten-core#19098

…ibraries Remove all `autoAddDeps` calls and redundant `__deps` entries across `src/lib/*.js` and the test suite. Fixes: emscripten-core#19098

sbc100 force-pushed the auto-js-deps branch from e70eb36 to cbb2a4f Compare October 16, 2026 18:19

sbc100 mentioned this pull request Oct 16, 2026

Minor cleanup and refactoring in src/jsifier.mjs. NFC #27997

Merged

sbc100 force-pushed the auto-js-deps branch 2 times, most recently from 9675b32 to 0a4cfff Compare October 16, 2026 18:57

sbc100 requested review from dschuff, juj and kripken October 16, 2026 19:00

juj reviewed Oct 16, 2026

View reviewed changes

Comment thread ChangeLog.md

sbc100 force-pushed the auto-js-deps branch from 0a4cfff to 518f258 Compare October 16, 2026 20:20

sbc100 mentioned this pull request Oct 16, 2026

Fix unused and overly-broad __deps entries in JS libraries #27999

Merged

sbc100 force-pushed the auto-js-deps branch from 518f258 to 0e9837d Compare October 16, 2026 22:07

sbc100 force-pushed the auto-js-deps branch 2 times, most recently from 3f60d55 to 64c9f6c Compare October 17, 2026 06:29

sbc100 force-pushed the auto-js-deps branch from 64c9f6c to 50181e9 Compare October 19, 2026 06:20

kripken reviewed Oct 19, 2026

View reviewed changes

Comment thread test/test_jslib.py

sbc100 force-pushed the auto-js-deps branch from 50181e9 to aa21bbc Compare October 20, 2026 18:18

sbc100 mentioned this pull request Oct 20, 2026

Emit JS library aliases after non-aliases #28032

Merged

sbc100 mentioned this pull request Oct 20, 2026

Fix false-positive $SYSCALLS dependency in wrapSyscallFunction. NFC #28034

Merged

sbc100 mentioned this pull request Oct 20, 2026

[GL] Fix unused and overly-broad __deps entries in WebGL JS libraries. NFC #28037

Merged

sbc100 force-pushed the auto-js-deps branch from aa21bbc to 0acf29c Compare October 21, 2026 21:03

sbc100 force-pushed the auto-js-deps branch 3 times, most recently from 4582b86 to 17dba40 Compare October 22, 2026 21:01

sbc100 requested a review from juj October 22, 2026 21:01

sbc100 mentioned this pull request Oct 22, 2026

Calculate transitive JS library deps after symbolHandler in --symbols-only mode. NFC #28052

Merged

kripken reviewed Oct 23, 2026

View reviewed changes

sbc100 mentioned this pull request Oct 23, 2026

Add missing direct __deps entries across JS libraries. NFC #28060

Merged

sbc100 force-pushed the auto-js-deps branch from 17dba40 to a38df35 Compare October 23, 2026 22:04

sbc100 force-pushed the auto-js-deps branch from a38df35 to ac7660a Compare October 23, 2026 23:20

sbc100 added 2 commits October 23, 2026 17:52

Remove redundant __deps entries and autoAddDeps calls across JS l…

8f14912

…ibraries Remove all `autoAddDeps` calls and redundant `__deps` entries across `src/lib/*.js` and the test suite. Fixes: emscripten-core#19098

sbc100 force-pushed the auto-js-deps branch from ac7660a to 8f14912 Compare October 24, 2026 00:53

Navigation Menu

Repository navigation

Automate JS-to-JS library dependencies using Acorn - #27995

Automate JS-to-JS library dependencies using Acorn#27995
sbc100 wants to merge 2 commits into
emscripten-core:mainfrom
sbc100:auto-js-deps

sbc100 commented Oct 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

juj commented Oct 16, 2026

Uh oh!

sbc100 commented Oct 16, 2026

Uh oh!

Uh oh!

sbc100 commented Oct 21, 2026

Uh oh!

sbc100 commented Oct 21, 2026

Uh oh!

kainino0x commented Oct 21, 2026

Uh oh!

kripken Oct 23, 2026

Uh oh!

sbc100 Oct 23, 2026

Uh oh!

sbc100 commented Oct 23, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

sbc100 commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

juj commented Oct 16, 2026

Uh oh!

sbc100 commented Oct 16, 2026

Uh oh!

Uh oh!

sbc100 commented Oct 21, 2026

Uh oh!

sbc100 commented Oct 21, 2026

Uh oh!

kainino0x commented Oct 21, 2026

Uh oh!

kripken Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

sbc100 Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

sbc100 commented Oct 23, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sbc100 commented Oct 16, 2026 •

edited

Loading