Implement native clang launcher by sbc100 · Pull Request #27401 · emscripten-core/emscripten

sbc100 · 2026-07-22T00:32:02Z

Implement Phase 1 of the native launcher design document. Provides
high-performance native C++ compiler drivers (emcc, em++) using C++20
that directly invoke Clang for compile steps (-c, -S, -E) and seamlessly
fall back to python (emcc.py / em++.py) for link steps or complex
post-processing options.

Output executables are placed in ./bin using CMake and Ninja. Adds CI
matrix testing for Linux, macOS, and Windows.

These new binaries are currently 100% optional and the python versions
can still be used without building them.

Tested using embuilder build libc --force (compiling 1,075
files sequentially with EMCC_CORES=1, EMCC_USE_NINJA=0, and
EMCC_BATCH_BUILD=0). Note that this is likely an outlier since
its perfectly suited to demonstrating to speedup from this change (i.e.
lots of small source files and no linking).

Platform	Before (Python Baseline)	After (Native Launcher)	Improvement	Speedup
Linux	181.96 s (169.3 ms/file)	64.82 s (60.3 ms/file)	-117.14 s (-109.0 ms/file)	2.81x
Windows	343.90 s (319.9 ms/file)	105.12 s (97.8 ms/file)	-238.78 s (-222.1 ms/file)	3.27x
macOS	162.08 s (150.6 ms/file)	64.73 s (60.1 ms/file)	-97.35 s (-90.5 ms/file)	2.50x

See: #26453

dschuff · 2026-07-22T23:35:21Z

Here are some high level comments:

Do we need response file support? Super long command lines are more common at link time, but the windows cmdline length limit is pretty low, so maybe a long path with a few include directives would be enough to push us over even at compile time.
If we are using C++20, we could use std::string_view instead of std::string in a lot of places to avoid a lot of allocations and copies in string operations, e.g. is_emscripten_only_warning, is_dash_s_setting, and other places. I got a bunch of other similar suggestions from Gemini for how to reduce allocations and redundant filesystem operations that might be worth considering to reduce overhead on startup.

sbc100 · 2026-07-23T22:38:17Z

Benchmark results for are in.. libc builds are much faster on all platforms! Yay

sbc100 · 2026-07-23T22:40:21Z

I think once we land this and we start using it everywhere we can just remove the old shell/bat/ps1/pylauncher launchers.

Implement Phase 1 of the native launcher design document. Provides high-performance native C++ compiler drivers (emcc, em++) using C++20 that directly invoke Clang for compile steps (-c, -S, -E) and seamlessly fall back to python (emcc.py / em++.py) for link steps or complex post-processing options. Output executables are placed in ./bin using CMake and Ninja. Adds CI matrix testing for Linux, macOS, and Windows. These new binaries are currently 100% optional and the python versions can still be used without building them. Tested using `embuilder build libc --force` on Linux CI (compiling 1,075 files sequentially with `EMCC_CORES=1`, `EMCC_USE_NINJA=0`, and `EMCC_BATCH_BUILD=0`): | Metric | Before (Python Baseline) | After (Native Launcher) | Improvement | | :--- | :---: | :---: | :---: | | **Total Time** | **181.96 s** (3m 2s) | **64.82 s** (1m 5s) | **-117.14 s (64.4% faster)** | | **Time per File** | 169.3 ms | 60.3 ms | **-109.0 ms / invocation** | | **Speedup Factor** | 1.0x | **2.81x** | — | **Key Takeaways:** * **64.4% Reduction in Build Time:** Bypassing Python interpreter startup shaves off **~109.0 ms of overhead per compiler invocation** on Linux CI (a **2.81x speedup**). * **Windows Impact:** Because process spawning and `python.exe` startup carry significantly higher overhead on Windows than on Linux, the percentage speedup on Windows CI is expected to be even larger. See: emscripten-core#26453

kripken · 2026-07-24T16:42:54Z

      return f.startswith(('-l', '-sEXPORT_ES6', '-sGL_TESTING', '-sPROXY_TO_PTHREAD',
                           '-sENVIRONMENT=', '--pre-js=', '--post-js=', '-sPTHREAD_POOL_SIZE=',
-                           '--profiling-funcs'))
+                           '--profiling-funcs', '--closure'))


This code seems unrelated, or am I not seeing it?

dschuff · 2026-07-24T20:07:56Z

+
+namespace emscripten {
+
+inline const std::unordered_set<std::string> LINK_ONLY_FLAGS = {


These can use std::string_view instead of std::string, and can be backed by the static C strings used in the initializers.
Gemini suggested even going further and using std::array instead of unordered_set, keeping them sorted, and use std::binary_search for the lookups. Not sure that's worth it though, it seems error-prone.

dschuff · 2026-07-24T20:14:39Z

+
+} // namespace
+
+void test_config_parsing() {


I guess another advantage if we eventually move to building this against LLVM is that we could use gtest.

dschuff · 2026-07-24T20:16:45Z

+add_executable(native_tests tests/test_native.cpp)
+target_link_libraries(native_tests PRIVATE native_launcher_lib)
+
+add_test(NAME native_tests COMMAND native_tests)


we should verify that even when CMake is configured in release mode, that the tests don't get compiled with -DNDEBUG because they are using raw assert(). We'd silently be just not testing anything!

sbc100 marked this pull request as draft July 22, 2026 00:32

sbc100 changed the title ~~[native] Implement native clang launcher frontend~~ Implement native clang launcher frontend Jul 22, 2026

sbc100 changed the title ~~Implement native clang launcher frontend~~ Implement native clang launcher Jul 22, 2026

sbc100 force-pushed the native_clang_impl branch 6 times, most recently from 198f9c8 to c5fd0d6 Compare July 22, 2026 21:53

sbc100 marked this pull request as ready for review July 22, 2026 21:54

sbc100 force-pushed the native_clang_impl branch from c5fd0d6 to 3d058e6 Compare July 22, 2026 22:04

sbc100 requested a review from dschuff July 22, 2026 22:05

sbc100 force-pushed the native_clang_impl branch 4 times, most recently from 470205f to bac6b35 Compare July 22, 2026 23:22

sbc100 requested a review from brendandahl July 22, 2026 23:32

dschuff reviewed Jul 22, 2026

View reviewed changes

Comment thread tools/emcc_native/main.cpp Outdated

Comment thread tools/emcc_native/main.cpp Outdated

Comment thread tools/emcc_native/driver.cpp Outdated

Comment thread tools/emcc_native/driver.cpp

sbc100 force-pushed the native_clang_impl branch 4 times, most recently from 06b94ef to 9fec6c9 Compare July 23, 2026 00:22

sbc100 requested a review from dschuff July 23, 2026 00:23

sbc100 force-pushed the native_clang_impl branch from 9fec6c9 to b5f76dc Compare July 23, 2026 00:36

dschuff reviewed Jul 23, 2026

View reviewed changes

Comment thread tools/emcc_native/driver.cpp

Comment thread tools/emcc_native/driver.cpp

Comment thread tools/emcc_native/driver.cpp Outdated

sbc100 force-pushed the native_clang_impl branch 4 times, most recently from 9ac1ab1 to c94e316 Compare July 23, 2026 19:16

sbc100 force-pushed the native_clang_impl branch 7 times, most recently from 0abdc68 to b7f2e37 Compare July 23, 2026 22:37

sbc100 requested review from dschuff, juj and kripken July 23, 2026 22:39

sbc100 mentioned this pull request Jul 24, 2026

Fix warning for link-only flags that contains = (e.g. --closure=1) #27410

Open

sbc100 force-pushed the native_clang_impl branch from b7f2e37 to f9bb2cb Compare July 24, 2026 00:37

Fix warning for link-only flags that contains = (e.g. --closure=1)

50fc65a

sbc100 force-pushed the native_clang_impl branch 4 times, most recently from 3a2cafc to ac011fa Compare July 24, 2026 04:30

sbc100 force-pushed the native_clang_impl branch from ac011fa to bc1e09d Compare July 24, 2026 06:31

kripken reviewed Jul 24, 2026

View reviewed changes

dschuff reviewed Jul 24, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Implement native clang launcher#27401

Implement native clang launcher#27401
sbc100 wants to merge 2 commits into
emscripten-core:mainfrom
sbc100:native_clang_impl

sbc100 commented Jul 22, 2026 •

edited

Loading

Uh oh!

dschuff commented Jul 22, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sbc100 commented Jul 23, 2026

Uh oh!

sbc100 commented Jul 23, 2026

Uh oh!

kripken Jul 24, 2026

Uh oh!

dschuff Jul 24, 2026

Uh oh!

dschuff Jul 24, 2026

Uh oh!

dschuff Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


		namespace emscripten {

		inline const std::unordered_set<std::string> LINK_ONLY_FLAGS = {

Uh oh!

Conversation

sbc100 commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dschuff commented Jul 22, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sbc100 commented Jul 23, 2026

Uh oh!

sbc100 commented Jul 23, 2026

Uh oh!

kripken Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

dschuff Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

dschuff Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

dschuff Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

sbc100 commented Jul 22, 2026 •

edited

Loading