Ship the thread pool as its own library so a process has one pool - #21522
Open
shoumikhin wants to merge 9 commits into
Open
Ship the thread pool as its own library so a process has one pool#21522shoumikhin wants to merge 9 commits into
shoumikhin wants to merge 9 commits into
Conversation
Contributor
Author
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21522
Note: Links to docs will display an error until the docs builds have been completed. ❌ 117 New Failures, 1 Pending, 2 Unrelated Failures, 49 Unclassified FailuresAs of commit 97e4007 with merge base d632341 ( NEW FAILURES - The following jobs have failed:
UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:
FLAKY - The following jobs failed but were likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
shoumikhin
added a commit
that referenced
this pull request
Jul 31, 2026
The thread pool that spreads CPU work across cores is meant to be a process-wide singleton, but it is held in a function-local static inside a static library, so every component that links it gets a private copy. The wheel ships two of them today: the pybindings extension and the training extension each define `get_threadpool`. Loading both into one interpreter gives the process two pools, and because each sizes itself to the machine's core count independently, they oversubscribe the CPU and contend with each other. Build the thread pool as a shared library for the wheel and let both extensions resolve it, so there is one pool per process again. `cpuinfo` and `pthreadpool` are forced static, so they are bundled inside this library rather than left for each consumer to supply, and the runtime is resolved from the shared runtime instead of embedding a second copy of the core. This is gated on the existing `EXECUTORCH_BUILD_SHARED` option, so only the Linux wheel changes; iOS, Android, and embedded builds keep linking the static library exactly as before. Test plan: The wheel smoke test now asserts that exactly one shipped library defines the thread pool accessor, alongside the existing single-backend-registry assertion, so a future change that reintroduces a second pool fails in CI rather than silently degrading performance. The new assertion was confirmed to fail against a wheel built before this change, where it correctly reports the two definers by name. Built the wheel from a clean checkout and verified against a fresh virtual environment with a normal dependency-resolving install: - `nm -DC` across every shipped shared object shows exactly one definition of `get_threadpool`, in the new library; the extensions import it rather than defining their own. - `readelf -d` shows the pybindings extension records the versioned thread pool library in `DT_NEEDED`, with a relocatable RUNPATH so it resolves the runtime as a sibling without `LD_LIBRARY_PATH`. - The single-backend-registry assertion still holds with the new library loaded. - `import executorch`, the registered backend list, and `.pte` execution through the Python bindings are unchanged, with outputs matching eager PyTorch. - With `EXECUTORCH_BUILD_SHARED` off, the thread pool is still a static library and no new shared object is produced, so every build that does not opt in is unaffected. ghstack-source-id: dfdb2bf ghstack-comment-id: 5146636071 Pull-Request: #21522
This was referenced Jul 31, 2026
shoumikhin
added
ciflow/periodic
ciflow/trunk
ciflow/binaries
ciflow/binaries/all
Contributor
There was a problem hiding this comment.
Pull request overview
This PR makes the extension thread pool a separately shipped shared library (when EXECUTORCH_BUILD_SHARED is enabled) so multiple wheel extensions loaded into one Python process share a single process-wide thread pool instead of each embedding its own static copy.
Changes:
- Build
extension_threadpoolas a shared library underEXECUTORCH_BUILD_SHARED, bundlingcpuinfo/pthreadpoolinto it and linking against the shared runtime. - Package the new
libexecutorch_threadpool.so.<major>into the wheel next tolibexecutorch.so.<major>. - Extend the wheel smoke test to assert there is exactly one definer of the thread pool accessor symbol across shipped shared objects.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| setup.py | Adds wheel packaging for libexecutorch_threadpool.so.<major> when building shared. |
| extension/threadpool/CMakeLists.txt | Switches threadpool to SHARED in shared builds; sets SONAME/RPATH; bundles static deps. |
| .ci/scripts/wheel/test_cpp_sdk.py | Adds “single threadpool definer” assertion alongside the existing backend-registry check. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The thread pool that spreads CPU work across cores is meant to be a
process-wide singleton, but it is held in a function-local static inside a
static library, so every component that links it gets a private copy. The
wheel ships two of them today: the pybindings extension and the training
extension each define
get_threadpool. Loading both into one interpretergives the process two pools, and because each sizes itself to the machine's
core count independently, they oversubscribe the CPU and contend with each
other.
Build the thread pool as a shared library for the wheel and let both
extensions resolve it, so there is one pool per process again.
cpuinfoandpthreadpoolare forced static, so they are bundled inside this libraryrather than left for each consumer to supply, and the runtime is resolved
from the shared runtime instead of embedding a second copy of the core.
This is gated on the existing
EXECUTORCH_BUILD_SHAREDoption, so only theLinux wheel changes; iOS, Android, and embedded builds keep linking the
static library exactly as before.
Test plan:
The wheel smoke test now asserts that exactly one shipped library defines
the thread pool accessor, alongside the existing single-backend-registry
assertion, so a future change that reintroduces a second pool fails in CI
rather than silently degrading performance. The new assertion was confirmed
to fail against a wheel built before this change, where it correctly reports
the two definers by name.
Built the wheel from a clean checkout and verified against a fresh virtual
environment with a normal dependency-resolving install:
nm -DCacross every shipped shared object shows exactly one definition ofget_threadpool, in the new library; the extensions import it rather thandefining their own.
readelf -dshows the pybindings extension records the versioned threadpool library in
DT_NEEDED, with a relocatable RUNPATH so it resolves theruntime as a sibling without
LD_LIBRARY_PATH.loaded.
import executorch, the registered backend list, and.pteexecutionthrough the Python bindings are unchanged, with outputs matching eager
PyTorch.
EXECUTORCH_BUILD_SHAREDoff, the thread pool is still a staticlibrary and no new shared object is produced, so every build that does not
opt in is unaffected.