Ship the XNNPACK delegate as its own library - #21526
Open
shoumikhin wants to merge 10 commits into
Open
Conversation
Contributor
Author
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21526
Note: Links to docs will display an error until the docs builds have been completed. ❌ 95 New Failures, 2 Pending, 1 Unrelated Failure, 58 Unclassified FailuresAs of commit 64027b7 with merge base d632341 ( NEW FAILURES - The following jobs have failed:
UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:
FLAKY - The following job failed but was likely due to flakiness present on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
This was referenced Jul 31, 2026
This was referenced Jul 31, 2026
shoumikhin
added a commit
that referenced
this pull request
Jul 31, 2026
The XNNPACK delegate runs many models on CPU, but it is compiled into whichever component links it. The wheel ships two copies today, one inside the Python bindings extension and another inside the training extension, and a C++ application cannot get the delegate at all without building it from source. Build the delegate as a shared library and ship it in the wheel, so a process has one copy and both the Python bindings and a C++ application can link the same one. XNNPACK and its microkernels are built as static libraries, so they are bundled inside this library rather than left for each consumer to supply. The runtime and the thread pool are resolved from their shared libraries instead of embedding another copy of either. Because the delegate now carries XNNPACK itself, the places that used to name those static libraries explicitly no longer do so when building shared, which would otherwise ship the same code twice. Registration still happens through a static initializer, and the delegate is retained on the link line so that initializer runs even though no symbol from it is referenced directly. This is gated on the existing `EXECUTORCH_BUILD_SHARED` option, so only the Linux wheel changes; iOS, Android, and embedded builds keep linking the static library exactly as before. Test plan: The wheel smoke test now asserts that exactly one shipped library defines the XNNPACK delegate, alongside the existing backend-registry, thread-pool, and CPU kernel assertions. The symbol it checks was confirmed to exist in the shipped libraries first, and the assertion was confirmed to fail against a wheel built before this change, where it correctly reports both copies by name. Built the wheel from a clean checkout and verified against a fresh virtual environment with a normal dependency-resolving install: - The wheel ships the delegate as its own versioned library next to the runtime, the thread pool, and the CPU kernels. - `nm -DC` across every shipped shared object shows exactly one definition of a representative delegate symbol, in the new library rather than in the two extensions that used to carry it. - The delegate still appears in the registered backend list at runtime, which confirms its static initializer still runs from the shared library. - `.pte` execution through the Python bindings is unchanged, with outputs matching eager PyTorch. - The backend-registry, thread-pool, and CPU kernel assertions all still hold with the new library loaded. - With `EXECUTORCH_BUILD_SHARED` off, the delegate remains a static library and no new shared object is produced, so every build that does not opt in is unaffected. ghstack-source-id: f241eb6 ghstack-comment-id: 5147561128 Pull-Request: #21526
Contributor
There was a problem hiding this comment.
Pull request overview
This PR restructures the XNNPACK backend build so that, when EXECUTORCH_BUILD_SHARED is enabled (Linux wheel via the pybind preset), the XNNPACK delegate is produced and shipped as its own shared library. The intent is to avoid embedding duplicate copies of the delegate/XNNPACK into multiple wheel components and to make the delegate linkable by C++ consumers.
Changes:
- Ship
libexecutorch_xnnpack_backend.so.<major>into the wheel’sexecutorch/lib/alongside the runtime and other shared components. - Adjust pybind build/link behavior for shared builds to avoid explicitly listing XNNPACK static archives where the delegate library is intended to bundle them, and retain registration-only shared libraries on the link line.
- Extend wheel smoke tests to assert that exactly one shipped shared object defines a representative XNNPACK delegate symbol.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| setup.py | Adds wheel packaging of the XNNPACK backend shared library when shared + XNNPACK are enabled. |
| CMakeLists.txt | Updates shared-build link deps for XNNPACK and adds shared-library “retain” logic for registration-only components. |
| backends/xnnpack/CMakeLists.txt | Builds xnnpack_backend as SHARED under EXECUTORCH_BUILD_SHARED and bundles static XNNPACK/microkernels into it. |
| .ci/scripts/wheel/test_cpp_sdk.py | Adds a wheel-level assertion that only one shipped library defines an XNNPACK delegate symbol. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The XNNPACK delegate runs many models on CPU, but it is compiled into whichever
component links it. The wheel ships two copies today, one inside the Python
bindings extension and another inside the training extension, and a C++
application cannot get the delegate at all without building it from source.
Build the delegate as a shared library and ship it in the wheel, so a process
has one copy and both the Python bindings and a C++ application can link the
same one.
XNNPACK and its microkernels are built as static libraries, so they are bundled
inside this library rather than left for each consumer to supply. The runtime
and the thread pool are resolved from their shared libraries instead of
embedding another copy of either. Because the delegate now carries XNNPACK
itself, the places that used to name those static libraries explicitly no
longer do so when building shared, which would otherwise ship the same code
twice.
Registration still happens through a static initializer, and the delegate is
retained on the link line so that initializer runs even though no symbol from
it is referenced directly.
This is gated on the existing
EXECUTORCH_BUILD_SHAREDoption, so only theLinux wheel changes; iOS, Android, and embedded builds keep linking the static
library exactly as before.
Test plan:
The wheel smoke test now asserts that exactly one shipped library defines the
XNNPACK delegate, alongside the existing backend-registry, thread-pool, and CPU
kernel assertions. The symbol it checks was confirmed to exist in the shipped
libraries first, and the assertion was confirmed to fail against a wheel built
before this change, where it correctly reports both copies by name.
Built the wheel from a clean checkout and verified against a fresh virtual
environment with a normal dependency-resolving install:
runtime, the thread pool, and the CPU kernels.
nm -DCacross every shipped shared object shows exactly one definition of arepresentative delegate symbol, in the new library rather than in the two
extensions that used to carry it.
confirms its static initializer still runs from the shared library.
.pteexecution through the Python bindings is unchanged, with outputsmatching eager PyTorch.
with the new library loaded.
EXECUTORCH_BUILD_SHAREDoff, the delegate remains a static library andno new shared object is produced, so every build that does not opt in is
unaffected.