[TRTLLM-14026][feat] BREAKING: Remove TensorRT from the C++ tree by Wanli-Jiang · Pull Request #16234 · NVIDIA/TensorRT-LLM

Wanli-Jiang · 2026-07-10T09:20:33Z

Line-by-line analysis of every code-bearing change in the commit
"[TRTLLM-14026][feat] BREAKING: Remove TensorRT from the C++ tree"
(base = Phase B a803a4f874 on origin/main @ ed3659a865, HEAD 53d64f3e74).

Overview

This PR severs the C++ tree's compile- and link-time dependency on TensorRT so the
shared core (runtime, batch manager, executor API, KV cache, sampling, kernels,
nanobind bridge) builds, links, and runs without the TensorRT library, and closes
the packaging gate (requirements.txt drops tensorrt). It is the C++ half of the
staged TensorRT-backend removal, following examples #15763, docs #15767, tests
#15810, the Triton C++ backend #15907, and the Python backend removal (TRTLLM-14022,
PR #15918).

512 files: 251 deletions, 258 modifications, 3 additions (+2,557 / −77,406).
Approach — internal types + delete the engine path in one PR: the shared core
used three nvinfer1 types as common currency (DataType, Dims, ILogger).
Those now live in common/tllmDataType.h as standalone tensorrt_llm:: types with
identical enumerator values and layout (serialization stays byte-compatible),
and every surviving file is respelled nvinfer1::X → tensorrt_llm::X. Because
enum class types do not interconvert, the TensorRT-engine execution path (which
must keep the real nvinfer1 types) cannot compile against the migrated core —
so its removal is forced into the same PR, matching the shape of the original
draft (Remove tensorrt backend QiJune/TensorRT-LLM#20, commit 59eb547a6d), re-derived on the live tree.
Test ownership rule (same as Phase B): this PR deletes the C++ binaries and
bindings, so it owns every test that drives them — engine-driven C++ gtests,
their Python integration wrappers, and all test-db/waives/.test_durations
entries are removed here.
Validation: full build_wheel.py without --trt_root green; readelf -d
on libtensorrt_llm.so/libth_common.so/bindings.*.so shows no libnvinfer
NEEDED entry; import tensorrt_llm + bindings succeed with the tensorrt
package hard-blocked; test_datatype_parity.py green (values 0–11 match legacy);
pytest --co tests/unittest collects clean; quickstart e2e on H100 produces
correct output; pre-commit fully green.
One row per distinct reason — a file changed for two different reasons gets two rows.

A. Internal types (serialization-compatible)

File	What changed	Why
`cpp/include/tensorrt_llm/common/tllmDataType.h` (ADDED)	Standalone `tensorrt_llm::DataType` (`enum class : int32_t`, values 0–11 = `kFLOAT..kE8M0`), `Dims` (`int32_t nbDims` + `int64_t d[8]`, `MAX_DIMS=8`), `ILoggerSeverity`/`ILogger`. Includes only `<cstdint>` + `common/config.h`; wrapped in `TRTLLM_NAMESPACE_BEGIN/END`.	The three nvinfer1 types the shared core uses as common currency need a TensorRT-free home. Enumerator values intentionally mirror `nvinfer1::DataType` and the `Dims` layout mirrors TRT-10 `Dims64` so previously-serialized executor configs and KV-cache metadata stay byte-compatible.
206 surviving files (list: `pr6-migrate-files.txt`)	Mechanical respell `nvinfer1::DataType` → `tensorrt_llm::DataType` (~2,000 uses), `nvinfer1::Dims` → `tensorrt_llm::Dims` (~60), `nvinfer1::ILogger` → `tensorrt_llm::ILogger`; 53 `NvInfer*.h` includes replaced by `tllmDataType.h`; 151 explicit `tllmDataType.h` includes added to files that had used the types only transitively (the new names are declared only in the new header).	Core migration. Breakdown: kernels 43, runtime 28, batch_manager 24, public `cpp/include` 23, thop 15, layers 9, nanobind 8, executor 6, common 6, tests 73, micro_benchmarks 1.
12 files with dead `NvInfer*.h` includes (`runtime/{loraCache,worldConfig}.h`, `batch_manager/cacheFormatter.h`, `kernels/…/moe_util_kernels.h`, `kernels/weightOnlyBatchedGemv/cudaCoreGemm{,NVFP4}.h`, `runtime/loraManager.cpp`, `thop/{allgather,reducescatter,noAuxTc,IndexerTopK}Op.cpp`, `unit_tests/kernels/cudaCoreGemm/cudaCoreGemmKernelTest.cpp`)	Include deleted (2 were already commented out).	They never referenced `nvinfer1` at all; each verified to include its cuda headers explicitly.
`cpp/include/tensorrt_llm/runtime/iTensor.h`	Also dropped the dead `namespace nvinfer1 { class IExecutionContext; }` fwd-decl.	Its only real users (`tllmRuntime.{h,cpp}`) are deleted and included `NvInferRuntime.h` themselves. (The draft leaked this into its end state.)
`cpp/tensorrt_llm/kernels/cutlass_kernels/moe_gemm/moe_kernels.cu`	File-local helper `moeLoraNvInferType` → `moeLoraDataType` (3 uses) + comment reword.	Identifier referenced nvinfer in name only.
`cpp/tests/unit_tests/multi_gpu/kernels/allReduce/gemmAllReduceTest.cu`	Dropped `using namespace nvinfer1;`.	All uses already qualified after the respell.
`tests/unittest/bindings/test_datatype_parity.py` (ADDED)	Asserts `tensorrt_llm.bindings.DataType` member set + integer values match the legacy `nvinfer1::DataType` values (FLOAT=0 … NVFP4=10).	Serialization/format-compatibility guard; auto-scheduled via the existing `unittest/bindings` entries in `l0_a10/h100/gh200.yml`.

B. Engine execution path — deleted C++ sources

Area	Files	What / why
`cpp/tensorrt_llm/plugins/**`	131	The TensorRT plugin library (`nvinfer_plugin_tensorrt_llm`) — only consumed by TRT engines.
`cpp/tensorrt_llm/batch_manager/`	20	TRT model adapters (`trtGptModel*`, `trtEncoderModel`, `trtGptModelFactory`) + engine-only buffers/algos (`runtimeBuffers`, `encoderBuffers`, `transformerBuffers`, `promptTuningBuffers`, `rnnStateBuffers`, `loraBuffers`, `handleContextLogits`, `handleGenerationLogits`, `logitsPostProcessor`, `makeDecodingBatchInputOutput`, `updateDecoderBuffers`) — the PyTorch pyexecutor has its own Python implementations.
`cpp/tensorrt_llm/runtime/`	7	Engine runtime wrappers: `tllmRuntime.{h,cpp}`, `tllmStreamReaders.{h,cpp}`, `layerProfiler.{h,cpp}`, `tllmLogger.cpp`.
`cpp/include/tensorrt_llm/`	12	Public headers of the above + `runtime/rawEngine.h`, `runtime/tllmLogger.h`, `executor/disaggServerUtil.h`, `plugins/api/tllmPlugin.h`.
`cpp/tensorrt_llm/executor/`	5	`executor.cpp`, `executorImpl.{h,cpp}` (the engine `Executor`), `disaggServerUtil.cpp`, and `model.h` (0 remaining consumers — verified).
`cpp/tensorrt_llm/executor_worker/`	2	The `executorWorker` orchestrator binary (engine-mode only).
`cpp/tensorrt_llm/nanobind/`	4	The C++ `Executor` binding (`executor/executor.{h,cpp}`) and the `ModelSpec` binding (`testing/modelSpecBinding.{h,cpp}`) — 0 Python users each after the test deletions below; `kvCacheManagerTestUtilBinding` (live users) kept.
`cpp/tensorrt_llm/testing/`	3	`modelSpec.{h,cpp}` + the `testing_src` CMake target — engine test-data path helper, orphaned. `kvCacheManagerTestUtil.h` (header-only, live binding) kept in place.
`cpp/cmake/modules/FindTensorRT.cmake`	1	No `find_package(TensorRT)` remains.

C. Relocations (retained symbols out of deleted files)

File	What changed	Why
`cpp/tensorrt_llm/executor/kvCacheEvent.cpp` (ADDED)	`KVCacheEvent` constructor + `executor::version()`, relocated from the deleted `executor.cpp`/`executorImpl` (taken verbatim from the draft).	Backend-agnostic; the retained KV-cache event path (kvCacheEventManager, serialization, nanobind) must keep linking.

D. Shared C++ files decoupled from the removed engine runtime

File	What changed	Why
`batch_manager/utils/inflightBatchingUtils.{h,cpp}`	Removed `copyGenerationLogits` (needs `RuntimeBuffers::GenerationLogitsCache`), `findOutputTensor`, `copyAdditionalOutputs`, and the `CudaGraphExecutor`/`CudaGraphExecutorCache` classes (drive `TllmRuntime` contexts). The PyTorch-used helpers stay.	Their parameter types are deleted with the engine path; PyTorch has its own cuda-graph and logits handling.
`runtime/lookaheadBuffers.{h,cpp}`	Removed the `LookaheadRuntimeBuffers` class (ctor takes `TllmRuntime const&`); kept `LookaheadDecodingBuffers`. `layers/lookaheadDecodingUtils.h` include → `common/cudaUtils.h` (for `divUp`).	Engine-side half of lookahead; the decoder-side half survives and is used by the shared decoder.
`batch_manager/medusaBuffers.{h,cpp}`	Removed the `MedusaBuffers` engine ctor (takes `TllmRuntime const&`, reads `engine.getTensorDataType("medusa_logits")`) + `medusaModule`/`speculativeChoicesUtils` includes; kept `reshape`/`insertInputTensors`.	Same split: engine ctor dies, shared methods stay.
`batch_manager/dataTransceiver.cpp`	Dropped the `runtimeBuffers.h` include.	Header deleted; nothing else used it.
`nanobind/runtime/bindings.cpp`	Removed the `TllmRuntime` class binding (~40 lines) + `tllmRuntime.h` include.	The Python TllmRuntime surface died with the Python backend; 0 remaining Python references (verified).
`nanobind/batch_manager/algorithms.cpp`	Removed the `LogitsPostProcessor` algorithm binding + include.	Its C++ impl is deleted; 0 Python users (pyexecutor uses its own Python logits path).
`nanobind/executor/bindings.cpp`	Dropped the deleted `Executor` class registration.	Config/type bindings (`ExecutorConfig`, request/response types) are untouched — the PyTorch backend keeps using them.
`nanobind/bindings.cpp`	Dropped the `modelSpecBinding.h` include + `initBindings(mInternalTesting)` call; kept `initKvCacheTestUtilBindings`.	ModelSpec binding deleted (§ B).
`common/opUtils.h` + `common/attentionOp.cpp` + `tensorrt_llm/_common.py`	Removed `isBuilding()` (env probe), the `if (isBuilding()) return 0;` gate in `attentionOp::initialize`, and the Python `_BuildingFlag`/`_is_building` writers of `IS_BUILDING` (+ now-unused `wraps` import).	The `IS_BUILDING` Python↔C++ contract only skipped NCCL setup during `trtllm-build`; no writer remains. Both halves deleted together (as the Phase B TODO in `_common.py` said).

E. Python executor layer (the engine path's last consumers)

File	What changed	Why
`tensorrt_llm/executor/base_worker.py`	Removed `_create_engine()` (constructed `tllm.Executor(engine_path, ModelType.DECODER_ONLY, executor_config)`); `setup_engine` now asserts `llm_args is not None` and always builds the PyExecutor; the three `isinstance(self.engine, tllm.Executor)` branches in `fetch_stats`/`fetch_kv_cache_capacity`/`fetch_kv_cache_events` collapsed to the PyTorch side.	The nanobind `Executor` class no longer exists — any `tllm.Executor` attribute access would raise at runtime. `llm_args=None` was unreachable from the LLM API since Phase B.
`tensorrt_llm/executor/worker.py`	`block_subordinates` drops the `isinstance(self.engine, tllm.Executor)` early-exit branch.	Same.
`tensorrt_llm/__init__.py`	Removed the `_preload_tensorrt_libs()` shim (import of the `tensorrt` package to preload `libnvinfer` for the bindings).	Its reason for existing — `bindings.*.so`'s DT_NEEDED on `libnvinfer.so.10` — is gone; verified by importing with `tensorrt` hard-blocked.

F. Build & packaging (the final gate)

File	What changed	Why
`cpp/CMakeLists.txt`	Removed `find_package(TensorRT 10 REQUIRED COMPONENTS OnnxParser)` + `set(TRT_LIB TensorRT::NvInfer)`, the `TensorRT::NvInfer` include-dir injection, the `BUILD_BENCHMARKS` option/messages/`add_subdirectory(benchmarks/cpp)`, and `nvinfer_plugin_tensorrt_llm` from the build-time measurement list.	Nothing links or includes TensorRT anymore.
`cpp/tensorrt_llm/CMakeLists.txt`	Removed `add_subdirectory(plugins)`, `add_subdirectory(executor_worker)`, `add_subdirectory(testing)`, `${TRT_LIB}` and `testing_src` from the link list.	Their targets are deleted.
`cpp/tensorrt_llm/{batch_manager,runtime,executor}/CMakeLists.txt`, `nanobind/CMakeLists.txt`	Deleted sources pruned from the `SRCS` lists; `kvCacheEvent.cpp` added; `executor/executor.cpp` + `testing/modelSpecBinding.cpp` dropped from the nanobind module.	Match the file deletions.
`scripts/build_wheel.py`	`trt_root` default `'/usr/local/tensorrt'` → `None`; the "Ensure base TRT is installed" venv check removed; `nvinfer_plugin_tensorrt_llm`/`executorWorker` dropped from targets; the plugin `.so`/`.dll` and `executorWorker` install steps and the `bin/` dir removed; the `--benchmarks` flag + target removed.	Building no longer needs TensorRT anywhere; C++ benchmarks are deleted.
`setup.py`	`package_data` drops `bin/executorWorker`, `libs/libnvinfer_plugin_tensorrt_llm.so`, `libs/nvinfer_plugin_tensorrt_llm.dll`.	Not built anymore.
`requirements.txt`	Drops `tensorrt~=10.16.1` — the global gate from `plan.md`.	Python tree TRT-free since Phase B; C++ tree TRT-free as of this PR.
`jenkins/Build.groovy`	Dropped `--benchmarks` from the wheel build line, the Step-4 benchmark packaging block (`bertBenchmark`/`gptManagerBenchmark`/`disaggServerBenchmark`/`libnvinfer_plugin` copies into the tarball), and the `build_cpp_examples.py` step.	Those binaries no longer exist; leaving the `cp` lines would fail every CI build.
`scripts/get_wheel_from_package.py`	Removed the benchmark-binary extraction loop + `benchmarks_dir` setup.	Tarball no longer carries them.
`scripts/build_cpp_examples.py` (DELETED) + `examples/cpp/**` (11 files)	The C++ executor examples (`executorExampleBasic/Advanced/Disaggregated/…`) all construct `tle::Executor(<engine path>)`.	Engine-only examples die with the class; the Jenkins step that built them is removed above.

G. C++ tests — deleted (engine-driven) vs fixed (kept)

Deleted

File(s)	Why
`cpp/tests/e2e_tests/batch_manager/{trtGptModelTest,trtGptModelRealDecoderTest,trtEncoderModelTest}.cpp`	Drive the deleted TRT model adapters.
`cpp/tests/e2e_tests/executor/**` (`executorTest.{h,cpp}`, `executorMockTest.cpp`, `encDecTest.cpp`, `disaggExecutor.h`, `disaggExecutorTest.cpp`, `CMakeLists.txt`)	Drive the deleted C++ `Executor` with built engines.
`cpp/tests/unit_tests/executor/executorTestSmall{,ArbitraryOutputTensors}.cpp`	Same, via in-memory engines from `tests/utils/engines`.
`cpp/tests/unit_tests/runtime/tllmRuntimeTest.cpp`, `cpp/tests/unit_tests/batch_manager/encDecBeamSearchTest.cpp`	Test `TllmRuntime` / `RuntimeBuffers::GenerationLogitsCache` — both deleted.
`cpp/tests/utils/{engines,executorUtils,common}.{h,cpp}` + `CMakeLists.txt` (the `testingUtils` lib)	`engines` builds TRT engines in-memory (the only survivor-file user of TRT graph APIs); `executorUtils` drives `executor::Executor`; `common` holds engine paths/logits helpers. All consumers deleted → whole `tests/utils/` dies.
`cpp/tests/resources/scripts/build__engines.py` + `generate_expected__output.py` + `generate_hf_gpt_output.py` + `io_converter.py` + `build_engines_utils.py` (22 files)	The engine-build machinery for the tests above (already broken since Phase B deleted the Python builder). Kept: `generate_test_lora_weights.py` — live user (`lora_setup` fixture feeds the surviving C++ LoRA unit tests, no engines involved).
`benchmarks/cpp/{bertBenchmark,gptManagerBenchmark,disaggServerBenchmark}.cpp` + `utils/utils.{h,cpp}` + `CMakeLists.txt`	Engine-driven C++ benchmarks (`bertBenchmark` uses `TllmRuntime`, the others the C++ `Executor`). Kept: `prepare_dataset.py` + the Python `utils/` — they feed `trtllm-bench`; `README.md` rewritten accordingly.

Fixed (kept tests that referenced deleted files — the draft missed all of these)

File	What changed	Why
`unit_tests/{layers/baseSamplingLayerTest.h, layers/eagleLayerTest.cpp, layers/explicitDraftTokensLayerTest.cpp, kernels/sampling/samplingTest.h, kernels/routing/routingTest.h, kernels/eaglePackDataTest.cpp, runtime/samplingTest.cpp}`	Removed the `tllmLogger.h` include, the `mLogger = std::make_shared<TllmLogger>()` line, and the member declaration.	The member was constructed but never consumed in all 7 files — vestigial; no replacement logger needed.
`unit_tests/runtime/gptDecoderBatchedTest.cpp`	The deleted `MakeDecodingBatchInputOutput::createDecoderBatchInputs` static (engine-free, ~50 lines) is inlined as a file-local helper; 4 call sites repointed.	Keeps real coverage of the surviving `GptDecoderBatched` (live in the PyTorch decoder path) without keeping a dead prod algorithm.
`unit_tests/kernels/ropeTest.cu`	`plugins/gptAttentionCommon/gptAttentionCommon.h` include → `kernels/gptKernels.h`.	It only needed the `AttentionMaskType`/`PositionEmbeddingType` enums, whose real home is `gptKernels.h`.
`unit_tests/multi_gpu/mpiUtilsTest.cpp`	Dropped the `plugins/common/plugin.h` include + the `GlobalSessionHandle` test (`plugins::getCommSessionHandle() == &COMM_SESSION`).	The only plugin-coupled assertion in an otherwise-kept shared-core test.
`cpp/tests/CMakeLists.txt`	`TensorRT::OnnxParser` dropped from the per-test link; `add_subdirectory(utils)` + the `tests/utils` include dir removed.	No TRT link; `testingUtils` deleted.
`cpp/tests/{e2e_tests,e2e_tests/batch_manager,unit_tests/executor,unit_tests/runtime,unit_tests/batch_manager}/CMakeLists.txt`	Registrations of every deleted test pruned (`guidedDecoderTest` stays — see below).	Match the deletions.
`cpp/tests/e2e_tests/batch_manager/guidedDecoderTest.cpp`	Migrated (respell + `tllmDataType.h`) and kept.	It lives under `e2e_tests/` but tests the surviving `GuidedDecoder` with no engine coupling. Flagged for relocation out of `e2e_tests/` in a follow-up.

H. Python test suite + CI metadata

File(s)	What changed	Why
`tests/integration/defs/cpp/test_e2e.py` (DELETED)	Whole file: builds TRT engines and drives the deleted `e2e_tests` binaries (`test_model`, `test_benchmarks`).	Owns-the-breakage rule.
`tests/integration/defs/cpp/test_multi_gpu.py`	Pruned to the 5 shared-core gtest wrappers (`test_mpi_utils`, `test_fused_gemm_allreduce`, `test_cache_transceiver`, `test_user_buffer`, `test_nccl_utils`) — exactly the KEEP list from `plan.md`. Deleted: `test_llama_executor{,_logits_proc,_guided_decoding}`, `test_enc_dec`, `test_trt_gpt_real_decoder`, `TestDisagg` (4 methods), their `run_*` helpers, and the engine-model fixtures (`prepare_multi_gpu_model_tests`, `prepare_model_multi_gpu`, `gpt/llama_single_gpu_model`, `llama_multi_gpu_model`, `prepare_models_disagg`, `multi_gpu_model`).	Per-function edit, not a file deletion (the file is MIXED — plan.md's explicit NB).
`tests/integration/defs/cpp/cpp_common.py`	`prepare_model_tests` (engine builder driving the deleted `build_*_engines.py`) removed.	0 callers after the fixture pruning.
`tests/integration/defs/cpp/conftest.py`	`prepare_model` + `build_benchmarks` fixtures removed. Kept: `lora_setup` (generates numpy LoRA weights — no engines — for the surviving C++ LoRA unit tests).	Their targets/scripts are deleted.
`tests/unittest/bindings/test_executor_bindings.py` (DELETED)	All-engine `trtllm.Executor` tests (was already waived since TRTLLM-13781).	The binding is gone; closes the waive.
`tests/unittest/bindings/binding_test_utils.py` (DELETED)	Engine-run helpers; only consumer was `test_executor_bindings.py`.	Orphaned.
`tests/unittest/utils/cpp_paths.py`	Trimmed to the sole surviving fixture `llm_root` (consumed by `tests/unittest/conftest.py` + one auto_deploy test). Deleted: the `ModelSpec` import (module-level — broke collection of the whole bindings suite), `get_base_model_spec`, and the engine-path/results-path fixtures (0 users each, verified fixture-by-fixture).	The late straggler found by the parity-test run; the other apparent "users" (`test_executor.py`, `test_base_worker.py`) define their own local names.
`tests/integration/defs/test_mlpf_results.py` (DELETED)	Runs `gptManagerBenchmark` on a built engine; unscheduled in any test list.	Binary deleted.
`tests/integration/test_lists/test-db/l0_{a10,a30,dgx_h100,h100}.yml`	10 entries delisted: the `cpp/test_e2e.py` model/benchmark entries + the `TestDisagg` symmetric-executor entries.	Tests deleted.
`tests/integration/test_lists/waives.txt`	11 waives removed (`cpp/test_e2e.py::`, `TestDisagg::`, the `test_executor_bindings.py` waive).	Waived tests no longer exist (duplicate/AST validators gate this).
`tests/integration/defs/.test_durations`	35 stale duration keys removed.	Same.
`benchmarks/cpp/README.md`	Rewritten: documents the kept dataset tools (`prepare_dataset.py`, `utils/*`) and points benchmarking at `trtllm-bench`/`trtllm-serve`; all `gptManagerBenchmark` content removed.	The old doc was entirely about the deleted binaries.

I. Review-driven scope change (chronological)

First cut (same day, superseded): tllmDataType.h was authored in a transitional
alias form (using DataType = nvinfer1::DataType; …) so the 206-file respell could
land as a standalone, provably-behavior-free PR while the plugin/engine path kept
compiling, with the real-enum flip + engine deletion deferred to the next PR
(plan.md's PR5/PR6 split).
Review direction: "remove nvinfer fully, not import nvinfer anymore" — the alias
staging was dropped; the header was flipped to the real standalone types and the
engine-path removal + packaging gate were folded into this PR (§ A–H above). The
alias-form design record is kept in pr6-cpp-datatype.md.
Stragglers beyond the draft (its base was ~3 weeks stale and its cpp/tests,
benchmarks, examples/cpp, and Jenkins handling were incomplete): the 12
deleted-header include stragglers (§ G "Fixed"), guidedDecoderTest classification,
executor/model.h, ModelSpec/testing_src, benchmarks/cpp, examples/cpp,
Build.groovy, cpp_paths.py — all found by systematic sweeps
(deleted-header include scan, python deleted-symbol scan, fixture-consumer counts).

J. Validation record

python3 scripts/build_wheel.py --use_ccache --skip_building_wheel --cuda_architectures native — no --trt_root — exit 0.
(Gotcha: BUILD_WHEEL_TARGETS is a CMake cache variable; the first incremental run
kept the stale target list and failed at the final aggregate target — resolved with
--configure_cmake.)
readelf -d on libtensorrt_llm.so, libth_common.so, bindings.*.so:
zero nvinfer/TensorRT NEEDED entries.
import tensorrt_llm + tensorrt_llm.bindings with the tensorrt package
hard-blocked via a sys.meta_path blocker: OK.
tests/unittest/bindings/test_datatype_parity.py: 2 passed.
pytest --co -q tests/unittest: exit 0 (collection clean).
Quickstart e2e (H100, cuda graphs): coherent generation. (The MoE FP8-block-scale
assert seen with one local checkpoint is a pre-existing Python weight-loader
incompatibility with that checkpoint — path untouched by this PR.)
pre-commit run on all 512 files: green (autoflake/clang-format/cmake-format
reformats absorbed into the commit).

K. Follow-ups (deliberately NOT in this PR)

Relocate guidedDecoderTest.cpp out of cpp/tests/e2e_tests/ (it is a unit test of
a surviving component; kept in place to avoid churn).
Rebase over the final Phase B HEAD (PR [TRTLLM-14022][feat] BREAKING: Remove python modules and tests for legacy TensorRT backend #15918, a7e54c552d): expect one trivial
tensorrt_llm/__init__.py conflict — its later amends added the
_preload_tensorrt_libs() shim this PR deletes, and the RTLD_GLOBAL re-dlopen
bridge in _common.py (kept here — it is about our own lib's symbols, not TRT;
its proper fix, a self-sufficient wrapper dlopen in transferAgent.cpp, remains
a C++ follow-up).
docker/common/install_tensorrt.sh + the TRT layers in the dev containers: the
wheel no longer needs TensorRT, but container-image slimming is a separate
infra-owned change.
Perf-test config keys with runtime: "cpp" in tests/integration/defs/perf/
(bertBenchmark-era label): dead config surface, prune in a perf-infra pass.
tensorrt_llm/models/unet/pp/__init__.py empty straggler from Phase B (tracked in
plan.md follow-ups).

@coderabbitai summary

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Remove the TensorRT execution backend from the Python package so that `import tensorrt_llm` and the PyTorch backend run without the `tensorrt` package installed. The `tensorrt` pip dependency itself is dropped in the follow-up C++/packaging step; `requirements.txt` is unchanged here. - Delete the TensorRT graph/build/runtime machinery: builder, network, functional graph ops, layers/, plugin/, the legacy models/ implementations, the runtime/ session and ModelRunner* classes, _tensorrt_engine, TRT quant layers and tools, and the trtllm-build/refit/prune CLIs. - Keep import paths stable: partly-TensorRT modules (functional.py, models/modeling_utils.py, _common.py, runtime/, bench/build) are gutted in place so the TensorRT-free symbols used by the PyTorch backend stay put; no `_torch/` import churn. - Drop the TensorRT public API surface (Builder/BuildConfig/TrtLlmArgs/...); `LLM(backend="tensorrt")` now raises and `trtllm-serve --backend` accepts pytorch/_autodeploy only (api_stability serve-CLI reference updated). - Remove or de-TRT the tests that exercised the deleted code (disaggregated trt-backend tests, TRT bench/openai e2e params, mixed llmapi unittest suites) and delist them from test-db/qa/waives; shared test harnesses keep their signatures for the PyTorch twins. - Port test_gather_generation_logits_cuda_graph to the PyTorch accuracy suite; repoint examples/apps to the PyTorch LLM; regenerate ruff-legacy-baseline.json. Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>

Sever the C++ tree's compile- and link-time dependency on TensorRT so the shared core (runtime, batch manager, executor API, KV cache, sampling, kernels, nanobind bridge) builds, links, and runs without the TensorRT library. Follows the Python TensorRT-backend removal (TRTLLM-14022); PyTorch is the sole backend. Internal types (serialization-compatible): - add tensorrt_llm::DataType/Dims/ILogger (common/tllmDataType.h); DataType enumerator values mirror nvinfer1::DataType for byte-compatible serialization, Dims layout mirrors nvinfer1::Dims - migrate nvinfer1::DataType/Dims/ILogger -> tensorrt_llm:: across 206 surviving files; replace NvInfer*.h includes with the internal header; drop 12 dead NvInfer includes from files that never referenced nvinfer1 - add tests/unittest/bindings/test_datatype_parity.py guarding the enumerator values (auto-scheduled via the existing unittest/bindings test-db entries) Remove the TensorRT-engine execution path (unused by the PyTorch backend): - plugins/ (nvinfer_plugin_tensorrt_llm), engine runtime wrappers (tllmRuntime, tllmStreamReaders, layerProfiler, rawEngine, tllmLogger), TRT model adapters (trtGptModel*/trtEncoderModel/trtGptModelFactory), executor.cpp/executorImpl, the C++ Executor + TllmRuntime + LogitsPostProcessor nanobind bindings (0 Python users each), executor_worker, disaggServerUtil, engine I/O buffers and engine-only logits/decoder algos, executor/model.h (0 consumers), the ModelSpec test helper + binding (orphaned) - relocate the retained KVCacheEvent ctor and executor::version() into executor/kvCacheEvent.cpp - decouple shared files (inflightBatchingUtils, medusaBuffers, lookaheadBuffers, dataTransceiver) from the removed engine runtime - delete the engine-driven C++ tests (e2e_tests engine tests, executorTestSmall*, tllmRuntimeTest, encDecBeamSearchTest, tests/utils engine builders), C++ benchmarks (bertBenchmark, gptManagerBenchmark, disaggServerBenchmark; the prepare_dataset.py tooling used by trtllm-bench stays), examples/cpp executor examples, and the cpp/tests/resources engine-build scripts - fix kept tests: strip vestigial never-consumed TllmLogger members (7 files), repoint ropeTest.cu to kernels/gptKernels.h, inline the engine-free createDecoderBatchInputs helper into gptDecoderBatchedTest - remove the IS_BUILDING build-time env contract end to end (common/opUtils.h isBuilding + attentionOp gate + _common.py half) - remove the llm_args=None engine path from executor/base_worker.py (tllm.Executor no longer exists); llm_args is now required Build/packaging (no TensorRT): - drop find_package(TensorRT)/TRT_LIB/NvInfer include injection and the plugins/executor_worker/benchmarks subdirs from the cpp CMake; delete FindTensorRT.cmake - build_wheel.py: trt_root optional (default None), drop the tensorrt venv check and the nvinfer_plugin/executorWorker/benchmarks targets - setup.py no longer packages libnvinfer_plugin_tensorrt_llm.so / executorWorker; requirements.txt drops tensorrt - jenkins/Build.groovy: drop --benchmarks, the benchmark/libnvinfer-plugin tarball packaging, and the build_cpp_examples.py step - delist the removed tests from test-db/waives.txt/.test_durations and prune tests/integration/defs/cpp to the shared-core gtest wrappers Signed-off-by: Wanli Jiang <35160485+Wanli-Jiang@users.noreply.github.com>

Wanli-Jiang · 2026-07-10T09:26:50Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-07-10T09:32:32Z

PR_Github #58652 [ run ] triggered by Bot. Commit: 4401321 Link to invocation

tensorrt-cicd · 2026-07-11T03:06:05Z

PR_Github #58652 [ run ] completed with state SUCCESS. Commit: 4401321
/LLM/main/L0_MergeRequest_PR pipeline #47240 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

github-actions Bot assigned Wanli-Jiang Jul 10, 2026

Wanli-Jiang added the api-breaking Accepted LLM API contract change that is backwards-incompatible label Jul 10, 2026

Wanli-Jiang added 2 commits July 10, 2026 17:21

Wanli-Jiang force-pushed the user/williamj/deprecated-trt-backend-cpp-removal branch from 53d64f3 to 4401321 Compare July 10, 2026 09:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[TRTLLM-14026][feat] BREAKING: Remove TensorRT from the C++ tree#16234

[TRTLLM-14026][feat] BREAKING: Remove TensorRT from the C++ tree#16234
Wanli-Jiang wants to merge 2 commits into
NVIDIA:mainfrom
Wanli-Jiang:user/williamj/deprecated-trt-backend-cpp-removal

Wanli-Jiang commented Jul 10, 2026 •

edited

Loading

Uh oh!

Wanli-Jiang commented Jul 10, 2026

Uh oh!

tensorrt-cicd commented Jul 10, 2026

Uh oh!

tensorrt-cicd commented Jul 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Wanli-Jiang commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

A. Internal types (serialization-compatible)

B. Engine execution path — deleted C++ sources

C. Relocations (retained symbols out of deleted files)

D. Shared C++ files decoupled from the removed engine runtime

E. Python executor layer (the engine path's last consumers)

F. Build & packaging (the final gate)

G. C++ tests — deleted (engine-driven) vs fixed (kept)

Deleted

Fixed (kept tests that referenced deleted files — the draft missed all of these)

H. Python test suite + CI metadata

I. Review-driven scope change (chronological)

J. Validation record

K. Follow-ups (deliberately NOT in this PR)

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

Wanli-Jiang commented Jul 10, 2026

Uh oh!

tensorrt-cicd commented Jul 10, 2026

Uh oh!

tensorrt-cicd commented Jul 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Wanli-Jiang commented Jul 10, 2026 •

edited

Loading