-
Notifications
You must be signed in to change notification settings - Fork 2.6k
- #15044 · laikhtewari opened
on Jun 6, 2026 1 - #3148 · juney-nvidia opened
on Mar 29, 2025 5 - #3124 · juney-nvidia opened
on Mar 27, 2025 11
Issues
is:issue state:open
is:issue state:open
Issue creation is restricted in this repository
Search results
[Bug] Default PDL enablement fails FlashInfer RMSNorm JIT on SM89 GPUs
bugSomething isn't workingSomething isn't workingCustomized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.Status: Open.#16330 In NVIDIA/TensorRT-LLM;[Bug] test_unittests_v2[unittest/_torch/executor] wrapper fails intermittently (~40%) with all inner tests passing — session teardown crash
Pytorch<NV>Pytorch backend related issues<NV>Pytorch backend related issuesTesting<NV>Continuous integration, build system, and testing infrastructure issues<NV>Continuous integration, build system, and testing infrastructure issuesStatus: Open.#16284 In NVIDIA/TensorRT-LLM;[Bug] test_nvfp4_gather_grouped_gemm_act_fusion_blackwell flaky on B300: ~0.05% mismatch vs rtol 1e-4, varying parametrizations
Pytorch<NV>Pytorch backend related issues<NV>Pytorch backend related issuesStatus: Open.#16281 In NVIDIA/TensorRT-LLM;[Bug] test_dflash_qwen3_5_4b flaky-marginal in DGX_H100-PyTorch-5: mean accepted 0.9909 < 1.0, alternating [True]/[False] variants
Speculative Decoding<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafterTesting<NV>Continuous integration, build system, and testing infrastructure issues<NV>Continuous integration, build system, and testing infrastructure issuesStatus: Open.#16277 In NVIDIA/TensorRT-LLM;[Bug] TestGemma4MoE::test_bf16 fails deterministically in pre-merge DGX_H100-4_GPUs-AutoDeploy-1: Not registered specs (bf16/auto)
AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy BackendStatus: Open.#16275 In NVIDIA/TensorRT-LLM;[Feature]: Extend the v1 KV cache manager with a disk (L3) tier and per-request retention
KV-Cache Managementkv-cache management for efficient LLM inferencekv-cache management for efficient LLM inferenceStatus: Open.#16251 In NVIDIA/TensorRT-LLM;[DeepSeek-V4] Loading nvidia/DeepSeek-V4-Pro-NVFP4 crashes: 'Linear' object has no attribute 'weight_scale' in fused_a loader (MIXED_PRECISION attn excluded → built BF16 but stored as FP8)
Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.Model optimization<NV>Model-specific performance optimizations and tuning<NV>Model-specific performance optimizations and tuningStatus: Open.#16196 In NVIDIA/TensorRT-LLM;[Bug]: Gemma4 INT4 AWQ quantization fails during export: Gemma4Config has no attribute vocab_size
bugSomething isn't workingSomething isn't workingCustomized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.Status: Open.#16171 In NVIDIA/TensorRT-LLM;[Feature]: Expose SmoothQuant alpha for int8_sq quantization in quantize.py
feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality supportModel optimization<NV>Model-specific performance optimizations and tuning<NV>Model-specific performance optimizations and tuningStatus: Open.#16151 In NVIDIA/TensorRT-LLM;[Bug]: 142 GiB/rank host memory leak (leading to host OOM if KV cache offloading is enabled) when serving Kimi-K2.6-NVFP4 TP4 on GB300 (aarch64), due to MetaInitException on aten.fill_.Scalar (nn.LayerNorm init) silently falling back to whole-model CPU construction
Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.Pytorch<NV>Pytorch backend related issues<NV>Pytorch backend related issuesStatus: Open.#16086 In NVIDIA/TensorRT-LLM;[AutoDeploy]: Migrate btk to support llmc
AutoDeploy/llmc-blocker<NV> Tag for issues that are blocking AutoDeploy standalone repo<NV> Tag for issues that are blocking AutoDeploy standalone repofeature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality supportStatus: Open.#16076 In NVIDIA/TensorRT-LLM;[Bug]: IndexError: tuple index out of range when loading Qwen 3.6 35B during Quantizing
bugSomething isn't workingSomething isn't workingCustomized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.Status: Open.#16075 In NVIDIA/TensorRT-LLM;