Skip to content

[NV] Add Qwen3.5 FP4 GB300 TRT-LLM Disaggregated AgentX / 新增 Qwen3.5 FP4 GB300 TRT-LLM 分离式 AgentX - #2584

Closed
richardhuo-nv wants to merge 3 commits into
mainfrom
rihuo/dynamo-trt-qwen3.5-fp4-agentx
Closed

[NV] Add Qwen3.5 FP4 GB300 TRT-LLM Disaggregated AgentX / 新增 Qwen3.5 FP4 GB300 TRT-LLM 分离式 AgentX#2584
richardhuo-nv wants to merge 3 commits into
mainfrom
rihuo/dynamo-trt-qwen3.5-fp4-agentx

Conversation

@richardhuo-nv

Copy link
Copy Markdown
Collaborator

Add Qwen3.5 FP4 GB300 TRT-LLM Disaggregated AgentX / 新增 Qwen3.5 FP4 GB300 TRT-LLM 分离式 AgentX

Summary / 摘要

Registers nine disaggregated Dynamo-TRT agentic benchmark recipes for Qwen3.5-397B-A17B-NVFP4-V2 on GB300, covering a range of topologies from single-node small-scale to large multi-node high-throughput.

新增九个 Qwen3.5-397B-A17B-NVFP4-V2 GB300 分离式 Dynamo-TRT AgentX Benchmark 配置,覆盖从单节点小规模到大型多节点高吞吐量的一系列拓扑结构。

Changes / 变更内容

configs/nvidia-master.yaml

  • New config key qwen3.5-fp4-gb300-dynamo-trt-agentic-disagg with scenario-type: agentic-coding, image nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc24, model nvidia/Qwen3.5-397B-A17B-NVFP4-V2, Dynamo router 1.4.0.dev20260807, and NIXL KV transfer.
  • Nine search-space entries with MTP speculative decoding, covering tep (attention-DP disabled) and dep (attention-DP enabled) topology families:
Recipe Prefill Decode conc
1p1d-tep2-tep4 1×TP2/EP2 1×TP4/EP4 8
2p3d-tep2-tep4 2×TP2/EP2 3×TP4/EP4 6, 24
2p3d-tep2-tep8 2×TP2/EP2 3×TP8/EP8 6
4p1d-dep4-dep8 4×TP4/EP4 1×TP8/EP8 896
15p1d-dep1-dep16 15×TP1/EP1 1×TP16/EP16 606
17p3d-dep2-dep8 17×TP2/EP2 3×TP8/EP8 1452
19p1d-dep1-dep16 19×TP1/EP1 1×TP16/EP16 225, 449

benchmarks/multi_node/srt-slurm-recipes/trtllm/qwen3.5/gb300-fp4/disagg/agentx/

  • Nine recipe YAMLs added, sourced from srt-slurm/recipes/trtllm/qwen3.5-fp4/agentx/submission/.

runners/launch_gb300-nv.sh

  • New branch for IS_AGENTIC=1 && dynamo-trt && qwen3.5: clones NVIDIA/srt-slurm at v1.0.48 and overlays the agentx recipe directory.

perf-changelog.yaml

  • New entry for qwen3.5-fp4-gb300-dynamo-trt-agentic-disagg.

Validation / 验证

Matrix generator produces all 9 expected agentic-coding rows:

qwen3.5_p1x2ep2_d1x4ep4_conc8
qwen3.5_p2x2ep2_d3x4ep4_conc6
qwen3.5_p2x2ep2_d3x8ep8_conc6
qwen3.5_p2x2ep2_d3x4ep4_conc24
qwen3.5_p4x4ep4dpa_d1x8ep8dpa_conc896
qwen3.5_p15x1dpa_d1x16ep16dpa_conc606
qwen3.5_p17x2ep2dpa_d3x8ep8dpa_conc1452
qwen3.5_p19x1dpa_d1x16ep16dpa_conc225
qwen3.5_p19x1dpa_d1x16ep16dpa_conc449

@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs


感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have

如需进行 PR 验证,请为此 PR 添加 full-sweep-fail-fast 标签(强烈推荐)— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

Comment thread perf-changelog.yaml Outdated
@github-actions

Copy link
Copy Markdown
Contributor

@github-actions

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Development

Successfully merging this pull request may close these issues.

1 participant