[NVIDIA][AgentX] GB300 DeepSeek-V4.1-Flash Dynamo+SGLang aggregated and 1PxD recipes / [NVIDIA][AgentX] GB300 DeepSeek-V4.1-Flash Dynamo+SGLang 聚合与 1PxD 配方 - #3598
Conversation
|
Thanks for the contribution!
中文感谢你的贡献！
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs. Because it changes SLURM launcher routing (new partition/account/shared-root branch) for multi-node GB300 jobs, a human look would still be worthwhile to confirm the operational details before merge.
What was reviewed:
- Launcher branch in launch_gb300-nv.sh: check_env_vars now covers both SLURM_PARTITION and SLURM_ACCOUNT, and the new values are set via if/else rather than
:-fallback. - GB300_SHARED_ROOT is "/data/home/slurm-shared/gharunners" on the new restricted route vs "sa-shared" used everywhere else in the repo — plausible if the restricted partition has its own mount, but not verifiable from source alone.
- perf-changelog.yaml entry is appended at the file's tail as required; its pr-link is a "pull/0" placeholder rather than the documented pre-PR "TBD" (docs/configuration-procedures.md:636).
- The new agg/disagg recipe files are paired with matching master-config entries as required for multi-node srt-slurm additions.
Extended reasoning...
The diff adds a GB300 multi-node AgentX benchmark route (two srt-slurm recipe files, master-config wiring, a changelog entry, and new conditional branching in the shared launch_gb300-nv.sh script that selects a different SLURM partition/account and a new shared-storage root for this model/framework combination). It touches no auth, crypto, or data-exposure surface; the risk is purely operational (job routing and shared-cache paths for a NVIDIA benchmark CI runner). The bug hunter reported no findings, but the launcher change introduces production-affecting branching logic and an unverified shared-root path divergence from repo convention, plus a changelog pr-link that deviates from the documented pre-merge placeholder convention, which together are enough that a human familiar with the cluster layout should confirm before merge.
This review covers commit 74b1996, which is no longer the latest commit on this pull request; later commits are not covered by it.
|
Heads-up: #3576 (merged) replaced the bash launchers with a Python launcher, so this PR will conflict when you merge |
Port the nine exercised aggregated and disaggregated AgentX points to the pluggable launcher. Add a named restricted GB300 Slurm route for the alternate partition and shared storage.\n\n将九个已验证的聚合与分离式 AgentX 点迁移到可插拔启动器，并为备用分区和共享存储添加命名的 GB300 restricted Slurm 路由。
6ff9e47 to
7ababb3
Compare
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=36843464922 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=36843464922 |
nvpohanh
changed the title
Cover every vertex of the measured Dynamo+SGLang Pareto curve in the two AgentX recipe files: add the two-GPU TP2/EP2 c1 arm, the 1P2D c48 and spread c16 cells, and the HiCache prefill tier at 1P1D c160 and 2P1D c256; move the multi-decode cells (1P2D c48/c64, 1P4D c64, 2P1D c256) to the 1 s session-affinity TTL they were measured and GSM8K-gated with; refresh the measured numbers in the file headers. 在两个 AgentX 配方文件中覆盖已测得的 Dynamo+SGLang Pareto 曲线的每个顶点：新增双 GPU 的 TP2/EP2 c1 配置、1P2D c48 与跨节点 c16 单元，以及 1P1D c160 与 2P1D c256 的 HiCache prefill 层；将多 decode 单元（1P2D c48/c64、1P4D c64、2P1D c256）改为其实际测量与 GSM8K 验证所用的 1 秒会话亲和 TTL；同时更新文件头中的测量数据。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Trim the aggregated and disaggregated AgentX recipes and the nvidia-master sweep to the eight measured Pareto vertices: agg TP4/EP1 c1 and TP2/EP2 c1; disagg 1P2D spread c16, 1P2D c48, 1P2D c64, 1P1D c96, 1P1D HiCache c160 and 2P1D HiCache c256. Drop the dominated aggregated c64 (prefill-decode-interval 8/16), 1P1D c8/c16/c64 and 1P4D c64 arms, add master entries for the new overrides, and update the perf-changelog description to the shipped set. 将聚合与分离式 AgentX 配方以及 nvidia-master sweep 精简为实测 Pareto 曲线的八个顶点：聚合 TP4/EP1 c1 与 TP2/EP2 c1；分离式 1P2D 跨节点 c16、1P2D c48、1P2D c64、1P1D c96、1P1D HiCache c160 与 2P1D HiCache c256。移除被支配的聚合 c64（prefill-decode-interval 8/16）、1P1D c8/c16/c64 与 1P4D c64 配置，为新增 override 添加 master 条目，并将 perf-changelog 描述更新为最终交付集合。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Re-append the perf-changelog entry after main's newer entries so the changelog stays append-only. 将 upstream/main 合并进 pohanh/dsv41flash-gb300-agentx-curve，并把 perf-changelog 条目重新追加到 main 的新条目之后，保持只增不删。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…m route Drop the named `restricted` GB300 Slurm route (batch_2 partition, restricted account, alternate shared caches). The CI runners are not associated with that account, so the canary failed at image import with "Invalid account or account/partition combination specified". The recipes now use the default gb300-nv partition, account and shared caches, like the other GB300 DeepSeek-V4.1-Flash configs. Revert the route plumbing in the Slurm settings, SRT launcher, tests and docs, and drop the matching changelog bullet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve the perf-changelog.yaml conflict by keeping main and appending this PR's entry at the end. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…egated recipe The aggregated AgentX power path now requires TP, PP_SIZE and PCP_SIZE before replay. Set PP_SIZE=1 and PCP_SIZE=1 in the base benchmark env, as the Qwen3.5 GB300 aggregated recipe does, so the replay no longer exits on missing inputs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
2 participants
Description
Add DeepSeek-V4.1-Flash FP4 AgentX recipes for GB300 with Dynamo + SGLang:
The recipes run on the default
gb300-nvSlurm partition, account and shared caches, like the other GB300 DeepSeek-V4.1-Flash configs. No launcher or cluster-schema changes.The stack is digest-pinned
lmsysorg/sglang:nightly-dev-20260928-81f27fb3with ai-dynamo1.6.0.dev20260928, Dynamo KV routing with session affinity, DSpark block size 5, and Mooncake KV transfer for disaggregated points.Validation
configs/runners.yaml(unchanged frommain);1/1/1/2/2/2/2/2/3;infx.workflows.validate_perf_changelogagainst the latestmain;AI model disclosure
main, resolved the launcher-refactor conflict, prepared this description, and monitored CI. Claude Code (Claude Opus 5.5) monitored the sweep and removed a GB300 Slurm route that CI runners could not use, so the recipes run on the default route. Final configuration selection and submission are by Po-Han Huang.Type of Change
Checklist
inferencex-e2e/perf-changelog.yaml中文
说明
新增 GB300 上 DeepSeek-V4.1-Flash FP4 的 Dynamo + SGLang AgentX 配方：
配方使用默认的
gb300-nvSlurm 分区、账号和共享缓存，与其他 GB300 DeepSeek-V4.1-Flash 配置一致；不修改启动器或集群 schema。软件栈固定为摘要锁定的
lmsysorg/sglang:nightly-dev-20260928-81f27fb3、ai-dynamo1.6.0.dev20260928、带会话亲和性的 Dynamo KV 路由、DSpark block size 5；分离式点使用 Mooncake KV 传输。验证
configs/runners.yaml（与main相同）；1/1/1/2/2/2/2/2/3；infx.workflows.validate_perf_changelog在最新main上通过；AI 模型披露
main，解决启动器重构冲突，准备本说明并监控 CI。Claude Code（Claude Opus 5.5）监控 sweep，并移除了 CI runner 无法使用的 GB300 Slurm 路由，使配方使用默认路由。最终配置选择和提交由 Po-Han Huang 完成。