Skip to content

feat(dsv4): add B300 AgentX NIXL configurations - #3597

Open
nvpohanh wants to merge 14 commits into
mainfrom
dsv4-b300-agentx-nixl
Open

feat(dsv4): add B300 AgentX NIXL configurations#3597
nvpohanh wants to merge 14 commits into
mainfrom
dsv4-b300-agentx-nixl

Conversation

@nvpohanh

@nvpohanh nvpohanh commented Sep 29, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

[by Codex]

Adds DeepSeek-V4-Pro-0813 FP4 B300 Dynamo+SGLang AgentX configurations using NIXL for disaggregated KV transfer.

Changes:

  • add aggregate TP8 c1 and TP4 c4 configurations
  • add disaggregated 1P1D DEP8/DEP8 c64 and c240 plus 2P1D DEP8/DEP8 c480
  • use NIXL with the LIBFABRIC backend for the B300 DSXE fabric
  • rely on the networking stack supplied by the pinned container and DSXE runtime, with no explicit host EFA/OFI mounts or runtime transport setup
  • keep HiCache DRAM KV offload and DSpark settings aligned across prefill and decode

Validation:

  • all five srt-slurm recipes migrate and resolve identically
  • matrix generation emits aggregate c1/c4 and disaggregated c64/c240/c480
  • perf-changelog, runner, and NVIDIA master-config validation pass
  • focused launcher, cluster, matrix, and changelog tests pass

@nvpohanh
nvpohanh requested a review from a team September 29, 2026 20:45
@github-actions

github-actions Bot commented Sep 29, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps only run on labeled PRs. Add full-sweep-fail-fast (strongly recommended); use full-sweep-enabled only when matrix jobs should continue after a failure.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献！

  • **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
  • **PR 验证：**扫描仅在带有标签的 PR 上运行。强烈建议添加 full-sweep-fail-fast；仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。
  • **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

functionstackx
functionstackx reviewed Sep 29, 2026
View reviewed changes
Comment thread inferencex-e2e/runners/srt-slurm/b300-dsxe.yaml Outdated
claude[bot]
claude Bot reviewed Sep 29, 2026
View reviewed changes
Comment thread inferencex-e2e/runners/srt-slurm/b300-dsxe.yaml Outdated
Comment thread inferencex-e2e/runners/srt-slurm/b300-dsxe.yaml Outdated
Comment thread inferencex-e2e/perf-changelog.yaml Outdated
@adibarra

adibarra commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator

Heads-up: #3576 (merged) replaced the bash launchers with a Python launcher, so this PR will conflict when you merge main, and the sweep won't start until that's resolved. Please merge main and move your launcher changes over to configs/runners.yaml / infx/launch/. Apologies for the churn, and thanks for your understanding as we wrap up the repo-wide refactoring push.

@nvpohanh
nvpohanh force-pushed the dsv4-b300-agentx-nixl branch from f151920 to edb8e4f Compare September 30, 2026 01:34
@github-actions

github-actions Bot commented Sep 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=36741786007

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=36741786007

@nvpohanh

nvpohanh commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator Author

[by Codex] Rebased onto latest main and migrated the remaining B300 AgentX cache behavior to configs/runners.yaml and infx/launch. The obsolete shell/YAML launcher changes and host EFA/OFI mounts are gone, all merge conflicts are resolved, and Mooncake installation/setup code remains removed. The full-sweep label was removed before pushing.

@nvpohanh
nvpohanh force-pushed the dsv4-b300-agentx-nixl branch from edb8e4f to f32f078 Compare September 30, 2026 01:37
@nvpohanh nvpohanh changed the title feat(dsv4): use NIXL for B300 disaggregation feat(dsv4): add B300 AgentX NIXL configurations Sep 30, 2026
@nvpohanh
nvpohanh force-pushed the dsv4-b300-agentx-nixl branch from f32f078 to d81b82b Compare September 30, 2026 02:21
nvpohanh and others added 14 commits September 30, 2026 09:05
@nvpohanh
fix(dsxe): migrate B300 AgentX recipes
e21ee13 
将 B300 AgentX 配方迁移到重构后的 inferencex-e2e 布局，并采用 EFA 1.50.0 安装流程。
@nvpohanh
fix(dsxe): avoid EFA preload during setup
916fae6 
移除 B300 分离式配方中的 LD_PRELOAD，避免 EFA 安装时向 dpkg 等初始化进程注入不兼容的 libfabric/libefa。
@nvpohanh
fix(dsxe): keep SGLang metrics directories stable
b7cebd1 
为 B300 分离式 prefill 和 decode 角色预建稳定的 Prometheus 多进程目录，避免调度器启动时临时目录被清理。
@nvpohanh
fix(dsxe): preserve Pyxis RDMA libraries for B300 AgentX
cee5a8e 
跳过 EFA 安装器的 rdma-core 升级，避免覆盖 Pyxis 挂载的 libibverbs，并确保安装失败时立即终止初始化。排除 TP8 c1 性能异常的 gpu-11 节点，追加性能变更记录。
@nvpohanh
fix(dsxe): use cluster RDMA stack for Mooncake EFA
86daade 
EFA 1.50 安装器与 DSXE Pyxis 挂载的 verbs 软件栈冲突；仅安装 Mooncake EFA wheel，并沿用集群提供的 RDMA 库。追加性能变更记录。
@nvpohanh
feat(dsxe): add B300 disaggregated c64 point
4366f34 
Replace the aggregate TP4 c8 point with a 1P1D DEP8 c64 recipe and remove the explicit Prometheus multiprocess-directory workaround.\n\n以 1P1D DEP8 c64 配方替换聚合式 TP4 c8 测试点，并移除显式的 Prometheus 多进程目录 workaround。
@nvpohanh
feat(dsv4): use NIXL for B300 disaggregation
12abcc3 
将三个 B300 Dynamo+SGLang 解耦配置从 Mooncake 切换为基于 UCX 的 NIXL，并移除 Mooncake EFA 安装脚本及相关环境变量。
@nvpohanh
fix(dsv4): use libfabric for B300 NIXL
58453ff
@nvpohanh
docs(perf): link B300 NIXL submission
8dfc5d8
@nvpohanh
fix(dsxe): migrate B300 cache mounts to Python launcher
5b923d1
@nvpohanh
docs(perf): describe final B300 NIXL configuration
148567f
@nvpohanh
fix(dsxe): defer B300 CPU allocation to runner
3763b4a
@nvpohanh @claude
Raise B300 DSV4 agg AgentX health-check budget to 8h
a27ae88 
The non-EP TP MoE aggregate recipes need a long weight-load and
post-load phase before the server is healthy: the TP8 c1 canary
took ~3h20m to become healthy on an otherwise idle node, and the
TP4 c4 worker was still loading when the 4h (1440 x 10s) health
budget expired. Raise max_attempts to 2880 (8h) for both agg
recipes; this stays within the 780-minute agentic job timeout and
does not change serving or benchmark settings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nvpohanh @claude
style(perf): restore changelog entry separator
8ddfe84 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nvpohanh
nvpohanh force-pushed the dsv4-b300-agentx-nixl branch from 540432c to 8ddfe84 Compare September 30, 2026 16:06

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@functionstackx functionstackx functionstackx left review comments
@claude claude[bot] claude[bot] left review comments
@Ankur-singh Ankur-singh Awaiting requested review from Ankur-singh Ankur-singh is a code owner
@kedarpotdar-nv kedarpotdar-nv Awaiting requested review from kedarpotdar-nv kedarpotdar-nv is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

full-sweep-fail-fast

Projects

InferenceX Board
Status: No status

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@nvpohanh @adibarra @functionstackx