Skip to content

[Klaud Cold] Update dsr1-fp4-b200-sglang-mtp SGLang image to v0.5.20-cu130 / [Klaud Cold] 将 dsr1-fp4-b200-sglang-mtp 的 SGLang 镜像更新至 v0.5.20-cu130 - #3634

Closed
Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-72260f70af9bbbbc-555679de269db36f
Closed

[Klaud Cold] Update dsr1-fp4-b200-sglang-mtp SGLang image to v0.5.20-cu130 / [Klaud Cold] 将 dsr1-fp4-b200-sglang-mtp 的 SGLang 镜像更新至 v0.5.20-cu130#3634
Klaud-Cold wants to merge 2 commits into
mainfrom
klaud/auto-72260f70af9bbbbc-555679de269db36f

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang:v0.5.16-cu130 to lmsysorg/sglang:v0.5.20-cu130@sha256:06e4f2ed21afde4ff513cda65070124e727ba23ccaeff7712b8c40e1097d611f.
Baseline: 2026-08-06 · lmsysorg/sglang:v0.5.16-cu130
Mean latency · Sources: API 1, API 2, API 3

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
8k/1k c1 TP4 EP1 1ecad2 N/A N/A N/A N/A
8k/1k c1 TP4 EP1 e3bd32 664.03 74.17 205.09 3.16
8k/1k c2 TP4 EP1 0ce74b 1,120.85 125.76 250.4 3.59
8k/1k c2 TP4 EP1 b79296 N/A N/A N/A N/A
8k/1k c4 TP4 EP1 0b6331 1,649.89 183.51 296.12 4.98
8k/1k c4 TP4 EP1 d56e03 N/A N/A N/A N/A
8k/1k c8 TP4 EP1 298215 N/A N/A N/A N/A
8k/1k c8 TP4 EP1 5f7481 2,321.26 260.92 412.66 7
8k/1k c16 TP4 EP1 5f8bdf 2,940.82 325.82 826.07 11.1
8k/1k c16 TP4 EP1 c2ec41 N/A N/A N/A N/A
8k/1k c32 TP4 EP1 28dffa 3,839.76 429.34 1,288.53 16.88
8k/1k c32 TP4 EP1 fb2b10 N/A N/A N/A N/A
8k/1k c64 TP4 EP4 6b9bd7 N/A N/A N/A N/A
8k/1k c64 TP4 EP4 c1061a 5,786.09 641.84 1,462.11 22.72
8k/1k c128 TP4 EP4 c02deb N/A N/A N/A N/A
8k/1k c128 TP4 EP4 c4328d 7,290.73 807.46 1,991.22 36.79
8k/1k c256 TP4 EP4 4cddc2 8,783.42 976.13 6,426.93 57.6
8k/1k c256 TP4 EP4 98a050 N/A N/A N/A N/A

Note: 8k/1k c1 TP4 EP1 1ecad2, 8k/1k c2 TP4 EP1 b79296, 8k/1k c4 TP4 EP1 d56e03, 8k/1k c8 TP4 EP1 298215, 8k/1k c16 TP4 EP1 c2ec41, 8k/1k c32 TP4 EP1 fb2b10, 8k/1k c64 TP4 EP4 6b9bd7, 8k/1k c128 TP4 EP4 c02deb, 8k/1k c256 TP4 EP4 98a050: unavailable.

Eval Score ↑ Samples
gsm8k/em_strict · c32 TP4 EP1 95.75% 1,319
gsm8k/em_strict · c128 TP4 EP4 DPA 95.83% 1,319
gsm8k/em_strict · c256 TP4 EP4 DPA 94.92% 1,319
中文

**目标：**将 SGLang 镜像从 lmsysorg/sglang:v0.5.16-cu130 更新为 lmsysorg/sglang:v0.5.20-cu130@sha256:06e4f2ed21afde4ff513cda65070124e727ba23ccaeff7712b8c40e1097d611f。
**基线：**2026-08-06 · lmsysorg/sglang:v0.5.16-cu130
平均延迟 · 来源： API 1, API 2, API 3；数值及异常说明见上表。

@github-actions @claude
perf(dsr1): bump B200 FP4 SGLang MTP image to v0.5.20-cu130
d809d79 
Pin lmsysorg/sglang:v0.5.20-cu130 by digest and replace the CLI flags
SGLang removed after v0.5.16: --cuda-graph-max-bs becomes
--cuda-graph-max-bs-decode and --disable-piecewise-cuda-graph becomes
--cuda-graph-backend-prefill=disabled (the exact v0.5.16 alias targets).

将 dsr1-fp4-b200-sglang-mtp 的 SGLang 镜像更新为按 digest 固定的
lmsysorg/sglang:v0.5.20-cu130，并替换 v0.5.16 之后移除的 CLI 参数：
--cuda-graph-max-bs 改为 --cuda-graph-max-bs-decode，
--disable-piecewise-cuda-graph 改为 --cuda-graph-backend-prefill=disabled
（即 v0.5.16 中这两个别名的实际目标）。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps only run on labeled PRs. Add full-sweep-fail-fast (strongly recommended); use full-sweep-enabled only when matrix jobs should continue after a failure.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献！

  • **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
  • **PR 验证：**扫描仅在带有标签的 PR 上运行。强烈建议添加 full-sweep-fail-fast；仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。
  • **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Passed · Run 36864810140 / attempt 1 · 2026-10-01 13:23 UTC
lmsysorg/sglang:v0.5.20-cu130@sha256:06e4f2ed21afde4ff513cda65070124e727ba23ccaeff7712b8c40e1097d611f · 1c5e6b9f5e7b · Mean latency
Change: Measured ref d809d79b: pin SGLang v0.5.20-cu130 by digest and rename the flags removed since v0.5.16 to their alias targets: cuda-graph-max-bs → cuda-graph-max-bs-decode and disable-piecewise-cuda-graph → cuda-graph-backend-prefill: disabled (v0.5.16 aliases, v0.5.20 fields); other recipe flags and env are unchanged in the v0.5.20 parser.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
8k/1k c1 TP4 EP1 1ecad2 78.3 (N/A) 211.29 (N/A) 2.97 (N/A)
8k/1k c64 TP4 EP4 6b9bd7 674.42 (N/A) 1,395.99 (N/A) 21.64 (N/A)

Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.

Eval Score ↑ Samples
gsm8k/em_strict · c32 TP4 EP1 95.53% (N/A) N/A/1,319 (old/new)
gsm8k/em_strict · c128 TP4 EP4 DPA 95.15% (N/A) N/A/1,319 (old/new)

Note: All rows: Δ N/A: no matched eval baseline.

Next: Stop without a final sweep: check-final reports the 9 published baseline points (producer 5430f5cf, runner b200, pre-srt-recipe launcher) absent from today's cluster:b200-nscale srt-recipe family, so they cannot receive updated-image results.

中文

初次尝试 · 已通过 · Run 36864810140 / attempt 1 · 2026-10-01 13:23 UTC
lmsysorg/sglang:v0.5.20-cu130@sha256:06e4f2ed21afde4ff513cda65070124e727ba23ccaeff7712b8c40e1097d611f · 1c5e6b9f5e7b · 平均延迟
**变更：**测量提交 d809d79b：按 digest 固定 SGLang v0.5.20-cu130，并将 v0.5.16 之后移除的参数改为其别名目标：cuda-graph-max-bs → cuda-graph-max-bs-decode，disable-piecewise-cuda-graph → cuda-graph-backend-prefill: disabled；其余配方参数与环境变量在 v0.5.20 解析器中保持不变。实测数值及异常说明见上表。
**下一步：**不运行最终 sweep 并停止：check-final 报告 9 个已发布基线点（producer 5430f5cf，runner b200，srt-recipe 之前的启动方式）不在当前 cluster:b200-nscale srt-recipe 配置族中，无法获得更新镜像结果。

@github-actions @claude
chore(changelog): record dsr1-fp4-b200-sglang-mtp SGLang v0.5.20 bump
caf4afc 
为 dsr1-fp4-b200-sglang-mtp 的 SGLang v0.5.20 镜像更新追加 perf-changelog 条目。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Klaud-Cold

Klaud-Cold commented Oct 1, 2026

Copy link
Copy Markdown
Collaborator Author

failed · Cleanup pending. Stop and confirm owned runs before closing.

中文

failed · 清理待完成。先停止并确认自有运行结束，再关闭 PR。

@Klaud-Cold Klaud-Cold closed this Oct 1, 2026
@Klaud-Cold
Klaud-Cold deleted the klaud/auto-72260f70af9bbbbc-555679de269db36f branch October 1, 2026 13:23
@Klaud-Cold

Klaud-Cold commented Oct 1, 2026

Copy link
Copy Markdown
Collaborator Author

failed · Repairs: 0 · Runs: 36864810140, 36868124483
All owned runs ended. PR closed; branch deleted for retry.

中文

failed · 修复次数：0 · 运行：36864810140, 36868124483
所有自有运行均已结束。PR 已关闭；分支已删除，可重新尝试。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Ankur-singh Ankur-singh Awaiting requested review from Ankur-singh Ankur-singh will be requested when the pull request is marked ready for review Ankur-singh is a code owner
@kedarpotdar-nv kedarpotdar-nv Awaiting requested review from kedarpotdar-nv kedarpotdar-nv will be requested when the pull request is marked ready for review kedarpotdar-nv is a code owner
@wzhao18 wzhao18 Awaiting requested review from wzhao18 wzhao18 will be requested when the pull request is marked ready for review wzhao18 is a code owner
@nvpohanh nvpohanh Awaiting requested review from nvpohanh nvpohanh will be requested when the pull request is marked ready for review nvpohanh is a code owner
@RohitNagraj RohitNagraj Awaiting requested review from RohitNagraj RohitNagraj will be requested when the pull request is marked ready for review RohitNagraj is a code owner

Assignees

No one assigned

Labels

None yet

Projects

InferenceX Board
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Klaud-Cold