Skip to content

[Klaud Cold] Update dsr1-fp4-b200-sglang-mtp SGLang image to v0.5.20-cu130 / [Klaud Cold] 将 dsr1-fp4-b200-sglang-mtp 的 SGLang 镜像更新至 v0.5.20-cu130 #3634

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Klaud-Cold wants to merge 2 commits into from
Closed

[Klaud Cold] Update dsr1-fp4-b200-sglang-mtp SGLang image to v0.5.20-cu130 / [Klaud Cold] 将 dsr1-fp4-b200-sglang-mtp 的 SGLang 镜像更新至 v0.5.20-cu130 #3634

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d809d79
perf(dsr1): bump B200 FP4 SGLang MTP image to v0.5.20-cu130
github-actions[bot] Oct 1, 2026
caf4afc
chore(changelog): record dsr1-fp4-b200-sglang-mtp SGLang v0.5.20 bump
github-actions[bot] Oct 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions inferencex-e2e/benchmarks/single_node/srt-slurm-recipes/dsr1/sglang/b200-fp4-mtp/8k1k.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ base:
name: dsr1-fp4-b200-sglang-mtp-8k1k
model:
path: hf:nvidia/DeepSeek-R1-0528-FP4-V2
container: lmsysorg/sglang:v0.5.16-cu130
container: lmsysorg/sglang:v0.5.20-cu130@sha256:06e4f2ed21afde4ff513cda65070124e727ba23ccaeff7712b8c40e1097d611f
precision: fp4
resources:
gpu_type: b200
Expand All @@ -25,7 +25,7 @@ base:
trust-remote-code: true
tensor-parallel-size: 4
data-parallel-size: 1
cuda-graph-max-bs: 256
cuda-graph-max-bs-decode: 256
max-running-requests: 256
mem-fraction-static: 0.85
kv-cache-dtype: fp8_e4m3
Expand All @@ -34,7 +34,7 @@ base:
quantization: modelopt_fp4
enable-flashinfer-allreduce-fusion: true
scheduler-recv-interval: 10
disable-piecewise-cuda-graph: true
cuda-graph-backend-prefill: disabled
attention-backend: trtllm_mla
moe-runner-backend: flashinfer_trtllm
stream-interval: 10
Expand Down
2 changes: 1 addition & 1 deletion inferencex-e2e/configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -893,7 +893,7 @@ dsr1-fp4-b200-sglang:
# - { tp: 8, ep: 8, offloading: none, conc-list: [1, 2, 4, 8, 12, 16, 32, 64, 128, 256, 512] }

dsr1-fp4-b200-sglang-mtp:
image: lmsysorg/sglang:v0.5.16-cu130
image: lmsysorg/sglang:v0.5.20-cu130@sha256:06e4f2ed21afde4ff513cda65070124e727ba23ccaeff7712b8c40e1097d611f
model: nvidia/DeepSeek-R1-0528-FP4-V2
model-prefix: dsr1
runner: cluster:b200-nscale
Expand Down
6 changes: 6 additions & 0 deletions inferencex-e2e/perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9168,3 +9168,9 @@
- "Relevant ATOM changes in the range: V4 decode reuses the sparse prefill ASM (ROCm/ATOM#2271), greedy sampler picks via aiter.topk_select (ROCm/ATOM#2244), and FP8 block scales declared scale_fmt ue8m0 are now stored as E8M0 on gfx950 by default (ROCm/ATOM#2419; previously FP32 unless ATOM_FP8_BLOCKSCALE_USE_E8M0_SCALE=1). The upstream DeepSeek-V4 recipes are unchanged across the range, and every recipe flag and choice (all2all-backend rccl, dp-load-balance least_tokens, moe-backend standard) remains valid."
- "No data-type or precision change to the DeepSeek-V4-Pro-0813 DSpark draft: no online quantization is configured, so it keeps its checkpoint precision; the E8M0 scale storage represents the checkpoint's power-of-two block scales exactly. kv-cache-dtype and index-cache-dtype touch cache storage only."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3605

- config-keys:
- dsr1-fp4-b200-sglang-mtp
description:
- "Update SGLang from v0.5.16-cu130 to v0.5.20-cu130 and rename two CUDA graph flags removed upstream."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3634
Loading