[AMD][AgentX] dsv4 vllm-agentic MTP: pin the 08-09 nightly + additive container patches / dsv4 vllm-agentic MTP：改钉 08-09 nightly 并增量打补丁

[jiacao-amd]

Summary

Waiver: docs/waiver/2589.md

dsv4-fp4-mi355x-vllm-agentic-mtp cannot start on its pinned image — the server memory-faults during the profile run, before it ever serves a request. This PR repins it to the 08-09 nightly (f8d03e77) and patches the image before serving via apply_dsv4_container_patches.sh, which is the combination local bring-up actually gets to Application startup complete on.

The fault is in the base, not in this recipe's patches. It reproduces identically on a pristine 08-12 nightly (3ee2df30) with no patches applied at all: the eight TP workers serialize on /tmp/aiter_configs/bf16_tuned_gemm.csv.lock, and the worker holding the baton dies in the post-MoE bf16 GEMM while the other seven survive parked on the lock. Two standalone repros clear the loudest suspects — the inter_dim=384 MoE returns finite results at the exact faulting shape with preshuffled weights and the same kernel pair, and the bf16 GEMM passes on all four paths (gemm_a16w16_opus, asm, triton, torch) at both M=65536 and M=16384 × N=7168 × K=7168. It is non-deterministic across runs (different worker, different address, different kernel selection), which is why it is filed as a base regression under separate investigation rather than fixed here.

• Image: nightly-821717118f… → nightly-f8d03e77416bf90c49acbe50e233275722f02c4b. The entry carries a comment telling the next person not to bump it without re-running bring-up.
• Patch stack (apply_dsv4_container_patches.sh, strictly additive + idempotent, marker-gated):

Upstream PR	What it changes	Why it is needed on this base
vllm #51473	native MXFP4 TP8 shard allocation (fused_moe/oracle/mxfp4.py)	merged 2026-08-11, i.e. after this base — the 08-12 nightly already had it, the 08-09 one does not
vllm #51714	opt-in AITER gluon sparse-MLA decode for gfx950	open upstream; dormant unless VLLM_ROCM_DSV4_SPARSE_GLUON=1
vllm #51918	vllm/config/kernel.py only — registers the flydsl_mega_moe backend name	model-side hunks deliberately NOT taken (see below)
aiter #4417	large-token FlyDSL MoE launch/output limits	the base's vendored aiter predates it; the upstream diff does not apply (base still reads Dict[str, Dict]), so the hunks are grafted by anchor via graft_aiter_4417.py
aiter gluon kernel	_gluon_kernels/gfx950/attention/pa_decode_sparse.py	the one file the base genuinely lacks

• Nothing that ships in the base is overwritten. Three of the four pa_decode_sparse variants already ship and are left alone. An earlier revision of this script copied all four out of aiter@97d0c6e4, silently replacing base files with a cross-version transplant; the script now skips anything already present, and a verified re-run is a clean no-op (keep base ×4, graft SKIP, all PRs already present).
• #51918 is taken partially on purpose. MegaMoE's intranode kernel imports mori.ir.flydsl, which this base does not have (verified ModuleNotFoundError), so applying the model hunks would buy a backend that raises on first use. Registering only the backend name means a DEP8 row is rejected at config time with a clear error instead of failing deep inside a kernel. DEP8/MegaMoE is out of scope on this route.
• VLLM_ROCM_DSV4_SPARSE_GLUON=1 is defaulted on (overridable). The knob is read via os.environ rather than envs.py, so vLLM logs it as Unknown vLLM environment variable — that warning is expected and does not mean it was ignored.
• The sweep grid is unchanged in this PR. Grid changes ride on a follow-up once this config is green.

Sweep label

full-sweep-enabled rather than the full-sweep-fail-fast that AGENTS.md:95 recommends as the default. Justification: this config is coming back from not starting at all, so the first full sweep is as much a bring-up probe as a measurement. Under fail-fast the first flaky point cancels that matrix's remaining jobs and throws away in-flight results from the concurrency points that did work — exactly the "flaky config where one flake would kill a matrix's in-flight results" case AGENTS.md:95 names as the reason to reach for full-sweep-enabled. Once the config has a green sweep this should go back to full-sweep-fail-fast.

8k-in / 1k-out @ concurrency 32

Run on the patched 08-09 image (hold job 6707, node crsuse2-m2m-100), mirroring CI's run_benchmark_serving invocation (--random-range-ratio 0, --num-prompts 320, --max-concurrency 32, --ignore-eos):

Successful requests:                     320
Failed requests:                         0
Benchmark duration (s):                  337.27
Total token throughput (tok/s):          8744.00
Output token throughput (tok/s):         971.56
Median TTFT (ms):                        1584.26     Mean TTFT (ms):  2828.64
Median TPOT (ms):                        29.00       Mean TPOT (ms):  30.10
Median ITL  (ms):                        46.11       Mean ITL  (ms):  74.92
---------------Speculative Decoding---------------
Acceptance rate (%):                     49.71
Acceptance length:                       2.49
Per-position acceptance (%):  P0 100.00 | P1 49.12 | P2 0.00

Acceptance length lands at 2.49, exactly the synthetic_acceptance_length pinned in the recipe — the MTP fan-out behaves as the campaign measured it. Position-2 acceptance is 0.00% by construction: synthetic rejection is configured to stop there.

Plumbing route

Picking explicitly, per the handover's "do not silently pick one": Option 1 — keep the knobs inside the benchmark recipe (env-overridable defaults), do not extend the sweep-config schema. Rationale: Option 2 touches validation.py, generate_sweep_configs.py and run-sweep.yml:596-630, i.e. shared sweep infrastructure every config rides on, in service of one config that today cannot even start. That is the wrong order — earn the schema change after the config is green. Option 1's cost is that per-arm knobs live in the recipe rather than the YAML search-space; the measured price of the recipe-side default at c32 was −4.9%, which is a known, bounded, documented tradeoff rather than a shared-infrastructure risk. This PR is the minimal form of Option 1: the patch invocation plus the gluon env, with no schema change. If a later arm genuinely needs a per-row knob the YAML cannot express, that is the moment to revisit Option 2 on its own PR.

Test plan

• apply_dsv4_container_patches.sh applies cleanly on a fresh container of the pinned image, and re-running it is a verified no-op
• py_compile + import checks pass in-container for every patched module
• generate_sweep_configs.py full-sweep --config-files configs/amd-master.yaml --model-prefix dsv4 --framework vllm generates all 8 agentic rows on the new image with no schema error
• TP8 server reaches Application startup complete on the patched 08-09 image — the profile run and cudagraph capture, where the old pin faulted, both complete
• 8k-in / 1k-out at concurrency 32: 320/320 successful, 0 failed, 8744 tok/s total, median TPOT 29.0 ms, MTP acceptance length 2.49 (see above)
• Full-sweep validation green on MI355X across the conc ladder

---

中文说明

dsv4-fp4-mi355x-vllm-agentic-mtp 在其固定镜像上根本起不来 —— 服务在 profile run 阶段发生显存访问越界，还没开始服务请求就崩了。本 PR 将镜像改钉到 08-09 的 nightly（f8d03e77），并通过 apply_dsv4_container_patches.sh 在启动前打补丁；这个组合是本地 bring-up 真正能跑到 Application startup complete 的组合。

该故障出在基座本身，与本 recipe 的补丁无关。 在完全未打任何补丁的 原始 08-12 nightly（3ee2df30）上可以复现出完全相同的签名：8 个 TP worker 在 /tmp/aiter_configs/bf16_tuned_gemm.csv.lock 上串行排队，持有 baton 的那个 worker 在 MoE 之后的 bf16 GEMM 中死亡，其余 7 个因阻塞在锁上而幸存。两个独立的最小复现排除了最显眼的嫌疑对象：inter_dim=384 的 MoE 在完全相同的形状、相同 kernel 组合、预 shuffle 权重下返回有限值；bf16 GEMM 在 M=65536 与 M=16384 × N=7168 × K=7168 下四条路径全部通过。它在不同运行之间是非确定性的（worker、地址、kernel 选择都不同），因此这里将其作为基座回归单独跟进，而不是在本 PR 中修复。

• 镜像：nightly-821717118f… → nightly-f8d03e77416bf90c49acbe50e233275722f02c4b。配置项中留有注释，提醒后来者不要在未重跑 bring-up 的情况下升级它。
• 补丁栈：严格增量、幂等、按 marker 判断是否已应用。其中 vllm #51473 于 2026-08-11 合入，晚于这个基座 —— 08-12 的 nightly 已自带，08-09 的没有，所以这里必须打；aiter #4417 因基座早于 aiter 的 typing 现代化改造，上游 diff 无法应用，改为按锚点移植。
• 基座自带的文件一律不覆盖。四个 pa_decode_sparse 变体中有三个基座已自带，脚本予以保留，只补真正缺失的 gfx950 那一个。早前版本曾把四个文件全部从 aiter@97d0c6e4 拷入，等于用跨版本文件静默替换了基座文件；现在脚本会跳过任何已存在的文件，重复执行已验证为 no-op。
• #51918 只取一部分是有意为之。MegaMoE 的 intranode kernel 需要 mori.ir.flydsl，这个基座没有（已验证 ModuleNotFoundError），打上模型侧的 hunk 只会得到一个一用就抛异常的后端。只注册后端名字，可以让 DEP8 行在配置阶段被明确拒绝，而不是深入 kernel 之后才失败。DEP8/MegaMoE 在本路线中不在范围内。
• 8k/1k @ c32 已跑通：320/320 成功、0 失败，总吞吐 8744 tok/s，median TPOT 29.0 ms，MTP acceptance length 2.49，与 recipe 中钉住的 synthetic_acceptance_length 完全一致。
• 管线路线明确选择 Option 1：把旋钮留在 benchmark recipe 内（环境变量可覆盖的默认值），不扩展 sweep 配置 schema。理由是 Option 2 要动 validation.py、generate_sweep_configs.py 和 run-sweep.yml:596-630 这些所有配置共用的 sweep 基础设施，而受益方只是一个目前连起都起不来的配置 —— 顺序不对，应当先让配置转绿再谈 schema 改动。Option 1 的代价是每个 arm 的旋钮位于 recipe 而非 YAML search-space；recipe 侧默认值在 c32 实测代价为 −4.9%，这是已知、有界、且已记录的取舍。
• sweep 网格在本 PR 中未做改动，网格调整留待本配置转绿后的后续 PR。
• 标签选择：使用 full-sweep-enabled 而非 AGENTS.md:95 推荐的默认 full-sweep-fail-fast。理由是这个配置是从「完全无法启动」恢复过来的，第一次全量 sweep 与其说是测量不如说是 bring-up 探测；在 fail-fast 下，第一个不稳定的点会取消该 matrix 其余作业并丢弃那些本已跑通的并发点的在途结果 —— 这正是 AGENTS.md:95 中所述应当改用 full-sweep-enabled 的情形。一旦该配置有了一次绿色 sweep，应改回 full-sweep-fail-fast。

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[jiacao-amd]

Superseded: reopening without any container patching, per the repo owner's rule that recipes must run the pinned image as published. The replacement PR keeps the 08-09 nightly pin and carries only recipe- and sweep-config-level tuning.

已关闭并由新 PR 取代：按仓库所有者的要求，配方不得对镜像打任何补丁，必须直接运行已发布的镜像。替代 PR 保留 08-09 nightly 镜像钉定，仅包含配方层与 sweep 配置层的调优。