Add MI300X GLM-5.2 FP8 AgentX MTP - #2581
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
cquil11
added
AMD
agentx
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM — new AgentX discovery recipe for GLM-5.2 FP8 on MI300X, following the established recipe pattern.
Extended reasoning...
Overview
This PR adds a new single-node AgentX benchmark recipe: a launch script (glm5.2_fp8_mi300x_mtp.sh), a config entry in configs/amd-master.yaml, and a perf-changelog.yaml entry. It enables native SGLang EAGLE MTP for GLM-5.2 FP8 on 8xMI300X, sweeping TP8/EP1 and TP8/EP8 across concurrency 1-8 to find the HBM/Pareto knee, mirroring the already-merged MI325X sibling recipe (glm5.2_fp8_mi325x_mtp.sh) and the same-SKU qwen3.5_fp8_mi300x_mtp.sh recipe.
Security risks
None. This is a benchmark harness invoking a pinned SGLang server image with fixed CLI flags derived from env vars validated via check_env_vars/require_agentic_kv_offload_none; there is no user-controlled input, credential handling, or network-facing service beyond the existing benchmark infrastructure.
Level of scrutiny
Low-to-moderate. This is config/recipe code (new model+hardware benchmark entry), not a change to core inference, scheduling, or shared library logic. It follows an established, widely-replicated template almost line-for-line.
Other factors
I diffed the new script against its two closest siblings (glm5.2_fp8_mi325x_mtp.sh and qwen3.5_fp8_mi300x_mtp.sh) and confirmed all helper calls (write_command, stop_background_process_tree, require_agentic_kv_offload_none, AIPERF_SERVER_METRICS_URLS/AIPERF_REQUIRED_SERVER_METRIC_PREFIX) exist in benchmark_lib.sh and are used with the current, more recently adopted calling convention (used by several other recent recipes, e.g. Kimi-K3 MI355X). The configs/amd-master.yaml entry (16 = 2 x 8 conc-list entries) and perf-changelog.yaml description are consistent with the PR's stated matrix. The bug hunting system found no issues, and the sweep is gated by CI labels per the standard recipe-reminder bot comment.
34ad8ba to
1e877d6
Compare
cquil11
changed the title
cquil11
added
full-sweep-enabled
and removed
agentx-fast
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31658080369 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31658080369 |
f396e86 to
6e26538
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31674031266 |
1 participant
Summary
Search-space rationale
Complete c1-c8 TP8/TEP8 fast discovery found a sharp content-quality cliff after c3. TP c1 is the low-latency point; at c3, TP has slightly higher input throughput while TEP materially lowers TTFT, so both remain. TP c2 exhausted HBM in two identical official long-context warmups and is excluded. TEP c1-c2 are dominated, and c4+ are unstable or off-frontier. An isolated TEP c3 HiCache test also exhausted device resources, so the official matrix remains GPU-resident.
Validation
bash -n benchmarks/single_node/agentic/glm5.2_fp8_mi300x_mtp.shuv run pytest utils/matrix_logic -q(231 passed)python3 utils/validate_perf_changelog.py --base-ref origin/main --head-ref HEAD