[doc] router replay

[hjh0119]

No description provided.

[gemini-code-assist]

Code Review

This pull request introduces documentation for Router Replay (R2/R3) strategies in both Chinese and English to address training-inference mismatch in MoE reinforcement learning. The review feedback correctly identifies several incorrect arXiv IDs in the text and references of both language versions, where the publication year was mistakenly written as 2025 instead of 2024.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The arXiv ID for the Group Sequence Policy Optimization (GSPO) paper is incorrect. It should be 2407.18071 instead of 2507.18071 (since the paper was published in July 2024).

Suggested change

	根据 [论文](https://arxiv.org/abs/2507.18071) 的分析，token 级重要性采样比可以分解为两个因子：
	根据 [论文](https://arxiv.org/abs/2407.18071) 的分析，token 级重要性采样比可以分解为两个因子：

[gemini-code-assist]

The arXiv IDs for the referenced papers are incorrect. They all use 25 instead of 24 for the year of publication (2024). Please update them to the correct IDs:\n- Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers: 2410.11370\n- Group Sequence Policy Optimization: 2407.18071\n- Stabilizing Reinforcement Learning with LLMs: Formulation and Practices: 2412.01374

Suggested change

	1. [Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers](https://arxiv.org/abs/2510.11370)
	2. [Group Sequence Policy Optimization](https://arxiv.org/abs/2507.18071)
	3. [Stabilizing Reinforcement Learning with LLMs: Formulation and Practices](https://arxiv.org/abs/2512.01374)
	1. [Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers](https://arxiv.org/abs/2410.11370)\n2. [Group Sequence Policy Optimization](https://arxiv.org/abs/2407.18071)\n3. [Stabilizing Reinforcement Learning with LLMs: Formulation and Practices](https://arxiv.org/abs/2412.01374)

[gemini-code-assist]

The arXiv ID for the Group Sequence Policy Optimization (GSPO) paper is incorrect. It should be 2407.18071 instead of 2507.18071 (since the paper was published in July 2024).

Suggested change

	According to the [paper](https://arxiv.org/abs/2507.18071), the token-level importance sampling ratio can be decomposed into two factors:
	According to the [paper](https://arxiv.org/abs/2407.18071), the token-level importance sampling ratio can be decomposed into two factors:

[gemini-code-assist]

The arXiv IDs for the referenced papers are incorrect. They all use 25 instead of 24 for the year of publication (2024). Please update them to the correct IDs:\n- Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers: 2410.11370\n- Group Sequence Policy Optimization: 2407.18071\n- Stabilizing Reinforcement Learning with LLMs: Formulation and Practices: 2412.01374

Suggested change

	1. [Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers](https://arxiv.org/abs/2510.11370)
	2. [Group Sequence Policy Optimization](https://arxiv.org/abs/2507.18071)
	3. [Stabilizing Reinforcement Learning with LLMs: Formulation and Practices](https://arxiv.org/abs/2512.01374)
	1. [Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers](https://arxiv.org/abs/2410.11370)\n2. [Group Sequence Policy Optimization](https://arxiv.org/abs/2407.18071)\n3. [Stabilizing Reinforcement Learning with LLMs: Formulation and Practices](https://arxiv.org/abs/2412.01374)