Skip to content

feat: client-side embedding training & multi-turn rollout support #247

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Yunnglin merged 11 commits into from
Jul 21, 2026
Merged

feat: client-side embedding training & multi-turn rollout support #247

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
d100f5d
feat: client-side embedding training & multi-turn rollout support
Yunnglin Jul 14, 2026
8060b2b
Merge remote-tracking branch 'origin/main' into feat/client-embedding…
Yunnglin Jul 15, 2026
47fda64
fix(megatron): PP-aware DP-group in set_loss + dedup _to_plain + clie…
Yunnglin Jul 16, 2026
50af737
refactor(mock_sampler): remove unused _MULTI_TURN_KNOBS
Yunnglin Jul 17, 2026
c716074
fix
Yunnglin Jul 20, 2026
2d8e473
docs: 补充 ClientMultiTurnRollout 与 client 侧 embedding 指南，并精简测试
Yunnglin Jul 20, 2026
3848c6d
Consolidate client examples and use OpenEnv rollout
Yunnglin Jul 21, 2026
70b2f19
Configure Megatron entrypoint runtime defaults
Yunnglin Jul 21, 2026
d7d6175
fix(processor): tolerate non-model keys in collate padding
Yunnglin Jul 21, 2026
76eb55e
style(processor): wrap long _pad_sequence call to satisfy flake8 (E501)
Yunnglin Jul 21, 2026
85b2b1b
fix lint
Yunnglin Jul 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion INSTALL_CLIENT.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -311,5 +311,5 @@ Write-Host " conda activate $EnvName" -ForegroundColor Yellow
Write-Host ""
Write-Host "Example usage (see cookbook/client/tinker/):"
Write-Host ' $env:MODELSCOPE_TOKEN = "your-token"' -ForegroundColor Yellow
Write-Host " python cookbook/client/tinker/modelscope_service/self_cognition.py" -ForegroundColor Yellow
Write-Host " python cookbook/client/tinker/self_cognition.py" -ForegroundColor Yellow
Write-Host ""
2 changes: 1 addition & 1 deletion INSTALL_CLIENT.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -233,5 +233,5 @@ echo " conda activate $ENV_NAME"
echo ""
echo "Example usage (see cookbook/client/tinker/):"
echo " export MODELSCOPE_TOKEN='your-token'"
echo " python cookbook/client/tinker/modelscope_service/self_cognition.py"
echo " python cookbook/client/tinker/self_cognition.py"
echo ""
6 changes: 2 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,10 +100,8 @@ sh INSTALL_MEGATRON.sh
| DPO multi-LoRA training | transformers | [Script](cookbook/rl/dpo/dpo_multi_lora.py) |
| GKD on-policy distillation | megatron | [Script](cookbook/rl/gkd/gkd_on_policy.py) |
| GKD off-policy distillation | megatron | [Script](cookbook/rl/gkd/gkd_off_policy.py) |
| Tinker client finetuning (self-host) | transformers | [Script](cookbook/client/tinker/self_host) |
| Tinker client finetuning (ModelScope) | transformers | [Script](cookbook/client/tinker/modelscope) |
| Twinkle client finetuning (self-host) | transformers | [Script](cookbook/client/twinkle/self_host) |
| Twinkle client finetuning (ModelScope) | transformers | [Script](cookbook/client/twinkle/modelscope) |
| Tinker client finetuning | transformers | [Script](cookbook/client/tinker) |
| Twinkle client finetuning | transformers | [Script](cookbook/client/twinkle) |
| Server startup scripts | transformers/megatron | [Script](cookbook/client/server) |

## Changelog
Expand Down
6 changes: 2 additions & 4 deletions README_ZH.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,10 +94,8 @@ sh INSTALL_MEGATRON.sh
| DPO 多 LoRA 训练 | transformers | [脚本](cookbook/rl/dpo/dpo_multi_lora.py) |
| GKD 在线蒸馏 | megatron | [脚本](cookbook/rl/gkd/gkd_on_policy.py) |
| GKD 离线蒸馏 | megatron | [脚本](cookbook/rl/gkd/gkd_off_policy.py) |
| Tinker 客户端微调（自部署） | transformers | [脚本](cookbook/client/tinker/self_host) |
| Tinker 客户端微调（ModelScope） | transformers | [脚本](cookbook/client/tinker/modelscope) |
| Twinkle 客户端微调（自部署） | transformers | [脚本](cookbook/client/twinkle/self_host) |
| Twinkle 客户端微调（ModelScope） | transformers | [脚本](cookbook/client/twinkle/modelscope) |
| Tinker 客户端微调 | transformers | [脚本](cookbook/client/tinker) |
| Twinkle 客户端微调 | transformers | [脚本](cookbook/client/twinkle) |
| 服务端启动脚本 | transformers/megatron | [脚本](cookbook/client/server) |

Twinkle✨支持相同的算法接口运行在单GPU、torchrun多机、Ray、Client等各场景下。其算法过程是外露的，非常便于修改和调试。完整的框架介绍请查看[快速开始](https://modelscope.github.io/twinkle-web/zh/docs/usage-guide/quick-start/)
Expand Down
Loading
Loading