[dependabot]

Bumps trl from 1.14.0 to 1.17.1.

Release notes

Sourced from trl's releases.

v1.17.1

What's Changed

• Keep a parenthesis inside an LFM2.5 tool-call argument from closing the call by @​albertvillanova in huggingface/trl#7942

Full Changelog: huggingface/trl@v1.17.0...v1.17.1

v1.17.0

Features

🔥 Mistral Large 4: fine-tune a 1T-parameter model on one node

TRL now supports mistralai/Mistral-Large-4-1T-A52B: 1T parameters, 52B active. The new LoRA example fine-tunes it with SFTTrainer on a single 8×B300 node, on top of its FP8 weights: the experts are split 8 ways with token dispatch, everything else is sharded across the 8 GPUs at load time by transformers' DistributedConfig, and LoRA trains attention and the dense and shared-expert MLPs.

Measured: 32.7k tokens per step in ~8 s, 142 GiB per GPU, with packing and assistant-only loss on reasoning conversations.

mistral-large4-trl_v7_hq.mp4

training_args = SFTConfig(
    model_init_kwargs={
        "dtype": "auto",  # keep the FP8 weights
        "distributed_config": DistributedConfig(fsdp_size=8, ep_size=8),
    },
    packing=True,
    assistant_only_loss=True,
    ...
)
trainer = SFTTrainer(model="mistralai/Mistral-Large-4-1T-A52B", args=training_args, peft_config=LoraConfig(...), ...)

assistant_only_loss=True works out of the box: TRL ships a training chat template that matches the released checkpoint's, including its reasoning_effort levels (low, medium, high). Needs transformers>=5.22.0, peft>=0.21.0 and accelerate>=1.15.0.

• Add Mistral Large 4 by @​qgallouedec in huggingface/trl#8017
• Sync the Mistral Large 4 chat template with the released checkpoint by @​albertvillanova in huggingface/trl#8018

🎯 Unbiased importance sampling under top-k, top-p and min-p

vLLM normalizes its log-probabilities over the tokens left after top_k, top_p and min_p filtering, while the trainer normalized over the whole vocabulary. Every sampled token's log-ratio was off by the log of the probability mass the filter kept, so the vLLM importance sampling ratio was biased towards 0 (around 1e-4 instead of 1 with top_p=0.9). The trainers now normalize over the candidate set each token was sampled from, following Keep Sampling Mask from DeepSeek-V3.2.

• AsyncGRPO: renormalize log-probs over the top-k/top-p candidate set by @​AmineDiro in huggingface/trl#7627
• GRPO: renormalize log-probs over the top-k/top-p candidate set (vLLM colocate mode, vLLM >= 0.28; server mode will follow) by @​albertvillanova in huggingface/trl#7866
• Require top_k with min_p in AsyncGRPO so the importance sampling ratio stays unbiased by @​albertvillanova in huggingface/trl#7937
• Track the support maximum in the chunked log-prob kernel, so a far-below-max candidate set no longer gives a log-probability of 0 and non-finite gradients, by @​kashif in huggingface/trl#7983

... (truncated)

Commits

• 44809ba Release: 1.17.1
• bf5c2c4 Keep a parenthesis inside an LFM2.5 tool-call argument from closing the call ...
• e257382 Release: v1.17 (#8021)
• 4a60b81 Sync the Mistral Large 4 chat template with the released checkpoint (#8018)
• a9131f2 Add Mistral Large 4 (#8017)
• 3c3c2af Return real VerifyResult objects from the OpenEnv test sessions (#7990)
• 551dfca Exit the broken session cleanly in the OpenEnv capture error test (#7986)
• 16e0599 Compare rollout sample arrays as lists in the OpenEnv harness worker test (#7...
• ea64740 Save AsyncGRPO and AsyncDistillation checkpoints in the mixed-precision dtype...
• 248210f Use a real tokenizer and shared values in the OpenEnv rollout loop tests (#7989)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[greptile-apps]

No reviewable files after applying ignore patterns.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	trl@​1.14.0 ⏵ 1.17.1	+1

View full report