Skip to content

[lr_scheduler] Forward OneCycle scheduling options - #6547

Open
Afloat16 wants to merge 1 commit into
hpcaitech:mainfrom
Afloat16:fix/onecycle-scheduler-options-20261112
Open

Afloat16 wants to merge 1 commit into
hpcaitech:mainfrom
Afloat16:fix/onecycle-scheduler-options-20261112

Conversation

@Afloat16

Copy link
Copy Markdown

What does this PR do?

Fixes #6544.

OneCycleLR captures optional arguments in kwargs but did not forward them to PyTorch. The documented three_phase=True option therefore silently used the two-phase curve, changing both learning rates and cycled momentum. total_steps=None with epochs and steps_per_epoch also failed to construct the documented inferred-length schedule.

Forward the optional arguments to the native scheduler. Explicit wrapper arguments and the maximum learning rates taken from optimizer groups remain unchanged. Unknown optional arguments now receive the native constructor's validation.

Validation

  • 15 native regression cases: upstream base 11 failed / 4 passed; patch 15 passed.
  • Two real SGD/Adam parameter groups are compared step by step against torch.optim.lr_scheduler.OneCycleLR for learning rate, momentum/betas and parameter updates, using cosine/linear annealing and two/three phases.
  • Epoch-derived total steps, real optimizer/scheduler checkpoint continuation and an unknown-argument control are covered.
  • python -m pytest tests/test_optimizer/test_onecycle_options.py tests/test_optimizer/test_lr_scheduler.py -q: 16 passed, including the existing scheduler save/load regression.
  • Applicable configured pre-commit hooks and git diff --check passed.
  • Python 3.12 and PyTorch 2.9.1+cpu. GPU/distributed training and the full repository suite were not run.

@Afloat16
Afloat16 requested a review from a team as a code owner November 12, 2026 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: OneCycleLR silently ignores three_phase and epoch-based schedule options

1 participant