Skip to content

WIP: align Qwen3.5 local Megatron modules - #9

Draft
zrr1999 wants to merge 1 commit into
PFCCLab:mainfrom
zrr1999:qwen35-bit-exact-alignment
Draft

WIP: align Qwen3.5 local Megatron modules#9
zrr1999 wants to merge 1 commit into
PFCCLab:mainfrom
zrr1999:qwen35-bit-exact-alignment

Conversation

@zrr1999

@zrr1999 zrr1999 commented Aug 5, 2026

Copy link
Copy Markdown

Status

Work-in-progress local Megatron module support used by the Qwen3.5 bit-exact reference path.

This is a partial implementation PR, not a complete cross-framework alignment claim.

Scope

  • 1-centered local RMSNorm
  • local mixed-block module spec
  • GDN cleanup and focused tests

Validation

  • formal Torch world4 EP4 CUDA/BF16 10-step reference is same-side deterministic x3
  • diff check passed

Not ready for merge; candidate retention and cross-framework forward/backward/optimizer/multimodal gates remain open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant