Support SVDQuant

[changwangss]

Description

Add SVDQuant as a pre-quantization transform that works with the existing AutoRound pipeline.

Keeps the residual branch as a normal Linear, so existing RTN / SignRound quantizers can process it with scheme="MXFP4".

API usage:

from auto_round import AutoRound

ar = AutoRound(
    "black-forest-labs/FLUX.1-dev",
    model_type="diffusion",
    scheme="MXFP4",
    algorithm="rtn",
    enable_svdquant=True,
    svdquant_rank=32,
    svdquant_smooth_alpha=0.5,
    disable_opt_rtn=True
)

ar.quantize_and_save("./flux-svdquant-mxfp4-rtn", format="fake")

CLI usage:

auto-round \
  --model black-forest-labs/FLUX.1-dev \
  --model_type diffusion \
  --scheme MXFP4 \
  --algorithm rtn \
  --disable_opt_rtn \
  --enable_svdquant \
  --svdquant-rank 32 \
  --svdquant-smooth-alpha 0.5 \
  --format fake \
  --output_dir ./flux-svdquant-mxfp4-rtn

Type of Change

feature.

Related Issues

Fixes or relates to #

Checklist Before Submitting

• My code has been tested locally.
• Documentation has been updated as needed.
• New or updated tests are included where applicable.
• The CUDA CI has passed. You can trigger it by commenting /azp run Unit-Test-CUDA-AutoRound.

[wenhuach21]

if it's linear, it will be quantized by AutoRound, I guess. Two solutions
1 add "lora" to predefined ignore layers for all models
or 2 change to F.linear

[wenhuach21]

I'd recommend using the awq+lora to generate svdquant, and share the common args with awq

[wenhuach21]

It might be better to implement this as a LoRABranch and decouple it from AWQ. That would make it easier to combine with other quantization algorithms in the future.

[wenhuach21]

you can refer gptqmodel about how to implement it