Perf/flux nvfp4 fusions

[jhaotingc]

Description

This PR improves FLUX.1-dev execution in TensorRT-LLM VisualGen on Blackwell:

• Adds configurable NVFP4 GEMM backend selection and removes redundant projection conversion work.
• Shares compatible NVFP4 activation quantization between the single-stream QKV and MLP projections. Sharing is enabled automatically for eligible static-NVFP4 projections and retains post-load compatibility checks.
• Fuses NVFP4 GEMM+GELU in the dual- and single-stream MLP paths. TRTLLM_FLUX_FUSED_MLP_GELU accepts case-insensitive single, dual, or full; 1 enables both paths and 0 disables both.
• Adds static per-tensor E4M3 Q/K/V scale plumbing for CUTEDSL attention (qk_dtype: fp8, v_dtype: fp8, block sizes 0), using checkpoint quantizer metadata instead of dynamic attention quantization.

The changes are API-compatible.

Performance

FLUX.1-dev, NVIDIA B300 SXM6 AC, TP1/Ulysses1/CFG1, CUDA Graph, 2048×2048 output:

Configuration	Summed GPU kernel time / transformer evaluation	Improvement vs. BF16
BF16 GEMMs + FA4 BF16 attention	324.7 ms	baseline
Static NVFP4 GEMMs + FA4 BF16 attention	217.6 ms	1.49× / 33.0% lower
Static NVFP4 GEMMs + CUTEDSL E4M3 attention	192.0 ms	1.69× / 40.9% lower

The CUTEDSL attention path reduces attention from 122.2 ms to 93.9 ms, including 14.8 ms of current scale/conversion overhead. Separately, sharing activation quantization and enabling both GEMM+GELU fusions reduces 512×512 NVFP4 step latency from 35 ms to 32 ms (8.6%).

The 2048×2048 values sum GPU kernel durations and exclude CPU gaps. The E4M3 attention capture is shape/model matched but used a different physical B300 and NVFP4 MLP tactic, so it is not a strict single-variable A/B.

Test Coverage

• Added focused unit coverage for VisualGen NVFP4 configuration propagation, CUTEDSL E4M3 dispatch and scale plumbing, fusion selector parsing, and automatic shared-quantization eligibility.
• pytest -q tests/unittest/_torch/visual_gen/test_flux_transformer.py tests/unittest/_torch/visual_gen/test_model_loader.py — 39 passed.
• pytest -q tests/unittest/_torch/visual_gen/test_attention_cute_dsl.py -k static_fp8_qkv_forward — 1 passed.
• pre-commit run -a — passed, including the full mypy check.
• End-to-end FLUX.1-dev static-NVFP4 VisualGen smoke test with FA4 and CUDA Graph at 1024×1024; confirmed 38 fused single-stream MLP blocks and 38 shared-quantization single-stream blocks.

PR Checklist

• The description explains the change and motivation.
• New code paths have unit coverage.
• All commits include DCO signoff.
• No new dependency is introduced.
• The change is API-compatible.