Perf/pytorch block fp8 moe decode

[qescccczmr]

Motivation

The existing PyTorch Block-FP8 MoE path may launch GEMM CTAs for every local
expert during small-batch decode, even though only a small subset of experts
receives tokens. For example, with M=1, top-k=8, and E=256, at most eight
experts are active while the original dense expert grid still covers all 256
experts.

The activation path also materializes the BF16 SwiGLU result before launching
a separate FP8 quantization kernel:

gate/up GEMM
  -> silu_and_mul
  -> BF16 intermediate
  -> quant_fp8
  -> FP8 down-projection input

This PR reduces both the inactive-expert launch overhead and the intermediate
activation traffic in Block-FP8 MoE decode.

Modification

Active-expert block scheduling

Add an opt-in active-block dispatch:

LMDEPLOY_MOE_ACTIVE_BLOCK_DECODE=1

The path reuses the existing compact MoE kernel and routed-block metadata, but
launches GEMM programs only for expert blocks that receive tokens.

Dispatch is selected using backend properties only:

• GPU compute capability
• input and output dtype
• FP8 group size
• token count
• expert count and top-k
• hidden and local projection dimensions

There are no model-name checks. Unknown or unsupported shapes retain the
existing schedule. The optimization also falls back for unsupported expert
layouts, dtypes, GPUs, and tensor shapes.

Fused SwiGLU and FP8 quantization

Fuse silu_and_mul with group-wise FP8 quantization for the default Block-FP8
MoE activation:

gate/up GEMM
  -> silu_and_mul_post_quant
  -> FP8 activation + scale
  -> down-projection GEMM

This removes one kernel launch and avoids materializing and rereading the BF16
activation. Custom activation functions continue using the existing fallback.

Performance

End-to-end active-block scheduling

Environment:

• 8 x NVIDIA H200
• PyTorch engine
• TP=8, EP=1
• CUDA Graph enabled
• Block-FP8 MoE model
• input length 128, output length 256
• symmetric-memory AllReduce baseline
• one warm-up and five measured repeats; median reported

Concurrency	Baseline TPOT	Active TPOT	TPOT reduction	Baseline tok/s	Active tok/s	Throughput gain
1	20.023 ms	17.527 ms	12.47%	48.371	55.053	13.82%
4	21.147 ms	18.769 ms	11.24%	183.429	205.707	12.14%

The baseline and candidate both used the fused SwiGLU/quant path, so this table
isolates the contribution of active-block scheduling.

SwiGLU and FP8 quantization microbenchmark

The benchmark used the TP8 local routed-MoE shape with top-k=8 and local
intermediate size 256.

Decode M	Separate kernels	Fused kernel	Latency reduction
1	7.403 us	5.466 us	26.17%
4	7.565 us	5.424 us	28.30%
16	7.690 us	5.777 us	24.87%
32	7.892 us	5.861 us	25.74%

This is an operator-level result and is not added directly to the end-to-end
active-block gain.

BC-breaking

No backward-incompatible API changes are introduced.

The active-block schedule is opt-in, and unsupported shapes fall back to the
existing implementation. The fused activation preserves the established
custom-activation fallback.

Correctness and validation

• The fused FP8 activation is bitwise equal to
  quant_fp8(silu_and_mul(x)) for the measured shapes.
• The generated FP8 scales are bitwise equal.
• Greedy end-to-end output digests matched across all measured repeats.
• Existing kernel tests pass: 97 passed.
• py_compile and git diff --check pass.

Commands:

python -m pytest -q \
  tests/pytorch/kernel/test_activation.py \
  tests/pytorch/kernel/test_fused_moe.py

python -m py_compile \
  lmdeploy/pytorch/kernels/cuda/activation.py \
  lmdeploy/pytorch/kernels/cuda/moe/blocked_fp8.py

git diff --check

Checklist

• Existing lint/static checks pass.
• Existing kernel tests pass.
• Unsupported configurations retain the established fallback.
• No new dependency or backward-incompatible API change is introduced.