[jinyouzhi]

Stacked on #9009. Until #9009 merges this branch also contains its commits; only the last commit (Reuse the static rollout decode graph across calls) is new here.

Summary

The default graph path (use_graph_capture=True without continuous_batch_size) currently rebuilds its DeepSpeedStaticCache, runs three warmup forwards and captures a new graph on every generate() call. In an RL loop that is a fixed setup cost per rollout round, paid even though the decode graph depends only on the decode shape, not on the prompt contents or the weight values.

This PR keeps the static path's graph and cache across calls, the same way #8732 / #9009 already do for the continuous path.

	Before	After
Per generate() call	Allocate DeepSpeedStaticCache, 3 warmups, capture	Eager prefill only; replay the retained graph
Recapture trigger	Every call	Graph signature changes: batch size, prompt width + max_new_tokens, device, dtype, or a parameter's address / shape / strides / dtype / device
Prompt width	n/a (always recaptured)	One graph serves every width with the same total length, since prefill runs eagerly and the decode mask is rebuilt per call
In-place weight updates	n/a	Replayed without recapture (same reasoning as #9009)
shutdown()	Releases the continuous state	Releases both the continuous and static states

Implementation

• _continuous_graph_signature → _graph_signature, shared by both paths (no behaviour change).
• New _StaticGraphState / _get_static_graph_state / _capture_static_graph. The warmup + capture code is moved unchanged out of _generate_graph. Capture now runs on a zeroed cache before the prefill KV is loaded; the replayed values never depend on capture-time contents.
• Each call resets the retained cache, zeros the 2-D mask, copies the prefill KV and rebuilds the 4-D mask before the first replay. The decode loop is unchanged.
• Memory: the static cache and graph pool stay allocated between rollouts (as on the continuous path) until the shape changes or shutdown() is called. Documented in inference-engine.rst.

Tests

New accelerator test test_static_graph_is_reused_across_rollouts_with_the_same_decode_shape (Llama, sdpa). Every call is compared against model.generate():

1. width 8 + 16 tokens → capture
2. in-place parameter update, same shape → reuse; tokens must change
3. width 12 + 12 tokens → reuse (same total length; catches a stale mask or KV from the previous call)
4. width 8 + 8 tokens → recapture

Then it asserts exactly 2 captures, and that shutdown() frees at least the cache bytes.

Mutation check (each must fail the new test): not zeroing the mask on reuse → fails; recapturing every call → fails (capture count); skipping the prefill KV copy → fails (all three static tests).

Validation

1× NVIDIA H20 (driver 580.178.04), torch 2.13.0+cu130, Transformers 5.15.1:

CUDA_VISIBLE_DEVICES=7 DS_ACCELERATOR=cuda OMP_NUM_THREADS=1 HF_HUB_OFFLINE=1 \
  python -m pytest -q -p no:cacheprovider -o addopts= \
  tests/unit/v1/test_hybrid_engine_rollout_v1.py \
  tests/unit/runtime/rollout/test_hybrid_engine_rollout.py tests/unit/utils/test_static_cache.py
# 105 passed, 2 skipped (optional pretrained model, native inference extension)

CPU (DS_ACCELERATOR=cpu, same files): 78 passed, 29 skipped (accelerator-only cases).

Pre-commit passes on the three changed files.

Not covered

• Pinning a cache capacity so different total lengths share one graph (the continuous path's continuous_cache_capacity); can follow if wanted.
• Linear-attention / hybrid caches (feat(rollout): segment-based kernel injection for hybrid-engine greedy decode #8602) — the static cache here is KV-only, as on master.
• ZeRO-3, offload, HybridEngine kernel injection.