[Performance] Serve process-slot inference passes from pinned staging batches

[vmoens]

Stack: #4305 → #4306 → #4307 → #4308; depends on #4305.

Batch process-slot inference through reusable pinned buffers, non-blocking transfers, and one CUDA event per pass. Gather/scatter replaces per-request copies; only response keys return to CPU.

server = InferenceServer(
    policy,
    transport=transport,  # ProcessSlotTransport; batched I/O is automatic
    policy_device="cuda",
)

Custom collate functions retain their existing path.

Validation: CPU inference suite and metadata regression tests pass. CUDA validation and throughput measurements pending.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/4306

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⏳ No Failures, 4 Pending

As of commit 9ee51a0 with merge base 5076ae1 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[vmoens]

Projected server pass cost: -40–50%
Projected server ceiling:   ~1,000 -> ~1,800–2,000 steps/s

These CUDA estimates remain unmeasured. No direct end-to-end gain is expected until #4305 removes the driver bottleneck.

[vmoens]

Measured on the target hardware (64-environment DreamerV3 run on a pixel task, one 4-GPU node, process-slot inference server with a static batch of 64, compiled and CUDA-graph learner step, bf16, train ratio 16), comparing the full stack #4305 + #4306 + #4307 + #4308 (with transition_chunk_size=64) against the same configuration without these four PRs, over the same elapsed window of the two runs (1,000 s to 4,428 s after training start):

	environment steps/s	learner updates/s (32x256)
without the stack	361	0.70
with the stack	1,249	2.44
speed-up	3.46x	3.46x

The unthrottled collection phase before training reached 1,254 steps/s (298 before), the learner GPU now shows sustained full-utilization bursts instead of about a quarter duty cycle, and the run had no errors over the measured 1.3 hours. The two rates move together because the train ratio couples them, so the stack lifted both the collection ceiling and the learner cadence past the previous limits. This measures the combined effect of the four PRs; per-PR attribution has not been measured.