[TRTLLM-14620][feat] Add confidence-guided dynamic verification for DeepSeek-V4 DSpark

[zhaoyuanh-nvidia]

@coderabbitai summary

Description

This draft adds opt-in confidence-guided dynamic verification for DeepSeek-V4 DSpark in the PyTorch runtime.

Static DSpark verifies the full K5 block for every request. The new path uses the checkpoint confidence head to predict prefix survival, selects a profitable measured (G,V) cell, and allocates the verifier-token budget raggedly across requests. G is the rank-local CUDA-graph row count and V is the rank-local verifier-token budget.

The implementation:

• preserves the native multi-G static K5 path as the fail-closed fallback;
• authenticates the measured SPS cost table against the active model, source, topology, GPU, and graph ladder;
• reuses the existing attention-DP shape-agreement collective to select one deterministic (G,V) across ranks, avoiding a second synchronization round;
• captures and reuses compact CUDA graphs for measured verifier-token buckets; and
• lets attention-DP ranks with no real requests join a selected compact cell through deterministic low/high padding dummies mapped to the DSpark scratch row; and
• preserves KV rewind and ownership, sampling layout, graph-key agreement, mandatory verification anchors, and full-K5 acceptance semantics.

This is intentionally a draft for architecture and correctness review. The matched static / feature-on forced-K5 / real-dynamic / trailing-static benchmark on the current PR head is still in progress. Therefore, this PR does not yet make a production performance claim. Matched stochastic-output, KV-state, graph-reuse, and rank-agreement validation remain release gates.

The new configuration fields are opt-in and should carry the api-compatible label.

Test Coverage

• 487 focused tests pass on 8xB300 at commit 247e2334 (487 passed, 9 warnings), covering planner and tier selection, ragged allocation, attention-DP agreement, CUDA-graph routing, sparse-row geometry, fused-Q handling, KV ownership and rewind, sampling, runtime controls, and static warmup without a speculative worker.
• The incremental idle-rank compact-execution path passes its native three-file suite on two independent B300 nodes with the exact public branch head 592a1ed6: AWS-PDX (171 passed, 10 warnings) and BIA (171 passed, 12 warnings).
• All 56 changed Python files compile.
• Eight-rank B300 collective and preflight validation pass.
• Before this PR leaves draft, the remaining validation is:
  • complete a matched current-main static / feature-on forced-K5 / real-dynamic / trailing-static bracket;
  • pass the strict output, KV, graph-reuse, and rank-agreement canary;
  • reproduce a dynamic gain whose lower confidence bound is positive; and
  • finish the architecture review and keep experimental-only tooling out of the production diff.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.