[None][feat] GVR V2 hint-free mode: pre_idx=None derives the top-k bracket from the current row

[longcheng-nv]

@coderabbitai summary

Description

run_varlen(pre_idx=None) runs the self-sampling GVR decode top-k kernel without the prev-step top-K hint tensor. The hinted path is unchanged; pre_idx stays in the API as the eligibility gate. Hint-free is auto-engine only, and k comes from indices.shape[1].

Callers that have no usable hint (first decode step, disagg generation side, deployments that drop the per-layer prior arena, ~256 MB at 61 layers / BS 1024) today pay a 3.6-6x register-family penalty (worst case 14.7x) if they pass degenerate hints. With pre_idx=None the same shapes run at hinted-path speed.

Implementation: one hf constexpr per kernel family, threaded through the compile caches and the varlen launcher key.

• reg / regimg: bracket = min/max fold of the first k row values, which already sit in the row-load register fragments (no extra loads); the hint prefetch and the hint-gather bracket arms (use_bm/use_img) are compiled out
• reg_clus: P0 samples the first k row elements instead of the hint (coalesced, cluster-uniform by construction, no barrier changes)
• main / clus: gather_hint rescue sites compiled out; a device-truth census on 886 real decode captures x BS {1,8,64,512} shows these sites never fire
• DEG short rows: unchanged (n <= 3k, the whole-row fold already covers the row)

Measurements

886 real decode captures (DSv3.2, V4 Flash, V4 Pro) x BS {1,8,64,512}, three arms in one process, cold-L2 nsys kernel-only, B200. own = hinted (production today), invalid = all -1 hints (dropping the hint with no kernel change), hf = this PR.

family (non-DEG)	invalid/own gm	hf/own gm	hf max
main (streaming)	1.00	1.01	1.08
clus	1.00	1.00	1.02
reg	4.44	1.01	1.15
regimg	3.59	0.98	1.08
reg_clus	6.04 (worst 14.7)	0.93	1.06

• Exactness: 10,632/10,632 tie-aware value-multiset checks pass, hf arm included.
• Overall hf/own gm 0.998 (BS1 0.994, BS8 0.993, BS64 1.007, BS512 0.997).
• Worst hf cases: ~9 reg cells at N = 2^k + 3, up to 1.15x. This is the schedule-quantization boundary the hinted path already has; invalid on the same cells is 5.3x.
• main at BS64 shows +3% from the extra compile arm shifting codegen; invalid on the same cells is at 1.007, so it is not an execution-path cost.

Full 886 x 11-BS grid on this head (9,746 cases, hinted vs pre_idx=None in one process, same protocol):

• exactness 19,492/19,492
• hint-free / hinted gm 0.992 (p90 1.025, max 1.15; worst cells are the same 2^k+3 set); N >= 4096 region gm 0.991; per-BS gm stays in 0.980-1.015 across BS 1..1024
• vs SGLang v2 plan+transform on 9,515 joinable pairs, measured in the same run: hint-free gm 1.656 (faster in 99.83%), hinted gm 1.643 (99.82%), so the mode keeps the kernel's external margin

Test Coverage

tests/unittest/_torch/thop/parallel/test_gvr_selfsampling_topk.py: a pre_idx=None arm per kernel family, exactness vs torch.topk, and None-vs-hinted output equivalence.

PR Checklist

• PR title and description convey what and why
• Test coverage for the change
• CI passing