Fused BBQ dequantize-distance kernels with tiered SIMD dispatch

[EnRaiha]

Problem

Per-candidate rerank distance for 1-bit BBQ codecs allocates a Vec<f32> reconstruction per candidate (encode → dequantize → subtract → L2). For a 768-dim collection at oversample × ef candidates this is the dominant rerank cost and defeats SIMD.

Proposed solution

A fused kernel that computes dist² = Σq² − 2·scale·Σ(sign·q) + dim·scale² directly over packed sign bits, dispatched through a small function-pointer table chosen at runtime:

• AVX-512 (512-bit server: Ice Lake+, Zen 4/5) — 32 lanes, _mm512_mask_blend_ps / masked ops
• AVX10/AVX-512VL (256-bit hybrid: Sierra/Clearwater Forest, Core Ultra) — _mm256_mask_blend_ps, __mmask8
• AVX2+FMA (Haswell..Alder Lake, Zen 1-3) — _mm256_blendv_ps
• NEON (aarch64) — vbslq_f32, with a vtbl1_u8 LUT for bit expansion
• WASM SIMD128 — v128_bitselect
• Portable scalar fallback (zero-allocation)
  Bit order is MSB-first; byte.reverse_bits() maps dim k → lane k. Correctness gate: golden tests vs scalar across dims 8..512, tolerance 1e-4; criterion benches vs the current alloc path.

Engine / area

Vector (rerank codecs).

Alternatives & workarounds

Current code path is correct, just slower; scale by reducing oversample.

Before submitting

• I searched existing issues and discussions for this proposal.

[farhan-syah]

Verdict: valid feature request.

• Finding: BbqRerank::distance_prepared allocates a Vec<f32> reconstruction per candidate.
• Implementation: Compute dist² = Σq² − 2·scale·Σ(sign·q) + dim·scale² directly from packed bits. Store the precomputed query sum in the prepared state. Extend the existing runtime dispatch table with scalar, AVX2+FMA, AVX-512, NEON, and WASM SIMD128 kernels. Handle partial bytes and preserve the scalar fallback.
• Tests: Compare every kernel with the scalar result across dimensions 8..512. Include dimensions that are not divisible by eight.
• Correction: AVX-512 provides 16 FP32 lanes per 512-bit vector, not 32.
• Benefit: Remove per-candidate reconstruction allocations and reduce BBQ rerank cost across oversample × ef_search candidates.
• Triage: Keep type:feature. Add engine:vector and priority:P2. No severity label applies.