[multi vector] Add tiled MinMax MaxSim kernels

[juchen-ms (partychen)]

• Does this PR have a descriptive title that could go in our release notes?
• Does this PR add any new dependencies?
• Does this PR modify any existing APIs?
• Is the change to the API backwards compatible?
• Should this result in any changes to our documentation, either updating existing docs or adding new ones?

Reference Issues/PRs

Builds on the matrix-kernel abstractions introduced in #1368.

What does this implement/fix? Briefly explain your changes.

Summary

Adds prepared MaxSim kernels for MinMax8 queries against packed MinMax4 documents.

• Uses one generic Driver -> PanelKernel -> MicroKernel traversal for every architecture.
• Reuses BlockTransposed, packed/unpacked views, panel visitors, fixed-size panels, remainder handling, and Bound-tracked pointers from the existing matrix-kernel infrastructure.
• Keeps persistent query ownership in the factory's Prepared; the matrix driver and kernels borrow query values, compensation metadata, canonical document rows, and output scores.
• Keeps documents in canonical packed MinMax4 form. Each micro-kernel expands only its current contraction group, so there is no document scratch allocation or second traversal framework.
• Keeps architecture-specific register representations opaque behind ExtraWide.
• Preserves output reset, empty-input behavior, shape validation, panel tails, and NaN compensation handling.

Grouped layout

Grouped<const N: usize> owns the grouping rules through count, from_query, and from_packed:

• Grouped<4> is used by Scalar, x86-64 V3, and ARM64 Neon. Every eight dimensions become [0, 2, 4, 6] and [1, 3, 5, 7], matching low/high nibble order.
• Grouped<8> is used by x86-64 V4. Query dimensions remain contiguous; four packed MinMax4 bytes are expanded with BMI2 pdep_u64 for 512-bit VNNI.

The final group and incomplete query panels are zero padded. Compensation always uses the original, unpadded dimension.

Kernels

ISA	MR x NR	Grouping	Integer dot product
Scalar	8 x 6	Grouped<4>	Portable u32 accumulation
x86-64 V3	16 x 8	Grouped<4>	AVX2 maddubs + madd + accumulation
DiskANN x86-64 V4	16 x 8	Grouped<8>	BMI2 expansion + 512-bit VNNI dpbusd; adjacent accumulator lanes are folded before compensation
ARM64 Neon + dotprod	8 x 8	Grouped<4>	UDOT

The micro-kernel enters the selected target-feature context through run_inline, allowing LLVM to emit direct architecture instructions rather than intrinsic shim calls inside the contraction loop.

API and compatibility

let kernel = build_minmax_max_sim(MaxSimIsa::Auto, query_minmax8, BoxErase)?;
let mut scores = vec![0.0; kernel.nrows()];
kernel.compute_max_sim(doc_minmax4, &mut scores)?;

The API is additive. Existing MaxSim entry points are unchanged and no dependencies are added. Invalid output lengths or document dimensions return an error before modifying scores. Unavailable requested ISAs return NotSupported. MaxSimIsa::Reference remains unsupported by this factory.

Any other comments?

Performance

Workload: 1,000 documents, 16 query vectors x 16 document vectors, with query preparation excluded. Both 250 and 256 dimensions are measured. Each executable measures 30 rounds and is run three times.

x86-64

Intel Xeon Platinum 8370C, Windows x86-64, Rust 1.97.1, release build with -Ctarget-cpu=x86-64-v3. All compared outputs matched exactly.

Dimensions	Existing Auto	V3	V3 speedup	V4	V4 speedup
250	11.47 us/doc	1.65 us/doc	6.9x	0.52 us/doc	22.0x
256	4.09 us/doc	1.61 us/doc	2.5x	0.49 us/doc	8.3x

The existing x86-64 8-bit by 4-bit distance path has a scalar remainder at 250 dimensions but not at 256, so the 250-dimensional speedup should not be relabeled as a 256-dimensional result.

ARM64

Snapdragon X Elite X1E80100, 12 cores / 12 logical processors, Windows 11 Pro ARM64 (build 26200), Rust 1.97.1, native release build with -Ctarget-feature=+neon,+dotprod. Measured at revision ed75906eaedef6c41e5155532ceaebee6e819c77 with the Balanced power plan and no explicit CPU affinity.

Dimensions	Existing Auto	Scalar	Neon	Neon speedup	Auto
250	18.62 us/doc	6.79 us/doc	2.18 us/doc	8.53x	2.21 us/doc
256	19.43 us/doc	6.92 us/doc	2.30 us/doc	8.44x	2.28 us/doc

ARM64 methodology:

• One query is quantized to MinMax8 and 1,000 documents to MinMax4 before timing. Each prepared kernel is built once; quantization and query preparation are excluded.
• Inputs are generated uniformly in [-1, 1) using StdRng seeded with 0x1394_2026 + dimensions. Each implementation receives identical inputs.
• Each implementation performs three warm-up sweeps, followed by 30 measured sweeps. Implementation order rotates between rounds; each sweep processes all 1,000 documents. Inputs and outputs are passed through black_box.
• The executable is run three times. Reported timings are the median of the three per-run medians, in microseconds per document. Speedups use unrounded timings.
• All 16,000 output scores per dimension and per prepared implementation are compared against Existing Auto before timing. Scalar, Neon, and Auto produced zero numerical differences and zero bit differences on these benchmark inputs in all three runs.

Neon's per-run medians ranged from 2.17 to 2.28 us/doc at 250 dimensions and 2.26 to 2.52 us/doc at 256 dimensions. These ranges reflect observed run-to-run variation rather than confidence intervals.

The ARM64 measurements use the same workload dimensions and document count as the x86-64 measurements, but independently generated inputs. Speedups are relative to the existing implementation on each host, not a controlled comparison between processors.

Validation

cargo fmt --all --check
cargo test -p diskann-quantization --release minmax
cargo clippy -p diskann-quantization --all-targets -- -D warnings
cargo clippy -p diskann-quantization --all-targets --target aarch64-pc-windows-msvc -- -D warnings
MIRIFLAGS="-Zmiri-disable-isolation -Zmiri-strict-provenance" \
  cargo +nightly miri test -p diskann-quantization v4_driver_and_registers

The x86-64 validation reports 47 passing MinMax tests, with Scalar, V3, V4, and Auto executing natively. The V4 Miri test passes using diskann-wide emulation, while the native release benchmark exercises BMI2 and AVX-512 VNNI. ARM64 all-target Clippy passes.

Native ARM64 validation at ed75906 additionally ran cargo test -p diskann-quantization --release minmax: 46 tests passed, including Neon driver/register coverage, equivalence with the existing implementation, grouped query packing, panel tails, shape validation, empty inputs, prepared-query ownership, and NaN compensation. Scalar, Neon, and Auto execute natively; V3/V4 factory tests exercise unsupported-ISA handling on this host.

[Copilot]

🔵 Needs a closer look

The public factory has a Reference ISA compatibility issue, and V3/V4 runtime paths remain unvalidated on x86-64.

Pull request overview

Adds prepared, tiled MaxSim kernels for MinMax8 queries against MinMax4 documents with ISA-specific dispatch.

Changes:

• Adds public prepared-kernel and factory APIs.
• Implements Scalar, AVX2/V3, AVX-512/V4, and ARM64 Neon kernels.
• Adds packing, fused compensation/reduction, validation, and reuse tests.

File summaries

File	Summary	Final review note
diskann-quantization/src/multi_vector/mod.rs	Re-exports the factory.	—
diskann-quantization/src/minmax/multi/mod.rs	Registers and exports kernel modules.	—
diskann-quantization/src/minmax/multi/kernel.rs	Defines kernel interfaces.	—
diskann-quantization/src/minmax/multi/factory.rs	Provides ISA dispatch and prepared-kernel construction.	Moderate: MaxSimIsa::Reference is reported available but rejected by this factory.
diskann-quantization/src/minmax/mod.rs	Exposes MinMax APIs.	—
diskann-quantization/src/matrix_kernels/maxsim/mod.rs	Registers the MinMax matrix kernel.	—
diskann-quantization/src/matrix_kernels/maxsim/minmax8_x_minmax4.rs	Implements packing, tiling, compensation, SIMD kernels, and tests.	Nit: V3/V4 paths lack x86-64 runtime coverage.

Review details

Suppressed comments (2)

diskann-quantization/src/matrix_kernels/maxsim/minmax8_x_minmax4.rs:785

• The V3/V4 implementations here are only compile-checked in the stated ARM64 validation: the ISA tests return early when the requested x86 ISA is unavailable, so the pdep/VNNI path and its lane reduction are not exercised. Please add or run an x86-64 runtime test (or an equivalent testable emulation) covering these kernels before relying on this new code.

    micro_kernel!(V3, 16, micro_kernel, {8, 7, 6, 5, 4, 3, 2, 1});
    micro_kernel!(V4, 16, micro_kernel, {8, 7, 6, 5, 4, 3, 2, 1});

diskann-quantization/src/minmax/multi/factory.rs:171

• MaxSimIsa::Reference is documented as an always-available selector (is_available() returns true), and the existing MinMax implementation provides the reference MaxSim path, but this new public factory rejects it unconditionally. Callers that use is_available() to preflight an ISA therefore receive NotSupported for a value reported as buildable. Please add a reference adapter (or make support for this factory explicit in the ISA API/documentation).

        MaxSimIsa::Reference => Err(NotSupported {
            isa,
            reason: "reference kernel unavailable",
        }),

• Files reviewed: 7/7 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Codecov Comments Bot (codecov-commenter)]

Codecov Report

❌ Patch coverage is 99.54463% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.74%. Comparing base (cc473fc) to head (2e4b053).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...ion/src/matrix_kernels/maxsim/minmax8_x_minmax4.rs	99.61%	3 Missing ⚠️
diskann-quantization/src/minmax/multi/factory.rs	99.36%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1394      +/-   ##
==========================================
+ Coverage   92.65%   92.74%   +0.08%     
==========================================
  Files         527      530       +3     
  Lines      103254   104352    +1098     
==========================================
+ Hits        95674    96778    +1104     
+ Misses       7580     7574       -6     

Flag	Coverage Δ
miri	92.74% <99.54%> (+0.08%)	⬆️
unittests	92.68% <99.16%> (+0.07%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...quantization/src/matrix_kernels/blocks/unpacked.rs	100.00% <ø> (ø)
diskann-quantization/src/matrix_kernels/bounds.rs	100.00% <ø> (ø)
diskann-quantization/src/matrix_kernels/mod.rs	100.00% <ø> (ø)
diskann-quantization/src/minmax/multi/kernel.rs	100.00% <100.00%> (ø)
diskann-quantization/src/minmax/multi/factory.rs	99.36% <99.36%> (ø)
...ion/src/matrix_kernels/maxsim/minmax8_x_minmax4.rs	99.61% <99.61%> (ø)

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Mark Hildebrand (hildebrandmw)]

There's a lot going on here. It does add a minmax8x4 kernel, but basically builds a second matrix-kernel decomposition architecture rather than extending the existing abstractions introduced in 1368.

The point of the machinery introduced there was to use enable better higher-level experimentation of cache tilings and ordering without fully relying on Miri or checked accesses for correctness debugging. All of these pieces could then be tested, reviewed, and reused independently.

Here, packing and traversal are all reimplemented manually with inconsistent bounds tracking applied. This is hard to test, validate, or reuse.

My suggestion would be to follow the design philosophy of 1368 and build the traversal mechanism out of lower-level parts (which we can then reuse for other quantization kernels). In addition, matrix kernels should not own any non-scratch state because that also makes them much more difficult to reuse.

I also have some other design concerns:

• The microkernel is currently doing a lot, and I feel that there has to be a more efficient unpacking than doing this much bit-twiddling in general purpose registers. One design worth considering incrementally unpacking B and reusing that for the whole A traversal (flipping the cache tiling order - which again is easier to do via paneled views). This can perhaps be coupled with a different permutation strategy of the A-side query for more efficiency.
• ExtraWide should as much as possible keep the associated types opaque to the caller (like ExtraWide in the f32 kernel). This makes it significantly easier to express architecture specific optimizations of coarser grained kernels without the profoundly heaver SIMDVector and such constraints.
• How many of the Miri exceptions are really needed? Miri can emulate a good number of Neon and AVX2 intrinsics.
• Directed testing of the various steps is pretty sparse. Especially given its ad-hoc mix of checked, unchecked, and Bounds based indexing. Using Bounds based checking in a more disciplined way at least provides higher confidence of integration-based tests.

Have you considered making the A-side packing even more aggressive, splitting into groups of even and odd indices? The idea there is that it would naturally fit the order of unpacked nibbles much better and avoid a lot of the interleaving logic. Basically, a lot the micro-kernels are concerned about restoring a dimension ordering that we anyways control.

[juchen-ms (partychen)]

There's a lot going on here. It does add a minmax8x4 kernel, but basically builds a second matrix-kernel decomposition architecture rather than extending the existing abstractions introduced in 1368.

The point of the machinery introduced there was to use enable better higher-level experimentation of cache tilings and ordering without fully relying on Miri or checked accesses for correctness debugging. All of these pieces could then be tested, reviewed, and reused independently.

Here, packing and traversal are all reimplemented manually with inconsistent bounds tracking applied. This is hard to test, validate, or reuse.

My suggestion would be to follow the design philosophy of 1368 and build the traversal mechanism out of lower-level parts (which we can then reuse for other quantization kernels). In addition, matrix kernels should not own any non-scratch state because that also makes them much more difficult to reuse.

I also have some other design concerns:

• The microkernel is currently doing a lot, and I feel that there has to be a more efficient unpacking than doing this much bit-twiddling in general purpose registers. One design worth considering incrementally unpacking B and reusing that for the whole A traversal (flipping the cache tiling order - which again is easier to do via paneled views). This can perhaps be coupled with a different permutation strategy of the A-side query for more efficiency.
• ExtraWide should as much as possible keep the associated types opaque to the caller (like ExtraWide in the f32 kernel). This makes it significantly easier to express architecture specific optimizations of coarser grained kernels without the profoundly heaver SIMDVector and such constraints.
• How many of the Miri exceptions are really needed? Miri can emulate a good number of Neon and AVX2 intrinsics.
• Directed testing of the various steps is pretty sparse. Especially given its ad-hoc mix of checked, unchecked, and Bounds based indexing. Using Bounds based checking in a more disciplined way at least provides higher confidence of integration-based tests.

Have you considered making the A-side packing even more aggressive, splitting into groups of even and odd indices? The idea there is that it would naturally fit the order of unpacked nibbles much better and avoid a lot of the interleaving logic. Basically, a lot the micro-kernels are concerned about restoring a dimension ordering that we anyways control.

Mark Hildebrand (@hildebrandmw) Thanks for the detailed review. I reworked the implementation around the abstractions from #1368 in ed75906e:

• There is now a single generic Driver -> PanelKernel -> MicroKernel traversal for Scalar, V3, V4, and Neon; the separate V4/direct traversal was removed.
• Grouped<N> owns both query packing and canonical MinMax4 group expansion. Scalar/V3/Neon use Grouped<4> with even/odd dimensions, while V4 uses contiguous Grouped<8> values.
• B remains in canonical packed MinMax4 form. The same micro-kernel pipeline expands only the current contraction group, so the matrix kernel has no persistent state or document scratch allocation.
• ExtraWide keeps its register types opaque; architecture backends only provide load/splat/dot/reduce behavior.
• The Miri exceptions are limited to operations Miri cannot execute natively: BMI2 expansion, the emulated V4 broadcast, and adjacent-lane reduction. The native V4 path and the Miri-emulated V4 test both pass.

I also added/ran directed packing, tail, bounds, driver/register, factory-equivalence, repeated-computation, empty-input, and NaN-compensation coverage. Native x86-64 V3/V4 tests, ARM64 cross-Clippy, and the V4 Miri test pass.

For the 1,000-document 16x16 workload on a Xeon Platinum 8370C, the latest release measurements are:

Dim	Existing	V3	V4
250	11.47 us/doc	1.65 us/doc	0.52 us/doc
256	4.09 us/doc	1.61 us/doc	0.49 us/doc

The 250-dimensional Existing path has a scalar remainder, so I am keeping the 250- and 256-dimensional results explicitly separate.