feat(wasm): SIMD128 kernels for l2_squared/inner_product distance (#675)#686
Open
ohdearquant wants to merge 1 commit into
Open
feat(wasm): SIMD128 kernels for l2_squared/inner_product distance (#675)#686ohdearquant wants to merge 1 commit into
ohdearquant wants to merge 1 commit into
Conversation
Adds wasm32 SIMD128 kernels (core::arch::wasm32, v128/f32x4 lanes) for
l2_squared and inner_product in ruvector-diskann's distance dispatch,
gated under #[cfg(all(target_arch = "wasm32", target_feature = "simd128"))],
mirroring the existing simd feature's dispatch shape (SimSIMD on native).
Dispatch priority becomes: GPU -> SimSIMD (native) -> WASM SIMD128 -> scalar.
PQ asymmetric-distance table construction speeds up transparently through
the same l2_squared dispatch (no separate kernel needed there).
Adds a getrandom 0.2 "js" feature shim (wasm32-only dependency, mirrors
ruvector-wasm/Cargo.toml) required for the crate to compile for wasm32 at
all -- rand's transitive getrandom dependency otherwise fails the build.
Correctness (wasm-bindgen-test, wasm-pack test --node) and an A/B timing
report cover dims {0,1,2,3,4,5,7,8,9,384,768,1000,1023,1024}; measured
geometric-mean speedup 4.18x across {l2_squared, inner_product} x
{384,768,1024} dims in Node (release build). Native builds are unaffected;
the new code is entirely cfg'd out off wasm32.
Fixes ruvnet#675
Contributor
Author
|
CI note — the two red checks on this PR are repo-level, not from this diff:
Every build/test/audit job relevant to this change passes. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
wasm32SIMD128 kernels (core::arch::wasm32,v128/f32x4lanes) for thetwo hot distance primitives in
ruvector-diskann's dispatch chain —l2_squaredand
inner_product— gated under#[cfg(all(target_arch = "wasm32", target_feature = "simd128"))], following the exact dispatch shape the crate's existingsimdfeature (SimSIMD, native NEON/AVX2/AVX-512) already uses. Native builds are
unaffected: the new code is compiled out entirely off
wasm32.PQ asymmetric-distance table construction (
ProductQuantizer::build_distance_tablein
pq.rs) callsl2_squaredin its inner loop and speeds up transparentlythrough the same dispatch — no separate kernel or code change needed there.
Why (#675)
Today,
wasm32builds of this crate always take the scalar fallback(
scalar_l2_squared/scalar_inner_product) because thesimdfeature routesthrough
simsimd, which only targets NEON/AVX2/AVX-512 — notwasm32. This addsthe missing leg of the dispatch chain: GPU → SimSIMD (native) → WASM SIMD128
(wasm32 + simd128) → scalar.
Gate (pre-registered before any measurement)
PASS iff:
wasm-pack test --node(SIMD128-enabled build) — everySIMD128-vs-scalar comparison across dims
{0, 1, 3, 5, 7, 384, 768, 1000, 1023, 1024}(covers empty/short/non-multiple-of-4/production-size) hasmax |diff| < 1e-6, on randomized inputs (seeded PRNG for reproducibility).scalar_time / simd128_timeacross the 6-cellgrid
{l2_squared, inner_product} × {384, 768, 1024}dims, measured in thesame Node process / same wasm instance (both code paths compiled into one
simd128-enabled binary, so the comparison isn't confounded by two separatebuilds) with a warmup phase before timing, is ≥ 1.2×.
If PASS: this PR is opened with the full table below. If FAIL: no PR is opened;
the numbers + analysis are committed to
AB_675.mdon the branch instead.Correctness-gate amendment (made before the speed run, once the correctness
tests were first executed): a flat
< 1e-6absolute tolerance turned out tobe numerically unreachable — e.g.
l2_squaredat dim=8 gave scalar=433.04245,simd128=433.04248, diff≈3.05e-5, which fails
< 1e-6outright. This isn't acorrectness bug: the scalar path sums with 4 interleaved f32 accumulators, the
simd128 path with 2
v128(8-wide) accumulators plus a horizontal-sum tail, sothe two reduction trees add terms in different orders. f32 addition isn't
associative, so reordering shifts rounding by a few ULPs, and a fixed absolute
bound can't hold across the requested magnitude range (results up to ~10^5 for
dim=1024). Switched to the standard combined tolerance
(
|diff| <= atol + rtol * max(|a|,|b|),atol=rtol=1e-5, numpy-allclosestyle) before running any correctness or speed numbers on the full dim grid —
i.e. this was a tolerance-formula fix caught by the very first correctness run
(which failed loudly on the fixed absolute bound), not a threshold relaxed
after seeing the speed result. Observed relative error is ~1.1e-7 (essentially
f32 machine epsilon) — a real bug (wrong lane order, dropped remainder tail)
would show relative error orders of magnitude larger than that, so this bound
stays tight.
A/B table
Measured via
RUSTFLAGS="-C target-feature=+simd128" wasm-pack test --node --release -- --no-default-features -- --nocapture(Node v25.6.0, macOSaarch64). Both
scalar_*andwasm_simd128_*are compiled into onesimd128-enabled release binary and called back-to-back from the same wasminstance — release profile is
opt-level = 3, lto = "fat", codegen-units = 1(workspace
[profile.release]; the debug/default profile is notrepresentative here — it left the
unsafeintrinsic calls uninlined andmeasured simd128 as ~8x slower, corrected by switching to
--release).Each cell: 2,000 warmup iterations, then 5 rounds of 300,000 timed iterations
(median of 5 taken), wall time via the global
performance.now()(sub-msresolution;
Date.now()'s 1ms granularity was too coarse at these iterationcounts and was replaced before the numbers below were collected).
Geometric mean speedup: 4.18x (gate: ≥ 1.2x) — PASS.
Correctness tests
crates/ruvector-diskann/src/distance.rs,mod wasm_simd128_tests(gated#[cfg(all(test, target_arch = "wasm32", target_feature = "simd128"))],wasm-bindgen-test, runs under Node by default):l2_squared_matches_scalar_across_dims/inner_product_matches_scalar_across_dims:seeded-PRNG random vectors (
rand::rngs::StdRng, fixed seeds) at dims{0, 1, 2, 3, 4, 5, 7, 8, 9, 384, 768, 1000, 1023, 1024}— empty, sub-lane,4/8-lane boundaries, non-multiples-of-4, and production embedding sizes —
scalar vs simd128 compared with the combined tolerance above. All pass;
max observed relative error ~1.1e-7.
identical_vectors_are_zero_distance:wasm_simd128_l2_squared(a, a) < 1e-10.Run recipe (also in the doc-comment above the test module):
Native builds are unaffected — the new code (kernels + tests) is entirely
cfg'd out offwasm32;cargo test -p ruvector-diskann(17 tests),cargo fmt -p ruvector-diskann -- --check, andcargo clippy -p ruvector-diskann --all-targetsall pass unchanged.cargo clippy -p ruvector-diskann --target wasm32-unknown-unknown --no-default-features --all-targets(withRUSTFLAGS="-C target-feature=+simd128") is also clean.The default
wasm32-unknown-unknownbuild (nosimd128target-feature) stillcompiles and takes the scalar fallback, unchanged.
Cargo.toml note
rand(used by PQ k-means training) pulls ingetrandom0.2 transitively,which needs its
"js"feature to build forwasm32-unknown-unknownat all —this blocked the crate from compiling for
wasm32even before this PR'skernel code, independent of SIMD128. Added the same
getrandom02 = { package = "getrandom", version = "0.2", features = ["js"] }shim
ruvector-wasm/Cargo.tomlalready carries for the same transitivedependency, target-gated to
wasm32only (zero effect on native builds).Without it, none of this PR's wasm32 tests could actually run.
Fixes #675