Skip to content

Use Fn for out-of-place lane kernels - #9419

Draft
connortsui20 wants to merge 1 commit into
developfrom
ct/map-into-fn
Draft

Use Fn for out-of-place lane kernels#9419
connortsui20 wants to merge 1 commit into
developfrom
ct/map-into-fn

Conversation

@connortsui20

@connortsui20 connortsui20 commented Aug 14, 2026

Copy link
Copy Markdown
Member

Narrows the out-of-place lane-kernel callback bounds from FnMut to Fn and propagates that contract through the current vortex-array adapters, so callers cannot introduce loop-carried state into per-lane work.

The decimal speedup comes from the callback shape after monomorphization. The old adapter reached the loop through &mut &mut Closure, while the Fn version uses shared references and lets LLVM vectorize the range check. Fn is not required to recover this codegen: passing the stateless closure by value through the existing FnMut API also vectorizes, but this PR chooses Fn because these kernels model independent per-lane work and do not need mutable callback state.

Codegen investigation

cast_decimal_buffer previously passed &mut cast into try_map_into. That monomorphized the kernel with F = &mut Closure, then its chunk helper received &mut F, producing the nested mutable receiver.

Variant Receiver in the chunk loop Optimized result
Existing FnMut adapter &mut &mut Closure Scalar range check with a captured-bound load on each lane
This PR &&Closure Both bounds hoisted and eight i64 values processed per iteration on arm64
Diagnostic FnMut variant &mut Closure Same vector loop as this PR

The diagnostic variant kept every FnMut bound and only moved the closure into try_map_into. A narrower fix could make that call-site change, or otherwise restructure the helper to avoid the nested mutable receiver, without restricting the public callback bound.

Local benchmark results

Compared b363fb7 (FnMut) with e973d82 (Fn) on arm64 macOS using rustc 1.97.1, -C target-cpu=native, the bench profile, and CodSpeed's wall-time runner. Each run used 1,000 samples, and the order alternated across three baseline/candidate pairs.

The copy_non_nullable case casts 65,536 non-nullable decimal values through the affected dense try_map_into path.

Run FnMut Fn
Pair 1 22.79 µs 13.70 µs
Pair 2 23.33 µs 13.37 µs
Pair 3 23.43 µs 13.37 µs

The median run fell from 23.33 µs to 13.37 µs, a 42.7% reduction. Other changed arithmetic, comparison, and cast monomorphizations showed no consistent separation, while the unchanged Boolean and in-place cast controls were flat.

A separate three-pair check of the CodSpeed-reported compress_fsst[(1000, 64, 8)] regression used 500 samples per run. Its median run was 112.4 µs for the baseline and 111.9 µs for this PR, so the local runs did not reproduce that regression.

Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
@connortsui20 connortsui20 added the changelog/performance A performance improvement label Aug 14, 2026
@codspeed-hq

codspeed-hq Bot commented Aug 14, 2026

Copy link
Copy Markdown

Merging this PR will regress 1 benchmark

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 6 improved benchmarks
❌ 1 regressed benchmark
✅ 1993 untouched benchmarks
⏩ 89 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation compress_fsst[(1000, 64, 8)] 1 ms 1.2 ms -11.08%
Simulation copy_non_nullable[65536] 1.7 ms 1.2 ms +39.85%
WallTime words_gather_scalar[65536] 9.4 µs 8.2 µs +13.6%
Simulation take[small_m/shuffled/primitive/nonnull/chunks=1024/indices=256] 2.3 ms 2 ms +13.27%
Simulation take[small_m/shuffled/primitive/nonnull/chunks=1024/indices=1000] 6.1 ms 5.4 ms +13.1%
Simulation take[small_m/shuffled/primitive/nonnull/chunks=16384/indices=1000] 10.2 ms 9 ms +12.82%
Simulation take[core/shuffled/primitive/nonnull/chunks=1024/indices=10000] 14.8 ms 13.3 ms +11.71%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ct/map-into-fn (e973d82) with develop (b363fb7)

Open in CodSpeed

Footnotes

  1. 89 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@connortsui20
connortsui20 marked this pull request as ready for review August 14, 2026 21:19
@connortsui20

Copy link
Copy Markdown
Member Author

@connortsui20 connortsui20 changed the title Use Fn for out-of-place lane kernels Use Fn for out-of-place lane kernels Aug 14, 2026
@connortsui20
connortsui20 marked this pull request as draft August 14, 2026 21:21
@connortsui20

Copy link
Copy Markdown
Member Author

I guess we could be a bit more precise with which is Fn vs which is FnMut but I feel like we should actually default to more constrained for now and relax it when we actually need to.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant