Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/memory/MEMORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,4 @@ them. See [README.md](README.md) for the layout and write protocol.
- [Singular package names](singular-package-names.md) — packages are singular (`io.spine.elastic.benchmark`) unless a plural is explicitly approved; modules/directories are unaffected.
- [`when (this)` in enum initializers](enum-when-in-initializers.md) — init-order trap (`WhenMappings` references entries that don't exist yet); use computed `get()` properties for exhaustive per-entry dispatch.
- [Kotest Double NaN](kotest-double-nan.md) — `shouldBe` compares primitive doubles by IEEE (`NaN != NaN`); filter NaN out of equality-based property-test value generators.
- [Codegen KDoc article check](codegen-template-article-check.md) — the generator's `badArticles` guard rejects "a [%CLASS%]" / "a `%K%`" (renders to `a [Int`); write "of [%CLASS%]" instead.
19 changes: 19 additions & 0 deletions .agents/memory/codegen-template-article-check.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Codegen KDoc: never put `a`/`an` before a bracketed `%CLASS%` or `%K%`/`%V%`

The `:codegen` `TemplateEngine` runs a blunt `badArticles` check after each
substitution and rejects four literal substrings: `a [Int`, `an [Long`, and the
backtick forms of the same. Because `%CLASS%` renders to `IntIntMap` etc. and
`%K%`/`%V%` render to `Int`/`Long`, a KDoc phrase like "a [%CLASS%]" or "a `%K%`"
becomes "a [IntIntMap]" / "a `Int`" and aborts
`./gradlew :elastic:generateSwissMaps` with an "article mismatch" error naming the
template and the 1-based line.

**Why:** the guard is a plain `contains`, not a word-boundary match, so it fires on
any `Int`/`Long`-prefixed identifier, not only a bare `Int`/`Long`. The existing
templates sidestep it by writing "of [%CLASS%]" or "each `%K%`", never "a [%CLASS%]".

**How to apply:** when editing `PrimitiveMap.kt.tpl` / `SwissMap.kt.tpl` (and their
spec templates), reference a substituted type with no indefinite article in front —
"of [%CLASS%]", "yielded by [%CLASS%.forEach]", "each `%K%` key". The same
validation pass enforces ≤100-char lines and zero placeholder residue, so read a
failed generate message literally. (Found adding the non-boxing `forEach` matrix.)
104 changes: 104 additions & 0 deletions .agents/tasks/primitive-map-foreach.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Task: non-boxing `forEach` for the Swiss map matrix

**Status:** implemented & verified (`:elastic:check` green) · **Opened:** 2026-07-04 · **Surfaced by:** Phase 6 benchmark work

## Goal

Give the primitive-value maps (`LongLongMap`, `LongIntMap`, `LongDoubleMap`,
`IntIntMap`, `IntLongMap`, `IntDoubleMap`) a non-boxing traversal API — the one
capability every competitor primitive-map library offers (fastutil `forEach`,
HPPC cursors, Eclipse `forEachKeyValue`, Agrona `longForEach`) and the reason the
comparative `iterate` benchmark op currently cannot include our maps. Add the
same to the boxed-value maps (`SwissLongMap`, `SwissIntMap`) where it composes
cleanly.

## Design decision (settled)

Callback shape: **a generated top-level `public fun interface` per cell**,
mirroring the house style of `LongHasher`/`IntHasher` (and the fastutil/Agrona
`…Consumer` convention). A plain `(K, V) -> Unit` was rejected: it erases to
`Function2`, whose `invoke` boxes each primitive — the exact cost this library
exists to eliminate. A public `inline fun` was rejected: it may not touch the
`private tables` or the `internal object Swar`, so it cannot be a published
entry point. `forEach` is therefore a **member** (not an extension), because it
needs that private/internal state.

- Primitive cells: `%K%%V%Consumer` with `fun accept(key: %K%, value: %V%)`
(e.g. `LongLongConsumer`, `IntDoubleConsumer`) — both key and value unboxed.
- Boxed cells: `%K%ObjectConsumer<V>` with `fun accept(key: %K%, value: V)`
(`LongObjectConsumer<V>`, `IntObjectConsumer<V>`) — primitive key unboxed; the
value is already an object.
- Each `Consumer` is **co-located in the same generated file as its map**,
following the `LongHasher.kt` precedent (an interface + a related free
declaration share one file; the class matching the filename keeps
`MatchingDeclarationName` satisfied).

Iteration-safety contract: documented as "must not be structurally modified
during the walk" (undefined behavior otherwise), matching `java.util.Map.forEach`
and the maps' existing "single-threaded, concurrent mutation undefined" stance.
No fail-fast guard: the primitive maps carry no `modCount`, and adding one would
tax the allocation-free hot path — against the maps' core promise. `forEach`
itself allocates nothing; it reuses the existing full-lane control-word walk
(`Swar.matchFull → firstLane → clearLowest`) already in `rehashOrGrow` /
`containsValue`.

## Files

Production templates (regenerate after):
- `codegen/.../template/PrimitiveMap.kt.tpl` — add `forEach` member after
`clear()`; add the `%K%%V%Consumer` interface after the class; bump the
`@Suppress("TooManyFunctions")` note (11 → 12 own-API functions).
- `codegen/.../template/SwissMap.kt.tpl` — add `forEach` member after
`asMutableMap()`; add `%K%ObjectConsumer<V>` after the class; bump the
`@Suppress("TooManyFunctions")` note (11 → 12).

Test templates:
- `codegen/.../template/PrimitiveMapSpec.kt.tpl` — add a `forEach` visits-once test.
- `codegen/.../template/PrimitiveMapPropertiesSpec.kt.tpl` — add a randomized
`forEach`-vs-model property.
- `codegen/.../template/SwissMapViewSpec.kt.tpl` — add a boxed `forEach` test.

Hand-written frozen-oracle mirrors (must track the templates test-for-test):
- `elastic/.../LongLongMapSpec.kt` — mirror the visits-once test, specialized to `(Long, Long)`.
- `elastic/.../LongLongMapPropertiesSpec.kt` — mirror the property.

Regeneration: `./gradlew :elastic:generateSwissMaps` rewrites the 6 primitive
maps, 2 boxed maps, and the generated specs.

## Benchmark op (added)

The `iterate` op now exists so the Phase 6 matrix can include our maps
(`benchmarks/src/commonMain/.../*Benchmark.kt`, compiles clean on JVM + Native):
- `LongLongMapBenchmark.iterate` — unboxed `Long → Long` via `forEach`.
- `SwissLongMapBenchmark.iterate` — unboxed key, boxed value (the gradient middle).
- `StdlibHashMapBenchmark.iterate` — boxed `HashMap<Long, Long>` baseline, the
partner for both Long-axis maps.
- `IntIntMapBenchmark.iterate` / `iterateBoxed` — self-contained primitive-vs-boxed.

Each sums `key xor value` into a primitive sink consumed once through the
`Blackhole`; the boxed arms iterate via entry destructuring (`{ (k, v) -> … }`) so
the call resolves on Native, not just the JVM `BiConsumer` overload. The one-time
captured-sink `Ref` is symmetric across arms, so the measured delta is the
per-entry boxing. Elastic/Funnel/SingleWriter benchmarks are untouched.

## Out of scope (deliberately)

- Running the JMH matrix on pinned hardware and publishing numbers — Phase 6.
- `forEach` on the `OpenAddressing{Long,Int}Map` interface — DP-10 keeps that
contract lean; `forEach` stays a concrete member of the Swiss classes for now.
- `SingleWriterSwissLongMap` — hand-written concurrent variant, not part of the
mechanically-substituted matrix (a lock-free snapshot walk is a separate task).

## Verification

- `./gradlew :elastic:verifyGeneratedSwissMaps` (drift-clean after regeneration).
- `./gradlew :codegen:test` (engine + drift specs still green).
- `./gradlew :elastic:build` / `check` — compile all targets, detekt, and the
generated + hand-written specs, including the new `forEach` coverage.
- Confirm no `%…%` residue and no >100-char line (the generator validates both).

## Follow-ups

- Phase 6: run the `iterate` column (with the rest of the matrix) on pinned
hardware and publish the boxing-gradient numbers
(`HashMap` boxes both · `SwissLongMap` boxes value only · `LongLongMap` boxes neither).
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ import kotlinx.benchmark.Warmup
* boxing. `lookupHitShuffled` is the fair random-access gate (see
* [SwissLongMapBenchmark]); the `FastHash` arm quantifies [IntHasher.Fibonacci]
* against the default finalizer, exactly like the `LongLongMap` benchmark.
* `iterate` / `iterateBoxed` contrast a full non-boxing [IntIntMap.forEach]
* traversal against the boxed `HashMap` entry-set walk.
*/
@State(Scope.Benchmark)
@BenchmarkMode(Mode.AverageTime)
Expand Down Expand Up @@ -125,6 +127,24 @@ class IntIntMapBenchmark {
blackhole.consume(sink)
}

@Benchmark
fun iterate(blackhole: Blackhole) {
val m = map
var sink = 0
m.forEach { key, value -> sink += key xor value }
blackhole.consume(sink)
}

@Benchmark
fun iterateBoxed(blackhole: Blackhole) {
val m = boxed
var sink = 0
for ((key, value) in m) {
sink += key xor value
}
blackhole.consume(sink)
}

@Benchmark
fun insertAllPresized(blackhole: Blackhole) {
val fresh = IntIntMap(expectedSize = size)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,9 @@ import kotlinx.benchmark.Warmup
* into a sink consumed once through the [Blackhole], so the hot path stays free of
* the boxing a per-result `consume` would introduce — the whole point of this map.
* `lookupHitShuffled` is the fair random-access gate (see [SwissLongMapBenchmark]).
* `iterate` walks every entry through the non-boxing [LongLongMap.forEach] — the
* traversal the boxed baseline in [StdlibHashMapBenchmark] cannot match without
* boxing each key and value.
*/
@State(Scope.Benchmark)
@BenchmarkMode(Mode.AverageTime)
Expand Down Expand Up @@ -115,6 +118,14 @@ class LongLongMapBenchmark {
blackhole.consume(sink)
}

@Benchmark
fun iterate(blackhole: Blackhole) {
val m = map
var sink = 0L
m.forEach { key, value -> sink += key xor value }
blackhole.consume(sink)
}

@Benchmark
fun insertAllPresized(blackhole: Blackhole) {
val fresh = LongLongMap(expectedSize = size)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,9 @@ import kotlinx.benchmark.Warmup
* baseline, with the map pre-sized in its own units (capacity for `size` entries
* at the JDK load factor) as the fairness gate requires; and `insertAllGrowing` —
* the same inserts into a default-capacity map, isolating resize/rehash cost.
* Each result is consumed through a [Blackhole] to defeat dead-code elimination.
* `iterate` walks the entry set — the boxed counterpart of the primitive maps'
* `forEach` traversal, boxing each key and value it visits. Each result is consumed
* through a [Blackhole] to defeat dead-code elimination.
*/
@State(Scope.Benchmark)
@BenchmarkMode(Mode.AverageTime)
Expand Down Expand Up @@ -99,6 +101,16 @@ class StdlibHashMapBenchmark {
blackhole.consume(sink)
}

@Benchmark
fun iterate(blackhole: Blackhole) {
val m = map
var sink = 0L
for ((key, value) in m) {
sink += key xor value
}
blackhole.consume(sink)
}

@Benchmark
fun insertAllPresized(blackhole: Blackhole) {
val fresh = HashMap<Long, Long>((size / JDK_LOAD_FACTOR).toInt() + 1)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,10 @@ import kotlinx.benchmark.Warmup
* steady-state insert with the map pre-sized in its own units ([SwissLongMap]'s
* `expectedSize`); and `insertAllGrowing`, the same inserts into a default-capacity
* map, folding in resize/rehash cost — plus `lookupMiss` (keys absent from the map,
* exercising the stop-on-empty probe). Each result is consumed through a [Blackhole]
* exercising the stop-on-empty probe). `iterate` walks every entry through
* [SwissLongMap.forEach], which hands out the `Long` key unboxed and the value as its
* stored object — so it sits between the fully unboxed [LongLongMap] and the
* doubly-boxed [StdlibHashMapBenchmark]. Each result is consumed through a [Blackhole]
* to defeat dead-code elimination.
*/
@State(Scope.Benchmark)
Expand Down Expand Up @@ -110,6 +113,14 @@ class SwissLongMapBenchmark {
blackhole.consume(sink)
}

@Benchmark
fun iterate(blackhole: Blackhole) {
val m = map
var sink = 0L
m.forEach { key, value -> sink += key xor value }
blackhole.consume(sink)
}

@Benchmark
fun insertAllPresized(blackhole: Blackhole) {
val fresh = SwissLongMap<Long>(expectedSize = size)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ import io.spine.elastic.internal.Swar
* @param absentValue the value returned for an absent key; defaults to %V_ZERO_DOC%
* @param hasher the hash function applied to keys; defaults to [%HASHER%.Default]
*/
@Suppress("TooManyFunctions") // The map's own API is eleven functions, one over the limit.
@Suppress("TooManyFunctions") // The map's own API is twelve functions, two over the limit.
public class %CLASS%(
expectedSize: Int = 0,
public val absentValue: %V% = %V_ZERO%,
Expand Down Expand Up @@ -177,6 +177,29 @@ public class %CLASS%(
growthLeft = Capacity.maxLoad(capacity)
}

/**
* Applies [action] to every key-value pair, in unspecified order.
*
* A cold, allocation-free traversal kept off the hot path: it walks the control
* words directly — the same full-lane scan a rebuild uses — and hands each `%K%`
* key and `%V%` value to [action] without boxing either. The map must not be
* structurally modified during the walk.
*/
public fun forEach(action: %K%%V%Consumer) {
val current = tables
val control = current.control
val keys = current.keys
val values = current.values
for (group in control.indices) {
var fulls = Swar.matchFull(control[group])
while (fulls != 0L) {
val slot = (group shl Swar.GROUP_SHIFT) + Swar.firstLane(fulls)
action.accept(keys[slot], values[slot])
fulls = Swar.clearLowest(fulls)
}
}
}

/**
* Returns the slot holding [key] in [table], or [SLOT_ABSENT] if it is not
* present. The probe stops at the first empty lane, which proves absence.
Expand Down Expand Up @@ -323,3 +346,17 @@ public class %CLASS%(
private const val SLOT_ABSENT = -1
}
}

/**
* A non-boxing callback over the entries of [%CLASS%].
*
* The functional-interface counterpart of `(%K%, %V%) -> Unit` that boxes neither
* argument: a Kotlin function type erases to `Function2`, whose `invoke` boxes each
* primitive, whereas [accept] takes them as primitives. It is the argument of
* [%CLASS%.forEach]; a lambda `{ key, value -> … }` converts to it.
*/
public fun interface %K%%V%Consumer {

/** Consumes one key-value pair yielded by [%CLASS%.forEach]. */
public fun accept(key: %K%, value: %V%)
}
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,25 @@ internal class %CLASS%PropertiesSpec {
}
}

@Test
fun `forEach visits exactly the entries the model holds`() = runTest {
checkAll(120, Arb.list(operations(%K_ARB%()), 0..600)) { ops ->
val map = %CLASS%()
val model = HashMap<%K%, %V%>()
for (op in ops) {
applyAndCompare(op, map, model, %V_ZERO%)
}
var count = 0
val collected = HashMap<%K%, %V%>()
map.forEach { key, value ->
collected[key] = value
count++
}
count shouldBe model.size
collected shouldBe model
}
}

private fun applyAndCompare(
op: Op,
map: %CLASS%,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,24 @@ internal class %CLASS%Spec {
map.size shouldBe 1
}

@Test
fun `visits every entry exactly once via forEach`() {
val map = %CLASS%()
val model = HashMap<%K%, %V%>()
for (key in %K(0)% until %K(100)%) {
map.put(key, (key * %K(2)%).to%V%())
model[key] = (key * %K(2)%).to%V%()
}
var count = 0
val seen = HashMap<%K%, %V%>()
map.forEach { key, value ->
seen[key] = value
count++
}
count shouldBe model.size
seen shouldBe model
}

@Test
fun `rejects a negative expected size`() {
shouldThrow<IllegalArgumentException> {
Expand Down
Loading
Loading