Skip to content

TailCache 05: add persisted mode and mixed workload matrix - #4

Open
Enixes wants to merge 8 commits into
mainfrom
tailcache-05-workloads-persistence-concurrency
Open

TailCache 05: add persisted mode and mixed workload matrix#4
Enixes wants to merge 8 commits into
mainfrom
tailcache-05-workloads-persistence-concurrency

Conversation

@Enixes

@Enixes Enixes commented Sep 12, 2026

Copy link
Copy Markdown
Owner

What this adds

TailCache 05 expands the validated operation-level harness into a controlled mixed-workload matrix while carrying forward the post-merge TailCache 04 peer-review hardening.

TailCache 04 peer-review follow-up

  • disables Chronicle vendor analytics in test and benchmark JVMs with -Dchronicle.analytics.disable=true
  • explicitly freezes/logs allowSegmentTiering=true
  • stops describing maxBloatFactor(1.0) as a guarantee that segments never tier
  • adds a wrong-value-size negative test for Chronicle's constant-size layout
  • describes backend configuration output as experiment-relevant rather than exhaustive

The hardened Chronicle adapter/operation-level harness was revalidated successfully on revision d98be0f34202d7bb3421d6d2c692741db6fc6501.

Primary backend/storage modes

  • CAFFEINE
  • CHRONICLE_IN_MEMORY
  • CHRONICLE_PERSISTED

Persisted Chronicle uses createPersistedTo(...) with a unique trial-local temporary file. The workload prepopulates outside measurement; teardown closes the map and explicitly deletes the file/directory. The adapter itself does not delete persisted data, and a unit test verifies that data survives close/reopen.

Chronicle's normal checksum semantics are frozen explicitly: in-memory maps use entryChecksums=false, persisted maps use entryChecksums=true. Persisted-vs-in-memory is therefore treated as a deployment-mode comparison, not a pure mmap-only cost comparison.

Workload matrix

  • access distributions: UNIFORM, ZIPFIAN
  • Zipf exponent: 0.99 (same numeric constant as YCSB's default Zipfian generators; TailCache does not claim trace equivalence)
  • complete-trace read/existing-key-put mixes: READ_95_WRITE_5, READ_70_WRITE_30
  • payloads: 256 B, 4 KiB
  • deterministic seed: 0x5EED

Key selection and operation selection use separate deterministic random streams, so changing the read/write ratio does not silently change the logical key sequence. Read counts are exact over each complete generated trace; a time-limited JMH measurement window can stop part-way through a trace cycle.

Every mixed read is a hit against the prepopulated resident working set and every mixed write is an existing-key put. Uniform vs Zipfian therefore studies locality/hot-key contention, not admission/eviction/hit-rate behavior.

Replacement values are preallocated per logical key to remove value allocation from the measured put path. Repeated puts to the same key may therefore become logically idempotent. The replacement bank is also a live heap shadow set for Chronicle, so this matrix is for latency/contention and must not be used to claim Chronicle heap/GC savings.

Shared-cache concurrency

The mixed workload deliberately uses:

  • CacheWorkloadState -> @State(Scope.Benchmark) for one cache shared by all workers
  • WorkloadCursorState -> @State(Scope.Thread) for one cursor per worker
  • deterministic staggered offsets over the same cyclic workload trace

Peer review moved thread-index lookup, trace-size lookup, and initial-offset calculation into @Setup(Level.Iteration). The measured path now only advances/wraps the initialized cursor before trace/key lookup and the selected cache operation.

This makes the 16-thread smoke a same-JVM shared-cache contention experiment under a staggered shared deterministic trace model. It is not 16 independently generated client streams. The older CacheSmokeBenchmark remains Scope.Thread and must not be reinterpreted as shared-cache contention by merely raising its thread count.

Two-JVM persisted sharing remains a separate secondary experiment.

Diagnostic tasks / config export

Adds:

./gradlew jmhWorkloadSmoke
./gradlew jmhWorkloadShared16Smoke

Both are deliberately short 1/1/1, 300 ms diagnostics. Each expands 3 storage modes x 2 payload sizes x 2 distributions x 2 mixes = 24 parameter combinations.

JMH JSON is retained at:

build/reports/jmh/workload-smoke-1t.json
build/reports/jmh/workload-smoke-16t.json

Validation status — COMPLETE

A complete validation bundle passed on executable revision d98be0f34202d7bb3421d6d2c692741db6fc6501:

./gradlew test --rerun-tasks
./gradlew jmhSmoke
./gradlew jmhChronicleAllocSmoke
./gradlew jmhChronicleJfrSmoke
./gradlew jmhWorkloadSmoke
./gradlew jmhWorkloadShared16Smoke

Confirmed there:

  • tests executed and passed
  • actual JMH forks used JDK 21.0.12.1
  • Chronicle analytics disable appeared exactly once per benchmark fork
  • allowSegmentTiering=true was logged
  • in-memory Chronicle logged entryChecksums=false; persisted logged entryChecksums=true
  • TailCache 04 allocation/JFR classifications remained intact
  • both workload tasks expanded all 24 combinations
  • the 16-thread task reported 16 workers against stateScope=Benchmark(shared-cache)
  • both JSON exports contained 24 benchmark records
  • persisted trial temp directories were absent after successful teardown

After that run, peer review removed thread-index/trace-size initialization from the measured mixed-workload path. The required narrow rerun then passed on revision a547f5d1bc735c6d47afff171c96feec10ea025a:

./gradlew test --rerun-tasks
./gradlew jmhWorkloadSmoke
./gradlew jmhWorkloadShared16Smoke

Confirmed again:

  • tests compiled and passed with the JMH state dependency in cursor setup
  • one-thread workload smoke completed and wrote 24 JSON benchmark records
  • 16-thread workload smoke completed with 16 workers and stateScope=Benchmark(shared-cache) and wrote 24 JSON benchmark records
  • persisted trial temp directories were absent after successful teardown
  • workload benchmark forks remained on JDK 21.0.12.1 with Chronicle analytics disabled

No additional Chronicle allocation/JFR rerun was required because the final hardening touched only mixed-workload cursor plumbing, not the Chronicle adapter or CacheSmokeBenchmark.

Methodology guardrails

  • smoke numbers are not research conclusions
  • mixed-workload percentiles combine read and existing-key-put operations; they are not read-only/write-only percentiles
  • persisted-warm means file-backed/memory-mapped steady-state intent, not synchronous durable disk latency
  • /tmp is acceptable for smoke only; reportable persisted runs need a configurable benchmark root plus filesystem/mount/device metadata
  • Caffeine maximumSize and Chronicle entries occupancy semantics still need to be frozen before reportable runs
  • two-JVM sharing remains separate from single-JVM persisted mode
  • the per-key replacement bank intentionally excludes heap-footprint/GC conclusions from this matrix

TailCache 05 is validated and merge-ready. Reportable benchmarking remains blocked on the trustworthy-harness controls in Milestone 1.

@Enixes
Enixes force-pushed the tailcache-05-workloads-persistence-concurrency branch from 19b8ab0 to d98be0f Compare September 12, 2026 18:35
@Enixes
Enixes marked this pull request as ready for review September 13, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant