Skip to content

flatkv cache#3027

Draft
cody-littley wants to merge 62 commits intomainfrom
cjl/flatkv-cache
Draft

flatkv cache#3027
cody-littley wants to merge 62 commits intomainfrom
cjl/flatkv-cache

Conversation

@cody-littley
Copy link
Contributor

@cody-littley cody-littley commented Mar 5, 2026

Describe your changes and provide context

Add a caching layer to FlatKV, more than doubling performance in cryptosim benchmarks.

Testing performed to validate your change

Unit tests, ran benchmark over several days.

@github-actions
Copy link

github-actions bot commented Mar 5, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedMar 12, 2026, 5:03 PM

@codecov
Copy link

codecov bot commented Mar 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 58.37%. Comparing base (2f6c4e5) to head (9ff2199).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3027      +/-   ##
==========================================
+ Coverage   58.27%   58.37%   +0.09%     
==========================================
  Files        2076     2078       +2     
  Lines      171480   170750     -730     
==========================================
- Hits        99933    99670     -263     
+ Misses      62615    62182     -433     
+ Partials     8932     8898      -34     
Flag Coverage Δ
sei-db 70.41% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
sei-cosmos/storev2/rootmulti/store.go 46.33% <ø> (-0.17%) ⬇️
sei-db/common/metrics/phase_timer.go 73.91% <ø> (ø)
sei-db/config/sc_config.go 100.00% <ø> (ø)
sei-db/db_engine/pebbledb/batch.go 100.00% <ø> (ø)
sei-db/db_engine/pebbledb/db.go 93.22% <ø> (-0.76%) ⬇️
sei-db/state_db/sc/composite/store.go 55.04% <ø> (+6.20%) ⬆️
sei-db/state_db/sc/flatkv/snapshot.go 66.04% <ø> (-0.53%) ⬇️
sei-db/state_db/sc/flatkv/store.go 56.75% <ø> (-3.25%) ⬇️
sei-db/state_db/sc/flatkv/store_lifecycle.go 46.51% <ø> (-9.94%) ⬇️
sei-db/state_db/sc/flatkv/store_read.go 55.31% <ø> (-1.52%) ⬇️
... and 1 more

... and 245 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.


coreCount := runtime.NumCPU()
readPoolSize := int(cfg.ReaderThreadsPerCore*float64(coreCount) + float64(cfg.ReaderConstantThreadCount))
miscPoolSize := int(cfg.MiscPoolThreadsPerCore*float64(coreCount) + float64(cfg.MiscConstantThreadCount))

Check notice

Code scanning / CodeQL

Floating point arithmetic Note

Floating point arithmetic operations are not associative and a possible source of non-determinism

coreCount := runtime.NumCPU()
readPoolSize := int(cfg.ReaderThreadsPerCore*float64(coreCount) + float64(cfg.ReaderConstantThreadCount))
miscPoolSize := int(cfg.MiscPoolThreadsPerCore*float64(coreCount) + float64(cfg.MiscConstantThreadCount))

Check notice

Code scanning / CodeQL

Floating point arithmetic Note

Floating point arithmetic operations are not associative and a possible source of non-determinism
ctx, cancel := context.WithCancel(ctx)

coreCount := runtime.NumCPU()
readPoolSize := int(cfg.ReaderThreadsPerCore*float64(coreCount) + float64(cfg.ReaderConstantThreadCount))

Check notice

Code scanning / CodeQL

Floating point arithmetic Note

Floating point arithmetic operations are not associative and a possible source of non-determinism
ctx, cancel := context.WithCancel(ctx)

coreCount := runtime.NumCPU()
readPoolSize := int(cfg.ReaderThreadsPerCore*float64(coreCount) + float64(cfg.ReaderConstantThreadCount))

Check notice

Code scanning / CodeQL

Floating point arithmetic Note

Floating point arithmetic operations are not associative and a possible source of non-determinism
"io"
"os"
"path/filepath"
"runtime"

Check notice

Code scanning / CodeQL

Sensitive package import Note

Certain system packages contain functions which may be a possible source of non-determinism
valueChan := entry.valueChan
s.lock.Unlock()
s.metrics.reportCacheMisses(1)
startTime := time.Now()

Check warning

Code scanning / CodeQL

Calling the system time Warning

Calling the system time may be a possible source of non-determinism
entry.valueChan = valueChan
s.lock.Unlock()
s.metrics.reportCacheMisses(1)
startTime := time.Now()

Check warning

Code scanning / CodeQL

Calling the system time Warning

Calling the system time may be a possible source of non-determinism
}

s.metrics.reportCacheMisses(int64(len(pending)))
startTime := time.Now()

Check warning

Code scanning / CodeQL

Calling the system time Warning

Calling the system time may be a possible source of non-determinism
}

s.metrics.reportCacheMissLatency(time.Since(startTime))
go s.bulkInjectValues(pending)

Check notice

Code scanning / CodeQL

Spawning a Go routine Note

Spawning a Go routine may be a possible source of non-determinism
missLatency: missLatency,
}

go cm.collectLoop(ctx, scrapeInterval, getSize)

Check notice

Code scanning / CodeQL

Spawning a Go routine Note

Spawning a Go routine may be a possible source of non-determinism
@cody-littley cody-littley marked this pull request as ready for review March 10, 2026 21:11
@cody-littley cody-littley marked this pull request as draft March 11, 2026 17:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant