[Prototype] FoundationDB by Kbhat1 · Pull Request #3481 · sei-protocol/sei-chain

Kbhat1 · 2026-05-20T22:49:30Z

Describe your changes and provide context

Testing performed to validate your change

cursor · 2026-05-20T22:49:55Z

PR Summary

Medium Risk
Adds new optional state-store read path that can route pruned Get/Has calls to external backends, plus new dependencies (Scylla driver, optional FoundationDB bindings). While gated by config, it touches state-store initialization and introduces new failure modes around remote connectivity and build tags.

Overview
Adds prototype historical-state offload support for the SS layer, allowing point reads (Get/Has) below local retention to fall back to an external store (ScyllaDB/Cassandra or FoundationDB), while iterators and writes remain on the local state store.

Wires new SS config/CLI flags and TOML template fields for both backends, updates parsing/tests, and updates ss.NewStateStore to wrap the primary store with a historical.FallbackStateStore (enforcing that only one backend can be configured).

Introduces a Kafka-based offload consumer tool (historical-scylla-consumer) with pluggable sinks for Scylla and FoundationDB (FoundationDB gated behind -tags foundationdb), adds schemas/docs/examples, and exports offload.NewSASLMechanism so the consumer can share producer auth logic.

^{Reviewed by Cursor Bugbot for commit eff7ead. Bugbot is set up for automated code reviews on this repo. Configure here.}

github-actions · 2026-05-20T22:51:08Z

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
`✅ passed`	`✅ passed`	`✅ passed`	`✅ passed`	May 22, 2026, 7:48 PM

codecov · 2026-05-20T22:53:42Z

Codecov Report

❌ Patch coverage is 42.88660% with 554 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.18%. Comparing base (654d40b) to head (e5b03f6).
⚠️ Report is 60 commits behind head on main.

Files with missing lines	Patch %	Lines
sei-db/state_db/ss/offload/consumer/consumer.go	0.00%	177 Missing ⚠️
sei-db/state_db/ss/offload/historical/scylla.go	49.29%	71 Missing and 1 partial ⚠️
...-db/state_db/ss/offload/historical/foundationdb.go	64.49%	47 Missing and 13 partials ⚠️
sei-db/state_db/ss/offload/consumer/scylla.go	62.17%	48 Missing and 11 partials ⚠️
sei-db/state_db/ss/store.go	18.51%	42 Missing and 2 partials ⚠️
sei-db/state_db/ss/offload/historical/store.go	53.76%	34 Missing and 9 partials ⚠️
sei-db/state_db/ss/offload/consumer/config.go	31.25%	28 Missing and 5 partials ⚠️
sei-db/state_db/ss/offload/consumer/kafka.go	44.82%	31 Missing and 1 partial ⚠️
...ad/consumer/cmd/historical-scylla-consumer/main.go	0.00%	21 Missing ⚠️
sei-db/state_db/ss/offload/consumer/compact.go	81.81%	5 Missing and 1 partial ⚠️
... and 3 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3481      +/-   ##
==========================================
- Coverage   59.25%   59.18%   -0.07%     
==========================================
  Files        2110     2122      +12     
  Lines      174181   176209    +2028     
==========================================
+ Hits       103210   104291    +1081     
- Misses      62044    62923     +879     
- Partials     8927     8995      +68

Flag	Coverage Δ
sei-chain-pr	`49.36% <42.88%> (?)`
sei-db	`70.41% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
sei-db/config/ss_config.go	`100.00% <ø> (ø)`
sei-db/state_db/ss/offload/kafka.go	`59.37% <100.00%> (ø)`
...te_db/ss/offload/consumer/foundationdb_disabled.go	`0.00% <0.00%> (ø)`
..._db/ss/offload/historical/foundationdb_disabled.go	`0.00% <0.00%> (ø)`
app/seidb.go	`81.65% <76.92%> (+9.65%)`	⬆️
sei-db/state_db/ss/offload/consumer/compact.go	`81.81% <81.81%> (ø)`
...ad/consumer/cmd/historical-scylla-consumer/main.go	`0.00% <0.00%> (ø)`
sei-db/state_db/ss/offload/consumer/kafka.go	`44.82% <44.82%> (ø)`
sei-db/state_db/ss/offload/consumer/config.go	`31.25% <31.25%> (ø)`
sei-db/state_db/ss/offload/historical/store.go	`53.76% <53.76%> (ø)`
... and 5 more

... and 7 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

cursor

Cursor Bugbot has reviewed your changes and found 3 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit eff7ead. Configure here.}

cursor · 2026-05-20T22:58:22Z

+	if s.cache == nil || value == nil || len(value) > maxHistoricalReadCacheValueBytes {
+		return
+	}
+	s.cache.Add(key, historicalReadCacheValue{value: bytes.Clone(value), found: true, valueKnown: true})


Cache skips nil values causing repeated reader calls

Low Severity

cacheValue bails out when value == nil, so legitimate empty-value keys (non-deleted state with zero-length data) are never cached. Every subsequent Get for such a key bypasses the cache and hits the remote historical reader again. The nil check conflates "no value to cache" with "value is the empty byte slice," which are distinct states in MVCC stores.

^{Reviewed by Cursor Bugbot for commit eff7ead. Configure here.}

cursor · 2026-05-20T22:58:22Z

+func scyllaHistoricalOffloadConfigured(cfg config.StateStoreConfig) bool {
+	return strings.TrimSpace(cfg.HistoricalOffloadScyllaHosts) != "" ||
+		strings.TrimSpace(cfg.HistoricalOffloadScyllaKeyspace) != ""
+}


Scylla offload detection too loose with OR check

Medium Severity

scyllaHistoricalOffloadConfigured returns true when either HistoricalOffloadScyllaHosts or HistoricalOffloadScyllaKeyspace is set. Setting only the keyspace (e.g., from a leftover example config) without hosts causes the node to fail at startup because NewScyllaReader validates that hosts are required. Worse, it also falsely conflicts with an enabled FoundationDB backend, producing a confusing "only one historical offload fallback" error. The FoundationDB check is stricter (single Enabled bool), creating an asymmetry.

^{Reviewed by Cursor Bugbot for commit eff7ead. Configure here.}

cursor · 2026-05-20T22:58:22Z

+		fdb.ApplyDefaults()
+		if err := fdb.Validate(); err != nil {
+			return fmt.Errorf("foundationdb: %w", err)
+		}


Consumer validates FoundationDB config on discarded copy

Low Severity

Validate() copies c.FoundationDB into a local variable, calls ApplyDefaults() on the copy, then validates the copy. The original c.FoundationDB remains unchanged (zero-valued fields intact). The Scylla path handles this differently — c.Scylla.Validate() internally creates its own copy. This asymmetry means post-Validate() inspection of c.FoundationDB sees un-defaulted values, which is fragile for any future code that reads the config after validation.

^{Reviewed by Cursor Bugbot for commit eff7ead. Configure here.}

Kbhat1 added 14 commits May 11, 2026 14:05

Add Scylla historical state offload prototype

e04092d

Avoid retry timer leak in Scylla consumer

32b55d2

Parallelize Scylla mutation ingest

6c6f589

Parallelize Scylla historical batch reads

4fcd242

Cache Scylla historical point reads

0fa1aef

Enable token-aware Scylla routing

5ef70ba

Compact duplicate Scylla mutation writes

37d0086

Pipeline Scylla batch row writes

4a14131

Add FoundationDB historical offload backend

3e7a2a2

Optimize FoundationDB historical reads and writes

34194d7

Pipeline FoundationDB historical batch writes

ca336d6

Clean up FoundationDB batch writer context

b0bad4b

Cache historical fallback misses

3e0adc0

Make FoundationDB MVCC ops configurable

eff7ead

Kbhat1 marked this pull request as draft May 20, 2026 22:49

cursor Bot reviewed May 20, 2026

View reviewed changes

Kbhat1 added 3 commits May 21, 2026 12:11

Disable FDB read-your-writes for MVCC ops

d54bdaa

Deduplicate FoundationDB batch lookups

f807ca1

Clean up historical offload helpers

e5b03f6

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Prototype] FoundationDB#3481

[Prototype] FoundationDB#3481
Kbhat1 wants to merge 17 commits into
mainfrom
mvcc-foundationdb

Kbhat1 commented May 20, 2026

Uh oh!

cursor Bot commented May 20, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 20, 2026 •

edited

Loading

Uh oh!

codecov Bot commented May 20, 2026 •

edited

Loading

Uh oh!

cursor Bot left a comment

Uh oh!

cursor Bot May 20, 2026

Uh oh!

cursor Bot May 20, 2026

Uh oh!

cursor Bot May 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Kbhat1 commented May 20, 2026

Describe your changes and provide context

Testing performed to validate your change

Uh oh!

cursor Bot commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

github-actions Bot commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented May 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot May 20, 2026

Choose a reason for hiding this comment

Cache skips nil values causing repeated reader calls

Uh oh!

cursor Bot May 20, 2026

Choose a reason for hiding this comment

Scylla offload detection too loose with OR check

Uh oh!

cursor Bot May 20, 2026

Choose a reason for hiding this comment

Consumer validates FoundationDB config on discarded copy

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

cursor Bot commented May 20, 2026 •

edited

Loading

github-actions Bot commented May 20, 2026 •

edited

Loading

codecov Bot commented May 20, 2026 •

edited

Loading