-
Notifications
You must be signed in to change notification settings - Fork 0
feat: external block provider for genesis sync #26
Copy link
Copy link
Open
Description
Summary
Nodes running in pruning mode (default) discard old blocks to bound disk growth. To support syncing new nodes from genesis without requiring every node to be a full archive node, we need an external block serving service.
Motivation
- Block archive storage now prunes by default (retention window of ~100k blocks)
- Chain-index (SQLite) serves RPC/explorer queries independently of the block archive
- Only seed/archive nodes need full block history
- New nodes joining the network need a way to sync from genesis without requiring every peer to store all blocks
Design
An external block provider service that:
- Stores all blocks from genesis (potentially using commonware's
immutable::Archivewith freezer for minimal memory + compression) - Serves block data over a well-defined API (HTTP/gRPC) to syncing nodes
- Can be operated independently of the P2P network -- e.g., a centralized service, cloud storage, or a dedicated archive node
Sync flow
New Node Block Provider
| |
|-- request blocks [0, N) --->|
|<--- stream ArchivedBlock ---|
| |
|-- apply blocks to STF --- |
|-- build chain-index --- |
| |
|-- join P2P, continue from |
| latest block |
Considerations
- The provider does not need to be trusted for correctness -- the node validates state roots against the chain's state transition function
- Provider could be backed by object storage (S3/R2) serving pre-compressed block segments
- Protocol should support range requests and resumable sync
- Consider using commonware's
immutable::Archive(freezer) as the backing store for the provider -- it's designed for exactly this workload (write-once, read-many, minimal memory, compressed)
Acceptance Criteria
- Define block provider API (gRPC service definition or HTTP endpoints)
- Implement block provider server backed by full block archive
- Implement client-side sync that fetches blocks from provider and applies them
- Support resumable sync (track last synced block)
- Node config option to specify block provider URL for initial sync
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels