Skip to content

add state snapshot uploads alongside log uploads#403

Open
snowp wants to merge 17 commits intomainfrom
state-uploads2-rebased
Open

add state snapshot uploads alongside log uploads#403
snowp wants to merge 17 commits intomainfrom
state-uploads2-rebased

Conversation

@snowp
Copy link
Contributor

@snowp snowp commented Feb 25, 2026

Implements state uploads by introducing a new StateUploadWorker that is notified whenever a log batch is uploaded and attempts to schedule the upload of any state snapshots that are necessary in order to provide the state updates required for hydrating the uploaded logs.

This is a generic mechanism that ends up working for both trigger buffer uploads as well as continuous uploads:

  • for trigger buffers the snapshots are kept on disk subject to the retention policy given by the data contained in the buffers, while uploads are scheduled whenever we observe logs being uploaded
  • for continuous buffers we rely on the upload signals to notify the worker about logs that are being uploaded, which may require periodic snapshot rotation in order to prepare the snapshots for upload

…e log uploads

Adds state snapshot upload capability to bd-logger:
- StateLogCorrelator tracks state changes and coordinates snapshot uploads
- State snapshots are uploaded before log uploads to ensure consistency
- Integrates with retention registry to manage snapshot lifecycle
- Adds skip_intent support and completion tracking to artifact uploader
@snowp snowp force-pushed the state-uploads2-rebased branch from 9d651d9 to be80130 Compare February 25, 2026 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant