Repository navigation
feat(plugins): add ReplicationPlugin to replicate external data into internal SQLite - #346
Open
khuxaiphi-stack wants to merge 5 commits into
Open
khuxaiphi-stack wants to merge 5 commits into
khuxaiphi-stack wants to merge 5 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
/claim #72
Fixes #72
Summary
Implements
ReplicationPlugin(plugins/replication) to replicate tables and rows from an external database source (PostgreSQL, MySQL, Turso, Cloudflare D1, or StarbaseDB) into the internal StarbaseDB Durable Object SQLite database.Key Changes
plugins/replication/index.ts:POST /replication/sync: Admin-authenticated endpoint to trigger on-demand replication across configured or explicitly specified tables.GET /replication/status: Admin-authenticated endpoint returning replication status and per-table metadata (table_name,rows_synced,last_synced_at,status) stored in_starbase_replication_meta.beforeQueryAuto-Sync: Optional periodic background replication whenautoSyncOnQueryis enabled andintervalSecondshas elapsed.tablesis omitted, infers SQLite column types (INTEGER,REAL,BLOB,TEXT), and creates local tables.batchSizechunks viaexecuteTransaction(with fallback toexecuteQuery) supporting bothreplace(INSERT OR REPLACE) andignore(INSERT OR IGNORE) conflict strategies./^[a-zA-Z_][a-zA-Z0-9_]*$/before constructing DDL/DML queries.plugins/replication/index.test.ts: 7 comprehensive Vitest unit tests covering initialization, batched transaction sync, empty table handling, SQL identifier validation, admin route authorization, andbeforeQueryauto-sync.plugins/replication/meta.json&plugins/replication/README.md: Plugin metadata and usage documentation.dist/plugins.ts: ExportsReplicationPlugin.Verification
npx vitest run plugins/replication/index.test.ts— 7 / 7 tests passingnpx tsc --noEmit— 0 TypeScript errorsSolana Payout Wallet:
FyexYHp1TZ6uzFLUvFNfofx14LQaBrYHbQksTo18xRgD