[almagas]

Description

Resolves #72

This PR introduces the Replication Plugin (starbasedb:replication) for StarbaseDB, providing pull-based data synchronization from external data sources (e.g. Supabase/PostgreSQL, MySQL) directly into the internal Durable Object SQLite instance. This enables StarbaseDB instances to act as high-speed, close-to-the-edge read replicas of central databases.

---

Key Capabilities

1. Incremental Cursor-Based Pull:

   • Supports tracking progress with cursorColumn (e.g., id, updated_at, timestamp).
   • Syncs only new or modified records (WHERE cursorColumn > previousCursor ORDER BY cursorColumn ASC LIMIT batchSize).
   • Supports forceFull mode to bypass cursors when a full re-sync is needed.

2. Idempotent Upserts:

   • Applies INSERT OR REPLACE INTO keyed on primaryKey (defaults to id), preventing duplicates and handling updates seamlessly.

3. Schema-Aware Auto Table Provisioning:

   • Automatically provisions destination tables in SQLite on first sync with proper type mapping (INTEGER, REAL, TEXT) and PRIMARY KEY.
   • Handles schema-qualified names (e.g. public.users replicates cleanly into target table users).
   • Escapes SQL identifiers to support reserved keywords and special characters.

4. State Persistence & Observability:

   • Tracks replication metadata in SQLite table _starbase_replication_state:
     • table_name: Primary key
     • cursor_value: Last processed cursor
     • last_sync_at: Epoch timestamp of last successful sync
     • total_rows_synced: Cumulative sync counter
     • status: 'idle' | 'syncing' | 'synced' | 'error'
     • last_error: Diagnostic message on failures

5. Concurrency Lock Guard:

   • In-flight table lock prevents overlapping sync executions on the same table.

6. REST API Interface (/replication/*):

   • GET /replication/status: Reports sync states, table count, and active synchronizations.
   • POST /replication/sync: Triggers immediate synchronization for a single table or all configured tables.
   • POST /replication/configure: Dynamically registers new table configurations at runtime.
   • POST /replication/reset: Resets replication state and cursors.

7. Scheduled Alarm / Cron Integration:

   • Exposes syncAll() and handleCron() for invocation via Cloudflare Worker scheduled events or CronPlugin.

---

Testing & Verification

• Added comprehensive Vitest suite in plugins/replication/index.test.ts (17 tests covering initialization, REST routes, incremental cursors, lock guards, error states, and cron handling).
• All 172 tests in the repository pass with 0 failures:

  Test Files  20 passed (20)
       Tests  172 passed (172)
  

---

/claim #72