Skip to content

Rebuild the PostgreSQL query store from JSON source data #149

Description

@atomantic

Port the derived-store write path so Layer 1 JSON data can populate PostgreSQL directly. Rebuilding from JSON is the migration strategy; do not copy rows from the legacy SQLite file.

Depends on #148

Scope:

  • Introduce a PostgreSQL writer with the batching/upsert semantics currently provided by server/src/lib/sqlite-writer.ts.
  • Run related inserts in PostgreSQL transactions and preserve ULID identity, source provenance, membership, media, claim, and relationship constraints.
  • Update the indexing/rebuild entry points so an explicitly configured PostgreSQL store can be rebuilt from data/person/*.json while SQLite remains available during the staged migration.
  • Add end-to-end fixture coverage that rebuilds a small tree and validates row counts, relationships, idempotency, and rollback on failure.

Acceptance criteria:

  • A clean PostgreSQL store can be rebuilt solely from JSON source files.
  • Re-running the rebuild is idempotent: no duplicate persons, edges, memberships, identities, claims, or FTS rows.
  • A failed batch leaves no partially committed person graph.
  • Existing JSON files remain unchanged and authoritative.

Files/areas: server/src/lib/sqlite-writer.ts or a new postgres-writer module, scripts/index.ts, scripts/rebuild.ts, transformation helpers, integration fixtures/tests.

Part of #120

After this issue ships, check its box in #120. If it was the last open child, comment on #120 with a summary of the delivered children and close #120 as completed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    effort:maxMaximum reasoning effort recommendedmodel:heavyBest suited to the strongest available coding modelplanClaimable backlog item

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions