Skip to content

Reference backend (PR 1,5,7). - #10

Merged
Chriztiaan merged 13 commits into
mainfrom
feat/base
Sep 17, 2026
Merged

Chriztiaan merged 13 commits into
mainfrom
feat/base

Conversation

@Chriztiaan

Copy link
Copy Markdown
Collaborator

Write API PoC: foundation, auth and transaction batching

feat/base merges PRs #1, #5 and #7 into a single branch against main, plus the integration work that only became visible once the three sat together.

The result is a runnable demo end to end: a React/Vite todo app, an Express backend, an OpenAPI contract both sides generate types from, and persisters for Postgres, MySQL, MSSQL and MongoDB, with docker-compose bringing up Postgres, MongoDB and the PowerSync service.

The three focus areas

A template write path (#1). Moves the translation problem from the client to the server edge. Instead of every developer writing a bespoke uploadData that fetches CRUD ops, maps types, talks to a backend and classifies errors, the client sends raw CRUD data over the wire and a standard server-side stack handles it from there: a generated endpoint handler, a mapping layer from SQLite CRUD data to the source database's shape, and a persister per supported database. Defining the wire format in openapi.yaml means the handler and client types can be regenerated rather than reimplemented, including for other languages.

Auth on the write path (#5). The write endpoint is gated behind a bearer JWT: the backend verifies a token on every write, the frontend attaches one to every upload. Verification sits behind a single TokenVerifier interface (backend/src/auth/types.ts), so an adopter swaps one file to accept tokens from Supabase, Clerk or anything else. The demo reuses the same PowerSync token the client already fetches in fetchCredentials() rather than minting a second one, and verifies it against the backend's own public key. auth-verifiers.md walks through the swap for Supabase and Clerk.

Transaction batching (#7). The client previously uploaded one transaction per attempt, so a queue built up offline drained one round-trip at a time. The write endpoint now takes an ordered run of whole transactions from the head of the upload queue and applies each one in its own database transaction. A transaction is never split across batches. The response holds one result per transaction sent, in the same order and always the same length as the request, so the client never has to infer which transactions were applied. Transactions the batch never reached come back as not_attempted rather than being omitted. The client then completes once, at the last result that is success or fatal_error; completing a transaction also completes everything before it, so a retry resumes from the failure instead of re-uploading transactions that already committed.

For a batch to know where to stop, every failure is sorted into retryable (connection loss, deadlock, resource exhaustion: will upload again after a delay) or fatal (missing required field, constraint violation, malformed or out-of-range value, schema mismatch: the data can never be stored). Classification is per engine, in backend/src/persistance/<engine>/<engine>-errors.ts.

on_fatal_error controls what a fatal failure does to the rest of the batch. stop (the default) ends the batch there and reports everything after it as not_attempted. skip drops the failing transaction and carries on, so a queue blocked by a poison operation can still drain; its result still reports fatal_error with the classification, so the client can record what it discarded, or divert it to a dead-letter queue instead of throwing it away. skip covers fatal failures only. A retryable failure always ends the batch.

AI disclosure

This PR was created with the help of Claude Code. Help constitutes assistance in research, planning, and rough outline of implementation. Beyond having a hand in the implementation, I have also manually tested this work.

Chriztiaan added 13 commits May 4, 2026 15:26
# Conflicts:
#	backend/src/api/data.ts
Three things the merge of #5 and #7 got textually clean but semantically wrong,
plus the regeneration the spec fix forces.

- openapi.yaml: /api/data/batch now declares `security: bearerAuth` and a 401
  response. #5 mounts requireAuth at router.use('/data', ...), which already
  covers the sub-path, so the spec was contradicting the implementation.

- OpenAPITransport: 401 recovery moves from postTransaction into an onResponse
  middleware. It was written per-method in #5, so the batch endpoint #7 added
  threw on 401 without ever clearing the cached token — wedging the batch path
  until a reload. The bearer header was already attached centrally; the
  rejection now is too, so endpoints added later inherit both.

- data.ts: the /batch handler logs the verified writer, matching what #5 does
  for single-transaction writes.

Regenerated types follow from the spec change.
CONTEXT.md had no auth terms at all. Adds Write Token, Verifier and Writer,
and notes under Transaction Batch that the /api/data/batch endpoint name
predates the batch / transaction batch distinction the glossary draws.
POST /api/data now takes a transaction batch and nothing else. The
single-transaction endpoint is gone; a client with one transaction to upload
sends a batch of one.

The win is deleting a response shape, not a route. Two endpoints sharing one
TransactionResponse schema had already leaked into the client as an apology:

    `not_attempted` is only ever returned for an entry in a batch result — the
    single-transaction endpoint never emits it. It is in the union because both
    endpoints share one response schema.

That comment is now untrue, so it is gone. So is the only behavioural difference
between the two endpoints: `message: 'Transaction completed'` on success. A
successful transaction carries a bare status; `message` is for failures.

The backend was already one implementation — applyTransaction() did the work and
/api/data was a wrapper that re-added the success string. On the client,
uploadSingleTransaction and uploadTransactionBatch become one uploadData, and
WriteAPIClient loses processTransaction, sendSingle and the create/update/delete
convenience methods, which had no callers anywhere in the repo.

Batching stops being a mode. BatchingConfig is no longer nullable and defaults to
10 transactions / 1000 operations; the env vars only tune how much of the queue
goes per request. Set VITE_BATCH_MAX_TRANSACTIONS=1 for one transaction per
round-trip. Leaving the default at 1 would have collapsed the contract while
leaving the drain-one-at-a-time behaviour that batching existed to fix.

CONTEXT.md: Batch moves to Out of scope beside Mutator — it names something the
system deliberately never does, and keeping it is what stops operation-splitting
being reintroduced by accident. Transaction Batch is now the only unit of upload,
and the note about the /batch endpoint name is obsolete.

Docs swept: backend/README.md, auth-verifiers.md (including the curl examples),
frontend/.env.template, and the data-flow diagram's labels.
@Chriztiaan
Chriztiaan marked this pull request as ready for review September 17, 2026 13:49
@Chriztiaan
Chriztiaan merged commit 5e24d89 into main Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants