Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .env
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Selects the run mode. Uncomment exactly one.
#
# The command is always a plain `docker compose up` — switching modes is this one line, so every
# other compose subcommand keeps working the way you expect.
COMPOSE_FILE=docker-compose.yaml:examples/postgres/compose.yaml
# COMPOSE_FILE=docker-compose.yaml:examples/mongodb/compose.yaml
# COMPOSE_FILE=docker-compose.yaml:examples/mysql/compose.yaml
# COMPOSE_FILE=docker-compose.yaml:examples/mssql/compose.yaml
# COMPOSE_FILE=docker-compose.yaml # Adopter Mode — your own database
#
# Append :docker-compose.dev.yaml to any of the above for the development loop, e.g.
# COMPOSE_FILE=docker-compose.yaml:examples/postgres/compose.yaml:docker-compose.dev.yaml

# DATABASE_TYPE is one of: postgres, mongodb, mysql, mssql
#
# Careful: PowerSync's own replication config in config/service.yaml spells the Postgres
# connector `postgresql`. This variable is read by the write API, which spells it `postgres`.
# Get it wrong and the backend refuses to start and tells you the supported values.
#
# DATABASE_URI shapes, one per type:
# postgres postgres://user:password@host:5432/database
# mongodb mongodb://user:password@host:27017/database
# mysql mysql://user:password@host:3306/database
# mssql mssql://user:password@host:1433/database
#
# A database running on this machine rather than in Docker is reachable at host.docker.internal,
# not localhost — inside a container, localhost is the container.

# Adopter Mode only. Example Mode overrides both of these with the bundled database.
# DATABASE_TYPE=postgres
# DATABASE_URI=

# ---------------------------------------------------------------------------------------------
# THROWAWAY SIGNING KEYS — PUBLIC, COMMITTED, AND KNOWN TO EVERYONE WHO HAS CLONED THIS REPO.
#
# They are here so the backend signs with a stable key across restarts. Without that, every
# restart mints a new key and PowerSync rejects tokens it just accepted with
# "PSYNC_S2101 — Could not find an appropriate key in the keystore".
#
# Replace them before this is anything other than a demo: `cd backend && pnpm generate-keys`.
# ---------------------------------------------------------------------------------------------
POWERSYNC_PRIVATE_KEY=eyJrdHkiOiJSU0EiLCJuIjoiNnJPRTVEdWRnbm5TYVh0NlZoVDNUQkpzRzNRdFVFQjA2N3dNZ29ocWstc3pvN3J3bVhOdTctdVJ4REM1OGdHYjBTVTM5R0tmNjVXMTlyVDhBSGVYNXg5bS1IbFUzZTdZMnYtanJLOW5OaGxlTUpfY3NzdElKdFJaaDlsYUJVb2ExejBtRVkzaHBMei1oRU1HNG5XQ3dZV1Q3SzZsR3lVYTFHOTRRSnYxaG5kZVkydmlGMWpMRHVERE1EckhwWkM4NEo3TWgxMkFpVXBUTkwxcWY0ckNoWEE2YXBWOW5DWmRzMERqcWhGUmFEWmhRUkR1MEJVZ1FRSFhkQWQ5X3hrX1JxYi1DQVVHX2tsSmVSRXlWU2p1LUV4M2tJWWlGMjFhSFNOWDFRN3c2amlKUWFnN25Bc2NmelEySVEwSFp0aGQ0Q3lTUzhHbE1RMFkwUFIyM09IYjBRIiwiZSI6IkFRQUIiLCJkIjoiSlFCQVJ5c08zZThPdVFwNmN1X0RQUDc2aENtQXExSS1ISnY4N09kTXhoMGlld0dSeE45cDhmRVZmZlNnbkFLYzZoQVFEanN1TXhuYkloWE9WTlNGNGk0Vk1iOFBIaDMxbWpFTFFNSTJaMVVBZ0hIemZVeUhCM2dhMVV2eTRUcVptSzFQUHgwN0labWFGb2Zxb2ZFY3VCMnpBSEZZSGp0dlMyWjNjdGdqa1JzYlBwbVloUWFoNUJWTFhlSmFEbTJHcTM3blBHZFRfa1pPN01KeVNRckVMTmxOMDZMNmJfMWk5RHRHc2d2WG5ZRmhWOUVPXzBpUk9BOTFDcTJCZzRHMHkzUF8yX0ZQMjRZNW9tQ2NDWGVTR0xrVmZsMVZBWmlhaXltTHVva05UZ3lnVnpYZHdYMnFKeGIwbk9kMmJCUHBvSm15Mnk2dmUwVGpoUzNOV2dCVzhRIiwicCI6Ii1LVWhXQW1RY2FxNFp4cEhWYnRlVmloREV4OU1TZm5kM0Q2TmZMUmYxMy1uZ192VE1JdHJkM2gyRjBuMjVTNmpLckhLQmlJUkNaOW9zcFdfSElCR1BwcGxYalIyaWI5R3BSdkdHeG4tMm5IM3pzcVZ3MjA1ODVqRWFkSElLa3hscnlIRks4b2xlSVc1ZUpzN0VkOU9ZOWxvS21LenU5emFyRUx4ZWpVLUFzOCIsInEiOiI4YVRNbDhQZUxoeTRSOG5rTHY4ZWpWajUxZDQ5elhiaS1GS2ZXZ2l3U2hRRTU2cXpTTHJuMG5kOXQ3cFVRdlpCUnk0d2NMLXJFMWhXNlZtRERJaU9hVTNKQm1mLVk0NVljVFYtcXNENDFXSjVMUC1fdVBydmFHZnNsYlJFVFZRaks3SWVOTTJfc0wyd1hhX3dtY1B3SGNyU2w0OXJ6U0lqTVBCTHE2X3NYMTgiLCJkcCI6InZxR3NETmxZV3kxdXItMmYzNFVGOEx4eG9JbVFiZThhUUNfZFBrejBaajVDNnBmNTlQQVBkc3R1anJCd2tJblBJMzZueTBmM0ZBLVpyOEhMZ2toLWtxVEJMeGEtQXlJWlFhRW5vOE9zZDBLRm9aQUVmbzZScmNma1h0VXR4X0JHelp6d2xJQXBkbHZnTlMyZWZqZGMzSVRrcmdwNmpuX25UOGNMYUl6RmZGVSIsImRxIjoiRXVzdkpYYXRUM2pxS0p5eTQ4Y1Bra3QwQ18yQll6TzZvMng4azJUNHdHUC0ybEJ3QnZLek1iUXZRSkl6QktjWkIwU2pnRUJSV1l0aUNwVDZnS0cwWEtROVotWC1jYmIwVDdDN2dRem9yblF1UG9xcmJRVWdkMUVqb2JqaVhCZUpSV09Gbi1hMzZsTl9tbVlxOVM3MF9yQWhlc0k4MDJ1bnk2NFVqcFdRY1FzIiwicWkiOiJmeV9GT2c4QzFvZG01c1lOejJkSEN4X0JtODdxYXNueG5WUXZYWWlZeXFHSlZrVy1ybkYtVHI4dl95UDZnWTBLNkFwQ1VrOV9yQlVVUEppU2pVcVJhMk0zaUlCQm9LUXVHM0lXQXRKOHc2RThRMVg1Z19kNlRXTnhxYmxRVlY4My11Szhtd25PNUpETzM3NXFCeVRaZkpWUUExSHVHV2ZEUld0cnlNSlNOdG8iLCJhbGciOiJSUzI1NiIsImtpZCI6InBvd2Vyc3luYy0yMTIyNjM5NzY1In0=
POWERSYNC_PUBLIC_KEY=eyJrdHkiOiJSU0EiLCJuIjoiNnJPRTVEdWRnbm5TYVh0NlZoVDNUQkpzRzNRdFVFQjA2N3dNZ29ocWstc3pvN3J3bVhOdTctdVJ4REM1OGdHYjBTVTM5R0tmNjVXMTlyVDhBSGVYNXg5bS1IbFUzZTdZMnYtanJLOW5OaGxlTUpfY3NzdElKdFJaaDlsYUJVb2ExejBtRVkzaHBMei1oRU1HNG5XQ3dZV1Q3SzZsR3lVYTFHOTRRSnYxaG5kZVkydmlGMWpMRHVERE1EckhwWkM4NEo3TWgxMkFpVXBUTkwxcWY0ckNoWEE2YXBWOW5DWmRzMERqcWhGUmFEWmhRUkR1MEJVZ1FRSFhkQWQ5X3hrX1JxYi1DQVVHX2tsSmVSRXlWU2p1LUV4M2tJWWlGMjFhSFNOWDFRN3c2amlKUWFnN25Bc2NmelEySVEwSFp0aGQ0Q3lTUzhHbE1RMFkwUFIyM09IYjBRIiwiZSI6IkFRQUIiLCJhbGciOiJSUzI1NiIsImtpZCI6InBvd2Vyc3luYy0yMTIyNjM5NzY1In0=
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,6 @@ docs/agents/

# Per-repo agent instructions
CLAUDE.md

# Dependencies. Lockfiles stay tracked — the Dockerfiles install with --frozen-lockfile.
node_modules/
180 changes: 150 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,49 +1,169 @@
# PowerSync Write API Demo
# PowerSync Write API

![Architecture diagram](./diagram.png)
A self-hostable backend for the PowerSync write path: a client uploads its queued local changes to
an HTTP API, which persists them to your source database. PowerSync replicates that database back
to clients.

## Project Layout
Clone it, point it at your own database, and change the code.

## Quickstart — see it work

This brings up a complete, self-contained system with **nothing for you to configure**: a seeded
Postgres, the PowerSync service, the write API, and a small demo client. The committed `.env`
already holds everything it needs, including a throwaway signing keypair.

```bash
docker compose up --build
```

- Demo client: http://localhost:5173
- Write API: http://localhost:6060
- PowerSync: http://localhost:8080
- Example Postgres: localhost:5432

Open the demo client, add a todo, and watch it land in Postgres and sync back.

**This is a smoke test, not the product.** The todo schema, the seed data and the demo client are
scaffolding to prove the machinery works before you wire in anything of your own. Everything
specific to it lives under `examples/` and can be deleted in one go.

## Point it at your own database

Edit `.env` and select Adopter Mode:

```bash
COMPOSE_FILE=docker-compose.yaml
DATABASE_TYPE=postgres
DATABASE_URI=postgres://user:password@your-host:5432/your-db
```

### Your database needs preparing first

PowerSync replicates by reading your database's change feed, and every flavour needs that turned
on before anything syncs. This is the part that silently produces an empty app if skipped.

| Flavour | What must be true of your database |
| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Postgres** | a publication named `powersync` covering the replicated tables; a user with `SELECT` on them and replication rights. **A Postgres source without a publication replicates nothing.** |
| **MongoDB** | A replica set — change streams and the multi-document transactions the write API uses both require one. Post-images configured (`post_images: auto_configure`), since change streams alone do not carry the pre-update document. |
| **MySQL** | `log_bin` on, `gtid_mode=ON`, `enforce_gtid_consistency=ON`, `binlog_format=ROW`, `binlog_row_image=FULL`, a unique `server-id`; a user with `REPLICATION SLAVE` and `SELECT`. On managed MySQL these usually live in a parameter group and need a restart. |
| **SQL Server** | CDC enabled at database level and per replicated table; a CDC-enabled `_powersync_checkpoints` table; SQL Server Agent **running**, or CDC captures nothing while appearing enabled; the user needs `cdc_reader`, `VIEW DATABASE PERFORMANCE STATE` in the database, and `VIEW SERVER PERFORMANCE STATE` in `master`. |

Each `examples/<flavour>/README.md` has the worked SQL and the managed-hosting wrinkles. Those
live under `examples/`, which you are invited to delete — so the table above is the version that
survives that, deliberately.

Then describe your own schema in `config/service.yaml` and `config/sync-config.yaml`. Those two
files are yours from the first minute — no example ever writes to them.

**Fill in `config/sync-config.yaml` before you start.** It ships empty, because only you know your
schema, and PowerSync will restart in a loop logging `'streams' are required` until it has at
least one stream. Remember `auto_subscribe: true` — without it a stream syncs nothing and reports
no error anywhere.

If your database runs on this machine rather than in Docker, reach it at `host.docker.internal`,
not `localhost` — inside a container, `localhost` is the container.

In Adopter Mode there is no bundled database and no demo client. Bring your own client.

> Bucket storage — PowerSync's own internal store — always runs in a container this project owns,
> in every mode. We never create schemas in a database you merely pointed us at.

## Switching modes

Mode selection is the `COMPOSE_FILE` line in `.env`, with the alternatives sitting there commented
out. The command stays a plain `docker compose up`, so `down`, `logs` and `ps` behave normally.

| `.env` line | What runs |
| ---------------------------------------------------- | ------------------------------------------------------------- |
| `docker-compose.yaml:examples/postgres/compose.yaml` | Example Mode, [Postgres](./examples/postgres/README.md) |
| `docker-compose.yaml:examples/mongodb/compose.yaml` | Example Mode, [MongoDB](./examples/mongodb/README.md) |
| `docker-compose.yaml:examples/mysql/compose.yaml` | Example Mode, [MySQL](./examples/mysql/README.md) (Beta) |
| `docker-compose.yaml:examples/mssql/compose.yaml` | Example Mode, [SQL Server](./examples/mssql/README.md) (Beta) |
| `docker-compose.yaml` | Adopter Mode, your database |

Only one runs at a time — they share ports, and each has its own Compose project name so switching
never reuses the previous flavour's volumes.

**Bring the current mode down before switching.** Because each mode is its own Compose project,
`docker compose down` only stops the mode currently selected in `.env`. Edit the line first and the
old containers keep running and holding ports, and the new mode fails with
`Bind for 0.0.0.0:6060 failed: port is already allocated`. Down first, then switch.

If you would rather be explicit, the same thing without `.env`:

```bash
docker compose -f docker-compose.yaml -f examples/postgres/compose.yaml up
```

## Layout

```
write-api/
├── docker-compose.yaml # Postgres + MongoDB + PowerSync
├── config/
│ ├── powersync.yaml # PowerSync service config
│ └── sync_rules.yaml # Sync rules (lists + todos)
├── init-scripts/
│ └── setup.sql # DB schema + seed data
├── powersync-nodejs-backend-todolist-demo/ # Backend (Express, port 6060)
│ └── .env
└── demo-app/ # Frontend (React/Vite, port 5173)
└── .env.local
├── docker-compose.yaml # Base: PowerSync, bucket storage, write API
├── .env # Mode selection and throwaway dev keys
├── config/ # ADOPTER MODE config — yours to edit
│ ├── service.yaml
│ └── sync-config.yaml
├── docker-compose.dev.yaml # Overlay: edit backend code without rebuilding
├── examples/ # Delete this when you no longer need it
│ ├── postgres/ # Seeded Postgres + demo client
│ │ ├── compose.yaml
│ │ ├── powersync/ # This example's PowerSync config
│ │ └── init-scripts/ # Demo schema + seed data
│ ├── mongodb/ # No source container — shares the bucket-storage replica set
│ ├── mysql/ # Binlog config + seeded schema
│ └── mssql/ # CDC bootstrap container + seeded schema
├── backend/ # The write API (Express, port 6060)
│ └── openapi.yaml # Shared contract, read by both packages
└── frontend/ # Demo client (React/Vite) — a test fixture
```

## Running Everything
## Changing the backend

Append the development overlay to whichever mode you are in:

```bash
# 1. Start infrastructure (Postgres, Mongo, PowerSync)
docker compose up -d
COMPOSE_FILE=docker-compose.yaml:examples/postgres/compose.yaml:docker-compose.dev.yaml
```

Your working tree is mounted into the container and the process restarts on save — an edit is
serving in about two seconds, with no image rebuild. It works in Adopter Mode too, which is
arguably where it matters more: wiring this into your own database is exactly when you are editing
`src/persistance/` and `src/auth/verifier.ts`.

# 2. Start backend (new terminal)
cd backend && pnpm install && pnpm start
Without the overlay, changes ship on rebuild — the deployment-shaped path:

# 3. Start frontend (new terminal)
cd frontend && pnpm install && pnpm dev
```bash
docker compose up --build
```

- Frontend: http://localhost:5173
- Backend: http://localhost:6060
- PowerSync: http://localhost:8080
- Postgres: localhost:5432
The demo client is a Vite app, so its own loop is the usual one, on the host:

```bash
cd frontend && pnpm dev
```

That reads `.env.local` at runtime, so changing a URL needs no rebuild. In the container the client
is a production build with its URLs baked in, which is why it is not part of the overlay.

## Generating Types from OpenAPI Spec
Auth seams worth knowing: `backend/src/auth/verifier.ts` is where you swap the demo's tokens for
your own identity provider — see [auth-verifiers.md](./auth-verifiers.md) for worked examples.

Both the backend and frontend generate TypeScript types from the shared `openapi.yaml` spec.
Replacing the throwaway signing keys is one command:

```bash
# Backend (generates src/generated/api.ts)
cd backend && pnpm generate-types
cd backend && pnpm generate-keys # prints both values for .env
```

> The signing keys in `.env` are a **public throwaway pair**, committed so the backend signs
> consistently across restarts. Replace them before this is anything but a demo.

## Generating types from the contract

# Frontend (generates src/generated/api.d.ts)
cd frontend && pnpm generate
Both packages generate TypeScript from `backend/openapi.yaml`:

```bash
cd backend && pnpm generate-types # -> src/generated/api.ts
cd frontend && pnpm generate # -> src/generated/api.d.ts
```
17 changes: 17 additions & 0 deletions backend/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Without this, `COPY / ./` overwrites the node_modules installed in the image with the host's —
# macOS-native binaries landing in a Linux container.
node_modules

# Secrets and local state must never reach the image.
.env
.envrc

# Build and run artefacts.
*.tsbuildinfo
*.log
dist

# Not needed at runtime.
Dockerfile
.dockerignore
**/*.test.ts
28 changes: 22 additions & 6 deletions backend/.env.template
Original file line number Diff line number Diff line change
@@ -1,8 +1,24 @@
# You probably do not need this file.
#
# The backend runs in Docker Compose, where these come from the root .env and the example
# overlays. See the root README for the two run modes.
#
# It is here for the unusual case of running the backend directly on the host with `pnpm start` —
# note that this collides with the containerised backend on port 6060, so stop that first.

# One of: postgres, mongodb, mysql, mssql
DATABASE_TYPE=postgres
# Required. The backend refuses to start without it.
DATABASE_URI=

PORT=6060

# Audience and issuer for the tokens this backend mints.
POWERSYNC_URL=powersync-dev
JWT_ISSUER=powersync-dev

# Base64-encoded JWKs. Leave blank and a temporary pair is generated at boot — convenient for a
# one-off run, but every restart invalidates tokens PowerSync has already accepted. Generate a
# stable pair with `pnpm generate-keys`.
POWERSYNC_PRIVATE_KEY=
POWERSYNC_PUBLIC_KEY=
POWERSYNC_URL=
PORT=
JWT_ISSUER=
# Either 'mongodb', 'mysql', 'mssql' or 'postgres'. This defaults to Postgres
DATABASE_TYPE=
DATABASE_URI=
Loading