Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
name: Test

on:
push:
branches:
- main
pull_request:

jobs:
backend:
name: Backend and connector tests
runs-on: ubuntu-latest
defaults:
run:
working-directory: backend
steps:
- uses: actions/checkout@v4

- uses: pnpm/action-setup@v4
with:
package_json_file: backend/package.json

- uses: actions/setup-node@v4
with:
node-version: 22
cache: pnpm
cache-dependency-path: backend/pnpm-lock.yaml

- run: pnpm install --frozen-lockfile
- run: pnpm check
- run: pnpm test

example-client:
name: Type-check connectors against the PowerSync SDK
runs-on: ubuntu-latest
defaults:
run:
working-directory: example-client
steps:
- uses: actions/checkout@v4

- uses: pnpm/action-setup@v4
with:
package_json_file: example-client/package.json

- uses: actions/setup-node@v4
with:
node-version: 22
cache: pnpm
cache-dependency-path: example-client/pnpm-lock.yaml

- run: pnpm install --frozen-lockfile
- run: pnpm check

supabase:
name: Supabase Edge Function tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: supabase/setup-cli@v1
with:
version: 2.98.2

- uses: denoland/setup-deno@v2
with:
deno-version: v2.x

- name: Start local Supabase
working-directory: backend
run: supabase start

- name: Serve functions
working-directory: backend
run: |
supabase functions serve > "$RUNNER_TEMP/functions.log" 2>&1 &
for i in $(seq 1 60); do
grep -q "Serving functions" "$RUNNER_TEMP/functions.log" && exit 0
sleep 2
done
cat "$RUNNER_TEMP/functions.log"
exit 1

- name: Run tests
working-directory: backend/supabase/functions/powersync-write
run: deno task test
28 changes: 19 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# PowerSync Reference Write Implementation

When your PowerSync app writes data using SQL INSERT, UPDATE, or DELETE statements,
PowerSync automatically queues these mutations for processing using the `uploadData` function.
PowerSync automatically queues these mutations and passes them to the upload callback you give `connect()`.

While there are no restrictions on the implementation of `uploadData` and the related write API,
While there are no restrictions on the implementation of the upload callback and the related write API,
this repository provides a client connector and two implementations of the write API for most simple use cases.
It provides extension points that will allow you to customize
the behavior to your own requirements, or you can modify it to suit your specific case.

- [backend/shared/powersync-reference-write-api.openapi.yaml](backend/shared/powersync-reference-write-api.openapi.yaml) provides a simple but flexible API definition that serves as a contract between the client connector and either backend.
- A [client connector](example-client/src/PowersyncConnector.ts) implementing `uploadData` handles most common workflows. It will marshal queued changes to the format specified by the API definition. A [single-file client connector](example-client/src/PowersyncConnector.singlefile.ts) is provided for ease of copy-paste.
- A [client connector](example-client/src/PowersyncConnector.ts) providing the upload callback handles most common workflows. It will marshal queued changes to the format specified by the API definition. A [single-file client connector](example-client/src/PowersyncConnector.singlefile.ts) is provided for ease of copy-paste.
- The write API (`POST /api/data`) accepts queued transactions from the client connector and applies them to your source database. Choose the [standalone Express backend](backend/standalone/README.md), or, for Supabase apps, the [Supabase Edge Function](backend/supabase/README.md), which implements the same write API and enforces your existing RLS policies. Both use the [shared core](backend/shared), which contains the transaction processor and adapters for Postgres, MongoDB, MySQL, and SQL Server.

The green blocks are provided by this repository; click them to explore the code. Solid arrows
Expand All @@ -20,7 +20,7 @@ Choose one backend to serve the write API (`POST /api/data`).
flowchart TB
subgraph app["Your app"]
sdk["PowerSync SDK<br/>Local database and upload queue"]
connector["Client connector<br/>uploadData()"]
connector["Client connector<br/>upload"]
sdk -->|Queued local writes| connector
end

Expand Down Expand Up @@ -120,12 +120,22 @@ Startup checks the database type, connection string, fatal-error policy, and ver

### 4. Connect your app

Copy a client connector from [example-client](example-client/README.md) into your app. Configure
its write API URL, PowerSync instance URL, and token retrieval, then pass it to your existing
PowerSync database's `connect()` method. The client connector implements:
Copy a client connector from [example-client](example-client/README.md) into your app and set its
write API URL. Then pass its `upload` callback to `connect()`, together with an `Authenticator`:

- `fetchCredentials()`: credentials for the PowerSync sync connection. If your app is already reading data from PowerSync then you will have already implemented this function, we recommend that you keep your existing implementation and configure the Express backend to trust the same issuers - see [auth setup](docs/auth/setup.md).
- `uploadData()`: sends queued local transactions to `POST /api/data`.
```ts
const connector = new PowersyncConnector({ authenticator });
await db.connect({ powerSyncEndpoint, authenticator, upload: connector.upload });
```

- `authenticator`: supplies tokens for the PowerSync sync connection, and the connector uses it for
write requests too. If your app already syncs with PowerSync, keep your existing authentication and
configure the Express backend to trust the same issuers; see [auth setup](docs/auth/setup.md).
- `upload`: sends queued local transactions to `POST /api/data`.

This needs `@powersync/web` 2.6.0 or `@powersync/react-native` 2.4.0 or later. If your app connects
with a `PowerSyncBackendConnector`, see
[migrating from a backend connector](example-client/README.md#migrating-from-a-backend-connector).

Use a URL reachable from the client device. `localhost` refers to that device, so another
computer or phone needs the Express backend's network address or a tunnel URL.
Expand Down
74 changes: 70 additions & 4 deletions backend/shared/test/connector.test.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { beforeEach, describe, expect, it, vi } from 'vitest';
vi.mock('uuid', () => ({ v4: () => 'test-user' }));
vi.mock('../../../example-client/src/library/powersync/OpenAPITransport', () => ({
AuthenticationError: class extends Error {},
AuthenticationError: class AuthenticationError extends Error {},
createOpenAPIClient: () => ({ transport: {} })
}));
vi.mock('../../../example-client/src/library/powersync/DemoConnectorConfig', () => ({
Expand All @@ -20,16 +20,29 @@ const splitPath = '../../../example-client/src/PowersyncConnector.ts';
const singlePath = '../../../example-client/src/PowersyncConnector.singlefile.ts';
const { PowersyncConnector: Split } = await import(new URL(splitPath, import.meta.url).href);
const { PowersyncConnector: Single } = await import(new URL(singlePath, import.meta.url).href);
const { AuthenticationError } = await import(
new URL('../../../example-client/src/library/powersync/OpenAPITransport.ts', import.meta.url).href
);

function stubAuthenticator() {
return {
resolveCredentials: vi.fn(async () => 'token'),
invalidateCredentials: vi.fn()
};
}

beforeEach(() => vi.stubGlobal('localStorage', { getItem: () => 'user' }));
beforeEach(() => {
vi.unstubAllGlobals();
vi.stubGlobal('localStorage', { getItem: () => 'user' });
});

for (const [name, Connector] of [
['split', Split],
['single', Single]
] as const) {
describe(`${name} connector queue completion`, () => {
async function upload(results: unknown[], decision: unknown = 'retain') {
const connector = new Connector();
const connector = new Connector({ authenticator: stubAuthenticator() });
const hook = vi.fn(async () => {
if (decision instanceof Error) throw decision;
return decision;
Expand Down Expand Up @@ -65,7 +78,7 @@ for (const [name, Connector] of [
};
let error: unknown;
try {
await connector.uploadData(database as any);
await connector.upload({ database: database as any });
} catch (e) {
error = e;
}
Expand Down Expand Up @@ -131,3 +144,56 @@ for (const [name, Connector] of [
});
});
}

describe('write API authentication failure', () => {
const database = (complete = vi.fn(async () => {})) => ({
complete,
db: {
getClientId: async () => 'client',
async *getCrudTransactions() {
yield { crud: [{ id: '1', table: 'items', op: 'PUT' }], complete };
}
}
});

it.each([
[
'split',
(connector: any) =>
Object.assign(connector, {
getWriteClient: async () =>
new WriteAPIClient({
transport: {
postTransactionBatch: async () => {
throw new AuthenticationError('Authentication failed (401)');
}
},
userId: 'user',
clientId: 'client'
})
}),
Split
],
[
'single',
() =>
vi.stubGlobal(
'fetch',
vi.fn(async () => new Response(JSON.stringify({ message: 'Unauthorized' }), { status: 401 }))
),
Single
]
] as const)(
'%s connector invalidates credentials on 401 and keeps the transaction queued',
async (_, stub, Connector) => {
const authenticator = stubAuthenticator();
const connector = new Connector({ authenticator });
stub(connector);
const { complete, db } = database();

await expect(connector.upload({ database: db as any })).rejects.toThrow();
expect(authenticator.invalidateCredentials).toHaveBeenCalledOnce();
expect(complete).not.toHaveBeenCalled();
}
);
});
3 changes: 2 additions & 1 deletion backend/shared/test/persister-ping.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@ vi.mock('mongodb', () => ({
MongoClient: class {
async connect() {}
db() {
return { command: db.mongoCommand };
// createMongoPersister discovers collection validators at startup.
return { command: db.mongoCommand, listCollections: () => ({ toArray: async () => [] }) };
}
}
}));
Expand Down
62 changes: 31 additions & 31 deletions backend/supabase/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ create policy "Owners manage their lists" on public.lists
> A write that a policy or missing grant denies returns `retryable_error` for the whole
> transaction. The client connector keeps it queued and retries, so a mistake in a policy or grant does not
> lose user writes, but it blocks that user's later uploads until the write succeeds. Have your
> client connector's `uploadData` give up after a number of retries and dead-letter or discard the
> client connector give up after a number of retries and dead-letter or discard the
> transaction. The Express backend instead returns a fatal `UNAUTHORIZED` that stays queued; see
> [error handling](../../docs/error-handling.md).
>
Expand Down Expand Up @@ -141,40 +141,40 @@ Supabase Auth tokens; see PowerSync's

### Using Supabase Auth in the connector

Pass your app's supabase-js client to the copied connector and read the session on every call.
Do not keep the connector's token cache: supabase-js refreshes the session itself, and the cache
is cleared only after a rejected write, so sync would keep receiving an expired token. In
`PowersyncConnector.ts`, replace `fetchCredentials()`, `fetchAuthToken()`, and `getAuthToken()`:
Use an `Authenticator` that reads the access token from your app's supabase-js client, and pass it
to both `connect()` and the connector. The SDK isn't guaranteed to cache tokens, so cache the token
and clear the cache in `invalidateCredentials()`. PowerSync calls `invalidateCredentials()` 30
seconds before the token expires, and the connector calls it when the write API rejects the token.
The next `getSession()` call then returns a valid session: in current supabase-js versions, it
refreshes the session when it expires within 90 seconds.

```ts
constructor(private readonly supabase: SupabaseClient) {
// existing setup
}

async fetchCredentials() {
const session = await this.getSession();
return {
endpoint: this.config.powersyncUrl,
token: session.access_token,
expiresAt: session.expires_at ? new Date(session.expires_at * 1000) : undefined
};
}

private async getSession() {
const { data: { session }, error } = await this.supabase.auth.getSession();
if (!session || error) throw new Error('Not signed in to Supabase');
return session;
}

// Write requests call this through the transport.
private async getAuthToken(): Promise<string> {
return (await this.getSession()).access_token;
}
import type { SupabaseClient } from '@supabase/supabase-js';
import type { Authenticator } from '@powersync/web';

declare const supabase: SupabaseClient; // your app's client

let cachedToken: string | null = null;

const authenticator: Authenticator = {
async resolveCredentials() {
if (!cachedToken) {
const { data: { session }, error } = await supabase.auth.getSession();
if (!session || error) throw new Error('Not signed in to Supabase');
cachedToken = session.access_token;
}
return cachedToken;
},
invalidateCredentials() {
cachedToken = null;
}
};

const connector = new PowersyncConnector({ authenticator });
await db.connect({ powerSyncEndpoint, authenticator, upload: connector.upload });
```

The single-file connector has the same methods; use `POWERSYNC_URL` instead of
`this.config.powersyncUrl`. Remove the now-unused `_authToken` field, including its reset in
`onTransportError()`. Uploads, batching, and error handling are unchanged.
Uploads, batching, and error handling are unchanged.

## Customize

Expand Down
1 change: 1 addition & 0 deletions backend/supabase/functions/powersync-write/rls.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ export async function asAuthenticatedUser(client: PG.PoolClient, auth: AuthConte
*/
export function permissionErrorsRetryable(persister: Persister): Persister {
return {
ping: () => persister.ping(),
async updateBatch(batch, auth) {
try {
await persister.updateBatch(batch, auth);
Expand Down
8 changes: 4 additions & 4 deletions backend/supabase/tests/powersync-write/connector.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { admin, applyTestSchema, FUNCTION_URL, listRow, signUp } from './helpers

/**
* Runs the unchanged single-file example connector against the function. Only its BACKEND_URL
* and AUTH_TOKEN constants are replaced.
* constant is replaced. The user's Supabase session token comes from an Authenticator.
*/
Deno.test({
name: 'example connector uploads through the function URL',
Expand All @@ -14,8 +14,7 @@ Deno.test({
const user = await signUp('connector');
const source = await Deno.readTextFile(new URL('../../../../example-client/src/PowersyncConnector.singlefile.ts', import.meta.url));
const configured = source
.replace(/^const BACKEND_URL = .*$/m, `const BACKEND_URL = ${JSON.stringify(FUNCTION_URL)};`)
.replace(/^const AUTH_TOKEN = .*$/m, `const AUTH_TOKEN = ${JSON.stringify(user.token)};`);
.replace(/^const BACKEND_URL = .*$/m, `const BACKEND_URL = ${JSON.stringify(FUNCTION_URL)};`);
const path = await Deno.makeTempFile({ suffix: '.ts' });
await Deno.writeTextFile(path, configured);

Expand All @@ -40,7 +39,8 @@ Deno.test({
};

try {
await new PowersyncConnector().uploadData(database);
const authenticator = { resolveCredentials: async () => user.token };
await new PowersyncConnector({ authenticator }).upload({ database });
assertEquals(completed, 1);
assertEquals(await listRow(list), { name: 'From connector', owner_id: user.id });
} finally {
Expand Down
Loading
Loading