Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions configuration/source-db/postgres-maintenance.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -74,3 +74,49 @@ select current_setting('max_replication_slots')

If this number is exceeded, you'll see an error such as "all replication slots are in use".

## Oversized Replication Messages

{/* TODO: Revisit this note and its troubleshooting entry when oversized-message handling changes. */}

Postgres replication currently has a **50 MiB receive buffer limit**. A message that cannot fit can stall replication, with `postgres sent too big message` in the Replicator logs.

The limit applies to every table in the publication, even those not referenced by Sync Streams. Publish only required tables and keep source rows within the separate [15 MiB row/document limit](/resources/performance-and-limits#limits).

<Note>With `REPLICA IDENTITY FULL`, a message can include both previous and new row values, which doubles the size of a single row.</Note>

### Recovery

This involves removal of oversized rows from replication and a potential reprocessing:

<Steps>
<Step title="Identify Source">
Review writes across all published tables for oversized rows. The time of the error's first occurrence can help identify the source table.
</Step>
<Step title="Correct the Source">
If the affected table is part of the PowerSync publication but not used by any Sync Streams, then remove the table from the publication and redeploy the service, and skip to step 4.

Otherwise, reduce the size of the oversized row or delete it from the source database and move on to step 3.
</Step>
<Step title="Start a New Snapshot">
Start [full reprocessing](/maintenance-ops/compacting-buckets#defragmenting-strategies) for each affected instance. This creates a new replication slot and snapshots the current source data. Deleting a row alone does not erase its earlier WAL entries.

<Tabs>
<Tab title="PowerSync Cloud">
In the PowerSync Dashboard, open **Settings**, then select **Defragment** in the **Compact operation history** section.
</Tab>
<Tab title="Self-Hosted">
Set `POWERSYNC_URL` to your Service URL. Set `PS_API_TOKEN` to an admin token configured under [`api.tokens`](/configuration/powersync-service/self-hosted-instances#api).

```bash
curl "${POWERSYNC_URL}/api/admin/v1/reprocess" \
Comment thread
joshuabrink marked this conversation as resolved.
-H "Authorization: Bearer ${PS_API_TOKEN}" \
--json '{}'
```
</Tab>
</Tabs>
</Step>
<Step title="Verify Recovery">
If you reprocessed, confirm in your logs that the new replication stream is active. Check that replication lag decreases and clients receive new source updates.
</Step>
</Steps>

4 changes: 4 additions & 0 deletions debugging/troubleshooting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@

To further diagnose and resolve this error, see [Reducing Bucket Count](/sync/advanced/reducing-bucket-count). For how buckets are generated and counted in the first place, see [Bucket Count](/sync/streams/bucket-count).

### `postgres sent too big message`

Postgres replication can stall when a message cannot fit in the receive buffer. See [Oversized Replication Messages](/configuration/source-db/postgres-maintenance#oversized-replication-messages) for guidance.

## Tools

Troubleshooting techniques depend on the type of issue:
Expand Down Expand Up @@ -132,5 +136,5 @@

* **Large initial sync**: if your Sync Streams/Rules result in a large dataset, the first sync after connecting will be slow. Inspect bucket sizes and sync state with the [Sync Diagnostics Client](/tools/diagnostics-client).
* **Upload queue blocking downloads**: by default, uploads are processed before downloads, so a backlogged upload queue delays receiving new data. Buckets and streams at [priority 0](/sync/advanced/prioritized-sync) are not blocked by uploads, but come with the trade-off of potential sync inconsistencies.
* **Replication lag on the source database**: high write volume, long-running transactions, bulk updates, or backfills can cause replication to fall behind faster than the service can drain it. See [Replication Lag](/maintenance-ops/replication-lag) for source-specific causes and fixes.

Check warning on line 139 in debugging/troubleshooting.mdx

View check run for this annotation

Mintlify / Mintlify Validation (powersync) - vale-spellcheck

debugging/troubleshooting.mdx#L139

Did you really mean 'backfills'?
* **Too many buckets per user**: incremental sync overhead scales roughly linearly with the number of buckets per user. See [Too Many Buckets](#psync_s2305-too-many-buckets-/-parameter-query-results) above.
2 changes: 1 addition & 1 deletion resources/performance-and-limits.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The PowerSync Cloud **Team** and **Enterprise** plans allow several of these lim
| **Component** | **Limit** | **Details** |
|---|---|---|
| **Synced buckets per user** | Maximum: 1,000 by default; Configurable up to 10,000 by request | Two limits apply per connection, both defaulting to 1,000: unique buckets and parameter query results before deduplication. Exceeding either causes sync to fail with `PSYNC_S2305`. Both limits can be increased for Team and Enterprise customers on request, but higher limits can reduce performance. See [Bucket Count and Limits](/sync/streams/bucket-count). |
| **Maximum row/document size** | 15MB | Applies to both source database rows and transformed rows synced to clients. |
| **Maximum row/document size** | 15 MiB | Applies to both source database rows and transformed rows synced to clients. |
| **Concurrent clients** | Maximum: configurable (50k+ per instance) | PowerSync Service instances have default limits configured based on the [Pricing plan](https://www.powersync.com/pricing). These limits can be increased upon request for Team and Enterprise customers, and currently scale to over 50,000 per instance. |
| **Data hosted** | Maximum: configurable | PowerSync Service instances have default limits configured based on the [Pricing plan](https://www.powersync.com/pricing). These limits can be increased upon request for Enterprise customers. |
| **Columns per table** | 1,999 | Hard limit of the client schema, excluding the `id` column. |
Expand Down
Loading