diff --git a/ci/vale/dictionary.txt b/ci/vale/dictionary.txt index 10ee9a02a4b..1c86447fc11 100644 --- a/ci/vale/dictionary.txt +++ b/ci/vale/dictionary.txt @@ -1468,6 +1468,8 @@ Matthes Matz Mavan mawk +max_replication_slots +max_wal_senders maxconn maxdepth maxretry @@ -2832,6 +2834,7 @@ vvv vz vznetaddbr wafs +wal_level walkthrough walkthroughs WAN diff --git a/docs/guides/platform/migrate-to-linode/logical-replication-to-a-linode-managed-postgresql-database/akamai-cloud-manager-database-host.png b/docs/guides/platform/migrate-to-linode/logical-replication-to-a-linode-managed-postgresql-database/akamai-cloud-manager-database-host.png new file mode 100644 index 00000000000..4355aa637a5 Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/logical-replication-to-a-linode-managed-postgresql-database/akamai-cloud-manager-database-host.png differ diff --git a/docs/guides/platform/migrate-to-linode/logical-replication-to-a-linode-managed-postgresql-database/index.md b/docs/guides/platform/migrate-to-linode/logical-replication-to-a-linode-managed-postgresql-database/index.md new file mode 100644 index 00000000000..bf282281ab0 --- /dev/null +++ b/docs/guides/platform/migrate-to-linode/logical-replication-to-a-linode-managed-postgresql-database/index.md @@ -0,0 +1,430 @@ +--- +slug: logical-replication-to-a-linode-managed-postgresql-database +title: "Logical Replication to a Linode Managed PostgreSQL Database" +description: "Learn how to configure PostgreSQL logical replication from a cloud-hosted source database to a Linode Managed Database." +authors: ["Akamai"] +contributors: ["Akamai"] +published: 2026-02-18 +keywords: ['logical replication', 'postgresql logical replication', 'linode managed database', 'akamai cloud', 'database migration', 'low downtime migration', 'postgresql subscription', 'postgresql publication', 'replication slot', 'write-ahead log', 'pg_stat_subscription', 'pg_current_wal_lsn', 'pg_dump schema only'] +license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' +--- + +Logical replication with PostgreSQL allows you to stream changes from one database (the publisher) to another (the subscriber) at the table level. This makes it possible to replicate data across environments in near real-time, without disrupting the source database by taking it offline. + +[Logical replication](https://www.postgresql.org/docs/current/logical-replication.html) is an alternative approach to the "dump-and-restore" method often used for data replication or migration. Logical replication is useful if you want: + +- **Zero Downtime Migration**: Keep the source database online while synchronizing data to the destination. +- **Continuous Synchronization**: Maintain an ongoing sync between source and destination databases. +- **Selective Table Replication**: Only replicate specific tables, rather than the entire database. + +This guide walks through the process of setting up logical replication from an existing PostgreSQL database to a [Linode Managed Database](https://www.linode.com/products/databases/) running PostgreSQL. + +## Things to Know Before Getting Started + +Before configuring logical replication, review the following key concepts and requirements. Understanding these can help you configure your databases correctly and avoid common issues during replication. + +### Key Terminology + +Logical replication in PostgreSQL follows a *publisher/subscriber* (pub/sub) model. Here, the source database publishes table-level changes and one or more subscribers receive and apply them. + +- **Publisher**: The source PostgreSQL database that exposes data changes through one or more publications. +- **Subscriber**: The destination PostgreSQL database that connects to a publisher and applies changes locally. +- **Publication**: A named set of tables on the publisher whose changes are made available to subscribers. +- **Subscription**: A replication object on the subscriber that connects to a publisher and retrieves changes from its publications. +- **Write-Ahead Log (WAL)**: A sequential log of all database changes. Logical replication reads table-level changes decoded from this log. +- **Replication Slot**: Tracks the progress of each subscriber and retains required WAL segments until they’ve been consumed. +- **WAL Sender (`walsender`)**: A background process on the publisher that streams WAL-derived changes to the subscriber. +- **Apply Worker**: A background process on the subscriber that replays incoming changes in transactional order. + +The diagram below illustrates the logical replication flow between the publisher and subscriber: + +(refer to diagram below) Logical Replication Flow + +![Diagram showing PostgreSQL logical replication flow from publisher to subscriber.](postgresql-logical-replication-flow.png) + +- The publisher defines what to replicate via a publication. +- Changes are captured from the Write-Ahead Log (WAL) and managed through a replication slot. +- A WAL sender streams these changes to the subscriber, which uses an apply worker to write them to local tables. + +### PostgreSQL Version Compatibility + +Logical replication works across major versions of PostgreSQL, but some features may have compatibility limitations. Confirm that your source and destination PostgreSQL versions both support logical replication and are compatible with each other. + +### Server Settings + +To enable logical replication, the source database server must be configured with the following parameters: + +- `wal_level`: Must be set to `logical` to enable logical decoding. +- `max_replication_slots`: At least one slot is required per subscription. +- `max_wal_senders`: Should be equal to or greater than `max_replication_slots`. + +### Roles and Permissions + +The source database should have a dedicated replication user. This user must have the `REPLICATION` attribute and `SELECT` privileges on any tables to be replicated. This is a security best practice, as it avoids using superuser or administrator accounts for replication purposes. + +### Network Access + +Replication requires that the destination database can connect to the source over the network. This means ensuring the source database accepts connections from the Linode Managed Database. + +## Before You Begin + +1. Follow our [Get started](https://techdocs.akamai.com/cloud-computing/docs/getting-started) guide to create an Akamai Cloud account if you do not already have one. + +1. Follow the steps in the Akamai Cloud documentation to [create a new database cluster](https://techdocs.akamai.com/cloud-computing/docs/aiven-manage-database#create-a-new-database-cluster). + +1. [Set up access control](https://techdocs.akamai.com/cloud-computing/docs/aiven-manage-database#access-control) so that you can connect to the database from your local machine. + +1. Install the [Linode CLI](https://techdocs.akamai.com/cloud-computing/docs/install-and-configure-the-cli) on your local machine. + +1. Install the `psql` client on your local machine: + + ```command + sudo apt install postgresql-client + ``` + +1. Ensure you have administrative access to your source PostgreSQL database. + +### Placeholders and Examples + +The following placeholders and example values are used in commands throughout this guide: + +| Parameter | Placeholder | Example Value | +|------------|--------------|----------------| +| Destination Hostname | {{< placeholder "DEST_HOST" >}} | `a334568-akamai-prod-183144-default.g2a.akamaidb.net` | +| Destination IP Address | {{< placeholder "DEST_IP" >}} | `172.235.135.114` | +| Destination Port | {{< placeholder "DEST_PORT" >}} | `10033` | +| Destination Database | {{< placeholder "DEST_DB" >}} | `defaultdb` | +| Destination Username | {{< placeholder "DEST_USER" >}} | `akmadmin` | +| Destination Password | {{< placeholder "DEST_PASSWORD" >}} | `thisismydestinationpassword` | +| Subscription Name | {{< placeholder "SUBSCRIPTION_NAME" >}} | `my_subscription` | + +Replace these placeholders with your own connection details when running commands in your environment. + +Additionally, the examples used in this guide assume the source database contains three tables (`customers`, `products`, and `orders`) that you want to replicate to a Linode Managed Database. + +### Example Scenario + +The examples used in this guide assume you have an existing PostgreSQL database hosted with a managed service from a cloud provider. This source database contains three tables (`customers`, `products`, and `orders`) that you want to replicate to a Linode Managed Database for PostgreSQL. + +## Configure the Source Database + +The source PostgreSQL database must be properly configured to support logical replication and accept incoming connections from your Linode Managed Database. + +### Obtain the IP Address of the Destination Database + +To configure access control on your source database, you first need the IP address of your Linode Managed Database. Begin by identifying your destination host ({{< placeholder "DEST_HOST" >}}). + +{{< tabs >}} +{{< tab "Akamai Cloud Manager" >}} +1. In the [Akamai Cloud Manager](https://cloud.linode.com/), navigate to **Databases**, then select your database cluster and copy the **Host** string: + + ![Screenshot of Akamai Cloud Manager showing Linode Managed Database host information.](akamai-cloud-manager-database-host.png) + + In the example above, the {{< placeholder "DEST_HOST" >}} value is `a334568-akamai-prod-183144-default.g2a.akamaidb.net` +{{< /tab >}} +{{< tab "Linode CLI" >}} +1. To find the host using the Linode CLI, run the following command to list your databases: + + ```command + linode databases list --json | jq + ``` + + Locate the `primary` host listed for your destination database: + + ```output + [ + { + "allow_list": [ + "2606:4700:3035::ac43:bc7a/128", + ], + "engine": "postgresql", + "hosts": { + "primary": "a334568-akamai-prod-183144-default.g2a.akamaidb.net" + }, + "id": 334568, + "label": "destination-db", + "port": 10033, + "region": "us-sea", + "status": "active", + "total_disk_size_gb": 9, + "type": "g6-nanode-1", + "version": "16.9", + ... + }, + ... + ] + ``` + + In the example above, the {{< placeholder "DEST_HOST" >}} value is `a334568-akamai-prod-183144-default.g2a.akamaidb.net` +{{< /tab >}} +{{< /tabs >}} + +2. Use a DNS lookup tool such as `nslookup` to resolve the IP address of your Linode Managed Database. Replace {{< placeholder "DEST_HOST" >}} with your actual value (e.g., `a334568-akamai-prod-183144-default.g2a.akamaidb.net`): + + ```command + nslookup {{< placeholder "DEST_HOST" >}} + ``` + + In this example, `172.232.188.122` is the {{< placeholder "DEST_IP" >}}: + + ```output + Non-authoritative answer: + Name: a334568-akamai-prod-183144-default.g2a.akamaidb.net + Address: 172.232.188.122 + Name: a334568-akamai-prod-183144-default.g2a.akamaidb.net + Address: 2600:3c0a::2000:fbff:fe65:756d + ``` + + Be sure to record your {{< placeholder "DEST_IP" >}}, as it is required in the cloud-specific preparation guides referenced in the next section. + +### Prepare the Source Database for Logical Replication + +With your Linode Managed Database’s IP address located, follow the corresponding guide to prepare your source database for logical replication: + +- [Preparing your AWS RDS PostgreSQL Database for Logical Replication to Linode Managed Database](/docs/guides/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/) +- [Preparing your Azure PostgreSQL Database for Logical Replication to Linode Managed Database](/docs/guides/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/) +- [Preparing your Google Cloud SQL PostgreSQL Database for Logical Replication to Linode Managed Database](/docs/guides/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/) + +After completing the steps in one of the above guides, gather the following information for your *source database*: + +| Parameter | Placeholder | Example Value | +|------------|--------------|----------------| +| Source Hostname or IP Address | {{< placeholder "SOURCE_HOST" >}} | `psql.managed-db-services.example.com` | +| Source Port Number | {{< placeholder "SOURCE_PORT" >}} | `5432` | +| Source Database | {{< placeholder "SOURCE_DB" >}} | `postgres` | +| Replication Username | {{< placeholder "REPL_USER" >}} | `linode_replicator` | +| Replication Password | {{< placeholder "REPL_PASSWORD" >}} | `thisismyreplicatorpassword` | +| Publication Name | {{< placeholder "PUBLICATION_NAME" >}} | `my_publication` | + +The remainder of this guide uses the example values shown above. + +{{< note title="Note: Source Host Value" >}} +{{< placeholder "SOURCE_HOST" >}} is your source database hostname or IP address. Depending on your source cloud provider, this may be an RDS endpoint, an Azure Flexible Server hostname, or a Cloud SQL public IP. +{{< /note >}} + +## Configure the Destination Database + +Configure your Linode Managed PostgreSQL database as the replication target. This involves two main tasks: + +- Recreate the schema +- Create a subscription to the source publication + +### Create the Database Schema + +Logical replication in PostgreSQL does not copy table definitions, it only replicates the data. This means you must manually create the destination tables to match the source database schema exactly. Use the same column names, types, constraints, and indexes to avoid replication errors. + +1. At the command line, use `pg_dump` with the `--schema-only` flag to connect to your source database and write the resulting SQL statements to a file. Replace {{< placeholder "SOURCE_HOST" >}} (e.g. `psql.managed-db-services.example.com`), {{< placeholder "REPL_USER" >}} (e.g., `linode_replicator`), and {{< placeholder "SOURCE_PORT" >}} (e.g. `5432`) with your own values: + + ```command + pg_dump \ + -h {{< placeholder "SOURCE_HOST" >}} \ + -U {{< placeholder "REPL_USER" >}} \ + -p {{< placeholder "SOURCE_PORT" >}} \ + --schema-only \ + --no-owner \ + --no-privileges \ + --schema=public \ + {{< placeholder "SOURCE_DB" >}} > schema.sql + ``` + + When prompted enter your Replication Password (e.g., `thisismyreplicatorpassword`) and press Enter. + + {{< note type="secondary" title="Alternative: Dump Specific Tables Only" isCollapsible=yes >}} + If you only want to create the schema for specific tables rather than the entire database, use the `--table` flag. For example, the following command only dumps the schema for the `customers` and `orders` tables: + + ```command + pg_dump \ + -h {{< placeholder "SOURCE_HOST" >}} \ + -U {{< placeholder "REPL_USER" >}} \ + -p {{< placeholder "SOURCE_PORT" >}} \ + --schema-only \ + --no-owner \ + --no-privileges \ + --schema=public \ + --table customers \ + --table orders \ + postgres > schema.sql + ``` + {{< /note >}} + +1. Connect to your Linode Managed Database using the `psql` client. Replace {{< placeholder "DEST_IP" >}} (e.g, `172.235.135.114`), {{< placeholder "DEST_USER" >}} (e.g., `akmadmin`), {{< placeholder "DEST_PORT" >}} (e.g, `10033`), and {{< placeholder "DEST_DB" >}} (e.g., `defaultdb`) with your own values: + + ```command + psql \ + -h {{< placeholder "DEST_IP" >}} \ + -U {{< placeholder "DEST_USER" >}} \ + -p {{< placeholder "DEST_PORT" >}} \ + -d {{< placeholder "DEST_DB" >}} + ``` + +1. From the `psql` prompt, run the schema file to create the tables on the destination database: + + ```command {title="Destination psql Prompt"} + \i schema.sql + ``` + +Unless otherwise noted, remain logged in to the destination database for the remainder of this section. + +### Create a Subscription + +Once the schema is in place, create a subscription on the destination database. This step connects your Linode Managed Database (the subscriber) to your source database (the publisher) and begins streaming changes. + +1. While still logged in to the `psql` prompt, run the [`CREATE SUBSCRIPTION`](https://www.postgresql.org/docs/current/sql-createsubscription.html) command to create a subscription (e.g., `my_subscription`) on the destination database. Remember to substitute placeholders for your own source database values: + + ```command {title="Destination psql Prompt"} + CREATE SUBSCRIPTION {{< placeholder "SUBSCRIPTION_NAME" >}} + CONNECTION 'host={{< placeholder "SOURCE_HOST" >}} port={{< placeholder "SOURCE_PORT" >}} user={{< placeholder "REPL_USER" >}} password={{< placeholder "REPL_PASSWORD" >}} dbname={{< placeholder "SOURCE_DB" >}} sslmode=require' + PUBLICATION {{< placeholder "PUBLICATION_NAME" >}}; + ``` + + ```output + CREATE SUBSCRIPTION + ``` + + Once the subscription is created, PostgreSQL begins replicating data from the source to the destination. + +### Check Subscription Status + +Once replication is active, monitor its status and verify that data is syncing correctly. + +1. While still logged in to the destination database, run the following SQL command to retrieve information about the subscription: + + ```command {title="Destination psql Prompt"} + SELECT * FROM pg_catalog.pg_stat_subscription; + ``` + + ```output + -[ RECORD 1 ]---------+------------------------------ + subid | 16492 + subname | my_subscription + pid | 463099 + leader_pid | + relid | + received_lsn | 0/D90038A0 + last_msg_send_time | 2025-07-24 17:38:44.539465+00 + last_msg_receipt_time | 2025-07-24 17:38:44.575312+00 + latest_end_lsn | 0/D90038A0 + latest_end_time | 2025-07-24 17:38:44.539465+00 + ``` + + Note the following important fields in the output: + + - `subname`: The name of the subscription + - `received_lsn`: The last WAL location received + - `latest_end_lsn`: The last WAL location applied + - `last_msg_send_time` and `last_msg_receipt_time`: Timestamps for the most recent activity + +### Compare WAL Positions + +To verify how far along the subscriber is, compare the `received_lsn` (or `latest_end_lsn`) from the destination with the current WAL position on the source. + +1. Open a new terminal window and use `psql` to establish a connection to the source (publisher) database: + + ```command + psql \ + -h {{< placeholder "SOURCE_HOST" >}} \ + -U {{< placeholder "REPL_USER" >}} \ + -p {{< placeholder "SOURCE_PORT" >}} \ + -d {{< placeholder "SOURCE_DB" >}} + ``` + +1. On the source database, run the following SQL command to retrieve the current WAL log sequence number: + + ```command {title="Source psql Prompt"} + SELECT pg_current_wal_lsn(); + ``` + + ```output + pg_current_wal_lsn + -------------------- + 0/DA0009D8 + (1 row) + ``` + + Compare this value with the `latest_end_lsn` shown on the destination: + + - If they match or are close, replication is current and lag-free. + - A large gap may indicate replication delay or a connectivity issue. + + Small differences are normal on active databases as new WAL entries are generated continuously. + +1. When done, type `\q` and press Enter to exit the source `psql` shell. + +### Validate Replicated Data + +Verify that the expected data appears in the destination (subscriber) database. + +1. Return to the terminal window connected to the Linode Managed Database (destination) `psql` prompt and run validation queries and other table checks to ensure that all rows were copied and continue to stay in sync, for example: + + ```command {title="Destination psql Prompt"} + SELECT COUNT(*) FROM public.customers; + ``` + + ```output + count + ------- + 2 + (1 row) + ``` + +1. When done, type `\q` and press Enter to exit the destination `psql` shell. + +You may want to repeat this validation periodically until the cutover is complete. + +## Finalize the Cutover + +Once replication is stable and the destination database is fully caught up, complete the migration by redirecting application traffic and decommissioning the replication setup. + +### Redirect Application Traffic + +Update your applications to point to the Linode Managed Database instead of the old source database. This involves changing the database host, port, username, and password in each application's connection settings. + +Test application connectivity to the new database in a staging environment or during a maintenance window before routing production traffic to the new database. + +### Remove the Subscription + +After all applications are using the Linode Managed Database as the primary data store, remove the subscription. This stops replication and turns the destination into an independent, writable database. + +1. Use `psql` to reconnect to your Linode Managed Database and enter the interactive prompt: + + ```command + psql \ + -h {{< placeholder "DEST_IP" >}} \ + -U {{< placeholder "DEST_USER" >}} \ + -p {{< placeholder "DEST_PORT" >}} \ + -d {{< placeholder "DEST_DB" >}} + ``` + +1. Run the following SQL command to remove the subscription from the destination database: + + ```command {title="Destination psql Prompt"} + DROP SUBSCRIPTION my_subscription; + ``` + + ```output + NOTICE: dropped replication slot "my_subscription" on publisher + DROP SUBSCRIPTION + ``` + + This command also removes the replication slot from the source database, preventing unnecessary WAL retention. + +1. When done, type `\q` and press Enter to exit the destination `psql` shell. + +### Retire the Source Database + +Once replication has stopped and all applications have switched to the destination database, you can safely decommission the source. Before doing so, make sure: + +- All applications connect exclusively to the new Linode Managed Database. +- Replicated data has been verified as complete and accurate. +- Backups exist for both the source and destination databases. + +After confirming these items, follow your cloud provider's procedures to delete or archive the source database. + +## Considerations and Potential Challenges + +Logical replication provides flexibility, but a few common challenges can arise: + +- **Schema Drift**: Changes to table structures on the source are not automatically replicated. Keep schemas aligned manually to avoid replication errors. +- **Conflicting Writes**: The subscriber should remain read-only during replication. Avoid making manual changes that could conflict with incoming data. +- **Network Reliability**: Logical replication requires a stable network connection between the destination and source databases. Temporary disconnects can cause lag or stalled replication until connectivity is restored. +- **Replication Slot Management**: Orphaned replication slots on the source can lead to WAL buildup. Always drop the subscription to clean up replication slots after cutover. \ No newline at end of file diff --git a/docs/guides/platform/migrate-to-linode/logical-replication-to-a-linode-managed-postgresql-database/postgresql-logical-replication-flow.png b/docs/guides/platform/migrate-to-linode/logical-replication-to-a-linode-managed-postgresql-database/postgresql-logical-replication-flow.png new file mode 100644 index 00000000000..c36686c36fe Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/logical-replication-to-a-linode-managed-postgresql-database/postgresql-logical-replication-flow.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-attach-parameter-group-to-instance.png b/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-attach-parameter-group-to-instance.png new file mode 100644 index 00000000000..844f964b7ed Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-attach-parameter-group-to-instance.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-create-custom-parameter-group.png b/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-create-custom-parameter-group.png new file mode 100644 index 00000000000..f2bd32acfa0 Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-create-custom-parameter-group.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-db-parameter-group-link.png b/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-db-parameter-group-link.png new file mode 100644 index 00000000000..2b84befb7e6 Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-db-parameter-group-link.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-edit-parameter-group-values.png b/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-edit-parameter-group-values.png new file mode 100644 index 00000000000..7f98f8af3b9 Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-edit-parameter-group-values.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-parameter-group-logical-replication-settings.png b/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-parameter-group-logical-replication-settings.png new file mode 100644 index 00000000000..3b9ec8d7381 Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-rds-parameter-group-logical-replication-settings.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-security-group-postgresql-inbound-rule.png b/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-security-group-postgresql-inbound-rule.png new file mode 100644 index 00000000000..5fabe5fd374 Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/aws-security-group-postgresql-inbound-rule.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/index.md b/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/index.md new file mode 100644 index 00000000000..8c51e70029c --- /dev/null +++ b/docs/guides/platform/migrate-to-linode/preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database/index.md @@ -0,0 +1,421 @@ +--- +slug: preparing-your-aws-rds-postgresql-database-for-logical-replication-to-linode-managed-database +title: "Preparing Your AWS RDS PostgreSQL Database for Logical Replication to Linode Managed Database" +description: "Prepare your Amazon RDS for PostgreSQL database for logical replication to a Linode Managed Database. Configure replication parameters, network access, and a replication user." +authors: ["Akamai"] +contributors: ["Akamai"] +published: 2026-02-18 +keywords: ['aws rds postgresql logical replication','linode managed database','rds parameter group','pg_publication','aws replication setup'] +license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' +external_resources: +- '[Setting up PostgreSQL logical replication](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_MultiAZDBCluster_LogicalRepl.html#multi-az-db-clusters-logical-replication)' +- '[Parameter groups for Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithParamGroups.html)' +- '[CLI documentation for AWS RDS](https://docs.aws.amazon.com/cli/latest/reference/rds/)' +--- + +[Logical replication](https://www.postgresql.org/docs/current/logical-replication.html) continuously synchronizes database tables, allowing you to prepare the destination database in advance. This approach minimizes downtime when you switch application traffic and retire the source database. + +This guide explains how to prepare an AWS RDS PostgreSQL database for logical replication to a [Linode Managed Database](https://www.linode.com/products/databases/). Follow this guide before returning to the [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) guide to [create the subscription](https://www.postgresql.org/docs/current/sql-createsubscription.html) on Akamai Cloud. + +Follow the steps in this guide to: + +- Configure your RDS instance to support logical replication. +- Ensure secure network access from Linode. +- Create a dedicated replication user. +- Set up a publication for the tables you wish to replicate. + +After completing these steps, return to [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) to configure the subscriber and finalize the setup. + +## Before You Begin + +1. Follow the [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) guide up to the **Prepare the Source Database for Logical Replication** section to obtain the public IP address or CIDR range of your Linode Managed Database. + +1. Ensure that you have administrative access to your AWS account, including permissions to modify RDS instance settings and security groups. + +1. Install the AWS CLI on your local machine and configure it with a user or role that has the necessary privileges listed above. + +### Placeholders and Examples + +The following placeholders and example values are used in commands throughout this guide: + +| Parameter | Placeholder | Example Value | +|------------|--------------|----------------| +| AWS RDS Instance ID | {{< placeholder "DB_INSTANCE_ID" >}} | `source-postgres` | +| AWS Region | {{< placeholder "AWS_REGION" >}} | `us-west-2` | +| AWS Security Group ID | {{< placeholder "SECURITY_GROUP_ID" >}} | `sg-78944f70` | +| Destination IP Address | {{< placeholder "DEST_IP" >}} | `172.232.188.122` | +| Source Hostname (RDS Endpoint) | {{< placeholder "SOURCE_HOST" >}} | `source-postgres.abc123.us-west-2.rds.amazonaws.com` | +| Source Port | {{< placeholder "SOURCE_PORT" >}} | `5432` | +| Source Username | {{< placeholder "SOURCE_USER" >}} | `democoder` | +| Source Database | {{< placeholder "SOURCE_DB" >}} | `postgres` | +| Source Password | {{< placeholder "SOURCE_PASSWORD" >}} | `thisismysourcepassword` | +| Replication Username | {{< placeholder "REPL_USER" >}} | `linode_replicator` | +| Replication Password | {{< placeholder "REPL_PASSWORD" >}} | `thisismyreplicatorpassword` | +| Publication Name | {{< placeholder "PUBLICATION_NAME" >}} | `my_publication` | + +Replace these placeholders with your own connection details when running commands in your environment. + +Additionally, the examples used in this guide assume the source database contains three tables (`customers`, `products`, and `orders`) that you want to replicate to a Linode Managed Database. + +## Configure RDS Parameter Group + +To support logical replication, you must configure your RDS instance with the correct parameter settings. Review your RDS instance’s current parameter group settings to determine if changes are required: + +{{< tabs >}} +{{< tab "AWS Console" >}} +1. In the AWS Management Console, navigate to **RDS > Databases** and select your instance. + +1. Open the **Configuration** tab and select the associated **DB instance parameter group** (not to be confused with the **option group**): + + ![AWS RDS database configuration page showing the DB instance parameter group link.](aws-rds-db-parameter-group-link.png) + +1. Filter the list of parameters to find the values for `rds.logical_replication`, `max_replication_slots`, and `max_wal_senders`: + + ![AWS RDS parameter group settings filtered to show rds.logical_replication, max_replication_slots, and max_wal_senders.](aws-rds-parameter-group-logical-replication-settings.png) + +{{< /tab >}} +{{< tab "AWS CLI" >}} +1. Run the following `aws` CLI command to obtain the name of the parameter group, replacing {{< placeholder "DB_INSTANCE_ID" >}} (e.g., `source-postgres`) and {{< placeholder "AWS_REGION" >}} (e.g., `us-west-2`) with your own values: + + ```command + aws rds describe-db-instances \ + --db-instance-identifier {{< placeholder "DB_INSTANCE_ID" >}} \ + --region {{< placeholder "AWS_REGION" >}} \ + --query "DBInstances[0].DBParameterGroups[*].DBParameterGroupName" + ``` + + ```output + [ + "default.postgres16" + ] + ``` + +1. Use the `describe-db-parameters` subcommand to obtain the parameter values for a given parameter group (e.g., `default.postgres16`): + + ```command + aws rds describe-db-parameters \ + --region {{< placeholder "AWS_REGION" >}} \ + --db-parameter-group-name default.postgres16 \ + --query "Parameters[?ParameterName=='rds.logical_replication' || ParameterName=='max_replication_slots' || ParameterName=='max_wal_senders'].[ParameterName, ParameterValue, ApplyType, Description]" + ``` + + ```output + [ + [ + "max_replication_slots", + "20", + "static", + "Sets the maximum number of replication slots that the server can support." + ], + [ + "max_wal_senders", + "35", + "static", + "Sets the maximum number of simultaneously running WAL sender processes." + ], + [ + "rds.logical_replication", + "0", + "static", + "Enables logical decoding." + ] + ] + ``` +{{< /tab >}} +{{< /tabs >}} + +In order for logical replication to succeed, these values should be as follows: + +- `max_replication_slots`: Greater than or equal to `1` +- `max_wal_senders`: Greater than or equal to `max_replication_slots`, depending on expected replication concurrency +- `rds.logical_replication`: `1` (which sets the PostgreSQL setting of `wal_level` to `logical`) + +If these values are already set correctly, skip the remainder of this section and continue with **Configure Network Access**. Otherwise, you need to modify the parameter group using the instructions below. + +### Create a Custom Parameter Group + +The default parameter group cannot be modified. If your RDS instance is using the default parameter group, you must create a custom parameter group to make the necessary modifications. + +{{< tabs >}} +{{< tab "AWS Console" >}} +1. Navigate to **Parameter groups** and select **Create parameter group** to create a new custom parameter group. + +1. Adjust the following fields to create a parameter group that begins with values that match the default parameter group: + + - **Parameter group name**: Specify a name (e.g., `with-logical-replication-on`). + - **Description**: Provide a description (e.g., `sets wal_level to logical`). + - **Engine type**: Choose **PostgreSQL**. + - **Parameter group family**: Choose based on the PostgreSQL version used by your RDS instance (e.g., **postgres16**). + +1. When done, click **Create**: + + ![AWS console form for creating a custom RDS PostgreSQL parameter group.](aws-rds-create-custom-parameter-group.png) + +1. On the details page for the newly created parameter group, click **Edit**. + +1. Search for each of the three parameters that may need modification and set their values as required: + + - `max_replication_slots`: Greater than or equal to `1` + - `max_wal_senders`: Greater than or equal to `max_replication_slots`, depending on expected replication concurrency + - `rds.logical_replication`: `1` (which sets the PostgreSQL setting of `wal_level` to `logical`) + +1. When done, click **Save Changes**: + + ![AWS RDS parameter group editor showing logical replication and WAL-related parameter values.](aws-rds-edit-parameter-group-values.png) + +1. Navigate to **Databases**, select your instance, and choose **Modify**. + +1. Under **Additional configuration**, set the **DB parameter group** to your newly created parameter group: + + ![AWS RDS modify database instance page showing selection of a custom DB parameter group.](aws-rds-attach-parameter-group-to-instance.png) + +1. Click **Continue**, select **Apply immediately** for the modification schedule, then click **Modify DB instance**. + +1. Reboot the RDS instance to apply the changes. +{{< /tab >}} +{{< tab "AWS CLI" >}} +1. Create a new parameter group based on the `postgres16` parameter group family: + + ```command + aws rds create-db-parameter-group \ + --region {{< placeholder "AWS_REGION" >}} \ + --db-parameter-group-name with-logical-replication-on \ + --db-parameter-group-family postgres16 \ + --description "sets wal_level to logical" + ``` + + This creates a parameter group that begins with values that match the default parameter group: + + ```output + { + "DBParameterGroup": { + "DBParameterGroupName": "with-logical-replication-on", + "DBParameterGroupFamily": "postgres16", + "Description": "sets wal_level to logical", + ... + } + } + ``` + +1. Modify parameter values for a custom parameter group as follows: + + ```command + aws rds modify-db-parameter-group \ + --region {{< placeholder "AWS_REGION" >}} \ + --db-parameter-group-name with-logical-replication-on \ + --parameters \ + "ParameterName=rds.logical_replication,ParameterValue=1,ApplyMethod=pending-reboot" \ + "ParameterName=max_replication_slots,ParameterValue=10,ApplyMethod=pending-reboot" \ + "ParameterName=max_wal_senders,ParameterValue=10,ApplyMethod=pending-reboot" + ``` + +1. Associate your RDS instance with the parameter group: + + ```command + aws rds modify-db-instance \ + --region {{< placeholder "AWS_REGION" >}} \ + --db-instance-identifier {{< placeholder "DB_INSTANCE_ID" >}} \ + --db-parameter-group-name with-logical-replication-on \ + --apply-immediately + ``` + +1. Reboot the RDS instance to apply the changes: + + ```command + aws rds reboot-db-instance \ + --region {{< placeholder "AWS_REGION" >}} \ + --db-instance-identifier {{< placeholder "DB_INSTANCE_ID" >}} + ``` +{{< /tab >}} +{{< /tabs >}} + +## Configure Network Access + +Ensure that the RDS instance allows network access from the Linode Managed Database. + +{{< tabs >}} +{{< tab "AWS Console" >}} +1. Under **Connectivity & security** for your database instance, find and click on the security group associated with the instance. + +1. Examine the inbound rules for the security group. If an existing rule allows access from `0.0.0.0/0`, your Linode Managed Database should already have access. Otherwise, you must create one. + +1. Click **Edit inbound rules**, then click **Add rule**. + +1. Set the rule type to "PostgreSQL" (TCP protocol to port `5432`). For source, add the CIDR block of your Linode Managed Database. + + ![AWS security group inbound rule allowing PostgreSQL traffic on port 5432 from a Linode Managed Database IP address.](aws-security-group-postgresql-inbound-rule.png) + +1. Click **Save rules**. +{{< /tab >}} +{{< tab "AWS CLI" >}} +1. Retrieve the name of the security group for your RDS instance: + + ```command + aws rds describe-db-instances \ + --db-instance-identifier {{< placeholder "DB_INSTANCE_ID" >}} \ + --region {{< placeholder "AWS_REGION" >}} \ + --query "DBInstances[0].{SecurityGroups:VpcSecurityGroups[*].VpcSecurityGroupId}" + ``` + + This value is used as {{< placeholder "SECURITY_GROUP_ID" >}} in the following command: + + ```output + { + "SecurityGroups": [ + "sg-78944f70" + ] + } + ``` + +1. Retrieve inbound rules for a security group: + + ```command + aws ec2 describe-security-groups \ + --group-ids {{< placeholder "SECURITY_GROUP_ID" >}} \ + --region {{< placeholder "AWS_REGION" >}} \ + --query "SecurityGroups[0].IpPermissions" + ``` + + Examine the inbound rules for the security group: + + ```output + [ + { + "FromPort": 5432, + "IpProtocol": "tcp", + "IpRanges": [ + { + "CidrIp": "..." + }, + { + "CidrIp": "..." + } + ], + "Ipv6Ranges": [], + "PrefixListIds": [], + "ToPort": 5432, + "UserIdGroupPairs": [] + }, + ... + ] + ``` + + If an existing rule allows access from `0.0.0.0/0`, your Linode Managed Database should already have access. Otherwise, you must create one. + +1. Add a firewall rule allowing access from your Linode Managed Database. Replace {{< placeholder "DEST_IP" >}} with the IP address from [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) (e.g., `172.232.188.122`): + + ```command + aws ec2 authorize-security-group-ingress \ + --group-id {{< placeholder "SECURITY_GROUP_ID" >}} \ + --region {{< placeholder "AWS_REGION" >}} \ + --protocol tcp \ + --port 5432 \ + --cidr {{< placeholder "DEST_IP" >}}/32 + ``` +{{< /tab >}} +{{< /tabs >}} + +With network access configured, your Linode Managed Database can reach the RDS instance during the subscription creation step in [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/). + +## Create a Replication User + +While logical replication can technically be performed using the primary database user, it's best practice to create a dedicated replication user. This user should have the `rds_replication` privilege and `SELECT` access only to the tables being published. + +Follow the steps below to create this limited-privileges user on your RDS instance. + +1. Before connecting to your source database, determine the RDS endpoint hostname: + + ```command + aws rds describe-db-instances \ + --region {{< placeholder "AWS_REGION" >}} \ + --db-instance-identifier {{< placeholder "DB_INSTANCE_ID" >}} \ + --query "DBInstances[0].Endpoint.Address" \ + --output text + ``` + + This value is used as {{< placeholder "SOURCE_HOST" >}} in the following command: + + ```output + source-postgres.abc123.us-west-2.rds.amazonaws.com + ``` + +1. Connect to your source PostgreSQL instance using the `psql` client. Replace {{< placeholder "SOURCE_HOST" >}} (e.g., `source-postgres.abc123.us-west-2.rds.amazonaws.com`), {{< placeholder "SOURCE_PORT" >}} (e.g., `5432`), {{< placeholder "SOURCE_USER" >}} (e.g., `democoder`), and {{< placeholder "SOURCE_DB" >}} (e.g., `postgres`) with your own values. You can find the endpoint on the **Connectivity & security** tab of your RDS instance. The default database name for RDS PostgreSQL instances is `postgres`. + + ```command + psql \ + -h {{< placeholder "SOURCE_HOST" >}} \ + -p {{< placeholder "SOURCE_PORT" >}} \ + -U {{< placeholder "SOURCE_USER" >}} \ + -d {{< placeholder "SOURCE_DB" >}} + ``` + + When prompted, enter your {{< placeholder "SOURCE_PASSWORD" >}} (e.g., `thisismysourcepassword`). + +1. Run the following commands from the source `psql` prompt. Replace {{< placeholder "REPL_USER" >}} (e.g., `linode_replicator`) and {{< placeholder "REPL_PASSWORD" >}} (e.g., `thisismyreplicatorpassword`) with your own values. For simplicity, this example assumes a public schema and three sample tables (customers, products, and orders). Replace the table names with your actual schema as needed. + + ```command {title="Source psql Prompt"} + CREATE ROLE {{< placeholder "REPL_USER" >}} LOGIN PASSWORD '{{< placeholder "REPL_PASSWORD" >}}'; + GRANT rds_replication TO {{< placeholder "REPL_USER" >}}; + GRANT SELECT ON customers, products, orders TO {{< placeholder "REPL_USER" >}}; + ``` + + ```output + CREATE ROLE + GRANT ROLE + GRANT + ``` + + {{< note type="secondary" title="Alternative: Grant on All Tables" isCollapsible=yes >}} + You can also grant privileges on *all* tables with the following command: + + ```command {title="Source psql Prompt"} + GRANT SELECT ON ALL TABLES IN SCHEMA public TO {{< placeholder "REPL_USER" >}}; + ``` + + ```output + GRANT + ``` + {{< /note >}} + +The newly created user is referenced by the Linode Managed Database when creating the subscription in [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/). + +## Create a Publication + +A publication defines which tables and changes (e.g., `INSERT`, `UPDATE`, and `DELETE`) should be streamed to the subscriber. At least one publication is required for logical replication, and the subscriber must have matching tables with compatible schemas for replication to succeed. + +1. While still connected to your source database via the `psql` client, use the following command to create a publication. Replace {{< placeholder "PUBLICATION_NAME" >}} (e.g., `my_publication`) and the specific tables you want to replicate (e.g., `customers`, `products`, and `orders`): + + ```command {title="Source psql Prompt"} + CREATE PUBLICATION {{< placeholder "PUBLICATION_NAME" >}} FOR TABLE customers, products, orders; + ``` + + ```output + CREATE PUBLICATION + ``` + + {{< note type="secondary" title="Alternative: Publish All Tables" isCollapsible=yes >}} + You can also create a publication for *all* tables in the database: + + ```command {title="Source psql Prompt"} + CREATE PUBLICATION {{< placeholder "PUBLICATION_NAME" >}} FOR ALL TABLES; + ``` + {{< /note >}} + +1. Run the following command to view all existing publications: + + ```command {title="Source psql Prompt"} + SELECT * FROM pg_publication_tables; + ``` + + ```output + pubname | schemaname | tablename | attnames + ----------------+------------+-----------+------------------------------------------------------- + my_publication | public | customers | {customer_id,name,email,created_at} + my_publication | public | products | {product_id,name,price,created_at} + my_publication | public | orders | {order_id,customer_id,product_id,quantity,created_at} + (3 rows) + ``` + +1. Type `\q` and press Enter to exit the source `psql` shell. + +Your RDS source database is now ready for logical replication. Return to [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) to configure the Linode Managed Database and create the subscription. \ No newline at end of file diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-firewall-rule-linode-managed-database.png b/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-firewall-rule-linode-managed-database.png new file mode 100644 index 00000000000..a64214d6982 Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-firewall-rule-linode-managed-database.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-navigation-server-parameters.png b/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-navigation-server-parameters.png new file mode 100644 index 00000000000..203d6a8c7e0 Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-navigation-server-parameters.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-networking-public-access.png b/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-networking-public-access.png new file mode 100644 index 00000000000..6e0b3de102c Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-networking-public-access.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-parameter-wal-level-default.png b/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-parameter-wal-level-default.png new file mode 100644 index 00000000000..d6a29891513 Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-parameter-wal-level-default.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-parameters-save-logical-replication.png b/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-parameters-save-logical-replication.png new file mode 100644 index 00000000000..276ec316b12 Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/azure-flexible-server-parameters-save-logical-replication.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/index.md b/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/index.md new file mode 100644 index 00000000000..5c9bd3e5f0d --- /dev/null +++ b/docs/guides/platform/migrate-to-linode/preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database/index.md @@ -0,0 +1,338 @@ +--- +slug: preparing-your-azure-postgresql-database-for-logical-replication-to-linode-managed-database +title: "Preparing Your Azure PostgreSQL Database for Logical Replication to Linode Managed Database" +description: "Prepare your Azure Database for PostgreSQL for logical replication to a Linode Managed Database. Configure server parameters, network access, and a replication user." +authors: ["Akamai"] +contributors: ["Akamai"] +published: 2026-02-18 +keywords: ['azure postgresql logical replication','linode managed database','flexible server','pg_publication','azure replication setup'] +license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' +external_resources: +- '[Logical replication and logical decoding in Azure Database for PostgreSQL](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-logical)' +- '[Server parameters in Azure Database for PostgreSQL](https://learn.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-server-parameters)' +- '[Azure CLI Documentation for `az postgres`](https://learn.microsoft.com/en-us/cli/azure/postgres?view=azure-cli-latest)' +--- + +[Logical replication](https://www.postgresql.org/docs/current/logical-replication.html) continuously synchronizes database tables, allowing you to prepare the destination database in advance. This approach minimizes downtime when you switch application traffic and retire the source database. + +This guide explains how to prepare an Azure Database for PostgreSQL for logical replication to a [Linode Managed Database](https://www.linode.com/products/databases/). Follow this guide before returning to the [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) guide to [create the subscription](https://www.postgresql.org/docs/current/sql-createsubscription.html) on Akamai Cloud. + +Follow the steps in this guide to: + +- Configure your Azure Database instance to support logical replication. +- Ensure secure network access from Linode. +- Create a dedicated replication user. +- Set up a publication for the tables you wish to replicate. + +After completing these steps, return to [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) to configure the subscriber and finalize the setup. + +## Before You Begin + +1. Follow the [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) guide up to the **Prepare the Source Database for Logical Replication** section to obtain the public IP address or CIDR range of your Linode Managed Database. + +1. Ensure your Azure account has permissions to modify PostgreSQL server parameters, networking settings, and firewall rules. + +1. Install and authenticate the [Azure CLI](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli) on your local machine: + + ```command + az login + az account set --subscription {{< placeholder "YOUR_AZURE_SUBSCRIPTION_ID" >}} + ``` + +{{< note title="Requirement: Flexible Server" >}} +Ensure you have an *Azure Database for PostgreSQL – Flexible Server* instance. Logical replication requires Flexible Server, as Single Server was retired in March 2025. +{{< /note >}} + +### Placeholders and Examples + +The following placeholders and example values are used in commands throughout this guide: + +| Parameter | Placeholder | Example Value | +|------------|--------------|----------------| +| Azure Server Name | {{< placeholder "AZURE_SERVER_NAME" >}} | `source-database` | +| Azure Resource Group | {{< placeholder "AZURE_RESOURCE_GROUP" >}} | `pg-repl-rg` | +| Destination IP Address | {{< placeholder "DEST_IP" >}} | `172.232.188.122` | +| Source Hostname | {{< placeholder "SOURCE_HOST" >}} | `source-database.postgres.database.azure.com` | +| Source Port | {{< placeholder "SOURCE_PORT" >}} | `5432` | +| Source Username | {{< placeholder "SOURCE_USER" >}} | `azureadmin` | +| Source Database | {{< placeholder "SOURCE_DB" >}} | `postgres` | +| Source Password | {{< placeholder "SOURCE_PASSWORD" >}} | `thisismysourcepassword` | +| Replication Username | {{< placeholder "REPL_USER" >}} | `linode_replicator` | +| Replication Password | {{< placeholder "REPL_PASSWORD" >}} | `thisismyreplicatorpassword` | +| Publication Name | {{< placeholder "PUBLICATION_NAME" >}} | `my_publication` | + +Replace these placeholders with your own connection details when running commands in your environment. + +Additionally, the examples used in this guide assume the source database contains three tables (`customers`, `products`, and `orders`) that you want to replicate to a Linode Managed Database. + +## Configure Server Parameters + +To support logical replication, you’ll need to adjust a few parameters on your Azure Database for PostgreSQL instance. + +{{< tabs >}} +{{< tab "Azure Portal" >}} +1. In the Azure Portal, locate your database resource, then navigate to **Settings > Server parameters**: + + ![Azure Database for PostgreSQL Flexible Server - Navigation menu with Server parameters selected.](azure-flexible-server-navigation-server-parameters.png) + +1. In the list of server parameters, use the search filter to find the values for `wal_level`, `max_replication_slots`, and `max_wal_senders`: + + ![Azure portal Server parameters page showing the wal_level parameter set to REPLICA by default.](azure-flexible-server-parameter-wal-level-default.png) + + In order for logical replication to succeed, these values should be as follows: + + - `max_replication_slots`: Greater than or equal to 1 + - `max_wal_senders`: Greater than or equal to `max_replication_slots`, depending on expected replication concurrency + - `wal_level`: `LOGICAL` + + If these values are already set correctly, skip the remainder of this section and continue with Configure Network Access. Otherwise, you need to modify the parameter group using the instructions below. + +1. Adjust the values as needed, then click **Save**: + + ![Azure portal Server parameters page showing wal_level set to LOGICAL, max_wal_senders and max_replication_slots set to 5, with the Save button highlighted.](azure-flexible-server-parameters-save-logical-replication.png) + +1. When Azure notifies you that you need to restart the server for changes to take effect, click **Save and Restart**. + +{{< /tab >}} +{{< tab "Azure CLI" >}} +1. Run the following `az` CLI command to list the relevant server parameters for the instance. Replace {{< placeholder "AZURE_SERVER_NAME" >}} with your Azure PostgreSQL server name (e.g., `source-database`) and {{< placeholder "AZURE_RESOURCE_GROUP" >}} with your resource group (e.g., `pg-repl-rg`): + + ```command + az postgres flexible-server parameter list \ + --server-name {{< placeholder "AZURE_SERVER_NAME" >}} \ + --resource-group {{< placeholder "AZURE_RESOURCE_GROUP" >}} \ + --output json \ + --query "[?name=='wal_level' || name=='max_replication_slots' || name=='max_wal_senders'].{name:name, description:description, dataType:dataType, value:value}" + ``` + + ```output + [ + { + "dataType": "Integer", + "description": "Specifies the maximum number of replication slots that the server can support.", + "name": "max_replication_slots", + "value": "10" + }, + { + "dataType": "Integer", + "description": "Sets the maximum number of simultaneously running WAL sender processes.", + "name": "max_wal_senders", + "value": "10" + }, + { + "dataType": "Enumeration", + "description": "It determines how much information is written to the WAL.", + "name": "wal_level", + "value": "REPLICA" + } + ] + ``` + + In order for logical replication to succeed, these values should be as follows: + + - `max_replication_slots`: Greater than or equal to 1 + - `max_wal_senders`: Greater than or equal to `max_replication_slots`, depending on expected replication concurrency + - `wal_level`: `LOGICAL` + + If these values are already set correctly, skip the remainder of this section and continue with Configure Network Access. Otherwise, you need to modify the parameter group using the instructions below. + + To adjust the values with the Azure CLI, you need to run the `parameter set` command for each parameter. + +1. Use the following command to adjust the value of `max_replication_slots` to `10`: + + ```command + az postgres flexible-server parameter set \ + --server-name {{< placeholder "AZURE_SERVER_NAME" >}} \ + --resource-group {{< placeholder "AZURE_RESOURCE_GROUP" >}} \ + --name max_replication_slots \ + --value 10 + ``` + +1. Use the following command to adjust the value of `max_wal_senders` to `10`: + + ```command + az postgres flexible-server parameter set \ + --server-name {{< placeholder "AZURE_SERVER_NAME" >}} \ + --resource-group {{< placeholder "AZURE_RESOURCE_GROUP" >}} \ + --name max_wal_senders \ + --value 10 + ``` + +1. Use the following command to adjust the value of `wal_level` to `logical`: + + ```command {title="Modify server parameters to support logical replication"} + az postgres flexible-server parameter set \ + --server-name {{< placeholder "AZURE_SERVER_NAME" >}} \ + --resource-group {{< placeholder "AZURE_RESOURCE_GROUP" >}} \ + --name wal_level \ + --value logical + ``` + +1. After modifying these parameters, restart the database instance: + + ```command + az postgres flexible-server restart \ + --name {{< placeholder "AZURE_SERVER_NAME" >}} \ + --resource-group {{< placeholder "AZURE_RESOURCE_GROUP" >}} + ``` +{{< /tab >}} +{{< /tabs >}} + +## Configure Network Access + +Before the Linode Managed Database can connect to your Azure Database instance, ensure that the instance allows network access from the Linode Managed Database. + +{{< tabs >}} +{{< tab "Azure Portal" >}} +1. Navigate to the **Settings > Networking** page for the instance. Make sure that the **Public access** option is checked. + + ![Azure Database for PostgreSQL Networking page showing the Public access option enabled.](azure-flexible-server-networking-public-access.png) + +1. In the list of firewall rules, add a rule to allow access to your Linode Managed Database. Specify a name for the firewall rule. Enter the IP address of your Linode Managed Database as both the **Start IP address** and the **End IP address**: + + ![Azure PostgreSQL Networking page showing a firewall rule named allow-linode-managed-database with the Linode host IP entered for Start and End addresses.](azure-flexible-server-firewall-rule-linode-managed-database.png) + +1. Click **Save** at the top of the page. +{{< /tab >}} +{{< tab "Azure CLI" >}} +1. Determine if public network access is enabled: + + ```command + az postgres flexible-server show \ + --resource-group {{< placeholder "AZURE_RESOURCE_GROUP" >}} \ + --name {{< placeholder "AZURE_SERVER_NAME" >}} \ + --query "network.publicNetworkAccess" + ``` + + If the output is `"Enabled"`, skip the following command: + + ```output + "Enabled" + ``` + +1. If the output is `"Disabled"`, use the following command to enable public network access: + + ```command + az postgres flexible-server update \ + --resource-group {{< placeholder "AZURE_RESOURCE_GROUP" >}} \ + --name {{< placeholder "AZURE_SERVER_NAME" >}} \ + --public-network-access Enabled + ``` + +1. Add a firewall rule allowing access from your Linode Managed Database. Replace {{< placeholder "DEST_IP" >}} with the IP address from [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) (e.g., `172.232.188.122`): + + ```command + az postgres flexible-server firewall-rule create \ + --resource-group {{< placeholder "AZURE_RESOURCE_GROUP" >}} \ + --name {{< placeholder "AZURE_SERVER_NAME" >}} \ + --rule-name allow-linode-managed-database \ + --start-ip-address {{< placeholder "DEST_IP" >}} \ + --end-ip-address {{< placeholder "DEST_IP" >}} + ``` + +1. Verify the firewall rules: + + ```command + az postgres flexible-server firewall-rule list \ + --resource-group {{< placeholder "AZURE_RESOURCE_GROUP" >}} \ + --name {{< placeholder "AZURE_SERVER_NAME" >}} \ + -o table + ``` + + ```output + EndIpAddress Name ResourceGroup StartIpAddress + --------------- ----------------------------- --------------- ---------------- + 172.235.145.182 allow-linode-managed-database pg-repl-rg 172.235.145.182 + ``` +{{< /tab >}} +{{< /tabs >}} + +With network access configured, your Linode Managed Database can reach the Azure Database instance during the subscription creation step in [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/). + +## Create a Replication User + +While logical replication can technically be performed using the primary database user, it's best practice to create a dedicated replication user. This user should have the `REPLICATION` privilege and `SELECT` access only to the tables being published. + +Follow the steps below to create this dedicated user on your Azure Database instance. + +1. Connect to your source PostgreSQL instance using the `psql` client. Replace {{< placeholder "SOURCE_HOST" >}} (e.g., `source-database.postgres.database.azure.com`), {{< placeholder "SOURCE_PORT" >}} (e.g., `5432`), {{< placeholder "SOURCE_USER" >}} (e.g., `azureadmin`), and {{< placeholder "SOURCE_DB" >}} (e.g., `postgres`) with your own values. You can find the connection details on the **Settings > Connect** page of your Azure Database instance. + + ```command + psql \ + -h {{< placeholder "SOURCE_HOST" >}} \ + -p {{< placeholder "SOURCE_PORT" >}} \ + -U {{< placeholder "SOURCE_USER" >}} \ + -d {{< placeholder "SOURCE_DB" >}} \ + "sslmode=require" + ``` + + When prompted, enter your {{< placeholder "SOURCE_PASSWORD" >}} (e.g., `thisismysourcepassword`). + +1. Run the following commands from the source `psql` prompt. Replace {{< placeholder "REPL_USER" >}} (e.g., `linode_replicator`) and {{< placeholder "REPL_PASSWORD" >}} (e.g., `thisismyreplicatorpassword`) with your own values. For simplicity, this example assumes a public schema and three sample tables (customers, products, and orders). Replace the table names with your actual schema as needed. + + ```command {title="Source psql Prompt"} + CREATE ROLE {{< placeholder "REPL_USER" >}} + WITH REPLICATION + LOGIN PASSWORD '{{< placeholder "REPL_PASSWORD" >}}'; + GRANT SELECT ON customers, products, orders TO {{< placeholder "REPL_USER" >}}; + ``` + + ```output + CREATE ROLE + GRANT + ``` + + {{< note type="secondary" title="Alternative: Grant on All Tables" isCollapsible=yes >}} + You can also grant privileges on *all* tables with the following command: + + ```command {title="Source psql Prompt"} + GRANT SELECT ON ALL TABLES IN SCHEMA public TO {{< placeholder "REPL_USER" >}}; + ``` + + ```output + GRANT + ``` + {{< /note >}} + +The newly created user is referenced by the Linode Managed Database when creating the subscription in [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/). + +## Create a Publication + +A publication defines which tables and changes (e.g., `INSERT`, `UPDATE`, and `DELETE`) should be streamed to the subscriber. At least one publication is required for logical replication, and the subscriber must have matching tables with compatible schemas for replication to succeed. + +1. While still connected to your source database via the `psql` client, use the following command to create a publication. Replace {{< placeholder "PUBLICATION_NAME" >}} (e.g., `my_publication`) and the specific tables you want to replicate (e.g., `customers`, `products`, and `orders`): + + ```command {title="Source psql Prompt"} + CREATE PUBLICATION {{< placeholder "PUBLICATION_NAME" >}} FOR TABLE customers, products, orders; + ``` + + ```output + CREATE PUBLICATION + ``` + + {{< note type="secondary" title="Alternative: Publish All Tables" isCollapsible=yes >}} + You can also create a publication for *all* tables in the database: + + ```command {title="Source psql Prompt"} + CREATE PUBLICATION {{< placeholder "PUBLICATION_NAME" >}} FOR ALL TABLES; + ``` + {{< /note >}} + +1. Run the following command to view all existing publications: + + ```command {title="Source psql Prompt"} + SELECT * FROM pg_publication_tables; + ``` + + ```output + pubname | schemaname | tablename | attnames | rowfilter + ----------------+------------+-----------+-------------------------------------------------------+----------- + my_publication | public | customers | {customer_id,name,email,created_at} | + my_publication | public | products | {product_id,name,price,created_at} | + my_publication | public | orders | {order_id,customer_id,product_id,quantity,created_at} | + (3 rows) + ``` + +1. Type `\q` and press Enter to exit the source `psql` shell. + +Your Azure source database is now ready for logical replication. Return to [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) to configure the Linode Managed Database and create the subscription. \ No newline at end of file diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-authorized-networks-add-rule.png b/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-authorized-networks-add-rule.png new file mode 100644 index 00000000000..f9af56f2455 Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-authorized-networks-add-rule.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-console-instance-overview.png b/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-console-instance-overview.png new file mode 100644 index 00000000000..818e1547323 Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-console-instance-overview.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-database-flags-configuration.png b/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-database-flags-configuration.png new file mode 100644 index 00000000000..276db86580d Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-database-flags-configuration.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-networking-public-ip.png b/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-networking-public-ip.png new file mode 100644 index 00000000000..6d40ff8a884 Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-networking-public-ip.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-restart-instance-apply-flags.png b/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-restart-instance-apply-flags.png new file mode 100644 index 00000000000..9a9861b722f Binary files /dev/null and b/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/cloud-sql-restart-instance-apply-flags.png differ diff --git a/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/index.md b/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/index.md new file mode 100644 index 00000000000..b8dfcdde165 --- /dev/null +++ b/docs/guides/platform/migrate-to-linode/preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database/index.md @@ -0,0 +1,261 @@ +--- +slug: preparing-your-google-cloud-sql-postgresql-database-for-logical-replication-to-linode-managed-database +title: "Preparing Your Google Cloud SQL PostgreSQL Database for Logical Replication to Linode Managed Database" +description: "Prepare your Google Cloud SQL for PostgreSQL database for logical replication to a Linode Managed Database. Configure server parameters, network access, and a replication user." +authors: ["Akamai"] +contributors: ["Akamai"] +published: 2026-02-18 +keywords: ['Google Cloud SQL', 'PostgreSQL', 'logical replication', 'Cloud SQL replication', 'Linode Managed Database', 'publication', 'replication user', 'database flags'] +license: '[CC BY-ND 4.0](https://creativecommons.org/licenses/by-nd/4.0)' +external_resources: +- '[Set up logical replication and decoding](https://cloud.google.com/sql/docs/postgres/replication/configure-logical-replication)' +- '[Cloud SQL for PostgreSQL Documentation](https://cloud.google.com/sql/docs/postgres)' +- '[gcloud sql CLI Documentation](https://cloud.google.com/sdk/gcloud/reference/sql)' +--- + +[Logical replication](https://www.postgresql.org/docs/current/logical-replication.html) continuously synchronizes database tables, allowing you to prepare the destination database in advance. This approach minimizes downtime when you switch application traffic and retire the source database. + +This guide explains how to prepare a Google Cloud SQL for PostgreSQL for logical replication to a [Linode Managed Database](https://www.linode.com/products/databases/). Follow this guide before returning to the [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) guide to [create the subscription](https://www.postgresql.org/docs/current/sql-createsubscription.html) on Akamai Cloud. + +Follow the steps in this guide to: + +- Configure your Cloud SQL instance to support logical replication. +- Ensure secure network access from Linode. +- Create a dedicated replication user. +- Set up a publication for the tables you wish to replicate. + +After completing these steps, return to [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) to configure the subscriber and finalize the setup. + +## Before You Begin + +1. Follow the [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) guide up to the **Prepare the Source Database for Logical Replication** section to obtain the public IP address or CIDR range of your Linode Managed Database. + +1. Ensure that you have administrative access to your GCP project, including permissions to modify Cloud SQL instance flags and authorized networks. + +1. Install and authenticate the Google Cloud CLI (`gcloud`) on your local machine. + +### Placeholders and Examples + +The following placeholders and example values are used in commands throughout this guide: + +| Parameter | Placeholder | Example Value | +|------------|--------------|----------------| +| GCP Instance Name | {{< placeholder "GCP_INSTANCE_NAME" >}} | `source-database` | +| Destination IP Address | {{< placeholder "DEST_IP" >}} | `172.232.188.122` | +| Source IP Address | {{< placeholder "SOURCE_HOST" >}} | `35.227.90.130` | +| Source Port | {{< placeholder "SOURCE_PORT" >}} | `5432` | +| Source Username | {{< placeholder "SOURCE_USER" >}} | `postgres` | +| Source Database | {{< placeholder "SOURCE_DB" >}} | `postgres` | +| Source Password | {{< placeholder "SOURCE_PASSWORD" >}} | `thisismysourcepassword` | +| Replication Username | {{< placeholder "REPL_USER" >}} | `linode_replicator` | +| Replication Password | {{< placeholder "REPL_PASSWORD" >}} | `thisismyreplicatorpassword` | +| Publication Name | {{< placeholder "PUBLICATION_NAME" >}} | `my_publication` | + +Replace these placeholders with your own connection details when running commands in your environment. + +Additionally, the examples used in this guide assume the source database contains three tables (`customers`, `products`, and `orders`) that you want to replicate to a Linode Managed Database. + +## Configure Database Flags + +Logical replication requires enabling specific PostgreSQL flags on your Cloud SQL for PostgreSQL instance. These flags can be configured using either the Google Cloud Console or the `gcloud` CLI. + +{{< tabs >}} +{{< tab "Google Cloud Console" >}} +1. In the Google Cloud Console, navigate to **SQL** and select your PostgreSQL instance: + + ![Google Cloud SQL instance overview in console.](cloud-sql-console-instance-overview.png) + +1. On the instance page, click **Edit**. + +1. Locate the **Flags and parameters** section, then click **Add a database flag**. + +1. Add the following flags: + + - `cloudsql.logical_decoding`: `On` (sets `wal_level` to `logical`) + - `max_replication_slots`: `10` or higher + - `max_wal_senders`: Greater than or equal to `max_replication_slots`, depending on expected replication concurrency + + ![Database flags configuration screen in Cloud SQL console.](cloud-sql-database-flags-configuration.png) + +1. Click **Save** at the bottom of the page. + +1. When prompted, click **Save and restart** to restart and apply the changes: + + ![Restart Cloud SQL instance after setting flags.](cloud-sql-restart-instance-apply-flags.png) + +{{< /tab >}} +{{< tab "gcloud CLI" >}} +Run the following `gcloud` command to set Cloud SQL instance database flags from the CLI. Replace {{< placeholder "GCP_INSTANCE_NAME" >}} with your Cloud SQL instance name (e.g., `source-database`) + +```command +gcloud sql instances patch {{< placeholder "GCP_INSTANCE_NAME" >}} \ + --database-flags=cloudsql.logical_decoding=on,max_replication_slots=10,max_wal_senders=10 +``` + +```output +The following message will be used for the patch API method. + +{ + "name": "source-database", + "settings": { + "databaseFlags": [ + {"name": "cloudsql.logical_decoding", "value": "off"}, + {"name": "max_replication_slots", "value": "10"}, + {"name": "max_wal_senders", "value": "10"} + ] + } +} + +WARNING: This patch modifies database flag values, which may require your +instance to be restarted. Check the list of supported flags - +https://cloud.google.com/sql/docs/postgres/flags - to see if your +instance will be restarted when this patch is submitted. + +Do you want to continue (Y/n)? +``` + +Confirm the request to restart the instance. +{{< /tab >}} +{{< /tabs >}} + +## Configure Network Access + +Ensure that your Cloud SQL instance allows network access from the Linode Managed Database. + +{{< tabs >}} +{{< tab "Google Cloud Console" >}} +1. In the Google Cloud Console, open your Cloud SQL instance. + +1. Navigate to the **Connections** page, then select the **Networking** tab. + +1. Ensure that the **Public IP** option is checked: + + ![Networking tab showing Public IP enabled in Cloud SQL instance.](cloud-sql-networking-public-ip.png) + +1. In the list of Authorized networks, add the CIDR range of your Linode Managed Database: + + ![Authorized networks list with Linode IP added.](cloud-sql-authorized-networks-add-rule.png) + +1. Click **Save** at the bottom of the page. +{{< /tab >}} +{{< tab "gcloud CLI" >}} +You can also configure authorized networks using the `gcloud` CLI. However, you can only specify a CIDR range (as a comma-separated list) and cannot assign a name for each network. + +Add a firewall rule allowing access from your Linode Managed Database. Replace {{< placeholder "DEST_IP" >}} with the IP address from [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) (e.g., `172.232.188.122`): + +```command +gcloud sql instances patch {{< placeholder "GCP_INSTANCE_NAME" >}} \ + --authorized-networks="{{< placeholder "DEST_IP" >}}/32" +``` + +{{< note type="warning" title="Warning: Existing Authorized Networks Are Replaced" >}} +The `--authorized-networks` flag *replaces* any existing authorized networks on the instance. If other networks are already configured, you must include them in the comma-separated list, for example: + +```command +gcloud sql instances patch {{< placeholder "GCP_INSTANCE_NAME" >}} \ + --authorized-networks="172.232.188.122/32,172.232.189.35/32" +``` +{{< /note >}} +{{< /tab >}} +{{< /tabs >}} + +With network access configured, your Linode Managed Database can reach the Cloud SQL instance during the subscription creation step in [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/). + +## Create a Replication User + +While logical replication can technically be performed using the primary database user, it's best practice to create a dedicated replication user. This user should have the `REPLICATION` privilege and `SELECT` access only to the tables being published. + +Follow the steps below to create this dedicated user on your Cloud SQL instance. + +1. Connect to your source PostgreSQL instance using the `psql` client. Replace {{< placeholder "SOURCE_HOST" >}} (e.g., `35.227.90.130`), {{< placeholder "SOURCE_PORT" >}} (e.g., `5432`), {{< placeholder "SOURCE_USER" >}} (e.g., `postgres`), and {{< placeholder "SOURCE_DB" >}} (e.g., `postgres`) with your own values. You can find the connection details under **Connections > Summary** in the Cloud SQL console. + + + ```command + psql \ + -h {{< placeholder "SOURCE_HOST" >}} \ + -p {{< placeholder "SOURCE_PORT" >}} \ + -U {{< placeholder "SOURCE_USER" >}} \ + -d {{< placeholder "SOURCE_DB" >}} \ + "sslmode=require" + ``` + + When prompted, enter your {{< placeholder "SOURCE_PASSWORD" >}} (e.g., `thisismysourcepassword`). + +1. Run the following commands from the source `psql` prompt. Replace {{< placeholder "REPL_USER" >}} (e.g., `linode_replicator`) and {{< placeholder "REPL_PASSWORD" >}} (e.g., `thisismyreplicatorpassword`) with your own values. For simplicity, this example assumes a public schema and three sample tables (customers, products, and orders). Replace the table names with your actual schema as needed. + + ```command {title="Source psql Prompt"} + CREATE ROLE {{< placeholder "REPL_USER" >}} + WITH REPLICATION + LOGIN PASSWORD '{{< placeholder "REPL_PASSWORD" >}}'; + GRANT SELECT ON customers, products, orders TO {{< placeholder "REPL_USER" >}}; + ``` + + ```output + CREATE ROLE + GRANT + ``` + + {{< note type="secondary" title="Alternative: Grant on All Tables" isCollapsible=yes >}} + You can also grant privileges on *all* tables with the following command: + + ```command {title="Source psql Prompt"} + GRANT SELECT ON ALL TABLES IN SCHEMA public TO {{< placeholder "REPL_USER" >}}; + ``` + + ```output + GRANT + ``` + {{< /note >}} + +The newly created user is referenced by the Linode Managed Database when creating the subscription in [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/). + +## Create a Publication + +A publication defines which tables and changes (e.g., `INSERT`, `UPDATE`, and `DELETE`) should be streamed to the subscriber. At least one publication is required for logical replication, and the subscriber must have matching tables with compatible schemas for replication to succeed. + +1. While still connected to your source database via the `psql` client, use the following command to create a publication. Replace {{< placeholder "PUBLICATION_NAME" >}} (e.g., `my_publication`) and the specific tables you want to replicate (e.g., `customers`, `products`, and `orders`): + + ```command {title="Source psql Prompt"} + CREATE PUBLICATION {{< placeholder "PUBLICATION_NAME" >}} FOR TABLE customers, products, orders; + ``` + + ```output + CREATE PUBLICATION + ``` + + {{< note type="secondary" title="Alternative: Publish All Tables" isCollapsible=yes >}} + You can also create a publication for *all* tables in the database: + + ```command {title="Source psql Prompt"} + CREATE PUBLICATION {{< placeholder "PUBLICATION_NAME" >}} FOR ALL TABLES; + ``` + {{< /note >}} + +1. Run the following command to view all existing publications: + + ```command {title="Source psql Prompt"} + SELECT * FROM pg_publication_tables; + ``` + + ```output + -[ RECORD 1 ]----------------------------------------------- + pubname | my_publication + schemaname | public + tablename | customers + attnames | {id,name,email,created_at} + rowfilter | + -[ RECORD 2 ]----------------------------------------------- + pubname | my_publication + schemaname | public + tablename | products + attnames | {id,name,price,in_stock} + rowfilter | + -[ RECORD 3 ]----------------------------------------------- + pubname | my_publication + schemaname | public + tablename | orders + attnames | {id,customer_id,product_id,quantity,order_date} + rowfilter | + ``` + +Your Google Cloud source database is now ready for logical replication. Return to [Logical Replication to a Linode Managed PostgreSQL Database](/docs/guides/logical-replication-to-a-linode-managed-postgresql-database/) to configure the Linode Managed Database and create the subscription. \ No newline at end of file