From 5dea41aee7509601fc85fc05af68a3a64e13adbe Mon Sep 17 00:00:00 2001 From: shiyuhang <1136742008@qq.com> Date: Sat, 28 Feb 2026 12:41:53 +0800 Subject: [PATCH 01/18] add msk doc --- ...s-private-link-connection-to-amazon-msk.md | 167 ++++++++++++++++++ .../serverless-private-link-connection.md | 50 ++++++ 2 files changed, 217 insertions(+) create mode 100644 tidb-cloud/serverless-private-link-connection-to-amazon-msk.md diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md new file mode 100644 index 0000000000000..c4447b1ddc223 --- /dev/null +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -0,0 +1,167 @@ +--- +title: Connect to Amazon MSK Provisioned via a Private Link Connection +summary: Learn how to connect to an Amazon MSK Provisioned cluster using an Amazon MSK Provisioned private link connection. +--- + +# Connect to Amazon MSK Provisioned via a Private Link Connection + +This document describes how to connect a {{{ .essential }}} cluster to an [Amazon MSK Provisioned](https://docs.aws.amazon.com/msk/latest/developerguide/msk-provisioned.html) cluster using an [Amazon MSK Provisioned private link connection](/tidb-cloud/serverless-private-link-connection.md#create-an-amazon-msk-provisioned-private-link-connection). + +## Limitations and requirements + +Before you begin, ensure the following: + +- **Region and AZ**: Your Amazon MSK Provisioned cluster is in the same AWS region as your {{{ .essential }}} cluster, and the availability zones of the MSK cluster is the same as your TiDB Cloud cluster. + +- **Authentication**: [SASL/SCRAM authentication](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password.html) is required for the MSK cluster. + +- **Broker type**: Do not use the `t4.small` broker type. It does not support private link. + +See [Amazon MSK multi-VPC private connectivity in a single Region](https://docs.aws.amazon.com/msk/latest/developerguide/aws-access-mult-vpc.html#mvpc-requirements) for more requirements. + +## Prerequisites + +- Your {{{ .essential }}} is hosted on AWS and is active. Retrieve and save the following for later use: + - AWS Account ID + - Availability zones (AZ) + +To view the AWS account ID and availability zones: + +1. In the [TiDB Cloud console](https://tidbcloud.com), go to the cluster overview page of your TiDB cluster, and then click **Settings** > **Networking** in the left navigation pane. +2. In the **Private Link Connection For Dataflow** area, click **Create Private Link Connection**. +3. In the dialog, note the AWS account ID and availability zones. + +## Step 1. Create an MSK cluster (if needed) + +If you do not have an Amazon MSK Provisioned cluster, [create one](https://docs.aws.amazon.com/msk/latest/developerguide/create-cluster.html) in the same region as your {{{ .essential }}} cluster and in the same availability zones as your TiDB Cloud cluster. + +## Step 2. Set up SASL/SCRAM authentication and ACLs + +### Step 2.1. Set up SASL/SCRAM authentication + +Configure [SASL/SCRAM authentication](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password-tutorial.html) for your MSK cluster. + +- **Secret name**: The secret name must start with `AmazonMSK_`. +- **Encryption**: Do not use the default encryption key; create a new one for the secret. + +### Step 2.2. Set ACLs for TiDB Cloud access + +You must set ACLs so that TiDB Cloud can access your MSK cluster. You can use either of the following methods: + +- **SASL/SCRAM (recommended)**: Set ACLs in your VPC using SASL/SCRAM authentication. See [Create ACLs using SASL/SCRAM](#create-acls-using-saslscram). +- **IAM**: Set ACLs in your VPC using IAM authentication. See [Create ACLs using IAM](#create-acls-using-iam). + +## Step 3. Configure the MSK cluster + +Update the following cluster configuration properties: + +- Set `auto.create.topics.enable=true`. +- Add `allow.everyone.if.no.acl.found=false` (required for SASL/SCRAM). +- Keep other properties unchanged or adjust them as needed. + +Apply the changes and wait for the cluster status to change from **Updating** to **Active**. + +## Step 4. Attach the cluster policy + +[Attach the cluster policy](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-cluster-owner-action-policy.html) to allow TiDB Cloud to connect to your MSK cluster. Use the TiDB Cloud AWS account ID you obtained in [Prerequisites](#prerequisites). + +## Step 5. Turn on multi-VPC connectivity + +After the cluster is active, [turn on multi-VPC connectivity](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-cluster-owner-action-turn-on.html) for the MSK cluster. Multi-VPC connectivity is required for private link and requires SASL/SCRAM authentication. + +Wait for the cluster status to change from **Updating** to **Active** again. + +## Step 6. Create an Amazon MSK Provisioned private link connection in TiDB Cloud + +Create the private link connection in TiDB Cloud using the arn of your MSK cluster. + +For detailed steps, see [Create an Amazon MSK Provisioned private link connection](/tidb-cloud/serverless-private-link-connection.md#create-an-amazon-msk-provisioned-private-link-connection). + +--- + +## Create ACLs using SASL/SCRAM + +Use this method to create ACLs in the same VPC as your MSK cluster using SASL/SCRAM authentication. + +1. Create an EC2 instance (Linux) in the VPC where your MSK cluster is located and SSH to it. + +2. Download Kafka and OpenJDK: + + ```shell + wget https://archive.apache.org/dist/kafka/3.7.1/kafka_2.13-3.7.1.tgz + tar -zxf kafka_2.13-3.7.1.tgz + wget https://download.java.net/java/GA/jdk22.0.2/c9ecb94cd31b495da20a27d4581645e8/9/GPL/openjdk-22.0.2_linux-x64_bin.tar.gz + tar -zxf openjdk-22.0.2_linux-x64_bin.tar.gz + ``` + +3. Set the environment (replace the path with your own if different): + + ```shell + export PATH=$PATH:/home/ec2-user/jdk-22.0.2/bin + ``` + +4. Create a file named `scram-client.properties` with the following content. Replace `username` and `pswd` with your SASL/SCRAM credentials: + + ```properties + security.protocol=SASL_SSL + sasl.mechanism=SCRAM-SHA-512 + sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \ + username="username" \ + password="pswd"; + ``` + +5. Create the ACLs. Replace `bootstrap-server` with your MSK bootstrap server address and port (for example, `b-2.xxxxx.c18.kafka.us-east-1.amazonaws.com:9096`), and replace the path to Kafka if needed: + + ```shell + /home/ec2-user/kafka_2.13-3.7.1/bin/kafka-acls.sh --bootstrap-server --command-config scram-client.properties --add --allow-principal User: --operation All --topic '*' + /home/ec2-user/kafka_2.13-3.7.1/bin/kafka-acls.sh --bootstrap-server --command-config scram-client.properties --add --allow-principal User: --operation All --group '*' + /home/ec2-user/kafka_2.13-3.7.1/bin/kafka-acls.sh --bootstrap-server --command-config scram-client.properties --add --allow-principal User: --operation All --cluster '*' + ``` + + The principal `User:` is the SASL/SCRAM user that TiDB Cloud uses to access your MSK cluster. Use the username you configured for TiDB Cloud in your MSK ACLs. + +--- + +## Create ACLs using IAM + +As an alternative to SASL/SCRAM, you can create ACLs in the same VPC as your MSK cluster using IAM authentication. The IAM user or role must have **Amazon MSK** and **Apache Kafka APIs for MSK** permissions. + +1. Create an EC2 instance (Linux) in the VPC where your MSK cluster is located and SSH to it. + +2. Download Kafka, OpenJDK, and the AWS MSK IAM auth JAR: + + ```shell + wget https://archive.apache.org/dist/kafka/3.7.1/kafka_2.13-3.7.1.tgz + tar -zxf kafka_2.13-3.7.1.tgz + wget https://download.java.net/java/GA/jdk22.0.2/c9ecb94cd31b495da20a27d4581645e8/9/GPL/openjdk-22.0.2_linux-x64_bin.tar.gz + tar -zxf openjdk-22.0.2_linux-x64_bin.tar.gz + wget https://github.com/aws/aws-msk-iam-auth/releases/download/v2.3.5/aws-msk-iam-auth-2.3.5-all.jar + ``` + +3. Set the environment (replace paths and credentials with your own): + + ```shell + export PATH=$PATH:/home/ec2-user/jdk-22.0.2/bin + export CLASSPATH=/home/ec2-user/aws-msk-iam-auth-2.3.5-all.jar + export AWS_ACCESS_KEY_ID= + export AWS_SECRET_ACCESS_KEY= + ``` + +4. Create a file named `iam-client.properties` with the following content: + + ```properties + security.protocol=SASL_SSL + sasl.mechanism=AWS_MSK_IAM + sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required; + sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler + ``` + +5. Create the ACLs. Replace `bootstrap-server` with your MSK bootstrap server address and port (for example, `b-1.xxxxx.c18.kafka.us-east-1.amazonaws.com:9098`), and replace the path to Kafka if needed: + + ```shell + /home/ec2-user/kafka_2.13-3.7.1/bin/kafka-acls.sh --bootstrap-server --command-config iam-client.properties --add --allow-principal User: --operation All --topic '*' + /home/ec2-user/kafka_2.13-3.7.1/bin/kafka-acls.sh --bootstrap-server --command-config iam-client.properties --add --allow-principal User: --operation All --group '*' + /home/ec2-user/kafka_2.13-3.7.1/bin/kafka-acls.sh --bootstrap-server --command-config iam-client.properties --add --allow-principal User: --operation All --cluster '*' + ``` + + The principal `User:` is the SASL/SCRAM user that TiDB Cloud uses to access your MSK cluster. Use the username you configured for TiDB Cloud in your MSK ACLs. diff --git a/tidb-cloud/serverless-private-link-connection.md b/tidb-cloud/serverless-private-link-connection.md index 61d43b530d8b5..99c4361504a83 100644 --- a/tidb-cloud/serverless-private-link-connection.md +++ b/tidb-cloud/serverless-private-link-connection.md @@ -19,6 +19,10 @@ This type of private link connection enables TiDB Cloud clusters on **AWS** to c The private link connection can access various AWS services, such as RDS instances and Kafka services, by associating them with the endpoint service. +### Amazon MSK Provisioned + +This type of private link connection enables TiDB Cloud clusters on **AWS** to connect to your [Amazon MSK Provisioned](https://docs.aws.amazon.com/msk/latest/developerguide/msk-provisioned.html) with private link. + ### Alibaba Cloud Endpoint Service This type of private link connection enables TiDB Cloud clusters on **Alibaba Cloud** to connect to your [Alibaba Cloud endpoint service](https://www.alibabacloud.com/help/en/privatelink/share-your-service/#51976edba8no7) powered by Alibaba Cloud PrivateLink. @@ -81,6 +85,50 @@ To create a private link connection using the TiDB Cloud CLI: +## Create an Amazon MSK Provisioned private link connection + +You can create an Amazon MSK Provisioned private link connection using the TiDB Cloud console or the TiDB Cloud CLI. + +Ensure that your Amazon MSK Provisioned cluster turn on multi-VPC connectivity. See [Connect to Amazon MSK Provisioned via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md) for more detials. + + +
+ +1. Log in to the [TiDB Cloud console](https://tidbcloud.com/) and navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project. + + > **Tip:** + > + > You can use the combo box in the upper-left corner to switch between organizations, projects, and clusters. + +2. Click the name of your target cluster to go to its overview page, and then click **Settings** > **Networking** in the left navigation pane. + +3. In the **Private Link Connection For Dataflow** area, click **Create Private Link Connection**. + +4. In the **Create Private Link Connection** dialog, enter the required information: + + - **Private Link Connection Name**: enter a name for the private link connection. + - **Connection Type**: select **Amazon MSK Provisioned**. If this option is not displayed, ensure that your cluster is created on AWS. + - **MSK Cluster Arn**: enter the arn of your Amazon MSK Provisioned cluster, for example, `arn:aws:kafka:us-east-1:385595570414:cluster/plc-test/11f9378a-ec59-4e08-bb42-ada47765a4e9-18`. + +5. Click **Create**. + +
+ +
+ +To create a private link connection using the TiDB Cloud CLI: + +1. Run the following command: + + ```shell + ticloud serverless private-link-connection create -c --display-name --type AMAZON_MSK_PROVISIONED --aws.endpoint-service-name + ``` + +2. Go to the detail page of your endpoint service on the [AWS console](https://console.aws.amazon.com). In the **Endpoint Connections** tab, accept the endpoint connection request from TiDB Cloud. + +
+ + ## Create an Alibaba Cloud Endpoint Service private link connection You can create an Alibaba Cloud Endpoint Service private link connection using the TiDB Cloud console or the TiDB Cloud CLI. @@ -147,6 +195,7 @@ Different private link connection types support attaching different domain types |--------------------------------|-------------------------------------------| | AWS Endpoint Service |
  • TiDB Cloud managed (`aws.tidbcloud.com`)
  • Confluent Dedicated (`aws.confluent.cloud`)
| | Alibaba Cloud Endpoint Service | TiDB Cloud managed (`alicloud.tidbcloud.com`) | +| Amazon MSK Provisioned | does not support attach domians | If your domain is not included in this table, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md) to request support. @@ -281,6 +330,7 @@ ticloud serverless private-link-connection delete -c --private-link - [Connect to Confluent Cloud via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-aws-confluent.md) - [Connect to Amazon RDS via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-aws-rds.md) +- [Connect to Amazon MSK Provisioned via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md) - [Connect to Alibaba Cloud ApsaraDB RDS for MySQL via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-alicloud-rds.md) - [Connect to AWS Self-Hosted Kafka via Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-self-hosted-kafka-in-aws.md) - [Connect to Alibaba Cloud Self-Hosted Kafka via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-self-hosted-kafka-in-alicloud.md) From a95ed4a9828bbd732a98a61a46ac4e96966c2523 Mon Sep 17 00:00:00 2001 From: shiyuhang <1136742008@qq.com> Date: Sat, 28 Feb 2026 12:46:10 +0800 Subject: [PATCH 02/18] add toc --- TOC-tidb-cloud-essential.md | 1 + 1 file changed, 1 insertion(+) diff --git a/TOC-tidb-cloud-essential.md b/TOC-tidb-cloud-essential.md index cdeb3c7ca755d..d53eb7ec44d85 100644 --- a/TOC-tidb-cloud-essential.md +++ b/TOC-tidb-cloud-essential.md @@ -160,6 +160,7 @@ - [Connect to Amazon RDS](/tidb-cloud/serverless-private-link-connection-to-aws-rds.md) - [Connect to Alibaba Cloud RDS](/tidb-cloud/serverless-private-link-connection-to-alicloud-rds.md) - [Connect to Confluent Cloud on AWS](/tidb-cloud/serverless-private-link-connection-to-aws-confluent.md) + - [Connect to Amazon MSK Provisioned](/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md) - [Connect to Self-Hosted Kafka on Alibaba Cloud](/tidb-cloud/serverless-private-link-connection-to-self-hosted-kafka-in-alicloud.md) - [Connect to Self-Hosted Kafka on AWS](/tidb-cloud/serverless-private-link-connection-to-self-hosted-kafka-in-aws.md) - Audit Management From 745f1b385dd1bba243d6bd19a9b631ada49cc692 Mon Sep 17 00:00:00 2001 From: shi yuhang <52435083+shiyuhang0@users.noreply.github.com> Date: Sat, 28 Feb 2026 12:47:15 +0800 Subject: [PATCH 03/18] Apply suggestions from code review Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- .../serverless-private-link-connection-to-amazon-msk.md | 4 ++-- tidb-cloud/serverless-private-link-connection.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md index c4447b1ddc223..987a0361d5fad 100644 --- a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -11,7 +11,7 @@ This document describes how to connect a {{{ .essential }}} cluster to an [Amazo Before you begin, ensure the following: -- **Region and AZ**: Your Amazon MSK Provisioned cluster is in the same AWS region as your {{{ .essential }}} cluster, and the availability zones of the MSK cluster is the same as your TiDB Cloud cluster. +- **Region and AZ**: Your Amazon MSK Provisioned cluster is in the same AWS region as your {{{ .essential }}} cluster, and the availability zones of the MSK cluster are the same as your TiDB Cloud cluster. - **Authentication**: [SASL/SCRAM authentication](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password.html) is required for the MSK cluster. @@ -73,7 +73,7 @@ Wait for the cluster status to change from **Updating** to **Active** again. ## Step 6. Create an Amazon MSK Provisioned private link connection in TiDB Cloud -Create the private link connection in TiDB Cloud using the arn of your MSK cluster. +Create the private link connection in TiDB Cloud using the `ARN` of your MSK cluster. For detailed steps, see [Create an Amazon MSK Provisioned private link connection](/tidb-cloud/serverless-private-link-connection.md#create-an-amazon-msk-provisioned-private-link-connection). diff --git a/tidb-cloud/serverless-private-link-connection.md b/tidb-cloud/serverless-private-link-connection.md index 99c4361504a83..54549ceb1a13d 100644 --- a/tidb-cloud/serverless-private-link-connection.md +++ b/tidb-cloud/serverless-private-link-connection.md @@ -108,7 +108,7 @@ Ensure that your Amazon MSK Provisioned cluster turn on multi-VPC connectivity. - **Private Link Connection Name**: enter a name for the private link connection. - **Connection Type**: select **Amazon MSK Provisioned**. If this option is not displayed, ensure that your cluster is created on AWS. - - **MSK Cluster Arn**: enter the arn of your Amazon MSK Provisioned cluster, for example, `arn:aws:kafka:us-east-1:385595570414:cluster/plc-test/11f9378a-ec59-4e08-bb42-ada47765a4e9-18`. + - **MSK Cluster Arn**: Enter the ARN of your Amazon MSK Provisioned cluster, for example, `arn:aws:kafka:us-east-1:385595570414:cluster/plc-test/11f9378a-ec59-4e08-bb42-ada47765a4e9-18`. 5. Click **Create**. @@ -195,7 +195,7 @@ Different private link connection types support attaching different domain types |--------------------------------|-------------------------------------------| | AWS Endpoint Service |
  • TiDB Cloud managed (`aws.tidbcloud.com`)
  • Confluent Dedicated (`aws.confluent.cloud`)
| | Alibaba Cloud Endpoint Service | TiDB Cloud managed (`alicloud.tidbcloud.com`) | -| Amazon MSK Provisioned | does not support attach domians | +| Amazon MSK Provisioned | Does not support attaching domains | If your domain is not included in this table, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md) to request support. From 254be8c3da4082a4514e396287a72472ec366398 Mon Sep 17 00:00:00 2001 From: shiyuhang <1136742008@qq.com> Date: Sat, 28 Feb 2026 12:47:55 +0800 Subject: [PATCH 04/18] msk --- .../serverless-private-link-connection.md | 22 +------------------ 1 file changed, 1 insertion(+), 21 deletions(-) diff --git a/tidb-cloud/serverless-private-link-connection.md b/tidb-cloud/serverless-private-link-connection.md index 54549ceb1a13d..b97179555e512 100644 --- a/tidb-cloud/serverless-private-link-connection.md +++ b/tidb-cloud/serverless-private-link-connection.md @@ -87,13 +87,10 @@ To create a private link connection using the TiDB Cloud CLI: ## Create an Amazon MSK Provisioned private link connection -You can create an Amazon MSK Provisioned private link connection using the TiDB Cloud console or the TiDB Cloud CLI. +You can create an Amazon MSK Provisioned private link connection using the TiDB Cloud console. Ensure that your Amazon MSK Provisioned cluster turn on multi-VPC connectivity. See [Connect to Amazon MSK Provisioned via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md) for more detials. - -
- 1. Log in to the [TiDB Cloud console](https://tidbcloud.com/) and navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project. > **Tip:** @@ -112,23 +109,6 @@ Ensure that your Amazon MSK Provisioned cluster turn on multi-VPC connectivity. 5. Click **Create**. -
- -
- -To create a private link connection using the TiDB Cloud CLI: - -1. Run the following command: - - ```shell - ticloud serverless private-link-connection create -c --display-name --type AMAZON_MSK_PROVISIONED --aws.endpoint-service-name - ``` - -2. Go to the detail page of your endpoint service on the [AWS console](https://console.aws.amazon.com). In the **Endpoint Connections** tab, accept the endpoint connection request from TiDB Cloud. - -
- - ## Create an Alibaba Cloud Endpoint Service private link connection You can create an Alibaba Cloud Endpoint Service private link connection using the TiDB Cloud console or the TiDB Cloud CLI. From eef70f99efdc46b548c7e70ead8c0541783777e0 Mon Sep 17 00:00:00 2001 From: shiyuhang <1136742008@qq.com> Date: Sat, 28 Feb 2026 15:21:24 +0800 Subject: [PATCH 05/18] some opt --- tidb-cloud/essential-changefeed-sink-to-kafka.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tidb-cloud/essential-changefeed-sink-to-kafka.md b/tidb-cloud/essential-changefeed-sink-to-kafka.md index 4326ec396012e..037ecabc2b288 100644 --- a/tidb-cloud/essential-changefeed-sink-to-kafka.md +++ b/tidb-cloud/essential-changefeed-sink-to-kafka.md @@ -33,13 +33,14 @@ Ensure that your {{{ .essential }}} cluster can connect to the Apache Kafka serv Private link connections leverage **Private Link** technologies from cloud providers to enable resources in your VPC to connect to services in other VPCs using private IP addresses, as if those services were hosted directly within your VPC. -{{{ .essential }}} currently supports Private Link connections only for self-hosted Kafka and Confluent Cloud Dedicated clusters. It does not support direct integration with MSK or other Kafka SaaS services. +{{{ .essential }}} currently supports Private Link connections only for self-hosted Kafka, Confluent Cloud Dedicated clusters, and Amazon MSK. It does not support direct integration with other Kafka SaaS services. To set up a Private Link connection based on your Kafka deployment and cloud provider, see the following guides: - [Connect to Confluent Cloud on AWS via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-aws-confluent.md) - [Connect to AWS Self-Hosted Kafka via Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-self-hosted-kafka-in-aws.md) - [Connect to Alibaba Cloud Self-Hosted Kafka via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-self-hosted-kafka-in-alicloud.md) +- [Connect to Amazon MSK via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md) @@ -90,7 +91,7 @@ The steps vary depending on the connectivity method you select. 1. In **Connectivity Method**, select **Private Link**. 2. In **Private Link Connection**, select the private link connection that you created in the [Network](#network) section. Make sure the Availability Zones of the private link connection match those of the Kafka deployment. -3. Fill in the **Bootstrap Port** that you obtained from the [Network](#network) section. +3. Fill in the **Bootstrap Port** that you obtained from the [Network](#network) section. The Amazon MSK Provisioned private link connection does not require a bootstrap port. 4. Select an **Authentication** option according to your Kafka authentication configuration. - If your Kafka does not require authentication, keep the default option **Disable**. From 7aeffb6e4bc392a422e991d0d55a3aed0d1f818f Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Mon, 2 Mar 2026 15:44:12 +0800 Subject: [PATCH 06/18] Apply suggestions from code review --- .../essential-changefeed-sink-to-kafka.md | 4 ++-- ...s-private-link-connection-to-amazon-msk.md | 20 ++++++++----------- 2 files changed, 10 insertions(+), 14 deletions(-) diff --git a/tidb-cloud/essential-changefeed-sink-to-kafka.md b/tidb-cloud/essential-changefeed-sink-to-kafka.md index 037ecabc2b288..eb543ffaad4e2 100644 --- a/tidb-cloud/essential-changefeed-sink-to-kafka.md +++ b/tidb-cloud/essential-changefeed-sink-to-kafka.md @@ -33,14 +33,14 @@ Ensure that your {{{ .essential }}} cluster can connect to the Apache Kafka serv Private link connections leverage **Private Link** technologies from cloud providers to enable resources in your VPC to connect to services in other VPCs using private IP addresses, as if those services were hosted directly within your VPC. -{{{ .essential }}} currently supports Private Link connections only for self-hosted Kafka, Confluent Cloud Dedicated clusters, and Amazon MSK. It does not support direct integration with other Kafka SaaS services. +{{{ .essential }}} currently supports Private Link connections only for self-hosted Kafka, Confluent Cloud Dedicated clusters, and Amazon MSK Provisioned. It does not support direct integration with other Kafka SaaS services. To set up a Private Link connection based on your Kafka deployment and cloud provider, see the following guides: - [Connect to Confluent Cloud on AWS via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-aws-confluent.md) - [Connect to AWS Self-Hosted Kafka via Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-self-hosted-kafka-in-aws.md) - [Connect to Alibaba Cloud Self-Hosted Kafka via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-self-hosted-kafka-in-alicloud.md) -- [Connect to Amazon MSK via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md) +- [Connect to Amazon MSK Provisioned via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md) diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md index 987a0361d5fad..7622931c69c6f 100644 --- a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -33,7 +33,7 @@ To view the AWS account ID and availability zones: ## Step 1. Create an MSK cluster (if needed) -If you do not have an Amazon MSK Provisioned cluster, [create one](https://docs.aws.amazon.com/msk/latest/developerguide/create-cluster.html) in the same region as your {{{ .essential }}} cluster and in the same availability zones as your TiDB Cloud cluster. +If you do not have an Amazon MSK Provisioned cluster, [create one](https://docs.aws.amazon.com/msk/latest/developerguide/create-cluster.html) in the same region and the same availability zone as your as your {{{ .essential }}} cluster. ## Step 2. Set up SASL/SCRAM authentication and ACLs @@ -41,15 +41,15 @@ If you do not have an Amazon MSK Provisioned cluster, [create one](https://docs. Configure [SASL/SCRAM authentication](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password-tutorial.html) for your MSK cluster. -- **Secret name**: The secret name must start with `AmazonMSK_`. -- **Encryption**: Do not use the default encryption key; create a new one for the secret. +- **Secret name**: the secret name must start with `AmazonMSK_`. +- **Encryption**: do not use the default encryption key. Create a new one for the secret. ### Step 2.2. Set ACLs for TiDB Cloud access You must set ACLs so that TiDB Cloud can access your MSK cluster. You can use either of the following methods: -- **SASL/SCRAM (recommended)**: Set ACLs in your VPC using SASL/SCRAM authentication. See [Create ACLs using SASL/SCRAM](#create-acls-using-saslscram). -- **IAM**: Set ACLs in your VPC using IAM authentication. See [Create ACLs using IAM](#create-acls-using-iam). +- **SASL/SCRAM (recommended)**: set ACLs in your VPC using SASL/SCRAM authentication. See [Create ACLs using SASL/SCRAM](#create-acls-using-saslscram). +- **IAM**: set ACLs in your VPC using IAM authentication. See [Create ACLs using IAM](#create-acls-using-iam). ## Step 3. Configure the MSK cluster @@ -75,9 +75,7 @@ Wait for the cluster status to change from **Updating** to **Active** again. Create the private link connection in TiDB Cloud using the `ARN` of your MSK cluster. -For detailed steps, see [Create an Amazon MSK Provisioned private link connection](/tidb-cloud/serverless-private-link-connection.md#create-an-amazon-msk-provisioned-private-link-connection). - ---- +For more information, see [Create an Amazon MSK Provisioned private link connection](/tidb-cloud/serverless-private-link-connection.md#create-an-amazon-msk-provisioned-private-link-connection). ## Create ACLs using SASL/SCRAM @@ -94,7 +92,7 @@ Use this method to create ACLs in the same VPC as your MSK cluster using SASL/SC tar -zxf openjdk-22.0.2_linux-x64_bin.tar.gz ``` -3. Set the environment (replace the path with your own if different): +3. Set the environment. Replace the path with your actual path. ```shell export PATH=$PATH:/home/ec2-user/jdk-22.0.2/bin @@ -120,8 +118,6 @@ Use this method to create ACLs in the same VPC as your MSK cluster using SASL/SC The principal `User:` is the SASL/SCRAM user that TiDB Cloud uses to access your MSK cluster. Use the username you configured for TiDB Cloud in your MSK ACLs. ---- - ## Create ACLs using IAM As an alternative to SASL/SCRAM, you can create ACLs in the same VPC as your MSK cluster using IAM authentication. The IAM user or role must have **Amazon MSK** and **Apache Kafka APIs for MSK** permissions. @@ -138,7 +134,7 @@ As an alternative to SASL/SCRAM, you can create ACLs in the same VPC as your MSK wget https://github.com/aws/aws-msk-iam-auth/releases/download/v2.3.5/aws-msk-iam-auth-2.3.5-all.jar ``` -3. Set the environment (replace paths and credentials with your own): +3. Set the environment. Replace paths and credentials with your own values. ```shell export PATH=$PATH:/home/ec2-user/jdk-22.0.2/bin From 1dadf02e9c1b8186adb51f2e6087ea877d5ff778 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Tue, 3 Mar 2026 10:03:37 +0800 Subject: [PATCH 07/18] Apply suggestions from code review --- ...serverless-private-link-connection-to-amazon-msk.md | 10 +++++----- tidb-cloud/serverless-private-link-connection.md | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md index 7622931c69c6f..a6f5278997510 100644 --- a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -44,12 +44,12 @@ Configure [SASL/SCRAM authentication](https://docs.aws.amazon.com/msk/latest/dev - **Secret name**: the secret name must start with `AmazonMSK_`. - **Encryption**: do not use the default encryption key. Create a new one for the secret. -### Step 2.2. Set ACLs for TiDB Cloud access +### Step 2.2. Set up ACLs for TiDB Cloud access -You must set ACLs so that TiDB Cloud can access your MSK cluster. You can use either of the following methods: +You must set up ACLs so that TiDB Cloud can access your MSK cluster. You can use either of the following methods: -- **SASL/SCRAM (recommended)**: set ACLs in your VPC using SASL/SCRAM authentication. See [Create ACLs using SASL/SCRAM](#create-acls-using-saslscram). -- **IAM**: set ACLs in your VPC using IAM authentication. See [Create ACLs using IAM](#create-acls-using-iam). +- **SASL/SCRAM (recommended)**: set up ACLs in your VPC using SASL/SCRAM authentication. See [Create ACLs using SASL/SCRAM](#create-acls-using-saslscram). +- **IAM**: set up ACLs in your VPC using IAM authentication. See [Create ACLs using IAM](#create-acls-using-iam). ## Step 3. Configure the MSK cluster @@ -134,7 +134,7 @@ As an alternative to SASL/SCRAM, you can create ACLs in the same VPC as your MSK wget https://github.com/aws/aws-msk-iam-auth/releases/download/v2.3.5/aws-msk-iam-auth-2.3.5-all.jar ``` -3. Set the environment. Replace paths and credentials with your own values. +3. Configure the environment. Replace paths and credentials with your own values. ```shell export PATH=$PATH:/home/ec2-user/jdk-22.0.2/bin diff --git a/tidb-cloud/serverless-private-link-connection.md b/tidb-cloud/serverless-private-link-connection.md index b97179555e512..c03a7f62923b5 100644 --- a/tidb-cloud/serverless-private-link-connection.md +++ b/tidb-cloud/serverless-private-link-connection.md @@ -105,7 +105,7 @@ Ensure that your Amazon MSK Provisioned cluster turn on multi-VPC connectivity. - **Private Link Connection Name**: enter a name for the private link connection. - **Connection Type**: select **Amazon MSK Provisioned**. If this option is not displayed, ensure that your cluster is created on AWS. - - **MSK Cluster Arn**: Enter the ARN of your Amazon MSK Provisioned cluster, for example, `arn:aws:kafka:us-east-1:385595570414:cluster/plc-test/11f9378a-ec59-4e08-bb42-ada47765a4e9-18`. + - **MSK Cluster ARN**: enter the ARN of your Amazon MSK Provisioned cluster, for example, `arn:aws:kafka:us-east-1:385595570414:cluster/plc-test/11f9378a-ec59-4e08-bb42-ada47765a4e9-18`. 5. Click **Create**. From f54409acf20ab8fb5f217b0a2e0717542399f55b Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Tue, 3 Mar 2026 13:56:35 +0800 Subject: [PATCH 08/18] Update tidb-cloud/serverless-private-link-connection-to-amazon-msk.md --- ...less-private-link-connection-to-amazon-msk.md | 16 ++++++---------- 1 file changed, 6 insertions(+), 10 deletions(-) diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md index a6f5278997510..43084baf06101 100644 --- a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -37,19 +37,15 @@ If you do not have an Amazon MSK Provisioned cluster, [create one](https://docs. ## Step 2. Set up SASL/SCRAM authentication and ACLs -### Step 2.1. Set up SASL/SCRAM authentication +1. Set up SASL/SCRAM authentication. Configure [SASL/SCRAM authentication](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password-tutorial.html) for your MSK cluster. -Configure [SASL/SCRAM authentication](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password-tutorial.html) for your MSK cluster. + - **Secret name**: the secret name must start with `AmazonMSK_`. + - **Encryption**: do not use the default encryption key. Create a new one for the secret. -- **Secret name**: the secret name must start with `AmazonMSK_`. -- **Encryption**: do not use the default encryption key. Create a new one for the secret. +2. Set up ACLs for TiDB Cloud access. You must set up ACLs so that TiDB Cloud can access your MSK cluster. You can use either of the following methods: -### Step 2.2. Set up ACLs for TiDB Cloud access - -You must set up ACLs so that TiDB Cloud can access your MSK cluster. You can use either of the following methods: - -- **SASL/SCRAM (recommended)**: set up ACLs in your VPC using SASL/SCRAM authentication. See [Create ACLs using SASL/SCRAM](#create-acls-using-saslscram). -- **IAM**: set up ACLs in your VPC using IAM authentication. See [Create ACLs using IAM](#create-acls-using-iam). + - **SASL/SCRAM (recommended)**: set up ACLs in your VPC using SASL/SCRAM authentication. See [Create ACLs using SASL/SCRAM](#create-acls-using-saslscram). + - **IAM**: set up ACLs in your VPC using IAM authentication. See [Create ACLs using IAM](#create-acls-using-iam). ## Step 3. Configure the MSK cluster From 46eae8d6437c056939b37555d4935b994d3c875d Mon Sep 17 00:00:00 2001 From: shi yuhang <52435083+shiyuhang0@users.noreply.github.com> Date: Wed, 4 Mar 2026 11:06:09 +0800 Subject: [PATCH 09/18] Apply suggestions from code review Co-authored-by: Grace Cai --- .../essential-changefeed-sink-to-kafka.md | 2 +- ...s-private-link-connection-to-amazon-msk.md | 27 +++++++++++-------- .../serverless-private-link-connection.md | 4 +-- 3 files changed, 19 insertions(+), 14 deletions(-) diff --git a/tidb-cloud/essential-changefeed-sink-to-kafka.md b/tidb-cloud/essential-changefeed-sink-to-kafka.md index eb543ffaad4e2..297ab350619ce 100644 --- a/tidb-cloud/essential-changefeed-sink-to-kafka.md +++ b/tidb-cloud/essential-changefeed-sink-to-kafka.md @@ -91,7 +91,7 @@ The steps vary depending on the connectivity method you select. 1. In **Connectivity Method**, select **Private Link**. 2. In **Private Link Connection**, select the private link connection that you created in the [Network](#network) section. Make sure the Availability Zones of the private link connection match those of the Kafka deployment. -3. Fill in the **Bootstrap Port** that you obtained from the [Network](#network) section. The Amazon MSK Provisioned private link connection does not require a bootstrap port. +3. Fill in the **Bootstrap Port** that you obtained from the [Network](#network) section. If you are using the Amazon MSK Provisioned private link connection, you can skip this field. 4. Select an **Authentication** option according to your Kafka authentication configuration. - If your Kafka does not require authentication, keep the default option **Disable**. diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md index 43084baf06101..d112f08c2f652 100644 --- a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -19,7 +19,7 @@ Before you begin, ensure the following: See [Amazon MSK multi-VPC private connectivity in a single Region](https://docs.aws.amazon.com/msk/latest/developerguide/aws-access-mult-vpc.html#mvpc-requirements) for more requirements. -## Prerequisites +## Prerequisites for {{{ .essential }}} - Your {{{ .essential }}} is hosted on AWS and is active. Retrieve and save the following for later use: - AWS Account ID @@ -31,18 +31,23 @@ To view the AWS account ID and availability zones: 2. In the **Private Link Connection For Dataflow** area, click **Create Private Link Connection**. 3. In the dialog, note the AWS account ID and availability zones. -## Step 1. Create an MSK cluster (if needed) +## Prerequisites for the Amazon MSK Provisioned cluster -If you do not have an Amazon MSK Provisioned cluster, [create one](https://docs.aws.amazon.com/msk/latest/developerguide/create-cluster.html) in the same region and the same availability zone as your as your {{{ .essential }}} cluster. - -## Step 2. Set up SASL/SCRAM authentication and ACLs - -1. Set up SASL/SCRAM authentication. Configure [SASL/SCRAM authentication](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password-tutorial.html) for your MSK cluster. +Before you begin, ensure the following for your Amazon MSK Provisioned cluster: + +- **Region and AZ**: Your Amazon MSK Provisioned cluster is in the same AWS region as your {{{ .essential }}} cluster, and the availability zones of the MSK cluster are the same as your TiDB Cloud cluster. +- **Authentication**: [SASL/SCRAM authentication](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password.html) is required for the MSK cluster. +- **Broker type**: Do not use the `t4.small` broker type. It does not support private link. + +For more requirements, see [Amazon MSK multi-VPC private connectivity in a single Region](https://docs.aws.amazon.com/msk/latest/developerguide/aws-access-mult-vpc.html#mvpc-requirements). + + If you do not have an Amazon MSK Provisioned cluster, [create one](https://docs.aws.amazon.com/msk/latest/developerguide/create-cluster.html) in the same region and the same availability zone as your {{{ .essential }}} cluster, and then [set up SASL/SCRAM authentication](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password-tutorial.html) for the created cluster. + - **Secret name**: the secret name must start with `AmazonMSK_`. + - **Encryption**: do not use the default encryption key. Create a new custom AWS KMS key for your secret. - - **Secret name**: the secret name must start with `AmazonMSK_`. - - **Encryption**: do not use the default encryption key. Create a new one for the secret. +## Step 1. Set up Kafka ACLs for TiDB Cloud access -2. Set up ACLs for TiDB Cloud access. You must set up ACLs so that TiDB Cloud can access your MSK cluster. You can use either of the following methods: +You must set up Kafka ACLs so that TiDB Cloud can access your Amazon MSK Provisioned cluster. You can use SASL/SCRAM authentication (recommended) or IAM authentication to set up ACLs. - **SASL/SCRAM (recommended)**: set up ACLs in your VPC using SASL/SCRAM authentication. See [Create ACLs using SASL/SCRAM](#create-acls-using-saslscram). - **IAM**: set up ACLs in your VPC using IAM authentication. See [Create ACLs using IAM](#create-acls-using-iam). @@ -63,7 +68,7 @@ Apply the changes and wait for the cluster status to change from **Updating** to ## Step 5. Turn on multi-VPC connectivity -After the cluster is active, [turn on multi-VPC connectivity](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-cluster-owner-action-turn-on.html) for the MSK cluster. Multi-VPC connectivity is required for private link and requires SASL/SCRAM authentication. +After the cluster is active, [turn on multi-VPC connectivity](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-cluster-owner-action-turn-on.html) for the MSK cluster. Multi-VPC connectivity is required for AWS PrivateLink. To connect from TiDB Cloud, you must enable SASL/SCRAM authentication. Wait for the cluster status to change from **Updating** to **Active** again. diff --git a/tidb-cloud/serverless-private-link-connection.md b/tidb-cloud/serverless-private-link-connection.md index c03a7f62923b5..24a01084ebf9d 100644 --- a/tidb-cloud/serverless-private-link-connection.md +++ b/tidb-cloud/serverless-private-link-connection.md @@ -89,7 +89,7 @@ To create a private link connection using the TiDB Cloud CLI: You can create an Amazon MSK Provisioned private link connection using the TiDB Cloud console. -Ensure that your Amazon MSK Provisioned cluster turn on multi-VPC connectivity. See [Connect to Amazon MSK Provisioned via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md) for more detials. +Before creating the Amazon MSK Provisioned private link connection, ensure that your Amazon MSK Provisioned cluster turns on multi-VPC connectivity. See [Connect to Amazon MSK Provisioned via a Private Link Connection](/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md) for more details. 1. Log in to the [TiDB Cloud console](https://tidbcloud.com/) and navigate to the [**Clusters**](https://tidbcloud.com/project/clusters) page of your project. @@ -175,7 +175,7 @@ Different private link connection types support attaching different domain types |--------------------------------|-------------------------------------------| | AWS Endpoint Service |
  • TiDB Cloud managed (`aws.tidbcloud.com`)
  • Confluent Dedicated (`aws.confluent.cloud`)
| | Alibaba Cloud Endpoint Service | TiDB Cloud managed (`alicloud.tidbcloud.com`) | -| Amazon MSK Provisioned | Does not support attaching domains | +| Amazon MSK Provisioned | Domain attachment is not supported. | If your domain is not included in this table, contact [TiDB Cloud Support](/tidb-cloud/tidb-cloud-support.md) to request support. From 2712eed21560c42fc9feb08c9f6d3b26b6dbfc00 Mon Sep 17 00:00:00 2001 From: shi yuhang <52435083+shiyuhang0@users.noreply.github.com> Date: Wed, 4 Mar 2026 11:09:38 +0800 Subject: [PATCH 10/18] Update tidb-cloud/serverless-private-link-connection-to-amazon-msk.md Co-authored-by: Grace Cai --- tidb-cloud/serverless-private-link-connection-to-amazon-msk.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md index d112f08c2f652..ba64b5915b845 100644 --- a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -49,8 +49,6 @@ For more requirements, see [Amazon MSK multi-VPC private connectivity in a singl You must set up Kafka ACLs so that TiDB Cloud can access your Amazon MSK Provisioned cluster. You can use SASL/SCRAM authentication (recommended) or IAM authentication to set up ACLs. - - **SASL/SCRAM (recommended)**: set up ACLs in your VPC using SASL/SCRAM authentication. See [Create ACLs using SASL/SCRAM](#create-acls-using-saslscram). - - **IAM**: set up ACLs in your VPC using IAM authentication. See [Create ACLs using IAM](#create-acls-using-iam). ## Step 3. Configure the MSK cluster From 7ead6b7e17eb64cb5b95f70f304fc3d0b0948009 Mon Sep 17 00:00:00 2001 From: shi yuhang <52435083+shiyuhang0@users.noreply.github.com> Date: Wed, 4 Mar 2026 11:09:47 +0800 Subject: [PATCH 11/18] Update tidb-cloud/serverless-private-link-connection-to-amazon-msk.md Co-authored-by: Grace Cai --- ...erverless-private-link-connection-to-amazon-msk.md | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md index ba64b5915b845..b08c484e8b817 100644 --- a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -7,17 +7,6 @@ summary: Learn how to connect to an Amazon MSK Provisioned cluster using an Amaz This document describes how to connect a {{{ .essential }}} cluster to an [Amazon MSK Provisioned](https://docs.aws.amazon.com/msk/latest/developerguide/msk-provisioned.html) cluster using an [Amazon MSK Provisioned private link connection](/tidb-cloud/serverless-private-link-connection.md#create-an-amazon-msk-provisioned-private-link-connection). -## Limitations and requirements - -Before you begin, ensure the following: - -- **Region and AZ**: Your Amazon MSK Provisioned cluster is in the same AWS region as your {{{ .essential }}} cluster, and the availability zones of the MSK cluster are the same as your TiDB Cloud cluster. - -- **Authentication**: [SASL/SCRAM authentication](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password.html) is required for the MSK cluster. - -- **Broker type**: Do not use the `t4.small` broker type. It does not support private link. - -See [Amazon MSK multi-VPC private connectivity in a single Region](https://docs.aws.amazon.com/msk/latest/developerguide/aws-access-mult-vpc.html#mvpc-requirements) for more requirements. ## Prerequisites for {{{ .essential }}} From 5e6bf074c43adb699d6f56ee51d0a5c4b4681f81 Mon Sep 17 00:00:00 2001 From: shiyuhang <1136742008@qq.com> Date: Wed, 4 Mar 2026 11:15:14 +0800 Subject: [PATCH 12/18] opt all --- ...s-private-link-connection-to-amazon-msk.md | 63 ++++++++++--------- .../serverless-private-link-connection.md | 2 +- 2 files changed, 35 insertions(+), 30 deletions(-) diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md index b08c484e8b817..f67125c358e8c 100644 --- a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -38,34 +38,8 @@ For more requirements, see [Amazon MSK multi-VPC private connectivity in a singl You must set up Kafka ACLs so that TiDB Cloud can access your Amazon MSK Provisioned cluster. You can use SASL/SCRAM authentication (recommended) or IAM authentication to set up ACLs. - -## Step 3. Configure the MSK cluster - -Update the following cluster configuration properties: - -- Set `auto.create.topics.enable=true`. -- Add `allow.everyone.if.no.acl.found=false` (required for SASL/SCRAM). -- Keep other properties unchanged or adjust them as needed. - -Apply the changes and wait for the cluster status to change from **Updating** to **Active**. - -## Step 4. Attach the cluster policy - -[Attach the cluster policy](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-cluster-owner-action-policy.html) to allow TiDB Cloud to connect to your MSK cluster. Use the TiDB Cloud AWS account ID you obtained in [Prerequisites](#prerequisites). - -## Step 5. Turn on multi-VPC connectivity - -After the cluster is active, [turn on multi-VPC connectivity](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-cluster-owner-action-turn-on.html) for the MSK cluster. Multi-VPC connectivity is required for AWS PrivateLink. To connect from TiDB Cloud, you must enable SASL/SCRAM authentication. - -Wait for the cluster status to change from **Updating** to **Active** again. - -## Step 6. Create an Amazon MSK Provisioned private link connection in TiDB Cloud - -Create the private link connection in TiDB Cloud using the `ARN` of your MSK cluster. - -For more information, see [Create an Amazon MSK Provisioned private link connection](/tidb-cloud/serverless-private-link-connection.md#create-an-amazon-msk-provisioned-private-link-connection). - -## Create ACLs using SASL/SCRAM + +
Use this method to create ACLs in the same VPC as your MSK cluster using SASL/SCRAM authentication. @@ -106,7 +80,9 @@ Use this method to create ACLs in the same VPC as your MSK cluster using SASL/SC The principal `User:` is the SASL/SCRAM user that TiDB Cloud uses to access your MSK cluster. Use the username you configured for TiDB Cloud in your MSK ACLs. -## Create ACLs using IAM +
+ +
As an alternative to SASL/SCRAM, you can create ACLs in the same VPC as your MSK cluster using IAM authentication. The IAM user or role must have **Amazon MSK** and **Apache Kafka APIs for MSK** permissions. @@ -149,3 +125,32 @@ As an alternative to SASL/SCRAM, you can create ACLs in the same VPC as your MSK ``` The principal `User:` is the SASL/SCRAM user that TiDB Cloud uses to access your MSK cluster. Use the username you configured for TiDB Cloud in your MSK ACLs. + +
+ + +## Step 2. Configure the MSK cluster + +Update the following cluster configuration properties: + +- Set `auto.create.topics.enable=true`. +- Add `allow.everyone.if.no.acl.found=false` (required for SASL/SCRAM). +- Keep other properties unchanged or adjust them as needed. + +Apply the changes and wait for the cluster status to change from **Updating** to **Active**. + +## Step 3. Attach the cluster policy + +[Attach the cluster policy](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-cluster-owner-action-policy.html) to allow TiDB Cloud to connect to your MSK cluster. Use the TiDB Cloud AWS account ID you obtained in [Prerequisites](#prerequisites). + +## Step 4. Turn on multi-VPC connectivity + +After the cluster is active, [turn on multi-VPC connectivity](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-cluster-owner-action-turn-on.html) for the MSK cluster. Multi-VPC connectivity is required for AWS PrivateLink. To connect from TiDB Cloud, you must enable SASL/SCRAM authentication. + +Wait for the cluster status to change from **Updating** to **Active** again. + +## Step 5. Create an Amazon MSK Provisioned private link connection in TiDB Cloud + +Create the private link connection in TiDB Cloud using the `ARN` of your MSK cluster. + +For more information, see [Create an Amazon MSK Provisioned private link connection](/tidb-cloud/serverless-private-link-connection.md#create-an-amazon-msk-provisioned-private-link-connection). \ No newline at end of file diff --git a/tidb-cloud/serverless-private-link-connection.md b/tidb-cloud/serverless-private-link-connection.md index 24a01084ebf9d..51f99dc2938f5 100644 --- a/tidb-cloud/serverless-private-link-connection.md +++ b/tidb-cloud/serverless-private-link-connection.md @@ -105,7 +105,7 @@ Before creating the Amazon MSK Provisioned private link connection, ensure that - **Private Link Connection Name**: enter a name for the private link connection. - **Connection Type**: select **Amazon MSK Provisioned**. If this option is not displayed, ensure that your cluster is created on AWS. - - **MSK Cluster ARN**: enter the ARN of your Amazon MSK Provisioned cluster, for example, `arn:aws:kafka:us-east-1:385595570414:cluster/plc-test/11f9378a-ec59-4e08-bb42-ada47765a4e9-18`. + - **MSK Cluster ARN**: enter the ARN of your Amazon MSK Provisioned cluster, for example, `arn:aws:kafka:us-east-1:385595570414:cluster//xxxx`. 5. Click **Create**. From 39bc10329fda0ce1d0b1c7ef7b7f2323b0881833 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Tue, 10 Mar 2026 20:24:32 +0800 Subject: [PATCH 13/18] Update tidb-cloud/serverless-private-link-connection-to-amazon-msk.md --- tidb-cloud/serverless-private-link-connection-to-amazon-msk.md | 1 - 1 file changed, 1 deletion(-) diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md index f67125c358e8c..26b29e7d61b30 100644 --- a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -7,7 +7,6 @@ summary: Learn how to connect to an Amazon MSK Provisioned cluster using an Amaz This document describes how to connect a {{{ .essential }}} cluster to an [Amazon MSK Provisioned](https://docs.aws.amazon.com/msk/latest/developerguide/msk-provisioned.html) cluster using an [Amazon MSK Provisioned private link connection](/tidb-cloud/serverless-private-link-connection.md#create-an-amazon-msk-provisioned-private-link-connection). - ## Prerequisites for {{{ .essential }}} - Your {{{ .essential }}} is hosted on AWS and is active. Retrieve and save the following for later use: From e9249a6fb6e46a8575f151ac189c7a5446507d24 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Tue, 10 Mar 2026 20:41:09 +0800 Subject: [PATCH 14/18] Apply suggestions from code review Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- .../serverless-private-link-connection-to-amazon-msk.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md index 26b29e7d61b30..6c7e75055d8a5 100644 --- a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -9,7 +9,7 @@ This document describes how to connect a {{{ .essential }}} cluster to an [Amazo ## Prerequisites for {{{ .essential }}} -- Your {{{ .essential }}} is hosted on AWS and is active. Retrieve and save the following for later use: +- Your {{{ .essential }}} cluster is hosted on AWS and is active. Retrieve and save the following for later use: - AWS Account ID - Availability zones (AZ) @@ -140,7 +140,7 @@ Apply the changes and wait for the cluster status to change from **Updating** to ## Step 3. Attach the cluster policy -[Attach the cluster policy](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-cluster-owner-action-policy.html) to allow TiDB Cloud to connect to your MSK cluster. Use the TiDB Cloud AWS account ID you obtained in [Prerequisites](#prerequisites). +[Attach the cluster policy](https://docs.aws.amazon.com/msk/latest/developerguide/mvpc-cluster-owner-action-policy.html) to allow TiDB Cloud to connect to your MSK cluster. Use the TiDB Cloud AWS account ID you obtained in [Prerequisites](#prerequisites-for-essential). ## Step 4. Turn on multi-VPC connectivity From 94041180de61988b5622e666179cd05f995cdda0 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Tue, 10 Mar 2026 20:50:55 +0800 Subject: [PATCH 15/18] Update tidb-cloud/serverless-private-link-connection-to-amazon-msk.md --- tidb-cloud/serverless-private-link-connection-to-amazon-msk.md | 1 + 1 file changed, 1 insertion(+) diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md index 6c7e75055d8a5..bcf7c597aae46 100644 --- a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -10,6 +10,7 @@ This document describes how to connect a {{{ .essential }}} cluster to an [Amazo ## Prerequisites for {{{ .essential }}} - Your {{{ .essential }}} cluster is hosted on AWS and is active. Retrieve and save the following for later use: + - AWS Account ID - Availability zones (AZ) From 7221d06adc098bdd53b97031d835c50bf44d085d Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Tue, 10 Mar 2026 21:16:00 +0800 Subject: [PATCH 16/18] Update tidb-cloud/serverless-private-link-connection-to-amazon-msk.md --- tidb-cloud/serverless-private-link-connection-to-amazon-msk.md | 1 + 1 file changed, 1 insertion(+) diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md index bcf7c597aae46..38d1744b5a920 100644 --- a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -31,6 +31,7 @@ Before you begin, ensure the following for your Amazon MSK Provisioned cluster: For more requirements, see [Amazon MSK multi-VPC private connectivity in a single Region](https://docs.aws.amazon.com/msk/latest/developerguide/aws-access-mult-vpc.html#mvpc-requirements). If you do not have an Amazon MSK Provisioned cluster, [create one](https://docs.aws.amazon.com/msk/latest/developerguide/create-cluster.html) in the same region and the same availability zone as your {{{ .essential }}} cluster, and then [set up SASL/SCRAM authentication](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password-tutorial.html) for the created cluster. + - **Secret name**: the secret name must start with `AmazonMSK_`. - **Encryption**: do not use the default encryption key. Create a new custom AWS KMS key for your secret. From dec07c4a29690064e6c31697106da5e6e69c5c39 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Tue, 10 Mar 2026 21:19:42 +0800 Subject: [PATCH 17/18] Apply suggestions from code review Co-authored-by: Grace Cai --- tidb-cloud/serverless-private-link-connection-to-amazon-msk.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md index 38d1744b5a920..580675e2bbb6f 100644 --- a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -7,7 +7,7 @@ summary: Learn how to connect to an Amazon MSK Provisioned cluster using an Amaz This document describes how to connect a {{{ .essential }}} cluster to an [Amazon MSK Provisioned](https://docs.aws.amazon.com/msk/latest/developerguide/msk-provisioned.html) cluster using an [Amazon MSK Provisioned private link connection](/tidb-cloud/serverless-private-link-connection.md#create-an-amazon-msk-provisioned-private-link-connection). -## Prerequisites for {{{ .essential }}} +## Prerequisites for {{{ .essential }}} {#prerequisites-for-essential} - Your {{{ .essential }}} cluster is hosted on AWS and is active. Retrieve and save the following for later use: From 33b5ffa800aa21c77ba5f2199cad2b080e4fb726 Mon Sep 17 00:00:00 2001 From: xixirangrang Date: Tue, 10 Mar 2026 21:30:38 +0800 Subject: [PATCH 18/18] Update tidb-cloud/serverless-private-link-connection-to-amazon-msk.md --- .../serverless-private-link-connection-to-amazon-msk.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md index 580675e2bbb6f..3672c46330bb0 100644 --- a/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md +++ b/tidb-cloud/serverless-private-link-connection-to-amazon-msk.md @@ -32,8 +32,8 @@ For more requirements, see [Amazon MSK multi-VPC private connectivity in a singl If you do not have an Amazon MSK Provisioned cluster, [create one](https://docs.aws.amazon.com/msk/latest/developerguide/create-cluster.html) in the same region and the same availability zone as your {{{ .essential }}} cluster, and then [set up SASL/SCRAM authentication](https://docs.aws.amazon.com/msk/latest/developerguide/msk-password-tutorial.html) for the created cluster. - - **Secret name**: the secret name must start with `AmazonMSK_`. - - **Encryption**: do not use the default encryption key. Create a new custom AWS KMS key for your secret. +- **Secret name**: the secret name must start with `AmazonMSK_`. +- **Encryption**: do not use the default encryption key. Create a new custom AWS KMS key for your secret. ## Step 1. Set up Kafka ACLs for TiDB Cloud access