Skip to content

SQL for BYOVPC#620

Open
kbatuigas wants to merge 11 commits into
mainfrom
DOC-2227-document-feature-byovpc-for-oxla-aws
Open

SQL for BYOVPC#620
kbatuigas wants to merge 11 commits into
mainfrom
DOC-2227-document-feature-byovpc-for-oxla-aws

Conversation

@kbatuigas

@kbatuigas kbatuigas commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Description

Adds a how-to guide for enabling Redpanda SQL on BYOVPC clusters on AWS, and updates surrounding pages to distinguish the BYOC and BYOVPC paths.

New page — sql/get-started/enable-sql-byovpc-aws.adoc:

  • Provision SQL resources with Terraform: set enable_redpanda_sql = true in the BYOVPC Terraform module to create the three SQL-specific AWS resources (IAM instance profile, S3 bucket, security group) and collect their ARNs.
  • Enable Redpanda SQL — split into two paths, each with tabs:
    • On a new cluster: Terraform provider or Cloud API (no Console path for BYOVPC creation).
    • On an existing cluster: Cloud Console, Terraform provider, or Cloud API.
  • Verify the engine via the Redpanda SQL tile, the operation state, or a psql/SQL editor connection.
  • Disable Redpanda SQL (Console/Terraform/API tabs), with explicit destructive-semantics coverage: catalog state is cleared, in-flight queries fail, topic/SR/Iceberg data is unaffected, and re-enabling provisions a fresh engine. CMR immutability and AWS resource cleanup are called out.
  • Single-AZ semantics for the SQL engine are stated at each <sql-az> parameter (per SME review: the engine always deploys to one AZ even on multi-AZ clusters; the AZ locks on first enable).

Terraform provider support: the page documents the Terraform path throughout — provider v2.1.0 ships the rpsql_* customer-managed resource fields (terraform-provider-redpanda#353).

Surrounding updates:

  • nav.adoc: retitled the pair to "Enable Redpanda SQL on BYOC" / "Enable Redpanda SQL on BYOVPC".
  • deploy-sql-cluster.adoc: retitled to "Enable Redpanda SQL on a BYOC Cluster"; NOTE directs BYOVPC users to the new guide.
  • vpc-byo-aws.adoc: next-steps link to the new guide.
  • What's New: "Redpanda SQL: BYOVPC support on AWS" entry (also carries the combined entry for SQL: Terraform provider #617).

Resolves DOC-2227
Review deadline:

Page previews

Checks

  • New feature
  • Content gap
  • Support Follow-up
  • Small fix (typos, links, copyedits, etc)

@netlify

netlify Bot commented Jun 16, 2026

Copy link
Copy Markdown

Deploy Preview for rp-cloud ready!

Name Link
🔨 Latest commit 5b988b9
🔍 Latest deploy log https://app.netlify.com/projects/rp-cloud/deploys/6a5eb8402b44b00008716b92
😎 Deploy Preview https://deploy-preview-620--rp-cloud.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Jun 16, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a4cd4b9e-97bd-41de-b3d0-d3f5c8a0c528

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR adds a new documentation page (enable-sql-byovpc-aws.adoc) describing how to enable Redpanda SQL on BYOVPC AWS clusters. The page covers prerequisites, Terraform provisioning of customer-managed AWS resources (enable_redpanda_sql = true), enable workflows for both new and existing clusters via Terraform, Cloud API, and Cloud Console, verification steps, and disablement instructions. Navigation in nav.adoc is updated to split the single "Enable Redpanda SQL" entry into separate BYOC and BYOVPC entries. The BYOVPC AWS VPC setup page gains a next-steps cross-reference, and the existing deploy-sql-cluster.adoc availability note is updated to direct BYOVPC users to the new page.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • redpanda-data/cloud-docs#261: Both PRs modify modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc, with this PR adding a SQL next-steps link and the retrieved PR adding an EIP prerequisite.
  • redpanda-data/cloud-docs#603: Both PRs edit the deploy-sql-cluster.adoc SQL enablement guide, with this PR updating the availability note and the retrieved PR modifying rpsql enablement examples.

Suggested reviewers

  • simonlord
  • micheleRP
  • Feediver1
  • david-yu
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Title check ❓ Inconclusive The title is vague and doesn't clearly state the main change beyond a generic BYOVPC SQL reference. Use a more specific title like "Add Redpanda SQL enablement docs for BYOVPC on AWS".
✅ Passed checks (4 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The description covers the change, previews, and checks, but omits the issue link and review deadline from the template.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch DOC-2227-document-feature-byovpc-for-oxla-aws

Comment @coderabbitai help to get the list of available commands.


[NOTE]
====
The Redpanda Terraform provider does not support SQL customer-managed resource fields. Use the Cloud API or Cloud Console to supply these resources and enable SQL on a BYOVPC cluster.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's PR that will add this in review.
redpanda-data/terraform-provider-redpanda#353

Comment thread modules/sql/pages/get-started/enable-sql-byovpc-aws.adoc
@kbatuigas
kbatuigas marked this pull request as ready for review June 22, 2026 20:32
@kbatuigas
kbatuigas requested a review from a team as a code owner June 22, 2026 20:32

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
modules/sql/pages/get-started/enable-sql-byovpc-aws.adoc (1)

364-369: 🧹 Nitpick | 🔵 Trivial | 💤 Low value

Clarify that scaling instructions on the BYOC SQL page apply to BYOVPC.

Line 366 links to deploy-sql-cluster.adoc for scaling, which is now labeled "Enable Redpanda SQL on BYOC" (per nav.adoc line 335). When BYOVPC users click this link, they will land on a page titled for BYOC clusters, which may cause momentary confusion.

However, the scaling mechanism is identical for both cluster types (adjusting replicas in the rpsql block), so the link target is functionally correct. Consider adding a brief note to clarify that the scaling instructions on that page apply to both BYOC and BYOVPC clusters, or alternatively, reference the specific section of the BYOC page (e.g., deploy-sql-cluster.adoc#scale-redpanda-sql) if AsciiDoc anchors are available.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@modules/sql/pages/get-started/enable-sql-byovpc-aws.adoc` around lines 364 -
369, In the "Next steps" section of the enable-sql-byovpc-aws.adoc file, the
link labeled "Scale the SQL engine" currently points to deploy-sql-cluster.adoc
which is titled for BYOC clusters. To clarify that the scaling instructions
apply to both BYOC and BYOVPC, either add a brief inline note (such as "(applies
to both BYOC and BYOVPC)" after the link text) or update the link to reference a
specific scaling section anchor (e.g.,
deploy-sql-cluster.adoc#scale-redpanda-sql) if such anchors exist in the target
document. This will prevent BYOVPC users from experiencing confusion when
landing on a page titled for BYOC clusters.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@modules/sql/pages/get-started/enable-sql-byovpc-aws.adoc`:
- Around line 364-369: In the "Next steps" section of the
enable-sql-byovpc-aws.adoc file, the link labeled "Scale the SQL engine"
currently points to deploy-sql-cluster.adoc which is titled for BYOC clusters.
To clarify that the scaling instructions apply to both BYOC and BYOVPC, either
add a brief inline note (such as "(applies to both BYOC and BYOVPC)" after the
link text) or update the link to reference a specific scaling section anchor
(e.g., deploy-sql-cluster.adoc#scale-redpanda-sql) if such anchors exist in the
target document. This will prevent BYOVPC users from experiencing confusion when
landing on a page titled for BYOC clusters.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 651baad6-bd95-472b-a754-72e12a7c8e95

📥 Commits

Reviewing files that changed from the base of the PR and between 8d6f00a and 85442a5.

📒 Files selected for processing (4)
  • modules/ROOT/nav.adoc
  • modules/get-started/pages/cluster-types/byoc/aws/vpc-byo-aws.adoc
  • modules/sql/pages/get-started/deploy-sql-cluster.adoc
  • modules/sql/pages/get-started/enable-sql-byovpc-aws.adoc

@kbatuigas
kbatuigas force-pushed the DOC-2227-document-feature-byovpc-for-oxla-aws branch from 8c51244 to b98fe30 Compare June 22, 2026 22:22
@kbatuigas kbatuigas mentioned this pull request Jun 22, 2026
4 tasks

@jacekseligarp jacekseligarp left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imho all good, but I suggest to wait for @rpdevmp tests of BYOVPC RP SQL.

}
}
}
rpsql = {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please confirm with @gene-redpanda which fields are required and if some fields have a built-in default

@kbatuigas
kbatuigas force-pushed the DOC-2227-document-feature-byovpc-for-oxla-aws branch from b98fe30 to 622a74d Compare July 13, 2026 19:18
@kbatuigas
kbatuigas requested review from ncole and removed request for ncole July 15, 2026 14:23
Comment thread modules/sql/pages/get-started/enable-sql-byovpc-aws.adoc Outdated

@rpdevmp rpdevmp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Addresses rpdevmp's review comment distinguishing multi-AZ streaming
clusters from the always-single-AZ SQL engine. Also updates the BYOC
note to reflect that Redpanda SQL now runs on GCP in addition to AWS,
alongside the BYOVPC guide pointer.
GCP BYOC/BYOVPC Redpanda SQL isn't customer-available yet (targeted
end of July/early August). Only AWS ships with this PR.

@Feediver1 Feediver1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final-pass review (docs-team-standards)

Overall: Thorough, SME-validated guide that's in strong shape at head — the final state is substantially better than the commit history suggests (the Terraform-provider path was fully incorporated after provider v2.1.0 shipped). One unresolved SME thread is the only thing I'd block on.

Jira ticket alignment

DOC-2227 (BYOVPC for Oxla AWS, epic ENG-1165) — ✅ fully addressed: enablement (new + existing cluster, three methods), verification, disable semantics, resource immutability, and cleanup.

SME thread verification

  • @rpdevmp confirmed the API call and approved (7/17) after testing — @jacekseligarp's conditional approval ("wait for rpdevmp tests") is satisfied.
  • ✅ rpdevmp's multi-AZ clarification (7/16) is incorporated at all four <sql-az> bullets.
  • ✅ The early NOTE claiming "the Terraform provider does not support SQL customer-managed resource fields" was correctly removed — verified that provider v2.1.0 ships the rpsql_* CMR fields (PR redpanda-data/terraform-provider-redpanda#353 merged just before the tag), so the What's New claim ("Console, Cloud API, or Terraform provider") and the page are consistent.

Must resolve before merge

  1. Open SME question on the new-cluster payload: @jacekseligarp asked (6/23) to confirm with @gene-redpanda which fields are required and which have built-in defaults for the POST /v1/clusters request. No reply landed in the thread; rpdevmp's testing confirmed the existing-cluster PATCH path. Please resolve (or note where it was answered) before merging.

Suggestions

  1. The <sql-az> bullet is duplicated verbatim 4× — a maintenance hazard (this PR itself had to fix it in four places after the SME comment proved the point). Consider a partial, or move the AZ semantics to a single NOTE above the tabs.
  2. <<On an existing cluster>> uses heading-text cross-reference form; it works, but the anchor form (<<on-an-existing-cluster>>) survives heading edits better.
  3. Follow-up (not in this PR): manage/terraform-provider.adoc's "Enable Redpanda SQL on a BYOC cluster" section could gain a one-line pointer to this new BYOVPC page, since the provider path now exists for both.

Verified clean

  • Nav entries retitled consistently; What's New entry added; sql/get-started/index.adoc uses page-layout: index so the new page auto-lists; the BYOC page retitle keeps its filename/anchors, so no inbound xrefs break.
  • feature-flag-rpcn.adoc include renders the correct BYOVPC/Premium-support gate (same as the three BYOVPC cluster pages) — the partial's filename is misleading legacy, worth a rename someday, not here.
  • Correct glossterm usage, 3 checkbox LOs with Bloom verbs, sentence-case H2/H3s, language specifiers on every code block.
  • CodeRabbit's single nitpick (scaling clarity) is already covered by the final Next steps wording.

What works well

  • The SME loop actually closed: tested before approval, API calls confirmed, AZ semantics corrected everywhere
  • The Disable section is honest about destructive semantics (catalog state loss, in-flight query failure, what survives, fresh-engine re-enable) — exactly the customer-experience detail this page needed

🤖 Automated final-pass review via docs-team-standards pr-review.


[IMPORTANT]
====
The SQL customer-managed resources (`rpsql_node_group_instance_profile`, `rpsql_cloud_storage_bucket`, `rpsql_security_group`) are immutable while SQL is enabled. To change any of these resources, disable SQL first.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The SQL customer-managed resources (`rpsql_node_group_instance_profile`, `rpsql_cloud_storage_bucket`, `rpsql_security_group`) are immutable while SQL is enabled. To change any of these resources, disable SQL first.
The SQL customer-managed resources (`rpsql_node_group_instance_profile`, `rpsql_cloud_storage_bucket`, `rpsql_security_group`) are immutable while SQL is enabled. To change any of these resources, you must first disable SQL.

@Feediver1 Feediver1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@david-yu

Copy link
Copy Markdown
Contributor

@kbatuigas We probably need the IAM permissions setup properly to allow RPSQL to communicate to glue: redpanda-data/terraform-aws-redpanda-byovpc#52

Introduces enable-sql-byovpc.adoc as an index page nesting
enable-sql-byovpc-aws.adoc under it in nav.adoc, so a GCP
sibling page can be added later without restructuring the
nav tree or moving files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants