Document Connect pipeline egress allowlist for BYOC clusters - #650
Document Connect pipeline egress allowlist for BYOC clusters#650micheleRP wants to merge 2 commits into
Conversation
Add a Networking how-to for the Redpanda Connect egress allowlist (redpanda_connect.allowed_destination_cidr_ports), including the Cloud API flow, which had no coverage, and document the default pipeline egress behavior where readers look for firewall boundaries. - Add networking:byoc/connect-egress-allowlist.adoc, plus a shared partial for the rule fields and limits - Document default-allowed, default-blocked, and never-allowlistable pipeline destinations in cloud-security-network.adoc - Reuse the shared partial in terraform-provider.adoc and correct the minimum provider version to 2.1.2 - Broaden the What's New entry from Terraform-only to the feature DOC-2069 Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
✅ Deploy Preview for rp-cloud ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
📝 WalkthroughWalkthroughAdds BYOC documentation for Redpanda Connect pipeline egress allowlists, covering default restrictions, blocked destinations, rule constraints, Cloud API and Terraform configuration, update behavior, troubleshooting, and navigation. It also updates Terraform provider guidance, the July 2026 release notes, and the Connect quickstart. Estimated code review effort: 2 (Simple) | ~10 minutes Sequence Diagram(s)sequenceDiagram
participant Operator
participant Cloud API
participant Cluster Update
participant Connect Pipeline
Operator->>Cloud API: PATCH cluster allowlist
Cloud API->>Cluster Update: Start update operation
Operator->>Cluster Update: Poll operation state
Cluster Update-->>Operator: Return STATE_READY
Connect Pipeline->>Cluster Update: Apply firewall rules on pod start or restart
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@modules/networking/pages/byoc/connect-egress-allowlist.adoc`:
- Around line 93-96: Remove the verbose -vv option from the curl command in the
cluster PATCH example, or replace it with non-sensitive error output such as
--fail-with-body, while preserving the existing request headers, payload, and
endpoint.
- Around line 32-36: Align the private-address egress behavior across all three
documentation sites: update the descriptions in
modules/networking/pages/byoc/connect-egress-allowlist.adoc (lines 32-36),
modules/networking/pages/cloud-security-network.adoc (lines 172-174), and
modules/manage/pages/terraform-provider.adoc (lines 384-386) so they
consistently state whether peered VPC/VNet addresses are allowed or blocked by
default, while preserving the documented exception for Redpanda cluster ports
and allowlist behavior.
In `@modules/networking/partials/connect-egress-allowlist-rules.adoc`:
- Around line 14-16: Update the port_end field description in the port allowlist
documentation to state the complete constraint: port_start <= port_end <= 65535,
while preserving the existing single-port behavior when omitted.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 56b8c5ff-343a-4acc-a627-296c3fd3efd2
📒 Files selected for processing (6)
modules/ROOT/nav.adocmodules/get-started/pages/whats-new-cloud.adocmodules/manage/pages/terraform-provider.adocmodules/networking/pages/byoc/connect-egress-allowlist.adocmodules/networking/pages/cloud-security-network.adocmodules/networking/partials/connect-egress-allowlist-rules.adoc
| | Private addresses outside the data plane VPC or VNet, such as a database in a peered network | ||
| | Allowed. | ||
|
|
||
| | Addresses inside the data plane VPC or VNet CIDR, and inside the Kubernetes pod and service CIDRs | ||
| | Blocked, except on the ports Redpanda uses to operate the cluster. Add an egress allowlist to open these destinations. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== File excerpts ==\n'
sed -n '1,120p' modules/networking/pages/byoc/connect-egress-allowlist.adoc
printf '\n----\n'
sed -n '150,210p' modules/networking/pages/cloud-security-network.adoc
printf '\n----\n'
sed -n '360,420p' modules/manage/pages/terraform-provider.adoc
printf '\n== Search for related wording ==\n'
rg -n "peered VPC|peered network|private addresses outside|publicly routable endpoints|allowed_destination_cidr_ports|egress allowlist|Blocked, except" modules -SRepository: redpanda-data/cloud-docs
Length of output: 17803
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Narrow the inspection to the three relevant documents and nearby references.
for f in \
modules/networking/pages/byoc/connect-egress-allowlist.adoc \
modules/networking/pages/cloud-security-network.adoc \
modules/manage/pages/terraform-provider.adoc
do
echo "===== $f ====="
nl -ba "$f" | sed -n '1,260p'
echo
doneRepository: redpanda-data/cloud-docs
Length of output: 275
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Look for all mentions of the specific behavior statements across the docs.
rg -n -C 2 "database in a peered|private address in a peered VPC|private addresses outside the data plane VPC or VNet|publicly routable endpoints|allowed_destination_cidr_ports|egress allowlist" modules -SRepository: redpanda-data/cloud-docs
Length of output: 10368
Align the BYOC egress docs. connect-egress-allowlist.adoc and cloud-security-network.adoc say private addresses in a peered VPC/VNet are allowed by default, while terraform-provider.adoc says they’re blocked. Make the three descriptions consistent.
📍 Affects 3 files
modules/networking/pages/byoc/connect-egress-allowlist.adoc#L32-L36(this comment)modules/networking/pages/cloud-security-network.adoc#L172-L174modules/manage/pages/terraform-provider.adoc#L384-L386
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@modules/networking/pages/byoc/connect-egress-allowlist.adoc` around lines 32
- 36, Align the private-address egress behavior across all three documentation
sites: update the descriptions in
modules/networking/pages/byoc/connect-egress-allowlist.adoc (lines 32-36),
modules/networking/pages/cloud-security-network.adoc (lines 172-174), and
modules/manage/pages/terraform-provider.adoc (lines 384-386) so they
consistently state whether peered VPC/VNet addresses are allowed or blocked by
default, while preserving the documented exception for Redpanda cluster ports
and allowlist behavior.
- Remove the leftover draft marker from the new page - Give in-VPC DNS resolution its own row, and state it consistently in cloud-security-network.adoc - Replace the brittle "fourth row" reference with the behavior it describes - Trim the Terraform lead-in that duplicated the shared partial, and point the partial's always-blocked bullet at Default pipeline egress - Use glossterm:data plane[] on first mention, and settle on "data plane VPC" - Name the required Cloud API service account instead of "ability to run PATCH requests"; document the port_end upper bound - Drop curl -vv so the bearer token is not echoed into logs (CodeRabbit) - Link the Connect quickstart to the new page for the Cloud API path DOC-2069 Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
modules/develop/pages/connect/connect-quickstart.adoc (1)
363-383: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winBump the Terraform provider minimum to
>= 2.1.2.
modules/develop/pages/connect/connect-quickstart.adoc:363still says>= 2.1.1, but this egress example needs the newer provider version.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@modules/develop/pages/connect/connect-quickstart.adoc` around lines 363 - 383, Update the Terraform provider version requirement in the egress allowlist guidance near the redpanda_connect example from >= 2.1.1 to >= 2.1.2, leaving the rest of the example and surrounding documentation unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@modules/develop/pages/connect/connect-quickstart.adoc`:
- Around line 363-383: Update the Terraform provider version requirement in the
egress allowlist guidance near the redpanda_connect example from >= 2.1.1 to >=
2.1.2, leaving the rest of the example and surrounding documentation unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 3eb667f6-110e-4a13-87a2-e072d40b24ee
📒 Files selected for processing (5)
modules/develop/pages/connect/connect-quickstart.adocmodules/manage/pages/terraform-provider.adocmodules/networking/pages/byoc/connect-egress-allowlist.adocmodules/networking/pages/cloud-security-network.adocmodules/networking/partials/connect-egress-allowlist-rules.adoc
🚧 Files skipped from review as they are similar to previous changes (4)
- modules/networking/pages/cloud-security-network.adoc
- modules/networking/partials/connect-egress-allowlist-rules.adoc
- modules/manage/pages/terraform-provider.adoc
- modules/networking/pages/byoc/connect-egress-allowlist.adoc
10.0.0.0/8, 172.16.0.0/16, are accepted only on these specific destination ports, 53, 9091, 9092, 30081, 30082, 30092
Existing pipeline pods will be restarted to pick up changes. |
2 participants
Closes DOC-2069. Documents the Redpanda Connect egress allowlist (
redpanda_connect.allowed_destination_cidr_ports), shipped in 26.2 per ENG-1154.The Terraform surface was already documented in #641. This adds the Cloud API surface, which had no coverage, gives the feature a home in Networking where network admins look for it, and documents the default pipeline egress behavior.
Changes
networking:byoc/connect-egress-allowlist.adoc— how-to covering default pipeline egress, the Cloud APIPATCHflow, Terraform, rule limits, when changes take effect, and troubleshooting. Nav entry under Networking > BYOC.networking/partials/connect-egress-allowlist-rules.adoc— shared rule fields and limits, so the field reference lives in one place.cloud-security-network.adoc— new "Redpanda Connect pipeline egress" section. This page documents every other firewall boundary but was silent on the pipeline sandbox.terraform-provider.adoc— uses the shared partial; corrects the minimum provider version to>= 2.1.2.whats-new-cloud.adoc— July entry broadened from Terraform-only to the feature, now mentioning Cloud API support.Source verification
Verified against
cloudv2@c9e588c(main), not from prior knowledge:ClusterCreateandClusterUpdate:proto/public/cloud/redpanda/api/controlplane/v1/cluster.proto:2438-2481; present in the prod OpenAPI.apps/redpanda-connect-sidecar/main.go:230-257. Default-blocked and never-allowlistable sets confirmed identical across AWS, GCP, and Azure provisioners.5e4898b(2026-07-13), shipped in v2.1.2 — hence the2.1.1→2.1.2correction.Local build passes with no errors or warnings on any file in this PR.
Preview pages
Two questions for reviewers
develop/connect/connect-quickstart.adoc:376andmanage/terraform-provider.adoc:384say a database "with a private address in a peered VPC" is blocked by default. Reading the nftables rule order, peer space (10.0.0.0/8,172.16.0.0/12,192.168.0.0/16) is accepted before those destinations would be dropped, so the statement looks correct only for BYOVPC, where the customer VPC is itself the internal CIDR. Deliberately left unchanged in this PR pending confirmation — @paulzhang97 or @gavinheavyside, can you confirm? If it needs correcting, I'll do it here.-setup, with no runtime refresh, so a running pipeline should keep its old ruleset until restarted. The new page says so and carries a// TODOasking whether the platform restarts pipeline pods as part of the cluster update. Confirmation would let me drop the TODO.🤖 Generated with Claude Code