Skip to content

docs: Terraform shadow link management (DOC-2022) - #1827

Merged
Feediver1 merged 2 commits into
betafrom
DOC-2022-terraform-shadow-link
Jul 28, 2026
Merged

docs: Terraform shadow link management (DOC-2022)#1827
Feediver1 merged 2 commits into
betafrom
DOC-2022-terraform-shadow-link

Conversation

@Feediver1

@Feediver1 Feediver1 commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Resolves https://redpandadata.atlassian.net/browse/DOC-2022

Adds Terraform as a management option on the Configure Shadowing page, covering the redpanda_shadow_link resource shipped in Redpanda Terraform provider v2.0.0 (PR #343; current release v2.1.2).

Changes

  • Create a shadow link: new Terraform tab in the cloud-gated tabs block (alongside Cloud UI / rpk / Control Plane API). Shows the redpanda_secret + redpanda_shadow_link pattern with TLS and SCRAM auth, the ${secrets.<name>} password reference (no plaintext in state), placeholder explanations in the page's existing style, and a NOTE covering the enable_shadow_linking prerequisite (settable via redpanda_cluster.cluster_configuration.custom_properties_json) and the allow_deletion = true destroy guard.
  • Update an existing shadow link: new Terraform tab noting that terraform apply sends only changed fields (the provider uses a sparse FieldMask), and that name and cluster IDs are immutable (changes plan a replacement).

Both edits are inside existing ifdef::env-cloud[] regions of the single-sourced page, so they render only in the cloud build; the Self-Managed build is unchanged. The cloud-docs stub picks the content up automatically.

Review notes

  • The HCL example is adapted from the provider's own examples/shadow_link/main.tf and trimmed to the minimal verified surface. Sync-option blocks are deliberately not shown inline; the tab links to the registry reference and the provider example for the full schema, to avoid drifting from generated docs.
  • The provider example's ACL blocks are not reproduced because they grant fewer permissions than this page's "Replication service permissions" prerequisites require (missing describe_configs); the tab points at the prerequisites instead. Flagged to the provider team separately.
  • The provider does no client-side check of enable_shadow_linking (verified in resource_shadowlink.go Create()), so the property is framed as a prerequisite, not a validated field.
  • No Private Link claims are made; PL/PSC validation for shadow links was still in flight per the engineering epic's last update.
  • Companion cloud-docs PR adds the shadow_link bullet to the Terraform provider resource list.

🤖 Generated with Claude Code

Preview pages

Note for reviewers: this preview renders the Self-Managed variant, which is intentionally unchanged (verified: zero Terraform mentions). The new Terraform tabs are inside ifdef::env-cloud[] regions and render only in the cloud build. Rendering was verified in a local cloud-docs build with the playbook pointed at this branch: both tabs appear in the create and update tab blocks, the HCL example and nested NOTE render cleanly, and no AsciiDoc directives leak.

Adds a Terraform tab to the cloud-gated create and update tab blocks
on the Configure Shadowing page, covering the redpanda_shadow_link
resource (provider 2.0.0+), the redpanda_secret password pattern,
the enable_shadow_linking prerequisite, and the allow_deletion guard.

Co-Authored-By: Claude Fable 5 <[email protected]>
@Feediver1
Feediver1 requested a review from a team as a code owner July 25, 2026 18:49
@netlify

netlify Bot commented Jul 25, 2026

Copy link
Copy Markdown

Deploy Preview for redpanda-docs-preview ready!

Name Link
🔨 Latest commit 769ca67
🔍 Latest deploy log https://app.netlify.com/projects/redpanda-docs-preview/deploys/6a68189e55ccc30008b7b694
😎 Deploy Preview https://deploy-preview-1827--redpanda-docs-preview.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai

coderabbitai Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 495187dd-a664-41cd-b3d9-64e777d6e72a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch DOC-2022-terraform-shadow-link

Comment @coderabbitai help to get the list of available commands.

@gene-redpanda gene-redpanda left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Feediver1 Feediver1 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review (docs-team-standards final pass)

Files reviewed: 1 (setup.adoc — Terraform tabs in the cloud-gated create and update blocks)
Overall assessment: Ready to merge. SME-approved, conflict-free, and every claim traces to verified sources. Disclosure: this is a docs-team self-review of docs-team-authored content — the pass re-verified everything fresh rather than assuming.

What changed since the PR opened

  • @gene-redpanda approved — the provider engineer's LGTM retroactively blesses the draft's two judgment calls: pointing at the page's "Replication service permissions" list instead of reproducing the provider example's under-privileged ACLs, and making no Private Link claims.
  • The branch is 10 commits behind beta, but setup.adoc is untouched on beta since the fork — mergeable with no re-merge needed.

Verified fresh

  • The $${secrets.${...}} HCL escape matches the shipped provider example and renders correctly (confirmed in a local cloud-docs cross-build).
  • The <<Replication service permissions>> internal anchor exists on the branch; all registry links target live pages (provider v2.0.0+ released).
  • The update-tab "sends only changed fields" claim is source-verified against the provider's sparse FieldMask implementation.
  • Both tabs sit inside existing ifdef::env-cloud[] regions — the Self-Managed build is untouched.

Impact

  • No collision with #1819: it edits setup.adoc at ~line 41; this PR's hunks are at lines 420+ and 906+. Clean in any merge order.
  • Companion cloud-docs#649 (resource-list bullet + cloud What's New entry) should merge around the 26.2 promotion so the cloud discovery surface lands with the tabs.
  • Upstream terraform-provider-redpanda#361 (under-privileged example ACLs) remains open — non-blocking here because the tab points at the docs' permission list, not the example.

What works well

  • Teaches the one pattern that matters (secret-backed password, nothing in state) and delegates the full schema to the registry — zero duplicated reference content to go stale.
  • Placeholder explanations carry the operational reality: which resources manage the source-side user and ACLs, and where the required permissions live.
  • The allow_deletion destroy-guard NOTE preempts the most likely support question.

No critical issues, no suggestions. With the SME approval in place, this is merge-ready for the 26.2 window.

🤖 Generated with Claude Code

@micheleRP

Copy link
Copy Markdown
Contributor

Minor, pre-existing (not from your diff) — since this PR already touches this file, could you fold in a one-line fix?

Line 42 of modules/manage/pages/disaster-recovery/shadowing/setup.adoc currently reads:

  • If you use Redpanda Console, ensure that it is running v3.30 or later.

There's no Console v3.30 — the latest release is v3.9.0 (2026-07-23), so this reads as a version that doesn't exist. It looks like a typo for v3.3.0, which shipped 2025-11-19, right after the shadow-link create wizard and edit support merged (console#2017, #2025, #2030).

Suggested:

- If you use Redpanda Console, ensure that it is running v3.3.0 or later.

One thing to confirm before you apply it: which capability was this prerequisite gating? If it was meant to cover Schema Registry shadowing specifically, the floor is later than v3.3.x — console#2058 ("support shadowing sr") merged 2025-12-02, after v3.3.2. If you're not sure, I'm happy to take it as a follow-up instead so it doesn't hold up this PR.

(Unrelated to this line: the new 26.2 API-mode Schema Registry forms need Console v3.9.0 and a cluster reporting Redpanda 26.2.0. I'm covering that separately in DOC-2335 — no need to add it here.)

@micheleRP

Copy link
Copy Markdown
Contributor

Another pass over the Terraform tabs, this time checking the HCL against the provider's shipped schema and source. One thing needs fixing before merge; the rest are suggestions.

Critical

1. setup.adoc:467 — the <secret-name> guidance omits the uppercase-only constraint, so a reader following the page's own naming style hits a terraform plan error.

redpanda_secret.name carries a hard validator (redpanda/resources/secret/schema_resource.go:49):

stringvalidator.RegexMatches(secretNamePattern, "must match ^[A-Z][A-Z0-9_]*$ (uppercase letters, digits, underscores; must start with a letter)")

The bullet currently reads "Name for the secret that stores the SASL/SCRAM password" — and the very next bullet gives production-dr as the shadow link name example, which primes lowercase-hyphen. A reader writes source-password and terraform plan fails validation. The provider's own examples/shadow_link/variables.tf documents the constraint ("uppercase, A-Z0-9_"), and our published cloud page already documents the pattern in Manage cluster secrets.

Suggested:

* `<secret-name>`: Name for the secret that stores the SASL/SCRAM password. Must be uppercase, matching `^[A-Z][A-Z0-9_]*$`, for example, `SOURCE_SCRAM_PASSWORD`.

Suggestions

2. setup.adoc:303 — the preceding "create the secret" step has no Terraform tab. That block offers Cloud UI / rpk / Data Plane API, so a reader on the Terraform path reaches step 1 with no Terraform option, then finds secret creation folded into step 2's Terraform tab. It also means tab selection doesn't sync between the two blocks. Either add a short Terraform tab there (just the redpanda_secret resource, or a pointer forward to the next step), or say in the step-2 Terraform tab that it covers both steps.

3. setup.adoc:913 — the replacement claim in the update tab is incomplete. "Terraform plans a replacement (destroy and recreate)" is accurate for name, shadow_redpanda_id, and source_redpanda_id (all RequiresReplace(), verified in schema_resource_gen.go) — but allow_deletion defaults to false, so the destroy half fails at apply time unless the resource sets it. A clause connecting this back to the create tab's NOTE would close the loop.

4. setup.adoc:433 — cross-link our own secret guidance, not just the registry. xref:manage:terraform-provider.adoc#manage-cluster-secrets[] already covers the name pattern, the write-only behavior, and rotation via secret_data_version. Since the tab is inside ifdef::env-cloud[], that xref resolves in the cloud build and never renders in the Self-Managed build.

5. setup.adoc:440secret_data_version = 1 is unexplained. One clause ("increment this when you rotate the password") stops it reading as boilerplate.

Still open from the earlier review

The Console version fix hasn't landed yet — setup.adoc:42 still reads v3.30, which is not a released version. Suggested: v3.3.0 (shipped 2025-11-19, right after the shadow-link create wizard and edit support merged). Worth confirming which capability the prerequisite was gating first: if it was meant to cover Schema Registry shadowing, the floor is later than v3.3.x (console#2058 merged 2025-12-02, after v3.3.2). Happy for this to be a follow-up if you'd rather not hold the PR.

@micheleRP micheleRP left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please see comments before merging!

Per Michele's review:
- Document the uppercase-only constraint on redpanda_secret names
  (provider validator ^[A-Z][A-Z0-9_]*$), with an example.
- Add a Terraform tab to the create-the-secret step pointing forward
  (the secret is defined with the shadow link), keeping tab selection
  in sync across the two blocks.
- Tie the update tab's replacement behavior back to allow_deletion.
- Cross-link Manage cluster secrets for naming/write-only/rotation.
- Explain secret_data_version with an inline rotation comment.
- Fix the pre-existing Console version typo: v3.30 -> v3.3.0 (the
  shadow-link Console support floor; the 26.2 SR-shadowing Console
  requirement is covered separately in DOC-2335).

Co-Authored-By: Claude Fable 5 <[email protected]>
@Feediver1

Copy link
Copy Markdown
Contributor Author

All feedback applied in 769ca67e — thanks @micheleRP, the schema-vs-prose pass caught exactly the failure a reader would hit:

  1. Critical (secret-name constraint): applied your suggested bullet verbatim — uppercase pattern ^[A-Z][A-Z0-9_]*$ with SOURCE_SCRAM_PASSWORD as the example.
  2. Secret-step Terraform tab: added a short forward-pointer tab (the secret is defined together with the shadow link in the next step's configuration), which also keeps tab selection synced across the two blocks.
  3. Update-tab replacement claim: now closes the loop — the destroy half of a replacement requires allow_deletion = true, referencing the create step's guard.
  4. Cross-link: the intro now points at xref:manage:terraform-provider.adoc#manage-cluster-secrets (naming, write-only behavior, rotation) alongside the registry link — safe because the tab is cloud-gated.
  5. secret_data_version: inline comment added ('Increment when you rotate the password.').
  6. Console version typo: v3.30v3.3.0. On your confirm-first question: the prerequisite predates the 26.2 Schema Registry shadowing work and sits in the general Shadowing requirements, so it gates the Console shadow-link UI (create wizard/edit, which shipped just before v3.3.0) — v3.3.0 is the right floor. The SR-shadowing Console requirement (v3.9.0 + 26.2 cluster) stays with DOC-2335 as you planned.

🤖 Generated with Claude Code

@Feediver1
Feediver1 merged commit a8458be into beta Jul 28, 2026
7 checks passed
@Feediver1
Feediver1 deleted the DOC-2022-terraform-shadow-link branch July 28, 2026 02:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants