Skip to content

feat: add support for HashiCorp Vault integration#10

Merged
yusufozturk merged 6 commits into
mainfrom
feature/vault-support
Jun 14, 2026
Merged

feat: add support for HashiCorp Vault integration#10
yusufozturk merged 6 commits into
mainfrom
feature/vault-support

Conversation

@erenaslandev

@erenaslandev erenaslandev commented Jun 11, 2026

Copy link
Copy Markdown
Member

Summary by CodeRabbit

  • New Features
    • Opt-in Vault secret-store topology: adds a Vault dev server with TLS, seeds declared secrets into KV paths, mounts Vault TLS into the subject, and gates subject startup on vault-init completion. Supports combining with Kafka orchestration, validates secret paths/fields and tokens, and prevents reserved-name collisions. Ensures secret values aren’t exposed in generated compose files or command outputs.
  • Bug Fixes
    • Fixed HTTP batching to prevent reused-buffer aliasing that could cause duplicate/blank lines.
  • Tests
    • Added coverage for Vault provisioning, compose rendering/validation, orchestration interactions, and the HTTP batching behavior.
  • Documentation
    • Documented the Vault topology and expected YAML shape in FUTURE-CAPABILITIES.md.

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: e8759e44-4489-45d1-ab0f-9920cd4e51f6

📥 Commits

Reviewing files that changed from the base of the PR and between c04f40c and 0562ca9.

📒 Files selected for processing (2)
  • internal/config/case.go
  • internal/orchestrator/docker.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • internal/config/case.go
  • internal/orchestrator/docker.go

Walkthrough

Adds an opt-in in-topology Vault capability (config, host prep, compose wiring, seeding, validation, tests, docs) and fixes an HTTP generator buffer-aliasing bug with validating tests.

Changes

Vault Dev-Server Topology

Layer / File(s) Summary
Vault configuration and validation
internal/config/case.go
VaultConfig type defines image/token/mount/secrets with defaults. TestCase.Vault enables the block; UsesVault() reports enablement. Validate() enforces non-empty secrets, regex restrictions on token/mount/path/key, and reserves vault/vault-init endpoint names.
Vault host directory and seed preparation
internal/orchestrator/vault.go
PrepareVault() creates TLS (mode 0777) and secrets (mode 0700) dirs, deterministically serializes v.Secrets to per-path JSON files (mode 0600) sorted by path, and returns VaultPaths mapping secret paths to filenames.
Docker compose template and runner integration
internal/orchestrator/docker.go, internal/runner/runner.go
RunConfig gains VaultTLSHost, VaultSecretsHost, and VaultSeeds. NewComposeRunner calls PrepareVault when needed. Compose template conditionally adds vault (TLS dev server, healthcheck) and vault-init (one-shot seeder), mounts /vault-tls:ro into the subject, gates subject on vault-init, and writeCompose builds VaultInitCmd iterating sorted seeds. Runner cleanup list extended to remove fixed bench-vault and bench-vault-init.
Compose rendering, tests, and capability documentation
internal/orchestrator/vault_test.go, FUTURE-CAPABILITIES.md
Tests: TestComposeRendersVault, TestComposeOmitsVaultByDefault, TestComposeRendersKafkaAndVaultTogether, TestWriteComposeRejectsUnpreparedVault, TestValidateRejectsBadVault, TestPrepareVaultWritesSeeds. FUTURE-CAPABILITIES.md documents the vault: schema, TLS/CA handling, vault-init seeding flow, output-safety guarantees, validation rules, reserved names, and kafka+vault coexistence.

HTTP Generator Buffer Aliasing Fix

Layer / File(s) Summary
HTTP batching defensive copy and regression test
containers/generator/main.go, containers/generator/http_test.go
runHTTPSingle trims trailing newline and copies each line into a new slice before appending to the batch to avoid buffer reuse/aliasing and doubled newlines on join. TestRunHTTPSingleBatchesDistinctLines verifies received payload contains non-empty distinct records with unique (conn,seq) pairs to detect aliasing regressions.

Sequence Diagram(s)

sequenceDiagram
  participant Runner
  participant PrepareVault
  participant NewComposeRunner
  participant writeCompose
  participant DockerCompose
  Runner->>PrepareVault: create TLS/secrets dirs
  PrepareVault->>PrepareVault: serialize secrets to JSON
  PrepareVault->>NewComposeRunner: return VaultPaths
  NewComposeRunner->>writeCompose: pass RunConfig with paths
  writeCompose->>writeCompose: build VaultInitCmd
  writeCompose->>DockerCompose: render vault + vault-init
  DockerCompose->>DockerCompose: subject depends_on vault-init
  DockerCompose->>DockerCompose: mount /vault-tls:ro
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • VirtualMetric/PipeBench#5: Extends the reserved-name endpoint validation introduced in PR #5 by adding vault and vault-init to reserved service names.

Suggested reviewers

  • yusufozturk

Poem

🐰 I seeded secrets in tidy, sorted rows,
TLS tucked in where the dev server glows.
Lines no longer borrow each other's face,
Each CONN and SEQ found its own happy place.
Hop, test, and compose — the harness hums with grace.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main feature addition: HashiCorp Vault integration support across configuration, orchestration, and testing.
Docstring Coverage ✅ Passed Docstring coverage is 83.33% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/vault-support

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
FUTURE-CAPABILITIES.md (1)

270-270: Verify Vault minimum version for -dev-tls

The -dev-tls server flag was added in Vault v1.12.0-rc1, so “needs >= 1.12” is accurate; consider clarifying it as “>= 1.12.0-rc1”.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@FUTURE-CAPABILITIES.md` at line 270, Update the Vault version note to
explicitly reference the pre-release that introduced the -dev-tls flag by
changing the “needs >= 1.12” text to “needs >= 1.12.0-rc1” (or add “(introduced
in 1.12.0-rc1)”) near the image line `image: "hashicorp/vault:1.20"` so readers
know the -dev-tls flag first appeared in v1.12.0-rc1.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@FUTURE-CAPABILITIES.md`:
- Line 287: The vmetric CA path example is inconsistent: the text shows an
absolute path `/vault-tls/vault-ca.pem` while the vmetric config uses a relative
value `ca_name: "vault-tls/vault-ca.pem"`. Update FUTURE-CAPABILITIES.md so both
the explanatory text and the vmetric example use the same path format (either
both absolute with leading slash or both relative without), and add a short note
explaining whether `ca_name` expects an absolute filesystem path or a
vault-style key (e.g., "use absolute filesystem path for ca_name" or "ca_name is
a vault key and should be specified relative"). Ensure you edit the lines
referencing `/vault-tls/vault-ca.pem` and the `ca_name:
"vault-tls/vault-ca.pem"` example to match.
- Around line 303-306: The documentation is inconsistent about whether the
compose key is kafka: or redpanda; clarify that by updating the
FUTURE-CAPABILITIES.md text to explicitly state the mapping between the two
names (e.g., that kafka: declarations in the schema are implemented/realized as
an internal redpanda service and therefore redpanda and redpanda-init are
reserved), and mention the corresponding reserved endpoint names (redpanda,
redpanda-init vs kafka-init) so readers know which identifiers to use with
endpoints:. Update the lines referencing kafka:, redpanda, redpanda-init, and
kafka-init to include a short parenthetical note explaining the relationship.

---

Nitpick comments:
In `@FUTURE-CAPABILITIES.md`:
- Line 270: Update the Vault version note to explicitly reference the
pre-release that introduced the -dev-tls flag by changing the “needs >= 1.12”
text to “needs >= 1.12.0-rc1” (or add “(introduced in 1.12.0-rc1)”) near the
image line `image: "hashicorp/vault:1.20"` so readers know the -dev-tls flag
first appeared in v1.12.0-rc1.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 6d122f58-2ba5-4f3f-a84c-a395080d145b

📥 Commits

Reviewing files that changed from the base of the PR and between 01a5a27 and e0d2d5f.

📒 Files selected for processing (8)
  • FUTURE-CAPABILITIES.md
  • containers/generator/http_test.go
  • containers/generator/main.go
  • internal/config/case.go
  • internal/orchestrator/docker.go
  • internal/orchestrator/vault.go
  • internal/orchestrator/vault_test.go
  • internal/runner/runner.go

Comment thread FUTURE-CAPABILITIES.md Outdated
Comment thread FUTURE-CAPABILITIES.md Outdated
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 11, 2026

Copy link
Copy Markdown

Deploying pipebench with  Cloudflare Pages  Cloudflare Pages

Latest commit: 0562ca9
Status: ✅  Deploy successful!
Preview URL: https://fa43e158.pipebench.pages.dev
Branch Preview URL: https://feature-vault-support.pipebench.pages.dev

View logs

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
internal/orchestrator/docker.go (1)

53-69: ⚠️ Potential issue | 🔴 Critical | ⚡ Quick win

Close the Vault block before the cloud-emulator sections.

Lines 53-69 and 457-555 currently scope the AWS/Azure blocks inside {{ if .VaultEnabled }}. That makes AWS/Azure-only cases render the wrong YAML (subject.depends_on: without the emulator entries, and no localstack/azurite services), and it leaves the template control flow unbalanced so template.Parse can fail.

🔧 Minimal fix
 {{- if .VaultEnabled }}
       vault-init:
         condition: service_completed_successfully
+{{- end }}
 {{- if .AWSEnabled }}
       localstack:
         condition: service_healthy
 {{- end }}
@@
   vault-init:
     image: "{{ .VaultImage }}"
@@
     command:
       - "{{ .VaultInitCmd }}"
+{{- end }}
 {{- if .AWSEnabled }}

   localstack:

Also applies to: 457-555

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/orchestrator/docker.go` around lines 53 - 69, The Vault conditional
block is left open so the AWS/Azure emulator blocks get nested inside {{ if
.VaultEnabled }} causing malformed YAML and unbalanced template flow; close the
Vault block (add the missing {{ end }} immediately after the vault-init section)
so that the subsequent {{ if .AWSEnabled }} (localstack) and {{ if .AzureEnabled
}} (azure-init/azurite) blocks are siblings under depends_on, and ensure every
opened {{ if ... }} (e.g., {{ if .VaultEnabled }}, {{ if .AWSEnabled }}, {{ if
.AzureEnabled }}) has a matching {{ end }} so template.Parse succeeds.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@internal/orchestrator/docker.go`:
- Around line 53-69: The Vault conditional block is left open so the AWS/Azure
emulator blocks get nested inside {{ if .VaultEnabled }} causing malformed YAML
and unbalanced template flow; close the Vault block (add the missing {{ end }}
immediately after the vault-init section) so that the subsequent {{ if
.AWSEnabled }} (localstack) and {{ if .AzureEnabled }} (azure-init/azurite)
blocks are siblings under depends_on, and ensure every opened {{ if ... }}
(e.g., {{ if .VaultEnabled }}, {{ if .AWSEnabled }}, {{ if .AzureEnabled }}) has
a matching {{ end }} so template.Parse succeeds.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: f32e062d-cbab-4707-aba2-eb141ee457cd

📥 Commits

Reviewing files that changed from the base of the PR and between 8e0f749 and 0a46960.

📒 Files selected for processing (4)
  • containers/generator/main.go
  • internal/config/case.go
  • internal/orchestrator/docker.go
  • internal/runner/runner.go
🚧 Files skipped from review as they are similar to previous changes (3)
  • internal/runner/runner.go
  • containers/generator/main.go
  • internal/config/case.go

yusufozturk and others added 2 commits June 15, 2026 01:39
Resolve conflicts where Vault and Minio support landed independently:

- internal/orchestrator/docker.go: combine both conditions into the
  subject depends_on guard ({{if or .Kafka .AWS .Azure .Vault .Minio}}).
- internal/config/case.go: keep main's regrouped `reserved` service-name
  map (cloud emulators incl. minio, kafka brokers) and add the vault/
  vault-init entries as their own group.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@yusufozturk yusufozturk merged commit bea48a9 into main Jun 14, 2026
5 checks passed
@yusufozturk yusufozturk deleted the feature/vault-support branch June 14, 2026 23:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants