Add fly:bats rake task for running BATs from a local branch#2753
Open
colins wants to merge 1 commit into
Open
Conversation
Introduces `bundle exec rake fly:bats` (from src/) so developers can run
the full BOSH Acceptance Test suite against their local branch without
waiting for the shared CI pipeline.
How it works
------------
The rake task sets a standalone Concourse pipeline (ci/fly-bats.yml),
then triggers and streams the single `bats` job. The pipeline:
1. Checks out the current branch from GitHub (branch is pushed
automatically before set-pipeline).
2. Runs make-candidate.yml to build a dev release tarball.
3. Compiles the tarball via bosh-deployment/ci/tasks/shared/bosh-agent-compile.yml.
4. Runs ci/tasks/run-bats-pipeline.sh which provisions a GCP environment
with terraform, deploys the BOSH director, runs BATs, and tears down.
All GCP credentials are resolved from the shared Concourse credential
store (((gcp_json_key)) / ((gcp_project_id))).
Files added
-----------
- ci/fly-bats.yml — standalone Concourse pipeline (replaces the
two separate fly-execute task files)
- ci/tasks/run-bats-pipeline.sh — orchestrates the full BATs lifecycle in GCP
Env-var overrides (all optional)
---------------------------------
BATS_ENV_NAME terraform env name (default: bats-local)
STEMCELL_NAME override GCP stemcell
DEPLOY_ARGS extra ops-files for bosh create-env
BAT_RSPEC_FLAGS extra RSpec flags (e.g. "--tag wip")
CONCOURSE_TARGET target alias (default: bosh)
WalkthroughAdds a new Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@ci/fly-bats.yml`:
- Around line 43-46: The stemcell resource is still hardcoded to a fixed name,
so the pipeline override is not applied end-to-end. Update the stemcell resource
definition in the fly config to use the existing stemcell_name variable instead
of the literal bosh-google-kvm-ubuntu-noble value, and keep the task/job param
wiring consistent with the same variable so the fetched stemcell matches the
override.
In `@ci/tasks/run-bats-pipeline.sh`:
- Around line 34-40: The Terraform install path in run-bats-pipeline.sh
downloads and unpacks a release directly into /usr/local/bin without any
integrity check. Update the Terraform installation block to either use a
pre-baked image/tooling layer or verify the downloaded archive against the
official checksum/signature before calling unzip and chmod, using the existing
terraform install flow and TERRAFORM_VERSION lookup as the entry point.
- Around line 48-72: Move the EXIT cleanup setup so it is registered before any
Terraform work starts in run-bats-pipeline.sh; the current flow around terraform
init, terraform apply, and terraform output can fail before teardown is armed,
leaving partial GCP resources behind. Update the script so the trap that calls
teardown is installed before entering the Terraform provisioning block, while
keeping the existing teardown function and ENV_NAME/TERRAFORM_DIR flow intact.
- Around line 157-166: Preserve caller-supplied BAT_RSPEC_FLAGS in
run-bats-pipeline.sh by saving its current value before sourcing
bats-config/bats.env, since that file sets a default export which overwrites it.
Update the sourcing flow around the bats.env load so the existing
BAT_RSPEC_FLAGS is restored or appended afterward, keeping the behavior in the
BAT_RSPEC_FLAGS handling block and the bats-config/bats.env source step
consistent.
- Line 17: The shell script currently uses set -eu without pipefail, so failures
in piped commands can be masked. Update the script header to use set -euo
pipefail in the run-bats-pipeline.sh flow, so the terraform output -json | jq
... pipeline fails correctly instead of allowing an empty environment/metadata
file. Use the existing pipeline in this script as the place to apply the change.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: aa46ba66-4144-4592-8494-bed921444c83
📒 Files selected for processing (3)
ci/fly-bats.ymlci/tasks/run-bats-pipeline.shsrc/tasks/fly.rake
Comment on lines
+43
to
+46
| - name: stemcell | ||
| type: bosh-io-stemcell | ||
| source: | ||
| name: bosh-google-kvm-ubuntu-noble |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Wire stemcell_name into the resource definition.
Line 46 is still pinned to bosh-google-kvm-ubuntu-noble, so STEMCELL_NAME only changes the task param later in the job, not the stemcell fetched by this pipeline. That means the advertised override is not actually end-to-end.
Suggested fix
- name: stemcell
type: bosh-io-stemcell
source:
- name: bosh-google-kvm-ubuntu-noble
+ name: ((stemcell_name))📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| - name: stemcell | |
| type: bosh-io-stemcell | |
| source: | |
| name: bosh-google-kvm-ubuntu-noble | |
| - name: stemcell | |
| type: bosh-io-stemcell | |
| source: | |
| name: ((stemcell_name)) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ci/fly-bats.yml` around lines 43 - 46, The stemcell resource is still
hardcoded to a fixed name, so the pipeline override is not applied end-to-end.
Update the stemcell resource definition in the fly config to use the existing
stemcell_name variable instead of the literal bosh-google-kvm-ubuntu-noble
value, and keep the task/job param wiring consistent with the same variable so
the fetched stemcell matches the override.
| # DEPLOY_ARGS – Extra ops-file args for bosh create-env | ||
| # BAT_RSPEC_FLAGS – Extra flags appended to the BAT run (optional) | ||
|
|
||
| set -eu |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the target script around the cited lines.
git ls-files ci/tasks/run-bats-pipeline.sh
wc -l ci/tasks/run-bats-pipeline.sh
cat -n ci/tasks/run-bats-pipeline.sh | sed -n '1,140p'Repository: cloudfoundry/bosh
Length of output: 7330
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check whether the script already uses pipefail or guards the pipeline in another way.
rg -n "pipefail|terraform output|jq|environment/metadata|metadata" ci/tasks/run-bats-pipeline.shRepository: cloudfoundry/bosh
Length of output: 658
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate any downstream consumers that assume the metadata file exists and is valid.
rg -n "environment/metadata|metadata" ci tasks .github . | sed -n '1,200p'Repository: cloudfoundry/bosh
Length of output: 23814
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Probe shell semantics relevant to the claim in a read-only way.
python3 - <<'PY'
import subprocess, textwrap, os, tempfile, pathlib, json, sys
scripts = {
"no_pipefail": "set -eu\nfalse | cat > /tmp/out1\nprintf done\n",
"with_pipefail": "set -euo pipefail\nfalse | cat > /tmp/out2\nprintf done\n",
}
for name, body in scripts.items():
p = subprocess.run(["bash", "-lc", body], capture_output=True, text=True)
print(name, "rc=", p.returncode, "stdout=", p.stdout.strip(), "stderr=", p.stderr.strip())
PYRepository: cloudfoundry/bosh
Length of output: 429
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check jq pipeline semantics and downstream consumers of the metadata file.
python3 - <<'PY'
import subprocess, textwrap
tests = {
"jq_with_entries_on_eof": "printf '' | jq 'with_entries(.value = .value.value)'; echo rc:$?",
"jq_raw_on_eof": "printf '' | jq -r '.director_public_ip // empty'; echo rc:$?",
}
for name, cmd in tests.items():
p = subprocess.run(["bash", "-lc", cmd], capture_output=True, text=True)
print(f"--- {name} ---")
print("returncode:", p.returncode)
print("stdout:", repr(p.stdout))
print("stderr:", repr(p.stderr))
PY
# Inspect the consumer script that reads environment/metadata.
wc -l ci/bats/iaas/gcp/director-vars ci/bats/iaas/gcp/prepare-bats-config.sh ci/bats/tasks/deploy-director.sh
cat -n ci/bats/iaas/gcp/director-vars | sed -n '1,160p'
echo '---'
cat -n ci/bats/iaas/gcp/prepare-bats-config.sh | sed -n '1,120p'
echo '---'
cat -n ci/bats/tasks/deploy-director.sh | sed -n '1,120p'Repository: cloudfoundry/bosh
Length of output: 8096
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Check jq pipeline semantics and downstream consumers of the metadata file.
python3 - <<'PY'
import subprocess, textwrap
tests = {
"jq_with_entries_on_eof": "printf '' | jq 'with_entries(.value = .value.value)'; echo rc:$?",
"jq_raw_on_eof": "printf '' | jq -r '.director_public_ip // empty'; echo rc:$?",
}
for name, cmd in tests.items():
p = subprocess.run(["bash", "-lc", cmd], capture_output=True, text=True)
print(f"--- {name} ---")
print("returncode:", p.returncode)
print("stdout:", repr(p.stdout))
print("stderr:", repr(p.stderr))
PY
# Inspect the consumer script that reads environment/metadata.
wc -l ci/bats/iaas/gcp/director-vars ci/bats/iaas/gcp/prepare-bats-config.sh ci/bats/tasks/deploy-director.sh
cat -n ci/bats/iaas/gcp/director-vars | sed -n '1,160p'
echo '---'
cat -n ci/bats/iaas/gcp/prepare-bats-config.sh | sed -n '1,120p'
echo '---'
cat -n ci/bats/tasks/deploy-director.sh | sed -n '1,120p'Repository: cloudfoundry/bosh
Length of output: 8096
Add pipefail to the script. terraform output -json | jq ... > environment/metadata can still create an empty metadata file if terraform output fails, because jq exits 0 on EOF. set -euo pipefail closes that gap.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ci/tasks/run-bats-pipeline.sh` at line 17, The shell script currently uses
set -eu without pipefail, so failures in piped commands can be masked. Update
the script header to use set -euo pipefail in the run-bats-pipeline.sh flow, so
the terraform output -json | jq ... pipeline fails correctly instead of allowing
an empty environment/metadata file. Use the existing pipeline in this script as
the place to apply the change.
Comment on lines
+34
to
+40
| if ! command -v terraform &>/dev/null; then | ||
| echo "--- Installing terraform ${TERRAFORM_VERSION} ---" | ||
| curl -sSL \ | ||
| "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip" \ | ||
| -o /tmp/terraform.zip | ||
| unzip -qo /tmp/terraform.zip -d /usr/local/bin terraform | ||
| chmod +x /usr/local/bin/terraform |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
git ls-files ci/tasks/run-bats-pipeline.sh
wc -l ci/tasks/run-bats-pipeline.sh
cat -n ci/tasks/run-bats-pipeline.sh | sed -n '1,140p'Repository: cloudfoundry/bosh
Length of output: 7330
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Look for any existing Terraform version pinning / checksum verification in CI scripts.
rg -n "terraform_|sha256|SHA256|shasum|openssl dgst|gpg|verify.*terraform|terraform.*zip" ci .github .gitlab-ci.yml README.mdRepository: cloudfoundry/bosh
Length of output: 5227
🏁 Script executed:
#!/bin/bash
set -euo pipefail
git ls-files ci/tasks/run-bats-pipeline.sh
wc -l ci/tasks/run-bats-pipeline.sh
cat -n ci/tasks/run-bats-pipeline.sh | sed -n '1,140p'Repository: cloudfoundry/bosh
Length of output: 7330
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Look for any existing Terraform version pinning / checksum verification in CI scripts.
rg -n "terraform_|sha256|SHA256|shasum|openssl dgst|gpg|verify.*terraform|terraform.*zip" ci .github .gitlab-ci.yml README.mdRepository: cloudfoundry/bosh
Length of output: 5227
Verify the Terraform download before installing it
ci/tasks/run-bats-pipeline.sh:34-40 fetches Terraform over the network and installs it into /usr/local/bin with no checksum or signature check. Bake Terraform into the image, or verify the release checksum/signature before unpacking it.
🧰 Tools
🪛 ast-grep (0.44.0)
[warning] 37-37: Writing to or reading from a hardcoded, predictable path under /tmp is vulnerable to symlink and TOCTOU attacks: a local attacker can pre-create the file (or a symlink pointing elsewhere) and hijack or corrupt the contents. Generate a unique, unpredictable temporary file with mktemp instead, e.g. tmpfile="$(mktemp)" (or mktemp -d for directories) and reference "$tmpfile".
Context: /tmp/terraform.zip
Note: [CWE-377] Insecure Temporary File.
(predictable-tmp-file-bash)
[warning] 38-38: Writing to or reading from a hardcoded, predictable path under /tmp is vulnerable to symlink and TOCTOU attacks: a local attacker can pre-create the file (or a symlink pointing elsewhere) and hijack or corrupt the contents. Generate a unique, unpredictable temporary file with mktemp instead, e.g. tmpfile="$(mktemp)" (or mktemp -d for directories) and reference "$tmpfile".
Context: /tmp/terraform.zip
Note: [CWE-377] Insecure Temporary File.
(predictable-tmp-file-bash)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ci/tasks/run-bats-pipeline.sh` around lines 34 - 40, The Terraform install
path in run-bats-pipeline.sh downloads and unpacks a release directly into
/usr/local/bin without any integrity check. Update the Terraform installation
block to either use a pre-baked image/tooling layer or verify the downloaded
archive against the official checksum/signature before calling unzip and chmod,
using the existing terraform install flow and TERRAFORM_VERSION lookup as the
entry point.
Comment on lines
+48
to
+72
| # ── Provision GCP environment via terraform ────────────────────────────────── | ||
| echo "--- Provisioning GCP environment (env: ${ENV_NAME}) ---" | ||
| pushd "${TERRAFORM_DIR}" >/dev/null | ||
|
|
||
| terraform init \ | ||
| -input=false \ | ||
| -reconfigure \ | ||
| -backend-config="bucket=bosh-director-pipeline" \ | ||
| -backend-config="prefix=bats-terraform/${ENV_NAME}" \ | ||
| -backend-config="credentials=${GCP_CREDS_FILE}" | ||
|
|
||
| terraform apply \ | ||
| -input=false \ | ||
| -auto-approve \ | ||
| -var "project_id=${GCP_PROJECT_ID}" \ | ||
| -var "gcp_credentials_json=${GCP_JSON_KEY}" \ | ||
| -var "name=${ENV_NAME}" | ||
|
|
||
| # Convert terraform outputs to the flat metadata JSON consumed by director-vars | ||
| # and prepare-bats-config.sh. | ||
| terraform output -json \ | ||
| | jq 'with_entries(.value = .value.value)' \ | ||
| > "${ROOT_DIR}/environment/metadata" | ||
|
|
||
| popd >/dev/null |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major
🧩 Analysis chain
🏁 Script executed:
sed -n '1,220p' ci/tasks/run-bats-pipeline.sh | cat -nRepository: cloudfoundry/bosh
Length of output: 8522
🏁 Script executed:
grep -nE 'trap|terraform|cleanup|EXIT' ci/tasks/run-bats-pipeline.shRepository: cloudfoundry/bosh
Length of output: 1523
🏁 Script executed:
python3 - <<'PY'
from pathlib import Path
p = Path('ci/tasks/run-bats-pipeline.sh')
text = p.read_text()
for i, line in enumerate(text.splitlines(), 1):
if 40 <= i <= 170:
print(f"{i:4}: {line}")
PYRepository: cloudfoundry/bosh
Length of output: 6362
🏁 Script executed:
python3 - <<'PY'
from pathlib import Path
p = Path('ci/tasks/run-bats-pipeline.sh')
text = p.read_text().splitlines()
for start in (1, 40, 70, 110, 130):
print(f"\n--- {start} ---")
for i in range(start, min(start+40, len(text)+1)):
print(f"{i:4}: {text[i-1]}")
PYRepository: cloudfoundry/bosh
Length of output: 9672
Move the EXIT trap before Terraform runs. terraform init, apply, and output currently run before trap teardown EXIT, so failures there skip cleanup and can leave partial GCP resources behind.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ci/tasks/run-bats-pipeline.sh` around lines 48 - 72, Move the EXIT cleanup
setup so it is registered before any Terraform work starts in
run-bats-pipeline.sh; the current flow around terraform init, terraform apply,
and terraform output can fail before teardown is armed, leaving partial GCP
resources behind. Update the script so the trap that calls teardown is installed
before entering the Terraform provisioning block, while keeping the existing
teardown function and ENV_NAME/TERRAFORM_DIR flow intact.
Comment on lines
+157
to
+166
| # Source the environment file that prepare-bats-config.sh wrote; this exports | ||
| # BOSH_ENVIRONMENT, BOSH_CLIENT, BOSH_CLIENT_SECRET, BOSH_CA_CERT, | ||
| # BOSH_ALL_PROXY, and the default BAT_RSPEC_FLAGS. | ||
| # shellcheck source=/dev/null | ||
| source bats-config/bats.env | ||
|
|
||
| # Allow the caller to append extra RSpec flags (e.g. "--tag wip"). | ||
| if [[ -n "${BAT_RSPEC_FLAGS:-}" ]]; then | ||
| export BAT_RSPEC_FLAGS="${BAT_RSPEC_FLAGS}" | ||
| fi |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate the relevant files and inspect the surrounding lines with numbers.
git ls-files 'ci/tasks/run-bats-pipeline.sh' 'ci/bats/iaas/gcp/prepare-bats-config.sh'
echo '--- run-bats-pipeline.sh ---'
sed -n '130,190p' ci/tasks/run-bats-pipeline.sh | cat -n
echo '--- prepare-bats-config.sh ---'
sed -n '1,120p' ci/bats/iaas/gcp/prepare-bats-config.sh | cat -nRepository: cloudfoundry/bosh
Length of output: 5730
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Search for BAT_RSPEC_FLAGS usage to understand precedence and intended behavior.
rg -n "BAT_RSPEC_FLAGS|bats\.env|prepare-bats-config" ci -SRepository: cloudfoundry/bosh
Length of output: 1932
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Probe shell semantics relevant to the review comment:
# 1) does sourcing a file overwrite a previously set variable?
# 2) what happens with the suggested concatenation logic?
python3 - <<'PY'
import os, tempfile, subprocess, textwrap, json
def run(script):
p = subprocess.run(["bash", "-lc", script], capture_output=True, text=True)
return p.returncode, p.stdout.strip(), p.stderr.strip()
cases = [
("user flags only", 'BAT_RSPEC_FLAGS="--tag wip"; source <(printf \'export BAT_RSPEC_FLAGS="--default"\'); echo "$BAT_RSPEC_FLAGS"'),
("concat if default exists", 'BAT_RSPEC_FLAGS="--default"; extra_bat_rspec_flags="--tag wip"; export BAT_RSPEC_FLAGS="${BAT_RSPEC_FLAGS:+${BAT_RSPEC_FLAGS} }${extra_bat_rspec_flags}"; echo "$BAT_RSPEC_FLAGS"'),
("concat if default empty", 'unset BAT_RSPEC_FLAGS; extra_bat_rspec_flags="--tag wip"; export BAT_RSPEC_FLAGS="${BAT_RSPEC_FLAGS:+${BAT_RSPEC_FLAGS} }${extra_bat_rspec_flags}"; echo "${BAT_RSPEC_FLAGS-<unset>}"'),
]
for name, script in cases:
rc, out, err = run(script)
print(f"[{name}] rc={rc} out={out!r} err={err!r}")
PYRepository: cloudfoundry/bosh
Length of output: 670
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the exact lines around the implementation to reason about precedence.
nl -ba ci/tasks/run-bats-pipeline.sh | sed -n '150,175p'
echo '---'
nl -ba ci/bats/iaas/gcp/prepare-bats-config.sh | sed -n '1,60p'Repository: cloudfoundry/bosh
Length of output: 195
Preserve BAT_RSPEC_FLAGS before sourcing bats.env.
prepare-bats-config.sh writes a default export BAT_RSPEC_FLAGS=... into bats-config/bats.env, so source bats-config/bats.env overwrites any caller-supplied flags. Save the existing value first, then append it after sourcing.
Suggested fix
+# Preserve user-supplied extra flags before sourcing the generated defaults.
+extra_bat_rspec_flags="${BAT_RSPEC_FLAGS:-}"
+
# shellcheck source=/dev/null
source bats-config/bats.env
# Allow the caller to append extra RSpec flags (e.g. "--tag wip").
-if [[ -n "${BAT_RSPEC_FLAGS:-}" ]]; then
- export BAT_RSPEC_FLAGS="${BAT_RSPEC_FLAGS}"
+if [[ -n "${extra_bat_rspec_flags}" ]]; then
+ export BAT_RSPEC_FLAGS="${BAT_RSPEC_FLAGS:+${BAT_RSPEC_FLAGS} }${extra_bat_rspec_flags}"
fi📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| # Source the environment file that prepare-bats-config.sh wrote; this exports | |
| # BOSH_ENVIRONMENT, BOSH_CLIENT, BOSH_CLIENT_SECRET, BOSH_CA_CERT, | |
| # BOSH_ALL_PROXY, and the default BAT_RSPEC_FLAGS. | |
| # shellcheck source=/dev/null | |
| source bats-config/bats.env | |
| # Allow the caller to append extra RSpec flags (e.g. "--tag wip"). | |
| if [[ -n "${BAT_RSPEC_FLAGS:-}" ]]; then | |
| export BAT_RSPEC_FLAGS="${BAT_RSPEC_FLAGS}" | |
| fi | |
| # Preserve user-supplied extra flags before sourcing the generated defaults. | |
| extra_bat_rspec_flags="${BAT_RSPEC_FLAGS:-}" | |
| # Source the environment file that prepare-bats-config.sh wrote; this exports | |
| # BOSH_ENVIRONMENT, BOSH_CLIENT, BOSH_CLIENT_SECRET, BOSH_CA_CERT, | |
| # BOSH_ALL_PROXY, and the default BAT_RSPEC_FLAGS. | |
| # shellcheck source=/dev/null | |
| source bats-config/bats.env | |
| # Allow the caller to append extra RSpec flags (e.g. "--tag wip"). | |
| if [[ -n "${extra_bat_rspec_flags}" ]]; then | |
| export BAT_RSPEC_FLAGS="${BAT_RSPEC_FLAGS:+${BAT_RSPEC_FLAGS} }${extra_bat_rspec_flags}" | |
| fi |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@ci/tasks/run-bats-pipeline.sh` around lines 157 - 166, Preserve
caller-supplied BAT_RSPEC_FLAGS in run-bats-pipeline.sh by saving its current
value before sourcing bats-config/bats.env, since that file sets a default
export which overwrites it. Update the sourcing flow around the bats.env load so
the existing BAT_RSPEC_FLAGS is restored or appended afterward, keeping the
behavior in the BAT_RSPEC_FLAGS handling block and the bats-config/bats.env
source step consistent.
colins
added a commit
that referenced
this pull request
Jun 26, 2026
Introduces `bundle exec rake fly:bats` (from src/) so developers can run
the full BOSH Acceptance Test suite against their local branch without
waiting for the shared CI pipeline.
How it works
------------
The rake task sets a standalone Concourse pipeline (ci/fly-bats.yml),
then triggers and streams the single `bats` job. The pipeline:
1. Checks out the current branch from GitHub (branch is pushed
automatically before set-pipeline).
2. Runs make-candidate.yml to build a dev release tarball.
3. Compiles the tarball via bosh-deployment/ci/tasks/shared/bosh-agent-compile.yml.
4. Runs ci/tasks/run-bats-pipeline.sh which provisions a GCP environment
with terraform, deploys the BOSH director, runs BATs, and tears down.
All GCP credentials are resolved from the shared Concourse credential
store (((gcp_json_key)) / ((gcp_project_id))).
Files added
-----------
- ci/fly-bats.yml — standalone Concourse pipeline (replaces the
two separate fly-execute task files)
- ci/tasks/run-bats-pipeline.sh — orchestrates the full BATs lifecycle in GCP
Env-var overrides (all optional)
---------------------------------
BATS_ENV_NAME terraform env name (default: bats-local)
STEMCELL_NAME override GCP stemcell
DEPLOY_ARGS extra ops-files for bosh create-env
BAT_RSPEC_FLAGS extra RSpec flags (e.g. "--tag wip")
CONCOURSE_TARGET target alias (default: bosh)
Cherry-picked from fly-bats-pipeline branch (PR #2753).
cf-foundation-community-automation
Bot
moved this to Inbox
in Foundational Infrastructure Working Group
github-project-automation
Bot
moved this from Inbox
to Waiting for Changes | Open for Contribution
in Foundational Infrastructure Working Group
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces
bundle exec rake fly:bats(run fromsrc/) so developers can test their local branch against the full BOSH Acceptance Test suite via the shared Concourse CI — without waiting for the main pipeline.ci/fly-bats.yml— single self-contained Concourse pipeline (replaces the earlier pair offly executetask files)ci/tasks/run-bats-pipeline.sh— orchestrates the complete BATs lifecycle: terraform GCP provisioning → BOSH director deploy → BATs → teardownsrc/tasks/fly.rake— adds thefly:batstask: pushes current branch, sets the pipeline, triggers and streams the jobHow it works
originso Concourse can check it outfly set-pipeline -p bats-local -c ci/fly-bats.yml ...with branch/env varsfly trigger-job -j bats-local/bats --watchto stream the runThe pipeline itself chains three tasks:
bosh-deployment/ci/tasks/shared/bosh-agent-compile.ymlGCP credentials are resolved from the Concourse credential store (
((gcp_json_key))/((gcp_project_id))).Optional env-var overrides
BATS_ENV_NAMEbats-localSTEMCELL_NAMEbosh-google-kvm-ubuntu-nobleDEPLOY_ARGS-o bosh-deployment/external-ip-not-recommended.ymlbosh create-envBAT_RSPEC_FLAGS--tag wipCONCOURSE_TARGETboshflytarget aliasTest plan
bundle exec rake fly:batsfromsrc/on a feature branchbats-localappears on Concourse with correct branch varsBATS_ENV_NAMEoverride prevents terraform conflicts when running concurrently