-
Notifications
You must be signed in to change notification settings - Fork 0
Add shared Docker build and infra-deploy actions #39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
cafa471
346d7d5
afb9379
2dc7ffd
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| name: Get image tags | ||
| description: >- | ||
| Resolve Docker image tags for a commit — always the default (unprefixed) tag, | ||
| plus a cluster-prefixed tag when a cluster is given | ||
|
|
||
| inputs: | ||
| image: | ||
| description: Full image name including registry/repository | ||
| required: true | ||
| cluster: | ||
| description: >- | ||
| Target cluster: development, staging, production, or aws. Leave empty to | ||
| get only the default (unprefixed) tag. | ||
| required: false | ||
| default: "" | ||
|
|
||
| outputs: | ||
| default_tag: | ||
| description: Unprefixed commit tag, always set (image:<short-sha>) | ||
| value: ${{ steps.tags.outputs.default_tag }} | ||
| cluster_tag: | ||
| description: Cluster-prefixed tag (image:<cluster>-<short-sha>); empty when no cluster given | ||
| value: ${{ steps.tags.outputs.cluster_tag }} | ||
| tags: | ||
| description: >- | ||
| Newline-delimited tag list — default tag always, plus the cluster tag when | ||
| given. Ready to pass to docker/build-push-action `tags`. | ||
| value: ${{ steps.tags.outputs.tags }} | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - name: Resolve image tags | ||
| id: tags | ||
| shell: bash | ||
| env: | ||
| IMAGE: ${{ inputs.image }} | ||
| CLUSTER: ${{ inputs.cluster }} | ||
| SHA: ${{ github.sha }} | ||
| run: | | ||
| SHORT_SHA="${SHA:0:7}" | ||
| DEFAULT_TAG="${IMAGE}:${SHORT_SHA}" | ||
| echo "default_tag=${DEFAULT_TAG}" >> "$GITHUB_OUTPUT" | ||
|
|
||
| if [ -z "$CLUSTER" ]; then | ||
| echo "cluster_tag=" >> "$GITHUB_OUTPUT" | ||
| { | ||
| echo "tags<<EOF" | ||
| echo "${DEFAULT_TAG}" | ||
| echo "EOF" | ||
| } >> "$GITHUB_OUTPUT" | ||
| else | ||
| case "$CLUSTER" in | ||
| development|staging|production|aws) ;; | ||
| *) | ||
| echo "::error::Invalid cluster '$CLUSTER'. Allowed: development, staging, production, aws (or empty)." | ||
| exit 1 | ||
| ;; | ||
| esac | ||
| CLUSTER_TAG="${IMAGE}:${CLUSTER}-${SHORT_SHA}" | ||
| echo "cluster_tag=${CLUSTER_TAG}" >> "$GITHUB_OUTPUT" | ||
| { | ||
| echo "tags<<EOF" | ||
| echo "${DEFAULT_TAG}" | ||
| echo "${CLUSTER_TAG}" | ||
| echo "EOF" | ||
| } >> "$GITHUB_OUTPUT" | ||
| fi | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,122 @@ | ||
| # Why this action exists: | ||
| # - all builds read from a shared "master" cache first because it is the | ||
| # warmest baseline across branches | ||
| # - non-master refs also read from a branch-specific cache so follow-up | ||
| # branch and PR runs can reuse work that exists only there | ||
| # - non-master refs write only to their branch cache so they do not | ||
| # constantly overwrite the shared master cache | ||
| # - master writes directly to the master cache, so there is no duplicate | ||
| # "master + master" cache-from list | ||
| # - docker/metadata-action gives us Docker-safe branch and PR tokens | ||
| # - the metadata config also includes a SHA tag, so there is always a | ||
| # fallback token even when no branch or PR ref tag applies | ||
| # | ||
| # Example outputs for image "harbor.bratislava.sk/app" and suffix "build": | ||
| # - on master: | ||
| # cache_from = "type=registry,ref=harbor.bratislava.sk/app:cache-master-build" | ||
| # cache_to = "type=registry,ref=harbor.bratislava.sk/app:cache-master-build,mode=max" | ||
| # - on branch "feature/foo": | ||
| # cache_from = "type=registry,ref=harbor.bratislava.sk/app:cache-master-build | ||
| # type=registry,ref=harbor.bratislava.sk/app:cache-feature-foo-build" | ||
| # cache_to = "type=registry,ref=harbor.bratislava.sk/app:cache-feature-foo-build,mode=max" | ||
| # - on pull_request for PR 4099: | ||
| # cache_from = "type=registry,ref=harbor.bratislava.sk/app:cache-master-build | ||
| # type=registry,ref=harbor.bratislava.sk/app:cache-pr-4099-build" | ||
| # cache_to = "type=registry,ref=harbor.bratislava.sk/app:cache-pr-4099-build,mode=max" | ||
| # - on an event without a branch/pr ref: | ||
| # cache_from = "type=registry,ref=harbor.bratislava.sk/app:cache-master-build | ||
| # type=registry,ref=harbor.bratislava.sk/app:cache-sha-abcdef1-build" | ||
| # cache_to = "type=registry,ref=harbor.bratislava.sk/app:cache-sha-abcdef1-build,mode=max" | ||
| # | ||
| # Example cache_refs output for suffixes "validate" and "build": | ||
| # { | ||
| # "validate": { | ||
| # "cache_from": "type=registry,ref=harbor.bratislava.sk/app:cache-master-validate\n | ||
| # type=registry,ref=harbor.bratislava.sk/app:cache-feature-foo-validate", | ||
| # "cache_to": "type=registry,ref=harbor.bratislava.sk/app:cache-feature-foo-validate,mode=max" | ||
| # }, | ||
| # "build": { | ||
| # "cache_from": "type=registry,ref=harbor.bratislava.sk/app:cache-master-build\n | ||
| # type=registry,ref=harbor.bratislava.sk/app:cache-feature-foo-build", | ||
| # "cache_to": "type=registry,ref=harbor.bratislava.sk/app:cache-feature-foo-build,mode=max" | ||
| # } | ||
| # } | ||
| name: Resolve Docker cache refs | ||
| description: Resolve stable Docker registry cache refs for master and the current branch/PR | ||
|
|
||
| inputs: | ||
| image: | ||
| description: Full image name including registry/repository | ||
| required: true | ||
| suffixes: | ||
| description: Newline-delimited cache suffixes to resolve in one call | ||
| required: true | ||
|
|
||
| outputs: | ||
| cache_refs: | ||
| description: JSON object keyed by suffix with cache_from and cache_to entries | ||
| value: ${{ steps.resolve.outputs.cache_refs }} | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - name: Resolve Docker-safe ref token | ||
| id: meta | ||
| uses: docker/metadata-action@v6 | ||
| with: | ||
| images: "" | ||
| tags: | | ||
| type=ref,event=branch | ||
| type=ref,event=pr | ||
| type=sha,prefix=sha- | ||
|
|
||
| - name: Build cache refs | ||
| id: resolve | ||
| uses: actions/github-script@v9 | ||
| env: | ||
| INPUT_IMAGE: ${{ inputs.image }} | ||
| INPUT_SUFFIXES: ${{ inputs.suffixes }} | ||
| INPUT_META_JSON: ${{ steps.meta.outputs.json }} | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Doesn't this mean that somewhere here it should also have |
||
| with: | ||
| script: | | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh, I really dislike that this runs a node script. I don't think that is something we want to do. First of all it takes longer to spin up as just plain runner. You introduce another image/dependency that you need to download to a runner. Not to mention the many issues that nodejs ecosystem had in a last year. If possible rewrite this into bash. First of all, it is consistent with the rest of the scripts in github-action repo. It lowers the footprint. Make is cleaner and leaner, as this is unnecessary convoluted. We have a javascript function within a very simple step. This can all be avoided with just something like token=$(jq -er '.["tag-names"][0]' <<< "$INPUT_META_JSON")
if [[ -z "$token" ]]; then
echo "::error::No metadata-action tag-names value found"
exit 1
fi
# Strip windows \r and trim
suffixes=$(tr -d '\r' <<<"$INPUT_SUFFIXES" | awk '{$1=$1;print}')
if [[ -z "${suffixes}" ]]; then
echo "::error::At least one suffix is required"
exit 1
fi
cache_refs='{}'
for suffix in $suffixes; do
master="type=registry,ref=${INPUT_IMAGE}:cache-master-${suffix}"
branch="type=registry,ref=${INPUT_IMAGE}:cache-${token}-${suffix}"
if [[ "$token" == master ]]; then
from="$master"
to="${master},mode=max"
else
from="${master}"$'\n'"${branch}"
to="${branch},mode=max"
fi
cache_refs=$(jq -c --arg suf "$suffix" --arg from "$from" --arg to "$to" \
'.[$suf] = {cache_from: $from, cache_to: $to}' <<< "$cache_refs")
done
echo "cache_refs=$cache_refs" >> "$GITHUB_OUTPUT"you can probably make it even more leaner, I just wrote this from top of my head. Moreover, consider if this is even necessary. I don't really know how you plan on using it (at the time of this review), but something like directly inline in |
||
| const image = process.env.INPUT_IMAGE | ||
| const rawSuffixes = process.env.INPUT_SUFFIXES | ||
| const rawMetaJson = process.env.INPUT_META_JSON | ||
|
|
||
| const suffixes = rawSuffixes | ||
| .split(/\r?\n/) | ||
| .map((suffix) => suffix.trim()) | ||
| .filter(Boolean) | ||
|
|
||
| if (suffixes.length === 0) { | ||
| throw new Error('At least one suffix is required') | ||
| } | ||
|
|
||
| const meta = JSON.parse(rawMetaJson) | ||
| const token = meta['tag-names']?.[0] | ||
| if (!token) { | ||
| throw new Error('No metadata-action tag-names value found') | ||
| } | ||
|
|
||
| function buildCacheRefs(suffix) { | ||
| const masterRef = `${image}:cache-master-${suffix}` | ||
| const branchRef = `${image}:cache-${token}-${suffix}` | ||
|
|
||
| if (token === 'master') { | ||
| return { | ||
| cache_from: `type=registry,ref=${masterRef}`, | ||
| cache_to: `type=registry,ref=${masterRef},mode=max`, | ||
| } | ||
| } | ||
|
|
||
| return { | ||
| cache_from: `type=registry,ref=${masterRef}\ntype=registry,ref=${branchRef}`, | ||
| cache_to: `type=registry,ref=${branchRef},mode=max`, | ||
| } | ||
| } | ||
|
|
||
| const cacheRefs = Object.fromEntries( | ||
| suffixes.map((suffix) => [suffix, buildCacheRefs(suffix)]), | ||
| ) | ||
|
|
||
| core.setOutput('cache_refs', JSON.stringify(cacheRefs)) | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| name: Setup Docker build | ||
| description: Resolve Docker cache refs, set up Docker Buildx, and log in to registries | ||
|
|
||
| inputs: | ||
| image: | ||
| description: Full image name including registry/repository | ||
| required: true | ||
| suffixes: | ||
| description: Newline-delimited cache suffixes to resolve in one call | ||
| required: true | ||
| docker_registry_username: | ||
| description: Docker Hub username | ||
| required: true | ||
| docker_registry_password: | ||
| description: Docker Hub password | ||
| required: true | ||
| harbor_registry: | ||
| description: Harbor registry hostname | ||
| required: true | ||
| harbor_registry_username: | ||
| description: Harbor registry username | ||
| required: true | ||
| harbor_registry_password: | ||
| description: Harbor registry password | ||
| required: true | ||
| buildx_driver_opts: | ||
| description: Optional newline-delimited Docker Buildx driver options | ||
| required: false | ||
| default: "" | ||
|
|
||
| outputs: | ||
| cache_refs: | ||
| description: JSON object keyed by suffix with cache_from and cache_to entries | ||
| value: ${{ steps.cache_ref.outputs.cache_refs }} | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| # Cross-repo composite actions cannot use a local "./" path here: GitHub | ||
| # resolves "./" relative to the caller's repository, not this one. Pin to | ||
| # the shared repo by ref instead. Repoint this ref to the released tag | ||
| # (e.g. @beta / @stable) once this branch is merged. | ||
| - name: Resolve Docker cache refs | ||
| id: cache_ref | ||
| uses: bratislava/github-actions/.github/actions/resolve-cache-refs@add-setup-docker-build-action | ||
| with: | ||
| image: ${{ inputs.image }} | ||
| suffixes: ${{ inputs.suffixes }} | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v4 | ||
| with: | ||
| driver-opts: ${{ inputs.buildx_driver_opts }} | ||
|
|
||
| - name: Log in to Docker Hub | ||
| uses: docker/login-action@v4 | ||
| with: | ||
| username: ${{ inputs.docker_registry_username }} | ||
| password: ${{ inputs.docker_registry_password }} | ||
|
|
||
| - name: Log in to Harbor | ||
| uses: docker/login-action@v4 | ||
| with: | ||
| registry: ${{ inputs.harbor_registry }} | ||
| username: ${{ inputs.harbor_registry_username }} | ||
| password: ${{ inputs.harbor_registry_password }} |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| # Reusable workflow: dispatch a deploy in the infrastructure-deployment-configuration | ||
| # repo and wait for it to finish. | ||
| # | ||
| # Required secret on the CALLING repository: | ||
| # INFRA_DEPLOY_PAT - a PAT with rights to dispatch and read workflow runs in | ||
| # bratislava/infrastructure-deployment-configuration. | ||
| # This workflow reads the secret directly via `secrets.INFRA_DEPLOY_PAT`, so the | ||
| # caller must make it available, e.g. by calling with `secrets: inherit` (or by | ||
| # explicitly forwarding the secret). The secret must exist on the caller repo. | ||
| name: Trigger Infrastructure Deploy (Reusable) | ||
|
|
||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| cluster: | ||
| description: "Target cluster (development, staging, production, aws)" | ||
| required: true | ||
| type: string | ||
| app: | ||
| description: >- | ||
| Application identifier passed to the deploy workflow as app. | ||
| Free-form, e.g. "parking_pricing_api" or | ||
| "konto.bratislava.sk/nest_city_account". | ||
| required: true | ||
| type: string | ||
| dispatch_ref: | ||
| description: "Git ref in the target repo to dispatch the deploy workflow on" | ||
| required: false | ||
| type: string | ||
| default: master | ||
|
|
||
| env: | ||
| TARGET_REPO: bratislava/infrastructure-deployment-configuration | ||
| DEPLOY_WORKFLOW_FILE: deploy.yml | ||
|
|
||
| jobs: | ||
| trigger-deploy: | ||
| name: Deploy to ${{ inputs.cluster }} | ||
| runs-on: bratislava | ||
| environment: ${{ inputs.cluster }} | ||
| steps: | ||
| - name: Trigger infrastructure deployment | ||
| env: | ||
| GH_TOKEN: ${{ secrets.INFRA_DEPLOY_PAT }} | ||
| run: | | ||
| echo "Dispatching deploy for cluster=${{ inputs.cluster }} app=${{ inputs.app }}" | ||
|
|
||
| RESPONSE=$(curl --fail --request POST \ | ||
| --header "Accept: application/vnd.github.v3+json" \ | ||
| --header "Authorization: Bearer $GH_TOKEN" \ | ||
| --data @- \ | ||
| "https://api.github.com/repos/${{ env.TARGET_REPO }}/actions/workflows/${{ env.DEPLOY_WORKFLOW_FILE }}/dispatches" \ | ||
| <<EOF | ||
| { | ||
| "ref": "${{ inputs.dispatch_ref }}", | ||
| "inputs": { | ||
| "cluster": "${{ inputs.cluster }}", | ||
| "app": "${{ inputs.app }}" | ||
| }, | ||
| "return_run_details": true | ||
| } | ||
| EOF | ||
| ) | ||
|
|
||
| RUN_ID=$(echo "$RESPONSE" | jq -r '.workflow_run_id // empty') | ||
| if [ -z "$RUN_ID" ]; then | ||
| echo "::error::Failed to get workflow run ID. Full response: $RESPONSE" | ||
| exit 1 | ||
| fi | ||
| RUN_URL=$(echo "$RESPONSE" | jq -r '.html_url // empty') | ||
| echo "run_id=$RUN_ID" >> $GITHUB_ENV | ||
| echo "run_url=$RUN_URL" >> $GITHUB_ENV | ||
| echo "Triggered workflow run: $RUN_ID ($RUN_URL)" | ||
|
|
||
| - name: Wait for triggered workflow to complete | ||
| env: | ||
| GH_TOKEN: ${{ secrets.INFRA_DEPLOY_PAT }} | ||
| run: | | ||
| API_BASE="https://api.github.com/repos/${{ env.TARGET_REPO }}/actions" | ||
| AUTH_HEADER="Authorization: Bearer $GH_TOKEN" | ||
| RUN_ID="${{ env.run_id }}" | ||
| RUN_URL="${{ env.run_url }}" | ||
|
|
||
| echo "Waiting for run to complete: $RUN_URL" | ||
| for i in $(seq 1 60); do | ||
| sleep 15 | ||
| RESPONSE=$(curl --silent \ | ||
| --header "Accept: application/vnd.github.v3+json" \ | ||
| --header "$AUTH_HEADER" \ | ||
| "$API_BASE/runs/$RUN_ID") | ||
| STATUS=$(echo "$RESPONSE" | jq -r '.status') | ||
| CONCLUSION=$(echo "$RESPONSE" | jq -r '.conclusion // empty') | ||
|
|
||
| if [ "$STATUS" = "completed" ]; then | ||
| echo "Workflow completed with conclusion: $CONCLUSION" | ||
|
|
||
| echo "### Infrastructure Deploy" >> $GITHUB_STEP_SUMMARY | ||
| echo "- Cluster: \`${{ inputs.cluster }}\`" >> $GITHUB_STEP_SUMMARY | ||
| echo "- App: \`${{ inputs.app }}\`" >> $GITHUB_STEP_SUMMARY | ||
| echo "- Result: \`$CONCLUSION\`" >> $GITHUB_STEP_SUMMARY | ||
| echo "- [Workflow run]($RUN_URL)" >> $GITHUB_STEP_SUMMARY | ||
|
|
||
| if [ "$CONCLUSION" != "success" ]; then | ||
| echo "::error::Triggered workflow finished with conclusion: $CONCLUSION" | ||
| exit 1 | ||
| fi | ||
| exit 0 | ||
| fi | ||
|
|
||
| echo "Attempt $i: status=$STATUS" | ||
| done | ||
|
|
||
| echo "::error::Timed out waiting for workflow run to complete" | ||
| exit 1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is so unnecessary convoluted. I would suggest something more lean like
Honest confession. I just wrote this, didn't test it, so leaving it up to the reader (? 🙈)