Skip to content

Set allow-failure on every try scenario#318

Merged
NullVoxPopuli merged 1 commit intoember-cli:nvp/its-vite-timefrom
NullVoxPopuli-ai-agent:fix/try-scenarios-continue-on-error
Apr 18, 2026
Merged

Set allow-failure on every try scenario#318
NullVoxPopuli merged 1 commit intoember-cli:nvp/its-vite-timefrom
NullVoxPopuli-ai-agent:fix/try-scenarios-continue-on-error

Conversation

@NullVoxPopuli-ai-agent
Copy link
Copy Markdown

Summary

Fixes the silent failure on the Vite branch where the try-scenarios job never appears in the CI run but the overall run is marked failure.

The workflow has:

try-scenarios:
  continue-on-error: ${{ matrix.allow-failure }}
  ...
  strategy:
    fail-fast: false
    matrix: ${{fromJson(needs.build_matrix.outputs.matrix)}}

@embroider/try's list command passes scenario objects through almost verbatim (only env: {} is defaulted in). Since none of the scenarios in test-app/.try.mjs defined allow-failure, ${{ matrix.allow-failure }} evaluated to '', which GitHub Actions rejects as a non-boolean. The annotation on run 24614721563 confirms:

Error when evaluating 'continue-on-error' for job 'try-scenarios'. .github/workflows/ci.yml (Line: 170, Col: 24): Unexpected value ''

Because the matrix failed to materialize, no try-scenarios job was created, yet the workflow run was marked failure — which is why the 13 visible checks are all green while the PR shows red.

This PR defines allow-failure on every scenario: true for the bleeding-edge ember-beta and ember-alpha scenarios, false everywhere else.

Test plan

  • CI on this PR shows the try-scenarios jobs (min-supported, ember-lts-4.12, ember-lts-5.12, ember-lts-6.4, ember-latest, ember-beta, ember-alpha) actually get created
  • Annotation about continue-on-error evaluating to '' no longer appears
  • Overall workflow conclusion matches the visible check results

🤖 Generated with Claude Code

The CI's try-scenarios job has `continue-on-error: ${{ matrix.allow-failure }}`,
but @embroider/try's list command passes scenario objects through verbatim
(apart from normalizing env). Because none of the scenarios defined an
allow-failure key, the expression evaluated to '' and GitHub Actions rejected
it as a non-boolean, preventing the job from being created at all. The overall
run was marked failed even though every other job passed.

Define allow-failure on each scenario: true for the bleeding-edge ember-beta
and ember-alpha scenarios, false everywhere else.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@NullVoxPopuli NullVoxPopuli merged commit c2d24e8 into ember-cli:nvp/its-vite-time Apr 18, 2026
2 checks passed
@github-actions github-actions Bot mentioned this pull request Apr 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants