Skip to content

ci: stop weekly arm64 mixed chaos from timing out - #133

Merged
fwerkor merged 4 commits into
mainfrom
fix/weekly-mixed-chaos-timeout
Aug 1, 2026
Merged

ci: stop weekly arm64 mixed chaos from timing out#133
fwerkor merged 4 commits into
mainfrom
fix/weekly-mixed-chaos-timeout

Conversation

@fwerkor

@fwerkor fwerkor commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Summary

  • run QEMU chaos shards on native architecture runners
  • size mixed-chaos job timeouts for both sequential QEMU phases
  • validate the production arm64 mixed-chaos limits on a native arm64 runner

Root cause

mixed_chaos.sh runs two sequential QEMU phases, each bounded by ARGOSFS_QEMU_TIMEOUT. The weekly arm64 shard allowed 90 minutes per phase but capped the whole job at 115 minutes. Healthy runs therefore reached phase 2 and were reported as cancelled when the outer job timeout expired. Running the arm64 guest under x86_64 TCG also made the mismatch repeatable.

Validation

  • actionlint on both modified workflows
  • YAML parsing with PyYAML
  • git diff --check
  • PR-only native arm64 mixed-chaos replay using a prior weekly arm64 image

Closes #132

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 62e9726ede

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/validate-native-arm64-mixed-chaos.yml Outdated
Comment thread .github/workflows/validate-native-arm64-mixed-chaos.yml Outdated
Comment thread .github/workflows/validate-native-arm64-mixed-chaos.yml Outdated
@fwerkor

fwerkor commented Aug 1, 2026

Copy link
Copy Markdown
Owner Author

The one-off native arm64 replay is running at https://github.com/fwerkor/ArgosFS/actions/runs/30689133228.

The previous native replay reached all phase-1 success markers and entered phase 2 before the old 115-minute outer timeout cancelled it. The final branch removes the one-off workflow so no hard-coded historical artifact run is retained in main; the production weekly workflow keeps the native runner selection and corrected two-phase timeout budgets.

@fwerkor
fwerkor enabled auto-merge (squash) August 1, 2026 07:18
@fwerkor
fwerkor merged commit e87253b into main Aug 1, 2026
28 checks passed
@fwerkor
fwerkor deleted the fix/weekly-mixed-chaos-timeout branch August 1, 2026 07:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Weekly four-matrix chaos CI failure

1 participant