Bound OpenCode review runtime and check lookup#276
Open
seonghobae wants to merge 5 commits into
Open
Conversation
There was a problem hiding this comment.
Pull request overview
This PR tightens the OpenCode review model-pool runtime behavior to reduce stalls in required PR review gates, prioritizes faster OpenAI model candidates ahead of slower fallbacks, and adjusts how paginated check-runs are aggregated for broader gh CLI compatibility.
Changes:
- Reordered
OPENCODE_MODEL_CANDIDATESto prefer OpenAI models before DeepSeek/Mistral/Meta fallbacks. - Reduced OpenCode runtime budgets (per-model timeout, total retry budget, and backoff max).
- Replaced
gh api --paginate --slurp --jq ...withgh api --paginate+jq -s ...for check-run aggregation.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
tests/test_opencode_agent_contract.py |
Updates contract assertions to match the new OpenCode timeouts and the new jq -s check-run aggregation approach. |
scripts/ci/test_strix_quick_gate.sh |
Updates Strix gate assertions/messages for the new timeout/retry budget values and the removal of gh --slurp --jq. |
.github/workflows/opencode-review.yml |
Reorders model candidates, reduces timeouts/backoff budgets, and changes check-run pagination aggregation to use jq -s over a paginated gh api output file. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
seonghobae
force-pushed
the
codex/bound-opencode-review-runtime
branch
from
2e755f4 to
c5eb83a
Compare
Contributor
OpenCode Review Overview
Pull request overviewOpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes. Findings1. HIGH review evidence:1 - OpenCode could not establish approval sufficiency
Summary
No PR approval was posted because model-output failure is not evidence that the PR has no blockers. Inline comment note: OpenCode could not find an added RIGHT-side diff line for this PR, so the model-exhaustion blocker is attached to the PR review body instead of a file line. Changed-File Evidence Mapflowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: test_strix_quick_gate.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_agent_contract.py"]
R3 --> V3["targeted test run"]
|
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes.
Findings
1. HIGH .github/workflows/opencode-review.yml:1 - OpenCode could not establish approval sufficiency
- Problem: every configured model path failed to produce a usable current-head control block.
- Root cause: model execution, timeout, export, normalization, or approval-gate validation did not complete after exponential retry across the configured model pool.
- Impact: approving from deterministic check state alone would miss PR-intent mismatches, missing files, edge-case bugs, robustness gaps, UX/DX regressions, security issues, and CodeGraph-backed base/head flow changes.
- Fix: rerun OpenCode after model availability recovers, or update the PR with the missing files, tests, docs, generated artifacts, and verification evidence needed for a source-backed review conclusion.
- Regression test: keep the approval gate posting REQUEST_CHANGES, not APPROVE or check-only failure, when no model produces a valid current-head review.
Summary
- Result: REQUEST_CHANGES
- Reason: coverage-evidence passed and peer GitHub Checks completed without failures, but no model produced a valid review control block.
- Deterministic evidence checked but not used for approval: current-head changed-file evidence (.github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
- Model outcome: model_pool=exhausted; selected_model=none.
- Head SHA:
c5eb83abc14ef5aa7a7d8099463d8821b5e93433 - Workflow run: 28505697105
- Workflow attempt: 1
No PR approval was posted because model-output failure is not evidence that the PR has no blockers.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: test_strix_quick_gate.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_agent_contract.py"]
R3 --> V3["targeted test run"]
…review-runtime # Conflicts: # .github/workflows/opencode-review.yml # tests/test_opencode_agent_contract.py
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes.
Findings
1. HIGH .github/workflows/opencode-review.yml:1 - OpenCode could not establish approval sufficiency
- Problem: every configured model path failed to produce a usable current-head control block.
- Root cause: model execution, timeout, export, normalization, or approval-gate validation did not complete after exponential retry across the configured model pool.
- Impact: approving from deterministic check state alone would miss PR-intent mismatches, missing files, edge-case bugs, robustness gaps, UX/DX regressions, security issues, and CodeGraph-backed base/head flow changes.
- Fix: rerun OpenCode after model availability recovers, or update the PR with the missing files, tests, docs, generated artifacts, and verification evidence needed for a source-backed review conclusion.
- Regression test: keep the approval gate posting REQUEST_CHANGES, not APPROVE or check-only failure, when no model produces a valid current-head review.
Summary
- Result: REQUEST_CHANGES
- Reason: coverage-evidence passed and peer GitHub Checks completed without failures, but no model produced a valid review control block.
- Deterministic evidence checked but not used for approval: current-head changed-file evidence (.github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
- Model outcome: model_pool=exhausted; selected_model=none.
- Head SHA:
228de0bf4692852e5e6cccb5e3818d5cc723a3a0 - Workflow run: 28508371944
- Workflow attempt: 1
No PR approval was posted because model-output failure is not evidence that the PR has no blockers.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: test_strix_quick_gate.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_agent_contract.py"]
R3 --> V3["targeted test run"]
| NPM_CONFIG_IGNORE_SCRIPTS: "true" | ||
| NO_COLOR: "1" | ||
| OPENCODE_MODEL_CANDIDATES: "github-models/openai/gpt-5-nano" | ||
| OPENCODE_MODEL_CANDIDATES: "github-models/openai/gpt-5-nano github-models/openai/gpt-5-mini github-models/openai/gpt-5-chat" |
Contributor
There was a problem hiding this comment.
HIGH OpenCode could not establish approval sufficiency
- Problem: the model pool exhausted without a valid current-head review control block, so this changed line cannot be approved from deterministic check state alone.
- Impact: PR-intent mismatches, missing files, robustness bugs, UX/DX regressions, and CodeGraph-backed flow changes could be missed.
- Fix: rerun OpenCode after model availability recovers, or add the missing source/test/docs/generated verification evidence needed for a source-backed approval.
- Verification: rerun the OpenCode Review workflow and confirm it emits APPROVE or source-backed REQUEST_CHANGES for this head SHA.
Contributor
Author
@copilot 결정론적 리뷰가 싫으니 exponential retry를 하든지 쫌 |
Contributor
Addressed in |
Contributor
There was a problem hiding this comment.
Pull request overview
OpenCode exhausted the configured model pool without a usable current-head review conclusion. This is not approval evidence, so the PR is blocked until a source-backed review can establish approval sufficiency or identify concrete fixes.
Findings
1. HIGH review evidence:1 - OpenCode could not establish approval sufficiency
- Problem: every configured model path failed to produce a usable current-head control block.
- Root cause: model execution, timeout, export, normalization, or approval-gate validation did not complete after exponential retry across the configured model pool.
- Impact: approving from deterministic check state alone would miss PR-intent mismatches, missing files, edge-case bugs, robustness gaps, UX/DX regressions, security issues, and CodeGraph-backed base/head flow changes.
- Fix: rerun OpenCode after model availability recovers, or update the PR with the missing files, tests, docs, generated artifacts, and verification evidence needed for a source-backed review conclusion.
- Regression test: keep the approval gate posting REQUEST_CHANGES, not APPROVE or check-only failure, when no model produces a valid current-head review.
Summary
- Result: REQUEST_CHANGES
- Reason: coverage-evidence passed and peer GitHub Checks completed without failures, but no model produced a valid review control block.
- Deterministic evidence checked but not used for approval: current-head changed-file evidence (.github/workflows/opencode-review.yml, scripts/ci/test_strix_quick_gate.sh, tests/test_opencode_agent_contract.py); coverage-evidence result success; peer checks from statusCheckRollup excluding this OpenCode check.
- Model outcome: model_pool=exhausted; selected_model=none.
- Head SHA:
db5a8da1879e83b6fe2098fb1782986a1654c12a - Workflow run: 28514249378
- Workflow attempt: 2
No PR approval was posted because model-output failure is not evidence that the PR has no blockers.
Inline comment note: OpenCode could not find an added RIGHT-side diff line for this PR, so the model-exhaustion blocker is attached to the PR review body instead of a file line.
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Workflow: opencode-review.yml"]
S1 --> I1["GitHub Actions review job"]
I1 --> R1["Review risk: Workflow: opencode-review.yml"]
R1 --> V1["actionlint plus required checks"]
Evidence --> S2["CI script: test_strix_quick_gate.sh"]
S2 --> I2["review and security gate shell path"]
I2 --> R2["Review risk: CI script: test_strix_quick_gate.sh"]
R2 --> V2["bash -n plus Strix self-test"]
Evidence --> S3["Test: test_opencode_agent_contract.py"]
S3 --> I3["regression suite"]
I3 --> R3["Review risk: Test: test_opencode_agent_contract.py"]
R3 --> V3["targeted test run"]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Verification
Note: bash -n scripts/ci/run_opencode_review_model_pool.sh timed out in this Windows shell before producing output.