Skip to content

fix: detect and break review retry loops when agent finds no changes #164

Description

@qinqon

Problem

When the agent processes review feedback and determines "no code changes needed" (e.g., stale reviews already addressed in prior commits), the cursor (`LastReviewID`) is never advanced because oompa only advances cursors after a successful push. This causes an infinite loop:

  1. Oompa finds N reviews on a PR
  2. Invokes the agent (~$0.50-1.00 per invocation)
  3. Agent says "all reviews are stale, no changes needed"
  4. No push → cursor not advanced
  5. Next poll cycle: same N reviews found → repeat from step 2

Cost impact

Observed on PR openshift/hypershift#8365: the loop ran every 2 minutes for 5 days (May 6-11), at ~$0.60-1.00 per invocation. Estimated total cost: $2,000-4,000 burned on stale review processing.

Root cause

In `pkg/agent/review.go` (or `loop.go`), the cursor advancement logic is:

```go
// Cursors only advance after successful push or when no changes detected
// BUT: "no changes detected" path still requires allRepliesPosted == true
// AND: the skill posts replies via GraphQL, but oompa doesn't track that
```

When the agent runs and concludes "no changes needed":

  • `changeDetected = false`
  • `pushed = false`
  • The cursor should advance (the reviews have been evaluated), but the current logic gates advancement on reply posting which may not happen

Fix

1. Advance cursors when agent completes with no changes

If the agent runs successfully and determines no code changes are needed, advance the review cursors anyway. The reviews have been evaluated — re-evaluating them on the next cycle is pure waste.

2. Add retry loop detection

Track consecutive no-op review cycles per PR. If the same reviews produce "no changes" N times in a row (e.g., 3), stop processing them and log a warning:

```go
type IssueWork struct {
// ... existing fields ...
ReviewNoOpCount int // consecutive cycles where reviews produced no changes
}
```

When `ReviewNoOpCount >= 3`:

  • Skip review processing for this PR
  • Log: "skipping stale reviews on PR #N after 3 no-op cycles"
  • Reset the counter when a new review arrives (review ID > LastReviewID)

3. Cost guard

Add a per-PR cost tracking field. If a single PR has consumed more than a configurable threshold (e.g., $10) in a single session, pause agent invocations and log a warning. This is a safety net against any type of retry loop, not just reviews.

Evidence

Logs from May 6-11 showing the loop on hypershift PR #8365:

```
May 11 05:51:17 addressing review feedback pr=8365 comments=0 reviews=3
May 11 05:51:29 "All CodeRabbit review comments are stale..." cost_usd=0.989
May 11 05:51:30 ERROR failed to amend commit (git corruption)
May 11 05:53:17 addressing review feedback pr=8365 comments=0 reviews=3
May 11 05:53:29 "All CodeRabbit review comments are stale..." cost_usd=0.994
... (repeating every 2 minutes for 5 days)
```

Workaround applied

Temporarily disabled `reviews` reaction for hypershift in the config to stop the bleeding. The worktree was also corrupted (invalid git objects) which caused an additional `git commit --amend` error on every cycle, compounding the issue.

Tests

  • Agent finds no changes on review → cursor advances, next cycle skips those reviews
  • Same reviews produce no changes 3 times → review processing paused with warning
  • New review arrives after pause → counter resets, processing resumes
  • Cost threshold exceeded → agent invocation paused with warning

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    good-for-aiIssue labeled with it will be processed by the project itself, generating a PR and iterating on it.

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions