Skip to content

Worktree teardown race crashes the run after a successful merge #139

Description

@tjoignant

Version: bmad-loop 0.8.1 (uv tool) · macOS 15 (Darwin 24.6.0) · git worktree isolation, adapter.name = "claude"

Summary

When a story's session leaves a process still writing into its worktree (for us: pytest), close_unit_workspace fails teardown and crashes the whole run — after unit-merged has already landed the work on the target branch. No code is lost, but the run dies and leaves an orphaned worktree directory with a dangling .git gitfile.

Hit 2 out of 2 backend stories in a single sweep run, with an identical fingerprint.

What I expected

A merged story tears down its worktree. If the teardown can't complete, that's housekeeping — it should be logged and reconciled, not turn a successful merge into a run crash.

Observed

crash.txt:

File ".../bmad_loop/workspace.py", line 157, in close_unit_workspace
    verify.worktree_remove(unit.repo_root, unit.path, force=not success)
GitError: git worktree remove .../worktrees/dw-db-schema-and-gate-falsifiability failed:
error: failed to delete '.../worktrees/dw-db-schema-and-gate-falsifiability': Directory not empty

During handling of the above exception, another exception occurred:
  File ".../bmad_loop/workspace.py", line 159, in close_unit_workspace
GitError: git worktree remove ... failed:
fatal: '.../worktrees/dw-db-schema-and-gate-falsifiability' is not a working tree

Timeline (journal ts + file mtimes on the leftover worktree):

16:13:02  session-end            review-2
16:13:04  unit-merged            <- work is safely on the target branch
16:13:04  worktree root mtime    <- git worktree remove starts
16:13:07  backend/.pytest_cache/{CACHEDIR.TAG,README.md,.gitignore} written
16:13:09  rmdir(worktree) -> ENOTEMPTY -> run-crash

Those are file content writes (own mtimes), not git deletions — a pytest process from the just-ended session was still alive and recreated the cache dir after git had scanned past it. The second story is the same to the second (cache written 11:42:20, crash 11:42:21). In both leftover worktrees the only surviving entry under backend/ is .pytest_cache.

Analysis (my reading, less certain than the above)

Two separate things combine:

1. Teardown doesn't wait for the session's process tree. _integrate_unit runs ~2s after session-end. A background/subagent-spawned test run outlives the model's turn and keeps writing into the tree git is deleting.

2. The --force retry cannot work, and that's what's fatal. Observed on disk: when git's work-tree deletion fails, it still removes .git/worktrees/<id> — the leftover directories have a .git gitfile pointing at an admin dir that no longer exists. So the retry in workspace.py:157:

try:
    verify.worktree_remove(unit.repo_root, unit.path, force=not success)
except verify.GitError:
    verify.worktree_remove(unit.repo_root, unit.path, force=True)   # -> "is not a working tree"

--force addresses a dirty tree, not a missing admin entry, so the second call raises uncaught and kills the run.

runs.py:507 already has the correct shape for exactly this case:

try:
    verify.worktree_remove(repo, wt, force=True)
except verify.GitError:
    shutil.rmtree(wt, ignore_errors=True)
...
verify.worktree_prune(repo)

Bug 2 looks like the cheap, high-value fix: giving the story teardown the same rmtree + prune fallback turns this class of race into a warning. Bug 1 is the deeper one.

Reproduce

With scm.isolation = "worktree", have a story's session leave a process writing into the worktree at teardown (any long-running test runner that outlives the turn). Teardown fails ENOTEMPTY, the retry reports "is not a working tree", run crashes post-merge.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions