diff --git a/AGENTS.md b/AGENTS.md index 0b6959e1e7..a5b203a314 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -478,7 +478,12 @@ A ship task's path from `done` to landed on `main` is set by the project's `mode - **no-mistakes** - the stages below as written: no-mistakes validation pipeline -> PR -> captain merge. - **direct-PR** - no pipeline. The crewmate pushes and opens the PR itself (its brief says so) and reports `done: PR `. Skip the Validate step and go straight to PR ready (run `fm-pr-check`, relay the PR). Teardown uses the normal landed-work check. -- **local-only** - no remote, no PR. The crewmate stops at `done: ready in branch fm/`. Review the diff with `bin/fm-review-diff.sh `, relay a one-paragraph summary to the captain, and on approval run `bin/fm-merge-local.sh ` to fast-forward local `main` (it refuses anything but a clean fast-forward - if it does, have the crewmate rebase). No `fm-pr-check`. Then teardown, whose safety check requires the branch already merged into local `main`, OR the work pushed to any remote (a fork counts - relevant for upstream-contribution PRs on a local-only-registered project). +- **local-only** - no remote, no PR. + The crewmate stops at `done: ready in branch fm/`. + Review the diff with `bin/fm-review-diff.sh `, relay a one-paragraph summary to the captain, and on approval run `bin/fm-merge-local.sh ` to fast-forward local `main`; it refuses anything but a clean fast-forward, and on success immediately invokes fail-closed auto-reaping. + If either the merge or automatic teardown refuses, inspect the retained state rather than weakening the safety check. + No `fm-pr-check`. + Teardown requires the branch already merged into local `main`, OR the work pushed to any remote; a fork counts, which is relevant for upstream-contribution PRs on a local-only-registered project. When reviewing any crewmate branch diff, use `bin/fm-review-diff.sh ` rather than `git diff ...branch` directly. Pooled clones keep their local default refs frozen at clone time and can lag `origin`; the helper always compares against the authoritative base. @@ -540,6 +545,11 @@ The helper defaults to `--squash`, accepts explicit merge-method flags such as ` bin/fm-teardown.sh ``` +The watcher normally invokes `bin/fm-auto-reap.sh` as soon as a terminal `done` task's recorded PR is provably merged. +The approved local-only merge helper and completed-scout signal take the same automatic path. +Auto-reaping cancels only an exactly attributed no-mistakes run, never guesses cross-branch process ownership, and then calls this ordinary teardown without `--force`. +Persistent secondmates are excluded, and X-mode-linked tasks wait for their required final follow-up. +An automatic refusal is an actionable wake and retains its metadata, worktree, and acquisition authority; after resolving the reported cause, retry with the ordinary command above. The script refuses if the worktree holds uncommitted changes or committed work that has not landed; treat a refusal as a stop-and-investigate, not an obstacle. Teardown validates that the recorded project and worktree are exact roots with the expected repository registration, quiesces every ordinary task endpoint, and then runs the final non-destructive safety checks before any Treehouse return. For a task whose metadata carries `report_required=1`, teardown also publishes the validated completion report before releasing the account lease or removing the worktree. @@ -566,7 +576,8 @@ A scout task follows Intake, Spawn, and Supervise exactly as above - scaffold th - There is no Validate or PR-ready stage. When the crewmate's status says `done`, read `data//report.md`. - Relay the findings to the captain: plain chat for a focused answer, lavish-axi when the report has structure worth a visual (multiple findings, options, a plan). -- Tear down immediately - no merge gate. For a post-cutover scout, `bin/fm-teardown.sh` requires the report's completion sections and publishes it before removing the declared scratch worktree; a missing or incomplete report refuses teardown because the findings are the work product. +- The watcher automatically tears down on the terminal `done` signal - no merge gate. + For a post-cutover scout, ordinary teardown still requires the report's completion sections and publishes it before removing the declared scratch worktree; a missing or incomplete report refuses auto-reaping because the findings are the work product. - Record it in Done with the report path instead of a PR link using `tasks-axi done` when the default tasks-axi backend is active and compatible, otherwise hand-edit `data/backlog.md` and keep Done to the 10 most recent, then re-evaluate the queue and dispatch only queued work whose blockers are gone and whose time/date gate, if any, has arrived. When the captain invokes `/reports` or asks to browse, open, search, or summarize completed work, load the `reports` skill. diff --git a/bin/backends/herdr.sh b/bin/backends/herdr.sh index 238669a7e8..924ccb9c6a 100644 --- a/bin/backends/herdr.sh +++ b/bin/backends/herdr.sh @@ -2653,22 +2653,18 @@ fm_backend_herdr_expected_label_matches() { # [expected-label] # launch-grade fm_backend_herdr_server_ensure used to spawn new crewmates. # # Reading a pane or sending keys to it does not care how the server was -# launched - the pane already exists and the server is up. It only needs the -# server to be running and adapter-owned (this HOME's own certificate names the -# live pid), which fm_backend_herdr_server_adapter_owned proves without the -# closed-shell launch certification. This is what keeps peek/steer working when -# FirstMate itself runs INSIDE the herdr session it manages (HERDR_ENV=1): that -# server was not launched through the crewmate adapter's own closed-shell path, so -# closed_shell_environment_ready is false and the full ensure would try to -# restart+recertify - impossible while the session is occupied by live crewmates and -# FirstMate itself. The SPAWN path deliberately keeps the strict ensure so a new -# crewmate still launches only in a certified closed-shell server. +# launched - the pane already exists and the server is up. Native agent routing +# removed production's server-certificate requirement, so exact pane identity is +# the authority boundary there. The explicit legacy certificate-lifecycle test +# mode retains its adapter-owned-server proof. fm_backend_herdr_server_reachable_for_readsteer() { # local session=$1 running running=$(fm_backend_herdr_cli "$session" status --json 2>/dev/null \ | fm_backend_herdr_control_jq -r '.server.running // false' 2>/dev/null) [ "$running" = true ] || return 1 - fm_backend_herdr_server_adapter_owned "$session" + if fm_backend_herdr_server_certificate_required; then + fm_backend_herdr_server_adapter_owned "$session" + fi } fm_backend_herdr_target_ready() { # [expected-label] diff --git a/bin/backends/orca.sh b/bin/backends/orca.sh index 7f6bc1b1ee..609f722411 100644 --- a/bin/backends/orca.sh +++ b/bin/backends/orca.sh @@ -153,7 +153,7 @@ fm_backend_orca_worktree_create() { # wt_path=$(printf '%s' "$out" | fm_backend_orca_json_get worktree-path 2>/dev/null || true) proof=unproven [ -z "$terminal" ] || proof=recorded - printf '%s\t%s\t%s\t%s\t%s' "$wt_id" "$wt_path" "$terminal" "$proof" "$repo_id" + printf '%s\t%s\t%s\t%s\t%s\t%s' "$wt_id" "$wt_path" "$terminal" "$proof" "$repo_id" "$name" if [ "$status" -ne 0 ] || [ -z "$wt_id" ] || [ -z "$wt_path" ]; then echo "error: orca worktree create returned incomplete or unsuccessful authority for $name" >&2 return 2 diff --git a/bin/fm-account-routing-lib.sh b/bin/fm-account-routing-lib.sh index fa65dccead..b3907fa157 100644 --- a/bin/fm-account-routing-lib.sh +++ b/bin/fm-account-routing-lib.sh @@ -604,6 +604,84 @@ fm_account_task_key() { # printf 'fm-%.16s-%s-%s\n' "$home_hash" "$task" "$attempt" } +fm_tasktmp_path() { # + local task=$1 generation=$2 token + fm_account_valid_id "$task" || return 1 + token=${generation##*:} + fm_account_valid_id "$token" || return 1 + printf '/tmp/fm-%s-%s\n' "$task" "$token" +} + +fm_tasktmp_owner_record() { # + local state=$1 task=$2 generation=$3 token + fm_account_valid_id "$task" || return 1 + token=${generation##*:} + fm_account_valid_id "$token" || return 1 + printf '%s/%s.tasktmp-owner.%s\n' "$state" "$task" "$token" +} + +fm_tasktmp_owner_write() { # + local state=$1 home=$2 task=$3 generation=$4 root=$5 home_real expected record tmp + home_real=$(cd "$home" 2>/dev/null && pwd -P) || return 1 + expected=$(fm_tasktmp_path "$task" "$generation") || return 1 + [ "$root" = "$expected" ] || return 1 + record=$(fm_tasktmp_owner_record "$state" "$task" "$generation") || return 1 + [ ! -e "$record" ] && [ ! -L "$record" ] || return 1 + tmp=$(mktemp "$state/.$task.tasktmp-owner.XXXXXX") || return 1 + { + printf 'home=%s\n' "$home_real" + printf 'task=%s\n' "$task" + printf 'generation=%s\n' "$generation" + printf 'root=%s\n' "$root" + } > "$tmp" || { rm -f "$tmp"; return 1; } + chmod 600 "$tmp" || { rm -f "$tmp"; return 1; } + if ! ln "$tmp" "$record" 2>/dev/null; then + rm -f "$tmp" + return 1 + fi + rm -f "$tmp" +} + +fm_tasktmp_owner_validate() { # + local record=$1 home=$2 task=$3 generation=$4 root=$5 home_real expected + [ -f "$record" ] && [ ! -L "$record" ] || return 1 + [ "$(wc -l < "$record" | tr -d ' ')" = 4 ] || return 1 + home_real=$(cd "$home" 2>/dev/null && pwd -P) || return 1 + expected=$(fm_tasktmp_path "$task" "$generation") || return 1 + [ "$root" = "$expected" ] || return 1 + [ "$(sed -n '1s/^home=//p' "$record")" = "$home_real" ] || return 1 + [ "$(sed -n '2s/^task=//p' "$record")" = "$task" ] || return 1 + [ "$(sed -n '3s/^generation=//p' "$record")" = "$generation" ] || return 1 + [ "$(sed -n '4s/^root=//p' "$record")" = "$root" ] || return 1 +} + +fm_tasktmp_create() { # + local state=$1 home=$2 task=$3 generation=$4 root=$5 proof="$5/.fm-tasktmp-owner" record + record=$(fm_tasktmp_owner_record "$state" "$task" "$generation") || return 1 + fm_tasktmp_owner_write "$state" "$home" "$task" "$generation" "$root" || return 1 + if ! mkdir "$root"; then + rm -f "$record" + return 1 + fi + if ! cp "$record" "$proof" || ! chmod 600 "$proof" || ! mkdir "$root/gotmp"; then + rm -rf "$root" + rm -f "$record" + return 1 + fi +} + +fm_tasktmp_remove_owned() { # + local state=$1 home=$2 task=$3 generation=$4 root=$5 record + record=$(fm_tasktmp_owner_record "$state" "$task" "$generation") || return 1 + fm_tasktmp_owner_validate "$record" "$home" "$task" "$generation" "$root" || return 1 + if [ -e "$root" ] || [ -L "$root" ]; then + [ -d "$root" ] && [ ! -L "$root" ] || return 1 + fm_tasktmp_owner_validate "$root/.fm-tasktmp-owner" "$home" "$task" "$generation" "$root" || return 1 + rm -rf "$root" || return 1 + fi + rm -f "$record" +} + fm_account_ps_bin() { if fm_account_test_lab_enabled \ && [ "${FM_ACCOUNT_TEST_HOOKS:-}" = firstmate-account-tests-v1 ] \ @@ -1195,7 +1273,7 @@ fm_account_meta_value() { # } fm_account_restore_artifacts() { - local state=$1 task=$2 backup_name=$3 tasktmp=${4:-} retain=${5:-0} backup name source + local state=$1 task=$2 backup_name=$3 tasktmp=${4:-} retain=${5:-0} backup name source generation owner_record local PATH=$FM_ACCOUNT_SYSTEM_PATH [ -n "$backup_name" ] || return 0 case "$backup_name" in @@ -1213,11 +1291,15 @@ fm_account_restore_artifacts() { fi done if [ -n "$tasktmp" ]; then - [ "$tasktmp" = "/tmp/fm-$task" ] || return 1 + generation=$(sed -n '3s/^generation=//p' "$tasktmp/.fm-tasktmp-owner" 2>/dev/null) + owner_record=$(fm_tasktmp_owner_record "$state" "$task" "$generation") || return 1 + fm_tasktmp_owner_validate "$owner_record" "${FM_HOME:?}" "$task" "$generation" "$tasktmp" || return 1 + fm_tasktmp_owner_validate "$tasktmp/.fm-tasktmp-owner" "${FM_HOME:?}" "$task" "$generation" "$tasktmp" || return 1 if [ -e "$backup/tasktmp-existed" ]; then [ -e "$backup/gotmp-existed" ] || fm_account_system_exec "$FM_ACCOUNT_SYSTEM_RM_BIN" -rf "$tasktmp/gotmp" || return 1 else fm_account_system_exec "$FM_ACCOUNT_SYSTEM_RM_BIN" -rf "$tasktmp" || return 1 + fm_account_system_exec "$FM_ACCOUNT_SYSTEM_RM_BIN" -f "$owner_record" || return 1 fi fi [ "$retain" = 1 ] || fm_account_system_exec "$FM_ACCOUNT_SYSTEM_RM_BIN" -rf "$backup" diff --git a/bin/fm-auto-reap.sh b/bin/fm-auto-reap.sh new file mode 100755 index 0000000000..764d0e8fc3 --- /dev/null +++ b/bin/fm-auto-reap.sh @@ -0,0 +1,495 @@ +#!/usr/bin/env bash +# Automatically reap terminal crewmate resources through fm-teardown.sh. +# +# `task ` validates the terminal event, +# reaps an exactly-attributed no-mistakes run when necessary, and delegates all +# endpoint, cleanliness, landing, report, and Treehouse-return proofs to ordinary +# teardown without --force. +# +# `maintenance` recovers pre-metadata Treehouse acquisitions left by a crashed +# spawn. A record is eligible only after an age threshold and exact PID/start-time +# death proof. Recovery installs fail-closed cleanup metadata, then invokes the +# same ordinary teardown proof. Every refusal stays on disk and is printed. +# Usage: fm-auto-reap.sh task +# fm-auto-reap.sh maintenance +set -eu + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +FM_ROOT="${FM_ROOT_OVERRIDE:-$(cd "$SCRIPT_DIR/.." && pwd)}" +FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$FM_ROOT}}" +STATE="${FM_STATE_OVERRIDE:-$FM_HOME/state}" +AUTO_REAP_STALE_SECS=${FM_AUTO_REAP_STALE_SECS:-300} +AUTO_REAP_COMMAND_TIMEOUT=${FM_AUTO_REAP_COMMAND_TIMEOUT:-20} + +# shellcheck source=bin/fm-gate-refuse-lib.sh +. "$SCRIPT_DIR/fm-gate-refuse-lib.sh" +fm_refuse_if_gate_agent +# shellcheck source=bin/fm-account-routing-lib.sh +. "$SCRIPT_DIR/fm-account-routing-lib.sh" +# shellcheck source=bin/fm-checkout-lock-lib.sh +. "$SCRIPT_DIR/fm-checkout-lock-lib.sh" +# shellcheck source=bin/fm-treehouse-lib.sh +. "$SCRIPT_DIR/fm-treehouse-lib.sh" +# shellcheck source=bin/fm-process-tree-lib.sh +. "$SCRIPT_DIR/fm-process-tree-lib.sh" + +case "$AUTO_REAP_STALE_SECS:$AUTO_REAP_COMMAND_TIMEOUT" in + *[!0-9:]*|0:*|*:0) + echo "error: auto-reap age and command timeouts must be positive integers" >&2 + exit 2 + ;; +esac +[ -d "$STATE" ] && [ ! -L "$STATE" ] || { + echo "error: auto-reap state must be a real directory: $STATE" >&2 + exit 1 +} + +auto_reap_tool() { # + local variable=$1 fallback=$2 value + eval "value=\${$variable:-}" + if [ -n "$value" ]; then + [ "${FM_AUTO_REAP_TEST_HOOKS:-}" = firstmate-auto-reap-tests-v1 ] || { + echo "error: $variable requires the explicit auto-reap test hook" >&2 + return 1 + } + [ -x "$value" ] || return 1 + printf '%s\n' "$value" + return 0 + fi + command -v "$fallback" +} + +meta_value() { # + local meta=$1 key=$2 + sed -n "s/^${key}=//p" "$meta" | tail -1 +} + +single_meta_value() { # + local file=$1 key=$2 values count + values=$(sed -n "s/^${key}=//p" "$file") + count=$(printf '%s\n' "$values" | awk 'NF { n++ } END { print n + 0 }') + [ "$count" -eq 1 ] || return 1 + printf '%s\n' "$values" +} + +status_last_verb() { # + local last + last=$(grep -v '^[[:space:]]*$' "$STATE/$1.status" 2>/dev/null | tail -1) + last=${last%%:*} + last=${last%%\[key=*} + printf '%s' "$last" | tr -d '[:space:]' +} + +refuse() { + printf 'auto-reap refused %s: %s\n' "${AUTO_REAP_ID:-maintenance}" "$*" >&2 + return 1 +} + +log_result() { + local line=$1 log="$STATE/.auto-reap.log" size tmp + [ ! -L "$log" ] && { [ ! -e "$log" ] || [ -f "$log" ]; } || return 0 + printf '[%s] %s\n' "$(date '+%Y-%m-%dT%H:%M:%S%z')" "$line" >> "$log" 2>/dev/null || return 0 + size=$(wc -c < "$log" 2>/dev/null | tr -d '[:space:]') + case "$size" in ''|*[!0-9]*) return 0 ;; esac + if [ "$size" -ge 262144 ]; then + tmp=$(mktemp "$STATE/.auto-reap-log.XXXXXX") || return 0 + tail -n 2000 "$log" > "$tmp" 2>/dev/null && mv "$tmp" "$log" 2>/dev/null || true + rm -f "$tmp" 2>/dev/null || true + fi +} + +run_capture() { # [args...] + local output_variable=$1 + shift + if fm_run_bounded_capture --combine-stderr "$output_variable" "$AUTO_REAP_COMMAND_TIMEOUT" "$@"; then + RUN_CAPTURE_STATUS=0 + else + RUN_CAPTURE_STATUS=$? + fi + fm_process_tree_cleanup_verified || { + echo "error: bounded auto-reap command cleanup could not be verified" >&2 + return 1 + } + return 0 +} + +parse_pr_url() { # + local url=$1 rest + case "$url" in + https://github.com/*/*/pull/[0-9]*) ;; + *) return 1 ;; + esac + rest=${url#https://github.com/} + PR_OWNER=${rest%%/*} + rest=${rest#*/} + PR_REPO=${rest%%/*} + rest=${rest#*/pull/} + PR_NUMBER=${rest%%[/?#]*} + case "$PR_OWNER:$PR_REPO:$PR_NUMBER" in + *[!A-Za-z0-9._:/-]*|::*|*::*|*:) return 1 ;; + esac + case "$PR_NUMBER" in ''|*[!0-9]*) return 1 ;; esac +} + +pr_is_merged() { # + local meta=$1 pr gh captured + pr=$(meta_value "$meta" pr) + parse_pr_url "$pr" || { + refuse "recorded PR URL is missing or invalid" + return 1 + } + gh=$(auto_reap_tool FM_AUTO_REAP_GH_AXI_BIN gh-axi) || { + refuse "gh-axi is unavailable" + return 1 + } + run_capture captured "$gh" pr view "$PR_NUMBER" -R "$PR_OWNER/$PR_REPO" || return 1 + [ "$RUN_CAPTURE_STATUS" -eq 0 ] || { + refuse "gh-axi could not verify PR $PR_NUMBER as merged" + return 1 + } + [ "$(printf '%s\n' "$captured" | grep -Ec '^[[:space:]]*state:[[:space:]]*merged[[:space:]]*$')" -eq 1 ] || { + refuse "PR $PR_NUMBER is not provably merged" + return 1 + } +} + +reap_no_mistakes_run() { # + local meta=$1 worktree branch nm captured run_branch run_id run_status outcome runs_output row status rest row_branch + [ "$(meta_value "$meta" mode)" = no-mistakes ] || return 0 + worktree=$(meta_value "$meta" worktree) + branch=$(git -C "$worktree" symbolic-ref --quiet --short HEAD 2>/dev/null || true) + [ "$branch" = "fm/$AUTO_REAP_ID" ] || { + refuse "no-mistakes run attribution requires exact branch fm/$AUTO_REAP_ID" + return 1 + } + nm=$(auto_reap_tool FM_AUTO_REAP_NO_MISTAKES_BIN no-mistakes) || { + refuse "no-mistakes is unavailable for exact run reaping" + return 1 + } + if run_capture captured env -C "$worktree" "$nm" axi status; then :; else return 1; fi + [ "$RUN_CAPTURE_STATUS" -eq 0 ] || { + refuse "no-mistakes status could not be inspected safely" + return 1 + } + run_branch=$(printf '%s\n' "$captured" | sed -n 's/^[[:space:]]*branch:[[:space:]]*//p' | head -1) + run_branch=${run_branch#\"} + run_branch=${run_branch%\"} + if [ "$run_branch" = "$branch" ]; then + run_id=$(printf '%s\n' "$captured" | sed -n 's/^[[:space:]]*id:[[:space:]]*//p' | head -1) + run_id=${run_id#\"} + run_id=${run_id%\"} + run_status=$(printf '%s\n' "$captured" | sed -n 's/^[[:space:]]*status:[[:space:]]*//p' | head -1) + run_status=${run_status#\"} + run_status=${run_status%\"} + outcome=$(printf '%s\n' "$captured" | sed -n 's/^[[:space:]]*outcome:[[:space:]]*//p' | head -1) + outcome=${outcome#\"} + outcome=${outcome%\"} + [ -n "$run_id" ] || { + refuse "matching no-mistakes run has no exact run ID" + return 1 + } + if [ -z "$outcome" ] && [ "$run_status" != completed ] \ + && [ "$run_status" != cancelled ] && [ "$run_status" != failed ]; then + if run_capture captured "$nm" axi abort --run "$run_id"; then :; else return 1; fi + [ "$RUN_CAPTURE_STATUS" -eq 0 ] || { + refuse "failed to cancel exact no-mistakes run $run_id" + return 1 + } + log_result "cancelled no-mistakes run $run_id for $AUTO_REAP_ID" + fi + return 0 + fi + if run_capture runs_output env -C "$worktree" "$nm" runs --limit 200; then :; else return 1; fi + [ "$RUN_CAPTURE_STATUS" -eq 0 ] || { + refuse "cross-branch no-mistakes attribution could not be inspected" + return 1 + } + while IFS= read -r row; do + row=$(printf '%s' "$row" | sed 's/^[[:space:]]*//') + [ -n "$row" ] || continue + status=${row%%[[:space:]]*} + rest=${row#"$status"} + rest=$(printf '%s' "$rest" | sed 's/^[[:space:]]*//') + row_branch=${rest%%[[:space:]]*} + [ "$row_branch" = "$branch" ] || continue + case "$status" in + running) + refuse "branch still has an active no-mistakes run but its exact run ID is unavailable" + return 1 + ;; + esac + return 0 + done < + local task=$1 teardown output + teardown=$(auto_reap_tool FM_AUTO_REAP_TEARDOWN_BIN fm-teardown.sh 2>/dev/null || true) + [ -n "$teardown" ] || teardown="$SCRIPT_DIR/fm-teardown.sh" + if output=$("$teardown" "$task" 2>&1); then + [ -z "$output" ] || printf '%s\n' "$output" + log_result "auto-reaped $task" + printf 'auto-reaped %s\n' "$task" + return 0 + fi + [ -z "$output" ] || printf '%s\n' "$output" >&2 + refuse "ordinary teardown refused; retained endpoint/worktree metadata" +} + +reap_task() { # + local id=$1 trigger=$2 meta kind mode + AUTO_REAP_ID=$id + fm_account_valid_id "$id" || refuse "invalid task id" + meta="$STATE/$id.meta" + [ -f "$meta" ] && [ ! -L "$meta" ] || refuse "task metadata is unavailable" + kind=$(meta_value "$meta" kind) + [ -n "$kind" ] || kind=ship + mode=$(meta_value "$meta" mode) + [ -z "$(meta_value "$meta" x_request)" ] || refuse "X-linked tasks require their final follow-up before teardown" + [ "$kind" != secondmate ] || refuse "persistent secondmates are never auto-reaped" + [ "$(status_last_verb "$id")" = "done" ] || refuse "last task status is not terminal done" + case "$trigger:$kind:$mode" in + pr-merged:ship:local-only) refuse "local-only tasks cannot use a PR-merged trigger" ;; + pr-merged:ship:*) + [ "${AUTO_REAP_PR_VERIFIED:-0}" = 1 ] || pr_is_merged "$meta" || return 1 + ;; + scout-done:scout:*) ;; + local-merged:ship:local-only) ;; + *) refuse "trigger $trigger does not match kind=$kind mode=${mode:-no-mistakes}" ;; + esac + reap_no_mistakes_run "$meta" || return 1 + run_teardown "$id" +} + +path_age() { + local path=$1 mtime + if [ "$(uname)" = Darwin ]; then + mtime=$(stat -f %m "$path" 2>/dev/null) || return 1 + else + mtime=$(stat -c %Y "$path" 2>/dev/null) || return 1 + fi + printf '%s\n' "$(( $(date +%s) - mtime ))" +} + +recover_acquisition() { # + local record=$1 id project holder recorded_worktree worktree snapshot owner_state lock tmp find_status absence_status + local recorded_home home_real generation tasktmp tasktmp_owner + [ -f "$record" ] && [ ! -L "$record" ] || return 0 + [ "$(path_age "$record")" -ge "$AUTO_REAP_STALE_SECS" ] || return 0 + AUTO_REAP_ID=${record##*/.worktree-acquire-} + AUTO_REAP_ID=${AUTO_REAP_ID%.pending} + if fm_account_lock_owner_state "$record"; then owner_state=0; else owner_state=$?; fi + case "$owner_state" in + 0) return 0 ;; + 1) ;; + 2) + refuse "stale acquisition owner liveness is indeterminate" + return 0 + ;; + *) + refuse "stale acquisition owner record is malformed" + return 0 + ;; + esac + id=$(single_meta_value "$record" id) || { + refuse "stale acquisition record has invalid task identity" + return 0 + } + if [ "$id" != "$AUTO_REAP_ID" ] || ! fm_account_valid_id "$id"; then + refuse "stale acquisition filename does not match its task identity" + return 0 + fi + project=$(single_meta_value "$record" project) || { + refuse "stale acquisition record has invalid project" + return 0 + } + holder=$(single_meta_value "$record" holder) || { + refuse "stale acquisition record has invalid lease holder" + return 0 + } + [ "$holder" = "firstmate-$id" ] || { + refuse "stale acquisition lease holder does not match task" + return 0 + } + recorded_home=$(single_meta_value "$record" home) || { + refuse "stale acquisition record has missing or malformed FM_HOME ownership" + return 0 + } + home_real=$(cd "$FM_HOME" 2>/dev/null && pwd -P) || return 0 + [ "$recorded_home" = "$home_real" ] || { + refuse "stale acquisition record belongs to a different canonical FM_HOME" + return 0 + } + generation=$(single_meta_value "$record" generation_id) || { + refuse "stale acquisition record has missing or malformed generation ownership" + return 0 + } + tasktmp=$(single_meta_value "$record" tasktmp) || { + refuse "stale acquisition record has missing or malformed task temp ownership" + return 0 + } + [ "$(fm_tasktmp_path "$id" "$generation" 2>/dev/null)" = "$tasktmp" ] || { + refuse "stale acquisition task temp path does not match its exact generation" + return 0 + } + snapshot=$(cat "$record") || return 0 + lock=$(fm_account_lifecycle_lock_acquire "$STATE" "$id") || { + refuse "could not serialize stale acquisition recovery" + return 0 + } + if [ "$(cat "$record" 2>/dev/null || true)" != "$snapshot" ]; then + fm_account_lifecycle_lock_release "$lock" >/dev/null 2>&1 || true + refuse "stale acquisition record changed during recovery" + return 0 + fi + if [ -e "$STATE/$id.meta" ] || [ -L "$STATE/$id.meta" ]; then + fm_account_lifecycle_lock_release "$lock" >/dev/null 2>&1 || true + refuse "task metadata exists; retained stale acquisition for operator reconciliation" + return 0 + fi + recorded_worktree=$(sed -n 's/^worktree=//p' "$record" | tail -1) + if [ -n "$recorded_worktree" ]; then + worktree=$(fm_checkout_trusted_dir "$recorded_worktree" 2>/dev/null || true) + [ -n "$worktree" ] && fm_treehouse_require_task_lease "$worktree" "$holder" >/dev/null 2>&1 || worktree= + else + worktree= + fi + if [ -z "$worktree" ]; then + if worktree=$(fm_treehouse_find_task_lease "$project" "$holder" 2>/dev/null); then + find_status=0 + else + find_status=$? + fi + if [ "$find_status" -eq 2 ]; then + if fm_treehouse_prove_task_lease_absent "$recorded_worktree" "$holder" >/dev/null 2>&1; then + absence_status=0 + else + absence_status=$? + fi + if [ "$absence_status" -eq 0 ]; then + tasktmp_owner=$(fm_tasktmp_owner_record "$STATE" "$id" "$generation") || { + fm_account_lifecycle_lock_release "$lock" >/dev/null 2>&1 || true + refuse "stale acquisition task temp ownership record is malformed" + return 0 + } + if [ -e "$tasktmp_owner" ] || [ -L "$tasktmp_owner" ]; then + fm_tasktmp_remove_owned "$STATE" "$FM_HOME" "$id" "$generation" "$tasktmp" || { + fm_account_lifecycle_lock_release "$lock" >/dev/null 2>&1 || true + refuse "Treehouse lease is absent but task temp ownership is forged or ambiguous; retained stale acquisition" + return 0 + } + elif [ -e "$tasktmp" ] || [ -L "$tasktmp" ]; then + fm_account_lifecycle_lock_release "$lock" >/dev/null 2>&1 || true + refuse "Treehouse lease is absent but task temp root has no authoritative ownership record; retained stale acquisition" + return 0 + fi + rm -f "$record" + fm_account_lifecycle_lock_release "$lock" >/dev/null 2>&1 || true + log_result "cleared owner-dead acquisition $id after proving it owns no Treehouse lease" + printf 'auto-reap cleared %s: owner is dead and no Treehouse lease exists\n' "$id" + return 0 + fi + fm_account_lifecycle_lock_release "$lock" >/dev/null 2>&1 || true + if [ "$absence_status" -eq 2 ]; then + log_result "CORRUPT authoritative Treehouse lease state retained owner-dead acquisition $id" + refuse "CORRUPT authoritative Treehouse lease state; retained stale acquisition" || true + return 0 + fi + log_result "retained owner-dead acquisition $id because Treehouse lease absence could not be proven" + refuse "retained stale acquisition because Treehouse lease absence could not be proven" || true + return 0 + fi + fi + [ -n "$worktree" ] || { + fm_account_lifecycle_lock_release "$lock" >/dev/null 2>&1 || true + refuse "exact stranded Treehouse lease is not uniquely provable" + return 0 + } + tmp=$(mktemp "$STATE/.$id.meta.auto-reap.XXXXXX") || { + fm_account_lifecycle_lock_release "$lock" >/dev/null 2>&1 || true + return 0 + } + { + printf 'window=\n' + printf 'worktree=%s\n' "$worktree" + printf 'project=%s\n' "$project" + printf 'harness=unknown\nkind=ship\n' + printf 'mode=%s\n' "$(single_meta_value "$record" mode 2>/dev/null || printf no-mistakes)" + printf 'yolo=%s\n' "$(single_meta_value "$record" yolo 2>/dev/null || printf off)" + printf 'tasktmp=%s\nmodel=default\neffort=default\n' "$tasktmp" + printf 'generation_id=%s\n' "$generation" + printf 'direct_spawn_endpoint=not-created\n' + printf 'direct_spawn_cleanup=pending\nrollback_pending=1\n' + } > "$tmp" + mv "$tmp" "$STATE/$id.meta" || { + rm -f "$tmp" + fm_account_lifecycle_lock_release "$lock" >/dev/null 2>&1 || true + return 0 + } + fm_account_lifecycle_lock_release "$lock" >/dev/null 2>&1 || { + refuse "failed to release recovery lifecycle lock" + return 0 + } + if run_teardown "$id"; then + rm -f "$record" + fi +} + +maintenance() { + local record meta id kind mode probe_status + for record in "$STATE"/.worktree-acquire-*.pending; do + [ -e "$record" ] || continue + recover_acquisition "$record" + done + # Backstop terminal events even when a task-specific check or turn-end signal + # was missed while no watcher was running. An ordinary open PR is expected and + # stays silent; once GitHub proves merged, every subsequent refusal is surfaced. + for meta in "$STATE"/*.meta; do + [ -f "$meta" ] && [ ! -L "$meta" ] || continue + id=${meta##*/} + id=${id%.meta} + fm_account_valid_id "$id" || continue + [ "$(status_last_verb "$id")" = "done" ] || continue + [ -z "$(meta_value "$meta" x_request)" ] || continue + kind=$(meta_value "$meta" kind) + [ -n "$kind" ] || kind=ship + mode=$(meta_value "$meta" mode) + case "$kind:$mode" in + scout:*) + reap_task "$id" scout-done + ;; + ship:local-only) + # The approved merge action calls auto-reap synchronously. Do not probe + # an unapproved local branch as if it were a terminal merge. + ;; + ship:*) + [ -n "$(meta_value "$meta" pr)" ] || continue + AUTO_REAP_ID=$id + if pr_is_merged "$meta" >/dev/null 2>&1; then + probe_status=0 + else + probe_status=$? + fi + [ "$probe_status" -eq 0 ] || continue + AUTO_REAP_PR_VERIFIED=1 reap_task "$id" pr-merged + ;; + esac + done +} + +case "${1:-}" in + task) + [ "$#" -eq 3 ] || { echo "usage: fm-auto-reap.sh task " >&2; exit 2; } + reap_task "$2" "$3" + ;; + maintenance) + [ "$#" -eq 1 ] || { echo "usage: fm-auto-reap.sh maintenance" >&2; exit 2; } + maintenance + ;; + *) + echo "usage: fm-auto-reap.sh task | maintenance" >&2 + exit 2 + ;; +esac diff --git a/bin/fm-checkout-refresh.sh b/bin/fm-checkout-refresh.sh index 91af403fb6..9aed34426f 100755 --- a/bin/fm-checkout-refresh.sh +++ b/bin/fm-checkout-refresh.sh @@ -1869,8 +1869,8 @@ record_alert() { record_reinspection_failure() { local checkout=$1 key alert output - key=$(checkout_key "$checkout") || { - printf '%s: skipped: covered checkout lock identity cannot be resolved\n' "$checkout" + key=$(fm_checkout_hash_value "$checkout" 24) || { + printf '%s: skipped: covered checkout alert identity cannot be resolved\n' "$checkout" return 1 } alert="$STATE_ROOT/$key.alert" diff --git a/bin/fm-merge-local.sh b/bin/fm-merge-local.sh index 8936f2ae28..e1b8543406 100755 --- a/bin/fm-merge-local.sh +++ b/bin/fm-merge-local.sh @@ -71,3 +71,7 @@ before=$(git -C "$PROJ" rev-parse --short "$DEFAULT") git -C "$PROJ" merge --ff-only "$BRANCH" >/dev/null after=$(git -C "$PROJ" rev-parse --short "$DEFAULT") echo "merged $BRANCH into local $DEFAULT ($before -> $after) in $PROJ" +if ! "$SCRIPT_DIR/fm-auto-reap.sh" task "$ID" local-merged; then + echo "error: local merge landed, but automatic teardown refused and retained task state for inspection" >&2 + exit 1 +fi diff --git a/bin/fm-spawn.sh b/bin/fm-spawn.sh index e66d231827..0ea32531db 100755 --- a/bin/fm-spawn.sh +++ b/bin/fm-spawn.sh @@ -602,7 +602,9 @@ reconcile_failed_direct_recovery() { echo "error: retained direct recovery artifacts are missing or unsafe for $task" >&2 return 1 fi - if [ "$tasktmp" != "/tmp/fm-$task" ] || [ -z "$generation" ] || [ -z "$target" ]; then + if ! fm_tasktmp_owner_validate "$(fm_tasktmp_owner_record "$STATE" "$task" "$generation")" "$FM_HOME" "$task" "$generation" "$tasktmp" \ + || ! fm_tasktmp_owner_validate "$tasktmp/.fm-tasktmp-owner" "$FM_HOME" "$task" "$generation" "$tasktmp" \ + || [ -z "$target" ]; then fm_account_meta_lock_release "$lock" >/dev/null 2>&1 || true echo "error: retained direct recovery metadata is incomplete for $task" >&2 return 1 @@ -864,10 +866,12 @@ if [ "$RECOVERY_ACCOUNT" = 1 ]; then rollback_tab=$(fm_account_meta_value "$RESUME_META" zellij_tab_id) rollback_home=$(fm_account_meta_value "$RESUME_META" home) rollback_tasktmp=$(fm_account_meta_value "$RESUME_META" tasktmp) + rollback_generation=$(fm_account_meta_value "$RESUME_META" generation_id) rollback_backup=$(fm_account_meta_value "$RESUME_META" account_rollback_backup) fm_account_meta_lock_release "$rollback_meta_lock" || exit 1 rollback_meta_lock= - if [ -n "$rollback_tasktmp" ] && [ "$rollback_tasktmp" != "/tmp/fm-$rollback_id" ]; then + if [ -n "$rollback_tasktmp" ] \ + && [ "$(fm_tasktmp_path "$rollback_id" "$rollback_generation" 2>/dev/null)" != "$rollback_tasktmp" ]; then echo "error: unsafe task temp path in rollback metadata for $rollback_id: $rollback_tasktmp" >&2 exit 1 fi @@ -892,8 +896,17 @@ if [ "$RECOVERY_ACCOUNT" = 1 ]; then fi rollback_profile=$(fm_account_meta_value "$RESUME_META" account_profile) if [ -z "$rollback_profile" ] && [ "$rollback_kind" = secondmate ] && [ -z "$rollback_backup" ]; then + rollback_tasktmp_owner=$(fm_tasktmp_owner_record "$STATE" "$rollback_id" "$rollback_generation") || exit 1 + if [ -e "$rollback_tasktmp_owner" ] || [ -L "$rollback_tasktmp_owner" ]; then + fm_tasktmp_remove_owned "$STATE" "$FM_HOME" "$rollback_id" "$rollback_generation" "$rollback_tasktmp" || { + echo "error: cleaned secondmate attempt retained ambiguous task temp ownership for $rollback_id" >&2 + exit 1 + } + elif [ -e "$rollback_tasktmp" ] || [ -L "$rollback_tasktmp" ]; then + echo "error: cleaned secondmate attempt retained an unowned task temp root for $rollback_id" >&2 + exit 1 + fi rm -f "$RESUME_META" "$STATE/$rollback_id.status" "$STATE/$rollback_id.turn-ended" "$STATE/$rollback_id.check.sh" "$STATE/$rollback_id.pi-ext.ts" "$STATE/$rollback_id.grok-turnend-token" - [ -z "$rollback_tasktmp" ] || rm -rf "$rollback_tasktmp" fi if [ -z "$rollback_profile" ]; then if [ -n "$rollback_backup" ]; then @@ -975,6 +988,8 @@ T= WORKTREE_CREATED=0 WORKTREE_RETAIN_ON_ABORT=0 WORKTREE_EXPECTED_TIP= +WORKTREE_ACQUIRE_RECORD= +WORKTREE_ACQUIRE_OWNER_START= META_INSTALLED=0 META_BACKUP= EXISTING_ARTIFACT_BACKUP= @@ -1006,8 +1021,13 @@ ORIGINAL_PI_EXT_PRESENT=-1 ORIGINAL_GROK_TOKEN_PRESENT=-1 ORIGINAL_TASK_TMP_PRESENT=-1 +spawn_test_lab_enabled() { + fm_account_test_lab_enabled \ + || [ "${FM_ACCOUNT_DIRECTORY_TEST_LAB:-}" = firstmate-account-directory-test-lab-v1 ] +} + snapshot_existing_artifacts() { - local backup name source tasktmp="/tmp/fm-$ID" + local backup name source tasktmp=${TASK_TMP:-} backup=$(mktemp -d "$STATE/.$ID.artifacts.rollback.XXXXXX") || return 1 for name in "$ID.status" "$ID.turn-ended" "$ID.check.sh" "$ID.pi-ext.ts" "$ID.grok-turnend-token"; do source="$STATE/$name" @@ -1042,9 +1062,10 @@ parse_orca_worktree_result() { ORCA_TERMINAL_PROOF=${rest%%$'\t'*} [ "$rest" != "$ORCA_TERMINAL_PROOF" ] || return 1 rest=${rest#*$'\t'} - ORCA_REPO_ID=$rest - case "$ORCA_REPO_ID" in *$'\t'*) return 1 ;; esac - ORCA_PROVIDER_TASK= + ORCA_REPO_ID=${rest%%$'\t'*} + [ "$rest" != "$ORCA_REPO_ID" ] || return 1 + ORCA_PROVIDER_TASK=${rest#*$'\t'} + case "$ORCA_PROVIDER_TASK" in *$'\t'*) return 1 ;; esac } persist_orca_cleanup_quarantine() { @@ -1262,7 +1283,7 @@ persist_failed_direct_recovery() { echo "kind=${KIND:-${RECORDED_KIND:-ship}}" echo "mode=${MODE:-${RECORDED_MODE:-no-mistakes}}" echo "yolo=${YOLO:-${RECORDED_YOLO:-off}}" - echo "tasktmp=${TASK_TMP:-${RECORDED_TASKTMP:-/tmp/fm-$ID}}" + echo "tasktmp=${TASK_TMP:-${RECORDED_TASKTMP:-}}" echo "model=${RECORDED_MODEL:-${MODEL:-default}}" echo "effort=${RECORDED_EFFORT:-${EFFORT:-default}}" echo "generation_id=${RECORDED_GENERATION:-${SPAWN_GENERATION_ID:-}}" @@ -1351,7 +1372,7 @@ persist_failed_direct_spawn() { # echo "kind=${KIND:-ship}" echo "mode=$retained_mode" echo "yolo=$retained_yolo" - echo "tasktmp=${TASK_TMP:-/tmp/fm-$ID}" + echo "tasktmp=${TASK_TMP:-}" echo "model=${MODEL:-default}" echo "effort=${EFFORT:-default}" echo "generation_id=${SPAWN_GENERATION_ID:-}" @@ -1425,6 +1446,68 @@ cleanup_continuation_launch_transport() { CONTINUATION_PROMPT_CONTENT_ID= } +create_worktree_acquisition_record() { + local record tmp start home_real + start=$(fm_account_process_start_time "$$") || { + echo "error: cannot record Treehouse acquisition owner for $ID" >&2 + return 1 + } + record="$STATE/.worktree-acquire-$ID.pending" + [ ! -e "$record" ] && [ ! -L "$record" ] || { + echo "error: stale or concurrent Treehouse acquisition record exists for $ID; let auto-reap reconcile it before retrying" >&2 + return 1 + } + tmp=$(mktemp "$STATE/.worktree-acquire-$ID.XXXXXX") || return 1 + home_real=$(cd "$FM_HOME" 2>/dev/null && pwd -P) || { rm -f "$tmp"; return 1; } + { + printf '%s\n%s\n' "$$" "$start" + printf 'id=%s\n' "$ID" + printf 'project=%s\n' "$PROJ_ABS" + printf 'holder=firstmate-%s\n' "$ID" + printf 'home=%s\n' "$home_real" + printf 'kind=%s\nmode=%s\nyolo=%s\n' "$KIND" "$MODE" "$YOLO" + printf 'generation_id=%s\ntasktmp=%s\nworktree=\n' "$SPAWN_GENERATION_ID" "$TASK_TMP" + } > "$tmp" || { + rm -f "$tmp" + return 1 + } + if ! ln "$tmp" "$record" 2>/dev/null; then + rm -f "$tmp" + echo "error: could not claim Treehouse acquisition record for $ID" >&2 + return 1 + fi + rm -f "$tmp" + WORKTREE_ACQUIRE_RECORD=$record + WORKTREE_ACQUIRE_OWNER_START=$start +} + +record_acquired_worktree() { + local tmp + [ -n "$WORKTREE_ACQUIRE_RECORD" ] || return 1 + [ "$(sed -n '1p' "$WORKTREE_ACQUIRE_RECORD" 2>/dev/null)" = "$$" ] \ + && [ "$(sed -n '2p' "$WORKTREE_ACQUIRE_RECORD" 2>/dev/null)" = "$WORKTREE_ACQUIRE_OWNER_START" ] || { + echo "error: Treehouse acquisition ownership changed for $ID" >&2 + return 1 + } + tmp=$(mktemp "$STATE/.worktree-acquire-$ID.update.XXXXXX") || return 1 + if ! sed '/^worktree=/d' "$WORKTREE_ACQUIRE_RECORD" > "$tmp" \ + || ! printf 'worktree=%s\n' "$WT" >> "$tmp" \ + || ! mv "$tmp" "$WORKTREE_ACQUIRE_RECORD"; then + rm -f "$tmp" + return 1 + fi +} + +clear_worktree_acquisition_record() { + [ -n "${WORKTREE_ACQUIRE_RECORD:-}" ] || return 0 + if [ "$(sed -n '1p' "$WORKTREE_ACQUIRE_RECORD" 2>/dev/null)" = "$$" ] \ + && [ "$(sed -n '2p' "$WORKTREE_ACQUIRE_RECORD" 2>/dev/null)" = "$WORKTREE_ACQUIRE_OWNER_START" ]; then + rm -f "$WORKTREE_ACQUIRE_RECORD" + fi + WORKTREE_ACQUIRE_RECORD= + WORKTREE_ACQUIRE_OWNER_START= +} + spawn_return_created_worktree() { local return_output return_status [ "$WORKTREE_CREATED" = 1 ] || return 0 @@ -1435,16 +1518,16 @@ spawn_return_created_worktree() { return 1 fi if [ -z "$WORKTREE_EXPECTED_TIP" ] \ - || ! "$SCRIPT_DIR/fm-checkout-refresh.sh" verify-returnable "$WT" "$PROJ_ABS" "$WORKTREE_EXPECTED_TIP"; then + || ! "$SCRIPT_DIR/fm-checkout-refresh.sh" verify-returnable "$WT" "${PROJ_ABS_REAL:-$PROJ_ABS}" "$WORKTREE_EXPECTED_TIP"; then echo "warning: retained acquired worktree $WT because repository identity and its expected detached tip could not be re-proven" >&2 return 1 fi rm -f "$WT/.claude/settings.local.json" "$WT/.opencode/plugins/fm-turn-end.js" "$WT/.fm-grok-turnend" - if ! "$SCRIPT_DIR/fm-checkout-refresh.sh" verify-returnable "$WT" "$PROJ_ABS" "$WORKTREE_EXPECTED_TIP"; then + if ! "$SCRIPT_DIR/fm-checkout-refresh.sh" verify-returnable "$WT" "${PROJ_ABS_REAL:-$PROJ_ABS}" "$WORKTREE_EXPECTED_TIP"; then echo "warning: retained acquired worktree $WT because post-cleanup repository safety could not be re-proven" >&2 return 1 fi - if return_output=$(fm_checkout_treehouse_return "$WT" "$CHECKOUT_LOCK_ROOT" "$PROJ_ABS" 2>&1); then + if return_output=$(fm_checkout_treehouse_return "$WT" "$CHECKOUT_LOCK_ROOT" "${PROJ_ABS_REAL:-$PROJ_ABS}" 2>&1); then [ -z "$return_output" ] || printf '%s\n' "$return_output" >&2 return 0 else @@ -1464,7 +1547,7 @@ spawn_restore_unmanaged_state_locked() { [ "${ACCOUNT_EFFECTIVE_MODE:-off}" != enforce ] || return 0 if [ -n "$EXISTING_ARTIFACT_BACKUP" ]; then artifact_backup_name=${EXISTING_ARTIFACT_BACKUP##*/} - fm_account_restore_artifacts "$STATE" "$ID" "$artifact_backup_name" "/tmp/fm-$ID" 1 || return 1 + fm_account_restore_artifacts "$STATE" "$ID" "$artifact_backup_name" "${TASK_TMP:-}" 1 || return 1 fi if [ -n "$META_BACKUP" ]; then [ -f "$META_BACKUP" ] && [ -f "$meta" ] || return 1 @@ -1636,7 +1719,7 @@ spawn_abort_cleanup() { fi if [ -n "$rollback_lock" ]; then artifact_backup_name=${EXISTING_ARTIFACT_BACKUP##*/} - if fm_account_restore_artifacts "$STATE" "$ID" "$artifact_backup_name" "${TASK_TMP:-/tmp/fm-$ID}" 1; then + if fm_account_restore_artifacts "$STATE" "$ID" "$artifact_backup_name" "${TASK_TMP:-}" 1; then if [ "$META_INSTALLED" = 1 ] && [ -n "$META_BACKUP" ] && [ -f "$META_BACKUP" ]; then if fm_account_meta_merge_extensions "$STATE/$ID.meta" "$META_BACKUP" \ && fm_account_safe_file_destination "$STATE/$ID.meta" \ @@ -1670,7 +1753,7 @@ spawn_abort_cleanup() { if [ -n "$rollback_lock" ] && [ "$worktree_clean" = 1 ]; then if [ -n "$META_BACKUP" ] && [ -f "$META_BACKUP" ]; then artifact_backup_name=${EXISTING_ARTIFACT_BACKUP##*/} - if fm_account_restore_artifacts "$STATE" "$ID" "$artifact_backup_name" "${TASK_TMP:-/tmp/fm-$ID}" 1 \ + if fm_account_restore_artifacts "$STATE" "$ID" "$artifact_backup_name" "${TASK_TMP:-}" 1 \ && fm_account_meta_merge_extensions "$STATE/$ID.meta" "$META_BACKUP" \ && fm_account_safe_file_destination "$STATE/$ID.meta" \ && mv "$META_BACKUP" "$STATE/$ID.meta"; then @@ -1691,7 +1774,7 @@ spawn_abort_cleanup() { [ "$ORIGINAL_CHECK_PRESENT" != 0 ] || rm -f "$STATE/$ID.check.sh" [ "$ORIGINAL_PI_EXT_PRESENT" != 0 ] || rm -f "$STATE/$ID.pi-ext.ts" [ "$ORIGINAL_GROK_TOKEN_PRESENT" != 0 ] || rm -f "$STATE/$ID.grok-turnend-token" - [ "$ORIGINAL_TASK_TMP_PRESENT" != 0 ] || { [ -z "${TASK_TMP:-}" ] || rm -rf "$TASK_TMP"; } + [ "$ORIGINAL_TASK_TMP_PRESENT" != 0 ] || { [ -z "${TASK_TMP:-}" ] || fm_tasktmp_remove_owned "$STATE" "$FM_HOME" "$ID" "$SPAWN_GENERATION_ID" "$TASK_TMP"; } fi fi if [ "$worktree_clean" != 1 ]; then @@ -1770,7 +1853,7 @@ spawn_abort_cleanup() { [ "$ORIGINAL_CHECK_PRESENT" != 0 ] || rm -f "$STATE/$ID.check.sh" [ "$ORIGINAL_PI_EXT_PRESENT" != 0 ] || rm -f "$STATE/$ID.pi-ext.ts" [ "$ORIGINAL_GROK_TOKEN_PRESENT" != 0 ] || rm -f "$STATE/$ID.grok-turnend-token" - [ "$ORIGINAL_TASK_TMP_PRESENT" != 0 ] || { [ -z "${TASK_TMP:-}" ] || rm -rf "$TASK_TMP"; } + [ "$ORIGINAL_TASK_TMP_PRESENT" != 0 ] || { [ -z "${TASK_TMP:-}" ] || fm_tasktmp_remove_owned "$STATE" "$FM_HOME" "$ID" "$SPAWN_GENERATION_ID" "$TASK_TMP"; } fi if [ "$account_clean" != 1 ] && [ -n "$rollback_lock" ]; then persist_failed_account_rollback || echo "warning: failed to persist Agent Fleet rollback state for ${ID:-unknown}" >&2 @@ -1790,6 +1873,14 @@ spawn_abort_cleanup() { fi fi [ -z "$rollback_lock" ] || fm_account_meta_lock_release "$rollback_lock" >/dev/null 2>&1 || true + if [ -n "${WORKTREE_ACQUIRE_RECORD:-}" ]; then + if [ -f "$STATE/${ID:-unknown}.meta" ] \ + || { [ "$WORKTREE_CREATED" = 1 ] && [ "$worktree_clean" = 1 ]; }; then + clear_worktree_acquisition_record + else + echo "warning: retained Treehouse acquisition record for ${ID:-unknown}; auto-reap will reconcile any lease after owner death" >&2 + fi + fi [ -z "$META_BACKUP" ] || [ -f "$META_BACKUP" ] || META_BACKUP= [ -z "$EXISTING_ARTIFACT_BACKUP" ] || [ -d "$EXISTING_ARTIFACT_BACKUP" ] || EXISTING_ARTIFACT_BACKUP= [ "${LIFECYCLE_LOCK_OWNED:-0}" != 1 ] || [ -z "${LIFECYCLE_LOCK:-}" ] || fm_account_lifecycle_lock_release "$LIFECYCLE_LOCK" >/dev/null 2>&1 || true @@ -1929,7 +2020,7 @@ if [ -e "$STATE/$ID.turn-ended" ] || [ -L "$STATE/$ID.turn-ended" ]; then ORIGIN if [ -e "$STATE/$ID.check.sh" ] || [ -L "$STATE/$ID.check.sh" ]; then ORIGINAL_CHECK_PRESENT=1; else ORIGINAL_CHECK_PRESENT=0; fi if [ -e "$STATE/$ID.pi-ext.ts" ] || [ -L "$STATE/$ID.pi-ext.ts" ]; then ORIGINAL_PI_EXT_PRESENT=1; else ORIGINAL_PI_EXT_PRESENT=0; fi if [ -e "$STATE/$ID.grok-turnend-token" ] || [ -L "$STATE/$ID.grok-turnend-token" ]; then ORIGINAL_GROK_TOKEN_PRESENT=1; else ORIGINAL_GROK_TOKEN_PRESENT=0; fi -if [ -e "/tmp/fm-$ID" ] || [ -L "/tmp/fm-$ID" ]; then ORIGINAL_TASK_TMP_PRESENT=1; else ORIGINAL_TASK_TMP_PRESENT=0; fi +if [ -n "${TASK_TMP:-}" ] && { [ -e "$TASK_TMP" ] || [ -L "$TASK_TMP" ]; }; then ORIGINAL_TASK_TMP_PRESENT=1; else ORIGINAL_TASK_TMP_PRESENT=0; fi if [ "$RECOVERY_ACCOUNT" = 1 ]; then RECORDED_KIND=$(fm_meta_get "$RESUME_META" kind) @@ -2007,7 +2098,8 @@ if [ "$RECOVERY_ACCOUNT" = 1 ]; then [ -n "$RECORDED_MODE" ] || { echo "error: direct account recovery metadata has no mode for $ID" >&2; exit 1; } [ -n "$RECORDED_YOLO" ] || { echo "error: direct account recovery metadata has no yolo setting for $ID" >&2; exit 1; } [ -n "$RECORDED_GENERATION" ] || { echo "error: direct account recovery metadata has no generation_id for $ID" >&2; exit 1; } - [ "$RECORDED_TASKTMP" = "/tmp/fm-$ID" ] || { echo "error: direct account recovery metadata has an invalid tasktmp for $ID" >&2; exit 1; } + [ "$(fm_tasktmp_path "$ID" "$RECORDED_GENERATION" 2>/dev/null)" = "$RECORDED_TASKTMP" ] \ + || { echo "error: direct account recovery metadata has an invalid tasktmp for $ID" >&2; exit 1; } RECORDED_META_WORKTREE_GIT_REF=$RECORDED_WORKTREE_GIT_REF RECORDED_META_WORKTREE_GIT_HEAD=$RECORDED_WORKTREE_GIT_HEAD RECORDED_META_WORKTREE_GIT_SETUP_REF=$RECORDED_WORKTREE_GIT_SETUP_REF @@ -2348,6 +2440,14 @@ elif [ "$ACCOUNT_EFFECTIVE_MODE" != off ]; then else SPAWN_GENERATION_ID="spawn:$(fm_account_attempt_id "$FM_HOME" "$ID")" || exit 1 fi +if [ "$DIRECT_ACCOUNT_RECOVERY" = 1 ] || [ "$RESUME_ACCOUNT" = 1 ]; then + TASK_TMP=$(fm_meta_get "$RESUME_META" tasktmp) +else + TASK_TMP=$(fm_tasktmp_path "$ID" "$SPAWN_GENERATION_ID") || { + echo "error: cannot derive task temp root for $ID generation $SPAWN_GENERATION_ID" >&2 + exit 1 + } +fi if [ "$ACCOUNT_EFFECTIVE_MODE" = enforce ]; then META_WRITE_LOCK=$(fm_account_meta_lock_acquire "$STATE" "$ID") || exit 1 if [ "$RECOVERY_ACCOUNT" = 1 ]; then @@ -2634,6 +2734,23 @@ else fi fi +# Per-project delivery mode + yolo flag (bin/fm-project-mode.sh; AGENTS.md +# project management and task lifecycle). Resolve it before Treehouse +# acquisition so the crash-recovery record carries exact teardown authority +# even if spawn dies before endpoint creation or metadata installation. +if [ "$DIRECT_ACCOUNT_RECOVERY" = 1 ]; then + MODE=$RECORDED_MODE + YOLO=$RECORDED_YOLO +elif [ "$KIND" = secondmate ]; then + MODE=secondmate + YOLO=off +else + PROJ_NAME=$(basename "$PROJ_ABS") + read -r MODE YOLO </dev/null) @@ -2701,7 +2818,7 @@ if [ "$KIND" = secondmate ]; then fi if [ "$ACCOUNT_EFFECTIVE_MODE" = enforce ]; then if ! secondmate_home_supports_account_routing "$PROJ_ABS"; then - echo "error: refusing account-routed secondmate launch for $PROJ_ABS: the home lacks Agent Fleet routing support. Fast-forward or otherwise reconcile the home to this Firstmate revision, run bin/fm-config-push.sh, and retry." >&2 + echo "error: refusing account-routed secondmate $ID launch for $PROJ_ABS: the home lacks Agent Fleet routing support. Fast-forward or otherwise reconcile the home to this Firstmate revision, run bin/fm-config-push.sh, and retry." >&2 exit 1 fi elif ! secondmate_home_supports_account_routing "$PROJ_ABS"; then @@ -2751,7 +2868,7 @@ real_path_or_raw() { # # and resets the selected clean pool worktree from that remote-tracking ref. # The post-acquisition verification below is the fail-closed freshness proof. if [ "$KIND" != secondmate ] && [ "$BACKEND" != orca ] && [ "$RECOVERY_ACCOUNT" != 1 ]; then - if CHECKOUT_PREFLIGHT_OUT=$("$SCRIPT_DIR/fm-checkout-refresh.sh" preflight "$PROJ_ABS" 2>&1); then + if CHECKOUT_PREFLIGHT_OUT=$("$SCRIPT_DIR/fm-checkout-refresh.sh" preflight "$PROJ_ABS_REAL" 2>&1); then CHECKOUT_PREFLIGHT_STATUS=0 else CHECKOUT_PREFLIGHT_STATUS=$? @@ -2839,12 +2956,13 @@ if [ "$DIRECT_ACCOUNT_RECOVERY" = 1 ]; then fi if [ "$KIND" != secondmate ] && [ "$BACKEND" != orca ] && [ "$RECOVERY_ACCOUNT" != 1 ]; then - "$SCRIPT_DIR/fm-checkout-refresh.sh" pool-preflight "$PROJ_ABS" || { + "$SCRIPT_DIR/fm-checkout-refresh.sh" pool-preflight "$PROJ_ABS_REAL" || { echo "error: refusing Treehouse acquisition because pool safety could not be inspected for $PROJ_ABS" >&2 exit 1 } + create_worktree_acquisition_record || exit 1 acquire_status=0 - WT=$("$SCRIPT_DIR/fm-checkout-refresh.sh" acquire-worktree "$PROJ_ABS" "firstmate-$ID") || acquire_status=$? + WT=$("$SCRIPT_DIR/fm-checkout-refresh.sh" acquire-worktree "$PROJ_ABS_REAL" "firstmate-$ID") || acquire_status=$? if [ "$acquire_status" -ne 0 ]; then if [ "$acquire_status" -eq 124 ]; then echo "error: refusing to spawn $ID after the bounded Treehouse acquisition timed out" >&2 @@ -2859,9 +2977,10 @@ if [ "$KIND" != secondmate ] && [ "$BACKEND" != orca ] && [ "$RECOVERY_ACCOUNT" } WORKTREE_CREATED=1 WORKTREE_RETAIN_ON_ABORT=1 + record_acquired_worktree || exit 1 validate_spawn_worktree "treehouse get --lease" "$PROJ_ABS" freshness_status=0 - "$SCRIPT_DIR/fm-checkout-refresh.sh" verify-worktree "$WT" "$PROJ_ABS" || freshness_status=$? + "$SCRIPT_DIR/fm-checkout-refresh.sh" verify-worktree "$WT" "$PROJ_ABS_REAL" || freshness_status=$? if [ "$freshness_status" -ne 0 ]; then echo "error: refusing to launch fm-$ID from a leased worktree whose repository identity, cleanliness, or default-tip freshness could not be proved" >&2 exit 1 @@ -2981,13 +3100,24 @@ if [ "$DIRECT_ACCOUNT_ROUTING" = 1 ] && [ "$DIRECT_ACCOUNT_RECOVERY" = 0 ] && [ } fi -# Per-task temp root: /tmp/fm-/ with Go's build temp nested at gotmp/. Go won't -# create GOTMPDIR, so mkdir before it is used; fm-teardown removes the whole root. -# Nested (not a bare /tmp/fm-/gotmp) so other per-task temp can live alongside -# later, and teardown cleans one deterministic path. GOTMPDIR (not TMPDIR) is the +# Per-generation temp root with Go's build temp nested at gotmp/. Go won't +# create GOTMPDIR, so create the exact owned root before it is used. +# GOTMPDIR (not TMPDIR) is the # targeted knob: TMPDIR is too broad (affects every program's temp, not just Go's). -TASK_TMP="/tmp/fm-$ID" -mkdir -p "$TASK_TMP/gotmp" +if [ "$DIRECT_ACCOUNT_RECOVERY" = 1 ] || [ "$RESUME_ACCOUNT" = 1 ]; then + if ! fm_tasktmp_owner_validate "$(fm_tasktmp_owner_record "$STATE" "$ID" "$SPAWN_GENERATION_ID")" "$FM_HOME" "$ID" "$SPAWN_GENERATION_ID" "$TASK_TMP" \ + || ! fm_tasktmp_owner_validate "$TASK_TMP/.fm-tasktmp-owner" "$FM_HOME" "$ID" "$SPAWN_GENERATION_ID" "$TASK_TMP"; then + echo "error: account recovery task temp ownership is not exact for $ID" >&2 + exit 1 + fi +else + if spawn_test_lab_enabled && [ "${FM_TEST_TASKTMP_CREATE_FAIL:-0}" = 1 ]; then + echo "error: test-only task temp creation failure for $ID" >&2 + exit 1 + fi + fm_tasktmp_create "$STATE" "$FM_HOME" "$ID" "$SPAWN_GENERATION_ID" "$TASK_TMP" \ + || { echo "error: cannot create exact owned task temp root for $ID" >&2; exit 1; } +fi # herdr sets GOTMPDIR natively at agent start. Every other backend exports it into # the pane shell just before the launch line, further down. CREW_PATH rides the same # two channels for the same reason. @@ -3103,23 +3233,6 @@ EOF esac fi -# Per-project delivery mode + yolo flag (bin/fm-project-mode.sh; AGENTS.md project management and task lifecycle). -# Recorded in meta so fm-teardown's safety check and the validate/merge stages can -# branch on them. Mode governs ship tasks; a scout's deliverable is a report, not a -# merge, so scout teardown ignores mode. -if [ "$DIRECT_ACCOUNT_RECOVERY" = 1 ]; then - MODE=$RECORDED_MODE - YOLO=$RECORDED_YOLO -elif [ "$KIND" = secondmate ]; then - MODE=secondmate - YOLO=off -else - PROJ_NAME=$(basename "$PROJ_ABS") - read -r MODE YOLO <&2 + exit 1 +fi if [ "$ACCOUNT_EFFECTIVE_MODE" = enforce ]; then persist_failed_account_rollback_short || exit 1 fi @@ -3646,6 +3763,7 @@ fi fm_account_safe_file_destination "$STATE/$ID.meta" || { echo "error: unsafe task metadata destination at $STATE/$ID.meta" >&2; exit 1; } mv "$META_TMP" "$STATE/$ID.meta" META_INSTALLED=1 +clear_worktree_acquisition_record [ -z "$META_WRITE_LOCK" ] || fm_account_meta_lock_release "$META_WRITE_LOCK" META_WRITE_LOCK= [ "$BACKEND" = orca ] && ORCA_ABORT_CLEANUP=0 diff --git a/bin/fm-teardown.sh b/bin/fm-teardown.sh index ea03eda10b..f04771db3e 100755 --- a/bin/fm-teardown.sh +++ b/bin/fm-teardown.sh @@ -114,6 +114,8 @@ CHECKOUT_LOCK_ROOT=$(fm_checkout_lock_root "$CHECKOUT_STATE_BASE") fm_refuse_if_gate_agent # shellcheck source=bin/fm-account-routing-lib.sh . "$SCRIPT_DIR/fm-account-routing-lib.sh" +# shellcheck source=bin/fm-treehouse-lib.sh +. "$SCRIPT_DIR/fm-treehouse-lib.sh" FM_LOCK_LOG_PREFIX=teardown "$FM_ROOT/bin/fm-guard.sh" || true TEARDOWN_UPSTREAM_TIMEOUT=${FM_CHECKOUT_REFRESH_PROBE_TIMEOUT:-15} @@ -202,13 +204,9 @@ if [ "$BACKEND" = orca ]; then fi HOME_PATH=$(grep '^home=' "$META" | cut -d= -f2- || true) PR_URL=$(grep '^pr=' "$META" | tail -1 | cut -d= -f2- || true) -# tasktmp is recorded by fm-spawn for tasks that set up a per-task temp root -# (/tmp/fm-/); absent for tasks spawned before that change, so tolerate empty. +# tasktmp is recorded by fm-spawn for tasks that set up a per-generation temp root. TASK_TMP=$(grep '^tasktmp=' "$META" | cut -d= -f2- || true) -if [ -n "$TASK_TMP" ] && [ "$TASK_TMP" != "/tmp/fm-$ID" ]; then - echo "REFUSED: unsafe task temp path in metadata for $ID: $TASK_TMP" >&2 - exit 1 -fi +TASK_GENERATION=$(fm_meta_get "$META" generation_id) ORCA_WORKTREE_ID=$(fm_meta_get "$META" orca_worktree_id) ORCA_PATH_MATCH_VERIFIED=0 DIRECT_SPAWN_CLEANUP=$(fm_meta_get "$META" direct_spawn_cleanup) @@ -400,7 +398,7 @@ quiesce_secondmate_endpoint() { quiesce_child_endpoint() { local meta=$1 task=$2 owner_home=$3 child_home=${4:-} - local backend target kind endpoint_home probe_home='' endpoint_status scoped_target + local backend target kind endpoint_home probe_home='' endpoint_status scoped_target zellij_tab backend=$(fm_backend_of_meta "$meta") target=$(teardown_backend_target_of_meta "$meta") kind=$(meta_value "$meta" kind) @@ -409,6 +407,21 @@ quiesce_child_endpoint() { [ "$endpoint_home" = "$FM_HOME" ] || probe_home=$endpoint_home scoped_target=$(meta_value "$meta" tmux_session_target) [ "$backend" != orca ] || scoped_target=$(meta_value "$meta" orca_worktree_id) + zellij_tab=$(meta_value "$meta" zellij_tab_id) + if [ "$backend" = zellij ] && [ -n "$zellij_tab" ]; then + if [ -n "$probe_home" ]; then + ( unset FM_ROOT_OVERRIDE; FM_HOME="$probe_home" FM_ROOT="$probe_home" \ + fm_backend_kill "$backend" "$target" "$zellij_tab" "fm-$task" "$scoped_target" ) 2>/dev/null || { + echo "error: failed to stop child endpoint for $task; refusing destructive cleanup" >&2 + return 1 + } + else + fm_backend_kill "$backend" "$target" "$zellij_tab" "fm-$task" "$scoped_target" 2>/dev/null || { + echo "error: failed to stop child endpoint for $task; refusing destructive cleanup" >&2 + return 1 + } + fi + fi if [ "$backend" = orca ]; then quiesce_authoritative_orca_endpoint "$target" "$scoped_target" "fm-$task" || { echo "error: child Orca endpoint authority or quiescence is unproven for $task" >&2 @@ -978,62 +991,6 @@ exact_git_worktree_root() { printf '%s\n' "$canonical" } -treehouse_state_for_worktree() { - local worktree=$1 slot pool state - slot=$(canonical_existing_dir "$(dirname "$worktree")") || return 1 - pool=$(canonical_existing_dir "$(dirname "$slot")") || return 1 - state="$pool/treehouse-state.json" - [ -f "$state" ] && [ ! -L "$state" ] || return 1 - printf '%s\n' "$state" -} - -require_treehouse_task_lease() { - local worktree=$1 expected_holder=$2 state - state=$(treehouse_state_for_worktree "$worktree") || { - echo "error: cannot resolve authoritative Treehouse state for $worktree" >&2 - return 1 - } - python3 - "$state" "$worktree" "$expected_holder" <<'PY' -import json -import os -import sys - -state_path, expected_path, expected_holder = sys.argv[1:] -try: - with open(state_path, encoding="utf-8") as stream: - state = json.load(stream) - worktrees = state["worktrees"] - if not isinstance(worktrees, list): - raise TypeError("worktrees must be an array") - matches = [] - for entry in worktrees: - if not isinstance(entry, dict): - continue - path = entry.get("path") - if not isinstance(path, str) or not path: - continue - if os.path.realpath(path) == expected_path: - matches.append(entry) - if len(matches) != 1: - raise ValueError("expected exactly one matching worktree entry") - entry = matches[0] - if entry.get("leased") is not True: - raise ValueError("worktree is not durably leased") - if entry.get("lease_holder") != expected_holder: - raise ValueError( - f"lease holder is {entry.get('lease_holder')!r}, expected {expected_holder!r}" - ) - if entry.get("destroying") is True: - raise ValueError("worktree is already being destroyed") -except (OSError, ValueError, TypeError, KeyError, json.JSONDecodeError) as error: - print( - f"error: Treehouse ownership for {expected_path} is unprovable: {error}", - file=sys.stderr, - ) - raise SystemExit(1) -PY -} - require_treehouse_return_authority() { local worktree=$1 project=$2 worktree_root project_root worktree_common project_common worktree_root=$(exact_git_worktree_root "$worktree") || return 1 @@ -1048,7 +1005,7 @@ require_treehouse_return_authority() { echo "error: Treehouse return target $worktree_root is not registered to $project_root" >&2 return 1 } - require_treehouse_task_lease "$worktree_root" "$3" + fm_treehouse_require_task_lease "$worktree_root" "$3" } validate_teardown_target_identity() { @@ -1059,6 +1016,14 @@ validate_teardown_target_identity() { echo "error: teardown project metadata is not an exact inspectable repository root: ${PROJ:-}" >&2 return 1 } + # A legacy scout can outlive its scratch worktree while its exact runtime + # endpoint remains. There is no worktree mutation left to authorize in that + # case; retain the project-root proof and let the backend's endpoint identity + # checks decide whether the recorded terminal can be quiesced. + if [ "$KIND" = scout ] && [ "$BACKEND" != orca ] \ + && [ ! -e "$WT" ] && [ ! -L "$WT" ]; then + return 0 + fi worktree_root=$(exact_git_worktree_root "$WT") || { echo "error: teardown worktree metadata is not an exact inspectable repository root: ${WT:-}" >&2 return 1 @@ -1083,7 +1048,7 @@ validate_teardown_target_identity() { echo "error: teardown worktree is not registered to the recorded project: $worktree_root" >&2 return 1 } - require_treehouse_task_lease "$worktree_root" "firstmate-$ID" + fm_treehouse_require_task_lease "$worktree_root" "firstmate-$ID" } retry_wait_secs_is_valid() { @@ -1826,9 +1791,21 @@ safe_rm_rf_child_worktree() { } safe_remove_task_tmp() { - local target=$1 base + local target=$1 base owner_record [ -n "$target" ] || return 0 - [ "$target" = "/tmp/fm-$ID" ] || return 1 + owner_record=$(fm_tasktmp_owner_record "$STATE" "$ID" "$TASK_GENERATION") || return 1 + if ! fm_tasktmp_owner_validate "$owner_record" "$FM_HOME" "$ID" "$TASK_GENERATION" "$target"; then + echo "REFUSED: task temp ownership is missing, malformed, or ambiguous for $ID: $target" >&2 + return 1 + fi + if [ ! -e "$target" ] && [ ! -L "$target" ]; then + rm -f "$owner_record" + return 0 + fi + if ! fm_tasktmp_owner_validate "$target/.fm-tasktmp-owner" "$FM_HOME" "$ID" "$TASK_GENERATION" "$target"; then + echo "REFUSED: task temp root proof is missing, malformed, or forged for $ID: $target" >&2 + return 1 + fi base=$(python3 - <<'PY' import os import stat @@ -1849,7 +1826,8 @@ if not stat.S_ISDIR(os.lstat(base).st_mode): print(base) PY ) || return 1 - removal_tree_operation "$base/fm-$ID" "task temp root" remove + removal_tree_operation "$base/${target##*/}" "task temp root" remove || return 1 + rm -f "$owner_record" } remove_worktree_compatibility_artifacts() { @@ -3398,6 +3376,7 @@ EOF validate_firstmate_home_for_removal() { local home=$1 label=$2 expected_id=${3:-} expected_source=${4:-$FM_ROOT} expected_registry=${5:-} expected_project local abs_home_path metadata_home_root marker_id source_authority=${7:-1} + local project_authority=${8:-1} expected_project=${6:-$home} [ -n "$home" ] && [ -e "$home" ] || { echo "REFUSED: missing $label removal target ${home:-}" >&2 @@ -3432,10 +3411,10 @@ validate_firstmate_home_for_removal() { "secondmate top-level source repository" "$abs_home_path" || return 1 fi if [ -n "$expected_id" ] && firstmate_home_has_treehouse_slot "$abs_home_path" "$expected_source"; then - require_treehouse_task_lease "$abs_home_path" "$expected_id" || return 1 + fm_treehouse_require_task_lease "$abs_home_path" "$expected_id" || return 1 fi validate_secondmate_home_landed_state "$abs_home_path" "$expected_source" || return 1 - if [ -n "$expected_id" ]; then + if [ -n "$expected_id" ] && [ "$project_authority" -eq 1 ]; then validate_secondmate_project_clones \ "$abs_home_path" "$expected_registry" "$expected_id" "$expected_source" || return 1 fi @@ -3567,7 +3546,7 @@ validate_firstmate_home_children_removal() { elif [ -n "$child_wt" ] && [ -d "$child_wt" ]; then child_proj=$(meta_value "$child_meta" project) validate_child_worktree_for_removal "$child_wt" "$child_proj" >/dev/null || return 1 - require_treehouse_task_lease "$(canonical_existing_dir "$child_wt")" "firstmate-$child_id" || return 1 + fm_treehouse_require_task_lease "$(canonical_existing_dir "$child_wt")" "firstmate-$child_id" || return 1 validate_child_worktree_landed_state "$child_meta" "$child_id" "$child_wt" "$child_proj" || return 1 else echo "error: retained child metadata for $child_id because its Treehouse worktree is missing or uninspectable" >&2 @@ -3659,7 +3638,7 @@ cleanup_firstmate_home_children() { return 1 fi validate_child_worktree_for_removal "$child_wt" "$child_proj" >/dev/null || return 1 - require_treehouse_task_lease "$(canonical_existing_dir "$child_wt")" "firstmate-$child_id" || return 1 + fm_treehouse_require_task_lease "$(canonical_existing_dir "$child_wt")" "firstmate-$child_id" || return 1 fi if managed_account_meta "$child_meta"; then child_endpoint_home=$(fm_backend_endpoint_home "$child_backend" "$child_kind" "$home" "$child_home") @@ -3900,7 +3879,7 @@ if [ "$KIND" = secondmate ]; then exit 1 } validate_firstmate_home_for_removal \ - "$HOME_PATH" "secondmate home" "$ID" "$FM_ROOT" "$SECONDMATE_REG" "$PROJ" 0 \ + "$HOME_PATH" "secondmate home" "$ID" "$FM_ROOT" "$SECONDMATE_REG" "$PROJ" 0 0 \ >/dev/null || exit 1 if [ "$FORCE" = "--force" ]; then validate_firstmate_home_children_removal "$HOME_PATH" || exit 1 @@ -3944,6 +3923,12 @@ quiesce_task_endpoint() { zellij_tab=$(meta_value "$META" zellij_tab_id) scoped_target=$(meta_value "$META" tmux_session_target) [ "$BACKEND" != orca ] || scoped_target=$ORCA_WORKTREE_ID + if [ "$BACKEND" = zellij ] && [ -n "$zellij_tab" ]; then + fm_backend_kill "$BACKEND" "$T" "$zellij_tab" "fm-$ID" "$scoped_target" 2>/dev/null || { + echo "error: failed to stop task endpoint for $ID; retaining metadata" >&2 + return 1 + } + fi if [ "$BACKEND" = orca ]; then quiesce_authoritative_orca_endpoint "$T" "$ORCA_WORKTREE_ID" "fm-$ID" || { echo "error: task Orca endpoint authority or quiescence is unproven for $ID; retaining metadata" >&2 @@ -4182,7 +4167,7 @@ EOF fi remove_grok_turnend_auth "$STATE" "$ID" fm_backend_clear_transition "$BACKEND" "$STATE" "$T" || true -# Remove the per-task temp root (/tmp/fm-/, incl. its gotmp/) recorded by spawn. +# Remove the exact owned per-generation temp root recorded by spawn. # Read before the state-file rm below; empty (pre-fix tasks without tasktmp=) is a no-op. [ -z "$TASK_TMP" ] || safe_remove_task_tmp "$TASK_TMP" || exit 1 rm -f "$STATE/$ID.status" "$STATE/$ID.turn-ended" "$STATE/$ID.check.sh" "$STATE/$ID.meta" "$STATE/$ID.pi-ext.ts" "$STATE/$ID.grok-turnend-token" diff --git a/bin/fm-treehouse-lib.sh b/bin/fm-treehouse-lib.sh new file mode 100755 index 0000000000..d0afa4e698 --- /dev/null +++ b/bin/fm-treehouse-lib.sh @@ -0,0 +1,173 @@ +#!/usr/bin/env bash +# Shared Treehouse lease authority helpers. +# +# Consumers may prove that one exact worktree is durably leased to one exact +# holder, or resolve the unique leased worktree for an exact project and holder. +# The latter is intentionally narrow: it walks only Git's registered worktrees +# for the trusted project and refuses zero or multiple matches. + +fm_treehouse_state_for_worktree() { # + local worktree=$1 slot pool state + worktree=$(fm_checkout_trusted_dir "$worktree") || return 1 + slot=$(fm_checkout_trusted_dir "$(dirname "$worktree")") || return 1 + pool=$(fm_checkout_trusted_dir "$(dirname "$slot")") || return 1 + state="$pool/treehouse-state.json" + [ -f "$state" ] && [ ! -L "$state" ] || return 1 + printf '%s\n' "$state" +} + +fm_treehouse_require_task_lease() { # + local worktree=$1 expected_holder=$2 state + worktree=$(fm_checkout_trusted_dir "$worktree") || { + echo "error: Treehouse worktree is unavailable or redirected: $1" >&2 + return 1 + } + state=$(fm_treehouse_state_for_worktree "$worktree") || { + echo "error: cannot resolve authoritative Treehouse state for $worktree" >&2 + return 1 + } + python3 - "$state" "$worktree" "$expected_holder" <<'PY' +import json +import os +import sys + +state_path, expected_path, expected_holder = sys.argv[1:] +try: + with open(state_path, encoding="utf-8") as stream: + state = json.load(stream) + worktrees = state["worktrees"] + if not isinstance(worktrees, list): + raise TypeError("worktrees must be an array") + matches = [] + for entry in worktrees: + if not isinstance(entry, dict): + continue + path = entry.get("path") + if not isinstance(path, str) or not path: + continue + if os.path.realpath(path) == expected_path: + matches.append(entry) + if len(matches) != 1: + raise ValueError("expected exactly one matching worktree entry") + entry = matches[0] + if entry.get("leased") is not True: + raise ValueError("worktree is not durably leased") + if entry.get("lease_holder") != expected_holder: + raise ValueError( + f"lease holder is {entry.get('lease_holder')!r}, expected {expected_holder!r}" + ) + if entry.get("destroying") is True: + raise ValueError("worktree is already being destroyed") +except (OSError, ValueError, TypeError, KeyError, json.JSONDecodeError) as error: + print( + f"error: Treehouse ownership for {expected_path} is unprovable: {error}", + file=sys.stderr, + ) + raise SystemExit(1) +PY +} + +fm_treehouse_find_task_lease() { # + local project=$1 expected_holder=$2 listed line candidate common project_common + local matches=0 match= + project=$(fm_checkout_trusted_dir "$project") || return 1 + [ "$(git -C "$project" rev-parse --show-toplevel 2>/dev/null)" = "$project" ] || return 1 + project_common=$(fm_checkout_git_common_dir "$project") || return 1 + listed=$(git -C "$project" -c core.quotePath=false worktree list --porcelain 2>/dev/null) || return 1 + while IFS= read -r line; do + case "$line" in + worktree\ *) + candidate=$(fm_checkout_trusted_dir "${line#worktree }" 2>/dev/null || true) + [ -n "$candidate" ] || continue + [ "$candidate" != "$project" ] || continue + common=$(fm_checkout_git_common_dir "$candidate" 2>/dev/null || true) + [ "$common" = "$project_common" ] || continue + if fm_treehouse_require_task_lease "$candidate" "$expected_holder" >/dev/null 2>&1; then + matches=$((matches + 1)) + match=$candidate + fi + ;; + esac + done <&2 + return 2 + ;; + *) + echo "error: multiple Treehouse leases exist for $expected_holder in $project" >&2 + return 3 + ;; + esac +} + +fm_treehouse_prove_task_lease_absent() { # + local recorded_worktree=$1 expected_holder=$2 slot pool state + [ -n "$recorded_worktree" ] || return 1 + slot=$(dirname "$recorded_worktree") + pool=$(fm_checkout_trusted_dir "$(dirname "$slot")") || return 1 + state="$pool/treehouse-state.json" + [ -f "$state" ] && [ ! -L "$state" ] || return 1 + python3 - "$state" "$expected_holder" <<'PY' +import json +import os +import sys + +state_path, expected_holder = sys.argv[1:] +try: + with open(state_path, encoding="utf-8") as stream: + state = json.load(stream) +except OSError as error: + print( + f"error: Treehouse lease absence for {expected_holder} is unprovable: {error}", + file=sys.stderr, + ) + raise SystemExit(1) +except json.JSONDecodeError as error: + print( + f"error: authoritative Treehouse lease state is corrupt: {error}", + file=sys.stderr, + ) + raise SystemExit(2) + +try: + worktrees = state["worktrees"] + if not isinstance(worktrees, list): + raise TypeError("worktrees must be an array") + for entry in worktrees: + if not isinstance(entry, dict): + raise TypeError("worktree entry must be an object") + leased = entry.get("leased") + holder = entry.get("lease_holder") + if not isinstance(leased, bool): + raise TypeError("leased must be a boolean") + if leased: + path = entry.get("path") + if ( + not isinstance(path, str) + or not path.strip() + or "\0" in path + or not os.path.isabs(path) + ): + raise TypeError("leased worktree path must be a non-empty absolute string") + if not isinstance(holder, str) or not holder.strip(): + raise TypeError("leased worktree holder must be a non-empty string") + if holder == expected_holder: + raise ValueError("matching Treehouse lease still exists") +except ValueError as error: + print( + f"error: Treehouse lease absence for {expected_holder} is unprovable: {error}", + file=sys.stderr, + ) + raise SystemExit(1) +except (TypeError, KeyError) as error: + print( + f"error: authoritative Treehouse lease state is corrupt: {error}", + file=sys.stderr, + ) + raise SystemExit(2) +PY +} diff --git a/bin/fm-watch.sh b/bin/fm-watch.sh index 9b4495edb6..1b41bb4cc8 100755 --- a/bin/fm-watch.sh +++ b/bin/fm-watch.sh @@ -37,6 +37,10 @@ # FM_PERMISSION_STALL_ESCALATE_SECS as a possible macOS # permission/system-dialog block. Unless afk is active. # check: