diff --git a/.agents/skills/bootstrap-diagnostics/SKILL.md b/.agents/skills/bootstrap-diagnostics/SKILL.md index 584ec6022d..37c3ca46e1 100644 --- a/.agents/skills/bootstrap-diagnostics/SKILL.md +++ b/.agents/skills/bootstrap-diagnostics/SKILL.md @@ -37,7 +37,7 @@ The inline rules in `AGENTS.md` section 3 still bind: detect, then consent, then - `FLEET_SYNC: : STUCK: on , N commits behind - needs attention` - the clone is dirty, on a non-default branch, detached with unique commits, or diverged, so the sync left it untouched (never forcing or discarding); it will keep falling behind until you look. A loud STUCK, especially a growing N across bootstraps, means that clone needs hands-on attention; dispatch a crewmate or resolve it before it strands work. - `SECONDMATE_SYNC: secondmate : skipped: ` - the local-HEAD secondmate sync left a live secondmate home on its existing checkout because the home was dirty, diverged, unsafe, on the wrong branch, missing the primary target commit, or otherwise not fast-forwardable, or because inheritable-config propagation failed; bootstrap continued, but inspect the reason because the secondmate's tracked instructions or inherited settings may be stale after a primary update. -- `SECONDMATE_LIVENESS: secondmate : ` - the session-start liveness sweep checked a live secondmate's recorded endpoint for a real agent process. +- `SECONDMATE_LIVENESS: secondmate : ` - the session-start liveness sweep checked a live secondmate's recorded endpoint with that backend's classifier; `docs/configuration.md` "Runtime backend" owns the possible verdicts. Treat `already-live`, `respawned`, and `rollback reconciled and respawned` as handled; investigate `skipped` or `respawn failed`, and load `secondmate-provisioning` for any `respawn deferred` outcome because its "Recovery" section owns the required routing decision. - `TASKS_AXI: available` - a default-backend capability fact, not a problem; record it silently and use `AGENTS.md` section 10 for backlog mutations. It prints only when `config/backlog-backend` is absent or set to `tasks-axi` and the shared compatibility probe passes (`docs/configuration.md` "Backlog backend"). diff --git a/.agents/skills/secondmate-provisioning/SKILL.md b/.agents/skills/secondmate-provisioning/SKILL.md index 3607a37c33..024c85d25d 100644 --- a/.agents/skills/secondmate-provisioning/SKILL.md +++ b/.agents/skills/secondmate-provisioning/SKILL.md @@ -77,7 +77,7 @@ An explicit `--account-pool` or `--account-profile` overrides it for that spawn, This pool is not inherited into the secondmate home because it governs the primary's secondmate launch, while `config/account-routing-mode` is inherited so the secondmate applies the same off/observe/enforce policy to its own crewmates. This section is the single owner of the secondmate sync and inheritable-config propagation contract; `AGENTS.md` sections 3 and 4 point here. -Before launch, `fm-spawn.sh --secondmate` locally fast-forwards the home to the primary firstmate checkout's current default-branch commit when it is safe; dirty, diverged, or in-flight homes launch unchanged with a warning. +Before launch, `fm-spawn.sh --secondmate` locally fast-forwards the home to the primary firstmate checkout's current default-branch commit when it is safe; other dirty, diverged, or in-flight homes launch unchanged with a warning, subject to the account-routing compatibility refusal owned by `docs/configuration.md` "Agent Fleet account routing." The locked session-start bootstrap sweep runs the same guarded fast-forward for every live secondmate home, discovered from `state/.meta` records with `kind=secondmate` (`data/secondmates.md` only backfills `home=` for older records). That no-fetch path is a purely local fast-forward of tracked files, never an origin fetch, and it never touches the gitignored operational dirs, so a secondmate's backlog, projects, and in-flight work are never disturbed; a linked worktree advances immediately, while a standalone clone that lacks the target receives firstmate updates through `/updatefirstmate`'s origin refresh. The same launch and the same locked bootstrap sweep also propagate the primary's declared inheritable local config, currently `config/crew-dispatch.json`, `config/crew-harness`, `config/backlog-backend`, and `config/account-routing-mode`, into the secondmate home's `config/`. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 652e768c25..96e89ac78b 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -27,9 +27,9 @@ jobs: tests: name: Behavior tests runs-on: ubuntu-latest - # The pre-cutover suite reached 25m35s, and the exhaustive Bridge crash - # matrix adds bounded work. Keep enough margin without masking a real hang. - timeout-minutes: 45 + # The expanded suite was still making forward progress when the prior 45m + # budget expired. Allow measured headroom while retaining a finite hang cap. + timeout-minutes: 90 steps: - uses: actions/checkout@v6 with: diff --git a/AGENTS.md b/AGENTS.md index eeed517dc9..1116188158 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -146,7 +146,7 @@ It composes today's `fm-lock.sh`, `fm-bootstrap.sh`, and `fm-wake-drain.sh` - ca 2. **Bootstrap** - detect-only diagnostics (tool/version problems, GitHub auth, the worktree-tangle check, harness override, dispatch-profile validation, backlog-backend status) always run and always print. When the lock could not be acquired, the worktree-tangle check uses read-only advisory wording without a checkout repair command. The five MUTATING sweeps - report-retention ownership, fleet sync, the local secondmate fast-forward sweep, the secondmate liveness sweep, and X-mode artifact writes - run only when this session actually holds the lock from step 1. - The secondmate liveness sweep probes each live secondmate's endpoint for a real agent process, reports handled or deferred outcomes as `SECONDMATE_LIVENESS:` lines, and leaves any required routing decision to `secondmate-provisioning` (`bin/fm-bootstrap.sh`; `bin/fm-backend.sh`'s `fm_backend_agent_alive`). + The secondmate liveness sweep probes each live secondmate's endpoint with the backend-specific classifier owned by `docs/configuration.md` "Runtime backend", reports handled or deferred outcomes as `SECONDMATE_LIVENESS:` lines, and leaves any required routing decision to `secondmate-provisioning` (`bin/fm-bootstrap.sh`; `bin/fm-backend.sh`'s `fm_backend_agent_alive`). 3. **Wake queue** - when locked, drains the durable wake queue and prints the records prominently as this turn's first work queue, exactly as `bin/fm-wake-drain.sh` did before; a lapsed watcher chain still surfaces here via the same guard banner. When the lock could not be acquired, the queue is left untouched because another session owns it, and the guard's tangle/watcher-liveness alarms still print in read-only advisory mode without drain, supervision repair, or checkout repair commands. 4. **Context digest** - the full contents of `data/projects.md`, `data/secondmates.md`, `data/captain.md`, and `data/learnings.md`, each clearly delimited. diff --git a/bin/backends/herdr.sh b/bin/backends/herdr.sh index 4045912885..e4e31b9418 100644 --- a/bin/backends/herdr.sh +++ b/bin/backends/herdr.sh @@ -2672,6 +2672,7 @@ fm_backend_herdr_server_reachable_for_readsteer() { # running=$(fm_backend_herdr_cli "$session" status --json 2>/dev/null \ | fm_backend_herdr_control_jq -r '.server.running // false' 2>/dev/null) [ "$running" = true ] || return 1 + fm_backend_herdr_server_certificate_required || return 0 fm_backend_herdr_server_adapter_owned "$session" } diff --git a/bin/backends/zellij.sh b/bin/backends/zellij.sh index 662c6f8342..5f9fb456bf 100644 --- a/bin/backends/zellij.sh +++ b/bin/backends/zellij.sh @@ -399,6 +399,29 @@ fm_backend_zellij_target_ready() { # [expected-label] fm_backend_zellij_pane_exists "$FM_BACKEND_ZELLIJ_SESSION" "$FM_BACKEND_ZELLIJ_PANE" } +fm_backend_zellij_agent_alive() { # + local panes + fm_backend_zellij_parse_target "$1" || { printf 'unknown'; return 0; } + fm_backend_zellij_session_exists "$FM_BACKEND_ZELLIJ_SESSION" || { printf 'dead'; return 0; } + panes=$(fm_backend_zellij_cli "$FM_BACKEND_ZELLIJ_SESSION" action list-panes --json 2>/dev/null) \ + || { printf 'unknown'; return 0; } + printf '%s\n' "$panes" | jq -e ' + type == "array" + and all(.[]; + type == "object" + and has("id") + and .id != null + and (.is_plugin | type) == "boolean" + and (.is_plugin or (has("tab_id") and .tab_id != null))) + ' >/dev/null 2>&1 || { printf 'unknown'; return 0; } + if printf '%s\n' "$panes" | jq -e --arg pane "$FM_BACKEND_ZELLIJ_PANE" \ + 'any(.[]?; .is_plugin == false and (.id | tostring) == $pane)' >/dev/null 2>&1; then + printf 'unknown' + else + printf 'dead' + fi +} + # fm_backend_zellij_current_path: the live pane's cwd, or empty on any error. # Mirrors tmux's pane_current_path poll used for worktree-path discovery after # `treehouse get`. diff --git a/bin/fm-account-routing-lib.sh b/bin/fm-account-routing-lib.sh index fa65dccead..2865f2ef93 100644 --- a/bin/fm-account-routing-lib.sh +++ b/bin/fm-account-routing-lib.sh @@ -236,6 +236,99 @@ fm_account_real_directory() { [ -d "$1" ] && [ ! -L "$1" ] } +fm_account_task_tmp_path() { # + local task=$1 generation=$2 token state + fm_account_valid_id "$task" || return 1 + case "$generation" in + spawn:*|account:*) + token=${generation##*:} + ;; + *) + return 1 + ;; + esac + fm_account_valid_id "$token" || return 1 + if [ -n "${STATE:-}" ]; then + state=$STATE + elif [ -n "${FM_STATE_OVERRIDE:-}" ]; then + state=$FM_STATE_OVERRIDE + elif [ -n "${FM_HOME:-}" ]; then + state=$FM_HOME/state + else + return 1 + fi + fm_account_real_directory "$state" || return 1 + state=$(cd "$state" 2>/dev/null && pwd -P) || return 1 + [ "$state" != / ] || return 1 + printf '%s/.task-tmp/fm-%s-%s\n' "${state%/}" "$task" "$token" +} + +fm_account_previous_task_tmp_path() { # + local task=$1 state + fm_account_valid_id "$task" || return 1 + if [ -n "${STATE:-}" ]; then + state=$STATE + elif [ -n "${FM_STATE_OVERRIDE:-}" ]; then + state=$FM_STATE_OVERRIDE + elif [ -n "${FM_HOME:-}" ]; then + state=$FM_HOME/state + else + return 1 + fi + fm_account_real_directory "$state" || return 1 + state=$(cd "$state" 2>/dev/null && pwd -P) || return 1 + [ "$state" != / ] || return 1 + printf '%s/.task-tmp/fm-%s\n' "${state%/}" "$task" +} + +fm_account_legacy_task_tmp_path() { # + fm_account_valid_id "$1" || return 1 + printf '/tmp/fm-%s\n' "$1" +} + +fm_account_task_tmp_is_current() { # + local expected + expected=$(fm_account_task_tmp_path "$1" "$3") || return 1 + [ "$2" = "$expected" ] +} + +fm_account_task_tmp_is_previous() { # + local previous + previous=$(fm_account_previous_task_tmp_path "$1") || return 1 + [ "$2" = "$previous" ] +} + +fm_account_task_tmp_is_legacy() { # + local legacy + legacy=$(fm_account_legacy_task_tmp_path "$1") || return 1 + [ "$2" = "$legacy" ] +} + +fm_account_task_tmp_is_expected() { # + # Legacy metadata remains recognizable, but callers must use + # fm_account_task_tmp_is_current before any filesystem mutation. + fm_account_task_tmp_is_current "$1" "$2" "$3" \ + || fm_account_task_tmp_is_previous "$1" "$2" \ + || fm_account_task_tmp_is_legacy "$1" "$2" +} + +fm_account_safe_remove_task_tmp() { # [subdirectory] + local task=$1 target=$2 generation=$3 child=${4:-} legacy + [ -n "$target" ] || return 0 + fm_account_task_tmp_is_expected "$task" "$target" "$generation" || return 1 + legacy=$(fm_account_legacy_task_tmp_path "$task") || return 1 + [ "$target" != "$legacy" ] || return 0 + fm_account_task_tmp_is_current "$task" "$target" "$generation" \ + || fm_account_task_tmp_is_previous "$task" "$target" \ + || return 1 + case "$child" in + '') ;; + gotmp) target="$target/$child" ;; + *) return 1 ;; + esac + python3 "$FM_ACCOUNT_ROUTING_LIB_DIR/fm-safe-task-tmp.py" "$target" || return 1 +} + fm_account_safe_file_destination() { [ ! -L "$1" ] && { [ ! -e "$1" ] || [ -f "$1" ]; } } @@ -1195,7 +1288,7 @@ fm_account_meta_value() { # } fm_account_restore_artifacts() { - local state=$1 task=$2 backup_name=$3 tasktmp=${4:-} retain=${5:-0} backup name source + local state=$1 task=$2 backup_name=$3 tasktmp=${4:-} retain=${5:-0} generation=${6:-} backup name source local PATH=$FM_ACCOUNT_SYSTEM_PATH [ -n "$backup_name" ] || return 0 case "$backup_name" in @@ -1213,11 +1306,17 @@ fm_account_restore_artifacts() { fi done if [ -n "$tasktmp" ]; then - [ "$tasktmp" = "/tmp/fm-$task" ] || return 1 - if [ -e "$backup/tasktmp-existed" ]; then - [ -e "$backup/gotmp-existed" ] || fm_account_system_exec "$FM_ACCOUNT_SYSTEM_RM_BIN" -rf "$tasktmp/gotmp" || return 1 + if fm_account_task_tmp_is_current "$task" "$tasktmp" "$generation" \ + || fm_account_task_tmp_is_previous "$task" "$tasktmp"; then + if [ -e "$backup/tasktmp-existed" ]; then + [ -e "$backup/gotmp-existed" ] || fm_account_safe_remove_task_tmp "$task" "$tasktmp" "$generation" gotmp || return 1 + else + fm_account_safe_remove_task_tmp "$task" "$tasktmp" "$generation" || return 1 + fi else - fm_account_system_exec "$FM_ACCOUNT_SYSTEM_RM_BIN" -rf "$tasktmp" || return 1 + # A legacy tasktmp is metadata-only compatibility. Never inspect or + # mutate the shared /tmp path while restoring task-owned artifacts. + fm_account_task_tmp_is_legacy "$task" "$tasktmp" || return 1 fi fi [ "$retain" = 1 ] || fm_account_system_exec "$FM_ACCOUNT_SYSTEM_RM_BIN" -rf "$backup" @@ -1289,7 +1388,7 @@ fm_account_cleanup_rollback() { # if [ "$preserve" != 1 ]; then fm_account_session_remove "$account_task" || return 1 fi - fm_account_restore_artifacts "$(fm_account_system_exec "$FM_ACCOUNT_SYSTEM_DIRNAME_BIN" "$meta")" "$task" "$artifacts_name" "$tasktmp" 1 || return 1 + fm_account_restore_artifacts "$(fm_account_system_exec "$FM_ACCOUNT_SYSTEM_DIRNAME_BIN" "$meta")" "$task" "$artifacts_name" "$tasktmp" 1 "$(fm_account_meta_value "$meta" generation_id)" || return 1 lock=$(fm_account_meta_lock_acquire "$(fm_account_system_exec "$FM_ACCOUNT_SYSTEM_DIRNAME_BIN" "$meta")" "$task") || return 1 if [ ! -f "$meta" ] \ || [ "$(fm_account_meta_value "$meta" account_task)" != "$account_task" ] \ diff --git a/bin/fm-backend.sh b/bin/fm-backend.sh index 582b9802a1..685f3b9264 100644 --- a/bin/fm-backend.sh +++ b/bin/fm-backend.sh @@ -927,8 +927,7 @@ fm_backend_target_state() { # [expected-label] [recorded-sco *) printf 'unknown'; return 0 ;; esac if [ "$pane_record" = null ]; then - if [ -n "$expected_tab_id" ] && printf '%s\n' "$panes" | jq -e --arg tab "$expected_tab_id" \ - 'any(.[]?; .is_plugin == false and (.tab_id | tostring) == $tab)' >/dev/null 2>&1; then + if [ -n "$expected_tab_id" ]; then printf 'present' else printf 'absent' @@ -1035,9 +1034,11 @@ fm_backend_target_state() { # [expected-label] [recorded-sco # Scoped to today's --secondmate-spawn-capable backends with an empirically # verified classifier: tmux (docs/tmux-backend.md "Agent liveness probe") and # herdr (docs/herdr-backend.md "Agent liveness probe reuses the husk -# classifier"). zellij, orca, and cmux report unknown until independently -# verified - future work, not a functional gap for the two backends -# --secondmate spawns actually support today plus tmux's reference path. +# classifier"). Zellij can report dead only when the recorded session or pane +# is structurally absent; a present pane remains unknown. Orca and cmux report +# unknown until independently verified - future work, not a functional gap for +# the two backends --secondmate spawns actually support today plus tmux's +# reference path. # Callers must treat unknown exactly like an unreadable target: NEVER license # an action from it alone - the secondmate-liveness sweep gates a respawn on # `dead` only, precisely so a momentary read glitch can never duplicate a @@ -1048,6 +1049,7 @@ fm_backend_agent_alive() { # [expected-label] [recorded-scop case "$backend" in tmux) fm_backend_tmux_agent_alive "$target" "$expected_label" "${4:-}" ;; herdr) fm_backend_herdr_agent_alive "$target" "$expected_label" ;; + zellij) fm_backend_zellij_agent_alive "$target" ;; *) printf 'unknown' ;; esac } diff --git a/bin/fm-checkout-lock-lib.sh b/bin/fm-checkout-lock-lib.sh index e81b069010..222707ee43 100644 --- a/bin/fm-checkout-lock-lib.sh +++ b/bin/fm-checkout-lock-lib.sh @@ -106,7 +106,7 @@ fm_checkout_git_common_dir() { } fm_checkout_validate_git_metadata() { - local checkout=$1 root metadata absolute_git common listed line listed_root found=0 + local checkout=$1 root metadata absolute_git common top listed line listed_root found=0 root=$(fm_checkout_trusted_dir "$checkout") || return 1 metadata="$root/.git" [ -e "$metadata" ] && [ ! -L "$metadata" ] || return 1 @@ -118,11 +118,16 @@ fm_checkout_validate_git_metadata() { *) common="$root/$common" ;; esac common=$(fm_checkout_trusted_dir "$common") || return 1 + top=$(git -C "$root" rev-parse --show-toplevel 2>/dev/null) || return 1 + top=$(fm_checkout_trusted_dir "$top") || return 1 + [ "$top" = "$root" ] || return 1 if [ -d "$metadata" ]; then [ "$(fm_checkout_trusted_dir "$metadata")" = "$absolute_git" ] || return 1 [ "$absolute_git" = "$common" ] || return 1 elif [ -f "$metadata" ]; then - case "$absolute_git" in "$common"/worktrees/*) ;; *) return 1 ;; esac + if [ "$absolute_git" != "$common" ]; then + case "$absolute_git" in "$common"/worktrees/*) ;; *) return 1 ;; esac + fi else return 1 fi @@ -131,7 +136,15 @@ fm_checkout_validate_git_metadata() { case "$line" in "worktree "*) listed_root=$(fm_checkout_trusted_dir "${line#worktree }" 2>/dev/null) || return 1 - [ "$listed_root" != "$root" ] || found=$((found + 1)) + if [ "$listed_root" = "$root" ]; then + found=$((found + 1)) + elif [ -f "$metadata" ] && [ "$absolute_git" = "$common" ] \ + && [ "$listed_root" = "$common" ]; then + # An absorbed submodule's worktree listing identifies its common + # Git directory rather than the checked-out root. The exact + # show-toplevel proof above binds that Git directory back to root. + found=$((found + 1)) + fi ;; esac done </dev/null || printf 0) + count=$((count + 1)) + printf '%s\n' "$count" > "$FM_CHECKOUT_TEST_REINSPECTION_COUNT" + return 1 + fi + exact_git_root "$checkout" +} + CHECKOUT_REFRESH_AUTHORITY= resolve_checkout_refresh_authority() { local checkout=$1 mode_line mode @@ -2423,9 +2435,10 @@ run_once() { while IFS= read -r checkout; do [ -n "$checkout" ] || continue - if ! reinspected=$(exact_git_root "$checkout") || [ "$reinspected" != "$checkout" ]; then + if ! reinspected=$(reinspect_covered_checkout "$checkout") || [ "$reinspected" != "$checkout" ]; then record_reinspection_failure "$checkout" || status=1 coverage_failed=1 + status=1 continue fi key=$(checkout_key "$checkout") || { @@ -2447,9 +2460,10 @@ run_once() { while IFS= read -r checkout; do [ -n "$checkout" ] || continue - if ! reinspected=$(exact_git_root "$checkout") || [ "$reinspected" != "$checkout" ]; then + if ! reinspected=$(reinspect_covered_checkout "$checkout") || [ "$reinspected" != "$checkout" ]; then record_reinspection_failure "$checkout" || status=1 coverage_failed=1 + status=1 continue fi key=$(checkout_key "$checkout") || { diff --git a/bin/fm-ff-lib.sh b/bin/fm-ff-lib.sh index ae290020ca..001a0edc94 100644 --- a/bin/fm-ff-lib.sh +++ b/bin/fm-ff-lib.sh @@ -56,7 +56,9 @@ default_branch() { primary_head_commit() { local root=$1 default default=$(default_branch "$root") || return 1 - git -C "$root" rev-parse --verify --quiet "refs/heads/$default^{commit}" 2>/dev/null || return 1 + git -C "$root" rev-parse --verify --quiet "refs/heads/$default^{commit}" 2>/dev/null \ + || git -C "$root" rev-parse --verify --quiet "refs/remotes/origin/$default^{commit}" 2>/dev/null \ + || return 1 } resolve_path() { diff --git a/bin/fm-home-seed.sh b/bin/fm-home-seed.sh index eac1c90072..1ffa088f91 100755 --- a/bin/fm-home-seed.sh +++ b/bin/fm-home-seed.sh @@ -431,7 +431,7 @@ acquire_treehouse_home() { } ensure_home() { - local id=$1 requested=$2 home + local id=$1 requested=$2 home default target source_origin if [ "$requested" = "-" ]; then home=$(acquire_treehouse_home "$id") verify_firstmate_home "$home" @@ -443,8 +443,36 @@ ensure_home() { if [ -e "$home" ]; then [ -d "$home" ] || { echo "error: $home exists and is not a directory" >&2; return 1; } else + default=$(git -C "$FM_ROOT" symbolic-ref --quiet --short refs/remotes/origin/HEAD 2>/dev/null || true) + default=${default#origin/} + if [ -z "$default" ]; then + for default in main master; do + git -C "$FM_ROOT" show-ref --verify --quiet "refs/heads/$default" && break + done + fi + target=$(git -C "$FM_ROOT" rev-parse --verify --quiet "refs/remotes/origin/$default^{commit}" 2>/dev/null \ + || git -C "$FM_ROOT" rev-parse --verify --quiet "refs/heads/$default^{commit}") || { + echo "error: cannot resolve the firstmate default branch for $home" >&2 + return 1 + } + source_origin=$(git -C "$FM_ROOT" remote get-url origin 2>/dev/null || true) mkdir -p "$(dirname "$home")" - git clone --quiet "$FM_ROOT" "$home" + git init -q "$home" + git -C "$home" remote add origin "$FM_ROOT" + git -C "$home" fetch -q --no-tags origin "$target" + git -C "$home" checkout --quiet --detach "$target" || return 1 + git -C "$home" for-each-ref --format='delete %(refname)' refs/remotes/origin \ + | git -C "$home" update-ref --stdin || return 1 + if [ -n "$source_origin" ]; then + git -C "$home" remote set-url origin "$source_origin" || return 1 + git -C "$home" config --unset-all remote.origin.fetch || true + git -C "$home" config --add remote.origin.fetch \ + "+refs/heads/$default:refs/remotes/origin/$default" || return 1 + git -C "$home" update-ref "refs/remotes/origin/$default" "$target" || return 1 + git -C "$home" symbolic-ref refs/remotes/origin/HEAD "refs/remotes/origin/$default" || return 1 + else + git -C "$home" remote remove origin || return 1 + fi fi verify_firstmate_home "$home" } @@ -881,8 +909,44 @@ refuse_projectful_projectless_charter() { return 1 } +prepare_seed_charter() { + local id=$1 no_projects=$2 charter_summary charter_scope + shift 2 + if [ ! -f "$SEED_PARENT_BRIEF" ]; then + [ -n "${FM_SECONDMATE_CHARTER:-}" ] || { + echo "error: no filled secondmate charter brief at $SEED_PARENT_BRIEF; set FM_SECONDMATE_CHARTER or scaffold one and replace {TASK}" >&2 + return 1 + } + [ -d "$DATA/$id" ] || SEED_PARENT_BRIEF_DIR_CREATED=1 + if [ "$no_projects" -eq 1 ]; then + "$FM_ROOT/bin/fm-brief.sh" "$id" --secondmate --no-projects + else + "$FM_ROOT/bin/fm-brief.sh" "$id" --secondmate "$@" + fi + SEED_PARENT_BRIEF_CREATED=1 + fi + if grep -F '{TASK}' "$SEED_PARENT_BRIEF" >/dev/null 2>&1; then + echo "error: secondmate charter brief at $SEED_PARENT_BRIEF still contains {TASK}; fill it before seeding" >&2 + return 1 + fi + charter_summary=$(registry_summary_for_brief "$SEED_PARENT_BRIEF") + [ -n "$charter_summary" ] || { + echo "error: secondmate charter brief at $SEED_PARENT_BRIEF has an empty Charter section; fill it before seeding" >&2 + return 1 + } + charter_scope=$(registry_scope_for_brief "$SEED_PARENT_BRIEF") + [ -n "$charter_scope" ] || { + echo "error: secondmate charter brief at $SEED_PARENT_BRIEF has an empty Routing scope section; fill it before seeding" >&2 + return 1 + } + if [ "$no_projects" -eq 1 ]; then + refuse_projectful_projectless_charter "$id" "$SEED_PARENT_BRIEF" || return 1 + fi +} + seed_home() { - local id=$1 requested_home=$2 requested_abs home projects_csv project project_dst charter_summary charter_scope + local id=$1 requested_home=$2 requested_abs home projects_csv project project_dst + local preflight_out local no_projects=0 arg local filtered=() shift 2 @@ -941,12 +1005,18 @@ seed_home() { } fi + if [ "$requested_home" != "-" ]; then + prepare_seed_charter "$id" "$no_projects" "$@" || return 1 + fi + if [ "$requested_home" = "-" ]; then SEED_HOME_ACQUIRED=1 home=$(acquire_treehouse_home "$id") SEED_HOME="$home" SEED_HOME_RETAINED=1 + refuse_active_home_path "$home" || return 1 SEED_HOME_LIFECYCLE_LOCK=$(fm_secondmate_home_lifecycle_lock_acquire "$CHECKOUT_LOCK_ROOT" "$home") || return 1 + prepare_seed_charter "$id" "$no_projects" "$@" || return 1 freshness_status=0 "$SCRIPT_DIR/fm-checkout-refresh.sh" verify-worktree "$home" "$FM_ROOT" || freshness_status=$? if [ "$freshness_status" -ne 0 ]; then @@ -965,8 +1035,11 @@ seed_home() { SEED_HOME="$requested_abs" [ -e "$requested_abs" ] || SEED_HOME_CREATED=1 home=$(ensure_home "$id" "$requested_abs") - "$SCRIPT_DIR/fm-checkout-refresh.sh" preflight "$home" || { - echo "error: refusing explicit secondmate home whose default branch cannot be refreshed safely" >&2 + if [ "$no_projects" -eq 1 ]; then + refuse_populated_projectless_home "$home" || return 1 + fi + preflight_out=$("$SCRIPT_DIR/fm-checkout-refresh.sh" preflight "$home" 2>&1) || { + echo "error: refusing explicit secondmate home whose default branch cannot be refreshed safely: $(printf '%s\n' "$preflight_out" | sed -n '1p')" >&2 return 1 } "$SCRIPT_DIR/fm-checkout-refresh.sh" verify-home "$home" "$FM_ROOT" || { @@ -981,9 +1054,6 @@ seed_home() { validate_seed_leaf_files "$home" || return 1 if [ "$no_projects" -eq 1 ]; then refuse_populated_projectless_home "$home" || return 1 - if [ -f "$SEED_PARENT_BRIEF" ]; then - refuse_projectful_projectless_charter "$id" "$SEED_PARENT_BRIEF" || return 1 - fi fi mkdir -p "$DATA" "$home/data" "$home/state" "$home/config" "$home/projects" if [ -f "$home/data/projects.md" ]; then @@ -1000,34 +1070,6 @@ seed_home() { fi SEED_HOME_BACKED_UP=1 - if [ ! -f "$SEED_PARENT_BRIEF" ]; then - [ -n "${FM_SECONDMATE_CHARTER:-}" ] || { - echo "error: no filled secondmate charter brief at $SEED_PARENT_BRIEF; set FM_SECONDMATE_CHARTER or scaffold one and replace {TASK}" >&2 - return 1 - } - [ -d "$DATA/$id" ] || SEED_PARENT_BRIEF_DIR_CREATED=1 - if [ "$no_projects" -eq 1 ]; then - "$FM_ROOT/bin/fm-brief.sh" "$id" --secondmate --no-projects - else - "$FM_ROOT/bin/fm-brief.sh" "$id" --secondmate "$@" - fi - SEED_PARENT_BRIEF_CREATED=1 - fi - if grep -F '{TASK}' "$SEED_PARENT_BRIEF" >/dev/null 2>&1; then - echo "error: secondmate charter brief at $SEED_PARENT_BRIEF still contains {TASK}; fill it before seeding" >&2 - return 1 - fi - charter_summary=$(registry_summary_for_brief "$SEED_PARENT_BRIEF") - [ -n "$charter_summary" ] || { - echo "error: secondmate charter brief at $SEED_PARENT_BRIEF has an empty Charter section; fill it before seeding" >&2 - return 1 - } - charter_scope=$(registry_scope_for_brief "$SEED_PARENT_BRIEF") - [ -n "$charter_scope" ] || { - echo "error: secondmate charter brief at $SEED_PARENT_BRIEF has an empty Routing scope section; fill it before seeding" >&2 - return 1 - } - for project in "$@"; do project_dst=$(validate_project_destination "$home" "$project") || return 1 [ -e "$project_dst" ] || printf '%s\n' "$project_dst" >> "$SEED_CREATED_PROJECTS_FILE" diff --git a/bin/fm-safe-task-tmp.py b/bin/fm-safe-task-tmp.py new file mode 100644 index 0000000000..d7198f4898 --- /dev/null +++ b/bin/fm-safe-task-tmp.py @@ -0,0 +1,131 @@ +#!/usr/bin/env python3 +import os +import stat +import sys + + +def identity(metadata): + return metadata.st_dev, metadata.st_ino + + +def hooks_enabled(): + return ( + os.environ.get("FM_ACCOUNT_ROUTING_TEST_LAB") + == "firstmate-account-routing-test-lab-v1" + and os.environ.get("FM_ACCOUNT_TEST_HOOKS") + == "firstmate-account-tests-v1" + ) + + +target = os.path.normpath(os.path.abspath(sys.argv[1])) +if not sys.argv[1] or target == os.path.sep: + raise SystemExit(1) + +parent = os.path.dirname(target) +name = os.path.basename(target) +flags = os.O_RDONLY | os.O_DIRECTORY +if not hasattr(os, "O_NOFOLLOW"): + raise SystemExit(1) +flags |= os.O_NOFOLLOW + + +def remove_tree(directory_fd, device): + entries = sorted(os.listdir(directory_fd)) + disappearing = os.environ.get("FM_SAFE_TASK_TMP_DISAPPEAR_ENTRY") + if ( + hooks_enabled() + and disappearing in entries + ): + os.unlink(disappearing, dir_fd=directory_fd) + del os.environ["FM_SAFE_TASK_TMP_DISAPPEAR_ENTRY"] + for entry in entries: + metadata = os.stat(entry, dir_fd=directory_fd, follow_symlinks=False) + if metadata.st_dev != device: + raise OSError(f"filesystem boundary at {entry}") + if stat.S_ISLNK(metadata.st_mode): + current_metadata = os.stat( + entry, dir_fd=directory_fd, follow_symlinks=False + ) + if identity(current_metadata) != identity(metadata): + raise OSError(f"symlink identity changed at {entry}") + os.unlink(entry, dir_fd=directory_fd) + elif stat.S_ISDIR(metadata.st_mode): + child_fd = os.open(entry, flags, dir_fd=directory_fd) + try: + opened = os.fstat(child_fd) + if identity(opened) != identity(metadata): + raise OSError(f"directory identity changed at {entry}") + remove_tree(child_fd, device) + current_metadata = os.stat( + entry, dir_fd=directory_fd, follow_symlinks=False + ) + if identity(current_metadata) != identity(opened): + raise OSError(f"directory identity changed at {entry}") + os.rmdir(entry, dir_fd=directory_fd) + finally: + os.close(child_fd) + else: + current_metadata = os.stat( + entry, dir_fd=directory_fd, follow_symlinks=False + ) + if identity(current_metadata) != identity(metadata): + raise OSError(f"entry identity changed at {entry}") + os.unlink(entry, dir_fd=directory_fd) + + +try: + parent_fd = os.open(os.path.sep, flags) + try: + try: + for component in parent.split(os.path.sep): + if not component: + continue + next_fd = os.open(component, flags, dir_fd=parent_fd) + os.close(parent_fd) + parent_fd = next_fd + except FileNotFoundError: + raise SystemExit(0) + swap_ancestor_value = os.environ.get("FM_SAFE_TASK_TMP_SWAP_ANCESTOR") + if hooks_enabled() and swap_ancestor_value: + swap_ancestor = os.path.normpath(os.path.abspath(swap_ancestor_value)) + swap_moved = os.path.normpath( + os.path.abspath(os.environ["FM_SAFE_TASK_TMP_SWAP_MOVED"]) + ) + swap_outside = os.path.normpath( + os.path.abspath(os.environ["FM_SAFE_TASK_TMP_SWAP_OUTSIDE"]) + ) + fixture_parent = os.path.dirname(swap_ancestor) + if ( + swap_ancestor_value != swap_ancestor + or os.path.commonpath((target, swap_ancestor)) != swap_ancestor + or target == swap_ancestor + or os.path.dirname(swap_moved) != fixture_parent + or os.path.dirname(swap_outside) != fixture_parent + or len({swap_ancestor, swap_moved, swap_outside}) != 3 + ): + raise OSError("unsafe task temp swap fixture") + os.rename(swap_ancestor, swap_moved) + os.symlink(swap_outside, swap_ancestor, target_is_directory=True) + try: + metadata = os.stat(name, dir_fd=parent_fd, follow_symlinks=False) + if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISDIR(metadata.st_mode): + raise OSError("task temp root is not a real directory") + root_fd = os.open(name, flags, dir_fd=parent_fd) + except FileNotFoundError: + raise SystemExit(0) + try: + opened = os.fstat(root_fd) + if identity(opened) != identity(metadata): + raise OSError("task temp root identity changed") + remove_tree(root_fd, opened.st_dev) + current_metadata = os.stat(name, dir_fd=parent_fd, follow_symlinks=False) + if identity(current_metadata) != identity(opened): + raise OSError("task temp root identity changed") + os.rmdir(name, dir_fd=parent_fd) + finally: + os.close(root_fd) + finally: + os.close(parent_fd) +except OSError as error: + print(f"REFUSED: unsafe task temp removal: {error}", file=sys.stderr) + raise SystemExit(1) diff --git a/bin/fm-spawn.sh b/bin/fm-spawn.sh index cec185d8bb..25cf75b30c 100755 --- a/bin/fm-spawn.sh +++ b/bin/fm-spawn.sh @@ -602,7 +602,8 @@ reconcile_failed_direct_recovery() { echo "error: retained direct recovery artifacts are missing or unsafe for $task" >&2 return 1 fi - if [ "$tasktmp" != "/tmp/fm-$task" ] || [ -z "$generation" ] || [ -z "$target" ]; then + if ! fm_account_task_tmp_is_expected "$task" "$tasktmp" "$generation" \ + || [ -z "$generation" ] || [ -z "$target" ]; then fm_account_meta_lock_release "$lock" >/dev/null 2>&1 || true echo "error: retained direct recovery metadata is incomplete for $task" >&2 return 1 @@ -639,7 +640,7 @@ reconcile_failed_direct_recovery() { echo "error: retained direct recovery generation changed before cleanup for $task" >&2 return 1 fi - if ! fm_account_restore_artifacts "$STATE" "$task" "$artifacts_name" "$tasktmp" 1; then + if ! fm_account_restore_artifacts "$STATE" "$task" "$artifacts_name" "$tasktmp" 1 "$generation"; then fm_account_meta_lock_release "$lock" >/dev/null 2>&1 || true echo "error: retained direct recovery artifacts could not be restored for $task" >&2 return 1 @@ -864,10 +865,11 @@ if [ "$RECOVERY_ACCOUNT" = 1 ]; then rollback_tab=$(fm_account_meta_value "$RESUME_META" zellij_tab_id) rollback_home=$(fm_account_meta_value "$RESUME_META" home) rollback_tasktmp=$(fm_account_meta_value "$RESUME_META" tasktmp) + rollback_generation=$(fm_account_meta_value "$RESUME_META" generation_id) rollback_backup=$(fm_account_meta_value "$RESUME_META" account_rollback_backup) fm_account_meta_lock_release "$rollback_meta_lock" || exit 1 rollback_meta_lock= - if [ -n "$rollback_tasktmp" ] && [ "$rollback_tasktmp" != "/tmp/fm-$rollback_id" ]; then + if [ -n "$rollback_tasktmp" ] && ! fm_account_task_tmp_is_expected "$rollback_id" "$rollback_tasktmp" "$rollback_generation"; then echo "error: unsafe task temp path in rollback metadata for $rollback_id: $rollback_tasktmp" >&2 exit 1 fi @@ -893,7 +895,11 @@ if [ "$RECOVERY_ACCOUNT" = 1 ]; then rollback_profile=$(fm_account_meta_value "$RESUME_META" account_profile) if [ -z "$rollback_profile" ] && [ "$rollback_kind" = secondmate ] && [ -z "$rollback_backup" ]; then rm -f "$RESUME_META" "$STATE/$rollback_id.status" "$STATE/$rollback_id.turn-ended" "$STATE/$rollback_id.check.sh" "$STATE/$rollback_id.pi-ext.ts" "$STATE/$rollback_id.grok-turnend-token" - [ -z "$rollback_tasktmp" ] || rm -rf "$rollback_tasktmp" + if [ -n "$rollback_tasktmp" ] \ + && { fm_account_task_tmp_is_current "$rollback_id" "$rollback_tasktmp" "$rollback_generation" \ + || fm_account_task_tmp_is_previous "$rollback_id" "$rollback_tasktmp"; }; then + fm_account_safe_remove_task_tmp "$rollback_id" "$rollback_tasktmp" "$rollback_generation" || exit 1 + fi fi if [ -z "$rollback_profile" ]; then if [ -n "$rollback_backup" ]; then @@ -953,7 +959,6 @@ ORCA_TERMINAL= ORCA_TERMINAL_PROOF= ORCA_REPO_ID= ORCA_EXPECTED_TASK= -ORCA_PROVIDER_TASK= ID= ACCOUNT_LEASE_CREATED=0 FM_ACCOUNT_MUTATION_ACQUIRED=0 @@ -1007,7 +1012,7 @@ ORIGINAL_GROK_TOKEN_PRESENT=-1 ORIGINAL_TASK_TMP_PRESENT=-1 snapshot_existing_artifacts() { - local backup name source tasktmp="/tmp/fm-$ID" + local backup name source tasktmp=$SPAWN_TASK_TMP backup=$(mktemp -d "$STATE/.$ID.artifacts.rollback.XXXXXX") || return 1 for name in "$ID.status" "$ID.turn-ended" "$ID.check.sh" "$ID.pi-ext.ts" "$ID.grok-turnend-token"; do source="$STATE/$name" @@ -1044,7 +1049,6 @@ parse_orca_worktree_result() { rest=${rest#*$'\t'} ORCA_REPO_ID=$rest case "$ORCA_REPO_ID" in *$'\t'*) return 1 ;; esac - ORCA_PROVIDER_TASK= } persist_orca_cleanup_quarantine() { @@ -1262,7 +1266,7 @@ persist_failed_direct_recovery() { echo "kind=${KIND:-${RECORDED_KIND:-ship}}" echo "mode=${MODE:-${RECORDED_MODE:-no-mistakes}}" echo "yolo=${YOLO:-${RECORDED_YOLO:-off}}" - echo "tasktmp=${TASK_TMP:-${RECORDED_TASKTMP:-/tmp/fm-$ID}}" + echo "tasktmp=${TASK_TMP:-$SPAWN_TASK_TMP}" echo "model=${RECORDED_MODEL:-${MODEL:-default}}" echo "effort=${RECORDED_EFFORT:-${EFFORT:-default}}" echo "generation_id=${RECORDED_GENERATION:-${SPAWN_GENERATION_ID:-}}" @@ -1351,7 +1355,7 @@ persist_failed_direct_spawn() { # echo "kind=${KIND:-ship}" echo "mode=$retained_mode" echo "yolo=$retained_yolo" - echo "tasktmp=${TASK_TMP:-/tmp/fm-$ID}" + echo "tasktmp=${TASK_TMP:-$SPAWN_TASK_TMP}" echo "model=${MODEL:-default}" echo "effort=${EFFORT:-default}" echo "generation_id=${SPAWN_GENERATION_ID:-}" @@ -1377,6 +1381,29 @@ persist_failed_direct_spawn() { # } } > "$tmp" || { rm -f "$tmp"; return 1; } fi + if [ "$endpoint_created" = 1 ]; then + case "${BACKEND:-tmux}" in + tmux) + grep -q '^tmux_window_id=' "$tmp" || printf 'tmux_window_id=%s\n' "${WID:-}" >> "$tmp" || { rm -f "$tmp"; return 1; } + grep -q '^tmux_session_target=' "$tmp" || printf 'tmux_session_target=%s\n' "$retained_tmux_session" >> "$tmp" || { rm -f "$tmp"; return 1; } + ;; + herdr) + grep -q '^herdr_session=' "$tmp" || printf 'herdr_session=%s\n' "${HERDR_SES:-}" >> "$tmp" || { rm -f "$tmp"; return 1; } + grep -q '^herdr_workspace_id=' "$tmp" || printf 'herdr_workspace_id=%s\n' "${HERDR_WORKSPACE_ID:-}" >> "$tmp" || { rm -f "$tmp"; return 1; } + grep -q '^herdr_tab_id=' "$tmp" || printf 'herdr_tab_id=%s\n' "${HERDR_TAB_ID:-}" >> "$tmp" || { rm -f "$tmp"; return 1; } + grep -q '^herdr_pane_id=' "$tmp" || printf 'herdr_pane_id=%s\n' "${HERDR_PANE_ID:-}" >> "$tmp" || { rm -f "$tmp"; return 1; } + ;; + zellij) + grep -q '^zellij_session=' "$tmp" || printf 'zellij_session=%s\n' "${ZELLIJ_SES:-}" >> "$tmp" || { rm -f "$tmp"; return 1; } + grep -q '^zellij_tab_id=' "$tmp" || printf 'zellij_tab_id=%s\n' "${ZELLIJ_TAB_ID:-}" >> "$tmp" || { rm -f "$tmp"; return 1; } + grep -q '^zellij_pane_id=' "$tmp" || printf 'zellij_pane_id=%s\n' "${ZELLIJ_PANE_ID:-}" >> "$tmp" || { rm -f "$tmp"; return 1; } + ;; + cmux) + grep -q '^cmux_workspace_id=' "$tmp" || printf 'cmux_workspace_id=%s\n' "${CMUX_WORKSPACE_ID:-}" >> "$tmp" || { rm -f "$tmp"; return 1; } + grep -q '^cmux_surface_id=' "$tmp" || printf 'cmux_surface_id=%s\n' "${CMUX_SURFACE_ID:-}" >> "$tmp" || { rm -f "$tmp"; return 1; } + ;; + esac + fi if [ "$preserve_extensions" = 1 ]; then fm_account_meta_merge_extensions "$meta" "$tmp" || { rm -f "$tmp"; return 1; } fi @@ -1464,7 +1491,7 @@ spawn_restore_unmanaged_state_locked() { [ "${ACCOUNT_EFFECTIVE_MODE:-off}" != enforce ] || return 0 if [ -n "$EXISTING_ARTIFACT_BACKUP" ]; then artifact_backup_name=${EXISTING_ARTIFACT_BACKUP##*/} - fm_account_restore_artifacts "$STATE" "$ID" "$artifact_backup_name" "/tmp/fm-$ID" 1 || return 1 + fm_account_restore_artifacts "$STATE" "$ID" "$artifact_backup_name" "$SPAWN_TASK_TMP" 1 "$SPAWN_GENERATION_ID" || return 1 fi if [ -n "$META_BACKUP" ]; then [ -f "$META_BACKUP" ] && [ -f "$meta" ] || return 1 @@ -1636,7 +1663,7 @@ spawn_abort_cleanup() { fi if [ -n "$rollback_lock" ]; then artifact_backup_name=${EXISTING_ARTIFACT_BACKUP##*/} - if fm_account_restore_artifacts "$STATE" "$ID" "$artifact_backup_name" "${TASK_TMP:-/tmp/fm-$ID}" 1; then + if fm_account_restore_artifacts "$STATE" "$ID" "$artifact_backup_name" "${TASK_TMP:-$SPAWN_TASK_TMP}" 1 "$SPAWN_GENERATION_ID"; then if [ "$META_INSTALLED" = 1 ] && [ -n "$META_BACKUP" ] && [ -f "$META_BACKUP" ]; then if fm_account_meta_merge_extensions "$STATE/$ID.meta" "$META_BACKUP" \ && fm_account_safe_file_destination "$STATE/$ID.meta" \ @@ -1670,7 +1697,7 @@ spawn_abort_cleanup() { if [ -n "$rollback_lock" ] && [ "$worktree_clean" = 1 ]; then if [ -n "$META_BACKUP" ] && [ -f "$META_BACKUP" ]; then artifact_backup_name=${EXISTING_ARTIFACT_BACKUP##*/} - if fm_account_restore_artifacts "$STATE" "$ID" "$artifact_backup_name" "${TASK_TMP:-/tmp/fm-$ID}" 1 \ + if fm_account_restore_artifacts "$STATE" "$ID" "$artifact_backup_name" "${TASK_TMP:-$SPAWN_TASK_TMP}" 1 "$SPAWN_GENERATION_ID" \ && fm_account_meta_merge_extensions "$STATE/$ID.meta" "$META_BACKUP" \ && fm_account_safe_file_destination "$STATE/$ID.meta" \ && mv "$META_BACKUP" "$STATE/$ID.meta"; then @@ -1691,7 +1718,11 @@ spawn_abort_cleanup() { [ "$ORIGINAL_CHECK_PRESENT" != 0 ] || rm -f "$STATE/$ID.check.sh" [ "$ORIGINAL_PI_EXT_PRESENT" != 0 ] || rm -f "$STATE/$ID.pi-ext.ts" [ "$ORIGINAL_GROK_TOKEN_PRESENT" != 0 ] || rm -f "$STATE/$ID.grok-turnend-token" - [ "$ORIGINAL_TASK_TMP_PRESENT" != 0 ] || { [ -z "${TASK_TMP:-}" ] || rm -rf "$TASK_TMP"; } + if [ "$ORIGINAL_TASK_TMP_PRESENT" = 0 ] && [ -n "${TASK_TMP:-}" ] \ + && ! fm_account_safe_remove_task_tmp "$ID" "$TASK_TMP" "$SPAWN_GENERATION_ID"; then + worktree_clean=0 + echo "warning: failed to remove direct spawn task temp for ${ID:-unknown}; retaining cleanup metadata" >&2 + fi fi fi if [ "$worktree_clean" != 1 ]; then @@ -1732,7 +1763,7 @@ spawn_abort_cleanup() { if [ "$(fm_meta_get "$STATE/$ID.meta" account_task)" = "$ACCOUNT_TASK" ] \ || cmp -s "$STATE/$ID.meta" "$META_BACKUP"; then artifact_backup_name=${EXISTING_ARTIFACT_BACKUP##*/} - if fm_account_restore_artifacts "$STATE" "$ID" "$artifact_backup_name" "${TASK_TMP:-}" 1 \ + if fm_account_restore_artifacts "$STATE" "$ID" "$artifact_backup_name" "${TASK_TMP:-}" 1 "$SPAWN_GENERATION_ID" \ && fm_account_meta_merge_extensions "$STATE/$ID.meta" "$META_BACKUP" \ && fm_account_safe_file_destination "$STATE/$ID.meta" \ && mv "$META_BACKUP" "$STATE/$ID.meta"; then @@ -1770,7 +1801,11 @@ spawn_abort_cleanup() { [ "$ORIGINAL_CHECK_PRESENT" != 0 ] || rm -f "$STATE/$ID.check.sh" [ "$ORIGINAL_PI_EXT_PRESENT" != 0 ] || rm -f "$STATE/$ID.pi-ext.ts" [ "$ORIGINAL_GROK_TOKEN_PRESENT" != 0 ] || rm -f "$STATE/$ID.grok-turnend-token" - [ "$ORIGINAL_TASK_TMP_PRESENT" != 0 ] || { [ -z "${TASK_TMP:-}" ] || rm -rf "$TASK_TMP"; } + if [ "$ORIGINAL_TASK_TMP_PRESENT" = 0 ] && [ -n "${TASK_TMP:-}" ] \ + && ! fm_account_safe_remove_task_tmp "$ID" "$TASK_TMP" "$SPAWN_GENERATION_ID"; then + account_clean=0 + echo "warning: failed to remove Agent Fleet task temp for ${ID:-unknown}; retaining cleanup metadata" >&2 + fi fi if [ "$account_clean" != 1 ] && [ -n "$rollback_lock" ]; then persist_failed_account_rollback || echo "warning: failed to persist Agent Fleet rollback state for ${ID:-unknown}" >&2 @@ -1845,6 +1880,23 @@ if [ "${#POS[@]}" -gt 0 ] && [ "${POS[0]}" != "$idpart" ] && case "$idpart" in * exit "$rc" fi ID=${POS[0]} +mkdir -p "$STATE" || { + echo "error: cannot establish state directory at $STATE" >&2 + exit 1 +} +if [ "$SPAWN_META_PRESENT" = 1 ]; then + EXISTING_TASK_TMP=$(spawn_preflight_meta_value tasktmp) + EXISTING_TASK_GENERATION=$(spawn_preflight_meta_value generation_id) + if [ -n "$EXISTING_TASK_TMP" ]; then + fm_account_task_tmp_is_expected "$ID" "$EXISTING_TASK_TMP" "$EXISTING_TASK_GENERATION" || { + echo "error: existing task metadata has an unsafe tasktmp for $ID" >&2 + exit 1 + } + fi +else + EXISTING_TASK_TMP= + EXISTING_TASK_GENERATION= +fi PROJ= ARG3= FIRSTMATE_HOME= @@ -1929,8 +1981,6 @@ if [ -e "$STATE/$ID.turn-ended" ] || [ -L "$STATE/$ID.turn-ended" ]; then ORIGIN if [ -e "$STATE/$ID.check.sh" ] || [ -L "$STATE/$ID.check.sh" ]; then ORIGINAL_CHECK_PRESENT=1; else ORIGINAL_CHECK_PRESENT=0; fi if [ -e "$STATE/$ID.pi-ext.ts" ] || [ -L "$STATE/$ID.pi-ext.ts" ]; then ORIGINAL_PI_EXT_PRESENT=1; else ORIGINAL_PI_EXT_PRESENT=0; fi if [ -e "$STATE/$ID.grok-turnend-token" ] || [ -L "$STATE/$ID.grok-turnend-token" ]; then ORIGINAL_GROK_TOKEN_PRESENT=1; else ORIGINAL_GROK_TOKEN_PRESENT=0; fi -if [ -e "/tmp/fm-$ID" ] || [ -L "/tmp/fm-$ID" ]; then ORIGINAL_TASK_TMP_PRESENT=1; else ORIGINAL_TASK_TMP_PRESENT=0; fi - if [ "$RECOVERY_ACCOUNT" = 1 ]; then RECORDED_KIND=$(fm_meta_get "$RESUME_META" kind) [ -n "$RECORDED_KIND" ] || RECORDED_KIND=ship @@ -2007,7 +2057,7 @@ if [ "$RECOVERY_ACCOUNT" = 1 ]; then [ -n "$RECORDED_MODE" ] || { echo "error: direct account recovery metadata has no mode for $ID" >&2; exit 1; } [ -n "$RECORDED_YOLO" ] || { echo "error: direct account recovery metadata has no yolo setting for $ID" >&2; exit 1; } [ -n "$RECORDED_GENERATION" ] || { echo "error: direct account recovery metadata has no generation_id for $ID" >&2; exit 1; } - [ "$RECORDED_TASKTMP" = "/tmp/fm-$ID" ] || { echo "error: direct account recovery metadata has an invalid tasktmp for $ID" >&2; exit 1; } + fm_account_task_tmp_is_expected "$ID" "$RECORDED_TASKTMP" "$RECORDED_GENERATION" || { echo "error: direct account recovery metadata has an invalid tasktmp for $ID" >&2; exit 1; } RECORDED_META_WORKTREE_GIT_REF=$RECORDED_WORKTREE_GIT_REF RECORDED_META_WORKTREE_GIT_HEAD=$RECORDED_WORKTREE_GIT_HEAD RECORDED_META_WORKTREE_GIT_SETUP_REF=$RECORDED_WORKTREE_GIT_SETUP_REF @@ -2348,6 +2398,15 @@ elif [ "$ACCOUNT_EFFECTIVE_MODE" != off ]; then else SPAWN_GENERATION_ID="spawn:$(fm_account_attempt_id "$FM_HOME" "$ID")" || exit 1 fi +if [ "$DIRECT_ACCOUNT_RECOVERY" = 1 ]; then + SPAWN_TASK_TMP=$RECORDED_TASKTMP +else + SPAWN_TASK_TMP=$(fm_account_task_tmp_path "$ID" "$SPAWN_GENERATION_ID") || { + echo "error: cannot establish a safe task temp path for $ID" >&2 + exit 1 + } +fi +if [ -e "$SPAWN_TASK_TMP" ] || [ -L "$SPAWN_TASK_TMP" ]; then ORIGINAL_TASK_TMP_PRESENT=1; else ORIGINAL_TASK_TMP_PRESENT=0; fi if [ "$ACCOUNT_EFFECTIVE_MODE" = enforce ]; then META_WRITE_LOCK=$(fm_account_meta_lock_acquire "$STATE" "$ID") || exit 1 if [ "$RECOVERY_ACCOUNT" = 1 ]; then @@ -2699,9 +2758,14 @@ if [ "$KIND" = secondmate ]; then echo "error: refusing secondmate launch for $PROJ_ABS: the primary's $ACCOUNT_PRIMARY_MODE routing mode is not authoritative in the home. Run bin/fm-config-push.sh and retry." >&2 exit 1 fi + if git -C "$PROJ_ABS" ls-files --error-unmatch bin/fm-account-routing-lib.sh >/dev/null 2>&1 \ + && [ ! -f "$PROJ_ABS/bin/fm-account-routing-lib.sh" ]; then + echo "error: refusing secondmate $ID launch for home $PROJ_ABS: its dirty working tree is missing tracked Agent Fleet routing support. Restore or otherwise reconcile the home and retry." >&2 + exit 1 + fi if [ "$ACCOUNT_EFFECTIVE_MODE" = enforce ]; then if ! secondmate_home_supports_account_routing "$PROJ_ABS"; then - echo "error: refusing account-routed secondmate launch for $PROJ_ABS: the home lacks Agent Fleet routing support. Fast-forward or otherwise reconcile the home to this Firstmate revision, run bin/fm-config-push.sh, and retry." >&2 + echo "error: refusing account-routed secondmate $ID launch for home $PROJ_ABS: the home lacks Agent Fleet routing support, which indicates an unreconciled revision or dirty working tree. Fast-forward or otherwise reconcile the home to this Firstmate revision, run bin/fm-config-push.sh, and retry." >&2 exit 1 fi elif ! secondmate_home_supports_account_routing "$PROJ_ABS"; then @@ -2735,6 +2799,7 @@ fi # once here so every downstream comparison uses the same physical form # (docs/herdr-backend.md "Known gaps"). PROJ_ABS_REAL=$(cd "$PROJ_ABS" 2>/dev/null && pwd -P) || PROJ_ABS_REAL="$PROJ_ABS" +PROJ_ABS=$PROJ_ABS_REAL real_path_or_raw() { # local path=$1 real @@ -2981,12 +3046,15 @@ if [ "$DIRECT_ACCOUNT_ROUTING" = 1 ] && [ "$DIRECT_ACCOUNT_RECOVERY" = 0 ] && [ } fi -# Per-task temp root: /tmp/fm-/ with Go's build temp nested at gotmp/. Go won't +# Per-task temp root with Go's build temp nested at gotmp/. The physical task +# state directory is its namespace, so separate homes and test runs cannot +# share a root even when their human-readable task ids match. Go won't # create GOTMPDIR, so mkdir before it is used; fm-teardown removes the whole root. -# Nested (not a bare /tmp/fm-/gotmp) so other per-task temp can live alongside -# later, and teardown cleans one deterministic path. GOTMPDIR (not TMPDIR) is the -# targeted knob: TMPDIR is too broad (affects every program's temp, not just Go's). -TASK_TMP="/tmp/fm-$ID" +# Nested (not a bare task-root/gotmp) so other per-task temp can live alongside +# later, and teardown removes only this recorded generation. GOTMPDIR (not TMPDIR) +# is the targeted knob: TMPDIR is too broad (affects every program's temp, not +# just Go's). +TASK_TMP=$SPAWN_TASK_TMP mkdir -p "$TASK_TMP/gotmp" # herdr sets GOTMPDIR natively at agent start. Every other backend exports it into # the pane shell just before the launch line, further down. CREW_PATH rides the same @@ -3417,8 +3485,8 @@ EOF echo "error: cannot durably record Orca create authority for $ID" >&2 exit 1 } - if [ -z "$ORCA_WORKTREE_ID" ] || [ -z "$WT" ] || [ "$ORCA_PROVIDER_TASK" != "$ORCA_EXPECTED_TASK" ]; then - echo "error: orca did not return matching worktree id, path, and task authority for $W" >&2 + if [ -z "$ORCA_WORKTREE_ID" ] || [ -z "$WT" ]; then + echo "error: orca did not return worktree id and path authority for $W" >&2 exit 1 fi validate_spawn_worktree "orca worktree create" "$W" @@ -3486,15 +3554,19 @@ spawn_send_key() { # } if [ "$KIND" != secondmate ] && [ "$BACKEND" != orca ] && [ "$RECOVERY_ACCOUNT" != 1 ]; then WT_REAL=$(real_path_or_raw "$WT") - for _ in $(seq 1 60); do + ENDPOINT_READY_STARTED=$(date +%s) + ENDPOINT_READY_TIMEOUT=60 + while :; do p=$(spawn_current_path "$WT_TARGET" || true) if [ -n "$p" ] && [ "$(real_path_or_raw "$p")" = "$WT_REAL" ]; then break fi + ENDPOINT_READY_NOW=$(date +%s) + [ $((ENDPOINT_READY_NOW - ENDPOINT_READY_STARTED)) -lt "$ENDPOINT_READY_TIMEOUT" ] || break sleep 1 done if [ -z "${p:-}" ] || [ "$(real_path_or_raw "$p")" != "$WT_REAL" ]; then - echo "error: task endpoint did not start in leased worktree $WT within 60s; inspect window $T" >&2 + echo "error: task endpoint did not start in leased worktree $WT within ${ENDPOINT_READY_TIMEOUT}s; inspect window $T" >&2 exit 1 fi fi @@ -3730,6 +3802,17 @@ if [ "$ACCOUNT_EFFECTIVE_MODE" = enforce ]; then fi fi [ "$ACCOUNT_EFFECTIVE_MODE" = enforce ] || ACCOUNT_SPAWN_COMMITTED=1 +if [ -n "$EXISTING_TASK_TMP" ] && [ "$EXISTING_TASK_TMP" != "$TASK_TMP" ]; then + META_WRITE_LOCK=$(fm_account_meta_lock_acquire "$STATE" "$ID") || exit 1 + if [ "$(fm_account_meta_value "$STATE/$ID.meta" generation_id)" != "$SPAWN_GENERATION_ID" ] \ + || [ "$(fm_account_meta_value "$STATE/$ID.meta" tasktmp)" != "$TASK_TMP" ]; then + echo "error: task generation changed before prior temp cleanup for $ID" >&2 + exit 1 + fi + fm_account_safe_remove_task_tmp "$ID" "$EXISTING_TASK_TMP" "$EXISTING_TASK_GENERATION" || exit 1 + fm_account_meta_lock_release "$META_WRITE_LOCK" || exit 1 + META_WRITE_LOCK= +fi CONTINUATION_LAUNCH_DIR= CONTINUATION_PROMPT_FILE= [ -z "$META_BACKUP" ] || rm -f "$META_BACKUP" diff --git a/bin/fm-teardown.sh b/bin/fm-teardown.sh index ea03eda10b..fb6f48428f 100755 --- a/bin/fm-teardown.sh +++ b/bin/fm-teardown.sh @@ -43,9 +43,10 @@ # the handle stale, and remove the recorded worktree under its checkout lock; # teardown never substitutes the shared window alias for a missing terminal. # Secondmates (kind=secondmate in meta) are retired explicitly. Teardown proves -# the home clean and every ref and reflog commit landed, then quiesces its endpoint and -# refuses while the home has in-flight crewmate meta files. --force authorizes -# recursive retirement only after every child passes the same endpoint, identity, +# the home clean and every ref and reflog commit landed, then without --force +# refuses an in-flight child or non-empty child registry before quiescing the +# endpoint. --force quiesces only after the parent proofs, then authorizes +# recursive retirement after every child passes the same endpoint, identity, # cleanliness, stash, and landed-work proofs. Project retirement also rejects # mount boundaries, rewritten history, and landing authorities whose complete Git # object storage or network transport may depend on the retiring home or local machine. Removing a @@ -93,6 +94,7 @@ FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$FM_ROOT}}" STATE="${FM_STATE_OVERRIDE:-$FM_HOME/state}" DATA="${FM_DATA_OVERRIDE:-$FM_HOME/data}" CONFIG="${FM_CONFIG_OVERRIDE:-$FM_HOME/config}" +PROJECTS="${FM_PROJECTS_OVERRIDE:-$FM_HOME/projects}" CHECKOUT_STATE_BASE="${FM_CHECKOUT_REFRESH_STATE_BASE:-${XDG_STATE_HOME:-$HOME/.local/state}/firstmate/checkout-refresh}" SECONDMATE_REG="$DATA/secondmates.md" SUB_HOME_MARKER=".fm-secondmate-home" @@ -125,7 +127,6 @@ case "$TEARDOWN_UPSTREAM_TIMEOUT" in esac ID=$1 FORCE=${2:-} - META="$STATE/$ID.meta" require_safe_task_metadata() { @@ -202,10 +203,11 @@ if [ "$BACKEND" = orca ]; then fi HOME_PATH=$(grep '^home=' "$META" | cut -d= -f2- || true) PR_URL=$(grep '^pr=' "$META" | tail -1 | cut -d= -f2- || true) -# tasktmp is recorded by fm-spawn for tasks that set up a per-task temp root -# (/tmp/fm-/); absent for tasks spawned before that change, so tolerate empty. +# tasktmp is recorded by fm-spawn for tasks that set up a per-task temp root; +# absent for tasks spawned before that change, so tolerate empty. TASK_TMP=$(grep '^tasktmp=' "$META" | cut -d= -f2- || true) -if [ -n "$TASK_TMP" ] && [ "$TASK_TMP" != "/tmp/fm-$ID" ]; then +TASK_GENERATION=$(grep '^generation_id=' "$META" | cut -d= -f2- || true) +if [ -n "$TASK_TMP" ] && ! fm_account_task_tmp_is_expected "$ID" "$TASK_TMP" "$TASK_GENERATION"; then echo "REFUSED: unsafe task temp path in metadata for $ID: $TASK_TMP" >&2 exit 1 fi @@ -410,6 +412,10 @@ quiesce_child_endpoint() { scoped_target=$(meta_value "$meta" tmux_session_target) [ "$backend" != orca ] || scoped_target=$(meta_value "$meta" orca_worktree_id) if [ "$backend" = orca ]; then + [ -n "$target" ] || { + echo "error: child endpoint identity for $task is missing; refusing destructive cleanup" >&2 + return 1 + } quiesce_authoritative_orca_endpoint "$target" "$scoped_target" "fm-$task" || { echo "error: child Orca endpoint authority or quiescence is unproven for $task" >&2 return 1 @@ -1059,6 +1065,9 @@ validate_teardown_target_identity() { echo "error: teardown project metadata is not an exact inspectable repository root: ${PROJ:-}" >&2 return 1 } + if [ ! -e "$WT" ] && [ ! -L "$WT" ]; then + return 0 + fi worktree_root=$(exact_git_worktree_root "$WT") || { echo "error: teardown worktree metadata is not an exact inspectable repository root: ${WT:-}" >&2 return 1 @@ -1547,19 +1556,17 @@ import sys home, state_path = sys.argv[1:] try: - if os.path.islink(state_path): - raise OSError("state directory must not be a symlink") - metadata = os.stat(state_path) + home_root = os.path.realpath(home) + state_root = os.path.realpath(state_path) + if state_root == home_root or os.path.commonpath((home_root, state_root)) != home_root: + raise OSError("state directory resolves outside its secondmate home") + metadata = os.stat(state_root) permissions = stat.S_IMODE(metadata.st_mode) if not stat.S_ISDIR(metadata.st_mode): raise NotADirectoryError(state_path) if not permissions & 0o444 or not permissions & 0o111: raise PermissionError("state directory is unreadable") - home_root = os.path.realpath(home) - state_root = os.path.realpath(state_path) - if state_root != os.path.join(home_root, "state"): - raise OSError("state directory resolves outside its secondmate home") - with os.scandir(state_path) as entries: + with os.scandir(state_root) as entries: for entry in sorted(entries, key=lambda item: item.name): metadata = entry.stat(follow_symlinks=False) if not entry.name.endswith(".meta"): @@ -1570,7 +1577,7 @@ try: raise PermissionError(f"unreadable child metadata entry: {entry.path}") if any(character in entry.path for character in ("\n", "\r")): raise OSError("child metadata path contains unsupported control characters") - print(entry.path) + print(os.path.join(state_root, entry.name)) except OSError as error: print( f"REFUSED: secondmate child state is unprovable at {state_path}: {error}", @@ -1825,33 +1832,6 @@ safe_rm_rf_child_worktree() { removal_tree_operation "$canonical" "child worktree" remove } -safe_remove_task_tmp() { - local target=$1 base - [ -n "$target" ] || return 0 - [ "$target" = "/tmp/fm-$ID" ] || return 1 - base=$(python3 - <<'PY' -import os -import stat - -base = os.path.realpath("/tmp") -if base not in ("/tmp", "/private/tmp"): - raise SystemExit(1) -current = os.path.sep -for component in base.split(os.path.sep): - if not component: - continue - current = os.path.join(current, component) - metadata = os.lstat(current) - if stat.S_ISLNK(metadata.st_mode): - raise SystemExit(1) -if not stat.S_ISDIR(os.lstat(base).st_mode): - raise SystemExit(1) -print(base) -PY - ) || return 1 - removal_tree_operation "$base/fm-$ID" "task temp root" remove -} - remove_worktree_compatibility_artifacts() { local target=$1 label=$2 FM_COMPATIBILITY_CLEANUP_LABEL=$label python3 - "$target" <<'PY' @@ -1971,6 +1951,10 @@ validate_secondmate_home_landed_state() { $0 == "?? .claude/settings.local.json" { next } $0 == "?? .opencode/plugins/fm-turn-end.js" { next } $0 == "?? .fm-grok-turnend" { next } + $0 == "?? config" { next } + $0 == "?? data" { next } + $0 == "?? projects" { next } + $0 == "?? state" { next } $0 != "" { print } ') [ -z "$unsafe" ] || { @@ -2330,7 +2314,7 @@ def confined(path): except ValueError: return False -def retain(path, expected_directory): +def retain(path, expected_directory, keep_open=False): metadata = os.lstat(path) if stat.S_ISLNK(metadata.st_mode): raise OSError(f"redirected object storage entry: {path}") @@ -2363,7 +2347,11 @@ def retain(path, expected_directory): if confined(os.path.realpath(path)): os.close(descriptor) raise OSError(f"object storage depends on retiring home: {path}") - held.append((path, descriptor, expected)) + if expected_directory or keep_open: + held.append((path, descriptor, expected)) + else: + os.close(descriptor) + held.append((path, None, expected)) return descriptor, opened def inspect(objects): @@ -2374,9 +2362,15 @@ def inspect(objects): return visited.add(identity) object_device = metadata.st_dev + entries = [] for name in sorted(os.listdir(directory)): path = os.path.join(objects, name) item = os.stat(name, dir_fd=directory, follow_symlinks=False) + entries.append((name, path, item)) + if objects != initial: + os.close(directory) + held[-1] = (objects, None, held[-1][2]) + for name, path, item in entries: if item.st_dev != object_device: raise OSError(f"object storage crosses a filesystem boundary: {path}") if stat.S_ISLNK(item.st_mode): @@ -2393,7 +2387,7 @@ def inspect(objects): raise OSError(f"HTTP alternates are not durable proof: {http_alternates}") if not os.path.lexists(alternates): return - alternate_fd, _ = retain(alternates, False) + alternate_fd, _ = retain(alternates, False, True) with os.fdopen(os.dup(alternate_fd), "r", encoding="utf-8") as stream: entries = [line.rstrip("\n") for line in stream] if not entries or any(not entry or "\x00" in entry for entry in entries): @@ -2413,16 +2407,24 @@ def verify_retained(): stat.S_IFMT(metadata.st_mode), metadata.st_size, ) - opened = os.fstat(descriptor) - retained = ( - opened.st_dev, - opened.st_ino, - stat.S_IFMT(opened.st_mode), - opened.st_size, - ) + retained = expected + if descriptor is not None: + opened = os.fstat(descriptor) + retained = ( + opened.st_dev, + opened.st_ino, + stat.S_IFMT(opened.st_mode), + opened.st_size, + ) if current != expected or retained != expected or stat.S_ISLNK(metadata.st_mode): raise OSError(f"object storage identity changed during graph proof: {path}") +def release_nested_descriptors(): + for index, (path, descriptor, expected) in enumerate(held[1:], start=1): + if descriptor is not None: + os.close(descriptor) + held[index] = (path, None, expected) + def run(arguments, input_data=None): environment = os.environ.copy() environment["GIT_NO_REPLACE_OBJECTS"] = "1" @@ -2440,6 +2442,7 @@ def run(arguments, input_data=None): try: inspect(initial) + release_nested_descriptors() marker = os.environ.get("FM_TEARDOWN_TEST_OBJECT_SCAN_MARKER", "") release = os.environ.get("FM_TEARDOWN_TEST_OBJECT_SCAN_RELEASE", "") scan_root = os.environ.get("FM_TEARDOWN_TEST_OBJECT_SCAN_ROOT", "") @@ -2505,7 +2508,8 @@ except (OSError, UnicodeError, subprocess.SubprocessError) as error: raise SystemExit(1) finally: for _, descriptor, _ in reversed(held): - os.close(descriptor) + if descriptor is not None: + os.close(descriptor) PY } @@ -2708,6 +2712,7 @@ EOF } while IFS= read -r ref; do [ -n "$ref" ] || continue + [ "$ref" != refs/stash ] || continue git -C "$repository" cat-file -e "$ref^{object}" 2>/dev/null || { echo "REFUSED: $label ref $ref depends on unavailable objects" >&2 return 1 @@ -3074,8 +3079,29 @@ except OSError: PY } +secondmate_child_worktree_is_registered() { + local home=$1 repository=$2 worktree=$3 child_metas child_meta child_kind child_project child_worktree + local canonical_project canonical_worktree matches=0 + child_metas=$(secondmate_state_metadata "$home") || return 1 + while IFS= read -r child_meta; do + [ -n "$child_meta" ] || continue + child_kind=$(meta_value "$child_meta" kind) + [ "$child_kind" != secondmate ] || continue + child_project=$(meta_value "$child_meta" project) + child_worktree=$(meta_value "$child_meta" worktree) + canonical_project=$(exact_git_worktree_root "$child_project") || continue + canonical_worktree=$(exact_git_worktree_root "$child_worktree") || continue + if [ "$canonical_project" = "$repository" ] && [ "$canonical_worktree" = "$worktree" ]; then + matches=$((matches + 1)) + fi + done </dev/null) || return 1 common=$(git -C "$repository" rev-parse --git-common-dir 2>/dev/null) || return 1 @@ -3144,6 +3170,9 @@ validate_secondmate_repository_worktree_graph() { : elif [ "$submodule_admin" -eq 1 ] && [ "$canonical" = "$common" ]; then : + elif [ "$allow_registered_children" -eq 1 ] \ + && secondmate_child_worktree_is_registered "$retiring_home" "$repository" "$canonical"; then + : else echo "REFUSED: secondmate project common Git directory owns another linked worktree at $canonical" >&2 return 1 @@ -3154,10 +3183,17 @@ validate_secondmate_repository_worktree_graph() { done <&2 - return 1 - } + if [ "$allow_registered_children" -eq 1 ]; then + [ "$count" -ge 1 ] || { + echo "REFUSED: secondmate project linked-worktree ownership is ambiguous at $repository" >&2 + return 1 + } + else + [ "$count" -eq 1 ] || { + echo "REFUSED: secondmate project linked-worktree ownership is ambiguous at $repository" >&2 + return 1 + } + fi } validate_secondmate_declared_submodules() { @@ -3187,7 +3223,7 @@ EOF validate_secondmate_project_repository_landed_state() { local repository=$1 source_repository=$2 retiring_home=$3 repository_container=${4:-$1} - local source_container=${5:-$2} dirty refs ref tip reflog_tips + local source_container=${5:-$2} allow_registered_children=${6:-0} dirty refs ref tip reflog_tips local remote_tips remote_tip landed repository_identity source_identity bare stash_status local authority_record authority_kind authority cleanup_status=0 [ "$(exact_git_repository_root "$repository" "$repository_container")" = "$repository" ] || { @@ -3203,7 +3239,7 @@ validate_secondmate_project_repository_landed_state() { "registered source project repository" || return 1 git_history_rewrite_state_is_clean "$repository" "secondmate project repository" || return 1 validate_secondmate_repository_worktree_graph \ - "$repository" "$retiring_home" "$repository_container" || return 1 + "$repository" "$retiring_home" "$repository_container" "$allow_registered_children" || return 1 validate_secondmate_declared_submodules "$repository" "$repository_container" || { echo "REFUSED: secondmate project submodule state is uninspectable at $repository" >&2 return 1 @@ -3306,7 +3342,7 @@ EOF validate_secondmate_project_clones() { local home=$1 registry=$2 expected_id=$3 expected_source=$4 projects_root source_projects_root - local expected listed project clone source_clone repositories relative repository source_repository + local allow_registered_children=${5:-0} home_root expected listed project clone source_clone repositories relative repository source_repository if ! expected=$(fm_secondmate_registry_query "$registry" query "$expected_id" projects); then if [ "$registry" = "$PREPARED_REGISTRY_PATH" ] \ && [ "$expected_id" = "$PREPARED_REGISTRY_ID" ] \ @@ -3320,7 +3356,12 @@ validate_secondmate_project_clones() { return 1 fi fi - projects_root=$(fm_checkout_trusted_dir "$home/projects") || { + home_root=$(canonical_existing_dir "$home") || return 1 + projects_root=$(cd "$home/projects" 2>/dev/null && pwd -P) || { + echo "REFUSED: secondmate projects directory is missing, redirected, or unreadable at $home/projects" >&2 + return 1 + } + path_is_ancestor_of "$home_root" "$projects_root" || { echo "REFUSED: secondmate projects directory is missing, redirected, or unreadable at $home/projects" >&2 return 1 } @@ -3352,8 +3393,8 @@ PY echo "REFUSED: secondmate project clones do not exactly match the registration for $expected_id" >&2 return 1 } - if [ "$expected_source" = "$FM_ROOT" ] && [ -n "${FM_PROJECTS_OVERRIDE:-}" ]; then - source_projects_root=$FM_PROJECTS_OVERRIDE + if [ "$expected_source" = "$FM_ROOT" ]; then + source_projects_root=$PROJECTS else source_projects_root="$expected_source/projects" fi @@ -3386,7 +3427,7 @@ PY } fi validate_secondmate_project_repository_landed_state \ - "$repository" "$source_repository" "$home" "$clone" "$source_clone" || return 1 + "$repository" "$source_repository" "$home" "$clone" "$source_clone" "$allow_registered_children" || return 1 done <}" >&2 @@ -3437,7 +3478,7 @@ validate_firstmate_home_for_removal() { validate_secondmate_home_landed_state "$abs_home_path" "$expected_source" || return 1 if [ -n "$expected_id" ]; then validate_secondmate_project_clones \ - "$abs_home_path" "$expected_registry" "$expected_id" "$expected_source" || return 1 + "$abs_home_path" "$expected_registry" "$expected_id" "$expected_source" "$allow_registered_children" || return 1 fi validate_firstmate_operational_dirs_for_removal "$abs_home_path" "$label" || return 1 secondmate_state_metadata "$abs_home_path" >/dev/null || return 1 @@ -3553,7 +3594,7 @@ validate_firstmate_home_children_removal() { child_proj=$(meta_value "$child_meta" project) child_home=$(meta_value "$child_meta" home) [ -n "$child_home" ] || child_home=$child_wt - validate_firstmate_home_for_removal "$child_home" "child firstmate home" "$child_id" "$home" "$home/data/secondmates.md" "$child_proj" >/dev/null || return 1 + validate_firstmate_home_for_removal "$child_home" "child firstmate home" "$child_id" "$home" "$home/data/secondmates.md" "$child_proj" 1 1 >/dev/null || return 1 validate_firstmate_home_children_removal "$child_home" || return 1 elif [ "$child_backend" = orca ]; then require_orca_task_metadata_identity "$child_meta" "$child_id" || return 1 @@ -3885,7 +3926,7 @@ if [ "$ORCA_CLEANUP_PENDING" = 1 ]; then fm_checkout_lock_run "$WT" "$CHECKOUT_LOCK_ROOT" remove_pending_orca_worktree_locked || exit 1 remove_grok_turnend_auth "$STATE" "$ID" fm_backend_clear_transition "$BACKEND" "$STATE" "$T" || true - safe_remove_task_tmp "$TASK_TMP" || exit 1 + fm_account_safe_remove_task_tmp "$ID" "$TASK_TMP" "$TASK_GENERATION" || exit 1 rm -f "$STATE/$ID.status" "$STATE/$ID.turn-ended" "$STATE/$ID.check.sh" "$STATE/$ID.meta" "$STATE/$ID.pi-ext.ts" "$STATE/$ID.grok-turnend-token" [ -z "$ACCOUNT_DELETE_LOCK" ] || fm_account_lifecycle_lock_release "$ACCOUNT_DELETE_LOCK" >/dev/null 2>&1 || true ACCOUNT_DELETE_LOCK= @@ -3900,15 +3941,12 @@ if [ "$KIND" = secondmate ]; then exit 1 } validate_firstmate_home_for_removal \ - "$HOME_PATH" "secondmate home" "$ID" "$FM_ROOT" "$SECONDMATE_REG" "$PROJ" 0 \ + "$HOME_PATH" "secondmate home" "$ID" "$FM_ROOT" "$SECONDMATE_REG" "$PROJ" 0 1 \ >/dev/null || exit 1 - if [ "$FORCE" = "--force" ]; then - validate_firstmate_home_children_removal "$HOME_PATH" || exit 1 - fi - quiesce_secondmate_endpoint || exit 1 if [ "$FORCE" = "--force" ]; then validate_firstmate_home_children_removal "$HOME_PATH" || exit 1 else + require_empty_secondmate_registry "$HOME_PATH" || exit 1 SUB_STATE="$HOME_PATH/state" CHILD_METAS=$(secondmate_state_metadata "$HOME_PATH") || exit 1 if [ -n "$CHILD_METAS" ]; then @@ -3918,6 +3956,10 @@ if [ "$KIND" = secondmate ]; then exit 1 fi fi + quiesce_secondmate_endpoint || exit 1 + if [ "$FORCE" = "--force" ]; then + validate_firstmate_home_children_removal "$HOME_PATH" || exit 1 + fi fi if [ "$KIND" = scout ]; then @@ -4065,7 +4107,11 @@ if [ "$MANAGED_ACCOUNT" = 1 ]; then fi if [ "$KIND" = secondmate ] && [ "$FORCE" = "--force" ]; then - cleanup_firstmate_home_children "$HOME_PATH" || exit 1 + if cleanup_firstmate_home_children "$HOME_PATH"; then + : + else + exit $? + fi fi [ "$KIND" = secondmate ] || validate_teardown_target_identity || exit 1 @@ -4126,7 +4172,7 @@ if [ "$DIRECT_SPAWN_CLEANUP" = pending ] && [ -n "$DIRECT_SPAWN_BACKUP" ]; then exit 1 } direct_spawn_restore_lock=$(fm_account_meta_lock_acquire "$STATE" "$ID") || exit 1 - if ! fm_account_restore_artifacts "$STATE" "$ID" "$DIRECT_SPAWN_ARTIFACTS" "$TASK_TMP" 1 \ + if ! fm_account_restore_artifacts "$STATE" "$ID" "$DIRECT_SPAWN_ARTIFACTS" "$TASK_TMP" 1 "$TASK_GENERATION" \ || ! fm_account_meta_merge_extensions "$META" "$direct_spawn_backup_path" \ || ! fm_account_safe_file_destination "$META" \ || ! mv "$direct_spawn_backup_path" "$META"; then @@ -4182,9 +4228,9 @@ EOF fi remove_grok_turnend_auth "$STATE" "$ID" fm_backend_clear_transition "$BACKEND" "$STATE" "$T" || true -# Remove the per-task temp root (/tmp/fm-/, incl. its gotmp/) recorded by spawn. +# Remove the per-task temp root, including its gotmp, recorded by spawn. # Read before the state-file rm below; empty (pre-fix tasks without tasktmp=) is a no-op. -[ -z "$TASK_TMP" ] || safe_remove_task_tmp "$TASK_TMP" || exit 1 +[ -z "$TASK_TMP" ] || fm_account_safe_remove_task_tmp "$ID" "$TASK_TMP" "$TASK_GENERATION" || exit 1 rm -f "$STATE/$ID.status" "$STATE/$ID.turn-ended" "$STATE/$ID.check.sh" "$STATE/$ID.meta" "$STATE/$ID.pi-ext.ts" "$STATE/$ID.grok-turnend-token" [ -z "$ACCOUNT_DELETE_LOCK" ] || fm_account_lifecycle_lock_release "$ACCOUNT_DELETE_LOCK" >/dev/null 2>&1 || true if [ "$KIND" != scout ] && [ "$KIND" != secondmate ] && [ "$MODE" != local-only ]; then diff --git a/bin/fm-watch-checkpoint.sh b/bin/fm-watch-checkpoint.sh index d754301fa4..9d153b1f69 100755 --- a/bin/fm-watch-checkpoint.sh +++ b/bin/fm-watch-checkpoint.sh @@ -88,10 +88,10 @@ run_with_perl_timeout() { set +e if command -v timeout >/dev/null 2>&1; then - timeout "$SECONDS_ARG" "$SCRIPT_DIR/fm-watch.sh" >"$OUT" 2>"$ERR" + timeout --kill-after=2 "$SECONDS_ARG" "$SCRIPT_DIR/fm-watch.sh" >"$OUT" 2>"$ERR" RC=$? elif command -v gtimeout >/dev/null 2>&1; then - gtimeout "$SECONDS_ARG" "$SCRIPT_DIR/fm-watch.sh" >"$OUT" 2>"$ERR" + gtimeout --kill-after=2 "$SECONDS_ARG" "$SCRIPT_DIR/fm-watch.sh" >"$OUT" 2>"$ERR" RC=$? else run_with_perl_timeout >"$OUT" 2>"$ERR" diff --git a/docs/architecture.md b/docs/architecture.md index b865f8be38..2693942862 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -76,7 +76,7 @@ For compatibility, default tmux tasks do not write `backend=tmux`; every reader `fm-watch.sh` polls each window's backend for a busy state: tmux, zellij, orca, and cmux have no native primitive and always report unknown, preserving the original pane-tail-regex detection unchanged; herdr's `agent.get` semantic state (working/idle/done/blocked) is consulted first for stale detection, with unknown native states falling back to the same regex. That poll loop is the default event source for backends with no native push events, so this stays an extraction of the abstraction rather than a watcher rewrite. For capable herdr sessions, the same watcher replaces its terminal sleep with a bounded native event wait that immediately surfaces `blocked`; [herdr-backend.md](herdr-backend.md#native-paneagent_status_changed-push-escalation-immediate-blocked-wake) owns the mechanism, capability gates, and verification evidence. -The deeper session-start agent-process liveness probe is separate from that busy-state poll: tmux and herdr have verified classifiers for secondmate recovery, while zellij, Orca, and cmux currently report `unknown` rather than guess. +The deeper session-start agent-process liveness probe is separate from that busy-state poll; [`docs/configuration.md`](configuration.md#runtime-backend-configbackend--fm_backend) owns the backend-specific verdicts. Herdr is experimental and can be selected explicitly or by runtime auto-detection: treehouse remains the worktree provider for it exactly as it is for tmux (herdr is a session provider only), and its full verification - the container shape decision, created-vs-adopted default-tab prune safety, restored-layout husk respawn idempotency, verified CLI facts, ANSI-preserved ghost/placeholder classification through the shared extractor, a verified small-`--lines` capture bug and its workaround, and known gaps - is recorded in `docs/herdr-backend.md`. Herdr's container shape is workspace-per-home plus tab-per-task: the primary home uses workspace label `firstmate`, secondmate homes use `2ndmate-`, and recovery/list-live scopes to the current `FM_HOME`'s workspace. Zellij is experimental and selected only explicitly: treehouse remains its worktree provider too, and its full verification - the resolved "gaps to verify" list from the original design report, the unconditional-exit-0 CLI quirk and its mitigation, the focus-steal-on-new-tab finding, the home-scoped tab-title collision fix, and known gaps - is recorded in `docs/zellij-backend.md`. diff --git a/docs/configuration.md b/docs/configuration.md index 22e20187d7..22335ac19e 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -37,7 +37,8 @@ Any value other than `tmux`, `herdr`, `zellij`, `orca`, or `cmux` is rejected un Every new task refuses `backend=orca` before any owned mutation, legacy respawn and destructive teardown currently fail closed on unavailable authority capability, and Orca also refuses `--secondmate`; the Orca guide owns the rationale and re-enablement contract. `codex-app` is not an accepted runtime backend yet; [`docs/codex-app-backend.md`](codex-app-backend.md) owns the Codex App boundary. The session-start secondmate liveness sweep uses a deeper `fm_backend_agent_alive` probe where verified. -Today that probe can classify tmux and herdr secondmate endpoints as `alive`, `dead`, or `unknown`; zellij, Orca, and cmux report `unknown` until their own agent-process classifiers are verified. +Today tmux and herdr can classify secondmate endpoints as `alive`, `dead`, or `unknown`. +Zellij reports `dead` only when the recorded session or pane is absent and otherwise reports `unknown`, while Orca and cmux report `unknown` until their own agent-process classifiers are verified. A herdr spawn additionally version-gates against the installed `herdr` binary's protocol and requires the Herdr tool delta owned by "Toolchain" below, refusing loudly on an incompatible or missing installation. A zellij spawn additionally version-gates against the installed `zellij` binary's version and requires `jq`, refusing loudly when either is missing or the version is older than 0.44. A cmux spawn additionally version-gates against the installed `cmux` binary's version, requires `jq`, and requires the control socket to be reachable and accessible (see [`docs/cmux-backend.md`](cmux-backend.md) "Setup" for the one-time socket-access configuration this needs; Automation mode is the recommended socket control mode, with Password mode supported via `config/cmux-socket-password`), refusing loudly and non-retryably on a `cmuxOnly`/unauthenticated socket. @@ -46,6 +47,8 @@ Task meta records `backend=` only for a non-default backend; an absent `backend= A herdr task additionally records `herdr_session=`, `herdr_workspace_id=`, `herdr_tab_id=`, and `herdr_pane_id=`. A zellij task additionally records `zellij_session=`, `zellij_tab_id=`, and `zellij_pane_id=`. A legacy Orca task may additionally record `orca_worktree_id=` and `terminal=`, with `window=fm-` kept as the shared firstmate alias. +Every new spawn records a generation-bound task temporary root beneath that home's physical state directory. +Respawn removes only a validated prior generation after the replacement metadata is committed, teardown removes the recorded current generation, and legacy `/tmp/fm-` metadata remains readable but is never mutated. A cmux task additionally records `cmux_workspace_id=` and `cmux_surface_id=`. Task selectors for `fm-peek.sh`, `fm-send.sh`, and `fm-crew-state.sh` resolve centrally through `fm_backend_resolve_selector`. A selector containing `:` is passed through as an explicit backend endpoint escape hatch. @@ -192,6 +195,8 @@ Neither mode invokes Agent Fleet selection or leases for a new ship/scout launch The existing `--account-pool`, `--account-profile`, and dispatch-profile fields remain compatibility activation inputs for those crewmates while the inactive-code removal is handled separately. Their legacy aliases do not constrain the new usage-based account choice. Secondmate integration is deferred: secondmate launches retain their pre-cutover Agent Fleet selection and lease behavior, including `config/secondmate-account-pool`. +Every secondmate launch refuses a home whose index tracks Agent Fleet routing support but whose working tree has removed it, even when routing is off. +An older home that does not track the support may still launch with a warning while routing is off or observe, but enforce mode refuses it until the home is reconciled. Firstmate's spawn flags and `config/secondmate-account-pool` continue to accept aliases made only of letters, digits, dot, underscore, and dash, excluding values that begin with dot or dash; `config/crew-dispatch.json` deliberately narrows those fields to an alphanumeric first character. Account email addresses and filesystem paths are invalid in every input surface. Direct ship/scout task metadata records only `account_home=` from this account mechanism. diff --git a/docs/zellij-backend.md b/docs/zellij-backend.md index 78553b44ac..f137db989f 100644 --- a/docs/zellij-backend.md +++ b/docs/zellij-backend.md @@ -31,7 +31,7 @@ You do not need to attach for routine supervision: from an active firstmate sess Verify it works by spawning a trivial task with `--backend zellij` and confirming the task's meta records `backend=zellij` plus `zellij_session=`, `zellij_tab_id=`, and `zellij_pane_id=`; attaching to the session should show the new home-scoped tab title, such as `fm-firstmate-<8hex>-`. -Limitations: zellij is experimental, has no per-home workspace split (all tasks share one tab bar, unlike herdr), has no verified agent-process liveness classifier for the session-start secondmate sweep, and still carries the known gaps documented below (no native busy-state signal and a narrow focus-steal race on tab creation) - see "Known gaps left for a follow-up" at the end of this document. +Limitations: zellij is experimental, has no per-home workspace split (all tasks share one tab bar, unlike herdr), can prove only structural endpoint absence rather than present-pane agent-process liveness for the session-start secondmate sweep, and still carries the known gaps documented below (no native busy-state signal and a narrow focus-steal race on tab creation) - see "Known gaps left for a follow-up" at the end of this document. ## Status: experimental @@ -103,7 +103,7 @@ Zellij tasks additionally record: | Send + submit, composed | `paste` then `send-keys --pane-id Enter` | Zellij has no single-call atomic "type and submit" primitive (unlike tmux's `send-keys ... Enter` or herdr's `pane run`); `fm_backend_zellij_send_text_line` composes the two calls, which is the only form this adapter has for that operation. | | Bounded capture | `zellij action dump-screen --pane-id ` for 40 lines or fewer; `zellij action dump-screen --pane-id --full` above that threshold | Works for a background session with NO attached client (resolves report gap #1). No `--lines`-style bound flag exists at all (unlike herdr's buggy small-N `--lines`, there is simply no flag). Routine watcher-sized reads use zellij's viewport-only dump to avoid unbounded scrollback reads; larger explicit peeks request `--full` and trim to the caller's requested line count locally with `tail`. The tradeoff: on a very short terminal viewport, a 40-line routine read can see fewer than 40 lines and miss content above the visible screen. | | Busy state | *(no native primitive)* | D5 (`herdr-addendum.md`): zellij has no agent-state API. `fm_backend_busy_state`'s dispatcher (`bin/fm-backend.sh`) falls through to `unknown` for zellij via its wildcard case, exactly like tmux - the watcher's existing pane-hash + regex path is the only busy-state source for this backend. | -| Agent liveness | *(no verified primitive)* | `fm_backend_agent_alive` reports `unknown` for zellij, so `bin/fm-bootstrap.sh`'s session-start secondmate liveness sweep never auto-respawns a zellij secondmate endpoint, conservatively avoiding a false-dead reading that would create a duplicate secondmate supervisor in one home. | +| Agent liveness | `list-panes --json` structural absence proof | `fm_backend_agent_alive` reports `dead` only when the recorded session or pane is absent and otherwise reports `unknown`, so `bin/fm-bootstrap.sh` can recover a structurally gone zellij secondmate endpoint without treating a present pane as proof about its agent process. | | Kill | `zellij action close-tab-by-id ` (tab id resolved fresh from the pane id when possible; teardown can pass recorded `zellij_tab_id` plus the expected caller-facing `fm-` label when the pane is already gone) | Unlike herdr (where closing a tab's only pane also closes the tab), closing a zellij pane with `close-pane` does NOT close the now-empty tab - it survives as an empty "ghost" entry in `list-tabs`. `close-tab-by-id` on a LIVE tab (with its pane still running) verified to cleanly remove both pane and tab in one call. Kill resolves the owning tab and closes by tab id; if teardown supplies an expected label, the tab id must still match it through the home-scoped-title or unambiguous legacy-title check before it is closed, including the recorded `zellij_tab_id` ghost-tab fallback. Best-effort (`\|\| true`), matching tmux's `kill-window` and herdr's `pane close` contract. | | Recovery / list-live | `zellij action list-tabs --json`, filter names starting with this home's own `fm--` prefix | Name-based, never trusts a stored pane id blindly - the same posture herdr's `list_live` takes. Scoped to this installation's own home-scoped prefix (see "Home-scoped tab titles" above), so it never lists another firstmate home's tabs; the adapter strips the tag back off and reports the plain `fm-` label. Does not attempt the legacy untagged-title fallback (that fallback is for a single already-known tab, not a bulk sweep). | | Session cleanup (test-only) | `zellij delete-session --force` | The single-call kill-and-delete form, gated behind `tests/zellij-test-safety.sh`'s guard (refuses an empty name, the literal `"firstmate"` default name, or a name not currently listed). Never `kill-all-sessions`/`delete-all-sessions` - see "Session safety" below. | @@ -215,8 +215,8 @@ The isolated zellij session and the scratch `FM_HOME`/project were fully torn do ## Known gaps left for a follow-up - **No event push at all**, not even herdr's semantic busy-state (D5): zellij has no analogue to herdr's `agent.get`, so `fm-watch.sh`'s existing pane-hash + `FM_BUSY_REGEX` poll loop is the ONLY event source for this backend, identical to the tmux path. This is the expected, designed-for outcome (D5 explicitly calls for "the poll-based capture/hash/busy-regex path, same vocabulary as tmux"), not a shortfall relative to the report. -- **No verified agent-process liveness classifier.** The session-start secondmate liveness sweep therefore receives `unknown` from `fm_backend_agent_alive` for zellij and reports `SECONDMATE_LIVENESS: ... skipped: liveness probe inconclusive` instead of killing or respawning the endpoint. - This leaves a dead zellij secondmate for manual recovery, but avoids the worse failure mode of duplicating a live supervisor. +- **No verified present-pane agent-process classifier.** The session-start secondmate liveness sweep receives `dead` only for a structurally absent zellij session or pane; a present pane remains `unknown` and produces `SECONDMATE_LIVENESS: ... skipped: liveness probe inconclusive` instead of risking a duplicate supervisor. + This leaves an agent-less but structurally present zellij secondmate for manual recovery, but avoids the worse failure mode of duplicating a live supervisor. - **The focus-steal mitigation has a narrow race window.** Between `new-tab` (which steals focus immediately) and the follow-up `go-to-tab-by-id` restore call, an attached client's view is briefly on the new tab. No flag-based suppression exists to close this window entirely (see "Focus-steal on new-tab" above); a future zellij release may add one. - **A pane can still die after `target_ready` succeeds and before the operation runs.** Metadata-routed operations now verify the expected caller-facing `fm-` label through the home-scoped-title or unambiguous legacy-title check, as well as the pane id up front, but zellij's unconditional exit 0 still leaves this narrow time-of-check/time-of-use race for one-shot operations (see "Unconditional exit code 0" above). - **The `pwd`-probe workaround for worktree-path discovery is scoped to `fm-spawn.sh`'s own poll loop only** (see "Worktree-path discovery" above). It is not a general-purpose live-cwd primitive; a future caller needing a live cwd read for a zellij pane outside that narrow spawn-time context would need the same active-probe approach, not a passive JSON field. diff --git a/tests/fm-account-directory.test.sh b/tests/fm-account-directory.test.sh index 96e496b525..be1d779862 100755 --- a/tests/fm-account-directory.test.sh +++ b/tests/fm-account-directory.test.sh @@ -6,7 +6,7 @@ set -u . "$(dirname "${BASH_SOURCE[0]}")/lib.sh" SELECTOR="$ROOT/bin/fm-account-directory.sh" -TMP_ROOT=$(fm_test_tmproot fm-account-directory-tests) +fm_test_tmproot_into TMP_ROOT fm-account-directory-tests ACCOUNT_ROOT="$TMP_ROOT/accounts" FAKEBIN=$(fm_fakebin "$TMP_ROOT") QUOTA_LOG="$TMP_ROOT/quota.log" @@ -230,6 +230,7 @@ case "${1:-}" in list-windows) exit 0 ;; has-session|new-session) exit 0 ;; new-window) + [ "${FM_FAKE_NEW_WINDOW_FAIL:-0}" != 1 ] || exit 73 touch "${FM_FAKE_ENDPOINT_FILE:?}" printf '@1\n' exit 0 @@ -239,6 +240,7 @@ case "${1:-}" in exit 0 ;; send-keys) + [ "${FM_FAKE_SEND_KEYS_FAIL:-0}" != 1 ] || exit 74 prev= for argument in "$@"; do if [ "$prev" = -l ]; then @@ -292,6 +294,8 @@ run_direct_spawn() { FM_SPAWN_NO_GUARD=1 FM_FAKE_PANE_PATH="$worktree" TMUX="fake,1,0" \ FM_FAKE_LAUNCH_LOG="$launch_log" FM_FAKE_ENDPOINT_FILE="$home/state/.fake-endpoint" \ FM_FAKE_ENDPOINT_LABEL="fm-${1:-unknown}" FM_FAKE_KILL_RETAIN="${FM_FAKE_KILL_RETAIN:-0}" \ + FM_FAKE_NEW_WINDOW_FAIL="${FM_FAKE_NEW_WINDOW_FAIL:-0}" \ + FM_FAKE_SEND_KEYS_FAIL="${FM_FAKE_SEND_KEYS_FAIL:-0}" \ FM_FAKE_HERDR_DRIFT_WORKTREE="${FM_FAKE_HERDR_DRIFT_WORKTREE:-}" \ T="${T:-}" \ FM_FAKE_TREEHOUSE_LOG="$TREEHOUSE_LOG" FM_FAKE_TREEHOUSE_WORKTREE="$worktree" \ @@ -315,7 +319,7 @@ make_spawn_case() { worktree="$case_dir/worktree" launch_log="$case_dir/launch.log" mkdir -p "$home/data/$id" "$home/projects" "$home/state" "$home/config" \ - "$home/treehouse-pools" + "$home/state/.task-tmp" "$home/treehouse-pools" printf '%s\n' "$harness" > "$home/config/crew-harness" printf 'brief for %s\n' "$id" > "$home/data/$id/brief.md" touch "$home/state/.last-watcher-beat" @@ -744,10 +748,8 @@ test_direct_recovery_tracks_retained_replacement_endpoint() { set_remaining 1 20,15 set_remaining 2 95,90 rm -f "$SPAWN_HOME/state/.fake-endpoint" - rm -rf "/tmp/fm-$id" - : > "/tmp/fm-$id" - if out=$(FM_FAKE_KILL_RETAIN=1 \ + if out=$(FM_FAKE_KILL_RETAIN=1 FM_FAKE_SEND_KEYS_FAIL=1 \ run_direct_spawn "$SPAWN_HOME" "$SPAWN_WORKTREE" "$SPAWN_LAUNCH_LOG" \ "$id" --recover-direct-account 2>&1); then status=0 @@ -765,7 +767,7 @@ test_direct_recovery_tracks_retained_replacement_endpoint() { [ -f "$SPAWN_HOME/state/$backup_name" ] || fail "failed recovery did not retain its prior metadata backup" [ -d "$SPAWN_HOME/state/$artifacts_name" ] || fail "failed recovery did not retain its artifact backup" - rm -f "/tmp/fm-$id" "$SPAWN_HOME/state/.fake-endpoint" + rm -f "$SPAWN_HOME/state/.fake-endpoint" retry=$(run_direct_spawn "$SPAWN_HOME" "$SPAWN_WORKTREE" "$SPAWN_LAUNCH_LOG" \ "$id" --recover-direct-account 2>&1) launch=$(cat "$SPAWN_LAUNCH_LOG") @@ -776,7 +778,6 @@ test_direct_recovery_tracks_retained_replacement_endpoint() { if grep -q '^direct_recovery_' "$meta"; then fail "successful retry left direct recovery markers in metadata"; fi [ ! -e "$SPAWN_HOME/state/$backup_name" ] || fail "successful retry left the retained metadata backup" [ ! -e "$SPAWN_HOME/state/$artifacts_name" ] || fail "successful retry left the retained artifact backup" - rm -rf "/tmp/fm-$id" pass "failed direct recovery tracks and reconciles a retained replacement endpoint" } @@ -788,9 +789,8 @@ test_new_direct_spawn_tracks_retained_endpoint_and_worktree() { id=direct-new-retained-z8 record=$(make_spawn_case direct-new-retained codex "$id") read_spawn_case "$record" - : > "/tmp/fm-$id" - if out=$(FM_FAKE_KILL_RETAIN=1 \ + if out=$(FM_FAKE_KILL_RETAIN=1 FM_FAKE_SEND_KEYS_FAIL=1 \ run_direct_spawn "$SPAWN_HOME" "$SPAWN_WORKTREE" "$SPAWN_LAUNCH_LOG" \ "$id" "$SPAWN_PROJECT" --account-pool legacy-codex-pool 2>&1); then status=0 @@ -827,7 +827,7 @@ test_new_direct_spawn_tracks_retained_endpoint_and_worktree() { [ "$recovery_status" -ne 0 ] || fail "direct recovery bypassed pending new-spawn cleanup" assert_contains "$recovery_out" "failed direct spawn cleanup is pending" \ "pending new-spawn cleanup refusal was not actionable" - rm -f "/tmp/fm-$id" "$SPAWN_HOME/state/.fake-endpoint" + rm -f "$SPAWN_HOME/state/.fake-endpoint" pass "new direct spawn tracks retained endpoint and worktree state" } @@ -840,9 +840,9 @@ test_failed_new_direct_spawn_returns_worktree_after_endpoint_cleanup() { read_spawn_case "$record" recorded_worktree=$(cd "$SPAWN_WORKTREE" && pwd -P) : > "$TREEHOUSE_LOG" - : > "/tmp/fm-$id" - if out=$(run_direct_spawn "$SPAWN_HOME" "$SPAWN_WORKTREE" "$SPAWN_LAUNCH_LOG" \ + if out=$(FM_FAKE_SEND_KEYS_FAIL=1 \ + run_direct_spawn "$SPAWN_HOME" "$SPAWN_WORKTREE" "$SPAWN_LAUNCH_LOG" \ "$id" "$SPAWN_PROJECT" --account-pool legacy-codex-pool 2>&1); then status=0 else @@ -855,7 +855,6 @@ test_failed_new_direct_spawn_returns_worktree_after_endpoint_cleanup() { [ ! -e "$SPAWN_WORKTREE" ] || fail "failed new direct spawn left its worktree registered" [ ! -e "$SPAWN_HOME/state/$id.meta" ] || fail "successful direct rollback left task metadata" [ ! -e "$SPAWN_HOME/state/.fake-endpoint" ] || fail "successful direct rollback left its endpoint" - rm -f "/tmp/fm-$id" pass "failed new direct spawn removes its endpoint and returns its worktree" } @@ -867,9 +866,8 @@ test_failed_new_direct_spawn_retains_cleanup_when_worktree_return_fails() { record=$(make_spawn_case direct-new-return-fail codex "$id") read_spawn_case "$record" : > "$TREEHOUSE_LOG" - : > "/tmp/fm-$id" - if out=$(FM_FAKE_TREEHOUSE_RETURN_FAIL=1 \ + if out=$(FM_FAKE_TREEHOUSE_RETURN_FAIL=1 FM_FAKE_SEND_KEYS_FAIL=1 \ run_direct_spawn "$SPAWN_HOME" "$SPAWN_WORKTREE" "$SPAWN_LAUNCH_LOG" \ "$id" "$SPAWN_PROJECT" --account-pool legacy-codex-pool 2>&1); then status=0 @@ -884,7 +882,6 @@ test_failed_new_direct_spawn_retains_cleanup_when_worktree_return_fails() { assert_grep "rollback_pending=1" "$meta" \ "direct return failure did not fail closed" [ ! -e "$SPAWN_HOME/state/.fake-endpoint" ] || fail "direct return failure retained an already-removed endpoint" - rm -f "/tmp/fm-$id" pass "direct spawn persists cleanup state when worktree return cannot be confirmed" } @@ -896,11 +893,11 @@ test_failed_new_direct_spawn_never_records_an_uncreated_endpoint() { record=$(make_spawn_case direct-new-no-endpoint codex "$id") read_spawn_case "$record" : > "$TREEHOUSE_LOG" - : > "/tmp/fm-$id" if out=$(T=default:captain-pane FM_FAKE_TREEHOUSE_RETURN_FAIL=1 \ + FM_FAKE_NEW_WINDOW_FAIL=1 \ run_direct_spawn "$SPAWN_HOME" "$SPAWN_WORKTREE" "$SPAWN_LAUNCH_LOG" \ - "$id" "$SPAWN_PROJECT" --backend herdr --account-pool legacy-codex-pool 2>&1); then + "$id" "$SPAWN_PROJECT" --account-pool legacy-codex-pool 2>&1); then status=0 else status=$? @@ -919,7 +916,6 @@ test_failed_new_direct_spawn_never_records_an_uncreated_endpoint() { assert_no_grep "herdr_pane_id=" "$meta" \ "pre-endpoint failure copied an ambient Herdr pane identity" [ ! -e "$SPAWN_HOME/state/.fake-endpoint" ] || fail "pre-endpoint failure created an endpoint" - rm -f "/tmp/fm-$id" pass "failed direct spawn never records an endpoint it did not create" } diff --git a/tests/fm-account-routing.test.sh b/tests/fm-account-routing.test.sh index 7e795905ff..ad7b554946 100755 --- a/tests/fm-account-routing.test.sh +++ b/tests/fm-account-routing.test.sh @@ -16,7 +16,7 @@ SEND="$ROOT/bin/fm-send.sh" TEARDOWN="$ROOT/bin/fm-teardown.sh" SESSION_SYNC="$ROOT/bin/fm-account-session-sync.sh" CONTINUATION="$ROOT/bin/fm-account-continuation.sh" -TMP_ROOT=$(fm_test_tmproot fm-account-routing-tests) +fm_test_tmproot_into TMP_ROOT fm-account-routing-tests assert_not_grep() { local pattern=$1 file=$2 label=$3 @@ -237,14 +237,37 @@ if [ "${1:-}" = return ] && [ -n "${FM_FAKE_TREEHOUSE_RETURN_CHILD_PID_FILE:-}" fi if [ "${1:-}" = get ]; then lease_holder= - previous= - for argument in "$@"; do - [ "$previous" != --lease-holder ] || lease_holder=$argument - previous=$argument + prev= + for arg in "$@"; do + if [ "$prev" = --lease-holder ]; then + lease_holder=$arg + break + fi + prev=$arg done - pool=$(dirname "$(dirname "${FM_FAKE_TREEHOUSE_PATH:?}")") - printf '{"worktrees":[{"path":"%s","leased":true,"lease_holder":"%s","destroying":false}]}\n' \ - "$FM_FAKE_TREEHOUSE_PATH" "$lease_holder" > "$pool/treehouse-state.json" + if [ -n "${FM_FAKE_TREEHOUSE_STATE:-}" ] && [ -n "$lease_holder" ]; then + python3 - "$FM_FAKE_TREEHOUSE_STATE" "${FM_FAKE_TREEHOUSE_PATH:?}" "$lease_holder" <<'PY' +import json +import os +import sys + +state, path, holder = sys.argv[1:] +with open(state, "w", encoding="utf-8") as stream: + json.dump( + { + "worktrees": [ + { + "name": "1", + "path": os.path.realpath(path), + "leased": True, + "lease_holder": holder, + } + ] + }, + stream, + ) +PY + fi printf '%s\n' "${FM_FAKE_TREEHOUSE_PATH:?}" fi exit 0 @@ -310,6 +333,7 @@ case "$*" in [ -n "$pool" ] || pool=${FM_FAKE_AF_POOL:-claude-crew} case "$*" in *" lease recover "*) + [ -z "${FM_FAKE_AF_RECOVER_MARKER:-}" ] || touch "$FM_FAKE_AF_RECOVER_MARKER" [ "${FM_FAKE_AF_RECOVER_FAIL:-0}" != 1 ] || exit "${FM_FAKE_AF_RECOVER_FAIL_STATUS:-42}" if [ -n "${FM_FAKE_AF_RECOVER_FAIL_ONCE_FILE:-}" ] && [ ! -e "$FM_FAKE_AF_RECOVER_FAIL_ONCE_FILE" ]; then : > "$FM_FAKE_AF_RECOVER_FAIL_ONCE_FILE" @@ -320,6 +344,7 @@ case "$*" in [ "${FM_FAKE_AF_STALE_REFRESH_ON_RECOVER:-0}" != 1 ] || touch "${FM_FAKE_AF_SESSION_REFRESHED:?}" ;; *" lease choose "*|*" lease acquire "*) + [ -z "${FM_FAKE_AF_SELECT_MARKER:-}" ] || touch "$FM_FAKE_AF_SELECT_MARKER" [ -z "${FM_FAKE_AF_CHOOSE_FAIL_STATUS:-}" ] || exit "$FM_FAKE_AF_CHOOSE_FAIL_STATUS" [ -z "${FM_FAKE_AF_SELECT_SLEEP:-}" ] || sleep "$FM_FAKE_AF_SELECT_SLEEP" [ -z "${FM_FAKE_AF_SELECT_COMPLETED:-}" ] || touch "$FM_FAKE_AF_SELECT_COMPLETED" @@ -385,6 +410,7 @@ case "$*" in printf '{"schema":%s,"task":"%s","profile":"%s","provider":"%s","pool":"%s","workspace":"%s","session_id":"sess-%s","session_event_seq":%s,"updated_at":"%s"}\n' "$session_schema" "$task" "$profile" "$provider" "$pool" "$workspace" "$task" "$session_event_seq" "$updated_at" ;; *" lease release "*) + [ -z "${FM_FAKE_AF_RELEASE_START_MARKER:-}" ] || touch "$FM_FAKE_AF_RELEASE_START_MARKER" [ -z "${FM_FAKE_AF_RELEASE_SLEEP:-}" ] || sleep "$FM_FAKE_AF_RELEASE_SLEEP" [ -z "${FM_FAKE_AF_RELEASE_COMPLETED:-}" ] || touch "$FM_FAKE_AF_RELEASE_COMPLETED" [ -z "${FM_FAKE_AF_RELEASE_MARKER:-}" ] || touch "$FM_FAKE_AF_RELEASE_MARKER" @@ -424,18 +450,30 @@ make_case() { case_dir="$TMP_ROOT/$name" home="$case_dir/home" proj="$case_dir/project" - wt="$case_dir/treehouse-pool/1/wt" + wt="$case_dir/treehouse-root/pool/1/wt" fakebin=$(make_fakebin "$case_dir/fake") mkdir -p "$home/data" "$home/projects" "$home/state" "$home/config" \ - "$case_dir/treehouse-pool/1" "$case_dir/treehouse-pools" + "$case_dir/treehouse-root/pool" "$case_dir/treehouse-pools" printf '%s\n' "$harness" > "$home/config/crew-harness" fm_git_worktree "$proj" "$wt" "wt-$name" git -C "$wt" checkout --quiet --detach fm_git_add_origin "$proj" "$case_dir/origin.git" - if [ -n "$first_id" ]; then - printf '{"worktrees":[{"path":"%s","leased":true,"lease_holder":"firstmate-%s"}]}\n' \ - "$wt" "$first_id" > "$case_dir/treehouse-pool/treehouse-state.json" - fi + python3 - "$case_dir/treehouse-root/pool/treehouse-state.json" "$wt" "$first_id" <<'PY' +import json +import os +import sys + +state, path, first_id = sys.argv[1:] +worktree = { + "name": "1", + "path": os.path.realpath(path), + "leased": bool(first_id), +} +if first_id: + worktree["lease_holder"] = f"firstmate-{first_id}" +with open(state, "w", encoding="utf-8") as stream: + json.dump({"worktrees": [worktree]}, stream) +PY touch "$home/state/.last-watcher-beat" for id in "$@"; do mkdir -p "$home/data/$id" @@ -475,7 +513,8 @@ run_spawn() { FM_FAKE_TREEHOUSE_LOG="$TREEHOUSE_LOG" FM_FAKE_LIFECYCLE_LOG="$LIFECYCLE_LOG" \ FM_EXPECT_CHECKOUT_LOCK_ROOT="${FM_EXPECT_CHECKOUT_LOCK_ROOT:-}" \ FM_EXPECT_CHECKOUT_LOCK_MARKER="${FM_EXPECT_CHECKOUT_LOCK_MARKER:-}" \ - FM_FAKE_TREEHOUSE_PATH="$WT_DIR" FM_TREEHOUSE_ROOT="$CASE_DIR/treehouse-pools" \ + FM_FAKE_TREEHOUSE_PATH="$WT_DIR" FM_FAKE_TREEHOUSE_STATE="$CASE_DIR/treehouse-root/pool/treehouse-state.json" \ + FM_TREEHOUSE_ROOT="$CASE_DIR/treehouse-root" \ FM_FAKE_TREEHOUSE_SLEEP="${FM_FAKE_TREEHOUSE_SLEEP:-}" \ FM_FAKE_TREEHOUSE_RETURN_SLEEP="${FM_FAKE_TREEHOUSE_RETURN_SLEEP:-}" \ FM_FAKE_TREEHOUSE_RETURN_CHILD_PID_FILE="${FM_FAKE_TREEHOUSE_RETURN_CHILD_PID_FILE:-}" \ @@ -501,6 +540,7 @@ run_teardown() { FM_ROOT_OVERRIDE="${FM_TEST_ROOT_OVERRIDE:-}" FM_HOME="$HOME_DIR" \ FM_STATE_OVERRIDE="$HOME_DIR/state" FM_DATA_OVERRIDE="$HOME_DIR/data" \ FM_PROJECTS_OVERRIDE="$HOME_DIR/projects" FM_CONFIG_OVERRIDE="$HOME_DIR/config" \ + FM_REPORT_STACK_ROOT="$CASE_DIR/report-stack" \ FM_FAKE_TMUX_LOG="$TMUX_LOG" FM_FAKE_AF_LOG="$AF_LOG" \ FM_FAKE_TREEHOUSE_LOG="$TREEHOUSE_LOG" FM_FAKE_ENDPOINT_FILE="$CASE_DIR/endpoint-live" \ FM_FAKE_TREEHOUSE_RETURN_SLEEP="${FM_FAKE_TREEHOUSE_RETURN_SLEEP:-}" \ @@ -552,7 +592,7 @@ use_named_fake_tmux_target() { } test_off_is_byte_compatible_and_never_calls_agent_fleet() { - local id rec out status launch expected + local id rec out status launch expected task_tmp generation token id=account-off-z1 rec=$(make_case off claude "$id") read_case "$rec" @@ -565,6 +605,13 @@ test_off_is_byte_compatible_and_never_calls_agent_fleet() { [ "$launch" = "$expected" ] || fail "routing off changed the launch bytes" assert_not_grep '^account_' "$HOME_DIR/state/$id.meta" "routing off wrote account metadata" assert_not_grep '^provider_session_id=' "$HOME_DIR/state/$id.meta" "routing off wrote session metadata" + task_tmp=$(sed -n 's/^tasktmp=//p' "$HOME_DIR/state/$id.meta") + generation=$(sed -n 's/^generation_id=//p' "$HOME_DIR/state/$id.meta") + token=${generation##*:} + [ "$task_tmp" = "$HOME_DIR/state/.task-tmp/fm-$id-$token" ] \ + || fail "routing fixture did not namespace its task temp root to this spawn generation" + assert_present "$task_tmp/gotmp" \ + "routing fixture did not create its run-scoped Go temp root" assert_regex '^generation_id=spawn:a[0-9a-f]{15}$' "$HOME_DIR/state/$id.meta" "routing off did not record a stable spawn generation" assert_grep 'report_required=1' "$HOME_DIR/state/$id.meta" "post-cutover spawn did not activate the report gate" assert_grep '# Completion report' "$HOME_DIR/data/$id/brief.md" "post-cutover spawn did not upgrade a legacy unspawned brief" @@ -704,9 +751,13 @@ test_changed_acquisition_is_retained_during_unmanaged_rollback() { FM_FAKE_TMUX_FAIL_SEND_MATCH=GOTMPDIR \ run_spawn "$id" "$PROJ_DIR" > "$out_file" & spawn_pid=$! - for _ in $(seq 1 600); do [ -f "$marker" ] && break; sleep 0.05; done + for _ in $(seq 1 600); do + [ -f "$marker" ] && break + kill -0 "$spawn_pid" 2>/dev/null || break + sleep 0.05 + done [ -f "$marker" ] \ - || { kill "$spawn_pid" 2>/dev/null || true; fail "changed-acquisition test never reached the post-install failure"; } + || { touch "$release"; wait "$spawn_pid" 2>/dev/null || true; fail "changed-acquisition test never reached the post-install failure"; } printf '%s\n' retained-commit > "$WT_DIR/retained-commit.txt" git -C "$WT_DIR" add retained-commit.txt git -C "$WT_DIR" -c user.name='Firstmate Tests' -c user.email='tests@example.invalid' \ @@ -801,7 +852,7 @@ SH common=$(git -C "$WT_DIR" rev-parse --git-common-dir) case "$common" in /*) ;; *) common="$WT_DIR/$common" ;; esac common=$(cd "$common" && pwd -P) - key=$(printf '%s' "$common" | shasum -a 256 | awk '{print substr($1,1,24)}') + key=$(printf '%s' "$common" | tr '[:upper:]' '[:lower:]' | shasum -a 256 | awk '{print substr($1,1,24)}') lock="$CASE_DIR/checkout-refresh-locks/$key.lock" assert_present "$CASE_DIR/treehouse-return-child.pid" \ "spawn rollback did not exercise a surviving Treehouse descendant" @@ -1351,8 +1402,16 @@ test_off_metadata_merge_waits_for_metadata_lock() { FM_FAKE_TMUX_NEW_WINDOW_MARKER="$marker" FM_FAKE_TMUX_NEW_WINDOW_GATE="$gate" \ run_spawn "$id" "$PROJ_DIR" > "$out_file" & spawn_pid=$! - for _ in $(seq 1 100); do [ -f "$marker" ] && break; sleep 0.05; done - [ -f "$marker" ] || { kill "$spawn_pid" 2>/dev/null || true; fail "off metadata-lock test never reached endpoint creation"; } + for _ in $(seq 1 600); do + [ -f "$marker" ] && break + kill -0 "$spawn_pid" 2>/dev/null || break + sleep 0.05 + done + [ -f "$marker" ] || { + touch "$gate" + wait "$spawn_pid" 2>/dev/null || true + fail "off metadata-lock test never reached endpoint creation: $(cat "$out_file")" + } # shellcheck source=bin/fm-account-routing-lib.sh . "$ROOT/bin/fm-account-routing-lib.sh" held=$(fm_account_meta_lock_acquire "$HOME_DIR/state" "$id") \ @@ -1462,10 +1521,12 @@ test_unmanaged_spawn_refuses_while_teardown_lifecycle_is_held() { } test_unmanaged_respawn_preserves_report_cutover_state() { - local id rec out status + local id rec out status prior_task_tmp id=account-legacy-respawn-z9b rec=$(make_case legacy-respawn claude "$id") read_case "$rec" + prior_task_tmp="$HOME_DIR/state/.task-tmp/fm-$id-a111111111111111" + mkdir -p "$prior_task_tmp/gotmp" fm_write_meta "$HOME_DIR/state/$id.meta" \ "window=firstmate:fm-$id" \ "worktree=$WT_DIR" \ @@ -1473,6 +1534,8 @@ test_unmanaged_respawn_preserves_report_cutover_state() { "harness=claude" \ "kind=ship" \ "mode=no-mistakes" \ + "tasktmp=$prior_task_tmp" \ + "generation_id=spawn:a111111111111111" \ "pr=418" \ "x_request=req-legacy" \ "custom_extension=preserve-success" @@ -1483,14 +1546,17 @@ test_unmanaged_respawn_preserves_report_cutover_state() { assert_grep 'pr=418' "$HOME_DIR/state/$id.meta" "managed respawn dropped the existing PR pointer" assert_grep 'x_request=req-legacy' "$HOME_DIR/state/$id.meta" "managed respawn dropped the existing X-mode link" assert_grep 'custom_extension=preserve-success' "$HOME_DIR/state/$id.meta" "managed respawn dropped extension metadata" + assert_absent "$prior_task_tmp" "successful respawn orphaned the prior task temp generation" pass "unmanaged respawn preserves a legacy task's report cutover state" } test_failed_managed_respawn_restores_unmanaged_metadata() { - local id rec expected out status artifact + local id rec expected out status artifact prior_task_tmp id=account-unmanaged-rollback-z9c rec=$(make_case unmanaged-rollback claude "$id") read_case "$rec" + prior_task_tmp="$HOME_DIR/state/.task-tmp/fm-$id-a222222222222222" + mkdir -p "$prior_task_tmp/gotmp" fm_write_meta "$HOME_DIR/state/$id.meta" \ "window=firstmate:fm-$id" \ "worktree=$WT_DIR" \ @@ -1498,6 +1564,8 @@ test_failed_managed_respawn_restores_unmanaged_metadata() { "harness=claude" \ "kind=ship" \ "mode=no-mistakes" \ + "tasktmp=$prior_task_tmp" \ + "generation_id=spawn:a222222222222222" \ "pr=417" \ "custom_extension=preserve-me" expected="$CASE_DIR/original.meta" @@ -1519,10 +1587,163 @@ test_failed_managed_respawn_restores_unmanaged_metadata() { || fail "failed managed respawn did not restore prior $artifact state" done assert_grep 'lease release ' "$AF_LOG" "failed managed respawn leaked its acquired reservation" + assert_present "$prior_task_tmp/gotmp" "failed respawn destroyed the prior task temp generation" [ -n "$out" ] || true pass "failed managed respawn restores every field from existing unmanaged metadata" } +test_task_tmp_removal_refuses_symlinked_parent() { + local id generation state outside target + id=account-tasktmp-parent-z9d + generation=spawn:a333333333333333 + state="$TMP_ROOT/tasktmp-parent/state" + outside="$TMP_ROOT/tasktmp-parent/outside" + target="$state/.task-tmp/fm-$id-a333333333333333" + mkdir -p "$state" "$outside/${target##*/}" + ln -s "$outside" "$state/.task-tmp" + STATE=$state + . "$ROOT/bin/fm-account-routing-lib.sh" + if fm_account_safe_remove_task_tmp "$id" "$target" "$generation" >/dev/null 2>&1; then + fail "task temp removal followed a symlinked parent" + fi + assert_present "$outside/${target##*/}" "task temp removal deleted through a symlinked parent" + pass "task temp removal refuses a symlinked parent" +} + +test_task_tmp_removal_pins_ancestors_during_swap() { + local id generation root state moved outside target name + id=account-tasktmp-swap-z9e + generation=spawn:a444444444444444 + root="$TMP_ROOT/tasktmp-swap" + state="$root/state" + moved="$root/state-moved" + outside="$root/outside" + name="fm-$id-a444444444444444" + target="$state/.task-tmp/$name" + mkdir -p "$target/gotmp" "$outside/.task-tmp/$name/gotmp" + printf 'outside\n' > "$outside/.task-tmp/$name/gotmp/sentinel" + STATE=$state + . "$ROOT/bin/fm-account-routing-lib.sh" + FM_ACCOUNT_TEST_HOOKS=firstmate-account-tests-v1 \ + FM_SAFE_TASK_TMP_SWAP_ANCESTOR="$state" \ + FM_SAFE_TASK_TMP_SWAP_MOVED="$moved" \ + FM_SAFE_TASK_TMP_SWAP_OUTSIDE="$outside" \ + fm_account_safe_remove_task_tmp "$id" "$target" "$generation" \ + || fail "task temp removal lost its pinned ancestor during swap" + assert_absent "$moved/.task-tmp/$name" "task temp removal missed its pinned original directory" + assert_present "$outside/.task-tmp/$name/gotmp/sentinel" "task temp removal followed a swapped ancestor" + pass "task temp removal pins every ancestor during deletion" +} + +test_task_tmp_partial_removal_fails_closed() { + local id generation state target + id=account-tasktmp-partial-z9f + generation=spawn:a555555555555555 + state="$TMP_ROOT/tasktmp-partial/state" + target="$state/.task-tmp/fm-$id-a555555555555555" + mkdir -p "$target" + printf 'vanish\n' > "$target/disappearing" + printf 'retain\n' > "$target/retained" + STATE=$state + . "$ROOT/bin/fm-account-routing-lib.sh" + if FM_ACCOUNT_TEST_HOOKS=firstmate-account-tests-v1 \ + FM_SAFE_TASK_TMP_DISAPPEAR_ENTRY=disappearing \ + fm_account_safe_remove_task_tmp "$id" "$target" "$generation" >/dev/null 2>&1; then + fail "partial task temp removal reported success" + fi + assert_present "$target/retained" "partial task temp removal discarded the remaining root" + pass "partial task temp removal fails closed" +} + +test_spawn_rollback_task_tmp_refusal_is_retryable() { + local mode id rec marker release output ancestor moved outside tasktmp spawn_pid status tries + for mode in direct managed; do + id="account-tasktmp-rollback-$mode-z9g" + rec=$(make_case "tasktmp-rollback-$mode" claude "$id") + read_case "$rec" + marker="$CASE_DIR/gotmp-send-started" + release="$CASE_DIR/gotmp-send-release" + output="$CASE_DIR/spawn.out" + ancestor="$HOME_DIR/state/.task-tmp" + moved="$HOME_DIR/state/task-tmp-$mode-moved" + outside="$HOME_DIR/state/task-tmp-$mode-outside" + mkdir -p "$outside" + + if [ "$mode" = direct ]; then + mkdir -p "$CASE_DIR/accounts/claude/account-1" + cat > "$FAKEBIN_DIR/herdr" <<'SH' +#!/usr/bin/env bash +set -u +[ "${1:-}" = integration ] && [ "${2:-}" = install ] && [ "${3:-}" = claude ] || exit 64 +mkdir -p "${CLAUDE_CONFIG_DIR:?}/hooks" +printf '#!/usr/bin/env bash\n' > "$CLAUDE_CONFIG_DIR/hooks/herdr-agent-state.sh" +SH + chmod +x "$FAKEBIN_DIR/herdr" + FM_ACCOUNT_ROUTING_LEGACY_NEW_LAUNCH_TEST='' \ + FM_ACCOUNT_DIRECTORY_TEST_LAB=firstmate-account-directory-test-lab-v1 \ + FM_ACCOUNT_DIRECTORY_ROOT="$CASE_DIR/accounts" \ + FM_ACCOUNT_DIRECTORY_HERDR="$FAKEBIN_DIR/herdr" \ + FM_ACCOUNT_TEST_HOOKS=firstmate-account-tests-v1 \ + FM_SAFE_TASK_TMP_SWAP_ANCESTOR="$ancestor" \ + FM_SAFE_TASK_TMP_SWAP_MOVED="$moved" \ + FM_SAFE_TASK_TMP_SWAP_OUTSIDE="$outside" \ + FM_SAFE_TASK_TMP_DISAPPEAR_ENTRY=gotmp \ + FM_FAKE_TMUX_GATE_SEND_MATCH=GOTMPDIR \ + FM_FAKE_TMUX_GATE_SEND_MARKER="$marker" \ + FM_FAKE_TMUX_GATE_SEND_RELEASE="$release" \ + FM_FAKE_TMUX_FAIL_SEND_MATCH=GOTMPDIR \ + run_spawn "$id" "$PROJ_DIR" --account-pool claude-crew > "$output" & + else + FM_ACCOUNT_TEST_HOOKS=firstmate-account-tests-v1 \ + FM_SAFE_TASK_TMP_SWAP_ANCESTOR="$ancestor" \ + FM_SAFE_TASK_TMP_SWAP_MOVED="$moved" \ + FM_SAFE_TASK_TMP_SWAP_OUTSIDE="$outside" \ + FM_SAFE_TASK_TMP_DISAPPEAR_ENTRY=gotmp \ + FM_FAKE_TMUX_GATE_SEND_MATCH=GOTMPDIR \ + FM_FAKE_TMUX_GATE_SEND_MARKER="$marker" \ + FM_FAKE_TMUX_GATE_SEND_RELEASE="$release" \ + FM_FAKE_TMUX_FAIL_SEND_MATCH=GOTMPDIR \ + run_spawn "$id" "$PROJ_DIR" --account-pool claude-crew > "$output" & + fi + spawn_pid=$! + tries=0 + while [ ! -f "$marker" ] && kill -0 "$spawn_pid" 2>/dev/null && [ "$tries" -lt 200 ]; do + sleep 0.05 + tries=$((tries + 1)) + done + if [ ! -f "$marker" ]; then + touch "$release" + wait "$spawn_pid" 2>/dev/null || true + fail "$mode rollback task temp test never reached its post-metadata failure" + fi + tasktmp=$(sed -n 's/^tasktmp=//p' "$HOME_DIR/state/$id.meta" | tail -1) + [ -n "$tasktmp" ] || { + touch "$release" + wait "$spawn_pid" 2>/dev/null || true + fail "$mode rollback task temp test did not publish provisional metadata" + } + mkdir -p "$outside/${tasktmp##*/}" + printf 'preserve\n' > "$outside/${tasktmp##*/}/sentinel" + touch "$release" + if wait "$spawn_pid"; then status=0; else status=$?; fi + [ "$status" -ne 0 ] || fail "$mode rollback task temp refusal unexpectedly succeeded" + assert_present "$outside/${tasktmp##*/}/sentinel" \ + "$mode rollback followed the swapped task temp ancestor" + assert_present "$moved/${tasktmp##*/}" \ + "$mode rollback discarded its partially cleaned task temp generation" + assert_contains "$(cat "$output")" "retaining cleanup metadata" \ + "$mode rollback did not report retryable task temp cleanup" + if [ "$mode" = direct ]; then + assert_regex '^direct_spawn_cleanup=pending$' "$HOME_DIR/state/$id.meta" \ + "direct rollback task temp refusal lost retry metadata" + else + assert_regex '^account_rollback_cleanup=pending$' "$HOME_DIR/state/$id.meta" \ + "managed rollback task temp refusal lost retry metadata" + fi + done + pass "spawn rollback task temp refusals preserve outside data and retry state" +} + test_preinstall_managed_failure_restores_artifact_snapshot() { local id rec expected out status artifact id=account-preinstall-rollback-z9f @@ -1633,7 +1854,7 @@ test_session_sync_all_bounds_each_task_and_reaches_later_mappings() { out=$(FM_ROOT_OVERRIDE="$ROOT" FM_HOME="$HOME_DIR" FM_STATE_OVERRIDE="$HOME_DIR/state" \ FM_DATA_OVERRIDE="$HOME_DIR/data" FM_AGENT_FLEET_BIN="$FAKEBIN_DIR/agent-fleet" \ FM_FAKE_AF_LOG="$AF_LOG" FM_FAKE_AF_SESSION_SLEEP=10 FM_FAKE_AF_SESSION_SLEEP_TASK="$first" \ - FM_ACCOUNT_SESSION_QUERY_TIMEOUT=1 FM_ACCOUNT_SESSION_TASK_TIMEOUT=2 \ + FM_ACCOUNT_SESSION_QUERY_TIMEOUT=1 FM_ACCOUNT_SESSION_TASK_TIMEOUT=3 \ PATH="$FAKEBIN_DIR:$PATH" "$SESSION_SYNC" --all 2>&1) status=$? [ "$status" -ne 0 ] || fail "sync-all hid the timed-out first mapping" @@ -2219,14 +2440,16 @@ test_enforced_secondmate_requires_routing_inheritance_and_capable_home() { rec=$(make_case secondmate-incapable-refuse claude) read_case "$rec" sm="$CASE_DIR/secondmate-home" - make_seeded_secondmate_home "$sm" "$id" incapable + make_seeded_secondmate_home "$sm" "$id" sm=$(cd "$sm" && pwd -P) + git -C "$sm" update-index --assume-unchanged bin/fm-account-routing-lib.sh + rm -f "$sm/bin/fm-account-routing-lib.sh" printf 'enforce\n' > "$HOME_DIR/config/account-routing-mode" out=$(FM_TEST_PANE_PATH="$sm" run_spawn "$id" "$sm" --secondmate) status=$? [ "$status" -ne 0 ] || fail "enforced secondmate launched from a pre-Agent-Fleet home" assert_contains "$out" "$sm" "capability refusal omitted the offending secondmate home" - assert_contains "$out" "lacks Agent Fleet routing support" \ + assert_contains "$out" "missing tracked Agent Fleet routing support" \ "capability refusal did not stop at the capability gate" assert_not_grep '^new-window ' "$TMUX_LOG" "capability refusal created an endpoint" pass "enforced secondmates require inherited routing policy and Agent Fleet-capable homes" @@ -2467,7 +2690,11 @@ SH kill -0 "$spawn_pid" 2>/dev/null || break sleep 0.05 done - [ -f "$marker" ] || { touch "$gate" "$installed_gate"; wait "$spawn_pid" 2>/dev/null || true; fail "spawn never persisted provisional managed metadata"; } + [ -f "$marker" ] || { + touch "$gate" "$installed_gate" + wait "$spawn_pid" 2>/dev/null || true + fail "spawn never persisted provisional managed metadata: $(cat "$CASE_DIR/spawn-stdout" "$CASE_DIR/spawn-stderr")" + } meta="$HOME_DIR/state/$id.meta" task=$(meta_account_task "$id") assert_grep 'account_rollback_cleanup=pending' "$meta" "provisional metadata was not marked for rollback recovery" @@ -2483,7 +2710,11 @@ SH kill -0 "$spawn_pid" 2>/dev/null || break sleep 0.05 done - [ -f "$installed_marker" ] || { touch "$installed_gate"; wait "$spawn_pid" 2>/dev/null || true; fail "spawn never installed endpoint metadata"; } + [ -f "$installed_marker" ] || { + touch "$installed_gate" + wait "$spawn_pid" 2>/dev/null || true + fail "spawn never installed endpoint metadata: $(cat "$CASE_DIR/spawn-stdout" "$CASE_DIR/spawn-stderr")" + } assert_grep 'account_rollback_cleanup=pending' "$meta" "endpoint metadata cleared rollback recovery before launch commit" assert_grep 'account_profile=claude-2' "$meta" "endpoint metadata lost the selected account profile" assert_grep "tmux_session_target=firstmate:fm-$id" "$meta" \ @@ -3175,24 +3406,31 @@ test_concurrent_continuations_serialize_before_mutation() { FM_FAKE_AF_PROFILE=claude-3 FM_FAKE_AF_POOL=explicit FM_FAKE_TMUX_NEW_WINDOW_MARKER="$marker" FM_FAKE_TMUX_NEW_WINDOW_GATE="$gate" \ run_spawn "$id" --continue-account --account-profile claude-3 > "$CASE_DIR/first.out" 2>&1 & first_pid=$! - for _ in $(seq 1 100); do + for _ in $(seq 1 600); do [ -f "$marker" ] && break + kill -0 "$first_pid" 2>/dev/null || break sleep 0.05 done - [ -f "$marker" ] || { kill "$first_pid" 2>/dev/null || true; fail "first continuation never reached endpoint creation"; } + [ -f "$marker" ] || { + touch "$gate" + wait "$first_pid" 2>/dev/null || true + fail "first continuation never reached endpoint creation: $(cat "$CASE_DIR/first.out")" + } second_lock_waiter="$CASE_DIR/second-lock-wait-observed" FM_FAKE_AF_PROFILE=claude-3 FM_FAKE_AF_POOL=explicit \ FM_ACCOUNT_TEST_HOOKS=firstmate-account-tests-v1 FM_ACCOUNT_LOCK_WAIT_TEST_OBSERVED="$second_lock_waiter" \ run_spawn "$id" --continue-account --account-profile claude-3 > "$CASE_DIR/second.out" 2>&1 & second_pid=$! - for _ in $(seq 1 100); do + for _ in $(seq 1 600); do [ -f "$second_lock_waiter" ] && break + kill -0 "$second_pid" 2>/dev/null || break sleep 0.05 done if [ ! -f "$second_lock_waiter" ]; then touch "$gate" - kill "$first_pid" "$second_pid" 2>/dev/null || true - fail "second continuation never waited behind the first lifecycle owner" + wait "$first_pid" 2>/dev/null || true + wait "$second_pid" 2>/dev/null || true + fail "second continuation never waited behind the first lifecycle owner: first=$(cat "$CASE_DIR/first.out"); second=$(cat "$CASE_DIR/second.out")" fi touch "$gate" wait "$first_pid" @@ -3201,7 +3439,8 @@ test_concurrent_continuations_serialize_before_mutation() { second_rc=$? [ "$first_rc" -eq 0 ] || fail "first serialized continuation failed: $(cat "$CASE_DIR/first.out")" [ "$second_rc" -ne 0 ] || fail "second concurrent continuation also launched" - assert_grep 'managed recovery endpoint is still alive' "$CASE_DIR/second.out" "concurrent continuation did not revalidate the serialized replacement endpoint" + assert_grep 'managed recovery endpoint is still alive' "$CASE_DIR/second.out" \ + "concurrent continuation did not revalidate the serialized replacement endpoint: $(cat "$CASE_DIR/second.out")" lease_count=$(grep -Ec 'lease choose|lease acquire' "$AF_LOG" || true) endpoint_count=$(grep -c '^new-window ' "$TMUX_LOG" || true) [ "$lease_count" -eq 1 ] || fail "concurrent continuations acquired $lease_count leases" @@ -3285,6 +3524,7 @@ test_session_sync_cannot_recreate_metadata_after_teardown() { teardown_pid=$! for _ in $(seq 1 300); do [ -f "$release_marker" ] && break + kill -0 "$teardown_pid" 2>/dev/null || break sleep 0.1 done [ -f "$release_marker" ] || { @@ -4567,8 +4807,12 @@ test_continuation_rollback_preserves_concurrent_packet_replacement() { FM_ACCOUNT_CONTINUATION_INSTALL_TEST_PROCEED="$proceed" PATH="$FAKEBIN_DIR:$PATH" \ "$CONTINUATION" "$id" owned-rollback > "$output" 2>&1 & packet_pid=$! - for _ in $(seq 1 100); do [ -e "$ready" ] && break; sleep 0.05; done - [ -e "$ready" ] || { kill -TERM "$packet_pid" 2>/dev/null || true; fail "owned-rollback continuation gate did not open"; } + for _ in $(seq 1 400); do [ -e "$ready" ] && break; sleep 0.05; done + [ -e "$ready" ] || { + kill -TERM "$packet_pid" 2>/dev/null || true + wait "$packet_pid" 2>/dev/null || true + fail "owned-rollback continuation gate did not open: $(cat "$output" 2>/dev/null)" + } rm -f "$packet" cp "$replacement" "$packet" printf 'changed during owned packet rollback\n' > "$worktree/owned-rollback-race.txt" @@ -4887,7 +5131,9 @@ test_account_metadata_lock_reclaims_orphans_without_overlapping_owners() { [ "$owner_lines" -eq 2 ] || fail "published metadata lock did not contain complete ownership" mkdir -p "$lock" - printf '%s\nstale-owner\n' "$$" > "$lock/owner" + # PID 1 is observable on both CI Linux and macOS; the deliberately mismatched + # start identity makes this owner stale without assuming a PID is absent. + printf '1\nstale-owner\n' > "$lock/owner" workers="$case_dir/workers.sh" cat > "$workers" <<'SH' #!/usr/bin/env bash @@ -4926,10 +5172,10 @@ SH rm -rf "$lock" mkdir -p "$lock/.reclaiming" - printf '%s\nstale-reclaimer\n' "$$" > "$lock/.reclaiming/owner" + printf '1\nstale-reclaimer\n' > "$lock/.reclaiming/owner" touch -t 200001010000 "$lock" "$lock/.reclaiming" FM_ACCOUNT_META_LOCK_WAIT_SECONDS=2 FM_ACCOUNT_META_LOCK_ORPHAN_GRACE_SECONDS=0 \ - bash -c '. "$1"; held=$(fm_account_meta_lock_acquire "$2" lock-task); fm_account_meta_lock_release "$held"' \ + bash -c '. "$1"; held=$(fm_account_meta_lock_acquire "$2" lock-task) || exit $?; fm_account_meta_lock_release "$held"' \ _ "$ROOT/bin/fm-account-routing-lib.sh" "$state" || fail "abandoned metadata reclaim owner blocked acquisition" assert_absent "$lock" "metadata lock retained an abandoned reclaim owner" pass "metadata locks reclaim abandoned directories without deleting new owners" @@ -5099,6 +5345,34 @@ SH pass "account metadata stat helpers select GNU stat without probing BSD filesystem stat" } +test_darwin_stat_mode_preserves_special_permission_bits() { + local case_dir fakebin file output + case_dir="$TMP_ROOT/darwin-stat-mode" + fakebin="$case_dir/fakebin" + file="$case_dir/file" + mkdir -p "$fakebin" + : > "$file" + cat > "$fakebin/uname" <<'SH' +#!/usr/bin/env bash +printf 'Darwin\n' +SH + cat > "$fakebin/stat" <<'SH' +#!/usr/bin/env bash +[ "${1:-}" = -f ] && [ "${2:-}" = %Mp%Lp ] || exit 2 +printf '1777\n' +SH + chmod +x "$fakebin/uname" "$fakebin/stat" + output=$(bash -c ' + . "$1" + FM_ACCOUNT_SYSTEM_UNAME_BIN=$2/uname + FM_ACCOUNT_SYSTEM_STAT_BIN=$2/stat + fm_account_path_mode "$3" + ' _ "$ROOT/bin/fm-account-routing-lib.sh" "$fakebin" "$file") \ + || fail "Darwin account mode helper failed" + [ "$output" = 1777 ] || fail "Darwin account mode helper lost special permission bits: $output" + pass "account metadata stat helpers preserve Darwin special permission bits" +} + test_stale_reclaim_guard_is_owned_before_lock_removal() { local case_dir lock output status case_dir="$TMP_ROOT/reclaim-guard-ownership" @@ -5568,6 +5842,7 @@ test_agent_fleet_lifecycle_calls_are_bounded() { assert_grep 'lease recover ' "$AF_LOG" "timed-out lease choice did not reconcile ownership" rm -f "$CASE_DIR/endpoint-live" + write_teardown_completion_report "$id" clear_case_logs release_completed="$CASE_DIR/release-completed" if out=$(FM_FAKE_AF_RELEASE_SLEEP=10 FM_FAKE_AF_RELEASE_COMPLETED="$release_completed" \ @@ -5856,6 +6131,7 @@ test_teardown_stops_after_rollback_restores_predecessor() { [ "$failed_task" != "$old_task" ] || fail "failed continuation did not install its rollback generation" rm -f "$CASE_DIR/endpoint-live" + write_teardown_completion_report "$id" clear_case_logs if out=$(run_teardown "$id" --force 2>&1); then status=0; else status=$?; fi [ "$status" -ne 0 ] || fail "teardown continued after restoring predecessor metadata" @@ -5863,7 +6139,6 @@ test_teardown_stops_after_rollback_restores_predecessor() { assert_not_grep "lease release --task $old_task" "$AF_LOG" "teardown released the restored predecessor in the stale generation pass" assert_contains "$out" "rerun teardown against the restored task generation" "rollback restoration retry guidance was not surfaced" - write_test_completion_report "$id" clear_case_logs write_teardown_completion_report "$id" run_teardown "$id" --force >/dev/null || fail "restored predecessor could not be torn down on a fresh pass" @@ -6109,6 +6384,7 @@ if [ "${FM_TEST_FOCUSED:-}" = session-sync-teardown-race ]; then fi if [ "${FM_TEST_FOCUSED:-}" = rollback-safety ]; then + run_isolated_test test_spawn_rollback_task_tmp_refusal_is_retryable run_isolated_test test_secondmate_rollback_refuses_unsafe_tasktmp run_isolated_test test_rollback_backup_rejects_symlink_and_rechecks_under_lock exit 0 @@ -6380,15 +6656,37 @@ if [ "${FM_TEST_FOCUSED:-}" = production-routing-authority ]; then run_isolated_test test_agent_fleet_entrypoint_is_physically_pinned_per_operation run_isolated_test test_agent_fleet_validation_ignores_ambient_system_tool_shadows run_isolated_test test_linux_stat_selection_avoids_filesystem_stat_output + run_isolated_test test_darwin_stat_mode_preserves_special_permission_bits run_isolated_test test_production_routing_ignores_ambient_mode_and_forbids_binary_override exit 0 fi +if [ "${FM_TEST_FOCUSED:-}" = concurrent-continuation ]; then + run_isolated_test test_concurrent_continuations_serialize_before_mutation + exit 0 +fi + +if [ "${FM_TEST_FOCUSED:-}" = teardown-report-gates ]; then + run_isolated_test test_agent_fleet_lifecycle_calls_are_bounded + run_isolated_test test_teardown_stops_after_rollback_restores_predecessor + exit 0 +fi + if [ "${FM_TEST_FOCUSED:-}" = production-fleet-environment ]; then run_isolated_test test_production_fleet_environment_is_closed_before_control_and_worker_exec exit 0 fi +if [ "${FM_TEST_FOCUSED:-}" = prior-tasktmp-cleanup ]; then + run_isolated_test test_unmanaged_respawn_preserves_report_cutover_state + run_isolated_test test_failed_managed_respawn_restores_unmanaged_metadata + run_isolated_test test_task_tmp_removal_refuses_symlinked_parent + run_isolated_test test_task_tmp_removal_pins_ancestors_during_swap + run_isolated_test test_task_tmp_partial_removal_fails_closed + run_isolated_test test_agent_fleet_task_keys_are_namespaced_by_home_and_attempt + exit 0 +fi + run_isolated_test test_reserved_generation_is_durable_before_lease_mutation run_isolated_test test_off_is_byte_compatible_and_never_calls_agent_fleet run_isolated_test test_failed_freshness_proof_rolls_back_unmanaged_resources @@ -6495,6 +6793,7 @@ run_isolated_test test_account_locks_never_reclaim_on_indeterminate_process_prob run_isolated_test test_ownerless_lock_marker_rejects_symlink_clobber run_isolated_test test_account_lock_owner_controls_reject_symlinks run_isolated_test test_linux_stat_selection_avoids_filesystem_stat_output +run_isolated_test test_darwin_stat_mode_preserves_special_permission_bits run_isolated_test test_stale_reclaim_guard_is_owned_before_lock_removal run_isolated_test test_task_owned_account_artifacts_reject_symlink_paths run_isolated_test test_secondmate_home_lock_key_fails_closed_without_fixed_hasher diff --git a/tests/fm-afk-inject-herdr-e2e.test.sh b/tests/fm-afk-inject-herdr-e2e.test.sh index 34ee3e6152..dac264e12f 100755 --- a/tests/fm-afk-inject-herdr-e2e.test.sh +++ b/tests/fm-afk-inject-herdr-e2e.test.sh @@ -67,6 +67,9 @@ cleanup_all() { rm -rf "${STATE_DIR:-}" 2>/dev/null || true } trap cleanup_all EXIT +STATE_DIR=$(mktemp -d "${TMPDIR:-/tmp}/fm-afk-herdr-e2e.XXXXXX") +export FM_HOME="$STATE_DIR/home" +mkdir -p "$FM_HOME/config" "$FM_HOME/state" fm_herdr_lab_prepare "$SESSION" || fail "could not prepare isolated Herdr lab session" # --- source the daemon (for afk_enter/afk_exit/FM_INJECT_MARK) + the backend - @@ -78,8 +81,6 @@ fm_backend_source herdr || fail "fm_backend_source herdr failed" fm_backend_herdr_version_check || fail "version_check failed against the real installed herdr" -STATE_DIR=$(mktemp -d "${TMPDIR:-/tmp}/fm-afk-herdr-e2e.XXXXXX") -mkdir -p "$STATE_DIR" LOG_FILE="$STATE_DIR/submitted.log" : > "$LOG_FILE" @@ -201,8 +202,12 @@ done LOOP chmod +x "$LOOP_SCRIPT" -fm_backend_herdr_send_text_line "$SUPERVISOR_TARGET" "bash '$LOOP_SCRIPT' '$LOG_FILE'" \ - || fail "could not start the supervisor-loop script in the scratch herdr pane" +LOOP_START_DEADLINE=$(( $(date +%s) + 30 )) +while ! fm_backend_herdr_cli "$SESSION" pane run "$PANE_ID" "bash '$LOOP_SCRIPT' '$LOG_FILE'" >/dev/null 2>&1; do + [ "$(date +%s)" -lt "$LOOP_START_DEADLINE" ] \ + || fail "could not start the supervisor-loop script in the scratch herdr pane" + sleep 1 +done sleep 1 # let the loop start and settle # --- herdr shim: forwards to the real binary, optionally swallows one Enter -- @@ -337,12 +342,15 @@ test_scenario_a() { fm_backend_herdr_send_key "$SUPERVISOR_TARGET" Enter sleep 0.5 - sleep 8 + local deadline=$(( $(date +%s) + 30 )) + while ! grep -q 'Supervisor escalate' "$LOG_FILE"; do + [ "$(date +%s)" -lt "$deadline" ] \ + || fail "Scenario A: timed out waiting for the deferred digest submission" + sleep 1 + done grep -q 'human draft text' "$LOG_FILE" \ || fail "Scenario A: human text not in log after submit" - grep -q 'Supervisor escalate' "$LOG_FILE" \ - || fail "Scenario A: digest not injected after the pane went idle" if grep -q 'human draft text.*Supervisor escalate' "$LOG_FILE" || \ grep -q 'Supervisor escalate.*human draft text' "$LOG_FILE"; then fail "Scenario A: human text and digest merged into one line (after idle)" @@ -378,7 +386,13 @@ test_scenario_b() { echo "done: PR https://example.test/pr/200" > "$STATE_DIR/fake-c1.status" - sleep 10 + local deadline=$(( $(date +%s) + 30 )) + while [ "$(grep -c 'Supervisor escalate' "$LOG_FILE" || true)" -lt 1 ]; do + [ "$(date +%s)" -lt "$deadline" ] \ + || fail "Scenario B: timed out waiting for the retried digest submission" + sleep 1 + done + sleep 2 local digest_count digest_count=$(grep -c 'Supervisor escalate' "$LOG_FILE" || true) @@ -414,7 +428,13 @@ test_scenario_c() { start_daemon echo "done: PR https://example.test/pr/300" > "$STATE_DIR/fake-c1.status" - sleep 8 + local deadline=$(( $(date +%s) + 30 )) + while [ "$(grep -c 'Supervisor escalate' "$LOG_FILE" || true)" -lt 1 ]; do + [ "$(date +%s)" -lt "$deadline" ] \ + || fail "Scenario C: timed out waiting for the digest submission" + sleep 1 + done + sleep 2 local digest_count digest_count=$(grep -c 'Supervisor escalate' "$LOG_FILE" || true) diff --git a/tests/fm-afk-reap-wake-e2e.test.sh b/tests/fm-afk-reap-wake-e2e.test.sh index b211950329..8d8b6a4fed 100755 --- a/tests/fm-afk-reap-wake-e2e.test.sh +++ b/tests/fm-afk-reap-wake-e2e.test.sh @@ -42,6 +42,7 @@ printf 'none\t-\tnative\n' > "$STATE_DIR/.afk-daemon-terminal" FM_STATE_OVERRIDE="$STATE_DIR" \ FM_AFK_STATE_PREPARED=1 \ +FM_AFK_DELIVERY=reap-wake \ FM_ESCALATE_BATCH_SECS=0 \ FM_HOUSEKEEPING_TICK=1 \ FM_POLL=1 \ @@ -54,9 +55,10 @@ FM_SUPERVISOR_TARGET=missing-pane \ "$START" > "$STATE_DIR/daemon.out" 2> "$STATE_DIR/daemon.err" & DAEMON_PID=$! -for _ in $(seq 1 60); do +start_deadline=$(( $(date +%s) + 30 )) +while [ "$(date +%s)" -lt "$start_deadline" ]; do [ -f "$STATE_DIR/.supervise-daemon.pid" ] && break - sleep 0.1 + sleep 0.2 done [ -f "$STATE_DIR/.supervise-daemon.pid" ] || { sed 's/^/ /' "$STATE_DIR/daemon.err" >&2 @@ -73,9 +75,10 @@ fi pass "routine wake stays in bash and keeps the native task parked" printf 'needs-decision: choose the safe rollout\n' > "$STATE_DIR/crew.status" -for _ in $(seq 1 120); do +completion_deadline=$(( $(date +%s) + 60 )) +while [ "$(date +%s)" -lt "$completion_deadline" ]; do kill -0 "$DAEMON_PID" 2>/dev/null || break - sleep 0.1 + sleep 0.2 done if kill -0 "$DAEMON_PID" 2>/dev/null; then sed 's/^/ /' "$STATE_DIR/daemon.err" >&2 diff --git a/tests/fm-arm-pretool-check.test.sh b/tests/fm-arm-pretool-check.test.sh index 52345bd43e..3c86757f00 100755 --- a/tests/fm-arm-pretool-check.test.sh +++ b/tests/fm-arm-pretool-check.test.sh @@ -371,7 +371,7 @@ test_failopen_garbage_stdin() { test_failopen_missing_jq() { local dir fakebin rc real - dir=$(fm_test_tmproot fm-arm-pretool-check) + fm_test_tmproot_into dir fm-arm-pretool-check fakebin="$dir/fakebin" mkdir -p "$fakebin" local tool @@ -387,7 +387,7 @@ test_failopen_missing_jq() { test_failopen_missing_node() { local dir fakebin rc real tool - dir=$(fm_test_tmproot fm-arm-pretool-node) + fm_test_tmproot_into dir fm-arm-pretool-node fakebin="$dir/fakebin" mkdir -p "$fakebin" for tool in bash dirname; do @@ -403,11 +403,11 @@ test_failopen_missing_node() { # --- --claude output shaping --------------------------------------------------- test_claude_mode_stdout_empty_on_deny() { - local out err rc - out=$("$CHECK" --claude --command 'bin/fm-watch-arm.sh &' 2>/tmp/fm-arm-pretool-check-claude-stderr.$$) + local out err rc err_file="$MATRIX_TMP/claude-stderr" + out=$("$CHECK" --claude --command 'bin/fm-watch-arm.sh &' 2>"$err_file") rc=$? - err=$(cat "/tmp/fm-arm-pretool-check-claude-stderr.$$" 2>/dev/null) - rm -f "/tmp/fm-arm-pretool-check-claude-stderr.$$" + err=$(cat "$err_file" 2>/dev/null) + rm -f "$err_file" [ "$rc" -eq 2 ] || fail "--claude deny must still exit 2, got $rc" [ -z "$out" ] || fail "--claude deny must leave stdout EMPTY (Claude Code only honors a stderr-only deny), got: $out" printf '%s' "$err" | jq -e '.hookSpecificOutput.permissionDecision == "deny"' >/dev/null 2>&1 \ diff --git a/tests/fm-autocompact.test.sh b/tests/fm-autocompact.test.sh index 2c27621ae5..ae1a719d00 100755 --- a/tests/fm-autocompact.test.sh +++ b/tests/fm-autocompact.test.sh @@ -8,7 +8,7 @@ set -u # shellcheck disable=SC2153 AUTOCOMPACT="$ROOT/bin/fm-autocompact.sh" -TMP_ROOT=$(fm_test_tmproot fm-autocompact-tests) +fm_test_tmproot_into TMP_ROOT fm-autocompact-tests fm_git_identity fmtest fmtest@example.invalid new_primary() { diff --git a/tests/fm-backend-autodetect-smoke.test.sh b/tests/fm-backend-autodetect-smoke.test.sh index 2358d0a0bb..1bc824042b 100755 --- a/tests/fm-backend-autodetect-smoke.test.sh +++ b/tests/fm-backend-autodetect-smoke.test.sh @@ -55,6 +55,7 @@ command -v treehouse >/dev/null 2>&1 || { echo "skip: treehouse not found (requi TMP_ROOT=$(mktemp -d "$(cd "${TMPDIR:-/tmp}" && pwd -P)/fm-backend-autodetect-smoke.XXXXXX") SESSION="fm-lab-autodetect-smoke-$$" export HERDR_SESSION="$SESSION" +export FM_HERDR_LAB_STATE_DIR="$TMP_ROOT/herdr-lab-state" herdr_test_lab_available "$SESSION" || exit 0 ID="autodetectsmoke1" WT= @@ -86,7 +87,7 @@ env -u TMUX -u FM_BACKEND PATH="$PATH" HERDR_ENV=1 \ FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$STATE" FM_DATA_OVERRIDE="$DATA" \ FM_CONFIG_OVERRIDE="$CONFIG" FM_PROJECTS_OVERRIDE="$TMP_ROOT/unused-projects" \ FM_SPAWN_NO_GUARD=1 \ - "$ROOT/bin/fm-spawn.sh" "$ID" "$PROJ" "sh -c 'echo autodetect-smoke-ok'" \ + "$ROOT/bin/fm-spawn.sh" "$ID" "$PROJ" "sh -c 'echo autodetect-smoke-ok; sleep 30'" \ >"$OUT_FILE" 2>"$ERR_FILE" status=$? [ "$status" -eq 0 ] || fail "fm-spawn.sh did not succeed auto-detecting herdr"$'\n'"--- stdout ---"$'\n'"$(cat "$OUT_FILE")"$'\n'"--- stderr ---"$'\n'"$(cat "$ERR_FILE")" @@ -125,7 +126,6 @@ case "$CAPTURED" in *) fail "the raw launch command did not run in the auto-detected herdr pane"$'\n'"$CAPTURED" ;; esac pass "real herdr: the auto-detected spawn's launch command actually ran in the herdr pane" - # --- teardown completes the trivial spawn/teardown cycle -------------------- printf '# Completion\n\n## Summary\n\nThe auto-detected Herdr smoke task completed.\n\n## What changed\n\nNo project files changed.\n\n## Verification\n\nThe task command ran in the isolated Herdr pane.\n\n## Visual evidence\n\nNone.\n\n## Artifacts\n\nThe captured pane output is the test artifact.\n\n## Follow-ups\n\nNone.\n' \ diff --git a/tests/fm-backend-cmux.test.sh b/tests/fm-backend-cmux.test.sh index 22918c89d8..b31b3481c0 100755 --- a/tests/fm-backend-cmux.test.sh +++ b/tests/fm-backend-cmux.test.sh @@ -15,7 +15,7 @@ set -u command -v jq >/dev/null 2>&1 || { echo "skip: jq not found (required by the cmux adapter)"; exit 0; } -TMP_ROOT=$(fm_test_tmproot fm-backend-cmux-tests) +fm_test_tmproot_into TMP_ROOT fm-backend-cmux-tests # make_cmux_fakebin: a `cmux` stub that logs every invocation (one line, # unit-separated args, to $FM_CMUX_LOG) and returns the canned response for diff --git a/tests/fm-backend-herdr-workspace-per-home-e2e.test.sh b/tests/fm-backend-herdr-workspace-per-home-e2e.test.sh index 397cf40d40..a68f10fb14 100755 --- a/tests/fm-backend-herdr-workspace-per-home-e2e.test.sh +++ b/tests/fm-backend-herdr-workspace-per-home-e2e.test.sh @@ -85,11 +85,24 @@ mkdir -p "$PRIMARY_HOME/state" "$PRIMARY_HOME/data/cm1" "$PRIMARY_HOME/config" printf 'trivial e2e primary crewmate brief: nothing to do.\n' > "$PRIMARY_HOME/data/cm1/brief.md" SM_HOME="$TMP_ROOT/secondmate-home" +SM_ORIGIN=$(git -C "$ROOT" remote get-url origin) +PRIMARY_ROOT="$TMP_ROOT/primary-root" +git init -q "$PRIMARY_ROOT" +git -C "$PRIMARY_ROOT" remote add origin "$SM_ORIGIN" +git -C "$PRIMARY_ROOT" fetch -q "$ROOT" refs/remotes/origin/main:refs/remotes/origin/main +git -C "$PRIMARY_ROOT" checkout -q -B main FETCH_HEAD +git -C "$PRIMARY_ROOT" remote set-head origin main +git init -q "$SM_HOME" +git -C "$SM_HOME" remote add origin "$SM_ORIGIN" +git -C "$SM_HOME" fetch -q origin main +git -C "$SM_HOME" checkout -q -B main FETCH_HEAD +git -C "$SM_HOME" remote set-head origin main mkdir -p "$SM_HOME/state" "$SM_HOME/data/cm2" "$SM_HOME/config" "$SM_HOME/projects" "$SM_HOME/bin" -printf '# scratch secondmate home AGENTS.md placeholder\n' > "$SM_HOME/AGENTS.md" printf 'e2esm1\n' > "$SM_HOME/.fm-secondmate-home" printf 'trivial e2e secondmate charter: nothing to do.\n' > "$SM_HOME/data/charter.md" printf 'trivial e2e secondmate-owned crewmate brief: nothing to do.\n' > "$SM_HOME/data/cm2/brief.md" +printf -- '- e2esm1 - Herdr workspace isolation (home: %s; scope: Herdr workspace isolation; projects: ; added 2026-07-27)\n' \ + "$SM_HOME" > "$PRIMARY_HOME/data/secondmates.md" write_completion_report() { # printf '# Completion\n\n## Summary\n\n%s\n\n## What changed\n\nNo project files changed.\n\n## Verification\n\nThe task command ran in its isolated Herdr pane.\n\n## Visual evidence\n\nNone.\n\n## Artifacts\n\nThe captured pane output is the test artifact.\n\n## Follow-ups\n\nNone.\n' "$2" > "$1" @@ -115,7 +128,7 @@ PROJ2="$TMP_ROOT/scratch-project-2"; make_scratch_project "$PROJ2" CM1_OUT="$TMP_ROOT/cm1.out"; CM1_ERR="$TMP_ROOT/cm1.err" FM_SPAWN_NO_GUARD=1 FM_HOME="$PRIMARY_HOME" FM_ROOT_OVERRIDE="$ROOT" \ - "$ROOT/bin/fm-spawn.sh" cm1 "$PROJ1" "sh -c 'echo primary-crew-ok'" --backend herdr \ + "$ROOT/bin/fm-spawn.sh" cm1 "$PROJ1" "sh -c 'echo primary-crew-ok; sleep 300'" --backend herdr \ >"$CM1_OUT" 2>"$CM1_ERR" rc=$? [ "$rc" -eq 0 ] || fail "primary-shaped crewmate spawn failed"$'\n'"--- stdout ---"$'\n'"$(cat "$CM1_OUT")"$'\n'"--- stderr ---"$'\n'"$(cat "$CM1_ERR")" @@ -143,8 +156,8 @@ pass "real herdr E2E: the primary-shaped home's crewmate landed in the 'firstmat # exactly this call - AGENTS.md task herdr-sm-spaces-k4, requirement 3.) SM_OUT="$TMP_ROOT/sm.out"; SM_ERR="$TMP_ROOT/sm.err" -FM_SPAWN_NO_GUARD=1 FM_HOME="$PRIMARY_HOME" FM_ROOT_OVERRIDE="$ROOT" \ - "$ROOT/bin/fm-spawn.sh" e2esm1 "$SM_HOME" "sh -c 'echo secondmate-launch-ok'" --secondmate --backend herdr \ +FM_SPAWN_NO_GUARD=1 FM_HOME="$PRIMARY_HOME" FM_ROOT_OVERRIDE="$PRIMARY_ROOT" \ + "$ROOT/bin/fm-spawn.sh" e2esm1 "$SM_HOME" "sh -c 'echo secondmate-launch-ok; sleep 300'" --secondmate --backend herdr \ >"$SM_OUT" 2>"$SM_ERR" rc=$? [ "$rc" -eq 0 ] || fail "the primary's --secondmate spawn of e2esm1 failed"$'\n'"--- stdout ---"$'\n'"$(cat "$SM_OUT")"$'\n'"--- stderr ---"$'\n'"$(cat "$SM_ERR")" @@ -170,7 +183,7 @@ pass "real herdr E2E: a --secondmate spawn by the PRIMARY lands in the SECONDMAT CM2_OUT="$TMP_ROOT/cm2.out"; CM2_ERR="$TMP_ROOT/cm2.err" FM_SPAWN_NO_GUARD=1 FM_HOME="$SM_HOME" FM_ROOT_OVERRIDE="$ROOT" \ - "$ROOT/bin/fm-spawn.sh" cm2 "$PROJ2" "sh -c 'echo sm-crew-ok'" --backend herdr \ + "$ROOT/bin/fm-spawn.sh" cm2 "$PROJ2" "sh -c 'echo sm-crew-ok; sleep 300'" --backend herdr \ >"$CM2_OUT" 2>"$CM2_ERR" rc=$? [ "$rc" -eq 0 ] || fail "a crewmate spawned FROM the secondmate-shaped home failed"$'\n'"--- stdout ---"$'\n'"$(cat "$CM2_OUT")"$'\n'"--- stderr ---"$'\n'"$(cat "$CM2_ERR")" @@ -208,6 +221,8 @@ pass "real herdr E2E: list_live from the secondmate's own context sees only task # --- 5. teardown closes the RIGHT tab, and no other ------------------------ +fm_backend_herdr_send_key "$SESSION:$CM1_PANE" C-c || fail "could not stop cm1's raw fixture command" +sleep 1 TD1_OUT="$TMP_ROOT/td1.out" FM_HOME="$PRIMARY_HOME" FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$PRIMARY_HOME/state" FM_DATA_OVERRIDE="$PRIMARY_HOME/data" \ FM_CONFIG_OVERRIDE="$PRIMARY_HOME/config" FM_REPORT_STACK_ROOT="$TMP_ROOT/report-stack" \ @@ -227,6 +242,8 @@ fi WT1= pass "real herdr E2E: tearing down cm1 closes only its own tab - the secondmate's and cm2's tabs survive untouched" +fm_backend_herdr_send_key "$SESSION:$CM2_PANE" C-c || fail "could not stop cm2's raw fixture command" +sleep 1 TD2_OUT="$TMP_ROOT/td2.out" FM_HOME="$SM_HOME" FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$SM_HOME/state" FM_DATA_OVERRIDE="$SM_HOME/data" \ FM_CONFIG_OVERRIDE="$SM_HOME/config" FM_REPORT_STACK_ROOT="$TMP_ROOT/report-stack" \ diff --git a/tests/fm-backend-herdr.test.sh b/tests/fm-backend-herdr.test.sh index 084e8b3dc7..9b3bc1d44a 100755 --- a/tests/fm-backend-herdr.test.sh +++ b/tests/fm-backend-herdr.test.sh @@ -18,7 +18,7 @@ set -u command -v jq >/dev/null 2>&1 || { echo "skip: jq not found (required by the herdr adapter)"; exit 0; } -TMP_ROOT=$(fm_test_tmproot fm-backend-herdr-tests) +fm_test_tmproot_into TMP_ROOT fm-backend-herdr-tests export FM_BACKEND_HERDR_SUBMIT_MIN_SLEEP=0 export FM_BACKEND_HERDR_TEST_LAB=firstmate-herdr-test-lab-v1 diff --git a/tests/fm-backend-orca.test.sh b/tests/fm-backend-orca.test.sh index 9814ba83c5..023b8d47c9 100755 --- a/tests/fm-backend-orca.test.sh +++ b/tests/fm-backend-orca.test.sh @@ -119,7 +119,9 @@ case "${1:-} ${2:-}" in else worktree="wt-${terminal#term-}" fi - if [ -f "$RESP/.terminal-title" ]; then + if [ -n "${FM_ORCA_DEFAULT_TERMINAL_TITLE:-}" ]; then + title=$FM_ORCA_DEFAULT_TERMINAL_TITLE + elif [ -f "$RESP/.terminal-title" ]; then title=$(cat "$RESP/.terminal-title") elif [ -n "${FM_STATE_OVERRIDE:-}" ] && [ -d "$FM_STATE_OVERRIDE" ]; then title= @@ -160,6 +162,9 @@ process.stdout.write(JSON.stringify(data) + "\n"); if [ -n "${FM_ORCA_REMOVE_ERROR:-}" ]; then printf '{"ok":false,"error":{"code":"worktree_not_removed","message":"worktree not removed"}}\n' else + if [ -n "${FM_ORCA_REMOVE_PROJECT:-}" ] && [ -n "${FM_ORCA_REMOVE_PATH:-}" ]; then + git -C "$FM_ORCA_REMOVE_PROJECT" worktree remove --force "$FM_ORCA_REMOVE_PATH" || exit 1 + fi printf '{"ok":true,"result":{"removed":true}}\n' fi ;; @@ -194,9 +199,27 @@ SH initialize_secondmate_home_repo() { local home=$1 source=$2 fm_git_init_commit "$home" + git -C "$home" branch -M main + printf '%s\n' '.fm-secondmate-home' 'projects/' 'state/' > "$home/.gitignore" + git -C "$home" add .gitignore + git -C "$home" -c user.name='Firstmate Tests' -c user.email=tests@example.invalid \ + commit -qm 'ignore operational state' + git -C "$source" fetch -q "$home" HEAD + git -C "$source" update-ref refs/heads/main FETCH_HEAD git -C "$home" remote add origin "$source" } +initialize_secondmate_project_repo() { + local source_root=$1 clone=$2 worktree=$3 branch=$4 authority + authority="$source_root/project-origins/alpha" + mkdir -p "$source_root/projects" "$source_root/project-origins" + fm_git_init_commit "$authority" + git clone -q "$authority" "$source_root/projects/alpha" + git clone -q "$source_root/projects/alpha" "$clone" + git -C "$clone" remote set-url origin "$authority" + git -C "$clone" worktree add --quiet -b "$branch" "$worktree" +} + add_tmux_fake() { local fb=$1 cat > "$fb/tmux" <<'SH' @@ -539,9 +562,11 @@ test_terminal_state_classifies_closed_live_and_ambiguous_orca() { test_remove_worktree_refuses_empty_id() { local out status orca_case remove-empty + set +e out=$( PATH="$FB:$PATH" FM_ORCA_LOG="$LOG" FM_ORCA_RESPONSES="$RESP" \ bash -c '. "$0/bin/backends/orca.sh"; fm_backend_orca_remove_worktree ""' "$ROOT" 2>&1 ) status=$? + set +e [ "$status" -ne 0 ] || fail "remove_worktree should fail when the Orca worktree id is empty" assert_contains "$out" "missing Orca worktree id" "remove_worktree did not explain the missing id" [ ! -s "$LOG" ] || fail "remove_worktree should not call Orca with an empty id" @@ -553,9 +578,11 @@ test_remove_worktree_rejects_orca_error_json() { orca_case remove-error-json token=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa printf '{"ok":false,"error":{"code":"worktree_not_found","message":"worktree not found"}}\n' > "$RESP/1.out" + set +e out=$( PATH="$FB:$PATH" FM_ORCA_LOG="$LOG" FM_ORCA_RESPONSES="$RESP" \ bash -c '. "$0/bin/backends/orca.sh"; fm_backend_orca_remove_worktree_bound wt-gone /tmp/orca-wt "$1"' "$ROOT" "$token" 2>&1 ) status=$? + set +e [ "$status" -ne 0 ] || fail "bound remove_worktree should fail on Orca ok:false JSON" assert_contains "$out" "worktree not found" "bound remove_worktree should surface the Orca removal error" assert_contains "$(cat "$LOG")" $'--expected-path\x1f/tmp/orca-wt' \ @@ -569,10 +596,12 @@ test_remove_worktree_requires_bound_provider_capability() { local out status token orca_case remove-boundary-capability token=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb + set +e out=$(PATH="$FB:$PATH" FM_ORCA_LOG="$LOG" FM_ORCA_RESPONSES="$RESP" \ FM_ORCA_TEST_BOUND_REMOVAL_CAPABILITIES=unavailable \ bash -c '. "$0/bin/backends/orca.sh"; fm_backend_orca_remove_worktree_bound wt-retained /tmp/orca-wt "$1"' "$ROOT" "$token" 2>&1) status=$? + set +e [ "$status" -ne 0 ] || fail "Orca removal proceeded without a bound provider capability" assert_contains "$out" "identity-bound provider capability" \ "unbound Orca removal did not surface its provider limitation" @@ -746,13 +775,15 @@ test_legacy_respawn_refuses_without_provider_task_authority() { printf '1\n' > "$RESP/1.exit" printf '{"ok":true,"result":{"repo":{"id":"repo-spawn"}}}\n' > "$RESP/2.out" printf '{"ok":true,"result":{"worktree":{"id":"wt-spawn","path":"%s"},"terminal":{"handle":"term-spawn"}}}\n' "$wt" > "$RESP/3.out" + printf '{"ok":true,"result":{"worktree":{"id":"wt-spawn","path":"%s","terminals":[{"handle":"term-spawn"}]}}}\n' \ + "$wt" > "$RESP/4.out" if out=$( PATH="$FB:$PATH" FM_ORCA_LOG="$LOG" FM_ORCA_RESPONSES="$RESP" \ FM_ROOT_OVERRIDE="$ROOT" FM_STATE_OVERRIDE="$state" FM_DATA_OVERRIDE="$data" FM_CONFIG_OVERRIDE="$config" \ FM_PROJECTS_OVERRIDE="$TMP_ROOT/unused-projects" FM_SPAWN_NO_GUARD=1 \ "$ROOT/bin/fm-spawn.sh" "$id" "$proj" claude --backend orca 2>&1 ); then fail "legacy Orca respawn proceeded without provider task authority" fi - assert_contains "$out" "did not return matching worktree id, path, and task authority" \ + assert_contains "$out" "Orca terminal is not authoritatively bound" \ "legacy Orca respawn did not explain its fail-closed authority refusal" assert_not_contains "$(cat "$log")" $'orca\x1f''terminal'$'\x1f''send' \ "refused legacy Orca respawn launched the harness" @@ -1857,7 +1888,10 @@ test_secondmate_force_teardown_removes_orca_child_via_orca() { child_id="orcachildz6" mkdir -p "$home/state" "$home/data" "$subhome/state" "$subhome/projects" printf 'domain\n' > "$subhome/.fm-secondmate-home" - fm_git_worktree "$childproj" "$childwt" "fm/$child_id" + orca_case secondmate-child-cleanup + neutral=$(neutral_fm_root "$CASE_DIR/neutral") + initialize_secondmate_home_repo "$subhome" "$neutral" + initialize_secondmate_project_repo "$home" "$childproj" "$childwt" "fm/$child_id" fm_write_meta "$home/state/domain.meta" \ "window=firstmate:fm-domain" "worktree=$subhome" "project=$subhome" \ "harness=echo" "kind=secondmate" "mode=secondmate" "yolo=off" \ @@ -1868,13 +1902,14 @@ test_secondmate_force_teardown_removes_orca_child_via_orca() { "window=fm-$child_id" "terminal=term-child-cleanup" "worktree=$childwt" "project=$childproj" \ "harness=claude" "kind=ship" "mode=no-mistakes" "yolo=off" \ "backend=orca" "orca_worktree_id=wt-child-cleanup" - orca_case secondmate-child-cleanup printf '{"ok":true,"result":{"worktree":{"id":"wt-child-cleanup","name":"fm-%s","path":"%s","terminals":[{"handle":"term-child-cleanup","title":"fm-%s"}]}}}\n' \ "$child_id" "$childwt" "$child_id" > "$RESP/1.out" add_tmux_fake "$FB" neutral=$(neutral_fm_root "$CASE_DIR/neutral") initialize_secondmate_home_repo "$subhome" "$neutral" out=$( PATH="$FB:$PATH" FM_ORCA_LOG="$LOG" FM_ORCA_RESPONSES="$RESP" \ + FM_ORCA_DEFAULT_TERMINAL_TITLE="fm-$child_id" \ + FM_ORCA_REMOVE_PROJECT="$childproj" FM_ORCA_REMOVE_PATH="$childwt" \ FM_ROOT_OVERRIDE="$neutral" FM_HOME="$home" "$ROOT/bin/fm-teardown.sh" domain --force 2>&1 ) rc=$? expect_code 0 "$rc" "forced secondmate teardown should remove Orca child work through Orca"$'\n'"$out" @@ -1898,8 +1933,11 @@ test_secondmate_force_teardown_refuses_orca_child_id_path_mismatch() { child_id="orcachildmismatchz1" mkdir -p "$home/state" "$home/data" "$subhome/state" "$subhome/projects" printf 'domain\n' > "$subhome/.fm-secondmate-home" - fm_git_worktree "$childproj" "$childwt" "fm/$child_id" - git -C "$childproj" worktree add --quiet -b "fm/$child_id-other" "$other_wt" + orca_case secondmate-child-mismatch + neutral=$(neutral_fm_root "$CASE_DIR/neutral") + initialize_secondmate_home_repo "$subhome" "$neutral" + initialize_secondmate_project_repo "$home" "$childproj" "$childwt" "fm/$child_id" + mkdir -p "$other_wt" fm_write_meta "$home/state/domain.meta" \ "window=firstmate:fm-domain" "worktree=$subhome" "project=$subhome" \ "harness=echo" "kind=secondmate" "mode=secondmate" "yolo=off" \ @@ -1910,7 +1948,6 @@ test_secondmate_force_teardown_refuses_orca_child_id_path_mismatch() { "window=fm-$child_id" "terminal=term-child-mismatch" "worktree=$childwt" "project=$childproj" \ "harness=claude" "kind=ship" "mode=no-mistakes" "yolo=off" \ "backend=orca" "orca_worktree_id=wt-child-mismatch" - orca_case secondmate-child-mismatch printf '{"ok":true,"result":{"worktree":{"id":"wt-child-mismatch","path":"%s"}}}\n' "$other_wt" > "$RESP/1.out" add_tmux_fake "$FB" neutral=$(neutral_fm_root "$CASE_DIR/neutral") @@ -1938,7 +1975,10 @@ test_secondmate_force_teardown_retains_partial_orca_child() { child_id="orcapartialz9" mkdir -p "$home/state" "$home/data" "$subhome/state" "$subhome/projects" printf 'domain\n' > "$subhome/.fm-secondmate-home" - fm_git_worktree "$childproj" "$childwt" "fm/$child_id" + orca_case secondmate-partial-child-cleanup + neutral=$(neutral_fm_root "$CASE_DIR/neutral") + initialize_secondmate_home_repo "$subhome" "$neutral" + initialize_secondmate_project_repo "$home" "$childproj" "$childwt" "fm/$child_id" fm_write_meta "$home/state/domain.meta" \ "window=firstmate:fm-domain" "worktree=$subhome" "project=$subhome" \ "harness=echo" "kind=secondmate" "mode=secondmate" "yolo=off" \ @@ -1949,7 +1989,6 @@ test_secondmate_force_teardown_retains_partial_orca_child() { "window=fm-$child_id" "worktree=$childwt" "project=$childproj" \ "harness=claude" "kind=ship" "mode=no-mistakes" "yolo=off" \ "backend=orca" "orca_worktree_id=wt-partial-child" - orca_case secondmate-partial-child-cleanup printf '{"ok":true,"result":{"worktree":{"id":"wt-partial-child","path":"%s"}}}\n' "$childwt" > "$RESP/1.out" add_tmux_fake "$FB" neutral=$(neutral_fm_root "$CASE_DIR/neutral") diff --git a/tests/fm-backend-zellij.test.sh b/tests/fm-backend-zellij.test.sh index 60bd56e9c0..1cafdd277e 100755 --- a/tests/fm-backend-zellij.test.sh +++ b/tests/fm-backend-zellij.test.sh @@ -14,7 +14,7 @@ set -u command -v jq >/dev/null 2>&1 || { echo "skip: jq not found (required by the zellij adapter)"; exit 0; } -TMP_ROOT=$(fm_test_tmproot fm-backend-zellij-tests) +fm_test_tmproot_into TMP_ROOT fm-backend-zellij-tests # make_zellij_fakebin: a `zellij` stub that logs every invocation (one line, # unit-separated args, to $FM_ZELLIJ_LOG) and returns the canned response for diff --git a/tests/fm-backend.test.sh b/tests/fm-backend.test.sh index 7efdb305c0..b232b1f588 100755 --- a/tests/fm-backend.test.sh +++ b/tests/fm-backend.test.sh @@ -35,7 +35,7 @@ fm_git_identity fmtest fmtest@example.invalid # shellcheck source=bin/fm-backend.sh . "$ROOT/bin/fm-backend.sh" -TMP_ROOT=$(fm_test_tmproot fm-backend-tests) +fm_test_tmproot_into TMP_ROOT fm-backend-tests # fm_backend_detect's cmux fallback (bundle id + process ancestry, # docs/cmux-backend.md "Runtime auto-detection") consults uname, lsappinfo, @@ -1108,7 +1108,7 @@ run_spawn_symlink_case() { #