Skip to content

feat(agent-fleet): add safe multi-account provider routing - #5

Closed
ruby-dlee wants to merge 0 commit into
mainfrom
codex/bridge-agent-fleet-v015
Closed

feat(agent-fleet): add safe multi-account provider routing#5
ruby-dlee wants to merge 0 commit into
mainfrom
codex/bridge-agent-fleet-v015

Conversation

@ruby-dlee

Copy link
Copy Markdown
Owner

Intent

Import Agent Fleet into the existing public ruby-dlee/firstmate repository as an independently versioned 0.2.0 provider-neutral package and ship it through a GitHub PR, not as local-only work or a separate/private repository. Make multi-account Claude and Codex routing safe enough that Desktop/default CLI sessions cannot silently share or eject worker identities: verify opaque remote identity sets, fail closed on base/Desktop overlap or unavailable proof, reread Claude Desktop identity at selection time, disable Claude login/logout commands in workers, scrub ambient provider variables, and keep worker launch task-bound with verified process ownership. Serialize profile changes, verification, quota refresh, selection, and enrollment under provider maintenance locks; refuse auth maintenance while same-provider leases are live. Make Codex enrollment non-destructive through an isolated staging home, atomic promotion, durable transaction journal, and crash recovery. Preserve sticky managed resume, isolated provider homes, and the existing Firstmate/Bridge architecture; do not touch live auth/config/tmux or merge without fresh owner authorization. Include regression coverage, locked uv metadata, provenance/release docs, root CI/no-mistakes integration, push the feature branch, and open a PR only after local and CI gates pass.

What Changed

  • Add Agent Fleet 0.2.0 as an independently versioned, provider-neutral package for isolated Claude and Codex profiles, identity verification, quota-aware scheduling, leases, enrollment, and crash recovery.
  • Integrate Firstmate account routing with explicit worktree validation, task-bound execution, and sticky managed session recovery.
  • Add regression coverage, locked uv metadata, provenance and release documentation, plus root CI and no-mistakes validation.

Risk Assessment

🚨 High: Captain, the change should not merge without approval because documented Claude aliases and subcommands can still bypass the task-bound launch boundary and outlive verified lease ownership.

Testing

The supplied shell baseline, locked Agent Fleet pytest suite, focused Firstmate routing checks, isolated synthetic-provider CLI workflows, and built-wheel installation smoke all passed; CLI/package artifacts capture the user-facing behavior without touching live provider auth, configuration, or tmux.

Evidence: End-to-end CLI transcript

=== independently versioned CLI contract ===
$ agent-fleet version
exit=0
{
  "cli_version": "0.2.0",
  "contract_version": 1
}
$ agent-fleet contract
exit=0
{
  "cli_version": "0.2.0",
  "commands": {
    "atomic_choose": "lease choose --pool <pool> --task <task> --workspace <path>",
    "dry_run": "choose --pool <pool> --task <task> --workspace <path> --dry-run",
    "exec": "exec --profile <profile> --task <task> --pool <pool> --workspace <path> -- <argv>",
    "explicit_acquire": "lease acquire --profile <profile> --task <task> --workspace <path>",
    "pool_summary": "pool status --pool <pool> [--provider <provider>]",
    "profile_enroll": "profile enroll <profile>",
    "profile_verify": "profile verify <profile>|--all",
    "project_list": "project list [--provider <provider>]",
    "project_register": "project register --provider <provider> <git-worktree>",
    "project_remove": "project remove --provider <provider> <git-worktree>",
    "recover": "lease recover --task <task> --workspace <path>",
    "release": "lease release --task <task>",
    "resume_explicit": "managed task/session mapping required",
    "resume_task": "resume --task <task> --workspace <path> -- <argv>",
    "session_remove": "session remove --task <task>",
    "session_status": "session status --task <task>"
  },
  "contract_version": 1,
  "formats": [
    "json",
    "toon",
    "human"
  ],
  "selection_fields": [
    "profile",
    "provider",
    "pool",
    "decision_reason",
    "quota_fresh",
    "headroom_percent",
    "active_lease_count",
    "degraded"
  ]
}

=== opaque remote identity evidence ===
$ agent-fleet quota show --all
exit=0
{
  "quota": [
    {
      "age_seconds": 0,
      "fresh": true,
      "headroom_percent": 80.0,
      "identity_fingerprint": "8a76657bbcee1e836570f5cf84a21cd9222846c0316e8c81a558b0ccb93e0626",
      "identity_source": "quota-account",
      "profile": "claude-1",
      "provider": "claude",
      "reason": null,
      "refreshed_at": "2026-07-18T05:56:18.796076+00:00",
      "reported_status": "fresh",
      "schema": 1,
      "status": "fresh",
      "verification_age_seconds": 0,
      "verified_at": "2026-07-18T05:56:18.793403+00:00",
      "verified_recent": true,
      "windows": [
        {
          "id": "five_hour",
          "kind": "session",
          "remaining_percent": 80.0,
          "resets_at": null
        }
      ]
    },
    {
      "age_seconds": 0,
      "fresh": true,
      "headroom_percent": 80.0,
      "identity_fingerprint": "027d629b77d7c38718f99b6a88a7eb7efee5736c47dd69e9ebdd061fde28dcd9",
      "identity_source": "quota-account",
      "profile": "codex-1",
      "provider": "codex",
      "reason": null,
      "refreshed_at": "2026-07-18T05:56:19.023508+00:00",
      "reported_status": "fresh",
      "schema": 1,
      "status": "fresh",
      "verification_age_seconds": 0,
      "verified_at": "2026-07-18T05:56:19.021054+00:00",
      "verified_recent": true,
      "windows": [
        {
          "id": "five_hour",
          "kind": "session",
          "remaining_percent": 80.0,
          "resets_at": null
        }
      ]
    }
  ]
}

=== managed Codex launch and sticky resume ===
$ agent-fleet exec --task sticky-codex-task --pool codex-crew --profile codex-1 --workspace /private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/registered-project -- record-session
exit=0
{
  "ambient_provider_variables_present": [],
  "claude_config_dir": null,
  "codex_home": "/private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/share/accounts/codex/1",
  "cwd": "/private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/registered-project",
  "disable_login_command": null,
  "disable_logout_command": null,
  "profile": "codex-1",
  "provider_argv": [
    "--disable",
    "plugins",
    "--disable",
    "plugin_sharing",
    "-c",
    "projects.\"/private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/registered-project\".trust_level=\"trusted\"",
    "--dangerously-bypass-hook-trust",
    "record-session"
  ],
  "session_recorded": true,
  "task": "sticky-codex-task"
}
$ agent-fleet session status --task sticky-codex-task
exit=0
{
  "pool": "codex-crew",
  "profile": "codex-1",
  "provider": "codex",
  "schema": 1,
  "session_id": "codex-session-001",
  "task": "sticky-codex-task",
  "updated_at": "2026-07-18T05:56:19.697414+00:00",
  "workspace": "/private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/registered-project"
}
$ agent-fleet resume --task sticky-codex-task --workspace /private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/registered-project -- resume-check
exit=0
{
  "ambient_provider_variables_present": [],
  "claude_config_dir": null,
  "codex_home": "/private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/share/accounts/codex/1",
  "cwd": "/private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/registered-project",
  "disable_login_command": null,
  "disable_logout_command": null,
  "profile": "codex-1",
  "provider_argv": [
    "resume",
    "--disable",
    "plugins",
    "--disable",
    "plugin_sharing",
    "-c",
    "projects.\"/private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/registered-project\".trust_level=\"trusted\"",
    "--dangerously-bypass-hook-trust",
    "codex-session-001",
    "resume-check"
  ],
  "session_recorded": false,
  "task": "sticky-codex-task"
}

=== managed Claude launch isolation ===
$ agent-fleet exec --task claude-env-task --pool claude-crew --profile claude-1 --workspace /private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/registered-project -- env-check
exit=0
{
  "ambient_provider_variables_present": [],
  "claude_config_dir": "/private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/share/accounts/claude/1",
  "codex_home": null,
  "cwd": "/private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/registered-project",
  "disable_login_command": "1",
  "disable_logout_command": "1",
  "profile": "claude-1",
  "provider_argv": [
    "env-check"
  ],
  "session_recorded": false,
  "task": "claude-env-task"
}

=== Claude Desktop overlap and unavailable proof fail closed ===
$ agent-fleet choose --task before-desktop-switch --pool claude-crew --provider claude --profile claude-1 --workspace /private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/registered-project --dry-run
exit=0
{
  "active_lease_count": 0,
  "decision_reason": "quota",
  "degraded": false,
  "dry_run": true,
  "headroom_percent": 80.0,
  "lease": null,
  "pool": "claude-crew",
  "profile": "claude-1",
  "provider": "claude",
  "quota_fresh": true,
  "schema": 1,
  "task": "before-desktop-switch"
}
$ agent-fleet choose --task after-desktop-switch --pool claude-crew --provider claude --profile claude-1 --workspace /private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/registered-project --dry-run
exit=2
{
  "error": "no remotely verified profile is routeable: duplicate_provider_identity",
  "ok": false
}
$ agent-fleet choose --task desktop-proof-unavailable --pool claude-crew --provider claude --profile claude-1 --workspace /private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/registered-project --dry-run
exit=2
{
  "error": "no remotely verified profile is routeable: duplicate_provider_identity",
  "ok": false
}

=== provider maintenance lock serializes registry changes ===
$ agent-fleet profile add claude-2 --provider claude
exit=2
{
  "error": "provider maintenance is already in progress for claude",
  "ok": false
}

=== live task ownership and same-provider auth maintenance ===
live worker pid=73984
$ agent-fleet exec --task live-owned-task --pool codex-crew --profile codex-1 --workspace /private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/registered-project -- second-owner-attempt
exit=2
{
  "error": "task lease is already owned by live process 73984",
  "ok": false
}
$ agent-fleet profile disable codex-1
exit=0
{
  "enabled": false,
  "home": "/private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/share/accounts/codex/1",
  "id": "codex-1",
  "max_concurrent": 3,
  "pools": [
    "codex-crew",
    "codex-manual"
  ],
  "provider": "codex",
  "reserve_percent": 15,
  "safety_policy": "worker",
  "weight": 1
}
$ agent-fleet profile enroll codex-1
exit=2
{
  "error": "refusing codex login while any same-provider Fleet lease is active",
  "ok": false
}
provider login invocations during refused enrollment=0
$ agent-fleet lease release --task live-owned-task --force
exit=0
{
  "profile": "codex-1",
  "released": true,
  "task": "live-owned-task"
}

=== Codex staged promotion crash recovery ===
staging parent=/private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/state/staging/codex
target home=/private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/share/accounts/codex/1
durable journal phase=prepared
target marker after atomic promotion=staged-new
$ agent-fleet profile status codex-1
exit=0
{
  "profiles": [
    {
      "active_leases": 0,
      "auth_status": "authenticated",
      "cooldown": null,
      "enabled": false,
      "home": "/private/var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e-sandbox/share/accounts/codex/1",
      "id": "codex-1",
      "max_concurrent": 3,
      "pools": [
        "codex-crew",
        "codex-manual"
      ],
      "provider": "codex",
      "provisioned": true,
      "quota": {
        "age_seconds": 13,
        "fresh": true,
        "headroom_percent": 80.0,
        "identity_fingerprint": "027d629b77d7c38718f99b6a88a7eb7efee5736c47dd69e9ebdd061fde28dcd9",
        "identity_source": "quota-account",
        "profile": "codex-1",
        "provider": "codex",
        "reason": null,
        "refreshed_at": "2026-07-18T05:56:19.023508+00:00",
        "reported_status": "fresh",
        "schema": 1,
        "status": "fresh",
        "verification_age_seconds": 13,
        "verified_at": "2026-07-18T05:56:19.021054+00:00",
        "verified_recent": true,
        "windows": [
          {
            "id": "five_hour",
            "kind": "session",
            "remaining_percent": 80.0,
            "resets_at": null
          }
        ]
      },
      "reserve_percent": 15,
      "routeability": {
        "eligible": true,
        "mode": "quota",
        "reason": "fresh"
      },
      "safety_policy": "worker",
      "weight": 1
    }
  ]
}
target marker after automatic recovery=installed-before
durable journal exists after recovery=False

=== result ===
PASS: isolated Agent Fleet CLI workflows satisfied every exercised safety invariant
Evidence: Reproducible E2E harness
from __future__ import annotations

import json
import os
import shutil
import stat
import subprocess
import sys
import time
from dataclasses import replace
from pathlib import Path


PACKAGE_ROOT = Path(sys.argv[1]).resolve()
EVIDENCE_ROOT = Path(sys.argv[2]).resolve()
SANDBOX = EVIDENCE_ROOT / "agent-fleet-e2e-sandbox"

if SANDBOX.parent != EVIDENCE_ROOT:
    raise SystemExit("unsafe evidence sandbox path")
if SANDBOX.exists():
    shutil.rmtree(SANDBOX)
SANDBOX.mkdir(mode=0o700, parents=True)

sys.path.insert(0, str(PACKAGE_ROOT / "src"))

from agent_fleet.config import initial_registry, load_registry, save_registry
from agent_fleet.enrollment import (
    activate_codex_promotion,
    create_codex_login_stage,
    discard_codex_promotion,
    discard_codex_stage,
    prepare_codex_promotion,
)
from agent_fleet.identity import refresh_provider_identity_anchors
from agent_fleet.locks import provider_enrollment_lock
from agent_fleet.provision import provision_profile
from agent_fleet.quota import refresh_quota, snapshot_quota_cache


CONFIG = SANDBOX / "config" / "accounts.toml"
STATE = SANDBOX / "state"
SHARE = SANDBOX / "share"
PROJECT = SANDBOX / "registered-project"
SUPERVISOR = SANDBOX / "orchestrator-cwd"
DESKTOP = SANDBOX / "desktop" / "claude-config.json"
READY = SANDBOX / "worker-ready"
PROVIDER_LOG = SANDBOX / "provider-invocations.jsonl"
PROVIDER = SANDBOX / "fake-provider"
QUOTA = SANDBOX / "quota-axi"

PROJECT.mkdir()
SUPERVISOR.mkdir()
DESKTOP.parent.mkdir()
subprocess.run(["git", "init", "-q", str(PROJECT)], check=True)
DESKTOP.write_text(
    json.dumps({"lastKnownAccountUuid": "desktop-captain-account"}),
    encoding="utf-8",
)

PROVIDER.write_text(
    """#!/usr/bin/env python3
import json
import os
import subprocess
import sys
import time
from pathlib import Path

args = sys.argv[1:]
log = Path(os.environ["EVIDENCE_PROVIDER_LOG"])
with log.open("a", encoding="utf-8") as handle:
    handle.write(json.dumps({"profile": os.environ.get("AGENT_FLEET_PROFILE"), "argv": args}) + "\\n")

if args[:3] == ["auth", "status", "--json"] or args[:2] == ["login", "status"]:
    print(json.dumps({"authenticated": True}))
    raise SystemExit(0)

if "hold-worker" in args:
    Path(os.environ["EVIDENCE_READY"]).write_text(str(os.getpid()), encoding="utf-8")
    time.sleep(60)
    raise SystemExit(0)

session_recorded = False
if "record-session" in args:
    payload = json.dumps({"hook_event_name": "SessionStart", "session_id": "codex-session-001"})
    result = subprocess.run(
        [
            sys.executable,
            "-m",
            "agent_fleet",
            "--format",
            "json",
            "--config",
            os.environ["AGENT_FLEET_CONFIG"],
            "hook",
            "session-start",
        ],
        input=payload,
        text=True,
        capture_output=True,
        check=False,
        env=os.environ,
    )
    if result.returncode != 0:
        print(result.stdout)
        print(result.stderr, file=sys.stderr)
        raise SystemExit(result.returncode)
    session_recorded = True

ambient = sorted(
    name
    for name in (
        "ANTHROPIC_API_KEY",
        "CLAUDE_CODE_OAUTH_TOKEN",
        "OPENAI_API_KEY",
        "CODEX_API_KEY",
    )
    if name in os.environ
)
print(
    json.dumps(
        {
            "provider_argv": args,
            "profile": os.environ.get("AGENT_FLEET_PROFILE"),
            "task": os.environ.get("AGENT_FLEET_TASK_ID"),
            "cwd": os.getcwd(),
            "codex_home": os.environ.get("CODEX_HOME"),
            "claude_config_dir": os.environ.get("CLAUDE_CONFIG_DIR"),
            "disable_login_command": os.environ.get("DISABLE_LOGIN_COMMAND"),
            "disable_logout_command": os.environ.get("DISABLE_LOGOUT_COMMAND"),
            "ambient_provider_variables_present": ambient,
            "session_recorded": session_recorded,
        },
        sort_keys=True,
    )
)
""",
    encoding="utf-8",
)
PROVIDER.chmod(PROVIDER.stat().st_mode | stat.S_IXUSR)

QUOTA.write_text(
    """#!/usr/bin/env python3
import json
import os
import sys
from datetime import UTC, datetime

provider = sys.argv[sys.argv.index("--provider") + 1]
profile = os.environ["AGENT_FLEET_PROFILE"]
print(json.dumps({
    "providers": [{
        "provider": provider,
        "account": {"accountId": profile + "-account"},
        "state": {"status": "fresh", "refreshedAt": datetime.now(UTC).isoformat()},
        "windows": [{"id": "five_hour", "kind": "session", "percentRemaining": 80}],
    }]
}))
""",
    encoding="utf-8",
)
QUOTA.chmod(QUOTA.stat().st_mode | stat.S_IXUSR)

ENV = dict(os.environ)
ENV.update(
    {
        "PYTHONPATH": str(PACKAGE_ROOT / "src"),
        "AGENT_FLEET_CONFIG": str(CONFIG),
        "AGENT_FLEET_STATE_DIR": str(STATE),
        "AGENT_FLEET_SHARE_DIR": str(SHARE),
        "EVIDENCE_READY": str(READY),
        "EVIDENCE_PROVIDER_LOG": str(PROVIDER_LOG),
        "ANTHROPIC_API_KEY": "synthetic-ambient-value",
        "CLAUDE_CODE_OAUTH_TOKEN": "synthetic-ambient-value",
        "OPENAI_API_KEY": "synthetic-ambient-value",
        "CODEX_API_KEY": "synthetic-ambient-value",
    }
)
os.environ.update(ENV)


def section(title: str) -> None:
    print(f"\n=== {title} ===")


def display(command: list[str], result: subprocess.CompletedProcess[str]) -> dict | None:
    print("$ agent-fleet " + " ".join(command))
    print(f"exit={result.returncode}")
    stdout = result.stdout.strip()
    stderr = result.stderr.strip()
    payload = None
    if stdout:
        try:
            payload = json.loads(stdout)
        except json.JSONDecodeError:
            print(stdout)
        else:
            print(json.dumps(payload, indent=2, sort_keys=True))
    if stderr:
        print("stderr=" + stderr)
    return payload


def cli(*command: str, timeout: int = 25) -> tuple[subprocess.CompletedProcess[str], dict | None]:
    result = subprocess.run(
        [
            sys.executable,
            "-m",
            "agent_fleet",
            "--format",
            "json",
            "--config",
            str(CONFIG),
            *command,
        ],
        cwd=SUPERVISOR,
        env=ENV,
        text=True,
        capture_output=True,
        timeout=timeout,
        check=False,
    )
    return result, display(list(command), result)


registry = initial_registry(1, 1)
registry = replace(registry, settings=replace(registry.settings, quota_binary=QUOTA))
providers = {}
for provider_name, provider in registry.providers.items():
    base = SANDBOX / "base" / provider_name
    base.mkdir(parents=True)
    hooks = base / ("settings.json" if provider_name == "claude" else "hooks.json")
    hooks.write_text(
        json.dumps(
            {
                "hooks": {
                    "SessionStart": [
                        {
                            "matcher": "startup",
                            "hooks": [{"type": "command", "command": "base-hook"}],
                        }
                    ]
                }
            }
        ),
        encoding="utf-8",
    )
    shared = "CLAUDE.md" if provider_name == "claude" else "AGENTS.md"
    (base / shared).write_text("synthetic workflow rules\n", encoding="utf-8")
    providers[provider_name] = replace(
        provider,
        binary=PROVIDER,
        base_home=base,
        hooks_source=hooks,
        shared_entries=(shared,),
        desktop_identity_file=DESKTOP if provider_name == "claude" else None,
        trusted_projects=(PROJECT.resolve(),),
    )
registry = replace(registry, providers=providers)
save_registry(registry, CONFIG)

for profile in registry.profiles.values():
    provision_profile(registry, profile)
    refresh_quota(registry, profile)
for provider_name in registry.providers:
    refresh_provider_identity_anchors(registry, provider_name)
profiles = {key: replace(value, enabled=True) for key, value in registry.profiles.items()}
registry = replace(registry, profiles=profiles)
save_registry(registry, CONFIG)

section("independently versioned CLI contract")
result, payload = cli("version")
assert result.returncode == 0 and payload["cli_version"] == "0.2.0"
result, payload = cli("contract")
assert result.returncode == 0
assert payload["commands"]["exec"].find("--workspace") >= 0

section("opaque remote identity evidence")
result, payload = cli("quota", "show", "--all")
assert result.returncode == 0
for item in payload["quota"]:
    assert len(item["identity_fingerprint"]) == 64
    assert "account" not in item

section("managed Codex launch and sticky resume")
result, first_exec = cli(
    "exec",
    "--task",
    "sticky-codex-task",
    "--pool",
    "codex-crew",
    "--profile",
    "codex-1",
    "--workspace",
    str(PROJECT),
    "--",
    "record-session",
)
assert result.returncode == 0
assert first_exec["profile"] == "codex-1"
assert first_exec["cwd"] == str(PROJECT)
assert first_exec["ambient_provider_variables_present"] == []
assert first_exec["provider_argv"][:4] == ["--disable", "plugins", "--disable", "plugin_sharing"]

result, mapping = cli("session", "status", "--task", "sticky-codex-task")
assert result.returncode == 0
assert mapping["profile"] == "codex-1"
assert mapping["session_id"] == "codex-session-001"
assert mapping["workspace"] == str(PROJECT)

result, resumed = cli(
    "resume",
    "--task",
    "sticky-codex-task",
    "--workspace",
    str(PROJECT),
    "--",
    "resume-check",
)
assert result.returncode == 0
assert resumed["profile"] == "codex-1"
assert resumed["codex_home"] == first_exec["codex_home"]
assert resumed["provider_argv"][0] == "resume"
assert "codex-session-001" in resumed["provider_argv"]

section("managed Claude launch isolation")
result, claude_exec = cli(
    "exec",
    "--task",
    "claude-env-task",
    "--pool",
    "claude-crew",
    "--profile",
    "claude-1",
    "--workspace",
    str(PROJECT),
    "--",
    "env-check",
)
assert result.returncode == 0
assert claude_exec["disable_login_command"] == "1"
assert claude_exec["disable_logout_command"] == "1"
assert claude_exec["ambient_provider_variables_present"] == []
assert claude_exec["claude_config_dir"].endswith("/claude/1")

section("Claude Desktop overlap and unavailable proof fail closed")
result, payload = cli(
    "choose",
    "--task",
    "before-desktop-switch",
    "--pool",
    "claude-crew",
    "--provider",
    "claude",
    "--profile",
    "claude-1",
    "--workspace",
    str(PROJECT),
    "--dry-run",
)
assert result.returncode == 0 and payload["profile"] == "claude-1"

DESKTOP.write_text(
    json.dumps({"lastKnownAccountUuid": "claude-1-account"}),
    encoding="utf-8",
)
result, payload = cli(
    "choose",
    "--task",
    "after-desktop-switch",
    "--pool",
    "claude-crew",
    "--provider",
    "claude",
    "--profile",
    "claude-1",
    "--workspace",
    str(PROJECT),
    "--dry-run",
)
assert result.returncode == 2
assert "duplicate_provider_identity" in payload["error"]

DESKTOP.unlink()
result, payload = cli(
    "choose",
    "--task",
    "desktop-proof-unavailable",
    "--pool",
    "claude-crew",
    "--provider",
    "claude",
    "--profile",
    "claude-1",
    "--workspace",
    str(PROJECT),
    "--dry-run",
)
assert result.returncode == 2
assert "duplicate_provider_identity" in payload["error"]
DESKTOP.write_text(
    json.dumps({"lastKnownAccountUuid": "desktop-captain-account"}),
    encoding="utf-8",
)

section("provider maintenance lock serializes registry changes")
registry = load_registry(CONFIG)
with provider_enrollment_lock(
    registry.settings.state_dir,
    "claude",
    registry.settings.lock_stale_seconds,
):
    result, payload = cli("profile", "add", "claude-2", "--provider", "claude")
assert result.returncode == 2
assert "provider maintenance is already in progress" in payload["error"]
assert "claude-2" not in load_registry(CONFIG).profiles

section("live task ownership and same-provider auth maintenance")
READY.unlink(missing_ok=True)
worker_command = [
    sys.executable,
    "-m",
    "agent_fleet",
    "--format",
    "json",
    "--config",
    str(CONFIG),
    "exec",
    "--task",
    "live-owned-task",
    "--pool",
    "codex-crew",
    "--profile",
    "codex-1",
    "--workspace",
    str(PROJECT),
    "--",
    "hold-worker",
]
worker = subprocess.Popen(
    worker_command,
    cwd=SUPERVISOR,
    env=ENV,
    text=True,
    stdout=subprocess.PIPE,
    stderr=subprocess.PIPE,
)
try:
    deadline = time.monotonic() + 15
    while not READY.exists() and time.monotonic() < deadline:
        if worker.poll() is not None:
            stdout, stderr = worker.communicate()
            raise AssertionError(f"worker exited before ready: {stdout} {stderr}")
        time.sleep(0.05)
    assert READY.exists(), "worker did not publish readiness"
    print(f"live worker pid={READY.read_text(encoding='utf-8').strip()}")

    result, payload = cli(
        "exec",
        "--task",
        "live-owned-task",
        "--pool",
        "codex-crew",
        "--profile",
        "codex-1",
        "--workspace",
        str(PROJECT),
        "--",
        "second-owner-attempt",
    )
    assert result.returncode == 2
    assert "already owned by live process" in payload["error"]

    result, payload = cli("profile", "disable", "codex-1")
    assert result.returncode == 0 and payload["enabled"] is False
    before_login = sum(
        1
        for line in PROVIDER_LOG.read_text(encoding="utf-8").splitlines()
        if json.loads(line)["argv"][:1] == ["login"]
    )
    result, payload = cli("profile", "enroll", "codex-1")
    assert result.returncode == 2
    assert "while any same-provider Fleet lease is active" in payload["error"]
    after_login = sum(
        1
        for line in PROVIDER_LOG.read_text(encoding="utf-8").splitlines()
        if json.loads(line)["argv"][:1] == ["login"]
    )
    print(f"provider login invocations during refused enrollment={after_login - before_login}")
    assert after_login == before_login
finally:
    worker.terminate()
    try:
        worker.communicate(timeout=5)
    except subprocess.TimeoutExpired:
        worker.kill()
        worker.communicate(timeout=5)

result, payload = cli("lease", "release", "--task", "live-owned-task", "--force")
assert result.returncode == 0 and payload["released"] is True

section("Codex staged promotion crash recovery")
registry = load_registry(CONFIG)
target = registry.require_profile("codex-1")
provision_profile(registry, target)
target_auth = target.home / "auth.json"
target_auth.write_text(json.dumps({"synthetic_marker": "installed-before"}), encoding="utf-8")
target_auth.chmod(0o600)
snapshot = snapshot_quota_cache(registry, target.id)
stage = create_codex_login_stage(registry, target)
promotion = None
try:
    stage_auth = stage.home / "auth.json"
    stage_auth.write_text(json.dumps({"synthetic_marker": "staged-new"}), encoding="utf-8")
    stage_auth.chmod(0o600)
    promotion = prepare_codex_promotion(registry, target, stage)
    transaction = activate_codex_promotion(registry, target, promotion, snapshot)
    journal = json.loads(transaction.journal_path.read_text(encoding="utf-8"))
    print(f"staging parent={stage.home.parent}")
    print(f"target home={target.home}")
    print(f"durable journal phase={journal['phase']}")
    print(
        "target marker after atomic promotion="
        + json.loads(target_auth.read_text(encoding="utf-8"))["synthetic_marker"]
    )
    assert stage.home.parent == STATE / "staging" / "codex"
    assert journal["phase"] == "prepared"

    result, payload = cli("profile", "status", "codex-1")
    assert result.returncode == 0
    restored = json.loads(target_auth.read_text(encoding="utf-8"))["synthetic_marker"]
    print(f"target marker after automatic recovery={restored}")
    print(f"durable journal exists after recovery={transaction.journal_path.exists()}")
    assert restored == "installed-before"
    assert not transaction.journal_path.exists()
finally:
    if promotion is not None and promotion.home.exists():
        discard_codex_promotion(promotion, target)
    if stage.home.exists():
        discard_codex_stage(registry, stage, target)

section("result")
print("PASS: isolated Agent Fleet CLI workflows satisfied every exercised safety invariant")
Evidence: Built-wheel installation smoke

installed_distribution=agent-fleet installed_version=0.2.0 console_entrypoint=agent_fleet.cli:main {"cli_version":"0.2.0","contract_version":1}

installed_distribution=agent-fleet
installed_version=0.2.0
console_entrypoint=agent_fleet.cli:main
{"cli_version":"0.2.0","contract_version":1}
- Evidence: Agent Fleet 0.2.0 wheel (local file: /var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-dist/agent_fleet-0.2.0-py3-none-any.whl) - Evidence: Agent Fleet 0.2.0 source distribution (local file: /var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-dist/agent_fleet-0.2.0.tar.gz) - Outcome: 🔧 1 issue found → auto-fixed ✅ across 2 runs (1h16m59s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 3 issues (1 error, 2 warnings)
  • 🚨 tools/agent-fleet/src/agent_fleet/enrollment.py:462 - The required criterion “durable transaction journal, and crash recovery” is not met when rolling back an existing credential. Restoring the copied backup creates a different inode from original_stat; if the process crashes before quota restoration and journal deletion, the next recovery rejects the already-restored auth as an outside change and permanently blocks recovery. Record an idempotent rollback phase/restored identity before replacing the file.
  • 🚨 tools/agent-fleet/src/agent_fleet/cli.py:417 - The criteria “verify opaque remote identity sets” and “fail closed on … unavailable proof” are contradicted here. After Claude login changes a profile’s credential, keychain_access_required can cause verification to reuse the pre-login cached fingerprint; auth_probe supplies no identity to bind that cache to the new credential, so a different account can be enabled using the old account’s proof. Invalidate proof after login or cryptographically bind cached proof to the current credential/home.
  • 🚨 tools/agent-fleet/src/agent_fleet/cli.py:992 - The criteria “refuse auth maintenance while same-provider leases are live,” non-destructive Codex enrollment, and sticky managed resume are bypassable because worker exec forwards arbitrary provider subcommands. For example, exec … -- login mutates Codex auth outside the maintenance lock/staging journal, while exec … -- resume &lt;id&gt; bypasses the recorded task/session mapping. Reject auth and resume subcommands in exec, leaving them to the dedicated enrollment/resume paths.
  • 🚨 tools/agent-fleet/src/agent_fleet/cli.py:694 - The required serialization of profile changes under provider maintenance locks is contradicted by init --force, which directly replaces the entire registry. It can remove or redirect profiles while their leases are live and can orphan pending Codex journals. Existing registries must be replaced only under all provider locks with live-lease and transaction checks, or force replacement should be removed.
  • 🚨 tools/agent-fleet/src/agent_fleet/enrollment.py:178 - The required non-destructive isolated Codex staging behavior is contradicted for custom profile homes. ensure_private_dir(profile.home.parent) unconditionally chmods an existing parent to 0700, so enrolling a profile under a user-owned shared/project directory silently changes that directory’s permissions. Require an already-private Fleet-owned parent or stage under a dedicated managed directory without mutating arbitrary parents.
  • 🚨 tools/agent-fleet/src/agent_fleet/config.py:171 - The isolated-home criterion and README invariant that auth/provider state is “never shared” are not enforced. shared_entries accepts any simple filename, so entries such as Codex auth.json or Claude credential/state files are symlinked from the base home during provisioning; enrollment may then operate on the base credential before identity checks run. Restrict this field to an explicit provider-specific workflow-asset allowlist.
  • ⚠️ tools/agent-fleet/src/agent_fleet/cli.py:1072 - Mutations complete before the default TOON encoder is checked. On a standalone installation without toon, documented commands such as init or profile enroll can successfully change registry/auth state and then exit 2 during output, encouraging unsafe retries. Preflight the selected encoder before _run, bundle the dependency, or use a built-in default format.
  • ⚠️ tools/agent-fleet/README.md:53 - The quickstart initializes three Claude workers but enrolls only claude-1; the subsequent enable always fails because complete-worker-set verification requires recent identity proof for claude-2 and claude-3. Initialize one profile per provider in this example or document enrollment and verification of the entire configured worker set before enabling any member.

🔧 Fix: Harden Agent Fleet trust and recovery
6 errors still open:

  • 🚨 tools/agent-fleet/src/agent_fleet/scheduler.py:158 - The required Gate-E behavior to refuse an unallowlisted cwd before lease creation, while preserving the Firstmate/Bridge architecture, is not met. Selection validates invocation_workspace(), but Firstmate calls lease choose from the supervisor process cwd while the actual task worktree remains in $WT. A registered task can therefore be rejected, or an unregistered task can receive a lease when the supervisor repo is registered and fail only during exec. Pass and validate the intended task worktree during choose/recover.
  • 🚨 tools/agent-fleet/src/agent_fleet/providers.py:47 - The required criterion “Preserve sticky managed resume” remains bypassable for Claude. -c, Claude’s short form of --continue, is absent from the blocked arguments, so exec … -- -c resumes the most recent unmanaged session without the recorded task/session mapping. Block Claude -c and attached -r&lt;session&gt; forms.
  • 🚨 tools/agent-fleet/src/agent_fleet/providers.py:92 - Codex configuration filtering handles only separated -c VALUE arguments. Attached short-option values such as -cfeatures.plugins=true bypass validation and appear after Fleet’s --disable plugins, allowing managed plugin, hook, or trust settings to be overridden. Reject or safely parse attached -c… and -p… forms.
  • 🚨 tools/agent-fleet/src/agent_fleet/provision.py:209 - The required Gate-E behavior to fail closed on changed hook markers is incomplete. session_command is only type-checked and then treated as the expected command; changing it in both the marker and hooks file and recomputing the unkeyed hooks_hash passes readiness with an arbitrary SessionStart command. Derive the expected command independently with session_hook_command() and compare the marker to it.
  • 🚨 tools/agent-fleet/src/agent_fleet/identity.py:207 - The criterion “fail closed on base/Desktop overlap or unavailable proof” is contradicted when a configured base home is absent or moved. Both anchor refresh and conflict checking skip the base boundary unless base_home.exists(), allowing workers to route without any proof of the default CLI identity. Treat a configured-but-missing base home as base_identity_unverified.
  • 🚨 tools/agent-fleet/src/agent_fleet/quota.py:283 - The criteria “verify opaque remote identity sets” and “fail closed on … unavailable proof” are bypassable because the installed CLI honors AGENT_FLEET_QUOTA_FIXTURE_DIR and accepts local JSON instead of invoking the remote quota reader. A stale ambient test variable can fabricate fresh identities and quota evidence. Replace this production environment seam with test-only dependency injection or an explicitly isolated test mode that cannot enable or lease profiles.

🔧 Fix: Harden Agent Fleet routing and trust boundaries
3 issues (1 error, 2 warnings) still open:

  • 🚨 tools/agent-fleet/src/agent_fleet/providers.py:93 - The required criterion “keep worker launch task-bound with verified process ownership” is still contradicted. This denylist misses Claude’s documented -w and --bg aliases plus positional setup-token and agents commands, allowing callers to escape the verified workspace, create background sessions that outlive the PID-bound lease, or perform auth/session administration (Claude CLI reference). Prefer an allowlist of the exact Firstmate-supported arguments.
  • ⚠️ bin/fm-account-routing-lib.sh:88 - Firstmate kills control commands after 10 seconds, but one Fleet auth probe alone permits 15 seconds, after base/quota refreshes and potentially once per profile. A healthy slow selection can therefore be terminated while holding the provider lock, leaving routing unavailable until stale-lock recovery. Align the outer deadline with Fleet’s bounded worst case or parallelize/bound the probes below it.
  • ⚠️ tools/agent-fleet/src/agent_fleet/config.py:397 - Every registry save unconditionally changes the existing config parent to mode 0700. Because --config accepts arbitrary paths, initializing or mutating a registry inside a project or shared dotfiles directory silently removes group/world access from that whole directory. Only harden a newly created Fleet-owned directory, or reject an unsuitable existing parent without modifying it.
🔧 **Test** - 1 issue found → auto-fixed ✅
  • 🚨 tests failed with exit code 1
  • command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; exit "$rc"

🔧 Fix: Captain, fix Agent Fleet lint parity assertion
✅ Re-checked - no issues remain.

  • command -v tmux >/dev/null || { echo "tmux is required for e2e tests" >&2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo "== $t =="; bash "$t" || rc=1; done; exit "$rc"
  • Supplied passing baseline: command -v tmux &gt;/dev/null || { echo &#34;tmux is required for e2e tests&#34; &gt;&amp;2; exit 1; }; tmux -V; rc=0; for t in tests/*.test.sh; do echo &#34;== $t ==&#34;; bash &#34;$t&#34; || rc=1; done; exit &#34;$rc&#34;
  • cd tools/agent-fleet && uv run --locked pytest
  • FM_TEST_FOCUSED=gate-e-workspace bash tests/fm-account-routing.test.sh
  • PYTHONDONTWRITEBYTECODE=1 python3 /var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-e2e.py /Users/dongkeun/.no-mistakes/worktrees/b174997b7206/01KXSJTEG548G0EWKQ9JWFTFQT/tools/agent-fleet /var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT
  • cd tools/agent-fleet && uv build --out-dir /var/folders/y_/bfdbj_vx20l9b9tw7crgkzwm0000gn/T/no-mistakes-evidence/01KXSJTEG548G0EWKQ9JWFTFQT/agent-fleet-dist
  • Installed agent_fleet-0.2.0-py3-none-any.whl without dependencies into an isolated virtual environment and ran agent-fleet --format json version
  • python3 -m py_compile tools/agent-fleet/src/agent_fleet/cli.py tools/agent-fleet/src/agent_fleet/enrollment.py
⚠️ **Document** - 1 error
  • 🚨 tools/agent-fleet/tests/test_quota_scheduler.py:1 - Locked ruff validation still fails with I001 for an unsorted import block. This executable test file is outside the documentation-only edit scope.

🔧 Fix: Correct Agent Fleet runtime documentation
1 error still open:

  • 🚨 tools/agent-fleet/tests/test_quota_scheduler.py:1 - Locked ruff validation still fails with I001 for an unsorted import block. Fixing it requires editing executable test code, outside this documentation-only scope.
✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@ruby-dlee ruby-dlee closed this Jul 19, 2026
@ruby-dlee
ruby-dlee force-pushed the codex/bridge-agent-fleet-v015 branch from 58cde83 to ac74baa Compare July 19, 2026 03:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant