Ordinary-resume orphans retained agents when the coordinator crashes before journaling resume-active

[davidfarah2003]

Summary

Post-merge review of space-backup (#244, commit 97123f9) found a crash-window bug in the ordinary-resume path that is the mirror of the cut-committed bug fixed in #251: an action (launching the retained agents) happens before the journal records it, so a crash in the gap orphans live agents that recovery cannot adopt.

Status: confirmed by code reading; NOT yet reproduced live. Per repo policy (reproduce live before fixing), a faithful repro needs infrastructure that does not yet exist — see "Reproducing" below. Filing with the exact analysis and fix design so a focused follow-up can build the repro and land the fix.

The window (confirmed)

implementations/cli/src/commands/up.ts, in the resume orchestration:

1. resumePreserved control RPC (~line 1198–1211) — the manager launches the retained agents and returns success.
2. markOrdinaryResumeActive (~line 1224) — journals resume-active.

A coordinator crash between (1) and (2) leaves the journal at resume-intent while the retained agents are already live. If the manager is then lost (ungraceful, children survive as detached processes), recovery reissues resumePreserved, whose preflight rejects every retained principal still present in the roster:

implementations/manager/src/manager.ts:2182

if (livePrincipals.has(principal))
  return { ok: false, agents: [], error: `retained principal "${principal}" is already live and this runtime cannot authoritatively adopt it` };

livePrincipals is built from the presence roster (manager.ts:2162–2164). So the replacement manager refuses to adopt the agents it is supposed to own, the journal can only go to resume-degraded, and there is no authoritative child ownership to adopt or terminate those processes — forward recovery is stranded.

Impact

An ordinary cotal up after down --preserve-state, interrupted at exactly this point with the manager subsequently lost, strands the mesh: retained agents are live but unowned, and the resume cannot complete or cleanly tear them down. POSIX-safe otherwise (crash-window only).

Reproducing (why it needs new infrastructure)

The harm only manifests with real retained agents — without them resumePreserved launches nothing and the window is harmless. A retained agent launched by the real cotal subprocess requires a connector the subprocess binary can launch and retain. The lightweight e2e test connector (used by spawn-detach-live.smoke.ts) is registered in-process only; every manifest live smoke deliberately uses agents: {} because booting a real connector subprocess is heavy.

A faithful live repro therefore needs:

1. A minimal installable test-connector package (a real package, not the in-process e2e object), added via cotal ext add, so the subprocess manager can launch + retain it.
2. A launch manifest that satisfies retention (launch.source.configPath as a retained dep, a runId, 0600 identity files — see inventoryReferenceError, manager.ts:802+).
3. An unsafe-side fault hook on the ordinary-resume path (the existing COTAL_SMOKE_EXIT_AFTER_RESUME_PRESERVED at up.ts:1218 is gated if (restored && …) — restore path only; an ordinary-path exit hook between the resumePreserved return and markOrdinaryResumeActive is needed).
4. Manager-kill with the retained agent surviving, then a recovery up that reproduces the degraded stranding.

An in-process simulation (skipping the journal write) does NOT satisfy the reproduce-live rule.

Proposed fix (mirror of #251)

1. Journal an activation intent (and per-child ownership) BEFORE the resumePreserved RPC launches agents, and clear/advance it once resume-active is durable — so recovery can tell "agents may already be live" from "not yet launched."
2. In the manager's resumePreserved preflight, when a retained principal is present AND it belongs to THIS attempt's inventory (i.e., an agent this resume launched and journaled ownership for), adopt it rather than rejecting as foreign. A truly foreign same-id principal still refuses.
3. Recovery finishes forward (adopt-in-place) or deterministically terminates the retained children using the journaled ownership, instead of degrading with no recourse.

Related

• fix(cli): space-backup post-merge review remediation (Windows ACLs + cut-committed crash window) #251 — the cut-committed sibling (fixed): same "journal-after-action" class, with the durable-marker + finish-forward pattern this fix should mirror.
• Verify: concurrent ordinary-resume may race two brokers / mis-bind the journal (unconfirmed) #250 — unverified concurrent-resume race (separate).