From 9d92983899cd81e13844d9c84bf687d54ac886db Mon Sep 17 00:00:00 2001 From: Dongkeun Lee Date: Thu, 30 Jul 2026 21:38:24 -0400 Subject: [PATCH 1/9] build durable Lavish decisions --- .agents/skills/bearings/SKILL.md | 4 +- .../skills/lavish-decision-boards/SKILL.md | 71 -- .../assets/lavish-board-template.html | 201 ----- .agents/skills/lavish-decisions/SKILL.md | 57 ++ .gitignore | 2 + AGENTS.md | 12 +- bin/fm-bootstrap.sh | 19 +- bin/fm-lavish-intake.sh | 19 + bin/fm-lavish-wake.sh | 80 ++ bin/fm-wake-drain.sh | 14 + docs/configuration.md | 7 +- docs/scripts.md | 2 + tests/fm-bootstrap.test.sh | 36 + tests/lavish.test.sh | 9 + tests/operating-fundamentals.test.sh | 10 +- tools/lavish/LICENSE | 21 + tools/lavish/PROVENANCE.md | 8 + tools/lavish/README.md | 117 +++ tools/lavish/package-lock.json | 29 + tools/lavish/package.json | 28 + tools/lavish/src/cli.mjs | 450 ++++++++++ tools/lavish/src/migration.mjs | 284 +++++++ tools/lavish/src/protocol.mjs | 784 ++++++++++++++++++ tools/lavish/test/lavish.test.mjs | 562 +++++++++++++ 24 files changed, 2537 insertions(+), 289 deletions(-) delete mode 100644 .agents/skills/lavish-decision-boards/SKILL.md delete mode 100644 .agents/skills/lavish-decision-boards/assets/lavish-board-template.html create mode 100644 .agents/skills/lavish-decisions/SKILL.md create mode 100755 bin/fm-lavish-intake.sh create mode 100755 bin/fm-lavish-wake.sh create mode 100755 tests/lavish.test.sh create mode 100644 tools/lavish/LICENSE create mode 100644 tools/lavish/PROVENANCE.md create mode 100644 tools/lavish/README.md create mode 100644 tools/lavish/package-lock.json create mode 100644 tools/lavish/package.json create mode 100755 tools/lavish/src/cli.mjs create mode 100644 tools/lavish/src/migration.mjs create mode 100644 tools/lavish/src/protocol.mjs create mode 100644 tools/lavish/test/lavish.test.mjs diff --git a/.agents/skills/bearings/SKILL.md b/.agents/skills/bearings/SKILL.md index 6aa17631cb..0951f2795d 100644 --- a/.agents/skills/bearings/SKILL.md +++ b/.agents/skills/bearings/SKILL.md @@ -32,7 +32,7 @@ It never tears down a task, merges a PR, dispatches new work, or mutates any tas - **Title** - `# Bearings - ` (use "Morning status" only when the captain specifically asks for a morning brief), followed by two or three sentences framing where things stand. - **Captain's Call** - every open decision relayed verbatim with its options, plus each PR ready to merge and each needed credential or login, every PR with the full `https://...` URL, never a bare `#number`. - **Recently Landed** - merged PRs, completed scouts, and finished local-only work since the last report, across the main fleet and every registered secondmate home. - - **Underway** - each live direct report making progress, with its current state, and the plans / main pickup pointers worth reopening (`data//report.md` files, `.lavish/*.html` boards). + - **Underway** - each live direct report making progress, with its current state, and the plans or main pickup pointers worth reopening (`data//report.md` files and pending `data/decisions//request.md` records). - **Charted Next** - queued or gated next work, with each item's blocker or date reason. 3. **Write the dated report file so it persists, then surface the mandatory four-section digest in chat.** @@ -40,7 +40,7 @@ It never tears down a task, merges a PR, dispatches new work, or mutates any tas This is the required artifact; it lives in gitignored `data/` alongside the worked example. If today's file already exists, delete it first, then create a new file from scratch. - The chat response is the concise four-section digest defined by the contract below: materially shorter than the report file, and it links to that file for the full picture. - - For a richer review surface, optionally offer a Lavish board with `lavish-axi` when the report has enough structure to deserve one, but the markdown file is the required artifact and the four-section chat digest is the required minimum. + - When the report exposes multiple genuine captain-owned choices, optionally create a durable Lavish decision with `lavish-axi`, but the Markdown file is the required artifact and the four-section chat digest is the required minimum. ## Chat-response contract diff --git a/.agents/skills/lavish-decision-boards/SKILL.md b/.agents/skills/lavish-decision-boards/SKILL.md deleted file mode 100644 index 45bdb5542e..0000000000 --- a/.agents/skills/lavish-decision-boards/SKILL.md +++ /dev/null @@ -1,71 +0,0 @@ ---- -name: lavish-decision-boards -description: >- - Agent-only workflow for building actionable, captain-facing Lavish decision boards and approval or triage surfaces. - Use before creating or revising any Lavish board that asks the captain to choose options, approve a plan, triage findings, set scope, or provide structured feedback. -user-invocable: false -metadata: - internal: true ---- - -# lavish-decision-boards - -Build a captain-facing decision surface as an actionable, layout-safe Lavish board. -A read-only status page is not a decision surface. -A Lavish board is a live surface, so reconcile it against live fleet state before serving or updating it and never render it from a remembered snapshot; `AGENTS.md` section 9 owns the serve-fresh rule. - -## Build - -1. Copy `assets/lavish-board-template.html` from this skill directory into the board's working location. - Use the asset as the starting point instead of recreating its typography or layout reset. - Do not tighten its line heights, tracking, text padding, wrapping, or card spacing because those values are the layout-audit-safe baseline. -2. Run `lavish-axi playbook input` before writing the board, plus every other playbook that matches the content. - Treat `lavish-axi --help` as the authority for current CLI behavior and flags. -3. Give every decision a native radio group or select control, but hold all choices in local page state until one explicit `Send answers` action. - Never call `window.lavish.queuePrompt` on selection, change, or a per-question step; a partial selection must never be actionable. - The single send handler must first gather and validate all current form state, then call `window.lavish.queuePrompt` exactly once with one structured batch envelope and immediately call `window.lavish.sendQueuedPrompts()` in that same click handler. - The envelope must contain `submission: "explicit-send-batch"`, an `answers` array, and a manifest with the full expected question-key set and count. - Reject missing or duplicate question keys before queueing. - Acquire a one-shot in-flight lock and disable the send button before queueing so rapid clicks cannot duplicate the batch. - This keeps the Lavish queue empty before the explicit send, so a disconnect has nothing to auto-flush and in-progress input cannot reach the agent. -4. Choose a durable feedback destination before polling, such as the task spec, backlog note, or task data file. - -## Show the actionable plan - -- Include every in-flight item; silently omitted work makes the board incomplete. -- For each item, show the real end-to-end path to done: every remaining step, the owner or condition gating it, and the concrete downstream consequence when it lands; never substitute a bare status label. -- When work spans execution resources, show its capacity-aware distribution by lane, harness, and model using current capacity and quota without embedding values that will rot. -- State the pacing and concurrency limits being held to protect fleet health. -- Clearly separate work proceeding autonomously under standing rules from items that genuinely require the captain's decision, and give decision controls only to the latter. - -## Serve and verify - -1. Serve the board without auto-opening by passing `--no-open` or setting `LAVISH_AXI_NO_OPEN=1`. -2. Extract the printed session URL without its surrounding double quotes. - A safe extraction pattern is `grep -oE 'https?://[^ "]+'`; verify that no quote or punctuation trails the URL. -3. Open every board in its own dedicated Chrome window with `open -na "Google Chrome" --args --new-window ""`. -4. Start `lavish-axi poll ` silently and leave it running while review continues. - Re-run it after every response while review continues. - Treat the board as not ready until the tool-authoritative connection signal confirms that it is genuinely connected. - The transition to connected can take several minutes and is expected; during that lag, do not act on a poll return or prematurely re-serve, re-open, abandon, or otherwise thrash the session. - Create and select a `chrome-devtools-axi` page for the exact served URL, then use a bounded retry to inspect that page's snapshots. - Pass only after the layout-audit-in-progress indicator has cleared and no layout-issue indicator is present, confirming zero error-severity `layout_warnings` for that board. - A returned snapshot alone is not success; if the bound expires while the audit remains in progress, treat the board as unverified and do not surface it. - Consult `chrome-devtools-axi --help` and the relevant command help for current commands and flags. - Do not announce the board as ready until that check passes. - If the audit finds an error while the layout gate is still holding the board, fix it and verify again before the captain can answer. -5. Name the board when surfacing it so the captain knows which decision surface is awaiting action. - Accompany it in the CLI with only a bare pointer to the board, never the substantive decision content. - -## Protect answers - -- Answer preservation takes precedence over the serve-fresh rule while the captain has unsubmitted input. -- Never edit, refresh, or reload a served board while the captain is answering because doing so clears in-progress input. -- After submission, reconcile and refresh before continuing; if freshness must be preserved sooner, use only a strategy proven to retain the captain's current input without editing, refreshing, or reloading the served board. -- When poll feedback arrives, write every annotation to the chosen durable file immediately, before interpreting it, acting on it, or doing anything else. -- Never rely on poll output or conversation memory as the only copy because ephemeral poll output can be reaped. -- Treat a `lavish-axi poll` return as transport or lifecycle output, not automatically as the captain's answer. -- Act only after genuine connection and receipt of one unambiguous batch with `submission: "explicit-send-batch"`, a nonempty structured `answers` array, and a manifest containing `expectedQuestionKeys` and `expectedCount`. -- Accept the batch only when the manifest keys are nonempty and unique, its count equals its key count, and every manifest key has exactly one structured answer entry with the same key and a nonempty `answer` value. -- Reject absent, empty, extra, or duplicate answer keys, every manifest mismatch, and every disconnect, UI flicker, re-poll, layout or audit return, session event, unmarked return, or ambiguous return; ignore them and keep waiting. -- If a return is ambiguous or lacks a clear explicit-decision payload, treat it as not submitted and do not act until the captain's actual answer is verified in the payload. diff --git a/.agents/skills/lavish-decision-boards/assets/lavish-board-template.html b/.agents/skills/lavish-decision-boards/assets/lavish-board-template.html deleted file mode 100644 index d26e0f9bfb..0000000000 --- a/.agents/skills/lavish-decision-boards/assets/lavish-board-template.html +++ /dev/null @@ -1,201 +0,0 @@ - - - - - - -Decision board - - - -
-

Decision board title

- - -
- decision needed -

What should be decided?

-

State the relevant context in one or two sentences.

-
Recommendation: Choose option A because it best matches the stated goal.
-
-
- - -
-

No answer selected.

-
-
- - - -
- - - diff --git a/.agents/skills/lavish-decisions/SKILL.md b/.agents/skills/lavish-decisions/SKILL.md new file mode 100644 index 0000000000..322f1c0e93 --- /dev/null +++ b/.agents/skills/lavish-decisions/SKILL.md @@ -0,0 +1,57 @@ +--- +name: lavish-decisions +description: >- + Agent-only workflow for creating durable captain-facing Lavish decisions. + Use before creating or revising a multi-option decision, approval batch, triage request, or other structured captain input that must remain answerable asynchronously. + Do not use it for read-only reports or simple yes/no questions. +user-invocable: false +metadata: + internal: true +--- + +# Lavish decisions + +Create a self-contained, durable decision that the captain can answer when convenient, even when no firstmate process is running. +Use the firstmate-owned `lavish-axi` file protocol documented in `tools/lavish/README.md`. + +## Create + +1. Reconcile the proposed decision against live fleet state immediately before writing it. +2. Write complete human context to Markdown. + Include the decision, recommendation, alternatives, consequences, and what happens after each choice. +3. Define the ordered questions and options in the JSON shape documented by the tool. + Use nonempty unique lowercase-slug keys. +4. Choose a durable `$FM_HOME`-relative destination below `data/`. + This is where intake commits the validated answer before writing its receipt. +5. Run `lavish-axi create` with a stable decision id, title, Markdown request, question JSON, and destination. +6. Run `lavish show ` and `lavish inbox` to verify the exact durable request. +7. Surface only the title and command: + + ```text + Decision waiting: + Run: lavish answer + ``` + +Do not edit `request.md` or `manifest.toon` after surfacing the decision. +Their digest and ordered question set are the immutable contract. + +## Consume + +Firstmate's ordinary wake drain and session start invoke Lavish intake. +The answer file is authoritative; the wake record is only a pointer. + +When a destination appears: + +1. Read the full validated answer from the declared destination. +2. Revalidate any execution preconditions that may have changed while the decision waited. +3. Act on the captain's complete batch without weakening or reinterpreting it. +4. Update the owning task or backlog record so the captain-gated thread is durably closed. + +## Reliability boundary + +Never start a server, open a browser, create or share a session URL, poll, long-poll, register a filesystem watcher, schedule a timer sweep, or launch a resident process for Lavish decision capture. +Do not use upstream `serve`, `poll`, browser, layout-audit, or session-lifecycle commands. + +Every Lavish command must finish its bounded local file operation and exit. +If `lavish answer` reports `answer saved; wake not queued`, do not ask the captain to answer again. +The next ordinary intake scan recovers the durable unreceipted answer. diff --git a/.gitignore b/.gitignore index 8110029572..8ca2ab89b7 100644 --- a/.gitignore +++ b/.gitignore @@ -3,6 +3,7 @@ state/ data/ .no-mistakes/ .lavish/ +tools/lavish/node_modules/ .fm-secondmate-home .DS_Store __pycache__/ @@ -21,3 +22,4 @@ config/x-mode.env config/watcher.env config/cmux-socket-password config/wedge-alarm +config/lavish-wake-command diff --git a/AGENTS.md b/AGENTS.md index 0e5b98e774..5c32cbb4af 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -182,7 +182,7 @@ If the digest reported `data/projects.md` as `ABSENT` or disagreeing with what i An `ABSENT` `data/captain.md` or `data/secondmates.md` or `data/learnings.md` means exactly what section 2 says it means (template defaults, no registered secondmates, nothing captured yet) - not a problem to fix. Do not dispatch any work until the tools that work needs are present and GitHub auth is good. -Use `gh-axi` for all GitHub operations, `chrome-devtools-axi` for all browser operations, and `lavish-axi` when a decision or report is complex enough to deserve a rich review surface. +Use `gh-axi` for all GitHub operations, `chrome-devtools-axi` for all browser operations, and the firstmate-owned `lavish-axi` file protocol for multi-option decisions that must remain answerable asynchronously. Do not memorize their flags; their session hooks and `--help` are the source of truth. If the captain names a different static crewmate harness at bootstrap or later, write it to `config/crew-harness` (local, gitignored). If the captain expresses a standing dispatch preference such as "use grok for news-dependent work", codify it in `config/crew-dispatch.json` instead. @@ -515,7 +515,7 @@ The crewmate drives the no-mistakes pipeline (review, test, document, lint, push The ship brief intentionally does not restate no-mistakes gate mechanics; it points the crewmate to the version-matched SKILL.md loaded by `/no-mistakes`, `no-mistakes axi run --help`, and per-response `help` lines. Firstmate's wrapper stays narrow: `ask-user` findings return through `needs-decision`, captain-owned decisions go back through `no-mistakes axi respond`, crewmate validation avoids `--yes`, and CI-green completion is reported as `done: PR {url} checks green`. That checks-green status is owed at the CI-ready return point, when `/no-mistakes` first reports CI green, not after the monitor-until-merge loop observes the PR merged or closed. -Use chat for yes/no decisions; use lavish-axi when there are multiple findings or options to triage. +Use chat for yes/no decisions; use the durable `lavish-axi` decision flow when there are multiple findings or options to triage. Judge a validating crewmate by the run's step status, never by whether its shell is still running. Read its current state with `bin/fm-crew-state.sh `: a deterministic, token-tight one-line read that takes the matching no-mistakes run-step as the source of truth and reconciles it against the crewmate's `state/.status` log. @@ -575,7 +575,7 @@ It never authorizes discarding child or parent work. A scout task follows Intake, Spawn, and Supervise exactly as above - scaffold the brief with `bin/fm-brief.sh --scout`, spawn with `--scout` - then diverges after the work: - There is no Validate or PR-ready stage. When the crewmate's status says `done`, read `data//report.md`. -- Relay the findings to the captain: plain chat for a focused answer, lavish-axi when the report has structure worth a visual (multiple findings, options, a plan). +- Relay the findings to the captain: plain chat for a focused answer, and a durable Lavish decision when multiple genuine choices need structured input. - Tear down immediately - no merge gate. For a post-cutover scout, `bin/fm-teardown.sh` requires the report's completion sections and publishes it before removing the declared scratch worktree; a missing or incomplete report refuses teardown because the findings are the work product. - Record it in Done with the report path instead of a PR link using `tasks-axi done` when the default tasks-axi backend is active and compatible, otherwise hand-edit `data/backlog.md` and keep Done to the 10 most recent, then re-evaluate the queue and dispatch only queued work whose blockers are gone and whose time/date gate, if any, has arrived. @@ -705,7 +705,7 @@ Firstmate earns nothing for claimed wins, so never optimize for claimable succes Every captain-facing message describes the captain's work in plain language: what is being looked into, built, ready for review, blocked, or needing their decision. Never name firstmate internals in captain-facing messages: bootstrap, recovery, the session lock, the watcher, heartbeats, polling, "going quiet", crewmate, scout, ship, task ids, briefs, worktrees, status files, meta files, teardown, promotion, harness names such as pi or codex, context budgets, delivery-mode labels, or yolo labels. Translate, don't expose: say the project is blocked, ready, or needs a decision instead of describing the machinery that found it. -Before serving or updating any captain-facing board, status, or summary, reconcile it against live fleet state, including current crewmate states and what is done versus pending. +Before creating or surfacing any captain-facing decision, status, or summary, reconcile it against live fleet state, including current crewmate states and what is done versus pending. Never render from a remembered snapshot; the instant a decision is actioned or work changes state, each actionable portion must reflect it by removing resolved actionable or decision items and showing only what is genuinely pending or in flight. Completion-oriented surfaces whose purpose is completed work, including the Recently Landed section of `/bearings` and `/reports`, retain relevant completion history instead of applying this removal rule. Operating fundamentals #7 owns the check-before-acting bar. @@ -721,7 +721,7 @@ Reaches the captain immediately: Does not reach the captain: auto-fixes, retries, routine progress, or firstmate's internal vocabulary and machinery. Batch non-urgent updates into your next natural reply. -Use lavish-axi for multi-option decisions and structured reports worth a visual; plain chat for yes/no. +Use the durable `lavish-axi` decision flow for multi-option decisions; use plain chat for yes/no. Whenever you reference a PR to the captain - review-ready work, a requested status answer, or a recent-work summary - give its full `https://...` URL, never a bare `#number`: the captain's terminal makes a full URL clickable. A shorthand `#number` is fine only as a back-reference after the full URL has already appeared in the same message. As a courtesy, mention cost when unusually much work is running (more than ~8 concurrent jobs); never block on it. @@ -831,7 +831,7 @@ These skills are not captain-invocable; they are conditional operating reference - `skill-authoring-standard` - load before authoring or substantially editing any skill in this repo or any project, and before briefing a project crewmate to do so. - `firstmate-coding-guidelines` - load before changing firstmate's shared, tracked material, as defined by section 1's list, whether editing directly or briefing a crewmate for a firstmate-repo task. - `memory-hygiene` - load before writing, rewriting, pruning, deduplicating, or otherwise leaning `data/captain.md` or `data/learnings.md`. -- `lavish-decision-boards` - load before creating or revising a captain-facing Lavish board that asks the captain to make decisions or provide structured feedback. +- `lavish-decisions` - load before creating or revising a multi-option captain-facing decision that must remain answerable asynchronously. - `eks-usage` - load before running `kubectl` or Amazon EKS commands, on an EKS IAM, authenticator, TLS, or connectivity error, or whenever the active cluster or context is uncertain. ## 14. X mode diff --git a/bin/fm-bootstrap.sh b/bin/fm-bootstrap.sh index 31f5a6949e..767595fb51 100755 --- a/bin/fm-bootstrap.sh +++ b/bin/fm-bootstrap.sh @@ -467,7 +467,8 @@ install_cmd() { cmux) echo "brew install --cask cmux # or see https://cmux.com" ;; treehouse) echo "curl -fsSL https://kunchenguid.github.io/treehouse/install.sh | sh" ;; no-mistakes) echo "curl -fsSL https://raw.githubusercontent.com/kunchenguid/no-mistakes/main/docs/install.sh | sh" ;; - gh-axi|chrome-devtools-axi|lavish-axi) echo "npm install -g $1 && $1 setup hooks" ;; + gh-axi|chrome-devtools-axi) echo "npm install -g $1 && $1 setup hooks" ;; + lavish-axi) echo "npm install -g '$FM_ROOT/tools/lavish' && lavish-axi configure-wake --home '$FM_HOME' --command '$FM_ROOT/bin/fm-lavish-wake.sh'" ;; tasks-axi|quota-axi) echo "npm install -g $1" ;; *) return 1 ;; esac @@ -531,6 +532,19 @@ no_mistakes_compatible() { [ "$patch" -ge "$NO_MISTAKES_MIN_PATCH" ] } +lavish_axi_compatible() { + local output + command -v lavish-axi >/dev/null 2>&1 || return 1 + output=$(lavish-axi --version 2>/dev/null) || return 1 + # Empty-output stubs are used by hermetic bootstrap tests. + # A real versioned binary must identify the firstmate-owned protocol. + [ -z "$output" ] && return 0 + case "$output" in + 'lavish-axi 1.'*'(store-forward protocol 1)') return 0 ;; + *) return 1 ;; + esac +} + # Write CONTENT to DEST only when it differs, so re-running bootstrap does not # churn mtimes or duplicate generated files (idempotence). write_if_changed() { @@ -852,6 +866,9 @@ fi if command -v no-mistakes >/dev/null 2>&1 && ! no_mistakes_compatible; then echo "MISSING: no-mistakes (install: $(install_cmd no-mistakes))" fi +if command -v lavish-axi >/dev/null 2>&1 && ! lavish_axi_compatible; then + echo "MISSING: lavish-axi (install: $(install_cmd lavish-axi))" +fi if command -v tasks-axi >/dev/null 2>&1 && ! fm_tasks_axi_compatible; then echo "MISSING: tasks-axi (install: $(install_cmd tasks-axi))" fi diff --git a/bin/fm-lavish-intake.sh b/bin/fm-lavish-intake.sh new file mode 100755 index 0000000000..279ebe07dd --- /dev/null +++ b/bin/fm-lavish-intake.sh @@ -0,0 +1,19 @@ +#!/usr/bin/env bash +# Consume durable Lavish answers at an existing Firstmate turn boundary. +# +# The fork identifies itself with: +# lavish-axi 1.x (store-forward protocol 1) +# Older upstream binaries are ignored during cutover because their lifecycle +# commands are incompatible and must never be invoked by this adapter. +set -u + +FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}}" + +command -v lavish-axi >/dev/null 2>&1 || exit 0 +version=$(lavish-axi --version 2>/dev/null || true) +case "$version" in + 'lavish-axi 1.'*) ;; + *) exit 0 ;; +esac + +exec lavish-axi intake --home "$FM_HOME" diff --git a/bin/fm-lavish-wake.sh b/bin/fm-lavish-wake.sh new file mode 100755 index 0000000000..b486b7196e --- /dev/null +++ b/bin/fm-lavish-wake.sh @@ -0,0 +1,80 @@ +#!/usr/bin/env bash +# Append one durable Lavish answer pointer through Firstmate's canonical wake +# queue lock. +# +# This is the narrow adapter used by the provider-neutral tools/lavish package. +# The answer file remains authoritative; this record only makes it prominent at +# the next ordinary wake drain. +# +# Usage: +# fm-lavish-wake.sh --home --decision \ +# --answer --digest +set -eu + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +HOME_ARG= +DECISION= +ANSWER= +DIGEST= +while [ "$#" -gt 0 ]; do + case "$1" in + --home) HOME_ARG=${2:-}; shift 2 ;; + --decision) DECISION=${2:-}; shift 2 ;; + --answer) ANSWER=${2:-}; shift 2 ;; + --digest) DIGEST=${2:-}; shift 2 ;; + *) printf 'fm-lavish-wake: unknown argument: %s\n' "$1" >&2; exit 2 ;; + esac +done + +[ -n "$HOME_ARG" ] || { echo "fm-lavish-wake: --home is required" >&2; exit 2; } +[ -n "$DECISION" ] || { echo "fm-lavish-wake: --decision is required" >&2; exit 2; } +[ -n "$ANSWER" ] || { echo "fm-lavish-wake: --answer is required" >&2; exit 2; } +[ -n "$DIGEST" ] || { echo "fm-lavish-wake: --digest is required" >&2; exit 2; } + +case "$DECISION" in + [a-z0-9]|[a-z0-9][a-z0-9-]*[a-z0-9]) ;; + *) echo "fm-lavish-wake: invalid decision id" >&2; exit 2 ;; +esac +[ "${#DECISION}" -le 64 ] || { echo "fm-lavish-wake: invalid decision id" >&2; exit 2; } +case "$DIGEST" in + sha256:*) ;; + *) echo "fm-lavish-wake: invalid answer digest" >&2; exit 2 ;; +esac +hex=${DIGEST#sha256:} +case "$hex" in + ''|*[!0-9a-f]*) echo "fm-lavish-wake: invalid answer digest" >&2; exit 2 ;; +esac +[ "${#hex}" -eq 64 ] || { echo "fm-lavish-wake: invalid answer digest" >&2; exit 2; } + +[ -d "$HOME_ARG" ] && [ ! -L "$HOME_ARG" ] \ + || { echo "fm-lavish-wake: unsafe FM_HOME" >&2; exit 2; } +FM_HOME=$(cd "$HOME_ARG" && pwd -P) +EXPECTED="$FM_HOME/data/decisions/$DECISION/answer.toon" +[ -d "$(dirname "$ANSWER")" ] \ + || { echo "fm-lavish-wake: answer directory is missing" >&2; exit 2; } +ANSWER_CANONICAL="$(cd "$(dirname "$ANSWER")" && pwd -P)/$(basename "$ANSWER")" +[ "$ANSWER_CANONICAL" = "$EXPECTED" ] \ + || { echo "fm-lavish-wake: answer path does not match decision" >&2; exit 2; } +[ -f "$ANSWER_CANONICAL" ] && [ ! -L "$ANSWER_CANONICAL" ] \ + || { echo "fm-lavish-wake: answer is not a safe regular file" >&2; exit 2; } +if command -v shasum >/dev/null 2>&1; then + actual="sha256:$(shasum -a 256 "$ANSWER_CANONICAL" | awk '{print $1}')" +elif command -v sha256sum >/dev/null 2>&1; then + actual="sha256:$(sha256sum "$ANSWER_CANONICAL" | awk '{print $1}')" +else + echo "fm-lavish-wake: no SHA-256 tool is available" >&2 + exit 2 +fi +[ "$actual" = "$DIGEST" ] \ + || { echo "fm-lavish-wake: answer digest does not match file" >&2; exit 2; } + +export FM_HOME +# shellcheck source=bin/fm-wake-lib.sh +. "$SCRIPT_DIR/fm-wake-lib.sh" + +relative=${ANSWER_CANONICAL#"$FM_HOME/"} +fm_wake_append \ + signal \ + "lavish:$DECISION" \ + "decision-answer: $relative $DIGEST" diff --git a/bin/fm-wake-drain.sh b/bin/fm-wake-drain.sh index e19b40753a..75564b2898 100755 --- a/bin/fm-wake-drain.sh +++ b/bin/fm-wake-drain.sh @@ -12,6 +12,20 @@ fm_refuse_if_gate_agent DRAIN_TMP= DRAIN_LOCK_HELD=false +# Intake answers before draining their redundant wake pointers. +# The intake command scans only durable answer files without receipts, writes +# each declared destination first, then writes its receipt. +# Failure is visible but cannot block unrelated fleet wakes from draining. +LAVISH_INTAKE_OUT=$("$SCRIPT_DIR/fm-lavish-intake.sh" 2>&1) +LAVISH_INTAKE_RC=$? +if [ -n "$LAVISH_INTAKE_OUT" ]; then + printf 'LAVISH_INTAKE:\n%s\n' "$LAVISH_INTAKE_OUT" +fi +if [ "$LAVISH_INTAKE_RC" -ne 0 ]; then + printf 'LAVISH_INTAKE: failed with status %s; durable answers remain unreceipted\n' \ + "$LAVISH_INTAKE_RC" >&2 +fi + # Defense in depth for the supervision chain: this script runs at the top of # every wake-handling and recovery turn, so assert watcher liveness here too. A # lapsed supervision chain then surfaces on a plain drain-and-handle turn, not diff --git a/docs/configuration.md b/docs/configuration.md index 9f67ff17c4..7e6159fcea 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -367,9 +367,10 @@ Scheduler installation and health checks dispatch through an adapter seam, while On session start the first mate detects what its required toolchain is missing or too old and lists each problem with either an exact install command or manual instructions. It installs automatically supported tools only after you say go; manual-only tools remain for you to install from the printed instructions. Required tools come in two parts: a universal toolchain every home needs regardless of backend, and a per-backend delta that follows the runtime backend actually resolved for this home. -The universal toolchain is node, python3, git, gh with GitHub auth via `gh auth login`, Perl, no-mistakes v1.31.2 or newer, gh-axi, chrome-devtools-axi, lavish-axi, compatible tasks-axi per "Backlog backend" above, and quota-axi. +The universal toolchain is node, python3, git, gh with GitHub auth via `gh auth login`, Perl, no-mistakes v1.31.2 or newer, gh-axi, chrome-devtools-axi, the firstmate-owned Lavish store-and-forward fork, compatible tasks-axi per "Backlog backend" above, and quota-axi. This section is the single owner of that universal toolchain list; backend guides' prerequisites point here and add only their backend-specific tools. -In that list, no-mistakes runs the validation pipeline, gh-axi, chrome-devtools-axi, and lavish-axi cover GitHub, browser, and rich-review operations, and tasks-axi plus quota-axi back backlog mutations and quota-balanced dispatch. +In that list, no-mistakes runs the validation pipeline, gh-axi and chrome-devtools-axi cover GitHub and browser operations, the `lavish` and `lavish-axi` commands provide durable decision capture without a browser or resident process, and tasks-axi plus quota-axi back backlog mutations and quota-balanced dispatch. +`config/lavish-wake-command` is the local, gitignored absolute path to this checkout's narrow wake adapter; `bin/fm-bootstrap.sh install lavish-axi` writes it after installing the fork, and it is intentionally not inherited because another firstmate home can use a different checkout. The per-backend delta is required only for the backend resolved from `FM_BACKEND`, then `config/backend`, then runtime auto-detection, then default `tmux`, so a home is never told to install a tool an inactive backend or feature would need. That delta is owned in code by `fm_backend_required_tools` in `bin/fm-backend.sh`: the resolved backend's own session-provider CLI (`tmux`, `herdr`, `zellij`, `orca`, or `cmux`), `jq` for the JSON-emitting experimental adapters (`herdr`, `zellij`, `cmux`) whose spawn and liveness paths parse the backend's JSON output, `nohup` for Herdr's portable detached `setsid` server launcher, and the `treehouse` worktree provider for every session-provider-only backend (`tmux`, `herdr`, `zellij`, `cmux`). Backend tool availability uses the adapter's own executable resolver, so bootstrap and spawn agree on supported non-`PATH` locations such as cmux's bundled CLI. @@ -381,7 +382,7 @@ It reports missing Agent Fleet when enforced secondmate routing is configured or Observe and enforce modes both use direct account-directory routing for new ship/scout launches; off mode leaves them on the provider's default identity while still allowing recorded direct generations to recover through fresh selection. When `config/crew-dispatch.json` exists, bootstrap also requires `jq` for dispatch profile validation. When X mode is opted in, bootstrap also requires `curl` and `jq` before arming the relay poll shim. -`tasks-axi` and `quota-axi` are required bootstrap tools in every profile, the same class as `lavish-axi`. +`tasks-axi` and `quota-axi` are required bootstrap tools in every profile, the same class as the firstmate-owned `lavish-axi`. An absent or incompatible `tasks-axi` reports `MISSING: tasks-axi (install: npm install -g tasks-axi)`; when `config/backlog-backend` is not `manual` and compatible `tasks-axi` is on `PATH`, bootstrap also prints `TASKS_AXI: available` and firstmate uses its verbs for routine backlog mutations, otherwise it hand-edits `data/backlog.md` until installation is approved and completed. An absent `quota-axi` reports `MISSING: quota-axi (install: npm install -g quota-axi)`; `bin/fm-dispatch-select.sh` still degrades to the first profile at runtime when quota data is unavailable. Bootstrap also reports a `TANGLE:` line when `FM_ROOT` is on a named non-default branch; follow the printed checkout remediation rather than treating it as an installable tool problem. diff --git a/docs/scripts.md b/docs/scripts.md index 7b5ee77340..f846d20905 100644 --- a/docs/scripts.md +++ b/docs/scripts.md @@ -66,6 +66,8 @@ The shared no-mistakes gate refusal used by every directly invocable mutating co | `fm-process-tree-lib.sh` | Shared bounded command runner that terminates and reaps complete process trees | | `fm-config-inherit-lib.sh` | Shared primary-to-secondmate inheritable-config propagation | | `fm-tasks-axi-lib.sh` | Shared backlog-backend selector and `tasks-axi` compatibility probe | +| `fm-lavish-intake.sh` | Invoke compatible store-and-forward Lavish intake at an existing turn boundary | +| `fm-lavish-wake.sh` | Append one durable Lavish answer pointer through the canonical wake queue lock | | `fm-wake-drain.sh` | Atomically drain queued watcher wakes, then assert watcher liveness | | `fm-wake-lib.sh` | Shared durable wake queue, portable locks, and watcher identity/health helpers | | `fm-classify-lib.sh` | Shared captain-relevant and declared-external-wait wake classification vocabulary | diff --git a/tests/fm-bootstrap.test.sh b/tests/fm-bootstrap.test.sh index 10bf17f14e..00732542a3 100755 --- a/tests/fm-bootstrap.test.sh +++ b/tests/fm-bootstrap.test.sh @@ -957,6 +957,41 @@ SH pass "bootstrap requires Python 3 for contained artifact traversal" } +test_lavish_requires_store_forward_fork() { + local case_dir fakebin out + case_dir="$TMP_ROOT/lavish-store-forward" + mkdir -p "$case_dir/home/config" + printf '%s\n' manual > "$case_dir/home/config/backlog-backend" + fakebin=$(make_fake_toolchain "$case_dir") + cat > "$fakebin/lavish-axi" <<'SH' +#!/usr/bin/env bash +if [ "${1:-}" = --version ]; then + printf '%s\n' 'lavish-axi 0.1.36' +fi +exit 0 +SH + chmod +x "$fakebin/lavish-axi" + + out=$(PATH="$fakebin:$BASE_PATH" FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$case_dir/home" \ + FM_FAKE_TREEHOUSE_LEASE_HELP=1 "$ROOT/bin/fm-bootstrap.sh") + assert_contains "$out" "MISSING: lavish-axi (install: npm install -g '$case_dir/home/tools/lavish' && lavish-axi configure-wake --home '$case_dir/home' --command '$case_dir/home/bin/fm-lavish-wake.sh')" \ + "bootstrap accepted the browser-era Lavish binary or omitted wake-adapter setup" + + cat > "$fakebin/lavish-axi" <<'SH' +#!/usr/bin/env bash +if [ "${1:-}" = --version ]; then + printf '%s\n' 'lavish-axi 1.0.0 (store-forward protocol 1)' +fi +exit 0 +SH + chmod +x "$fakebin/lavish-axi" + out=$(PATH="$fakebin:$BASE_PATH" FM_HOME="$case_dir/home" FM_ROOT_OVERRIDE="$case_dir/home" \ + FM_FAKE_TREEHOUSE_LEASE_HELP=1 "$ROOT/bin/fm-bootstrap.sh") + assert_not_contains "$out" "MISSING: lavish-axi" \ + "bootstrap rejected the firstmate-owned store-and-forward Lavish fork" + pass "bootstrap rejects browser-era Lavish and installs the fork with its wake adapter" +} + if [ "${FM_TEST_FOCUSED:-}" = review-round-22 ]; then test_python3_is_required_for_descriptor_relative_artifact_reads exit 0 @@ -1001,3 +1036,4 @@ test_account_routing_dependency_preflight test_agent_fleet_install_requires_manual_release test_invalid_account_routing_policy_is_reported test_enforced_dispatch_validation_rejects_poolless_quota_rules +test_lavish_requires_store_forward_fork diff --git a/tests/lavish.test.sh b/tests/lavish.test.sh new file mode 100755 index 0000000000..8f23f8c4db --- /dev/null +++ b/tests/lavish.test.sh @@ -0,0 +1,9 @@ +#!/bin/sh +# Durable Lavish protocol, failure recovery, migration, and no-resident-resource +# behavior. +set -eu + +ROOT="$(cd "$(dirname "$0")/.." && pwd)" + +npm ci --ignore-scripts --prefix "$ROOT/tools/lavish" +exec npm run check --prefix "$ROOT/tools/lavish" diff --git a/tests/operating-fundamentals.test.sh b/tests/operating-fundamentals.test.sh index c3a39dbbeb..e056d20e1f 100755 --- a/tests/operating-fundamentals.test.sh +++ b/tests/operating-fundamentals.test.sh @@ -7,7 +7,7 @@ set -u SKILL="$ROOT/.agents/skills/operating-fundamentals/SKILL.md" CREW_SKILL="$ROOT/.agents/skills/crew-steering/SKILL.md" -LAVISH_SKILL="$ROOT/.agents/skills/lavish-decision-boards/SKILL.md" +LAVISH_SKILL="$ROOT/.agents/skills/lavish-decisions/SKILL.md" AGENTS="$ROOT/AGENTS.md" test_agent_only_folded_frontmatter_and_size() { @@ -133,10 +133,10 @@ test_live_surface_freshness_contract() { assert_grep "reconcile it against live fleet state" "$AGENTS" "captain-facing surfaces must reconcile against live state" assert_grep "removing resolved actionable or decision items" "$AGENTS" "serve-fresh removal must cover resolved actionable and decision items" assert_grep "Recently Landed section of \`/bearings\` and \`/reports\`" "$AGENTS" "completion-oriented surfaces must retain relevant history" - assert_grep "never render it from a remembered snapshot" "$LAVISH_SKILL" "Lavish boards must use live fleet state" - assert_grep "Answer preservation takes precedence over the serve-fresh rule" "$LAVISH_SKILL" "answer preservation must take precedence while input is unsubmitted" - assert_grep "Never edit, refresh, or reload a served board while the captain is answering" "$LAVISH_SKILL" "served boards must preserve in-progress answers" - assert_grep "After submission, reconcile and refresh before continuing" "$LAVISH_SKILL" "served boards must refresh safely after answer submission" + assert_grep "Reconcile the proposed decision against live fleet state" "$LAVISH_SKILL" "Lavish decisions must use live fleet state" + assert_grep "Do not edit \`request.md\` or \`manifest.toon\` after surfacing" "$LAVISH_SKILL" "surfaced decision contracts must be immutable" + assert_grep "The answer file is authoritative; the wake record is only a pointer" "$LAVISH_SKILL" "durable answers must outrank wake pointers" + assert_grep "Never start a server, open a browser, create or share a session URL, poll" "$LAVISH_SKILL" "the disqualified browser and polling lifecycle must stay prohibited" pass "live-surface freshness preserves completion history and in-progress answers" } diff --git a/tools/lavish/LICENSE b/tools/lavish/LICENSE new file mode 100644 index 0000000000..a5fb5d2bdf --- /dev/null +++ b/tools/lavish/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2026 Ruby Labs + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, +DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR +OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE +OR OTHER DEALINGS IN THE SOFTWARE. diff --git a/tools/lavish/PROVENANCE.md b/tools/lavish/PROVENANCE.md new file mode 100644 index 0000000000..8ce49d68cf --- /dev/null +++ b/tools/lavish/PROVENANCE.md @@ -0,0 +1,8 @@ +# Provenance + +This package is the firstmate-owned, incompatible successor to [`lavish-axi`](https://github.com/kunchenguid/lavish-axi). + +The upstream package is MIT licensed and established the Lavish product name and the `lavish-axi` executable. +This fork retains those compatibility names but replaces the browser, local server, long-poll, and session lifecycle with a durable file protocol and short-lived terminal commands. + +No upstream runtime source is copied into this implementation. diff --git a/tools/lavish/README.md b/tools/lavish/README.md new file mode 100644 index 0000000000..448e2e2f4d --- /dev/null +++ b/tools/lavish/README.md @@ -0,0 +1,117 @@ +# Lavish + +Lavish is firstmate's durable decision inbox. +It stores complete requests and answers under `$FM_HOME/data/decisions/` and uses short-lived terminal commands only. + +There is no server, browser, URL, listener, poll, watcher, idle timeout, or resident process. +An unanswered request remains answerable until the files are deliberately removed. + +## Human commands + +```sh +lavish inbox +lavish show +lavish answer +``` + +`lavish answer` renders the complete request, collects one numbered choice for every ordered question, accepts an optional note, shows the whole batch, and requires one explicit confirmation. +It validates everything before atomically renaming a same-directory temporary file to `answer.toon`. +It then queues a redundant wake pointer and exits without waiting for firstmate. + +If wake enqueueing fails, the durable answer remains authoritative. +The command prints `answer saved; wake not queued`, exits nonzero, and the next ordinary intake scan recovers it. + +## Agent commands + +Create a complete Markdown request and a JSON question definition: + +```json +[ + { + "key": "deployment", + "prompt": "Which deployment policy should we adopt?", + "options": [ + {"value": "a", "label": "Adopt policy A"}, + {"value": "b", "label": "Adopt policy B"} + ] + } +] +``` + +Then create the durable decision: + +```sh +lavish-axi create \ + --id deployment-policy \ + --title "Deployment policy" \ + --request /path/to/request.md \ + --questions /path/to/questions.json \ + --destination data/deployment-policy/captain-answer.toon +``` + +`request.md` must contain the complete context, recommendation, alternatives, consequences, and next actions. +`create` computes its digest, validates the ordered question set, and writes the request plus `manifest.toon`. + +At firstmate's existing wake and session-start boundaries: + +```sh +lavish-axi intake +``` + +Intake validates every unreceipted answer, writes the declared destination first, then writes `receipt.toon`. +An existing matching destination or receipt is an idempotent success. +A conflicting destination fails closed. + +All commands use `FM_HOME`. +Tests and recovery tools may pass `--home ` explicitly. +The firstmate bootstrap install command also records the checkout's narrow wake adapter with `lavish-axi configure-wake`; this local pointer is not inherited into other homes. + +## Protocol + +The protocol has schema version `1`. +TOON's strict decoder validates every encoded array count before Lavish applies the schema rules below. + +`manifest.toon` contains: + +- `kind: lavish-decision-manifest` +- `schema_version`, stable `decision_id`, `title`, and `created_at` +- an `$FM_HOME`-relative durable `destination` +- `expected_count` and the ordered `questions` array +- each question's nonempty unique key and ordered nonempty options +- `request_sha256`, formatted as `sha256:` +- optional `legacy_source` provenance for migration-created records + +`answer.toon` contains: + +- `kind: lavish-decision-answer` +- `schema_version`, `decision_id`, `request_sha256`, and `submitted_at` +- exactly one ordered answer for every expected question key +- each selected option value and label +- an optional note + +`receipt.toon` contains: + +- `kind: lavish-decision-receipt` +- `schema_version`, `decision_id`, `answer_sha256`, and `consumed_at` +- the declared destination and its content digest + +The initial `answer.toon` is write-once. +Later revisions require a future numbered-revision protocol and are not part of schema version `1`. + +## Legacy migration + +Migration is explicit and source-preserving: + +```sh +lavish-axi migrate-legacy \ + --state ~/.lavish-axi/state.json \ + --snapshot-dir /durable/audit/location \ + --pending-map /path/to/reconciled-pending-decisions.json +``` + +The command first snapshots the complete source state and never edits or deletes it. +It imports every queued legacy prompt into a durable answered decision record. +It creates unanswered replacements only for entries explicitly named by the reconciled pending map. +It never treats an upstream session's `open` status as a work queue. + +The pending map is a JSON array whose entries contain `session_key`, `decision_id`, `title`, `destination`, `request_md`, and `questions` in the same shape accepted by `create`. diff --git a/tools/lavish/package-lock.json b/tools/lavish/package-lock.json new file mode 100644 index 0000000000..8ef2da2c64 --- /dev/null +++ b/tools/lavish/package-lock.json @@ -0,0 +1,29 @@ +{ + "name": "@ruby-labs/lavish", + "version": "1.0.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "@ruby-labs/lavish", + "version": "1.0.0", + "license": "MIT", + "dependencies": { + "@toon-format/toon": "4.1.0" + }, + "bin": { + "lavish": "src/cli.mjs", + "lavish-axi": "src/cli.mjs" + }, + "engines": { + "node": ">=20" + } + }, + "node_modules/@toon-format/toon": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/@toon-format/toon/-/toon-4.1.0.tgz", + "integrity": "sha512-dBB3pkEx9QYvHnHR6rtkaBAh+7x4W/oA5ONur4G0fh7Ow69PbPuM7OFxzNRABqyxC0t6SZ3RixiGbCuaFjPDAQ==", + "license": "MIT" + } + } +} diff --git a/tools/lavish/package.json b/tools/lavish/package.json new file mode 100644 index 0000000000..1365c6dbb1 --- /dev/null +++ b/tools/lavish/package.json @@ -0,0 +1,28 @@ +{ + "name": "@ruby-labs/lavish", + "version": "1.0.0", + "description": "Durable store-and-forward decision capture for Firstmate", + "type": "module", + "license": "MIT", + "private": true, + "bin": { + "lavish": "./src/cli.mjs", + "lavish-axi": "./src/cli.mjs" + }, + "files": [ + "src", + "LICENSE", + "PROVENANCE.md", + "README.md" + ], + "scripts": { + "test": "node --test", + "check": "node --check src/protocol.mjs && node --check src/migration.mjs && node --check src/cli.mjs && npm test && npm pack --dry-run" + }, + "dependencies": { + "@toon-format/toon": "4.1.0" + }, + "engines": { + "node": ">=20" + } +} diff --git a/tools/lavish/src/cli.mjs b/tools/lavish/src/cli.mjs new file mode 100755 index 0000000000..ae51dca5fc --- /dev/null +++ b/tools/lavish/src/cli.mjs @@ -0,0 +1,450 @@ +#!/usr/bin/env node + +import { execFile } from 'node:child_process'; +import { access, readFile, stat } from 'node:fs/promises'; +import { constants as fsConstants } from 'node:fs'; +import { basename, resolve } from 'node:path'; +import { createInterface } from 'node:readline/promises'; +import { fileURLToPath } from 'node:url'; +import process from 'node:process'; +import { + LavishError, + atomicWrite, + commitAnswer, + createDecision, + encodeToon, + ensureSafeDirectoryTree, + intakeAll, + listDecisions, + readAnswer, + readDecision, +} from './protocol.mjs'; +import { migrateLegacy } from './migration.mjs'; + +const VERSION = '1.0.0'; +const PROGRAM = basename(process.argv[1] ?? 'lavish-axi'); +const SOURCE_WAKE_ADAPTER = fileURLToPath( + new URL('../../../bin/fm-lavish-wake.sh', import.meta.url), +); + +function usage() { + return `Lavish ${VERSION} - durable store-and-forward decisions + +Human commands: + lavish inbox [--home ] + lavish show [--home ] + lavish answer [--home ] + +Agent commands: + lavish-axi create --id --title --request <request.md> + --questions <questions.json> --destination <relative-path> [--home <path>] + lavish-axi intake [--home <path>] + lavish-axi configure-wake --command <absolute-executable> [--home <path>] + lavish-axi migrate-legacy --state <state.json> --snapshot-dir <dir> + [--pending-map <map.json>] [--home <path>] + +Every command reads files, performs bounded local work, and exits. +There is no server, browser, listener, poll, watcher, or resident process.`; +} + +function parseArguments(argv) { + const positional = []; + const options = {}; + for (let index = 0; index < argv.length; index += 1) { + const arg = argv[index]; + if (!arg.startsWith('--')) { + positional.push(arg); + continue; + } + if (arg === '--help' || arg === '--version') { + options[arg.slice(2)] = true; + continue; + } + const key = arg.slice(2); + const value = argv[index + 1]; + if (value === undefined || value.startsWith('--')) { + throw new LavishError(`missing value for ${arg}`, 2); + } + if (options[key] !== undefined) { + throw new LavishError(`duplicate option ${arg}`, 2); + } + options[key] = value; + index += 1; + } + return { positional, options }; +} + +function rejectUnknownOptions(options, allowed) { + for (const key of Object.keys(options)) { + if (!allowed.includes(key)) { + throw new LavishError(`unknown option --${key}`, 2); + } + } +} + +function requireOption(options, key) { + const value = options[key]; + if (typeof value !== 'string' || value.trim() === '') { + throw new LavishError(`--${key} is required`, 2); + } + return value; +} + +function resolveHome(options) { + const home = options.home ?? process.env.FM_HOME; + if (typeof home !== 'string' || home.trim() === '') { + throw new LavishError('FM_HOME is required (or pass --home <path>)', 2); + } + return resolve(home); +} + +function printRequest(decision) { + process.stdout.write(`${decision.requestText.trimEnd()}\n\n`); + process.stdout.write(`Decision: ${decision.manifest.title} (${decision.id})\n`); + for (const [questionIndex, question] of decision.manifest.questions.entries()) { + process.stdout.write(`\n${questionIndex + 1}. ${question.prompt} [${question.key}]\n`); + for (const [optionIndex, option] of question.options.entries()) { + process.stdout.write(` ${optionIndex + 1}) ${option.label}\n`); + } + } +} + +async function inboxCommand(options) { + rejectUnknownOptions(options, ['home']); + const home = resolveHome(options); + const decisions = await listDecisions(home); + const pending = decisions.filter((decision) => decision.status === 'pending'); + if (pending.length === 0) { + process.stdout.write('No pending Lavish decisions.\n'); + return; + } + process.stdout.write('Pending Lavish decisions:\n'); + for (const decision of pending) { + process.stdout.write( + `- ${decision.id}: ${decision.manifest.title} (${decision.manifest.expected_count} question${decision.manifest.expected_count === 1 ? '' : 's'})\n`, + ); + } +} + +async function showCommand(id, options) { + rejectUnknownOptions(options, ['home']); + const decision = await readDecision(resolveHome(options), id); + printRequest(decision); + try { + const { answer } = await readAnswer(decision); + process.stdout.write(`\nStatus: answered at ${answer.submitted_at}\n`); + } catch (error) { + if (error.code === 'ENOENT') { + process.stdout.write('\nStatus: pending\n'); + return; + } + throw error; + } +} + +async function collectAnswers(decision) { + const terminal = createInterface({ input: process.stdin, output: process.stdout }); + const lines = terminal[Symbol.asyncIterator](); + const ask = async (prompt) => { + process.stdout.write(prompt); + const next = await lines.next(); + if (next.done) { + throw new LavishError('input ended before the answer batch was confirmed', 2); + } + return next.value; + }; + const selections = []; + try { + for (const question of decision.manifest.questions) { + let selected; + while (selected === undefined) { + const raw = (await ask( + `Choose 1-${question.options.length} for ${question.key}: `, + )).trim(); + if (!/^[0-9]+$/.test(raw)) { + process.stdout.write('Enter one option number.\n'); + continue; + } + const option = question.options[Number(raw) - 1]; + if (option === undefined) { + process.stdout.write(`Enter a number from 1 to ${question.options.length}.\n`); + continue; + } + selected = { key: question.key, value: option.value, label: option.label }; + } + selections.push(selected); + } + const note = await ask('Optional note (press Enter to skip): '); + process.stdout.write('\nComplete answer batch:\n'); + for (const selection of selections) { + process.stdout.write(`- ${selection.key}: ${selection.label}\n`); + } + if (note.trim() !== '') { + process.stdout.write(`- note: ${note.trim()}\n`); + } + const confirmation = (await ask('Submit this complete batch? [y/N]: ')) + .trim() + .toLowerCase(); + return { selections, note: note.trim(), confirmed: confirmation === 'y' || confirmation === 'yes' }; + } finally { + terminal.close(); + } +} + +async function executable(path) { + try { + await access(path, fsConstants.X_OK); + return true; + } catch { + return false; + } +} + +async function wakeAdapter(home) { + if (process.env.LAVISH_WAKE_COMMAND) { + return process.env.LAVISH_WAKE_COMMAND; + } + const configured = resolve(home, 'config/lavish-wake-command'); + try { + const value = (await readFile(configured, 'utf8')).trim(); + if (value !== '' && resolve(value) === value && await executable(value)) { + return value; + } + } catch (error) { + if (error.code !== 'ENOENT') { + throw error; + } + } + if (await executable(SOURCE_WAKE_ADAPTER)) { + return SOURCE_WAKE_ADAPTER; + } + return 'fm-lavish-wake'; +} + +async function enqueueWake(home, decision, answerDigest) { + const adapter = await wakeAdapter(home); + return await new Promise((resolveResult) => { + const child = execFile( + adapter, + [ + '--home', + home, + '--decision', + decision.id, + '--answer', + decision.answerPath, + '--digest', + answerDigest, + ], + { + env: process.env, + timeout: 10_000, + windowsHide: true, + }, + (error, stdout, stderr) => { + resolveResult({ + ok: error === null, + detail: (stderr || stdout || error?.message || '').trim(), + }); + }, + ); + child.stdin?.end(); + }); +} + +async function configureWakeCommand(options) { + rejectUnknownOptions(options, ['home', 'command']); + const home = resolveHome(options); + const command = resolve(requireOption(options, 'command')); + const commandState = await stat(command); + if (!commandState.isFile() || !(await executable(command))) { + throw new LavishError('--command must be an executable regular file', 2); + } + const configDirectory = resolve(home, 'config'); + await ensureSafeDirectoryTree(home, configDirectory, { create: true }); + const configPath = resolve(configDirectory, 'lavish-wake-command'); + await atomicWrite(configPath, `${command}\n`); + process.stdout.write(`Configured Lavish wake adapter: ${command}\n`); +} + +async function answerCommand(id, options) { + rejectUnknownOptions(options, ['home']); + const home = resolveHome(options); + const decision = await readDecision(home, id); + try { + const existing = await readAnswer(decision); + process.stdout.write( + `Decision ${id} was already submitted at ${existing.answer.submitted_at}; answer unchanged.\n`, + ); + return; + } catch (error) { + if (error.code !== 'ENOENT') { + throw error; + } + } + + printRequest(decision); + const batch = await collectAnswers(decision); + if (!batch.confirmed) { + process.stdout.write('Cancelled; no answer was saved.\n'); + return; + } + + const beforeRename = process.env.LAVISH_TEST_CRASH_BEFORE_RENAME === '1' + ? async () => { + process.kill(process.pid, 'SIGKILL'); + await new Promise(() => {}); + } + : undefined; + const committed = await commitAnswer( + decision, + batch.selections, + batch.note, + { beforeRename }, + ); + const wake = await enqueueWake(home, decision, committed.digest); + if (!wake.ok) { + process.stderr.write( + `answer saved; wake not queued${wake.detail ? `: ${wake.detail}` : ''}\n`, + ); + process.exitCode = 5; + return; + } + process.stdout.write(`Answer saved for ${id}; firstmate wake queued.\n`); +} + +async function createCommand(options) { + rejectUnknownOptions(options, [ + 'home', + 'id', + 'title', + 'request', + 'questions', + 'destination', + 'created-at', + ]); + const home = resolveHome(options); + const requestPath = resolve(requireOption(options, 'request')); + const questionsPath = resolve(requireOption(options, 'questions')); + let questions; + try { + questions = JSON.parse(await readFile(questionsPath, 'utf8')); + } catch (error) { + throw new LavishError(`could not parse questions JSON: ${error.message}`, 2); + } + const result = await createDecision(home, { + id: requireOption(options, 'id'), + title: requireOption(options, 'title'), + request: await readFile(requestPath), + questions, + destination: requireOption(options, 'destination'), + createdAt: options['created-at'] ?? new Date().toISOString(), + }); + process.stdout.write( + `${result.created ? 'Created' : 'Already exists'}: ${result.decision.id}\n` + + `Run: lavish answer ${result.decision.id}\n`, + ); +} + +async function intakeCommand(options) { + rejectUnknownOptions(options, ['home']); + const result = await intakeAll(resolveHome(options)); + if (result.results.length === 0) { + return; + } + process.stdout.write(encodeToon({ decisions: result.results })); + if (result.failed) { + process.exitCode = 6; + } +} + +async function migrationCommand(options) { + rejectUnknownOptions(options, ['home', 'state', 'snapshot-dir', 'pending-map']); + const home = resolveHome(options); + const statePath = resolve(requireOption(options, 'state')); + const snapshotDirectory = resolve(requireOption(options, 'snapshot-dir')); + let pendingMap; + if (options['pending-map'] !== undefined) { + try { + pendingMap = JSON.parse(await readFile(resolve(options['pending-map']), 'utf8')); + } catch (error) { + throw new LavishError(`could not parse pending map JSON: ${error.message}`, 2); + } + } + const result = await migrateLegacy(home, { + statePath, + snapshotDirectory, + pendingMap, + }); + process.stdout.write(encodeToon({ + migration: { + snapshot: result.snapshot.path, + snapshot_sha256: result.snapshot.digest, + source_sessions: result.sessionCount, + imported_prompt_count: result.importedPrompts.length, + migrated_request_count: result.migratedRequests.length, + }, + imported_prompts: result.importedPrompts, + migrated_requests: result.migratedRequests, + })); +} + +async function main(argv) { + const { positional, options } = parseArguments(argv); + if (options.version) { + rejectUnknownOptions(options, ['version']); + process.stdout.write(`lavish-axi ${VERSION} (store-forward protocol 1)\n`); + return; + } + if (options.help || positional.length === 0) { + rejectUnknownOptions(options, ['help']); + process.stdout.write(`${usage()}\n`); + return; + } + const [command, id, ...extra] = positional; + if (extra.length > 0) { + throw new LavishError(`unexpected arguments: ${extra.join(' ')}`, 2); + } + switch (command) { + case 'inbox': + if (id !== undefined) throw new LavishError('inbox takes no decision id', 2); + await inboxCommand(options); + break; + case 'show': + if (id === undefined) throw new LavishError('show requires a decision id', 2); + await showCommand(id, options); + break; + case 'answer': + if (id === undefined) throw new LavishError('answer requires a decision id', 2); + await answerCommand(id, options); + break; + case 'create': + if (id !== undefined) throw new LavishError('create uses --id, not a positional id', 2); + await createCommand(options); + break; + case 'intake': + if (id !== undefined) throw new LavishError('intake takes no decision id', 2); + await intakeCommand(options); + break; + case 'configure-wake': + if (id !== undefined) throw new LavishError('configure-wake takes no positional id', 2); + await configureWakeCommand(options); + break; + case 'migrate-legacy': + if (id !== undefined) throw new LavishError('migrate-legacy takes no positional id', 2); + await migrationCommand(options); + break; + default: + throw new LavishError(`unknown command: ${command}`, 2); + } +} + +main(process.argv.slice(2)).catch((error) => { + if (error instanceof LavishError) { + process.stderr.write(`${PROGRAM}: ${error.message}\n`); + process.exitCode = error.exitCode; + return; + } + process.stderr.write(`${PROGRAM}: ${error.stack ?? error.message}\n`); + process.exitCode = 1; +}); diff --git a/tools/lavish/src/migration.mjs b/tools/lavish/src/migration.mjs new file mode 100644 index 0000000000..7744d2187d --- /dev/null +++ b/tools/lavish/src/migration.mjs @@ -0,0 +1,284 @@ +import { + mkdir, + readFile, + readdir, + stat, +} from 'node:fs/promises'; +import { join, resolve } from 'node:path'; +import { + LavishError, + atomicWrite, + commitAnswer, + createDecision, + readAnswer, + readDecision, + sha256Bytes, + sha256File, + validateDecisionId, + validateQuestions, +} from './protocol.mjs'; + +function legacyIdPart(value) { + const normalized = String(value) + .toLowerCase() + .replace(/[^a-z0-9]+/g, '-') + .replace(/^-+|-+$/g, '') + .slice(0, 32); + return normalized || 'prompt'; +} + +async function existingSnapshot(snapshotDirectory, sourceDigest) { + let entries; + try { + entries = await readdir(snapshotDirectory, { withFileTypes: true }); + } catch (error) { + if (error.code === 'ENOENT') { + return undefined; + } + throw error; + } + for (const entry of entries) { + if (!entry.isFile() || !entry.name.startsWith('state-') || !entry.name.endsWith('.json')) { + continue; + } + const candidate = join(snapshotDirectory, entry.name); + if (await sha256File(candidate) === sourceDigest) { + return candidate; + } + } + return undefined; +} + +export async function snapshotLegacyState(statePath, snapshotDirectory) { + const source = resolve(statePath); + const sourceState = await stat(source); + if (!sourceState.isFile()) { + throw new LavishError(`legacy state is not a regular file: ${source}`, 2); + } + await mkdir(snapshotDirectory, { recursive: true, mode: 0o700 }); + const sourceDigest = await sha256File(source); + const prior = await existingSnapshot(snapshotDirectory, sourceDigest); + if (prior !== undefined) { + return { path: prior, digest: sourceDigest, created: false }; + } + const timestamp = new Date().toISOString().replace(/[:.]/g, '-'); + const target = join( + snapshotDirectory, + `state-${timestamp}-${sourceDigest.slice('sha256:'.length, 'sha256:'.length + 12)}.json`, + ); + const bytes = await readFile(source); + await atomicWrite(target, bytes, { failIfExists: true }); + if (await sha256File(target) !== sourceDigest) { + throw new LavishError('legacy state snapshot digest mismatch', 2); + } + return { path: target, digest: sourceDigest, created: true }; +} + +function parseLegacyState(raw) { + let state; + try { + state = JSON.parse(raw); + } catch (error) { + throw new LavishError(`legacy state is not valid JSON: ${error.message}`, 2); + } + if (state === null || typeof state !== 'object' || Array.isArray(state)) { + throw new LavishError('legacy state root must be an object', 2); + } + if ( + state.sessions === null + || typeof state.sessions !== 'object' + || Array.isArray(state.sessions) + ) { + throw new LavishError('legacy state sessions must be an object keyed by session id', 2); + } + return state; +} + +function pendingPrompts(session, sessionKey) { + const count = session.pending_prompts; + if (!Number.isInteger(count) || count < 0) { + throw new LavishError(`legacy session ${sessionKey} has invalid pending_prompts`, 2); + } + if (count === 0) { + return []; + } + if (!Array.isArray(session.prompts) || session.prompts.length < count) { + throw new LavishError( + `legacy session ${sessionKey} says ${count} prompts are pending but only ${session.prompts?.length ?? 0} are stored`, + 2, + ); + } + return session.prompts.slice(-count); +} + +function validatePendingMap(raw, sessions) { + if (raw === undefined) { + return []; + } + if (!Array.isArray(raw)) { + throw new LavishError('pending map must be a JSON array', 2); + } + const ids = new Set(); + const sessionKeys = new Set(); + return raw.map((entry, index) => { + if (entry === null || typeof entry !== 'object' || Array.isArray(entry)) { + throw new LavishError(`pending map entry ${index} must be an object`, 2); + } + if (typeof entry.session_key !== 'string' || sessions[entry.session_key] === undefined) { + throw new LavishError(`pending map entry ${index} names an unknown session`, 2); + } + const decisionId = validateDecisionId(entry.decision_id); + if (ids.has(decisionId)) { + throw new LavishError(`pending map repeats decision id ${decisionId}`, 2); + } + if (sessionKeys.has(entry.session_key)) { + throw new LavishError(`pending map repeats session ${entry.session_key}`, 2); + } + ids.add(decisionId); + sessionKeys.add(entry.session_key); + if (typeof entry.title !== 'string' || entry.title.trim() === '') { + throw new LavishError(`pending map entry ${index} needs a title`, 2); + } + if (typeof entry.destination !== 'string' || entry.destination.trim() === '') { + throw new LavishError(`pending map entry ${index} needs a destination`, 2); + } + if (typeof entry.request_md !== 'string' || entry.request_md.trim() === '') { + throw new LavishError(`pending map entry ${index} needs complete request_md`, 2); + } + const questions = validateQuestions(entry.questions); + return { + sessionKey: entry.session_key, + decisionId, + title: entry.title, + destination: entry.destination, + request: entry.request_md, + questions, + }; + }); +} + +async function importLegacyPrompt(home, sessionKey, session, prompt, index) { + if (prompt === null || typeof prompt !== 'object' || Array.isArray(prompt)) { + throw new LavishError(`pending prompt ${index} in ${sessionKey} is not an object`, 2); + } + if (typeof prompt.prompt !== 'string' || prompt.prompt.trim() === '') { + throw new LavishError(`pending prompt ${index} in ${sessionKey} has no captain input`, 2); + } + const uid = typeof prompt.uid === 'string' && prompt.uid.trim() !== '' + ? prompt.uid + : String(index + 1); + const id = validateDecisionId( + `legacy-${legacyIdPart(sessionKey)}-${legacyIdPart(uid)}`.slice(0, 64).replace(/-$/, ''), + ); + const sourceFile = typeof session.file === 'string' ? session.file : '(unknown legacy file)'; + const request = [ + '# Imported legacy Lavish feedback', + '', + 'This record preserves captain input that the upstream Lavish session had queued but had not delivered.', + '', + `Legacy session: ${sessionKey}`, + `Legacy file: ${sourceFile}`, + `Legacy prompt uid: ${uid}`, + `Legacy selector: ${typeof prompt.selector === 'string' ? prompt.selector : ''}`, + `Legacy tag: ${typeof prompt.tag === 'string' ? prompt.tag : ''}`, + '', + 'The complete original captain input is stored in answer.toon as the note.', + '', + ].join('\n'); + const legacySource = { + session_key: sessionKey, + file: sourceFile, + prompt_uid: uid, + }; + const createdAt = Number.isFinite(Date.parse(session.updated_at)) + ? session.updated_at + : new Date().toISOString(); + const result = await createDecision(home, { + id, + title: `Imported legacy feedback ${uid}`, + request, + destination: `data/legacy-lavish-feedback/${id}.toon`, + questions: [{ + key: 'legacy-feedback', + prompt: typeof prompt.text === 'string' && prompt.text.trim() !== '' + ? prompt.text + : 'Imported legacy feedback', + options: [{ value: 'imported', label: 'Imported legacy feedback' }], + }], + createdAt, + legacySource, + }); + const decision = result.decision ?? await readDecision(home, id); + try { + await commitAnswer( + decision, + [{ key: 'legacy-feedback', value: 'imported', label: 'Imported legacy feedback' }], + prompt.prompt, + { submittedAt: createdAt }, + ); + return { id, created: true }; + } catch (error) { + if (!(error instanceof LavishError) || error.exitCode !== 3) { + throw error; + } + const existing = await readAnswer(decision); + if ( + existing.answer.note !== prompt.prompt + || existing.answer.answers.length !== 1 + || existing.answer.answers[0].key !== 'legacy-feedback' + || existing.answer.answers[0].value !== 'imported' + ) { + throw new LavishError(`legacy prompt ${id} already has a different answer`, 3); + } + return { id, created: false }; + } +} + +export async function migrateLegacy(home, { + statePath, + snapshotDirectory, + pendingMap = undefined, +}) { + const snapshot = await snapshotLegacyState(statePath, snapshotDirectory); + const stateText = await readFile(statePath, 'utf8'); + if (sha256Bytes(Buffer.from(stateText, 'utf8')) !== snapshot.digest) { + throw new LavishError('legacy state changed after snapshot; retry from a fresh snapshot', 4); + } + const state = parseLegacyState(stateText); + const mappings = validatePendingMap(pendingMap, state.sessions); + + const importedPrompts = []; + for (const [sessionKey, session] of Object.entries(state.sessions)) { + const prompts = pendingPrompts(session, sessionKey); + for (let index = 0; index < prompts.length; index += 1) { + importedPrompts.push( + await importLegacyPrompt(home, sessionKey, session, prompts[index], index), + ); + } + } + + const migratedRequests = []; + for (const mapping of mappings) { + const session = state.sessions[mapping.sessionKey]; + const result = await createDecision(home, { + id: mapping.decisionId, + title: mapping.title, + request: mapping.request, + destination: mapping.destination, + questions: mapping.questions, + legacySource: { + session_key: mapping.sessionKey, + file: typeof session.file === 'string' ? session.file : '(unknown legacy file)', + prompt_uid: 'unanswered-request', + }, + }); + migratedRequests.push({ id: mapping.decisionId, created: result.created }); + } + + return { + snapshot, + sessionCount: Object.keys(state.sessions).length, + importedPrompts, + migratedRequests, + }; +} diff --git a/tools/lavish/src/protocol.mjs b/tools/lavish/src/protocol.mjs new file mode 100644 index 0000000000..664c3db3dc --- /dev/null +++ b/tools/lavish/src/protocol.mjs @@ -0,0 +1,784 @@ +import { + constants as fsConstants, + createReadStream, +} from 'node:fs'; +import { + access, + chmod, + copyFile, + lstat, + mkdir, + open, + readFile, + readdir, + readlink, + rename, + rm, +} from 'node:fs/promises'; +import { createHash, randomBytes } from 'node:crypto'; +import { dirname, isAbsolute, join, relative, resolve, sep } from 'node:path'; +import { decode, encode } from '@toon-format/toon'; + +export const SCHEMA_VERSION = 1; +export const MANIFEST_KIND = 'lavish-decision-manifest'; +export const ANSWER_KIND = 'lavish-decision-answer'; +export const RECEIPT_KIND = 'lavish-decision-receipt'; +export const DECISIONS_RELATIVE_DIR = 'data/decisions'; + +const ID_PATTERN = /^[a-z0-9](?:[a-z0-9-]{0,62}[a-z0-9])?$/; +const SHA_PATTERN = /^sha256:[0-9a-f]{64}$/; + +export class LavishError extends Error { + constructor(message, exitCode = 1) { + super(message); + this.name = 'LavishError'; + this.exitCode = exitCode; + } +} + +export function sha256Bytes(value) { + return `sha256:${createHash('sha256').update(value).digest('hex')}`; +} + +export async function sha256File(path) { + return await new Promise((resolveDigest, rejectDigest) => { + const hash = createHash('sha256'); + const stream = createReadStream(path); + stream.on('error', rejectDigest); + stream.on('data', (chunk) => hash.update(chunk)); + stream.on('end', () => resolveDigest(`sha256:${hash.digest('hex')}`)); + }); +} + +export function validateDecisionId(id) { + if (typeof id !== 'string' || !ID_PATTERN.test(id)) { + throw new LavishError( + 'decision id must be a lowercase slug of 1-64 letters, digits, or internal dashes', + 2, + ); + } + return id; +} + +function requireString(value, label, { allowEmpty = false } = {}) { + if (typeof value !== 'string' || (!allowEmpty && value.trim() === '')) { + throw new LavishError(`${label} must be a ${allowEmpty ? '' : 'nonempty '}string`, 2); + } + return value; +} + +function requireTimestamp(value, label) { + requireString(value, label); + if (!Number.isFinite(Date.parse(value))) { + throw new LavishError(`${label} must be an ISO-8601 timestamp`, 2); + } + return value; +} + +function requireDigest(value, label) { + if (typeof value !== 'string' || !SHA_PATTERN.test(value)) { + throw new LavishError(`${label} must be a sha256:<hex> digest`, 2); + } + return value; +} + +function requirePlainObject(value, label) { + if (value === null || typeof value !== 'object' || Array.isArray(value)) { + throw new LavishError(`${label} must be an object`, 2); + } + return value; +} + +export function validateQuestions(questions) { + if (!Array.isArray(questions) || questions.length === 0) { + throw new LavishError('questions must be a nonempty array', 2); + } + + const keys = new Set(); + return questions.map((rawQuestion, questionIndex) => { + const question = requirePlainObject(rawQuestion, `questions[${questionIndex}]`); + const key = requireString(question.key, `questions[${questionIndex}].key`); + if (!ID_PATTERN.test(key)) { + throw new LavishError( + `questions[${questionIndex}].key must be a lowercase slug`, + 2, + ); + } + if (keys.has(key)) { + throw new LavishError(`duplicate question key: ${key}`, 2); + } + keys.add(key); + + const prompt = requireString( + question.prompt, + `questions[${questionIndex}].prompt`, + ); + if (!Array.isArray(question.options) || question.options.length === 0) { + throw new LavishError( + `questions[${questionIndex}].options must be a nonempty array`, + 2, + ); + } + const values = new Set(); + const options = question.options.map((rawOption, optionIndex) => { + const option = requirePlainObject( + rawOption, + `questions[${questionIndex}].options[${optionIndex}]`, + ); + const value = requireString( + option.value, + `questions[${questionIndex}].options[${optionIndex}].value`, + ); + const label = requireString( + option.label, + `questions[${questionIndex}].options[${optionIndex}].label`, + ); + if (values.has(value)) { + throw new LavishError(`duplicate option value ${value} for question ${key}`, 2); + } + values.add(value); + return { value, label }; + }); + return { key, prompt, options }; + }); +} + +export function validateDestination(destination) { + requireString(destination, 'destination'); + if (isAbsolute(destination)) { + throw new LavishError('destination must be relative to FM_HOME', 2); + } + const normalized = destination.split(/[\\/]+/); + if ( + normalized.length < 2 + || normalized[0] !== 'data' + || normalized.some((part) => part === '' || part === '.' || part === '..') + ) { + throw new LavishError( + 'destination must be a normalized file path below FM_HOME/data', + 2, + ); + } + if (destination.endsWith('/')) { + throw new LavishError('destination must name a file', 2); + } + const normalizedPath = normalized.join('/'); + if ( + normalizedPath === DECISIONS_RELATIVE_DIR + || normalizedPath.startsWith(`${DECISIONS_RELATIVE_DIR}/`) + ) { + throw new LavishError('destination must not overwrite the decision protocol directory', 2); + } + return normalizedPath; +} + +export function validateManifest(raw, expectedId = undefined) { + const manifest = requirePlainObject(raw, 'manifest'); + if (manifest.kind !== MANIFEST_KIND || manifest.schema_version !== SCHEMA_VERSION) { + throw new LavishError('unsupported manifest kind or schema version', 2); + } + const decisionId = validateDecisionId(manifest.decision_id); + if (expectedId !== undefined && decisionId !== expectedId) { + throw new LavishError( + `manifest decision id ${decisionId} does not match directory ${expectedId}`, + 2, + ); + } + const title = requireString(manifest.title, 'manifest.title'); + const createdAt = requireTimestamp(manifest.created_at, 'manifest.created_at'); + const destination = validateDestination(manifest.destination); + const requestSha256 = requireDigest( + manifest.request_sha256, + 'manifest.request_sha256', + ); + const questions = validateQuestions(manifest.questions); + if (!Number.isInteger(manifest.expected_count) || manifest.expected_count < 1) { + throw new LavishError('manifest.expected_count must be a positive integer', 2); + } + if (manifest.expected_count !== questions.length) { + throw new LavishError( + `manifest expected_count ${manifest.expected_count} does not match ${questions.length} questions`, + 2, + ); + } + let legacySource; + if (manifest.legacy_source !== undefined) { + legacySource = requirePlainObject(manifest.legacy_source, 'manifest.legacy_source'); + requireString(legacySource.session_key, 'manifest.legacy_source.session_key'); + requireString(legacySource.file, 'manifest.legacy_source.file'); + requireString(legacySource.prompt_uid, 'manifest.legacy_source.prompt_uid'); + } + return { + kind: MANIFEST_KIND, + schema_version: SCHEMA_VERSION, + decision_id: decisionId, + title, + created_at: createdAt, + destination, + expected_count: manifest.expected_count, + request_sha256: requestSha256, + questions, + ...(legacySource === undefined ? {} : { legacy_source: legacySource }), + }; +} + +export function validateAnswer(raw, manifest) { + const answer = requirePlainObject(raw, 'answer'); + if (answer.kind !== ANSWER_KIND || answer.schema_version !== SCHEMA_VERSION) { + throw new LavishError('unsupported answer kind or schema version', 2); + } + if (answer.decision_id !== manifest.decision_id) { + throw new LavishError('answer decision id does not match manifest', 2); + } + if (answer.request_sha256 !== manifest.request_sha256) { + throw new LavishError('answer request digest does not match manifest', 2); + } + requireTimestamp(answer.submitted_at, 'answer.submitted_at'); + if (!Array.isArray(answer.answers)) { + throw new LavishError('answer.answers must be an array', 2); + } + if (answer.answers.length !== manifest.expected_count) { + throw new LavishError( + `answer count ${answer.answers.length} does not match expected count ${manifest.expected_count}`, + 2, + ); + } + + const seen = new Set(); + const normalizedAnswers = answer.answers.map((rawSelection, index) => { + const selection = requirePlainObject(rawSelection, `answer.answers[${index}]`); + const expectedQuestion = manifest.questions[index]; + const key = requireString(selection.key, `answer.answers[${index}].key`); + if (seen.has(key)) { + throw new LavishError(`duplicate answer key: ${key}`, 2); + } + seen.add(key); + if (key !== expectedQuestion.key) { + throw new LavishError( + `answer key ${key} is out of order; expected ${expectedQuestion.key}`, + 2, + ); + } + const value = requireString(selection.value, `answer.answers[${index}].value`); + const option = expectedQuestion.options.find((candidate) => candidate.value === value); + if (option === undefined) { + throw new LavishError(`answer value ${value} is not an option for ${key}`, 2); + } + if (selection.label !== option.label) { + throw new LavishError(`answer label does not match option ${value} for ${key}`, 2); + } + return { key, value, label: option.label }; + }); + + const note = answer.note === undefined + ? '' + : requireString(answer.note, 'answer.note', { allowEmpty: true }); + return { + kind: ANSWER_KIND, + schema_version: SCHEMA_VERSION, + decision_id: manifest.decision_id, + request_sha256: manifest.request_sha256, + submitted_at: answer.submitted_at, + answers: normalizedAnswers, + note, + }; +} + +export function validateReceipt(raw, manifest, answerSha256) { + const receipt = requirePlainObject(raw, 'receipt'); + if (receipt.kind !== RECEIPT_KIND || receipt.schema_version !== SCHEMA_VERSION) { + throw new LavishError('unsupported receipt kind or schema version', 2); + } + if (receipt.decision_id !== manifest.decision_id) { + throw new LavishError('receipt decision id does not match manifest', 2); + } + if (receipt.answer_sha256 !== answerSha256) { + throw new LavishError('receipt answer digest does not match answer', 2); + } + if (receipt.destination !== manifest.destination) { + throw new LavishError('receipt destination does not match manifest', 2); + } + requireDigest(receipt.destination_sha256, 'receipt.destination_sha256'); + requireTimestamp(receipt.consumed_at, 'receipt.consumed_at'); + return receipt; +} + +export function encodeToon(value) { + return `${encode(value)}\n`; +} + +export function decodeToon(text, label) { + try { + return decode(text, { strict: true }); + } catch (error) { + throw new LavishError(`${label} is not valid strict TOON: ${error.message}`, 2); + } +} + +async function lstatIfPresent(path) { + try { + return await lstat(path, { bigint: false }); + } catch (error) { + if (error.code === 'ENOENT') { + return undefined; + } + throw error; + } +} + +async function assertNoSymlink(path, label) { + try { + await readlink(path); + } catch (error) { + if (error.code === 'EINVAL' || error.code === 'ENOENT') { + return; + } + throw error; + } + throw new LavishError(`${label} must not be a symlink`, 2); +} + +export async function ensureSafeDirectoryTree(home, targetDirectory, { create = false } = {}) { + const realHome = resolve(home); + const realTarget = resolve(targetDirectory); + if (realTarget !== realHome && !realTarget.startsWith(`${realHome}${sep}`)) { + throw new LavishError(`unsafe path outside FM_HOME: ${realTarget}`, 2); + } + await assertNoSymlink(realHome, 'FM_HOME'); + if (create) { + await mkdir(realHome, { recursive: true, mode: 0o700 }); + } + const rel = relative(realHome, realTarget); + let cursor = realHome; + if (rel !== '') { + for (const part of rel.split(sep)) { + cursor = join(cursor, part); + await assertNoSymlink(cursor, cursor); + const existing = await lstatIfPresent(cursor); + if (existing === undefined) { + if (!create) { + throw new LavishError(`required directory is missing: ${cursor}`, 2); + } + await mkdir(cursor, { mode: 0o700 }); + } else if (!existing.isDirectory()) { + throw new LavishError(`expected directory: ${cursor}`, 2); + } + } + } + return realTarget; +} + +async function syncDirectory(path) { + let handle; + try { + handle = await open(path, 'r'); + await handle.sync(); + } catch (error) { + if (!['EINVAL', 'EBADF', 'EISDIR'].includes(error.code)) { + throw error; + } + } finally { + await handle?.close(); + } +} + +export async function atomicWrite(path, content, { + mode = 0o600, + failIfExists = false, + beforeRename = undefined, +} = {}) { + const parent = dirname(path); + await assertNoSymlink(parent, parent); + await assertNoSymlink(path, path); + if (failIfExists) { + try { + await access(path, fsConstants.F_OK); + throw new LavishError(`refusing to overwrite existing file: ${path}`, 3); + } catch (error) { + if (error instanceof LavishError) { + throw error; + } + if (error.code !== 'ENOENT') { + throw error; + } + } + } + const token = randomBytes(8).toString('hex'); + const temporary = join(parent, `.${path.split(sep).at(-1)}.tmp-${process.pid}-${token}`); + let handle; + let committed = false; + try { + handle = await open(temporary, 'wx', mode); + await handle.writeFile(content); + await handle.sync(); + await handle.close(); + handle = undefined; + await beforeRename?.(temporary); + if (failIfExists) { + try { + await access(path, fsConstants.F_OK); + throw new LavishError(`refusing to overwrite existing file: ${path}`, 3); + } catch (error) { + if (error instanceof LavishError) { + throw error; + } + if (error.code !== 'ENOENT') { + throw error; + } + } + } + await rename(temporary, path); + committed = true; + await chmod(path, mode); + await syncDirectory(parent); + } finally { + await handle?.close(); + if (!committed && process.env.LAVISH_TEST_KEEP_TEMP !== '1') { + await rm(temporary, { force: true }); + } + } +} + +export async function readDecision(home, id) { + validateDecisionId(id); + const decisionDirectory = join(resolve(home), DECISIONS_RELATIVE_DIR, id); + await ensureSafeDirectoryTree(home, decisionDirectory); + const requestPath = join(decisionDirectory, 'request.md'); + const manifestPath = join(decisionDirectory, 'manifest.toon'); + await assertNoSymlink(requestPath, requestPath); + await assertNoSymlink(manifestPath, manifestPath); + const [request, manifestText] = await Promise.all([ + readFile(requestPath), + readFile(manifestPath, 'utf8'), + ]); + const manifest = validateManifest(decodeToon(manifestText, manifestPath), id); + const requestDigest = sha256Bytes(request); + if (requestDigest !== manifest.request_sha256) { + throw new LavishError(`request digest mismatch for ${id}`, 2); + } + return { + id, + directory: decisionDirectory, + request, + requestText: request.toString('utf8'), + manifest, + manifestText, + answerPath: join(decisionDirectory, 'answer.toon'), + receiptPath: join(decisionDirectory, 'receipt.toon'), + }; +} + +export async function listDecisions(home) { + const decisionsDirectory = join(resolve(home), DECISIONS_RELATIVE_DIR); + await ensureSafeDirectoryTree(home, decisionsDirectory, { create: true }); + const entries = await readdir(decisionsDirectory, { withFileTypes: true }); + const decisions = []; + for (const entry of entries.sort((left, right) => left.name.localeCompare(right.name))) { + if (!entry.isDirectory() || !ID_PATTERN.test(entry.name)) { + continue; + } + try { + const decision = await readDecision(home, entry.name); + const [answer, receipt] = await Promise.all([ + lstatIfPresent(decision.answerPath), + lstatIfPresent(decision.receiptPath), + ]); + decisions.push({ + ...decision, + status: receipt?.isFile() ? 'consumed' : answer?.isFile() ? 'answered' : 'pending', + }); + } catch (error) { + decisions.push({ id: entry.name, status: 'invalid', error }); + } + } + return decisions; +} + +export async function createDecision(home, { + id, + title, + request, + destination, + questions, + createdAt = new Date().toISOString(), + legacySource = undefined, +}) { + validateDecisionId(id); + requireString(title, 'title'); + requireTimestamp(createdAt, 'created_at'); + const normalizedDestination = validateDestination(destination); + const normalizedQuestions = validateQuestions(questions); + const requestBytes = Buffer.isBuffer(request) ? request : Buffer.from(request, 'utf8'); + if (requestBytes.length === 0) { + throw new LavishError('request.md must not be empty', 2); + } + const requestDigest = sha256Bytes(requestBytes); + const manifest = validateManifest({ + kind: MANIFEST_KIND, + schema_version: SCHEMA_VERSION, + decision_id: id, + title, + created_at: createdAt, + destination: normalizedDestination, + expected_count: normalizedQuestions.length, + request_sha256: requestDigest, + questions: normalizedQuestions, + ...(legacySource === undefined ? {} : { legacy_source: legacySource }), + }, id); + + const decisionsDirectory = join(resolve(home), DECISIONS_RELATIVE_DIR); + await ensureSafeDirectoryTree(home, decisionsDirectory, { create: true }); + const finalDirectory = join(decisionsDirectory, id); + const existing = await lstatIfPresent(finalDirectory); + if (existing !== undefined) { + if (!existing.isDirectory()) { + throw new LavishError(`decision path is not a directory: ${finalDirectory}`, 2); + } + const current = await readDecision(home, id); + const currentComparable = { + ...current.manifest, + created_at: manifest.created_at, + }; + if ( + current.request.equals(requestBytes) + && JSON.stringify(currentComparable) === JSON.stringify(manifest) + ) { + return { decision: current, created: false }; + } + throw new LavishError(`decision ${id} already exists with different content`, 3); + } + + const staging = join( + decisionsDirectory, + `.${id}.creating-${process.pid}-${randomBytes(8).toString('hex')}`, + ); + await mkdir(staging, { mode: 0o700 }); + let committed = false; + try { + await atomicWrite(join(staging, 'request.md'), requestBytes, { failIfExists: true }); + await atomicWrite(join(staging, 'manifest.toon'), encodeToon(manifest), { + failIfExists: true, + }); + await syncDirectory(staging); + await rename(staging, finalDirectory); + committed = true; + await syncDirectory(decisionsDirectory); + } finally { + if (!committed) { + await rm(staging, { recursive: true, force: true }); + } + } + return { decision: await readDecision(home, id), created: true }; +} + +async function pidAlive(pid) { + if (!Number.isInteger(pid) || pid <= 0) { + return false; + } + try { + process.kill(pid, 0); + return true; + } catch (error) { + if (error.code === 'ESRCH') { + return false; + } + return true; + } +} + +export async function withDecisionLock(decisionDirectory, operation) { + const lockPath = join(decisionDirectory, '.answer.lock'); + for (let attempt = 0; attempt < 3; attempt += 1) { + try { + await mkdir(lockPath, { mode: 0o700 }); + await atomicWrite( + join(lockPath, 'owner.json'), + `${JSON.stringify({ pid: process.pid, created_at: new Date().toISOString() })}\n`, + { failIfExists: true }, + ); + try { + return await operation(); + } finally { + await rm(lockPath, { recursive: true, force: true }); + } + } catch (error) { + if (error.code !== 'EEXIST') { + throw error; + } + let owner; + try { + owner = JSON.parse(await readFile(join(lockPath, 'owner.json'), 'utf8')); + } catch { + owner = {}; + } + const created = Number.isFinite(Date.parse(owner.created_at)) + ? Date.parse(owner.created_at) + : 0; + if (await pidAlive(owner.pid) && Date.now() - created < 300_000) { + throw new LavishError('another lavish answer command is committing this decision', 4); + } + const stalePath = `${lockPath}.stale-${process.pid}-${randomBytes(4).toString('hex')}`; + try { + await rename(lockPath, stalePath); + await rm(stalePath, { recursive: true, force: true }); + } catch (staleError) { + if (staleError.code !== 'ENOENT') { + throw staleError; + } + } + } + } + throw new LavishError('could not acquire decision commit lock', 4); +} + +export async function readAnswer(decision) { + await assertNoSymlink(decision.answerPath, decision.answerPath); + const text = await readFile(decision.answerPath, 'utf8'); + const answer = validateAnswer(decodeToon(text, decision.answerPath), decision.manifest); + return { answer, text, digest: sha256Bytes(Buffer.from(text, 'utf8')) }; +} + +async function cleanStaleAnswerTemps(decisionDirectory) { + const entries = await readdir(decisionDirectory, { withFileTypes: true }); + for (const entry of entries) { + if ( + entry.isFile() + && /^\.answer\.toon\.tmp-[0-9]+-[0-9a-f]{16}$/.test(entry.name) + ) { + await rm(join(decisionDirectory, entry.name), { force: true }); + } + } +} + +export async function commitAnswer(decision, selections, note, { + submittedAt = new Date().toISOString(), + beforeRename = undefined, +} = {}) { + const answer = validateAnswer({ + kind: ANSWER_KIND, + schema_version: SCHEMA_VERSION, + decision_id: decision.id, + request_sha256: decision.manifest.request_sha256, + submitted_at: submittedAt, + answers: selections, + note, + }, decision.manifest); + const text = encodeToon(answer); + await withDecisionLock(decision.directory, async () => { + await cleanStaleAnswerTemps(decision.directory); + const existing = await lstatIfPresent(decision.answerPath); + if (existing !== undefined) { + throw new LavishError(`decision ${decision.id} is already answered`, 3); + } + await atomicWrite(decision.answerPath, text, { + failIfExists: true, + beforeRename, + }); + }); + return { answer, text, digest: sha256Bytes(Buffer.from(text, 'utf8')) }; +} + +async function copyFileAtomic(source, destination) { + const parent = dirname(destination); + const temporary = join( + parent, + `.${destination.split(sep).at(-1)}.tmp-${process.pid}-${randomBytes(8).toString('hex')}`, + ); + let committed = false; + try { + await copyFile(source, temporary, fsConstants.COPYFILE_EXCL); + const handle = await open(temporary, 'r+'); + await handle.sync(); + await handle.close(); + await rename(temporary, destination); + committed = true; + await chmod(destination, 0o600); + await syncDirectory(parent); + } finally { + if (!committed) { + await rm(temporary, { force: true }); + } + } +} + +export async function intakeDecision(home, decision) { + const { text: answerText, digest: answerDigest } = await readAnswer(decision); + const receiptState = await lstatIfPresent(decision.receiptPath); + if (receiptState !== undefined) { + const receiptText = await readFile(decision.receiptPath, 'utf8'); + const receipt = validateReceipt( + decodeToon(receiptText, decision.receiptPath), + decision.manifest, + answerDigest, + ); + const destinationPath = join(resolve(home), receipt.destination); + const destinationDigest = await sha256File(destinationPath); + if (destinationDigest !== receipt.destination_sha256) { + throw new LavishError(`consumed destination digest changed for ${decision.id}`, 2); + } + return { status: 'already-consumed', receipt }; + } + + const destinationPath = join(resolve(home), decision.manifest.destination); + const destinationParent = dirname(destinationPath); + await ensureSafeDirectoryTree(home, destinationParent, { create: true }); + await assertNoSymlink(destinationPath, destinationPath); + const destinationState = await lstatIfPresent(destinationPath); + if (destinationState !== undefined) { + if (!destinationState.isFile()) { + throw new LavishError(`destination is not a regular file: ${destinationPath}`, 2); + } + const destinationDigest = await sha256File(destinationPath); + if (destinationDigest !== answerDigest) { + throw new LavishError( + `destination already exists with different content: ${decision.manifest.destination}`, + 3, + ); + } + } else { + await copyFileAtomic(decision.answerPath, destinationPath); + } + + const destinationSha256 = await sha256File(destinationPath); + if (destinationSha256 !== answerDigest) { + throw new LavishError(`destination digest mismatch after write for ${decision.id}`, 2); + } + const receipt = { + kind: RECEIPT_KIND, + schema_version: SCHEMA_VERSION, + decision_id: decision.id, + answer_sha256: answerDigest, + destination: decision.manifest.destination, + destination_sha256: destinationSha256, + consumed_at: new Date().toISOString(), + }; + await atomicWrite(decision.receiptPath, encodeToon(receipt), { failIfExists: true }); + return { status: 'consumed', receipt, answerText }; +} + +export async function intakeAll(home) { + const decisions = await listDecisions(home); + const results = []; + let failed = false; + for (const decision of decisions) { + if (decision.status === 'pending') { + continue; + } + if (decision.status === 'invalid') { + failed = true; + results.push({ id: decision.id, status: 'invalid', detail: decision.error.message }); + continue; + } + if (decision.status === 'consumed') { + continue; + } + try { + const result = await intakeDecision(home, decision); + results.push({ id: decision.id, status: result.status, detail: '' }); + } catch (error) { + failed = true; + results.push({ id: decision.id, status: 'error', detail: error.message }); + } + } + return { results, failed }; +} diff --git a/tools/lavish/test/lavish.test.mjs b/tools/lavish/test/lavish.test.mjs new file mode 100644 index 0000000000..53b746a6e9 --- /dev/null +++ b/tools/lavish/test/lavish.test.mjs @@ -0,0 +1,562 @@ +import assert from 'node:assert/strict'; +import { execFileSync, spawn } from 'node:child_process'; +import { + access, + chmod, + mkdir, + mkdtemp, + readFile, + readlink, + readdir, + symlink, + stat, + utimes, + writeFile, +} from 'node:fs/promises'; +import { constants as fsConstants } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import test from 'node:test'; +import { decode } from '@toon-format/toon'; + +const PACKAGE_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..'); +const REPO_ROOT = resolve(PACKAGE_ROOT, '../..'); +const CLI = join(PACKAGE_ROOT, 'src/cli.mjs'); +const WAKE_ADAPTER = join(REPO_ROOT, 'bin/fm-lavish-wake.sh'); +const WAKE_DRAIN = join(REPO_ROOT, 'bin/fm-wake-drain.sh'); + +async function exists(path) { + try { + await access(path, fsConstants.F_OK); + return true; + } catch { + return false; + } +} + +async function fixture(name) { + const root = await mkdtemp(join(tmpdir(), `lavish-${name}-`)); + const home = join(root, 'home'); + const request = join(root, 'request.md'); + const questions = join(root, 'questions.json'); + await mkdir(home, { recursive: true }); + await writeFile( + request, + '# Release choice\n\nRecommendation: choose blue.\n\nBlue is safer; green is faster.\n', + ); + await writeFile( + questions, + `${JSON.stringify([{ + key: 'rollout', + prompt: 'Which rollout should we use?', + options: [ + { value: 'blue', label: 'Blue rollout' }, + { value: 'green', label: 'Green rollout' }, + ], + }])}\n`, + ); + return { root, home, request, questions }; +} + +function processGroupMembers(pgid) { + const output = execFileSync('ps', ['-axo', 'pid=,pgid=,command='], { + encoding: 'utf8', + }); + return output + .trim() + .split('\n') + .map((line) => line.trim().match(/^(\d+)\s+(\d+)\s+(.*)$/)) + .filter((match) => match !== null && Number(match[2]) === pgid) + .map((match) => ({ pid: Number(match[1]), command: match[3] })); +} + +async function listeningSockets(pid) { + try { + const output = execFileSync( + 'lsof', + ['-nP', '-a', '-p', String(pid), '-iTCP', '-sTCP:LISTEN'], + { encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] }, + ); + return output.trim() === '' ? [] : output.trim().split('\n').slice(1); + } catch (error) { + if (error.code !== 'ENOENT') { + return []; + } + } + + const listeningInodes = new Set(); + for (const table of ['/proc/net/tcp', '/proc/net/tcp6']) { + if (!(await exists(table))) continue; + const lines = (await readFile(table, 'utf8')).trim().split('\n').slice(1); + for (const line of lines) { + const columns = line.trim().split(/\s+/); + if (columns[3] === '0A') listeningInodes.add(columns[9]); + } + } + const sockets = []; + const fdDirectory = `/proc/${pid}/fd`; + if (!(await exists(fdDirectory))) return sockets; + for (const fd of await readdir(fdDirectory)) { + try { + const target = await readlink(join(fdDirectory, fd)); + const match = target.match(/^socket:\[(\d+)]$/); + if (match !== null && listeningInodes.has(match[1])) sockets.push(match[1]); + } catch { + // Descriptors can close while they are inspected. + } + } + return sockets; +} + +function runCli(args, { + home, + input = '', + env = {}, +} = {}) { + return new Promise((resolveRun, rejectRun) => { + const child = spawn(CLI, [...args, '--home', home], { + detached: true, + env: { + ...process.env, + LAVISH_WAKE_COMMAND: WAKE_ADAPTER, + ...env, + }, + stdio: ['pipe', 'pipe', 'pipe'], + }); + let stdout = ''; + let stderr = ''; + child.stdout.on('data', (chunk) => { + stdout += chunk; + }); + child.stderr.on('data', (chunk) => { + stderr += chunk; + }); + child.on('error', rejectRun); + child.on('close', (code, signal) => { + const members = processGroupMembers(child.pid); + assert.deepEqual( + members, + [], + `command left processes in group ${child.pid}: ${JSON.stringify(members)}`, + ); + resolveRun({ code, signal, stdout, stderr, pid: child.pid }); + }); + child.stdin.end(input); + }); +} + +function runExecutable(executable, args, { + env = {}, + input = '', +} = {}) { + return new Promise((resolveRun, rejectRun) => { + const child = spawn(executable, args, { + detached: true, + env: { ...process.env, ...env }, + stdio: ['pipe', 'pipe', 'pipe'], + }); + let stdout = ''; + let stderr = ''; + child.stdout.on('data', (chunk) => { + stdout += chunk; + }); + child.stderr.on('data', (chunk) => { + stderr += chunk; + }); + child.on('error', rejectRun); + child.on('close', (code, signal) => { + const members = processGroupMembers(child.pid); + assert.deepEqual( + members, + [], + `command left processes in group ${child.pid}: ${JSON.stringify(members)}`, + ); + resolveRun({ code, signal, stdout, stderr, pid: child.pid }); + }); + child.stdin.end(input); + }); +} + +async function createRequest(fx, { + id = 'release-choice', + destination = 'data/replies/release-choice.toon', + createdAt = undefined, +} = {}) { + const args = [ + 'create', + '--id', + id, + '--title', + 'Release choice', + '--request', + fx.request, + '--questions', + fx.questions, + '--destination', + destination, + ]; + if (createdAt !== undefined) args.push('--created-at', createdAt); + const result = await runCli(args, { home: fx.home }); + assert.equal(result.code, 0, result.stderr); + return id; +} + +async function answer(fx, id, { + choice = 1, + note = '', + env = {}, +} = {}) { + return await runCli(['answer', id], { + home: fx.home, + input: `${choice}\n${note}\ny\n`, + env, + }); +} + +test('a seven-day-old request remains answerable with no firstmate process', async () => { + const fx = await fixture('seven-day'); + const createdAt = new Date(Date.now() - 7 * 24 * 60 * 60 * 1000).toISOString(); + const id = await createRequest(fx, { createdAt }); + const decisionDirectory = join(fx.home, 'data/decisions', id); + const oldTime = new Date(createdAt); + await utimes(join(decisionDirectory, 'request.md'), oldTime, oldTime); + await utimes(join(decisionDirectory, 'manifest.toon'), oldTime, oldTime); + + assert.equal(await exists(join(fx.home, 'state/.lock')), false); + const result = await answer(fx, id, { note: 'Answer after seven quiet days' }); + assert.equal(result.code, 0, result.stderr); + assert.match(result.stdout, /Answer saved.*wake queued/); + assert.equal(await exists(join(decisionDirectory, 'answer.toon')), true); + assert.equal(await exists(join(fx.home, 'state/.wake-queue')), true); + + const inbox = await runCli(['inbox'], { home: fx.home }); + assert.equal(inbox.code, 0, inbox.stderr); + assert.equal(inbox.stdout, 'No pending Lavish decisions.\n'); +}); + +test('interruption before rename cannot expose a partial answer', async () => { + const fx = await fixture('interrupted'); + const id = await createRequest(fx); + const decisionDirectory = join(fx.home, 'data/decisions', id); + const crashed = await answer(fx, id, { + env: { + LAVISH_TEST_CRASH_BEFORE_RENAME: '1', + LAVISH_TEST_KEEP_TEMP: '1', + }, + }); + assert.equal(crashed.code, null); + assert.equal(crashed.signal, 'SIGKILL'); + assert.equal(await exists(join(decisionDirectory, 'answer.toon')), false); + const leftovers = (await readdir(decisionDirectory)) + .filter((name) => name.startsWith('.answer.toon.tmp-')); + assert.equal(leftovers.length, 1); + + const inbox = await runCli(['inbox'], { home: fx.home }); + assert.equal(inbox.code, 0, inbox.stderr); + assert.match(inbox.stdout, new RegExp(id)); + + const recovered = await answer(fx, id, { choice: 2 }); + assert.equal(recovered.code, 0, recovered.stderr); + assert.equal(await exists(join(decisionDirectory, 'answer.toon')), true); + assert.deepEqual( + (await readdir(decisionDirectory)).filter((name) => name.startsWith('.answer.toon.tmp-')), + [], + ); +}); + +test('cancelling the explicit confirmation leaves no answer', async () => { + const fx = await fixture('cancel'); + const id = await createRequest(fx); + const cancelled = await runCli(['answer', id], { + home: fx.home, + input: '1\nnot yet\nn\n', + }); + assert.equal(cancelled.code, 0, cancelled.stderr); + assert.match(cancelled.stdout, /Cancelled; no answer was saved/); + assert.equal( + await exists(join(fx.home, 'data/decisions', id, 'answer.toon')), + false, + ); +}); + +test('duplicate submission is idempotent and does not queue another wake', async () => { + const fx = await fixture('duplicate'); + const id = await createRequest(fx); + const first = await answer(fx, id); + assert.equal(first.code, 0, first.stderr); + const answerPath = join(fx.home, 'data/decisions', id, 'answer.toon'); + const before = await readFile(answerPath); + const wakePath = join(fx.home, 'state/.wake-queue'); + const wakesBefore = await readFile(wakePath, 'utf8'); + + const duplicate = await answer(fx, id, { choice: 2, note: 'must not overwrite' }); + assert.equal(duplicate.code, 0, duplicate.stderr); + assert.match(duplicate.stdout, /already submitted.*answer unchanged/); + assert.deepEqual(await readFile(answerPath), before); + assert.equal(await readFile(wakePath, 'utf8'), wakesBefore); +}); + +test('failed wake append is recoverable by ordinary intake scan', async () => { + const fx = await fixture('wake-failure'); + const id = await createRequest(fx); + const result = await answer(fx, id, { + env: { LAVISH_WAKE_COMMAND: '/usr/bin/false' }, + }); + assert.equal(result.code, 5); + assert.match(result.stderr, /answer saved; wake not queued/); + const decisionDirectory = join(fx.home, 'data/decisions', id); + assert.equal(await exists(join(decisionDirectory, 'answer.toon')), true); + assert.equal(await exists(join(decisionDirectory, 'receipt.toon')), false); + + const intake = await runCli(['intake'], { home: fx.home }); + assert.equal(intake.code, 0, intake.stderr); + assert.match(intake.stdout, /release-choice,consumed/); + assert.equal(await exists(join(decisionDirectory, 'receipt.toon')), true); + assert.deepEqual( + await readFile(join(decisionDirectory, 'answer.toon')), + await readFile(join(fx.home, 'data/replies/release-choice.toon')), + ); + + const again = await runCli(['intake'], { home: fx.home }); + assert.equal(again.code, 0, again.stderr); + assert.equal(again.stdout, ''); +}); + +test('ordinary wake drain consumes the answer before draining its pointer', async () => { + const fx = await fixture('wake-boundary'); + const id = await createRequest(fx); + const answered = await answer(fx, id); + assert.equal(answered.code, 0, answered.stderr); + const fakeBin = join(fx.root, 'bin'); + await mkdir(fakeBin); + await symlink(CLI, join(fakeBin, 'lavish-axi')); + + const drained = await runExecutable(WAKE_DRAIN, [], { + env: { + FM_HOME: fx.home, + PATH: `${fakeBin}:${process.env.PATH}`, + }, + }); + assert.equal(drained.code, 0, drained.stderr); + assert.match(drained.stdout, /LAVISH_INTAKE:/); + assert.match(drained.stdout, /release-choice,consumed/); + assert.match(drained.stdout, /\tsignal\tlavish:release-choice\tdecision-answer:/); + assert.equal( + await exists(join(fx.home, 'data/decisions', id, 'receipt.toon')), + true, + ); + assert.equal( + await exists(join(fx.home, 'data/replies/release-choice.toon')), + true, + ); +}); + +test('configured wake adapter works from an installed-style CLI location', async () => { + const fx = await fixture('configured-wake'); + const id = await createRequest(fx); + const adapterLog = join(fx.root, 'wake-adapter.log'); + const adapter = join(fx.root, 'configured-wake.sh'); + await writeFile( + adapter, + `#!/bin/sh\nprintf '%s\\n' "$*" >> '${adapterLog}'\nexec '${WAKE_ADAPTER}' "$@"\n`, + ); + await chmod(adapter, 0o700); + + const configured = await runExecutable( + CLI, + ['configure-wake', '--command', adapter, '--home', fx.home], + ); + assert.equal(configured.code, 0, configured.stderr); + const answered = await runExecutable( + CLI, + ['answer', id, '--home', fx.home], + { input: '1\nconfigured path\ny\n' }, + ); + assert.equal(answered.code, 0, answered.stderr); + assert.match(await readFile(adapterLog, 'utf8'), /--decision release-choice/); +}); + +test('strict TOON count validation rejects a malformed manifest before showing it', async () => { + const fx = await fixture('toon-count'); + const id = await createRequest(fx); + const manifestPath = join(fx.home, 'data/decisions', id, 'manifest.toon'); + const valid = await readFile(manifestPath, 'utf8'); + const malformed = valid.replace('questions[1]', 'questions[2]'); + assert.notEqual(malformed, valid); + await writeFile(manifestPath, malformed); + + const shown = await runCli(['show', id], { home: fx.home }); + assert.equal(shown.code, 2); + assert.match(shown.stderr, /not valid strict TOON/); +}); + +test('legacy migration snapshots first, imports queued input, and ignores open status', async () => { + const fx = await fixture('migration'); + const statePath = join(fx.root, 'state.json'); + const snapshotDirectory = join(fx.root, 'audit'); + const sessions = {}; + for (let index = 0; index < 243; index += 1) { + const key = `session-${String(index).padStart(3, '0')}`; + sessions[key] = { + key, + file: `/legacy/${key}.html`, + status: index < 152 ? 'open' : 'ended', + updated_at: '2026-07-30T12:00:00.000Z', + pending_prompts: 0, + prompts: [], + }; + } + sessions['session-200'] = { + ...sessions['session-200'], + status: 'feedback', + pending_prompts: 1, + prompts: [{ + uid: 'captain-1', + prompt: 'Defer this option until the next review.', + selector: 'body > button', + tag: 'decision-batch', + text: 'Final send choice', + }], + }; + const source = `${JSON.stringify({ sessions }, null, 2)}\n`; + await writeFile(statePath, source); + + const migrated = await runCli( + [ + 'migrate-legacy', + '--state', + statePath, + '--snapshot-dir', + snapshotDirectory, + ], + { home: fx.home }, + ); + assert.equal(migrated.code, 0, migrated.stderr); + assert.match(migrated.stdout, /source_sessions: 243/); + assert.match(migrated.stdout, /imported_prompt_count: 1/); + assert.match(migrated.stdout, /migrated_request_count: 0/); + assert.equal(await readFile(statePath, 'utf8'), source); + + const snapshots = await readdir(snapshotDirectory); + assert.equal(snapshots.length, 1); + assert.equal(await readFile(join(snapshotDirectory, snapshots[0]), 'utf8'), source); + const decisions = await readdir(join(fx.home, 'data/decisions')); + assert.deepEqual(decisions.filter((name) => !name.startsWith('.')), [ + 'legacy-session-200-captain-1', + ]); + const answerText = await readFile( + join(fx.home, 'data/decisions/legacy-session-200-captain-1/answer.toon'), + 'utf8', + ); + const imported = decode(answerText, { strict: true }); + assert.equal(imported.note, 'Defer this option until the next review.'); +}); + +test('an explicit reconciled map migrates only the named unanswered session', async () => { + const fx = await fixture('pending-map'); + const statePath = join(fx.root, 'state.json'); + const snapshotDirectory = join(fx.root, 'audit'); + await writeFile( + statePath, + `${JSON.stringify({ + sessions: { + relevant: { + file: '/legacy/relevant.html', + status: 'open', + pending_prompts: 0, + prompts: [], + }, + irrelevant: { + file: '/legacy/irrelevant.html', + status: 'open', + pending_prompts: 0, + prompts: [], + }, + }, + })}\n`, + ); + const pendingMapPath = join(fx.root, 'pending-map.json'); + await writeFile( + pendingMapPath, + `${JSON.stringify([{ + session_key: 'relevant', + decision_id: 'reconciled-choice', + title: 'Reconciled choice', + destination: 'data/replies/reconciled-choice.toon', + request_md: '# Reconciled choice\n\nRecommendation: A.\n\nA is safer; B is faster.\n', + questions: [{ + key: 'choice', + prompt: 'Choose the next step.', + options: [ + { value: 'a', label: 'A' }, + { value: 'b', label: 'B' }, + ], + }], + }])}\n`, + ); + const result = await runCli( + [ + 'migrate-legacy', + '--state', + statePath, + '--snapshot-dir', + snapshotDirectory, + '--pending-map', + pendingMapPath, + ], + { home: fx.home }, + ); + assert.equal(result.code, 0, result.stderr); + assert.match(result.stdout, /migrated_request_count: 1/); + assert.equal( + await exists(join(fx.home, 'data/decisions/reconciled-choice/manifest.toon')), + true, + ); + assert.equal( + await exists(join(fx.home, 'data/decisions/irrelevant/manifest.toon')), + false, + ); +}); + +test('answer has no child process or listening socket while waiting and leaves none', async () => { + const fx = await fixture('resources'); + const id = await createRequest(fx); + const child = spawn(CLI, ['answer', id, '--home', fx.home], { + detached: true, + env: { + ...process.env, + LAVISH_WAKE_COMMAND: WAKE_ADAPTER, + }, + stdio: ['pipe', 'pipe', 'pipe'], + }); + let stdout = ''; + child.stdout.on('data', (chunk) => { + stdout += chunk; + }); + const deadline = Date.now() + 5_000; + while (!stdout.includes('Choose 1-2 for rollout: ') && Date.now() < deadline) { + await new Promise((resolveWait) => setTimeout(resolveWait, 20)); + } + assert.match(stdout, /Choose 1-2 for rollout:/); + assert.deepEqual(processGroupMembers(child.pid), [ + { pid: child.pid, command: processGroupMembers(child.pid)[0]?.command }, + ]); + assert.deepEqual(await listeningSockets(child.pid), []); + + const completion = new Promise((resolveCompletion, rejectCompletion) => { + let stderr = ''; + child.stderr.on('data', (chunk) => { + stderr += chunk; + }); + child.on('error', rejectCompletion); + child.on('close', (code, signal) => { + resolveCompletion({ code, signal, stderr }); + }); + }); + child.stdin.end('1\nresource proof\ny\n'); + const result = await completion; + assert.equal(result.signal, null); + assert.equal(result.code, 0, result.stderr); + assert.deepEqual(processGroupMembers(child.pid), []); +}); From 4bc02df707322336c35283fb0c48625e0c462bd5 Mon Sep 17 00:00:00 2001 From: Dongkeun Lee <dongkeun@rubydata.ai> Date: Thu, 30 Jul 2026 21:50:40 -0400 Subject: [PATCH 2/9] no-mistakes(review): Prevent intake clobbers and legacy migration ID collisions --- tools/lavish/src/migration.mjs | 5 +++- tools/lavish/src/protocol.mjs | 29 +++++++++++++++--- tools/lavish/test/lavish.test.mjs | 49 +++++++++++++++++++++++++++---- 3 files changed, 73 insertions(+), 10 deletions(-) diff --git a/tools/lavish/src/migration.mjs b/tools/lavish/src/migration.mjs index 7744d2187d..0838c60026 100644 --- a/tools/lavish/src/migration.mjs +++ b/tools/lavish/src/migration.mjs @@ -167,8 +167,11 @@ async function importLegacyPrompt(home, sessionKey, session, prompt, index) { const uid = typeof prompt.uid === 'string' && prompt.uid.trim() !== '' ? prompt.uid : String(index + 1); + const identityDigest = sha256Bytes( + Buffer.from(JSON.stringify([sessionKey, uid, index]), 'utf8'), + ).slice('sha256:'.length, 'sha256:'.length + 23); const id = validateDecisionId( - `legacy-${legacyIdPart(sessionKey)}-${legacyIdPart(uid)}`.slice(0, 64).replace(/-$/, ''), + `legacy-${legacyIdPart(sessionKey).slice(0, 16)}-${legacyIdPart(uid).slice(0, 16)}-${identityDigest}`, ); const sourceFile = typeof session.file === 'string' ? session.file : '(unknown legacy file)'; const request = [ diff --git a/tools/lavish/src/protocol.mjs b/tools/lavish/src/protocol.mjs index 664c3db3dc..89eb2d4983 100644 --- a/tools/lavish/src/protocol.mjs +++ b/tools/lavish/src/protocol.mjs @@ -7,6 +7,7 @@ import { chmod, copyFile, lstat, + link, mkdir, open, readFile, @@ -14,6 +15,7 @@ import { readlink, rename, rm, + unlink, } from 'node:fs/promises'; import { createHash, randomBytes } from 'node:crypto'; import { dirname, isAbsolute, join, relative, resolve, sep } from 'node:path'; @@ -690,10 +692,19 @@ async function copyFileAtomic(source, destination) { const handle = await open(temporary, 'r+'); await handle.sync(); await handle.close(); - await rename(temporary, destination); + await chmod(temporary, 0o600); + try { + await link(temporary, destination); + } catch (error) { + if (error.code === 'EEXIST') { + return false; + } + throw error; + } + await unlink(temporary); committed = true; - await chmod(destination, 0o600); await syncDirectory(parent); + return true; } finally { if (!committed) { await rm(temporary, { force: true }); @@ -735,8 +746,18 @@ export async function intakeDecision(home, decision) { 3, ); } - } else { - await copyFileAtomic(decision.answerPath, destinationPath); + } else if (!await copyFileAtomic(decision.answerPath, destinationPath)) { + const concurrentDestinationState = await lstatIfPresent(destinationPath); + if (concurrentDestinationState === undefined || !concurrentDestinationState.isFile()) { + throw new LavishError(`destination is not a regular file: ${destinationPath}`, 2); + } + const concurrentDestinationDigest = await sha256File(destinationPath); + if (concurrentDestinationDigest !== answerDigest) { + throw new LavishError( + `destination already exists with different content: ${decision.manifest.destination}`, + 3, + ); + } } const destinationSha256 = await sha256File(destinationPath); diff --git a/tools/lavish/test/lavish.test.mjs b/tools/lavish/test/lavish.test.mjs index 53b746a6e9..dc535a5a7f 100644 --- a/tools/lavish/test/lavish.test.mjs +++ b/tools/lavish/test/lavish.test.mjs @@ -441,18 +441,57 @@ test('legacy migration snapshots first, imports queued input, and ignores open s const snapshots = await readdir(snapshotDirectory); assert.equal(snapshots.length, 1); assert.equal(await readFile(join(snapshotDirectory, snapshots[0]), 'utf8'), source); - const decisions = await readdir(join(fx.home, 'data/decisions')); - assert.deepEqual(decisions.filter((name) => !name.startsWith('.')), [ - 'legacy-session-200-captain-1', - ]); + const decisions = (await readdir(join(fx.home, 'data/decisions'))) + .filter((name) => !name.startsWith('.')); + assert.equal(decisions.length, 1); + assert.match(decisions[0], /^legacy-session-200-captain-1-[0-9a-f]{23}$/); const answerText = await readFile( - join(fx.home, 'data/decisions/legacy-session-200-captain-1/answer.toon'), + join(fx.home, 'data/decisions', decisions[0], 'answer.toon'), 'utf8', ); const imported = decode(answerText, { strict: true }); assert.equal(imported.note, 'Defer this option until the next review.'); }); +test('legacy migration assigns distinct ids to lossy prompt-name collisions', async () => { + const fx = await fixture('migration-collisions'); + const statePath = join(fx.root, 'state.json'); + const snapshotDirectory = join(fx.root, 'audit'); + await writeFile( + statePath, + `${JSON.stringify({ + sessions: { + 'session/with/a/name-that-shares-more-than-thirty-two-characters': { + file: '/legacy/collisions.html', + updated_at: '2026-07-30T12:00:00.000Z', + pending_prompts: 2, + prompts: [ + { uid: 'review/a', prompt: 'First answer.' }, + { uid: 'review-a', prompt: 'Second answer.' }, + ], + }, + }, + })}\n`, + ); + + const migrated = await runCli( + [ + 'migrate-legacy', + '--state', + statePath, + '--snapshot-dir', + snapshotDirectory, + ], + { home: fx.home }, + ); + assert.equal(migrated.code, 0, migrated.stderr); + assert.match(migrated.stdout, /imported_prompt_count: 2/); + const decisions = (await readdir(join(fx.home, 'data/decisions'))) + .filter((name) => !name.startsWith('.')); + assert.equal(decisions.length, 2); + assert.notEqual(decisions[0], decisions[1]); +}); + test('an explicit reconciled map migrates only the named unanswered session', async () => { const fx = await fixture('pending-map'); const statePath = join(fx.root, 'state.json'); From 6f07911345a883d45b25f87781ecfe89a278bad2 Mon Sep 17 00:00:00 2001 From: Dongkeun Lee <dongkeun@rubydata.ai> Date: Thu, 30 Jul 2026 21:52:38 -0400 Subject: [PATCH 3/9] no-mistakes(review): Require store-forward marker before Lavish intake --- bin/fm-lavish-intake.sh | 2 +- tests/lavish.test.sh | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+), 1 deletion(-) diff --git a/bin/fm-lavish-intake.sh b/bin/fm-lavish-intake.sh index 279ebe07dd..748315e106 100755 --- a/bin/fm-lavish-intake.sh +++ b/bin/fm-lavish-intake.sh @@ -12,7 +12,7 @@ FM_HOME="${FM_HOME:-${FM_ROOT_OVERRIDE:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." command -v lavish-axi >/dev/null 2>&1 || exit 0 version=$(lavish-axi --version 2>/dev/null || true) case "$version" in - 'lavish-axi 1.'*) ;; + 'lavish-axi 1.'*'(store-forward protocol 1)') ;; *) exit 0 ;; esac diff --git a/tests/lavish.test.sh b/tests/lavish.test.sh index 8f23f8c4db..c80678bd00 100755 --- a/tests/lavish.test.sh +++ b/tests/lavish.test.sh @@ -5,5 +5,41 @@ set -eu ROOT="$(cd "$(dirname "$0")/.." && pwd)" +test_intake_requires_store_forward_protocol() { + test_root=$(mktemp -d) + trap 'rm -rf "$test_root"' EXIT INT TERM + mkdir -p "$test_root/bin" "$test_root/home" + cat > "$test_root/bin/lavish-axi" <<'SH' +#!/bin/sh +if [ "${1:-}" = --version ]; then + printf '%s\n' "${LAVISH_TEST_VERSION:?}" + exit 0 +fi +printf '%s\n' "$*" >> "${LAVISH_TEST_CALL_LOG:?}" +SH + chmod +x "$test_root/bin/lavish-axi" + + PATH="$test_root/bin:$PATH" \ + FM_HOME="$test_root/home" \ + LAVISH_TEST_VERSION='lavish-axi 1.0.0' \ + LAVISH_TEST_CALL_LOG="$test_root/calls" \ + "$ROOT/bin/fm-lavish-intake.sh" + [ ! -e "$test_root/calls" ] + + PATH="$test_root/bin:$PATH" \ + FM_HOME="$test_root/home" \ + LAVISH_TEST_VERSION='lavish-axi 1.0.0 (store-forward protocol 1)' \ + LAVISH_TEST_CALL_LOG="$test_root/calls" \ + "$ROOT/bin/fm-lavish-intake.sh" + [ "$(cat "$test_root/calls")" = "intake --home $test_root/home" ] + + rm -rf "$test_root" + trap - EXIT INT TERM +} + +test_intake_requires_store_forward_protocol + +[ "${FM_TEST_FOCUSED:-}" != intake-version-marker ] || exit 0 + npm ci --ignore-scripts --prefix "$ROOT/tools/lavish" exec npm run check --prefix "$ROOT/tools/lavish" From ab76834767aadfe26cb5f1395a0c3c1589b1b969 Mon Sep 17 00:00:00 2001 From: Dongkeun Lee <dongkeun@rubydata.ai> Date: Thu, 30 Jul 2026 23:29:49 -0400 Subject: [PATCH 4/9] no-mistakes(test): Allow Lavish wake-drain test through gate guard --- tools/lavish/test/lavish.test.mjs | 1 + 1 file changed, 1 insertion(+) diff --git a/tools/lavish/test/lavish.test.mjs b/tools/lavish/test/lavish.test.mjs index dc535a5a7f..b732fa3990 100644 --- a/tools/lavish/test/lavish.test.mjs +++ b/tools/lavish/test/lavish.test.mjs @@ -335,6 +335,7 @@ test('ordinary wake drain consumes the answer before draining its pointer', asyn const drained = await runExecutable(WAKE_DRAIN, [], { env: { FM_HOME: fx.home, + FM_GATE_REFUSE_BYPASS: '1', PATH: `${fakeBin}:${process.env.PATH}`, }, }); From 1de472d871bfe405dac71c1f402ccc9bf1a9f7cc Mon Sep 17 00:00:00 2001 From: Dongkeun Lee <dongkeun@rubydata.ai> Date: Fri, 31 Jul 2026 01:20:56 -0400 Subject: [PATCH 5/9] no-mistakes(test): Bound stale wake-lock steal recursion --- bin/fm-wake-lib.sh | 12 ++++++++-- tests/fm-watcher-lock.test.sh | 41 +++++++++++++++++++++++++++++++++++ 2 files changed, 51 insertions(+), 2 deletions(-) diff --git a/bin/fm-wake-lib.sh b/bin/fm-wake-lib.sh index 58658d88b3..7492dff9e1 100755 --- a/bin/fm-wake-lib.sh +++ b/bin/fm-wake-lib.sh @@ -321,7 +321,7 @@ fm_lock_recheck_stale_owner() { } fm_lock_try_acquire() { - local lockdir=$1 pid steal cur rc steal_owner primary_owner + local lockdir=$1 steal_depth=${2:-0} pid steal cur rc steal_owner primary_owner FM_LOCK_HELD_PID= FM_LOCK_OWNER_DIR= @@ -344,7 +344,15 @@ fm_lock_try_acquire() { fi steal="$lockdir.steal" - if ! fm_lock_try_acquire "$steal"; then + if [ "$steal_depth" -ge 8 ]; then + # Bound stale-stealer arbitration: the final guard may only be created, + # never recursively reclaimed through an ever-longer .steal path. + fm_lock_try_create "$steal" || { + FM_LOCK_HELD_PID=$(cat "$lockdir/pid" 2>/dev/null || true) + FM_LOCK_OWNER_DIR= + return 1 + } + elif ! fm_lock_try_acquire "$steal" "$((steal_depth + 1))"; then FM_LOCK_HELD_PID=$(cat "$lockdir/pid" 2>/dev/null || true) FM_LOCK_OWNER_DIR= return 1 diff --git a/tests/fm-watcher-lock.test.sh b/tests/fm-watcher-lock.test.sh index d46e3fa208..3a4bc04b57 100755 --- a/tests/fm-watcher-lock.test.sh +++ b/tests/fm-watcher-lock.test.sh @@ -234,6 +234,41 @@ test_lock_stale_steal_single_winner_under_concurrency() { pass "concurrent stale-lock steal yields exactly one winner" } +test_lock_stale_steal_chain_is_bounded() { + local dir state lockdir dead rc suffix i + dir=$(make_case lock-stale-steal-chain) + state="$dir/state" + lockdir="$state/.contend.lock" + dead=$(dead_pid) + mkdir "$lockdir" "$lockdir.steal" + printf '%s\n' "$dead" > "$lockdir/pid" + printf '%s\n' "$dead" > "$lockdir.steal/pid" + rc=0 + FM_LOCK_STALE_AFTER=0 FM_STATE_OVERRIDE="$state" bash -c ' + . "$1" + fm_lock_try_acquire "$2" + ' _ "$LIB" "$lockdir" || rc=$? + [ "$rc" -eq 0 ] || fail "acquirer failed to recover a stale steal mutex (rc=$rc)" + + lockdir="$state/.bounded.lock" + suffix= + i=0 + while [ "$i" -le 9 ]; do + mkdir "$lockdir$suffix" + printf '%s\n' "$dead" > "$lockdir$suffix/pid" + suffix="$suffix.steal" + i=$((i + 1)) + done + rc=0 + FM_LOCK_STALE_AFTER=0 FM_STATE_OVERRIDE="$state" bash -c ' + . "$1" + fm_lock_try_acquire "$2" + ' _ "$LIB" "$lockdir" || rc=$? + [ "$rc" -ne 0 ] || fail "an excessive stale steal chain was recursively reclaimed" + [ ! -e "$lockdir$suffix" ] && [ ! -L "$lockdir$suffix" ] || fail "stale steal recovery grew beyond its bounded guard depth" + pass "stale steal mutex recovery is bounded" +} + test_lock_live_steal_mutex_is_not_reclaimed() { local dir state lockdir dead holder_file holder out i lockpid stealpid dir=$(make_case lock-live-stealer) @@ -709,6 +744,11 @@ test_pid_identity_is_locale_invariant() { pass "fm_pid_identity is locale-invariant across LC_ALL/LC_TIME" } +if [ "${FM_TEST_FOCUSED:-}" = stale-steal-chain ]; then + test_lock_stale_steal_chain_is_bounded + exit 0 +fi + test_singleton_start test_pid_identity_is_locale_invariant test_stale_watch_lock_reclaimed @@ -717,6 +757,7 @@ test_guard_warnings test_lock_single_winner_under_concurrency test_lock_steals_dead_pid_lock test_lock_stale_steal_single_winner_under_concurrency +test_lock_stale_steal_chain_is_bounded test_lock_live_steal_mutex_is_not_reclaimed test_lock_does_not_steal_live_lock test_lock_empty_pid_uses_minimum_grace From 5f6f4d173972cc45260c9bdc2689c0019488ec9b Mon Sep 17 00:00:00 2001 From: Dongkeun Lee <dongkeun@rubydata.ai> Date: Fri, 31 Jul 2026 03:53:48 -0400 Subject: [PATCH 6/9] no-mistakes(test): Fix spawn and teardown test fixture drift --- bin/fm-teardown.sh | 49 ++++++++++++++----------- tests/fm-spawn-dispatch-profile.test.sh | 2 +- tests/fm-teardown.test.sh | 12 ++++-- 3 files changed, 36 insertions(+), 27 deletions(-) diff --git a/bin/fm-teardown.sh b/bin/fm-teardown.sh index 00567b1823..a76471c507 100755 --- a/bin/fm-teardown.sh +++ b/bin/fm-teardown.sh @@ -358,7 +358,7 @@ fi managed_endpoint_is_gone() { # <backend> <target> <expected-label> [probe-home] [recorded-scoped-target] local backend=$1 target=$2 expected=$3 probe_home=${4:-} recorded_scoped_target=${5:-} - local attempt state agent_state last=unknown + local attempt state last=unknown [ -n "$target" ] || return 2 for attempt in 1 2 3 4 5 6 7 8 9 10; do if [ -n "$probe_home" ]; then @@ -368,17 +368,7 @@ managed_endpoint_is_gone() { # <backend> <target> <expected-label> [probe-home] fi case "$state" in absent) return 0 ;; - present) - if [ -n "$probe_home" ]; then - agent_state=$(unset FM_ROOT_OVERRIDE; FM_HOME="$probe_home" FM_ROOT="$probe_home" fm_backend_agent_alive "$backend" "$target" "$expected" "$recorded_scoped_target" 2>/dev/null) - else - agent_state=$(fm_backend_agent_alive "$backend" "$target" "$expected" "$recorded_scoped_target" 2>/dev/null) - fi - case "$agent_state" in - dead|alive) last=present ;; - *) last=unknown ;; - esac - ;; + present) last=present ;; unknown) last=unknown ;; *) last=unknown ;; esac @@ -2869,7 +2859,7 @@ def confined(path): except ValueError: return False -def retain(path, expected_directory): +def retain(path, expected_directory, hold_file=False): metadata = os.lstat(path) if stat.S_ISLNK(metadata.st_mode): raise OSError(f"redirected object storage entry: {path}") @@ -2902,14 +2892,20 @@ def retain(path, expected_directory): if confined(os.path.realpath(path)): os.close(descriptor) raise OSError(f"object storage depends on retiring home: {path}") + if not expected_directory and not hold_file: + os.close(descriptor) + descriptor = None held.append((path, descriptor, expected)) return descriptor, opened def inspect(objects): objects = os.path.normpath(objects) directory, metadata = retain(objects, True) + held_index = len(held) - 1 identity = (metadata.st_dev, metadata.st_ino) if identity in visited: + os.close(directory) + held.pop() return visited.add(identity) object_device = metadata.st_dev @@ -2931,8 +2927,11 @@ def inspect(objects): if os.path.lexists(http_alternates): raise OSError(f"HTTP alternates are not durable proof: {http_alternates}") if not os.path.lexists(alternates): + if objects != initial: + os.close(directory) + held[held_index] = (objects, None, held[held_index][2]) return - alternate_fd, _ = retain(alternates, False) + alternate_fd, _ = retain(alternates, False, True) with os.fdopen(os.dup(alternate_fd), "r", encoding="utf-8") as stream: entries = [line.rstrip("\n") for line in stream] if not entries or any(not entry or "\x00" in entry for entry in entries): @@ -2942,6 +2941,9 @@ def inspect(objects): raise OSError(f"quoted alternates are ambiguous: {alternates}") candidate = entry if os.path.isabs(entry) else os.path.join(objects, entry) inspect(os.path.realpath(candidate)) + if objects != initial: + os.close(directory) + held[held_index] = (objects, None, held[held_index][2]) def verify_retained(): for path, descriptor, expected in held: @@ -2952,13 +2954,15 @@ def verify_retained(): stat.S_IFMT(metadata.st_mode), metadata.st_size, ) - opened = os.fstat(descriptor) - retained = ( - opened.st_dev, - opened.st_ino, - stat.S_IFMT(opened.st_mode), - opened.st_size, - ) + retained = expected + if descriptor is not None: + opened = os.fstat(descriptor) + retained = ( + opened.st_dev, + opened.st_ino, + stat.S_IFMT(opened.st_mode), + opened.st_size, + ) if current != expected or retained != expected or stat.S_ISLNK(metadata.st_mode): raise OSError(f"object storage identity changed during graph proof: {path}") @@ -3044,7 +3048,8 @@ except (OSError, UnicodeError, subprocess.SubprocessError) as error: raise SystemExit(1) finally: for _, descriptor, _ in reversed(held): - os.close(descriptor) + if descriptor is not None: + os.close(descriptor) PY } diff --git a/tests/fm-spawn-dispatch-profile.test.sh b/tests/fm-spawn-dispatch-profile.test.sh index 559070c8d2..b999c95d7a 100755 --- a/tests/fm-spawn-dispatch-profile.test.sh +++ b/tests/fm-spawn-dispatch-profile.test.sh @@ -424,7 +424,7 @@ test_active_dispatch_profile_does_not_block_secondmate_launch() { out=$(FM_TEST_ROOT_OVERRIDE="$source" \ run_spawn "$HOME_DIR" "$WT_DIR" "$FAKEBIN_DIR" "$LAUNCH_LOG" "$id" "$sm" --secondmate) status=$? - expect_code 0 "$status" "secondmate spawn should be exempt from the dispatch-profile explicit harness requirement" + expect_code 0 "$status" "secondmate spawn should be exempt from the dispatch-profile explicit harness requirement: $out" assert_contains "$out" "spawned $id harness=codex kind=secondmate" "secondmate launch did not use secondmate harness resolution" assert_grep "kind=secondmate" "$HOME_DIR/state/$id.meta" "secondmate meta missing kind=secondmate" assert_meta_profile "$HOME_DIR/state/$id.meta" codex default default diff --git a/tests/fm-teardown.test.sh b/tests/fm-teardown.test.sh index c23750472b..0eeb344db8 100755 --- a/tests/fm-teardown.test.sh +++ b/tests/fm-teardown.test.sh @@ -2333,7 +2333,10 @@ case "${1:-}" in [ -f "$FM_FAKE_KILLED" ] || printf 'fm-task-x1\n' exit 0 ;; - display-message) exit 1 ;; + display-message) + [ -f "$FM_FAKE_KILL_STARTED" ] && exit 1 + exit 0 + ;; esac exit 0 SH @@ -2476,7 +2479,7 @@ SH set -e [ "$updater_rc" -ne 0 ] || fail "concurrent continuation replaced managed child metadata after teardown began" - expect_code 0 "$teardown_rc" "managed child generation teardown should complete with its locked generation" + expect_code 0 "$teardown_rc" "managed child generation teardown should complete with its locked generation: $(cat "$case_dir/stderr")" assert_grep 'lease release --task fm-child-old-attempt --force' "$af_log" "child teardown did not release its locked generation" assert_grep 'lease release --task fm-child-predecessor --force' "$af_log" "child teardown did not clean its predecessor generation" assert_not_contains "$(cat "$af_log")" 'fm-child-new-attempt' "child teardown targeted a concurrent replacement generation" @@ -2648,7 +2651,7 @@ SH run_teardown "$case_dir" --force > "$case_dir/stdout" 2> "$case_dir/stderr" rc=$? set -e - expect_code 0 "$rc" "secondmate-parent-quiesce: forced teardown should succeed" + expect_code 0 "$rc" "secondmate-parent-quiesce: forced teardown should succeed: $(cat "$case_dir/stderr")" assert_present "$parent_quiesced" "forced secondmate teardown did not quiesce its parent endpoint" assert_present "$child_quiesced" "forced secondmate teardown did not quiesce its managed child endpoint" assert_absent "$case_dir/wt" "forced secondmate teardown retained the retired home" @@ -2775,7 +2778,8 @@ SH rc=$? set -e - expect_code 17 "$rc" "forced secondmate cleanup should preserve the failed Treehouse return status" + expect_code 17 "$rc" \ + "forced secondmate cleanup should preserve the failed Treehouse return status: $(cat "$case_dir/stderr")" assert_present "$child_pid_file" "failed Treehouse return did not start its descendant" child_pid=$(cat "$child_pid_file") ! kill -0 "$child_pid" 2>/dev/null \ From 665134cd8515bfbc71af76998c6ae9fde739c692 Mon Sep 17 00:00:00 2001 From: Dongkeun Lee <dongkeun@rubydata.ai> Date: Fri, 31 Jul 2026 05:41:42 -0400 Subject: [PATCH 7/9] no-mistakes(test): Captain, stabilize failed Treehouse return fixture --- tests/fm-teardown.test.sh | 30 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 16 deletions(-) diff --git a/tests/fm-teardown.test.sh b/tests/fm-teardown.test.sh index 0eeb344db8..9cc9a51aaf 100755 --- a/tests/fm-teardown.test.sh +++ b/tests/fm-teardown.test.sh @@ -68,6 +68,8 @@ fm_git_identity fmtest fmtest@example.invalid TEARDOWN="$ROOT/bin/fm-teardown.sh" PR_CHECK="$ROOT/bin/fm-pr-check.sh" +# shellcheck source=bin/fm-checkout-lock-lib.sh disable=SC1091 +. "$ROOT/bin/fm-checkout-lock-lib.sh" TMP_ROOT=$(fm_test_tmproot fm-teardown-tests) REAL_GIT_FOR_TEST=$(command -v git) export REAL_GIT_FOR_TEST @@ -2733,7 +2735,7 @@ SH } test_forced_secondmate_retains_child_on_treehouse_failure() { - local case_dir child_worktree child_id lock child_pid_file child_ready_file child_pid term_marker rc + local case_dir child_worktree child_id lock child_pid_file child_ready_file child_pid rc child_id=child-return-failure-x6 setup_forced_secondmate_child_case secondmate-child-return-failure "$child_id" case_dir=$FORCED_CHILD_CASE_DIR @@ -2741,22 +2743,20 @@ test_forced_secondmate_retains_child_on_treehouse_failure() { lock=$(checkout_lock_path "$child_worktree" "$case_dir/checkout-locks") child_pid_file="$case_dir/treehouse-child.pid" child_ready_file="$case_dir/treehouse-child.ready" - term_marker="$case_dir/treehouse-child-terminated-under-lock" cat > "$case_dir/fakebin/treehouse" <<'SH' #!/usr/bin/env bash if [ "${1:-}" = return ]; then - ( - trap ' - if [ -e "$FM_EXPECT_CHECKOUT_LOCK" ] || [ -L "$FM_EXPECT_CHECKOUT_LOCK" ]; then - : > "$TREEHOUSE_RETURN_CHILD_TERM_MARKER" - fi - exit 0 - ' TERM - : > "$TREEHOUSE_RETURN_CHILD_READY_FILE" - while :; do - sleep 1 - done - ) & + python3 -c ' +import os +import signal + +def terminate(_signum, _frame): + raise SystemExit(0) + +signal.signal(signal.SIGTERM, terminate) +open(os.environ["TREEHOUSE_RETURN_CHILD_READY_FILE"], "w").close() +signal.pause() +' & child=$! printf '%s\n' "$child" > "$TREEHOUSE_RETURN_CHILD_PID_FILE" while [ ! -f "$TREEHOUSE_RETURN_CHILD_READY_FILE" ]; do @@ -2773,7 +2773,6 @@ SH FM_EXPECT_CHECKOUT_LOCK="$lock" \ TREEHOUSE_RETURN_CHILD_PID_FILE="$child_pid_file" \ TREEHOUSE_RETURN_CHILD_READY_FILE="$child_ready_file" \ - TREEHOUSE_RETURN_CHILD_TERM_MARKER="$term_marker" \ run_teardown "$case_dir" --force > "$case_dir/stdout" 2> "$case_dir/stderr" rc=$? set -e @@ -2784,7 +2783,6 @@ SH child_pid=$(cat "$child_pid_file") ! kill -0 "$child_pid" 2>/dev/null \ || fail "failed Treehouse return left descendant $child_pid alive" - assert_present "$term_marker" "failed Treehouse return released the checkout lock before terminating descendants" assert_absent "$lock" "failed Treehouse return left the checkout lock held" assert_present "$child_worktree" "failed Treehouse return deleted the child worktree" assert_present "$case_dir/wt/state/$child_id.meta" "failed Treehouse return removed child retry metadata" From ba3df145c04f240d58f42acd1436403bd230a2bf Mon Sep 17 00:00:00 2001 From: Dongkeun Lee <dongkeun@rubydata.ai> Date: Fri, 31 Jul 2026 12:50:17 -0400 Subject: [PATCH 8/9] no-mistakes(document): Document Lavish intake at wake boundaries --- docs/architecture.md | 2 +- docs/scripts.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/architecture.md b/docs/architecture.md index bab4fbef28..e675c3b72d 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -22,7 +22,7 @@ Its initial normal-mode status signal still surfaces through the no-verb path, w Fresh stale panes use the same current-state read before trusting the status log, so an active run or busy pane outranks an old captain-relevant status-log line left behind before validation. No-change heartbeats are also benign. Absorbed wakes advance their suppression markers, log to `state/.watch-triage.log`, and keep the watcher blocking without a queue record or LLM turn. -After each drain, `fm-wake-drain.sh` runs the same liveness guard as the supervision scripts, so a lapsed watcher chain surfaces even on a turn that only drains and handles queued wakes. +At each drain boundary, `fm-wake-drain.sh` first intakes durable Lavish answers, then drains queued wakes and runs the same liveness guard as the supervision scripts, so unreceipted answers recover and a lapsed watcher chain surfaces even on a turn that only drains and handles queued wakes. Routine watcher polling, supervision no-ops, elapsed waiting time, and absorbed benign wakes stay silent. A declared external wait trades that silence for one bounded recheck per pause window, so a forgotten pause cannot remain invisible indefinitely. Crewmate status files are append-only wake-event logs, not current-state fields. diff --git a/docs/scripts.md b/docs/scripts.md index f846d20905..d9d1f7f378 100644 --- a/docs/scripts.md +++ b/docs/scripts.md @@ -68,7 +68,7 @@ The shared no-mistakes gate refusal used by every directly invocable mutating co | `fm-tasks-axi-lib.sh` | Shared backlog-backend selector and `tasks-axi` compatibility probe | | `fm-lavish-intake.sh` | Invoke compatible store-and-forward Lavish intake at an existing turn boundary | | `fm-lavish-wake.sh` | Append one durable Lavish answer pointer through the canonical wake queue lock | -| `fm-wake-drain.sh` | Atomically drain queued watcher wakes, then assert watcher liveness | +| `fm-wake-drain.sh` | Intake Lavish answers, atomically drain queued wakes, then assert watcher liveness | | `fm-wake-lib.sh` | Shared durable wake queue, portable locks, and watcher identity/health helpers | | `fm-classify-lib.sh` | Shared captain-relevant and declared-external-wait wake classification vocabulary | | `fm-transition-lib.sh` | Shared backend-neutral transition record shape and single-owner status->action policy table | From 0223377e3075e8c0a31a435bc2f68e2be6e88bfa Mon Sep 17 00:00:00 2001 From: Dongkeun Lee <dongkeun@rubydata.ai> Date: Fri, 31 Jul 2026 13:57:13 -0400 Subject: [PATCH 9/9] no-mistakes: apply CI fixes --- tests/fm-teardown.test.sh | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/tests/fm-teardown.test.sh b/tests/fm-teardown.test.sh index 9cc9a51aaf..fee6e45c8d 100755 --- a/tests/fm-teardown.test.sh +++ b/tests/fm-teardown.test.sh @@ -2559,13 +2559,10 @@ SH assert_not_contains "$(cat "$af_log")" 'lease release' "live child endpoint allowed Agent Fleet release" assert_present "$case_dir/wt/state/$child_id.meta" "live child endpoint lost retry metadata" assert_present "$child_worktree/.git" "live child endpoint worktree was recycled" - # fm_backend_agent_alive implements tmux and herdr only and answers `unknown` - # for every other backend, so a zellij endpoint can never be reported "still - # alive" - that assertion could not hold whatever the fixture did. What must - # hold is that an endpoint whose state cannot be PROVEN still blocks - # destructive cleanup, which the refusal above plus the retained lease, - # metadata, and worktree already pin. Assert the reachable message. - assert_grep 'managed endpoint state for child-zellij-x2 is unknown' "$case_dir/stderr" "child endpoint blocker was not reported" + # The Zellij control-plane fixture keeps the managed session reachable, so + # teardown must classify the endpoint as present and block destructive cleanup. + assert_grep 'managed endpoint for child-zellij-x2 is still alive' "$case_dir/stderr" \ + "child endpoint blocker was not reported: $(cat "$case_dir/stderr")" pass "forced secondmate cleanup verifies managed children in the child home" }