UX v2 · Temperature + Operations/Tactics + Operate (bottom-cam→SPIM) suite

.gitignore

@@ -146,5 +146,12 @@ electron/
 gently/ui/tui/node_modules/
 gently/ui/tui/dist/
 
-# Runtime storage accidentally created on Linux when GENTLY_STORAGE_PATH="D:/" resolves literally
-D:/
+# Stray local storage: GENTLY_STORAGE_PATH default (D:\Gently3) resolves
+# literally to ./D:/ under the repo on Linux. Not data we track.
+/D:/
+.superpowers/
+
+# Runtime operator overrides (written by the Settings panel)
+config/config.local.yml
+config/settings.local.yml
+config/dashboard_defaults.json

README.md

@@ -175,6 +175,9 @@ uv run python launch_gently.py --no-api
 # Don't auto-open a browser — open the printed URL yourself
 uv run python launch_gently.py --no-browser
 
+# Skip login — disable accounts (localhost-control mode; same as GENTLY_NO_AUTH=1)
+uv run python launch_gently.py --no-auth
+
 # Resume a previous session
 uv run python launch_gently.py --resume            # interactive picker
 uv run python launch_gently.py --resume latest     # most recent session
@@ -211,8 +214,9 @@ First-run admin account created — sign in at the URL above:
 - **Lost it?** There's no reset command yet — delete
   `<GENTLY_STORAGE_PATH>/auth/users.yaml` and restart to bootstrap a fresh
   `admin` (this clears all accounts).
-- **Just trying it locally?** `GENTLY_NO_AUTH=1` disables accounts entirely
-  (legacy mode: localhost gets control, remote callers need `X-Gently-Token`).
+- **Just trying it locally?** Launch with `--no-auth` (or set `GENTLY_NO_AUTH=1`)
+  to disable accounts entirely (localhost gets control, remote callers need
+  `X-Gently-Token`). Handy if you've lost the admin password.
 
 Accounts live under `<GENTLY_STORAGE_PATH>/auth/` (`users.yaml` + `secret.key`),
 outside the repo.

docs/ARCHITECTURE.md

@@ -0,0 +1,171 @@
+# Gently — Concurrency & Runtime Architecture
+
+How the system runs temperature telemetry, device-state polling, experiments,
+image acquisition, and perception/VLM "at the same time" without stepping on
+itself. The short version: **almost nothing runs truly in parallel — the design
+quarantines the blocking work and serializes the hardware, then keeps both event
+loops responsive by offloading and decoupling everything slow.**
+
+> Scope: the concurrency model. For storage layout see `CLAUDE.md`; for the
+> device/hardware plugin model see `docs/asi-plugin-architecture.md`.
+
+## 1. Two processes, bridged by HTTP + a shared filesystem
+
+This split is the load-bearing fact of the whole design.
+
+```
+┌────────────────────────────────────────┐        ┌────────────────────────────────────────┐
+│  APP / VIZ process   (FastAPI :8080)    │        │  DEVICE-LAYER process  (aiohttp :60610) │
+│  gently/app/agent.py                    │        │  gently/hardware/dispim/device_layer.py │
+│                                         │  HTTP  │                                          │
+│  • agent + TimelapseOrchestrator        │◄──────►│  • the ONLY code touching hardware:      │
+│  • Perceiver (gently_perception, VLM)   │  (DiSPIM│    MMCore/pymmcore, Ophyd, Bluesky RE    │
+│  • AsyncAnthropic clients               │  Client)│  • plan queue + single executor          │
+│  • EventBus + WebSocket ConnectionMgr   │        │  • 3 state pollers + camera/LS streamers │
+│  • TemperatureSampler, DeviceStateMonitor│       │                                          │
+└──────────────┬──────────────────────────┘        └───────────────┬──────────────────────────┘
+               │                     shared filesystem (incoming/ TIFF staging, session store)
+               └──────────────────────────────────────────────────┘
+```
+
+- The **device-layer process** (`device_layer.py`) is the *only* code that ever
+  touches the microscope: MMCore/pymmcore, the Ophyd devices, and a Bluesky
+  `RunEngine`. All truly blocking hardware work is quarantined here.
+- The **app/viz process** (`app/agent.py`, FastAPI+uvicorn) owns everything
+  cognitive and user-facing: the agent, the `TimelapseOrchestrator`, the
+  `Perceiver` (external `gently_perception` VLM package), the Anthropic clients,
+  the in-process `EventBus`, the WebSocket `ConnectionManager`, and the pollers.
+- The app reaches hardware **only** through `DiSPIMClient` (`client.py`), over a
+  single shared `aiohttp.ClientSession`. It never imports MMCore.
+
+Because blocking hardware calls live in a separate process, the app-side loop
+stays a light cooperative-asyncio world.
+
+## 2. One asyncio event loop per process
+
+- **Device loop:** HTTP routes + the RunEngine driver + the plan queue/executor +
+  three state pollers (XY ~5 Hz, piezo/galvo ~1 Hz, full property cache ~15 s) +
+  subscriber-gated camera/lightsheet SSE streamers.
+- **App loop:** FastAPI + the `TimelapseOrchestrator` acquisition loop +
+  `TemperatureSampler` (1 Hz) + `DeviceStateMonitor` + `Perceiver` calls + one
+  coroutine per open browser WebSocket.
+
+## 3. The core trick: hardware is serialized, not parallelized
+
+Every Bluesky plan — a move, a snap, a volume/burst acquisition, a focus/calibration
+sweep — is submitted as a `PlanRequest` onto **one `asyncio.Queue`
+(`self._plan_queue`), drained by one `_plan_executor` task**. Only one plan owns
+the hardware at a time.
+
+- `submit_plan` enqueues and `await`s a per-request `asyncio.Future`; the executor
+  sets the result/exception, which flows back to the awaiting HTTP handler. On
+  failure the executor records it in `_plan_execution_log` and continues to the
+  next queued plan.
+- Underneath, **pymmcore's internal `g_core_lock` is the real mutex** serializing
+  every actual core call across pollers *and* plans. The `DiSPIMSystem` facade is
+  the single place the process touches MMCore.
+
+A microscope has one stage and one camera; single-file execution is correct, not
+a limitation.
+
+## 4. How constant polling coexists with long experiments
+
+Four cooperating mechanisms keep the loops responsive while a plan runs:
+
+1. **Offload blocking reads.** Every MMCore read, camera grab, SAM call, and
+   transient temperature probe goes through `asyncio.to_thread`; the camera
+   sequence runs on its own `threading.Thread` returning an Ophyd `Status`. The
+   loop itself never sits on I/O.
+2. **Split pollers by cadence.** The three device-state pollers are independent
+   tasks so a slow (~1.5 s) full-state-cache read cannot stall the ~5 Hz XY path.
+3. **`pause_state_updates()` — a reference-counted async context manager.** Heavy
+   plans (a `frozenset` of names) wrap execution in it, incrementing a counter;
+   every poller/streamer checks `if self._state_pause_counter > 0` and *skips its
+   MMCore read*, emitting only ~2 s heartbeats. The plan gets full serial/camera
+   bandwidth instead of the pollers fighting it for `g_core_lock`. Nested heavy
+   sections stack and unwind cleanly.
+4. **Telemetry bypasses the plan queue.** `GET /api/temperature/status` reads the
+   temperature Ophyd device directly (`temp.read()` over serial/MQTT — a device
+   wholly separate from MMCore); `GET /api/devices/state` returns the cached
+   `_state_latest` snapshot. Neither sits behind a running experiment, so status
+   polls are never blocked by a long acquisition. (MMCore push callbacks also
+   mirror joystick/property changes into `_state_latest` via
+   `loop.call_soon_threadsafe` with ~50 ms debouncing.)
+
+### Temperature specifically
+The vendor SDK backend (serial or MQTT) runs its own **background daemon thread**
+that ingests the controller's 500 ms telemetry broadcast into a cache
+(`self.telemetry`); `get_water_temp()`/`get_system_state()` return the cached
+value (non-blocking). `TemperatureController.read()` returns that cache, and
+`TemperatureSampler` (`interval_sec=1.0`) polls it at 1 Hz → persists + emits
+`TEMPERATURE_UPDATE`. So we ride the telemetry indirectly, resampled at 1 Hz.
+
+## 5. Image data travels via the filesystem, not JSON
+
+Arrays over ~1 MB are written as **TIFF into the shared `incoming/` staging dir**;
+only a small `{__file_ref__, path, shape, dtype}` dict crosses HTTP (`serialize_value`).
+The client resolves the ref with `tifffile` and hands the decoded array to
+`register_volume`, which renames the file into the session store and stamps it
+with the latest temperature sample — avoiding a multi-GB JSON blob and a double
+decode.
+
+## 6. Perception / VLM / events are decoupled (fire-and-forget)
+
+- **Perception never gates acquisition.** The `TimelapseOrchestrator` does
+  `asyncio.create_task(self._run_perception(...))` rather than awaiting it inline,
+  so a slow VLM call doesn't hold up the next embryo. Inside the task the
+  `Perceiver` and the Claude client are awaited cooperatively, and every Claude
+  call is wrapped in `asyncio.wait_for(timeout=30)` returning a safe fallback
+  instead of raising into the loop.
+- **The `EventBus` publishes without awaiting handlers** (`event_bus.py`): sync
+  handlers run inline, async handlers are scheduled via `asyncio.ensure_future` /
+  `loop.call_soon_threadsafe`; high-volume telemetry types skip the bounded
+  history deque, so one slow WebSocket client delays only its own broadcast.
+
+## 7. Backpressure & failure isolation
+
+- **SSE streams** use per-subscriber queues bounded at `maxsize=4`; device-state
+  broadcasts drop slow subscribers, camera/lightsheet streams drop the *oldest*
+  frame and push the newest so steady clients keep fresh frames.
+- **WebSocket fan-out** (`ConnectionManager.broadcast`) sends per client under an
+  `asyncio.Lock` and drops clients that error.
+- **Failure = a gap, not a crash.** A failed temperature poll logs once and backs
+  off (`1.0 s → min(interval*10, 30 s)`); a stalled SSE forces a watchdog
+  reconnect after 60 s (chosen to tolerate the quiet windows during heavy plans);
+  volume acquisition always disables lasers on error to protect the sample.
+- **Config safety.** `POST /api/temperature/config` refuses (`409`) while the
+  RunEngine is not idle or a ramp holds the controller lock; `health_check()` is
+  read-only and never flips the connected flag, so a transient status-poll timeout
+  can't disconnect an in-flight acquisition.
+
+## 8. Known limits & bottlenecks
+
+- **RunEngine on the loop.** `self.RE(plan)` is invoked synchronously on the
+  device-layer loop, so while a plan runs the loop is largely occupied — which is
+  exactly why heavy plans quiet the pollers and the app-side watchdog tolerates
+  60 s of silence. Whether the telemetry HTTP handlers stay fully responsive
+  mid-acquisition depends on Bluesky's internal threading (external package) and
+  is not verified from repo code.
+- **Unbounded in-flight perception.** `_perception_tasks` is a plain set that
+  self-prunes; sustained fast acquisition against a slow VLM could grow concurrent
+  Claude calls without an explicit cap. A semaphore is the obvious guard if
+  cadence is ever pushed.
+- **~~Synchronous O(n) prediction writes~~ — FIXED.** `store_prediction` used to
+  re-parse the entire `predictions.jsonl` on every append to compute the next id.
+  Now O(1) via a bounded tail read (`_last_jsonl_record`, `file_store.py`). Other
+  `FileStore` JSONL writes remain synchronous on the app loop but are single-line
+  appends.
+- **External VLM internals.** How `gently_perception`'s `Perceiver` implements its
+  VLM call (async httpx vs sync-in-thread) is not inspectable from this repo; the
+  orchestrator awaits it, implying a coroutine.
+
+## The model in one line
+
+Two single-threaded event loops; hardware fenced into one process behind a
+one-at-a-time plan queue; everything blocking pushed into threads; and the
+slow/cognitive work (VLM, persistence, UI) decoupled with fire-and-forget tasks —
+so the system *feels* concurrent while the microscope itself stays strictly
+serialized.
+
+---
+*Generated from a code-grounded architecture pass (Claude Opus 4.8), 2026-07-01.*

docs/HEURISTICS-AUDIT.md

@@ -0,0 +1,112 @@
+# Heuristics audit — where to use the model (as a typed-output function) instead
+
+Codebase sweep (5 parallel scanners + synthesis) for heuristics that **fake
+judgment** an LLM would do better — in the spirit of the genotype→channel
+refactor (drop the lookup table, let the model infer, keep a typed provenance
+record + confirm-when-unsure). The flip side — logic that **must stay
+deterministic** (safety, math, calibration, transport) — is listed at the end so
+we don't mistakenly LLM-ify it.
+
+The unifying move for every candidate: **LLM with a typed structured-output
+schema + provenance + a confirm/UNCERTAIN escape**, never free-text-then-parse.
+
+## Model candidates (ranked)
+
+### High value
+
+1. **Hatching / time-to-stage prediction** — `organisms/celegans/developmental_tracker.py`
+   *(the closest twin of genotype→channel; medium effort)*
+   Three hardcoded 20 °C lookup tables (`STAGE_TIMING_20C`, `TIME_TO_HATCHING`,
+   `TIMING_VARIABILITY`) plus magic `{HIGH:1.0, MEDIUM:1.5, LOW:2.0}` uncertainty
+   fudge factors. Structurally **can't use the rig's actual temperature** (we run
+   a TEC), the strain, or the embryo's observed progression rate. Let the model
+   produce a calibrated, explained interval; **keep the literature table as a
+   deterministic sanity bracket** and flag when the estimate falls outside it.
+   → `{ predicted_minutes_to_hatching, low, high, basis, assumptions{temperature_c,strain,used_observed_rate}, confidence, reasoning }`
+
+2. **Citation → PubMed query** — `harness/plan_mode/tools/research.py` (`_search_pmid`)
+   A regex that only handles "Surname et al YEAR …" + six hand-rolled query-
+   relaxation strategies + a stopword/word-position ladder that drops load-bearing
+   nouns. The model parses the sloppy citation and proposes relaxed queries; **code
+   keeps the deterministic esearch call and never fabricates a PMID.**
+   → `{ author_last, year, journal, topic_keywords[], organism, pubmed_query, alt_queries[], confidence }`
+
+3. **Lab-history retrieval** — `harness/plan_mode/tools/lab_context.py`, `harness/memory/interface.py`
+   Semantic recall faked by substring-OR over query tokens (matches "we"/"before",
+   misses every paraphrase). Feed the model the candidate records and have it
+   **rank/select from provided ids only** (no fabrication). Read-only, no
+   acquisition risk.
+   → `{ matches:[{kind,id,summary,relevance,why_relevant}], answer }`
+
+4. **Stage-label parse via 22-entry synonym dict** — `developmental_tracker.py` (`_parse_stage_name`)
+   *(small effort, pure robustness win)* The Vision call already classifies; the
+   brittleness is a plain-text `STAGE:/CONFIDENCE:` block scraped line-by-line, with
+   off-vocabulary phrasings silently collapsing to `UNKNOWN` (which kills the
+   downstream hatching prediction). Constrained-enum structured output deletes the
+   parser + synonym table.
+   → `{ stage: enum(...), confidence: enum(high|medium|low), is_transitional, reasoning }`
+
+### Medium value (mostly small — fix the output contract, not the judgment)
+
+5. **Calibration Vision calls** — `hardware/dispim/claude_client.py`
+   Four Vision calls return positional free text recovered by `'yes' in first_line`
+   / `re.search(r'\d+')` / first-valid-letter, with silent defaults (so "no, this is
+   not yes…" reads as *yes*). Typed output deletes the parse + silent-default layer.
+
+6. **ML architecture ranking** — `ml/architectures.py` (`get_suitable_architectures`)
+   Hard feasibility gates (VRAM / dataset) are correct **and stay**; the `+2/+1/+1`
+   point-score ranking that follows discards the per-arch prose. Let the model rank
+   the *pre-filtered feasible set* (ids constrained to that set).
+
+7. **Training label normalization** — `ml/data_loader.py` (`build_labels_from_store`)
+   Class space built by exact-string identity over free-text human annotations —
+   "1.5-fold" and "1.5 fold" become different classes. Model normalizes to the
+   canonical staging vocabulary, flags novel/ambiguous ones.
+
+### Lower value
+
+8. **"Plan has a control?"** — `plan_mode/tools/validation.py` — substring scan of a
+   6-word keyword set; a scientific judgment over the whole plan. Non-blocking
+   warning → safe for the model.
+9. **CGC HTML scraping** — `research.py` (`_cgc_search`) — positional multi-group
+   regex over fetched HTML; structured extraction the model does better (HTTP GET
+   stays code; **mark strain names low-confidence to avoid sending someone to order
+   a hallucinated strain**).
+
+### Cross-cutting batch (small each): typed output for the detector/verifier cluster
+`harness/detection/verifier.py`, `app/detectors/hatching.py`,
+`app/detectors/dopaminergic_signal.py`, `hardware/dispim/sam_detection.py` — all
+already make the right model call but reconstruct the verdict via
+`startswith`/regex-JSON-scraping with silent defaults. A batch move to native
+structured output **strictly reduces parse-induced false negatives** without
+touching the deterministic vote-tally/consensus/enum-dispatch downstream.
+
+**Reference implementations already in the repo (imitate, don't change):**
+`dopaminergic_signal`'s perceiver→classifier rubric (typed enums, UNCERTAIN
+escape, conservative-on-tie) and onboarding's `_extract_with_llm` (typed
+extraction, degrade-to-verbatim fallback).
+
+## Keep deterministic (do NOT LLM-ify)
+Safety, math, calibration, and transport — where a hallucinated value is unsafe
+or breaks reproducibility:
+- Laser-power safety limits + wavelength→MM-property map (`hardware/dispim/devices/optical.py`)
+- SPIM trigger-timing arithmetic, piezo–galvo calibration, MM framing (`dispim/config.py`)
+- Calibration prior EMA + R²≥0.75 slope-lock gate (`dispim/calibration.py`)
+- SwitchBot GATT byte commands / status decoding (`hardware/switchbot.py`)
+- Temperature setpoint bound [0,99.9] °C + stabilization I/O (`hardware/temperature.py`)
+- Autofocus signal-processing, curve fitting, adaptive-sweep stop rules (`analysis/core.py`, `analysis/focus.py`)
+- Classical-CV ROI detection + pixel→stage coordinate transforms (`detection.py`, `sam_detection.py` geometry)
+- Timelapse rule dispatch + `confirm_timepoints` debounce + monotonic power ramp (`app/orchestration/timelapse.py`)
+- Volume→b64 dark/flat calibration + fixed brightness scaling (`dopaminergic_signal._volume_to_b64` — deliberately non-adaptive)
+- Wake-router debounce/throttle/stage-transition gate (`app/wake_router.py`)
+- Plan hardware limits, detector-preset membership, dependency-cycle DFS, stage-order normalization (`plan_mode/tools/validation.py`)
+- Ensemble vote tally + 0.70 quorum / unanimity consensus (`detection/verifier.py`)
+- ML metric/aggregation math: confusion matrix, F1, federated averaging (`ml/evaluation.py`, `federated.py`)
+- Core imaging geometry (max-projection, crop bounds, Euler rotations) + UI event reduction/routing/security (`core/imaging.py`, `ui/web/*`)
+- Device-state SSE watchdog/staleness timers (`app/device_state_monitor.py`)
+- Reference-type dispatch (PMID/DOI/URL by canonical syntax), `os.path.isfile` checks (`research.py`)
+
+## Note
+`gap_assessment.conversation_weight` (the 0.25/0.1/0.05 readiness scalar) is now
+largely **vestigial** — it only returns 'heavy' (lab onboarding) or 'none' — so
+it's not worth an API call. Left off the candidate list.