v4.9.0 — non-Bash error detection, verified scrubbing, restored diagnostics

.github/workflows/tests.yml

@@ -67,6 +67,10 @@ jobs:
         shell: bash
         run: bash tests/test-crossos-registry.sh
 
+      - name: Run error detection tests
+        shell: bash
+        run: bash tests/test-error-detection.sh
+
       - name: Run v4.5 Opus 4.7 tests
         shell: bash
         run: bash tests/test-v45-opus47.sh

CHANGELOG.md

@@ -1,16 +1,27 @@
 # Changelog
 
-## Unreleased
+## v4.9.0 (2026-07-25)
 
 ### Added
 
 - **Cross-OS project de-dup in the registry** (`_session-learner.sh`): the per-project id is `sha256(remote || root)`, so a project with **no git remote** gets a different id on each OS (its root differs — `/Users/me/Proj` vs `C:/Users/Me/Proj`). When the registry is shared between machines (e.g. a synced folder), the same project surfaced as two entries — duplicating it in `/projects`, `/eod` and cross-project instinct search. The learner now matches on a cross-OS-stable key (the git remote when present, otherwise the project name), so the two sightings collapse into one entry: each per-OS id is preserved in `aliases[]`, and each observed root is stored under `roots.{posix,windows}`. The name key is **restricted to the genuine cross-OS scenario**: it only matches an entry whose root belongs to a different OS family and that has no root of the candidate's family recorded, so two same-named projects on the same machine (`~/work/app` vs `~/personal/app`) always stay separate. Projects **with** a remote already shared one id and are unaffected; the remote key takes precedence over the name. Purely additive — the new `aliases`, `roots` and `crossKey` fields are optional and ignored by readers that don't use them.
 - **Migration pass for pre-existing duplicates**: registries that already contain both per-OS entries (written by pre-feature learners and synced since) are cured on the next Stop event — entries with the same cross-OS key merge under the same family guard as the lookup, with union of `aliases`/`roots`, the oldest `created` and the newest `last_seen`; `crossKey` is backfilled for legacy entries.
 - **Alias-aware consumers**: `_eod-gather.sh` resolves observations recorded under an aliased per-OS id to the merged project, and `/projects` counts observations from each alias's `homunculus/projects/{alias}/` directory.
+- **Name-key fusions are stamped and logged.** The name fallback cannot rule out two *unrelated* projects that merely share a folder name (`app`, `api`, `docs`) seen from a Windows and a posix machine — with no remote there is nothing left to tell them apart. Guarding it away would cost more than it saves: requiring a matching parent directory breaks the legitimate case, where one project lives at a different path on each OS. So the limit is accepted and made visible instead — every name-based fusion sets `name_merged` on the entry, records the offending root pair in `name_merge_log`, and appends a line to `_session-learner.log`. A wrong fusion becomes an auditable event rather than silent registry drift.
+
+### Fixed
+
+- **Session-learner false positives eliminated** (#29, @juanparisma): `observe_v3.py` set `is_error` whenever the output contained the substring `error`/`failed` — *including the content of files read successfully*, so a `Read` of source code containing `throw new Error` was recorded as a tool failure. One real session produced 190 proposals of which 189 were junk. Detection is now (a) structural — harness-reported failure shapes, every tool; (b) hard failure markers over execution output (`Bash`, `PowerShell`), which is a verdict rather than arbitrary content; (c) hard markers over bare, short output from other tools. `user_correction` no longer counts files re-edited by design (journals, control panels, daily summaries), and `workflow_chain` requires 5+ repetitions with at least two distinct tools and one non-generic tool instead of proposing every `Bash>Bash>Bash` trigram. Measured over a real 8,000-observation window: 1,074 flagged observations drop to 22, `error_resolution` proposals from 11 to 3, `workflow_chain` from 507 to 51.
+- **Genuine non-Bash tool failures are no longer discarded.** Restricting hard markers to `Bash` left real `Edit` and `Read` failures (`String to replace not found`, `File does not exist`) flagged by nobody — they are not harness-shaped, so the structural branch missed them too. Detection for these tools keys on *shape*: a failure returns a bare verdict string, a success returns a payload that serialises starting with `{` or `[`, and payloads are never scanned for marker words — that scan is the very bug this release fixes. Verified against 7,931 real non-Bash observations: zero false positives. `isRealError()` in the learner, which re-validates flags written by older observers still sitting in the observation window, stays deliberately tool-agnostic so PowerShell and MCP failures survive re-validation.
+- **The session learner no longer discards its own diagnostics.** The node block ended in `2>/dev/null`, swallowing every message it emits — registry lock contention, failed upserts — and overriding the `SINAPSIS_DEBUG=1` redirect, so the debug switch produced nothing for the 95% of the script that runs inside that block. Diagnostics now append to `_session-learner.log`.
+- **The cross-OS test suite could not run under Git Bash.** `MSYS_NO_PATHCONV=1` is needed suite-wide so posix fixtures survive argv conversion, but `newhome()` still returned a posix `mktemp -d` path, and node — a native Windows binary — resolves `/tmp/...` to a nonexistent `C:	mp...`. Every fixture write died with `ENOENT`: 0 of 10 tests actually ran on Windows while reporting green elsewhere. The suite now hands node a native path.
 
 ### Tests
 
 - New `tests/test-crossos-registry.sh` — 10 hermetic tests (real hook driven via a sandbox `HOME`, `MSYS_NO_PATHCONV` exported for Git Bash): two-OS no-remote sighting collapses to one entry, both `roots.{posix,windows}` recorded, the second per-OS id registered as an alias, remote key beats name for same-named projects, idempotency on re-run, same-named same-family projects stay separate (anti-regression, both roots intact), pre-existing duplicate migration (merge + union semantics), and Git Bash `/c/` roots classifying as the windows family. Registered in `.github/workflows/tests.yml`. Existing suites re-run clean: `test-install-upgrade` 21/21, `test-registry-isolation`, `test-eod-gather`, `test-bom-decay-python`, `test-v45-opus47`, `test-v46-opus48`.
+- **Secret scrubbing is verified for the first time.** Test Group 4 of `test-security.sh` called `importlib.exec_module()` on `observe_v3.py` and then looked for a function named `scrub_secrets`. Both were wrong — the observer is a script (importing it runs `main()` and blocks on stdin) and the real function is `scrub`, nested inside `main` and unreachable from outside. Every assertion returned `LOAD_FAIL` and was reported as a SKIP, so a headline of "11/11 passed" covered scrubbing that had never been exercised. The group now drives the hook through the front door — a `PostToolUse` payload carrying a real secret — and asserts the secret is absent from what reached disk, across JWT, GitHub, AWS, Stripe and Slack formats, plus a control asserting non-secret output survives so an over-broad scrubber cannot pass by destroying everything.
+- New `tests/test-error-detection.sh` (10 tests): successful `Read`/`Bash`/`Grep` whose content mentions errors stay unflagged, real `Bash`/`Edit`/`Read` failures are flagged, long payloads carrying a marker inside their content stay clean, and `isRealError()` remains tool-agnostic. Note for anyone extending it: the observer resolves its config dir with `expanduser("~")`, which on Windows reads `USERPROFILE`, not `HOME` — both must be redirected or the test writes into real learning data.
+- Suite: **217 tests across 16 files**, green on ubuntu, macos and windows.
 
 ---
 

README.md

@@ -1,7 +1,7 @@
-# Sinapsis v4.8
+# Sinapsis v4.9
 
-[![Version](https://img.shields.io/badge/version-4.8.1-blue.svg)](https://github.com/Luispitik/sinapsis)
-[![Tests](https://img.shields.io/badge/tests-112%20passing-green.svg)](tests/)
+[![Version](https://img.shields.io/badge/version-4.9.0-blue.svg)](https://github.com/Luispitik/sinapsis)
+[![Tests](https://img.shields.io/badge/tests-217%20passing-green.svg)](tests/)
 [![CI](https://github.com/Luispitik/sinapsis/actions/workflows/tests.yml/badge.svg)](https://github.com/Luispitik/sinapsis/actions)
 [![License: MIT](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
 
@@ -28,6 +28,14 @@ Think of it as going from a dumb terminal to an assistant that actually knows yo
 
 ---
 
+## What's New in v4.9.0 — Cleaner signal, one project per project
+
+**The learner stopped inventing work.** `observe_v3.py` used to flag a tool call as failed whenever its output contained the word `error` — including the *content of files it had just read successfully*. A `Read` of source code with `throw new Error` looked exactly like a crash. One real session produced 190 proposals of which 189 were noise. Detection is now structural (harness-reported failure shapes), plus hard failure markers over execution output, plus a shape-and-length check for other tools, so a genuine `Edit`/`Read` failure is caught while a payload that merely *mentions* an error is not. On a real 8,000-observation window, flagged observations dropped from 1,074 to 22. Thanks to [@juanparisma](https://github.com/juanparisma) (#29).
+
+**One project, one entry.** A project with no git remote used to get a different id on every machine, because the id derives from its root and roots differ by OS. On a shared registry it showed up twice, forever. Entries now carry a cross-OS-stable key, keep every per-OS id as an alias, and record both roots — and a migration pass heals registries that already drifted. Where the match rests on the folder name alone it can still be wrong, so it stamps itself and logs, instead of merging quietly. Thanks to [@Sergio-LPA](https://github.com/Sergio-LPA) (#22).
+
+**Secret scrubbing is actually tested now.** The assertions that were supposed to prove it had been silently reporting SKIP since they were written — they tried to import a script that cannot be imported, then looked for a function that does not exist. They now drive the real hook with a real secret and check what reached disk.
+
 ## What's New in v4.8.0 — Plexus moves to the team edition
 
 Sinapsis returns to what it has been since the v4.3.2 separation: **individual autonomous

commands/dashboard-sinapsis.md

@@ -23,7 +23,7 @@ Incluye 8 KPIs, velocity, timings por hora del día, maduración de instincts, h
 Run the Python script directly (not via LLM reasoning — es determinista):
 
 ```bash
-python C:/Users/luis/.claude/skills/norteia-continuous-learning/hooks/generate-dashboard.py
+python ~/.claude/skills/_generate-dashboard.py
 ```
 
 El script lee en paralelo:
@@ -42,21 +42,23 @@ Y escribe `~/.claude/skills/_dashboard.html` autocontenido (Chart.js + Google Fo
 Tras ejecutar el script, muestra al usuario:
 
 ```
-Dashboard regenerado: C:/Users/luis/.claude/skills/_dashboard.html
+Dashboard regenerado: ~/.claude/skills/_dashboard.html
 
 Acceso:
   · Preview server (si está arrancado): http://localhost:8080/_dashboard.html
-  · Directamente:                       file:///C:/Users/luis/.claude/skills/_dashboard.html
+  · Directamente:                       file://<HOME>/.claude/skills/_dashboard.html
 
 Próxima regeneración automática: diaria por sinapsis-linting
 ```
 
+Sustituye `<HOME>` por la ruta absoluta del home real del usuario — `file://` no expande `~`.
+
 Si el preview server de `.claude/launch.json` (name=`dashboard-static`) está activo, basta con navegar a la URL.
 Si no está activo, ofrece arrancarlo con `preview_start` del MCP Claude_Preview (solo si el usuario lo pide).
 
 ### Step 3: Handle errors
 
-- Si falla el Python con "template not found": el template está en `hooks/dashboard-template.html` — verificar que existe.
+- Si falla el Python con "template not found": el instalador deja el template en `~/.claude/skills/_dashboard-template.html` (origen: `core/_dashboard-template.html`) — verificar que existe.
 - Si falla por fichero JSON corrupto: el script ya gestiona `default` en `load_json`, seguirá con ceros.
 - Si no hay `_instinct.log`: el heatmap y hour-distribution estarán a cero, el resto del dashboard funciona.
 

core/_session-learner.sh

@@ -1,9 +1,9 @@
 #!/bin/bash
 # Session Learner - Sinapsis v4.3.3
 # Stop hook: five detectors —
-#   1. error-fix pairs (error → same tool success within 5 events)
-#   2. user-corrections (same file edited 2+ times within 10 events)
-#   3. workflow-chains (tool trigram repeated 2+ times)
+#   1. error-fix pairs (validated error → same tool success within 5 events)
+#   2. user-corrections (same file edited 2+ times within 10 events; by-design files excluded)
+#   3. workflow-chains (tool trigram repeated 5+ times, >=2 distinct tools, >=1 non-generic)
 #   4. repetitions (same error pattern across 3+ sessions — cross-session memory)
 #   5. agent-patterns (subagent tool sequences captured from Agent tool calls)
 # Also writes context.md per project.
@@ -15,6 +15,13 @@ if [ "${SINAPSIS_DEBUG:-}" = "1" ]; then
   exec 2>>"$HOME/.claude/skills/_sinapsis-debug.log"
 fi
 
+# v4.9.0: the node block below used to end in 2>/dev/null, which discarded every
+# diagnostic it emits — lock contention on the registry, failed upserts, and the
+# SINAPSIS_DEBUG redirect above, which the node invocation silently overrode. Those
+# messages had never reached a human. They now append to the learner log. The directory
+# is created here because the hook can fire before any installer has run.
+mkdir -p "$HOME/.claude/skills" 2>/dev/null || true
+
 INDEX_FILE="$HOME/.claude/skills/_instincts-index.json"
 PROPOSALS_FILE="$HOME/.claude/skills/_instinct-proposals.json"
 LOG_FILE="$HOME/.claude/skills/_session-learner.log"
@@ -58,6 +65,15 @@ try {
 
 if (lines.length < 3) process.exit(0);
 
+// v4.8.1: validate is_error against hard failure markers. Observations written by
+// observers < v4.8.1 flagged is_error on any output whose CONTENT mentioned "error"
+// (e.g. a successful Read of source code). Do not trust the flag alone.
+const HARD_ERR_RE = /(tool_use_error|Permission denied|command not found|No such file or directory|InputValidationError|String to replace not found|File does not exist|has not been read yet|Traceback \(most recent call last\)|fatal: |npm ERR!|\bEPERM\b|\bENOENT\b|\bEACCES\b|UnicodeEncodeError|exit code [1-9]|syntax error|was blocked|Web search error)/;
+function isRealError(l) {
+  if (!l || !l.is_error) return false;
+  return HARD_ERR_RE.test((l.err_msg || "") + "\n" + (l.output || "").slice(0, 600));
+}
+
 // ── JOB 1: Write project context.md (ALWAYS — not just when proposals exist) ──
 const projectDir = path.dirname(obsFile);
 const projectHash = path.basename(projectDir);
@@ -132,9 +148,9 @@ try {
   // Error patterns count (for proposals hint)
   let errorCount = 0;
   for (let i = 0; i < lines.length - 1; i++) {
-    if (!lines[i].is_error) continue;
+    if (!isRealError(lines[i])) continue;
     for (let j = i+1; j < Math.min(i+6, lines.length); j++) {
-      if (lines[j].tool === lines[i].tool && !lines[j].is_error) {
+      if (lines[j].tool === lines[i].tool && !isRealError(lines[j])) {
         errorCount++;
         break;
       }
@@ -397,16 +413,17 @@ try {
 const proposedIds = new Set(proposals.proposals.map(p => p.id));
 const found = [];
 
-// PATTERN 1: error → same tool success within 5 events (uses is_error flag from observe_v3)
+// PATTERN 1: error → same tool success within 5 events (uses is_error flag from observe_v3,
+// re-validated against hard failure markers — see isRealError)
 // Dedup: one proposal per tool per day
 for (let i = 0; i < lines.length - 1; i++) {
-  if (!lines[i].is_error) continue;
+  if (!isRealError(lines[i])) continue;
 
   const toolId = "fix-" + lines[i].tool.toLowerCase().replace(/[^a-z]/g, "");
   if (existing.has(toolId) || proposedIds.has(toolId)) continue;
 
   for (let j = i+1; j < Math.min(i+6, lines.length); j++) {
-    if (lines[j].tool === lines[i].tool && !lines[j].is_error) {
+    if (lines[j].tool === lines[i].tool && !isRealError(lines[j])) {
       found.push({
         type: "error_resolution",
         id: toolId,
@@ -430,11 +447,15 @@ const editEvents = lines
   .map((l, idx) => ({ ...l, _idx: idx }))
   .filter(l => l.event === "tool_complete" && (l.tool === "Edit" || l.tool === "Write"));
 
+// v4.8.1: files that are re-edited BY DESIGN (journals, control panels, wiki pages
+// appended item-by-item during /eod dual save) are not correction signals.
+const BYDESIGN_RE = /(?:^|[\\\/])(hot\.md|brief\.md|working-memory\.md|MEMORY\.md|CLAUDE\.md|\d{4}-\d{2}-\d{2}\.md)$|[\\\/](cerebro|briefs|_daily-summaries|daily-summaries)[\\\/]/i;
+
 const correctedFiles = {};
 for (let i = 0; i < editEvents.length - 1; i++) {
   let fileA = "";
   try { const inp = JSON.parse(editEvents[i].input || "{}"); fileA = inp.file_path || ""; } catch(e) {}
-  if (!fileA) continue;
+  if (!fileA || BYDESIGN_RE.test(fileA)) continue;
 
   for (let j = i + 1; j < editEvents.length; j++) {
     if (editEvents[j]._idx - editEvents[i]._idx > 10) break; // window of 10 events
@@ -470,6 +491,11 @@ const toolSeq = lines
   .filter(l => l.event === "tool_complete")
   .map(l => l.tool);
 
+// v4.8.1: generic-tool trigrams (Bash>Bash>Bash x380...) are coding activity, not
+// workflows — one session produced 154 junk proposals. Only propose trigrams that
+// include a NON-generic tool (MCP, custom), have >= 2 distinct tools, and repeat 5+.
+const GENERIC_TOOLS = new Set(["Bash","Read","Edit","Write","Grep","Glob","ToolSearch","AskUserQuestion","Skill","Task","TodoWrite","WebSearch","WebFetch","Agent","Monitor","ScheduleWakeup","NotebookEdit","TaskCreate","TaskUpdate"]);
+
 if (toolSeq.length >= 6) {
   const trigramCounts = {};
   for (let i = 0; i <= toolSeq.length - 3; i++) {
@@ -478,8 +504,10 @@ if (toolSeq.length >= 6) {
   }
 
   for (const [seq, count] of Object.entries(trigramCounts)) {
-    if (count < 2) continue;
+    if (count < 5) continue;
     const parts = seq.split(">");
+    if (new Set(parts).size < 2) continue;              // homogeneous = noise
+    if (parts.every(p => GENERIC_TOOLS.has(p))) continue; // all-generic = coding activity
     const wfId = "workflow-" + parts.map(p => p.toLowerCase().replace(/[^a-z]/g, "")).join("-");
     if (existing.has(wfId) || proposedIds.has(wfId)) continue;
     found.push({
@@ -543,8 +571,9 @@ for (const ae of agentEvents) {
   // \u0027 is single-quote: avoids closing the bash single-quoted node -e block
   const agentTypeMatch = output.match(/subagent_type[=:]?\s*["\u0027]?(\w+)/i);
   const agentType = agentTypeMatch ? agentTypeMatch[1] : "general";
-  // If agent output contains error keywords, propose a pattern
-  const hasError = /\berror\b|\bfailed\b|\bexception\b/i.test(output);
+  // v4.8.1: substring keywords flagged any agent whose report MENTIONED errors;
+  // require a validated hard failure marker instead.
+  const hasError = isRealError(ae);
   if (hasError) {
     const agId = "agent-error-" + agentType.toLowerCase();
     if (existing.has(agId) || proposedIds.has(agId)) continue;
@@ -591,6 +620,6 @@ if (found.length > 0) {
     "\nRevisa con /analyze-session.";
   console.log(JSON.stringify({ systemMessage: msg }));
 }
-' "$OBS_FILE" "$INDEX_FILE" "$PROPOSALS_FILE" "$LOG_FILE" 2>/dev/null
+' "$OBS_FILE" "$INDEX_FILE" "$PROPOSALS_FILE" "$LOG_FILE" 2>>"$LOG_FILE"
 
 exit 0