Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@ Layered prompt-injection / approval-fatigue defenses. Full reference: [docs/SECU
- **Audit log** (`cmd/odek/audit.go` + `internal/session/audit.go`) — every `wrapUntrusted` call records source + content-hash + turn into `<sessions>/audit/<id>.json`. After each turn a divergence heuristic flags `suspicious_divergence=true` when the agent ingested untrusted content AND its actions or final response reference resources that either did not appear in the user's message or were introduced by the untrusted content itself (closing response-only exfiltration and reused-resource injection bypasses). Inspect with `odek audit <session-id>` / `odek audit --list`.
- **Memory taint** (`internal/memory/provenance.go`) — `EpisodeProvenance` tracks Untrusted/Sources/UserApproved. Tainted episodes are stored but `Search()` filters them out, so a one-shot injection cannot persist via the episode pipeline. User must explicitly promote.
- **Skill provenance gate** (`internal/skills/loader.go` + `cache.go` + `tools.go`) — `Skill.Provenance{Untrusted, Sources, NeedsReview}`. NeedsReview skills pin to Lazy regardless of `auto_load` and are excluded from lazy trigger matching (the matchers in `reloadLocked` are built only from promotable skills), so a flagged skill cannot be injected into context on a keyword match. Skills scanned from the project-local `./.odek/skills/` directory are distrusted like `./odek.json`: they are forced to `NeedsReview` with `"project"` in `Sources` until promoted. The auto-save path declines tainted suggestions by default. Agent-created skills via `skill_save` and patched skills via `skill_patch` are forced to `Untrusted` + `NeedsReview`, and `skill_patch` refuses edits that touch the YAML frontmatter, blocking an injected agent from flipping `auto_load` or clearing `needs_review`. All skill-body scans (load, save, patch, suggestion) go through `guard.ScanContentWithScope`, so the fast local rule scan runs even when the `skills` guard scope or the guard itself is disabled; the sidecar second opinion only runs when the scope is enabled (default: on). Taint derives from `memory.ToolCallTaints`, whose always-external set covers `browser` / `http_batch` / `transcribe` / `session_search` / `web_search` / `vision` / `delegate_tasks` (`shell` is deliberately excluded as the primary work tool). `odek skill promote <name> --force` clears the flag after explicit user review.
- **Skill scope gates** (`internal/skills/selfimprove.go` + `curator.go` + `candidates.go` + `loader.go`) — the auto-save pipeline classifies every suggestion by scope before writing. Machine-specific suggestions (`NonReusableReason`: absolute home-directory paths) are dropped entirely. Project-specific suggestions (`ProjectScopeReason`: repo-rooted `./scripts/...` invocations, hardcoded release version tags) are redirected to the project skills dir `./.odek/skills` — project-related skills are never promoted to the global `~/.odek/skills`, and the redirect is refused (suggestion dropped) when the project dir resolves to the same location as the global dir (e.g. odek run from `$HOME`). Post-session micro-curation (`MicroCuration` / `RunAutoCurate` / `ExecuteMicroCuration`) is confined via `Skill.Source.Dir` to skills physically stored under the global dir, so a project skill can never be merged into a global skill or deleted by the curator. `skill_save` accepts `scope: "project"` and the interactive learn prompt offers `p` for the same boundary; both refuse when no distinct project dir exists. `DetectCorrection` additionally requires explicit correction phrases plus a shared lead verb between the corrected commands, and derived trigger keywords filter shell operators, flags, and version fragments — together these suppress the "garbage skill" class (one-off commands saved as reusable knowledge).
- **Skill save-time hygiene** (`internal/skills/selfimprove.go` + `candidates.go` + `loader.go`) — four gates keep the skill store clean: (1) recurrence: a suggestion must appear in `auto_save.min_occurrences` distinct sessions (default 2) before saving; first sightings are tracked in `~/.odek/skills/.candidates.json` (30-day pruning) and reported as `Pending`. (2) Score ranking: eligible suggestions are sorted by a deterministic substance score so `max_per_run` keeps the strongest. (3) Near-duplicate rejection: stopword-filtered Jaccard word-set similarity ≥ 0.85 against an existing skill in the target dir skips the save (`DuplicateOf`); same-name saves are updates and bypass the gate. (4) Secret scan: `WriteSkill` runs `internal/redact` over body and description — matches are replaced with `[REDACTED]` and the skill is pinned to `NeedsReview`. With `llm_curate`, merge bodies are LLM-synthesized (structure/size sanity-checked, mechanical fallback) while keeping the worst-of provenance union, so a merge can never launder taint.
- **Sub-agent damage cap** (`cmd/odek/subagent.go::applySubagentTrust`) — `delegate_tasks` carries `trust_level` + `max_risk`. The mutated `DangerousConfig` is passed into the sub-agent's `odek.Config` so the engine enforces the cap. Untrusted ⇒ NonInteractive=deny, Destructive/CodeExec/Install/SystemWrite/NetworkEgress/Unknown/Blocked all forced to Deny. `max_risk` ⇒ everything above cap forced to Deny. MCP tools are not loaded into untrusted sub-agents, because the MCP `ToolAdapter` does not perform its own danger check and would otherwise bypass the cap.
- **FD-based API key handoff** (`cmd/odek/subagent_key.go`) — parent writes key to a 0600 tempfile, immediately `unlink()`s, passes the FD via `cmd.ExtraFiles`. Sub-agent reads from `$ODEK_API_KEY_FD` and closes. Key never in `/proc/<pid>/environ`.
- **Approver friction** (`internal/danger/approver.go`, `cmd/odek/wsapprover.go`, `cmd/odek/shell.go`, `cmd/odek/perf_tools.go`) — both TTYApprover and WSApprover engage friction mode after 3 approvals of the same class in 60s: require typing literal `approve`, 1.5s pause. Trust-class shortcut disabled for `destructive` + `blocked` regardless. CLI `shell`/`parallel_shell` reuse a single TTYApprover instance per process so the counter and trust cache persist across prompts.
Expand Down
24 changes: 21 additions & 3 deletions cmd/odek/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -2011,7 +2011,7 @@ func runLearnLoop(messages []llm.Message, sm *skills.SkillManager, llmClient ski
if skillsCfg.Verbose {
verbose = os.Stderr
}
if skills.RunAutoSaveLoop(filtered, userDir, sm, llmClient, skillsCfg, g, guardCfg, verbose) {
if skills.RunAutoSaveLoop(filtered, userDir, skills.ProjectSkillsDir(), sm, llmClient, skillsCfg, g, guardCfg, verbose) {
return
}

Expand All @@ -2024,15 +2024,15 @@ func runLearnLoop(messages []llm.Message, sm *skills.SkillManager, llmClient ski
}

// interactiveSavePrompt walks the user through each suggestion, reading
// y/n/s from stdin. Lives in cmd/odek because it couples to the TTY.
// y/n/p/s from stdin. Lives in cmd/odek because it couples to the TTY.
func interactiveSavePrompt(filtered []skills.SkillSuggestion, userDir string, sm *skills.SkillManager, g guard.Guard, guardCfg guard.Config) {
fmt.Fprintf(os.Stderr, "\n🔍 Learning: detected %d skill pattern(s)\n", len(filtered))
for _, s := range filtered {
fmt.Fprint(os.Stderr, skills.FormatSuggestionWithPreview(s, true, 400))
if s.IsTainted() {
fmt.Fprintf(os.Stderr, " ⚠ This suggestion is tainted (sources: %s). It will be saved but cannot be auto-loaded until promoted with --force.\n", strings.Join(s.Provenance.Sources, ", "))
}
fmt.Fprintf(os.Stderr, " Save as skill? [Y/n/s=skip always]: ")
fmt.Fprintf(os.Stderr, " Save as skill? [Y/n/p=save to project/s=skip always]: ")

var response string
fmt.Scanf("%s", &response)
Expand All @@ -2047,6 +2047,24 @@ func interactiveSavePrompt(filtered []skills.SkillSuggestion, userDir string, sm
sm.MarkDirty()
sm.Reload()
}
case "p", "project":
// Project-scoped save: keeps project-specific procedures out
// of the global dir. Refused when the project dir resolves to
// the global one (odek run from $HOME).
projDir := skills.ProjectSkillsDir()
pa, perr := filepath.Abs(projDir)
ua, uerr := filepath.Abs(userDir)
if perr != nil || uerr != nil || pa == ua {
fmt.Fprintf(os.Stderr, " ✗ Cannot save to project: no distinct project skills dir from here\n")
break
}
if err := skills.SaveSuggestionWithGuard(context.Background(), projDir, s, g, guardCfg); err != nil {
fmt.Fprintf(os.Stderr, " ✗ Error saving skill: %v\n", err)
} else {
fmt.Fprintf(os.Stderr, " ✓ Saved project skill %q to %s (promote with `odek skill promote` to use)\n", s.Name, projDir)
sm.MarkDirty()
sm.Reload()
}
case "s", "skip":
sl := skills.LoadSkipList(userDir)
sl.RecordSkip(userDir, s.Name, s.Heuristic)
Expand Down
6 changes: 4 additions & 2 deletions cmd/odek/main_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1396,8 +1396,10 @@ func TestRunLearn_MultiStepProcedure(t *testing.T) {
}()

// Create local odek.json with auto_save enabled, LLM enhancement disabled
// (mock server can't handle enhancement prompts)
configContent := `{"skills": {"verbose": true, "auto_save": {"enabled": true, "require_llm": false}, "llm_learn": false}}`
// (mock server can't handle enhancement prompts). min_occurrences=1
// bypasses the recurrence gate — this test covers the save path; the
// gate itself has dedicated unit tests in internal/skills.
configContent := `{"skills": {"verbose": true, "auto_save": {"enabled": true, "require_llm": false, "min_occurrences": 1}, "llm_learn": false}}`
os.WriteFile("odek.json", []byte(configContent), 0644)
defer os.Remove("odek.json")

Expand Down
4 changes: 2 additions & 2 deletions cmd/odek/serve.go
Original file line number Diff line number Diff line change
Expand Up @@ -1211,13 +1211,13 @@ func handlePrompt(
resolved.Skills.Curation.SkipThreshold, resolved.Skills.Curation.SkipResetDays)
_ = skipped
if resolved.Skills.AutoSave.Enabled {
result := skills.AutoSaveSuggestions(filtered, userDir, resolved.Skills, g, resolved.Guard, false)
result := skills.AutoSaveSuggestions(filtered, userDir, skills.ProjectSkillsDir(), resolved.Skills, g, resolved.Guard, false)
for _, name := range result.Saved {
sm.Notifier.Notify(skills.SkillEvent{
Type: "saved", SkillName: name, Timestamp: time.Now().UTC(),
})
}
if len(result.Saved) > 0 {
if len(result.Saved)+len(result.ProjectSaved) > 0 {
sm.MarkDirty()
sm.Reload()
}
Expand Down
9 changes: 7 additions & 2 deletions cmd/odek/telegram.go
Original file line number Diff line number Diff line change
Expand Up @@ -2047,13 +2047,18 @@ func handleChatMessage(

// Auto-save if enabled
if skillsCfg.AutoSave.Enabled {
result := skills.AutoSaveSuggestions(filtered, userDir, *skillsCfg, telegramGuard, telegramGuardCfg, false)
result := skills.AutoSaveSuggestions(filtered, userDir, skills.ProjectSkillsDir(), *skillsCfg, telegramGuard, telegramGuardCfg, false)
for _, name := range result.Saved {
sm.Notifier.Notify(skills.SkillEvent{
Type: "saved", SkillName: name, Timestamp: time.Now().UTC(),
})
}
if len(result.Saved) > 0 {
for _, name := range result.ProjectSaved {
sm.Notifier.Notify(skills.SkillEvent{
Type: "saved", SkillName: name, Timestamp: time.Now().UTC(),
})
}
if len(result.Saved)+len(result.ProjectSaved) > 0 {
sm.MarkDirty()
sm.Reload()
// Run micro-curation
Expand Down
6 changes: 4 additions & 2 deletions docs/CONFIG.md
Original file line number Diff line number Diff line change
Expand Up @@ -289,7 +289,8 @@ The `skills` section controls the skill system:
"auto_save": {
"enabled": true,
"require_llm": true,
"max_per_run": 3
"max_per_run": 3,
"min_occurrences": 2
}
}
}
Expand All @@ -313,7 +314,8 @@ The `skills` section controls the skill system:
| `curation.skip_reset_days` | — | 30 | Days after which a skip expires (re-allows suggestion) |
| `auto_save.enabled` | — | true | Auto-save quality skill suggestions without prompting |
| `auto_save.require_llm` | — | true | Only auto-save if LLM enhancement was applied |
| `auto_save.max_per_run` | — | 3 | Max skills to auto-save per session |
| `auto_save.max_per_run` | — | 3 | Max skills to auto-save per session (score-ranked) |
| `auto_save.min_occurrences` | — | 2 | Distinct sessions a pattern must recur in before auto-save (1 disables) |
| `embedding` | — | *(inherits top-level `embedding`)* | Optional override of the shared embedding backend for semantic skill matching. When unset, skills inherit the top-level `embedding` default with the per-turn query timeout bounded to 2s. See [Shared embedding backend](#shared-embedding-backend-embedding--memory-sessions--skills). |

## Memory configuration
Expand Down
22 changes: 14 additions & 8 deletions docs/LEARNING.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,10 @@ odek run --learn "set up CI with GitHub Actions"
- **Auto-curation runs silently** — after every session where skills were saved, overlaps are merged, duplicates removed, and stale skills pruned.
- **Learning is non-blocking** — skill detection and auto-save run in a background goroutine after the agent's response is delivered. The process exits immediately; learning completes asynchronously on a best-effort basis.
- **Tainted skills require explicit promotion** — skills learned from `browser`, MCP tools, or sensitive file reads are saved with `Provenance.Untrusted=true` and `NeedsReview=true`. They cannot be auto-loaded until you run `odek skill promote <name> --force` after reviewing the body.
- **Scope gates keep garbage out** — machine-specific suggestions (absolute home-directory paths) are dropped entirely. Project-specific suggestions (repo-rooted `./scripts/...` invocations, hardcoded release version tags) are redirected to the project skills dir `./.odek/skills` instead of the global `~/.odek/skills` — project-related skills are never promoted to global, and project-dir skills stay pinned to `NeedsReview` until promoted locally. Auto-curation (merge/prune) is confined to the global dir, so project skills can never be merged into or deleted by the global curator.
- **Recurrence before persistence** — a pattern must be seen in at least `auto_save.min_occurrences` distinct sessions (default 2) before it is eligible for auto-save; first sightings are recorded in `~/.odek/skills/.candidates.json` and reported as pending.
- **Save-time hygiene** — suggestions are score-ranked (verification section, command-log evidence, LLM-judged) so the `max_per_run` budget goes to the strongest candidates; near-duplicates of existing skills (Jaccard word-set similarity ≥ 0.85) are skipped instead of creating parallel skills; and every SKILL.md write is secret-scanned, with detected credentials replaced by `[REDACTED]` and the skill pinned to `NeedsReview`.
- **LLM curation** — with `llm_curate` enabled, merge bodies are synthesized by the model (deduplicated, coherent) instead of mechanically concatenated; failures fall back to concatenation.

## CLI Usage

Expand Down Expand Up @@ -118,10 +122,10 @@ odek skill promote procedure-browser --force
- Verify each step's output before proceeding
- Exit code 0 means success

Save as skill? [Y/n/s=skip always]:
Save as skill? [Y/n/p=save to project/s=skip always]:
```

Type `y` (or Enter) to save, `n` to skip (temporarily), `s` to skip permanently.
Type `y` (or Enter) to save globally, `p` to save to the project skills dir (`./.odek/skills`, for project-specific procedures), `n` to skip (temporarily), `s` to skip permanently.

### Skip Persistence

Expand Down Expand Up @@ -378,7 +382,7 @@ After the agent completes:
## Overview
Procedure for: docker
...
Save as skill? [Y/n/s=skip always]: y
Save as skill? [Y/n/p=save to project/s=skip always]: y
✓ Saved skill "procedure-docker"
```

Expand All @@ -393,7 +397,7 @@ odek skill list
```
📝 Skill suggestion: repeated-ls
...
Save as skill? [Y/n/s=skip always]: s
Save as skill? [Y/n/p=save to project/s=skip always]: s
Skipped permanently. Use `odek skill reset-skips` to re-enable.
```

Expand All @@ -407,7 +411,7 @@ odek skill reset-skips repeated-ls
- **Heuristic detection is deterministic** — same tool calls always produce the same suggestions. Skip persistence prevents repeats (one skip = permanent suppression).
- **Max 1 per heuristic** — if an agent session has 10 multi-step sequences, only the first is suggested.
- **Max 5 suggestions total** — one per heuristic type.
- **Auto-curation handles dedup** — overlapping skills are automatically merged after each session.
- **Dedup at save and curation time** — near-duplicates of existing skills are rejected at save time (Jaccard similarity ≥ 0.85); remaining overlaps are merged by auto-curation after each session.
- **Command-only** — the heuristics work on terminal (`shell`) tool calls. Other tools (read_file, write_file) are visible in the transcript but aren't analyzed for patterns.
- **LLM enhancement requires API calls** — when `llm_learn: true`, each suggestion triggers an LLM call for enrichment. Set to `false` for zero-cost heuristic-only mode.
- **No REPL integration** — learning currently only works with `odek run`, not in REPL mode.
Expand All @@ -423,7 +427,8 @@ odek skill reset-skips repeated-ls
"auto_save": {
"enabled": true,
"require_llm": true,
"max_per_run": 3
"max_per_run": 3,
"min_occurrences": 2
},
"curation": {
"staleness_days": 90,
Expand All @@ -440,10 +445,11 @@ odek skill reset-skips repeated-ls
|-------|---------|-------------|
| `learn` | `true` | Enable skill learning |
| `llm_learn` | `true` | Use LLM to enhance detected patterns |
| `llm_curate` | `true` | Use LLM for curation suggestions |
| `llm_curate` | `true` | Use LLM for curation suggestions and merge-body synthesis |
| `auto_save.enabled` | `true` | Auto-save without prompting |
| `auto_save.require_llm` | `true` | Only auto-save LLM-enhanced skills |
| `auto_save.max_per_run` | `3` | Max skills to auto-save per session |
| `auto_save.max_per_run` | `3` | Max skills to auto-save per session (strongest suggestions win the budget) |
| `auto_save.min_occurrences` | `2` | Sessions a pattern must recur in before it may be auto-saved (`1` disables the recurrence gate) |
| `curation.auto_curate` | `true` | Run auto-curation after sessions |
| `curation.skip_threshold` | `1` | Skips needed for permanent suppression |
| `curation.skip_reset_days` | `30` | Days before skip expires |
Expand Down
2 changes: 2 additions & 0 deletions docs/SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,8 @@ Skills created or edited through the agent-facing `skill_save` and `skill_patch`

The non-interactive auto-save path (`RunAutoSaveLoop`) now **declines to persist tainted suggestions by default**, so a prompt-injected turn cannot silently leave a poisoned skill on disk. Tainted suggestions are still surfaced in the interactive TUI and can be saved explicitly by the user after review.

The auto-save pipeline additionally classifies every suggestion by **scope** before writing: machine-specific suggestions (absolute home-directory paths) are dropped, and project-specific ones (repo-rooted `./scripts/...` invocations, hardcoded release version tags) are redirected to `./.odek/skills` — project-related skills are never promoted to the global `~/.odek/skills`, and micro-curation is confined to the global dir via `Skill.Source.Dir` so a project skill can never be merged into a global one. Save-time hygiene gates further require cross-session recurrence (`auto_save.min_occurrences`, default 2), reject near-duplicates of existing skills, and run `internal/redact` over every SKILL.md write — detected credentials are replaced with `[REDACTED]` and the skill pinned to `NeedsReview`.

After reviewing the skill body, promote it with `--force`:

```bash
Expand Down
3 changes: 3 additions & 0 deletions internal/config/loader.go
Original file line number Diff line number Diff line change
Expand Up @@ -1842,6 +1842,9 @@ func resolveSkills(cfg *SkillsConfig) skills.SkillsConfig {
def.AutoSave.MaxPerRun = cfg.AutoSave.MaxPerRun
}
def.AutoSave.RequireLLM = cfg.AutoSave.RequireLLM
if cfg.AutoSave.MinOccurrences > 0 {
def.AutoSave.MinOccurrences = cfg.AutoSave.MinOccurrences
}
}
if cfg.LLMLearn != nil {
def.LLMLearn = *cfg.LLMLearn
Expand Down
Loading
Loading