diff --git a/CHANGELOG.md b/CHANGELOG.md index 6101d47..8253f4f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,24 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [Unreleased] +### Added + +- **`forge docs impact` — a reusable documentation-impact graph.** Where `docs check` + reconciles a fixed list of registries and `docs sync` scans a diff for raw identifiers, + `docs impact` answers the general question the project kept forgetting: _"I changed X — + which documented surfaces mention X and are now potentially stale?"_ It works in three + data-driven stages: (1) a pluggable extractor registry derives the **typed** entities + the project documents (command names, CLI flags, `FORGE_*` env vars, MCP tool names, + exported symbols, brand tokens, the version, `package.json` fields) from their canonical + sources — reusing `docs_check`'s `envVarsRead`/`srcFiles` and the `COMMANDS`/`TOOLS` + registries; (2) a word-boundary- and code-fence-aware scan of every doc surface (all + tracked `*.md`, `CITATION.cff`, the plugin manifests, the landing page, `package.json`) + builds an inverted index entity → `file:line`; (3) an impact query maps the entities a + git diff changed to every doc location that references them, ranked by confidence. + Advisory by default (`--strict` exits non-zero for CI); `--since `, `--staged`, + `--min-confidence `, and `--json` flags; `docs check` prints a one-line advisory + pointing here when the working tree touched a documented entity. New `src/docs_impact.js`. + ## [0.23.2] - 2026-07-20 ### Fixed diff --git a/README.md b/README.md index c9fe2b3..5bfdcda 100644 --- a/README.md +++ b/README.md @@ -217,7 +217,7 @@ that never clobbers your existing settings (skip it with `install.sh --no-settin | | `forge tools` | primary-tool config — gitignore secondary-tool artifacts (`.cursor/`, `.gemini/`, …) for tools this repo doesn't use; `forge tools ` sets it, `--reset` clears | | | `forge doctor` | pass/fail health check: tools, guards, MCP, drift, update | | | `forge update` | self-update — `--check` reports if a newer version exists, bare applies it, `--to ` pins/downgrades | -| | `forge docs` | docs↔code drift — `check` reconciles commands/env/MCP/CHANGELOG; `sync` sweeps the diff for stale doc mentions | +| | `forge docs` | docs↔code drift — `check` reconciles commands/env/MCP/CHANGELOG; `sync` sweeps the diff for stale doc mentions; `impact` maps which docs reference the entities THIS diff changed | | | `forge config` | provider setup — show / switch / add providers, set the default model | | | `forge integrations` | opt-in third-party MCP servers (e.g. context7) — `add` records the managed set and writes only with `--yes` (`--adopt` claims a same-name entry you already had); `remove` reverses it | | | `forge harden` | wire the pre-commit gate (gitleaks + commit gate) + sandbox settings | diff --git a/docs/GUIDE.md b/docs/GUIDE.md index 7ce123f..e08021b 100644 --- a/docs/GUIDE.md +++ b/docs/GUIDE.md @@ -410,6 +410,46 @@ touched in the same diff still answers for **removed** symbols (updated-for-one- isn't updated-for-the-rename); and `.forge/state.md` is never scanned — handoff writes the changed-file list into it by design, so the gate's mtime check covers it instead. +### `forge docs impact` — change X, which docs mention X? + +`docs sync` sweeps a diff for raw identifiers; `docs impact` is the general, reusable +version of the same idea, built as a **documentation-reference graph** rather than a +per-file rule. It works in three data-driven stages: + +1. **Entity extraction.** A pluggable extractor registry derives the _typed_ set of + things the project documents — command names, CLI flags, `FORGE_*` env vars, MCP tool + names, exported symbols, brand tokens, the version, and `package.json` fields — from + their canonical sources (the same `COMMANDS`/`TOOLS` registries and `envVarsRead` + scanner `docs check` already uses). Adding a new entity type is one record in the + registry, not a new special case. +2. **Reference index.** A word-boundary- and code-fence-aware scan of every doc surface + (all tracked `*.md`, `CITATION.cff`, the plugin manifests, the landing page, + `package.json`) builds an inverted index: entity → every `file:line` that names it. + English-word-shaped entities (bare symbols like `build`) only count inside backticks + or fenced blocks, so prose doesn't false-positive. +3. **Impact query.** From a git diff of the canonical sources it computes which entities + _changed_ (a command renamed, a flag added, an env var removed, an export touched, the + version bumped) and returns every doc location that references them, ranked by + confidence. + +```console +$ forge docs impact --since main +docs impact — diff vs main: 2 changed entities, 1 doc file(s) potentially stale + env `APP_CACHE_MODE` (REMOVED) — 2 reference(s): + README.md:88 (code, 0.95) [Environment] Set `APP_CACHE_MODE` to tune the cache. + version `0.23.0` (REMOVED) — 1 reference(s): + README.md:3 (prose, 0.75) install @scope/pkg@0.23.0 + → review each surface: update it, or confirm the mention is still correct. +``` + +Advisory by default (matching the fail-open ethos) — `--strict` exits non-zero when any +doc surface is impacted, for CI. `--since ` sets the diff base (default: the session +baseline, then `HEAD`), `--staged` reads the index, `--min-confidence ` drops low-signal +hits, `--json` for tooling. `docs check` also prints a one-line advisory pointing here when +the working tree touched a documented entity. **Honest limits:** a token index catches a +doc that _names_ a changed entity; it cannot see a paraphrase that never uses the name, a +screenshot or diagram, or a design/wording choice with no textual anchor. + ### `forge verify` — did it actually work? The independent check: runs the real test suite and flags edited symbols that aren't in @@ -997,6 +1037,7 @@ Plain `forge cost` remains the per-day spend view via `ccusage`. | `forge doctor` | Health check: layers, install, drift, cortex. `forge doctor --fix` auto-repairs the safely fixable findings (missing settings hooks/permissions, ledger union-merge rule, stale `AGENTS.md`, non-executable guards) by reusing the same idempotent functions `forge init`/`forge sync` use, then re-checks. | | `forge docs check` | Docs↔code drift: commands, env vars, MCP tools, CHANGELOG reconciled against the code (CI-gated on the forge repo itself). | | `forge docs sync` | Diff-driven stale-docs sweep: UPDATED / STALE (file:line hits) / VERIFIED-UNAFFECTED per artifact (see the full section above). | +| `forge docs impact` | Documentation-reference graph: extract typed entities (commands/flags/env/MCP/symbols/version), index every doc surface, and report which docs reference the entities THIS diff changed — ranked by confidence (see the full section above). | | `forge catalog` | Start-Here index of every tool / crew / guard. | | `forge integrations` | Opt-in third-party MCP servers (e.g. `context7`, no longer installed by default). `integrations add ` prints the package, its network behaviour, and the files it touches, then writes only with `--yes`; the install is recorded in `.forge/forge.config.json` (`mcp.integrations`), so every later sync keeps emitting it. A same-name server you configured yourself is never overwritten unless you pass `--adopt` (recorded under `mcp.adopted`). `integrations remove ` reverses the add — it deletes only forge-owned entries/blocks/files and is a no-op when nothing is installed. | | `forge brain` / `forge remember` | Portable project memory inlined into `AGENTS.md`. | diff --git a/src/cli.js b/src/cli.js index 4754714..a69cc54 100755 --- a/src/cli.js +++ b/src/cli.js @@ -472,6 +472,22 @@ async function run(argv) { if (r.error || (argv.includes("--strict") && r.stale.length)) process.exitCode = 1; return; } + // `impact` — reusable documentation-impact graph: which documented surfaces + // reference the entities THIS diff changed (advisory; --strict exits 1). + if (sub === "impact") { + const { docsImpact, renderDocsImpact } = await import("./docs_impact.js"); + const { newestBaseline } = await import("./docs_sync.js"); + const staged = argv.includes("--staged"); + const sinceIdx = argv.indexOf("--since"); + const base = sinceIdx >= 0 ? argv[sinceIdx + 1] : newestBaseline(process.cwd()) || undefined; + const mcIdx = argv.indexOf("--min-confidence"); + const minConfidence = mcIdx >= 0 ? Number(argv[mcIdx + 1]) : 0; + const r = docsImpact(process.cwd(), { base, staged, minConfidence }); + if (json) console.log(JSON.stringify(r, null, 2)); + else console.log(renderDocsImpact(r)); + if (argv.includes("--strict") && r.impacted.length) process.exitCode = 1; + return; + } // `check` — self-check of the forge package's own docs against its code (commands // table, env reads, MCP registry, CHANGELOG). const { docsCheck } = await import("./docs_check.js"); @@ -489,6 +505,16 @@ async function run(argv) { console.log(`\n${r.issues.filter((i) => i.severity === "error").length} problem(s)`); process.exitCode = 1; } + // Advisory: if the working tree changed a documented entity, point at the impacted + // prose. Never fails the check (fail-open); best-effort, so a git-less tree is silent. + try { + const { docsImpact } = await import("./docs_impact.js"); + const imp = docsImpact(process.cwd()); + if (imp.impacted.length) + console.log( + `\n ! ${imp.summary.impactedFiles} doc file(s) may be stale from this diff — run \`${BRAND.cli} docs impact\` to review`, + ); + } catch {} return; } if (cmd === "integrations") { diff --git a/src/commands.js b/src/commands.js index 859b66f..f482128 100644 --- a/src/commands.js +++ b/src/commands.js @@ -109,7 +109,35 @@ export const COMMANDS = { dash: "live dashboard: ledger, metrics trends, radar, memory browser, timeline, blast radius", report: "emit a static, self-contained HTML snapshot of .forge/ — opens offline, no server", brand: "print the active brand token map", - docs: "docs↔code drift — check (registry reconcile) / sync (diff-driven stale-docs sweep)", + docs: { + summary: + "docs↔code drift — check (registry reconcile) / sync (diff-driven stale-docs sweep) / impact (reusable doc-reference graph: which docs mention what THIS diff changed)", + usage: "forge docs [check | sync | impact] [--since | --staged] [--strict] [--json]", + flags: [ + { + flag: "--since ", + desc: "impact: diff against this git ref (default: session baseline, then HEAD)", + }, + { + flag: "--staged", + desc: "impact/sync: diff the staged index instead of the working tree", + }, + { + flag: "--min-confidence ", + desc: "impact: drop reference hits below this confidence (0..1)", + }, + { + flag: "--strict", + desc: "exit non-zero when stale/impacted docs are found (for CI; advisory otherwise)", + }, + ], + examples: [ + "forge docs check", + "forge docs sync", + "forge docs impact", + "forge docs impact --since main", + ], + }, integrations: "opt-in third-party MCP servers (e.g. context7) — add records the managed set and writes only with --yes (--adopt claims a same-name entry); remove reverses it", }; diff --git a/src/docs_check.js b/src/docs_check.js index ef80e5b..01519d1 100644 --- a/src/docs_check.js +++ b/src/docs_check.js @@ -37,7 +37,9 @@ function readDoc(root, rel) { return existsSync(p) ? readFileSync(p, "utf8") : ""; } -function srcFiles(root) { +// Exported so docs_impact.js reuses the SAME src-file inventory (recursive, .js only) +// the env-var extractor scans — one definition of "what counts as a source file". +export function srcFiles(root) { const dir = join(root, "src"); if (!existsSync(dir)) return []; // recursive: src/emit/*.js reads are part of the same env contract. diff --git a/src/docs_impact.js b/src/docs_impact.js new file mode 100644 index 0000000..a71b3b0 --- /dev/null +++ b/src/docs_impact.js @@ -0,0 +1,600 @@ +// forge docs impact — a REUSABLE documentation-impact graph. Where `docs check` +// reconciles a fixed list of registries and `docs sync` scans a git diff for raw +// identifiers, this module answers the general question the project keeps forgetting: +// "I changed X — which documented surfaces mention X and are now potentially stale?" +// +// It works in three data-driven stages, none of them hardcoded per file: +// +// 1. ENTITY EXTRACTION. A data-driven EXTRACTORS registry derives the TYPED set of +// entities the project documents (commands, flags, env vars, MCP tools, exported +// symbols, brand tokens, the version, package.json fields) from their CANONICAL +// sources. Adding a new entity type = pushing one entry onto EXTRACTORS, never +// editing N call sites. The heavy extractors are REUSED from docs_check.js +// (envVarsRead, srcFiles) and the existing registries (COMMANDS, TOOLS, BRAND). +// +// 2. REFERENCE INDEX. A generic token scan over every prose/doc SURFACE (all tracked +// *.md, CITATION.cff, the plugin manifests, the landing page, package.json's +// description/keywords) builds an inverted index entity → [{file,line,section, +// context,confidence}]. The scan is word-boundary aware and code-fence aware: +// English-word-shaped entities (bare symbols) are only counted inside code spans, +// exactly like docs_sync's "soft" identifiers, so prose doesn't false-positive. +// +// 3. IMPACT QUERY. Given the entities a git diff CHANGED (a command renamed/removed, +// a flag added, an env var deleted, an export touched, the version bumped), it +// returns every indexed doc location that references them, ranked by confidence. +// This is the reusable core: change → index lookup → the exact stale-risk surfaces. +// +// Honest about limits (see the CLI help + README): a token index catches a doc that +// NAMES a changed entity. It cannot catch a paraphrase that never uses the name, a +// screenshot/diagram/image, or a design/wording choice with no textual anchor. It is +// advisory by construction — a high-recall "look here", not a proof of staleness. + +import { execFileSync } from "node:child_process"; +import { existsSync, readFileSync } from "node:fs"; +import { join } from "node:path"; +import { BRAND } from "./brand.js"; +import { COMMANDS, HIDDEN_COMMANDS } from "./commands.js"; +import { envVarsRead, srcFiles } from "./docs_check.js"; +import { TOOLS } from "./mcp_tools.js"; + +const git = (root, args) => { + try { + return execFileSync("git", args, { + cwd: root, + encoding: "utf8", + stdio: ["ignore", "pipe", "ignore"], + }); + } catch { + return ""; + } +}; + +const esc = (s) => String(s).replace(/[.*+?^${}()|[\]\\]/g, "\\$&"); + +/** A tracked file is a canonical SOURCE (never scanned as a doc — that would flag every + * registry as a reference to itself). */ +const isSrcJs = (rel) => rel.startsWith("src/") && rel.endsWith(".js"); + +// History files are append-only and correct-by-design: their mentions of a changed +// entity (a renamed command in an old changelog line) are NOT staleness. Same exemption +// docs_sync applies. Kept out of both the reference index and the impacted output. +const HISTORY_RE = /(^|\/)(CHANGELOG\.md|\.forge\/decisions\.md)$/i; + +// --------------------------------------------------------------------------- +// Stage 1 — the data-driven extractor registry. +// --------------------------------------------------------------------------- +// Each extractor is a plain record: +// type stable entity-type label (also the confidence bucket) +// weight base confidence a plain-prose reference of this type earns (0..1) +// codeOnly entity names shaped like English words → only count inside code spans +// extract() the canonical current entities: {name, meta?}[] +// ref(names) build the doc-scan RegExp (group 1 = matched entity name) +// isSource (rel) → is this file a canonical source that DEFINES this type? +// discover (text) → entity names found in arbitrary source text (used on a diff's +// REMOVED lines to detect entities that were DELETED, and on package.json +// to recover the OLD version). Optional. +// +// Adding "config keys" or any new type is one more record here — nothing else changes. + +const boundaryRe = (names) => + new RegExp(`(?:^|[^\\w$/.-])(${names.map(esc).join("|")})(?![\\w$])`, "g"); + +/** @type {Array<{type:string,weight:number,codeOnly?:boolean,extract:(root:string)=>{name:string,meta?:object}[],ref:(names:string[])=>RegExp,isSource:(rel:string)=>boolean,discover?:(text:string)=>string[]}>} */ +export const EXTRACTORS = [ + { + // Command names — reference form is ` ` (BRAND.cli, never a literal), + // which keeps a common word like "docs" from matching bare prose. + type: "command", + weight: 0.9, + extract: () => [ + ...Object.keys(COMMANDS).map((name) => ({ name })), + ...HIDDEN_COMMANDS.map((name) => ({ name, meta: { hidden: true } })), + ], + ref: (names) => + new RegExp(`\\b${esc(BRAND.cli)}\\s+(${names.map(esc).join("|")})(?![\\w-])`, "g"), + isSource: (rel) => rel === "src/commands.js", + // Table keys in commands.js: ` init: {` / ` taste: "…"`. + discover: (text) => [...text.matchAll(/^\s{2}([a-z][a-z-]+):/gm)].map((m) => m[1]), + }, + { + // CLI flags, canonically declared in the COMMANDS table's `flags`/`usage`. + type: "flag", + weight: 0.8, + extract: () => { + const flags = new Set(); + for (const entry of Object.values(COMMANDS)) { + if (typeof entry !== "object") continue; + for (const f of entry.flags ?? []) + for (const m of String(f.flag).matchAll(/--[a-z][\w-]*/g)) flags.add(m[0]); + for (const m of String(entry.usage ?? "").matchAll(/--[a-z][\w-]*/g)) flags.add(m[0]); + } + return [...flags].map((name) => ({ name })); + }, + ref: (names) => boundaryRe(names), + isSource: (rel) => rel === "src/commands.js", + discover: (text) => [...text.matchAll(/--[a-z][\w-]*/g)].map((m) => m[0]), + }, + { + // Env vars — REUSES docs_check.envVarsRead (the one scanner of process.env + guards). + type: "env", + weight: 0.9, + extract: (root) => [...envVarsRead(root)].map((name) => ({ name })), + ref: (names) => boundaryRe(names), + isSource: (rel) => isSrcJs(rel) || /^global\/guards\/.*\.sh$/.test(rel), + discover: (text) => [ + ...[...text.matchAll(/process\.env\.([A-Z_][A-Z0-9_]*)/g)].map((m) => m[1]), + ...[ + ...text.matchAll( + /\b((?:FORGE|ANTHROPIC|LITELLM|OPENROUTER|ENABLE_CORTEX|CLAUDE)_[A-Z0-9_]+)\b/g, + ), + ].map((m) => m[1]), + ], + }, + { + // MCP tool names — from the same TOOLS registry docs_check reconciles. + type: "mcp-tool", + weight: 0.9, + extract: () => TOOLS.map((t) => ({ name: t.name })), + ref: (names) => boundaryRe(names), + isSource: (rel) => rel === "src/mcp_tools.js", + discover: (text) => [...text.matchAll(/name:\s*["']([a-z][a-z0-9_]+)["']/gi)].map((m) => m[1]), + }, + { + // Exported public symbols — the JS API surface. Word-shaped (build, load, has), so + // codeOnly: a symbol counts only inside a code span, never in running prose. + type: "symbol", + weight: 0.6, + codeOnly: true, + extract: (root) => { + const names = new Set(); + for (const file of srcFiles(root)) { + let text = ""; + try { + text = readFileSync(file, "utf8"); + } catch { + continue; + } + for (const m of text.matchAll( + /export\s+(?:async\s+)?(?:function|const|let|class)\s+([A-Za-z_$][\w$]*)/g, + )) + names.add(m[1]); + } + return [...names].map((name) => ({ name })); + }, + ref: (names) => boundaryRe(names), + isSource: isSrcJs, + discover: (text) => + [ + ...text.matchAll( + /export\s+(?:async\s+)?(?:function|const|let|class)\s+([A-Za-z_$][\w$]*)/g, + ), + ].map((m) => m[1]), + }, + { + // Brand tokens — from src/brand.js. Ubiquitous, so low confidence, but a rebrand + // genuinely touches every surface, which is exactly what this should surface. + type: "brand", + weight: 0.35, + extract: () => + [...new Set([BRAND.brand, BRAND.cli, BRAND.pkg, BRAND.home].filter(Boolean))].map((name) => ({ + name, + })), + ref: (names) => boundaryRe(names), + isSource: (rel) => rel === "brand.json", + }, + { + // The version string — a bump makes every hardcoded version reference stale. + // `v?` so `v0.23.1` and `0.23.1` both match; group 1 is the bare version. + type: "version", + weight: 0.75, + // Read the TARGET repo's package.json (root), not forge's own BRAND.version, so a + // version bump is detected in whatever repo `docs impact` runs against. + extract: (root) => { + try { + const v = JSON.parse(readFileSync(join(root, "package.json"), "utf8")).version; + return v ? [{ name: v }] : []; + } catch { + return []; + } + }, + ref: (names) => new RegExp(`(? rel === "package.json", + discover: (text) => [...text.matchAll(/"version":\s*"([^"]+)"/g)].map((m) => m[1]), + }, + { + // package.json identity fields — name + keywords. Low weight (keywords are common + // words) but a rename or keyword churn should still point at the docs that echo them. + type: "pkg-field", + weight: 0.4, + extract: (root) => { + try { + const pkg = JSON.parse(readFileSync(join(root, "package.json"), "utf8")); + const names = new Set(); + if (pkg.name) names.add(pkg.name); + for (const k of pkg.keywords ?? []) if (String(k).length >= 3) names.add(String(k)); + return [...names].map((name) => ({ name })); + } catch { + return []; + } + }, + ref: (names) => boundaryRe(names), + isSource: (rel) => rel === "package.json", + }, +]; + +/** + * The full current TYPED entity set, one flat list. Fail-open per extractor so a single + * broken source can't blank the whole graph. + * @param {string} [root] + * @returns {{type:string, name:string, weight:number, codeOnly:boolean, meta:object}[]} + */ +export function extractEntities(root = BRAND.root) { + const out = []; + const seen = new Set(); + for (const ex of EXTRACTORS) { + let items = []; + try { + items = ex.extract(root) ?? []; + } catch { + items = []; + } + for (const it of items) { + // A single-character name (e.g. a stray `process.env.X` in a comment) is + // indistinguishable from prose and never a real documented entity — drop it. + if (!it.name || String(it.name).length < 2) continue; + const key = `${ex.type} ${it.name}`; + if (seen.has(key)) continue; + seen.add(key); + out.push({ + type: ex.type, + name: it.name, + weight: ex.weight, + codeOnly: !!ex.codeOnly, + meta: it.meta ?? {}, + }); + } + } + return out; +} + +// --------------------------------------------------------------------------- +// Stage 2 — the generic, code-fence-aware reference index. +// --------------------------------------------------------------------------- + +/** Byte ranges of inline `code` spans on a single line (matched backtick runs). */ +function inlineCodeRanges(line) { + const ranges = []; + const re = /(`+)/g; + let m; + let openIdx = -1; + let openLen = 0; + while ((m = re.exec(line))) { + if (openIdx < 0) { + openIdx = m.index; + openLen = m[1].length; + } else if (m[1].length === openLen) { + ranges.push([openIdx, m.index + m[1].length]); + openIdx = -1; + openLen = 0; + } + } + return ranges; +} + +/** Every prose/doc SURFACE the index owes an answer for — discoverable, not a per-file + * allow-list: all tracked *.md and *.cff, the plugin manifests, the landing page, and + * package.json (its description/keywords are prose). Canonical .js sources and history + * files are excluded. Falls back to a fixed set when git is unavailable (tmp fixtures). */ +export function docSurfaces(root = BRAND.root) { + const set = new Set(); + const tracked = git(root, ["ls-files"]); + const list = tracked + ? tracked.split("\n").filter(Boolean) + : ["README.md", "docs/GUIDE.md", "ARCHITECTURE.md", "ROADMAP.md", "package.json"]; + for (const rel of list) { + if (HISTORY_RE.test(rel)) continue; + if (/\.(md|cff)$/i.test(rel)) set.add(rel); + } + for (const rel of [ + ".claude-plugin/plugin.json", + ".claude-plugin/marketplace.json", + ".codex-plugin/plugin.json", + "landing/index.html", + "package.json", + ]) + if (existsSync(join(root, rel))) set.add(rel); + return [...set].sort(); +} + +// GitHub-style heading slug is overkill here; the raw heading text is a good enough +// "section" label for a human reviewer. +const headingOf = (line) => { + const m = /^#{1,6}\s+(.+?)\s*#*\s*$/.exec(line); + return m ? m[1].trim() : null; +}; + +/** + * Build the inverted index: entityKey → occurrences. Generic over ANY entity list, so it + * indexes the current entities AND the reconstructed removed ones (old version, deleted + * command) the impact query needs. + * @param {string} root + * @param {{type:string,name:string,weight:number,codeOnly?:boolean}[]} entities + * @param {string[]} [surfaces] + * @returns {{index: Map, surfaces: string[]}} + */ +export function buildReferenceIndex(root, entities, surfaces = docSurfaces(root)) { + // One RegExp per (type) — an alternation of every name of that type — so a doc line is + // scanned once per type, not once per entity. group 1 is the matched name. + const byType = new Map(); + for (const e of entities) { + if (!byType.has(e.type)) byType.set(e.type, { spec: e, names: [] }); + byType.get(e.type).names.push(e.name); + } + const scanners = []; + for (const [type, { spec, names }] of byType) { + const ex = EXTRACTORS.find((x) => x.type === type); + if (!ex || !names.length) continue; + scanners.push({ type, spec, re: ex.ref(names) }); + } + + const index = new Map(); + const record = (type, name, occ) => { + const key = `${type} ${name}`; + if (!index.has(key)) index.set(key, []); + index.get(key).push(occ); + }; + + for (const rel of surfaces) { + let text = ""; + try { + text = readFileSync(join(root, rel), "utf8"); + } catch { + continue; + } + const lines = text.split("\n"); + let inFence = false; + let section = null; + for (let i = 0; i < lines.length; i += 1) { + const line = lines[i]; + if (/^\s*(```|~~~)/.test(line)) { + inFence = !inFence; + continue; // the fence marker itself is not content + } + if (!inFence) { + const h = headingOf(line); + if (h) section = h; + } + const codeRanges = inFence ? null : inlineCodeRanges(line); + const inCode = (pos) => + inFence || (codeRanges?.some(([a, b]) => pos >= a && pos < b) ?? false); + for (const { type, spec, re } of scanners) { + re.lastIndex = 0; + let m; + while ((m = re.exec(line))) { + const name = m[1]; + const pos = m.index + m[0].indexOf(name); + const code = inCode(pos); + if (spec.codeOnly && !code) continue; // word-shaped entity in plain prose — skip + record(type, name, { + file: rel, + line: i + 1, + section, + context: inFence ? "fence" : code ? "code" : "prose", + confidence: Math.min(1, spec.weight + (code ? 0.05 : 0)), + snippet: line.trim().slice(0, 160), + }); + } + } + } + } + return { index, surfaces }; +} + +// --------------------------------------------------------------------------- +// Stage 3 — change detection over canonical sources, then the impact query. +// --------------------------------------------------------------------------- + +/** Parse a unified diff into per-file added/removed line arrays. Handles new files + * (untracked, when not --staged) as all-added. */ +function diffHunks(root, { base, staged }) { + const args = staged + ? ["diff", "--unified=0", "--cached"] + : ["diff", "--unified=0", base || "HEAD"]; + const diff = git(root, args); + const perFile = new Map(); + const ensure = (rel) => { + if (!perFile.has(rel)) perFile.set(rel, { added: [], removed: [] }); + return perFile.get(rel); + }; + let current = null; + for (const line of diff.split("\n")) { + const add = /^\+\+\+ b\/(.*)$/.exec(line); + if (add) { + current = add[1] === "/dev/null" ? null : add[1]; + if (current) ensure(current); + continue; + } + const del = /^--- a\/(.*)$/.exec(line); + if (del && del[1] !== "/dev/null") ensure(del[1]); + if (!current) continue; + if (line.startsWith("+") && !line.startsWith("+++")) ensure(current).added.push(line.slice(1)); + else if (line.startsWith("-") && !line.startsWith("---")) + ensure(current).removed.push(line.slice(1)); + } + if (!staged) { + for (const f of git(root, ["ls-files", "--others", "--exclude-standard"]) + .split("\n") + .filter(Boolean)) { + if (perFile.has(f)) continue; + let text = ""; + try { + text = readFileSync(join(root, f), "utf8"); + } catch {} + if (text) perFile.set(f, { added: text.split("\n"), removed: [] }); + } + } + return perFile; +} + +/** + * The entities a diff changed, TYPED. For each type: (a) a current entity is CHANGED if + * its name-token appears on a changed line of one of that type's canonical sources; (b) a + * name the extractor can DISCOVER in that type's REMOVED lines but which is absent from + * the current set is a REMOVED entity (rename/deletion) — its old doc mentions are stale. + * Version bumps additionally recover the OLD version from package.json's removed lines. + * @param {string} root + * @param {{base?:string, staged?:boolean}} [opts] + * @returns {{type:string,name:string,reason:string,removed:boolean}[]} + */ +export function changedEntities(root, { base, staged } = {}) { + const hunks = diffHunks(root, { base, staged }); + const current = extractEntities(root); + const currentByType = new Map(); + for (const e of current) { + if (!currentByType.has(e.type)) currentByType.set(e.type, new Set()); + currentByType.get(e.type).add(e.name); + } + + const changed = []; + const emitted = new Set(); + const emit = (type, name, reason, removed) => { + if (!name || String(name).length < 2) return; + const key = `${type} ${name}`; + if (emitted.has(key)) return; + emitted.add(key); + changed.push({ type, name, reason, removed }); + }; + + for (const ex of EXTRACTORS) { + // Gather this type's changed source hunks. + let addedText = ""; + let removedText = ""; + let touched = false; + for (const [rel, { added, removed }] of hunks) { + if (!ex.isSource(rel)) continue; + touched = true; + addedText += `${added.join("\n")}\n`; + removedText += `${removed.join("\n")}\n`; + } + if (!touched) continue; + const changedText = `${addedText}\n${removedText}`; + const currentNames = currentByType.get(ex.type) ?? new Set(); + + // (a) current entities whose token is on a changed line. + for (const name of currentNames) { + const re = new RegExp(`(?:^|[^\\w$-])${esc(name)}(?![\\w$])`); + if (re.test(changedText)) emit(ex.type, name, "modified", false); + } + // (b) discovered names in removed/added text that no longer exist → renamed/removed. + if (ex.discover) { + const seen = new Set([...ex.discover(removedText), ...ex.discover(addedText)]); + for (const name of seen) if (!currentNames.has(name)) emit(ex.type, name, "removed", true); + } + } + return changed; +} + +/** + * The reusable core. Compute the changed entities, index every doc surface for BOTH the + * current and the removed entities, and return each changed entity with the doc locations + * that reference it — ranked by confidence. + * @param {string} root + * @param {{base?:string, staged?:boolean, minConfidence?:number, surfaces?:string[]}} [opts] + */ +export function docsImpact(root = BRAND.root, opts = {}) { + const { base, staged, minConfidence = 0, surfaces } = opts; + const changed = changedEntities(root, { base, staged }); + + // Index the CURRENT entities plus the reconstructed removed ones (so an old version or a + // deleted command still has doc occurrences to point at). + const entities = extractEntities(root); + const known = new Set(entities.map((e) => `${e.type} ${e.name}`)); + for (const c of changed) { + const key = `${c.type} ${c.name}`; + if (known.has(key)) continue; + known.add(key); + const ex = EXTRACTORS.find((x) => x.type === c.type); + entities.push({ + type: c.type, + name: c.name, + weight: ex?.weight ?? 0.5, + codeOnly: !!ex?.codeOnly, + meta: { removed: true }, + }); + } + + const { index, surfaces: scanned } = buildReferenceIndex( + root, + entities, + surfaces ?? docSurfaces(root), + ); + + const impacted = []; + for (const c of changed) { + const occ = (index.get(`${c.type} ${c.name}`) ?? []) + .filter((o) => o.confidence >= minConfidence) + .sort( + (a, b) => b.confidence - a.confidence || a.file.localeCompare(b.file) || a.line - b.line, + ); + if (occ.length) impacted.push({ ...c, occurrences: occ }); + } + // Entities with the most / highest-confidence hits first. + impacted.sort( + (a, b) => + (b.occurrences[0]?.confidence ?? 0) - (a.occurrences[0]?.confidence ?? 0) || + b.occurrences.length - a.occurrences.length, + ); + + const files = new Set(); + for (const e of impacted) for (const o of e.occurrences) files.add(o.file); + + return { + base: staged ? "--staged" : base || "HEAD", + changed, + impacted, + surfaces: scanned, + summary: { + changedEntities: changed.length, + impactedEntities: impacted.length, + impactedFiles: files.size, + occurrences: impacted.reduce((n, e) => n + e.occurrences.length, 0), + }, + }; +} + +/** Human rendering — grouped by changed entity, each hit cited file:line with confidence. */ +export function renderDocsImpact(report, { maxPerEntity = 8 } = {}) { + const out = [ + `docs impact — diff vs ${report.base}: ${report.summary.changedEntities} changed entit${ + report.summary.changedEntities === 1 ? "y" : "ies" + }, ${report.summary.impactedFiles} doc file(s) potentially stale`, + ]; + if (!report.changed.length) { + out.push(" clean — no documented entity changed in this diff."); + return out.join("\n"); + } + if (!report.impacted.length) { + out.push( + ` ${report.summary.changedEntities} entit${ + report.summary.changedEntities === 1 ? "y" : "ies" + } changed, but no doc surface references them — nothing to review.`, + ); + return out.join("\n"); + } + for (const e of report.impacted) { + const tag = e.removed ? "REMOVED" : "changed"; + out.push(` ${e.type} \`${e.name}\` (${tag}) — ${e.occurrences.length} reference(s):`); + for (const o of e.occurrences.slice(0, maxPerEntity)) { + const where = o.section ? ` [${o.section}]` : ""; + out.push( + ` ${o.file}:${o.line} (${o.context}, ${o.confidence.toFixed(2)})${where} ${o.snippet}`, + ); + } + if (e.occurrences.length > maxPerEntity) + out.push(` (+${e.occurrences.length - maxPerEntity} more)`); + } + out.push( + " → review each surface: update it, or confirm the mention is still correct. Advisory — image/design/paraphrase impact is not detectable by a token scan.", + ); + return out.join("\n"); +} diff --git a/test/docs_impact.test.js b/test/docs_impact.test.js new file mode 100644 index 0000000..4ed8e14 --- /dev/null +++ b/test/docs_impact.test.js @@ -0,0 +1,257 @@ +import assert from "node:assert/strict"; +import { execFileSync } from "node:child_process"; +import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { test } from "node:test"; +import { COMMANDS } from "../src/commands.js"; +import { + buildReferenceIndex, + changedEntities, + docSurfaces, + docsImpact, + EXTRACTORS, + extractEntities, +} from "../src/docs_impact.js"; +import { TOOLS } from "../src/mcp_tools.js"; + +// A real git repo — docs impact reads git diffs, so the pipeline can't be exercised +// against a bare directory. Writes `files`, commits them as the baseline, and returns +// helpers to mutate + inspect. +function repo(files = {}) { + const root = mkdtempSync(join(tmpdir(), "forge-impact-")); + const g = (...args) => execFileSync("git", args, { cwd: root, encoding: "utf8" }); + const write = (rel, content) => { + const p = join(root, rel); + mkdirSync(join(p, ".."), { recursive: true }); + writeFileSync(p, content); + }; + for (const [rel, content] of Object.entries(files)) write(rel, content); + g("init", "-q"); + g("config", "user.email", "t@t.t"); + g("config", "user.name", "t"); + g("add", "-A"); + g("commit", "-qm", "baseline"); + return { root, g, write }; +} + +// A directory (no git) for pure reference-index unit tests where we pass surfaces explicitly. +function scratch(files = {}) { + const root = mkdtempSync(join(tmpdir(), "forge-idx-")); + for (const [rel, content] of Object.entries(files)) { + const p = join(root, rel); + mkdirSync(join(p, ".."), { recursive: true }); + writeFileSync(p, content); + } + return root; +} + +// --- Stage 1: entity extraction ------------------------------------------------------ + +test("extractEntities: the registry yields the typed entity set from canonical sources", () => { + const { root } = repo({ + "package.json": JSON.stringify({ + name: "widget", + version: "3.1.0", + keywords: ["forge-test"], + }), + "src/app.js": "const m = process.env.FORGE_TEST_KNOB;\nexport function runWidget() {}\n", + }); + const ents = extractEntities(root); + const by = (type) => ents.filter((e) => e.type === type).map((e) => e.name); + + // Root-parametrized types read the TARGET repo. + assert.ok(by("env").includes("FORGE_TEST_KNOB"), "env var from process.env scan"); + assert.ok(by("symbol").includes("runWidget"), "exported symbol"); + assert.ok(by("version").includes("3.1.0"), "version from the repo's package.json"); + assert.ok(by("pkg-field").includes("widget"), "package name"); + + // Registry-backed types come from forge's own COMMANDS / TOOLS (self-check semantics). + assert.ok(by("command").includes(Object.keys(COMMANDS)[0]), "a command name"); + assert.ok(by("mcp-tool").includes(TOOLS[0].name), "an MCP tool name"); +}); + +test("extractEntities: single-character names are dropped (no `process.env.X` prose noise)", () => { + const root = scratch({ "src/a.js": "const x = process.env.X;\n" }); + const ents = extractEntities(root); + assert.ok(!ents.some((e) => e.name.length < 2)); +}); + +test("EXTRACTORS is data-driven — a new entity type is one record, no call-site edits", () => { + // Every extractor exposes the same shape, so the pipeline never special-cases a type. + for (const ex of EXTRACTORS) { + assert.equal(typeof ex.type, "string"); + assert.equal(typeof ex.extract, "function"); + assert.equal(typeof ex.ref, "function"); + assert.equal(typeof ex.isSource, "function"); + } +}); + +// --- Stage 2: reference index (word boundaries + code fences) ------------------------- + +test("buildReferenceIndex: word-boundary aware — a longer token is not a match", () => { + const root = scratch({ + "README.md": [ + "# t", + "Run `forge docs` to check. But forge docsxyz is unrelated.", + "Set FORGE_ABC now; FORGE_ABCDEF is a different var.", + ].join("\n"), + }); + const entities = [ + { type: "command", name: "docs", weight: 0.9 }, + { type: "env", name: "FORGE_ABC", weight: 0.9 }, + ]; + const { index } = buildReferenceIndex(root, entities, ["README.md"]); + assert.equal(index.get("command docs")?.length, 1, "matches `forge docs`, not `forge docsxyz`"); + assert.equal(index.get("env FORGE_ABC")?.length, 1, "matches FORGE_ABC, not FORGE_ABCDEF"); +}); + +test("buildReferenceIndex: code-fence aware — a word-shaped symbol only counts in code", () => { + const root = scratch({ + "README.md": [ + "# t", + "The word build appears in plain prose here.", // prose → must NOT count + "Call `build` to compile.", // inline code → counts + "```js", + "build();", // fenced code → counts + "```", + ].join("\n"), + }); + const entities = [{ type: "symbol", name: "build", weight: 0.6, codeOnly: true }]; + const { index } = buildReferenceIndex(root, entities, ["README.md"]); + const hits = index.get("symbol build") ?? []; + assert.equal(hits.length, 2, "only the inline-code and fenced occurrences"); + assert.ok(hits.every((h) => h.context !== "prose")); +}); + +test("buildReferenceIndex: records section, line, and a confidence bonus inside code", () => { + const root = scratch({ + "README.md": ["## Setup", "Set FORGE_ABC in your shell.", "Or `FORGE_ABC` inline."].join("\n"), + }); + const { index } = buildReferenceIndex( + root, + [{ type: "env", name: "FORGE_ABC", weight: 0.9 }], + ["README.md"], + ); + const hits = index.get("env FORGE_ABC"); + assert.equal(hits.length, 2); + assert.equal(hits[0].section, "Setup"); + assert.equal(hits[0].line, 2); + const code = hits.find((h) => h.context === "code"); + const prose = hits.find((h) => h.context === "prose"); + assert.ok(code.confidence > prose.confidence, "inline-code hit is more confident"); +}); + +// --- docSurfaces discovery ----------------------------------------------------------- + +test("docSurfaces: discovers markdown but exempts CHANGELOG (append-only history)", () => { + const { root } = repo({ + "README.md": "# r\n", + "docs/GUIDE.md": "# g\n", + "CHANGELOG.md": "# c\n", + "package.json": JSON.stringify({ name: "w", version: "1.0.0" }), + }); + const surfaces = docSurfaces(root); + assert.ok(surfaces.includes("README.md")); + assert.ok(surfaces.includes("docs/GUIDE.md")); + assert.ok(surfaces.includes("package.json")); + assert.ok(!surfaces.includes("CHANGELOG.md"), "history is exempt"); +}); + +// --- Stage 3: change detection + impact query ---------------------------------------- + +test("impact: a removed env var flags the README that documents it", () => { + const { root, write } = repo({ + "package.json": JSON.stringify({ name: "w", version: "1.0.0" }), + "src/app.js": "const m = process.env.FORGE_TEST_KNOB;\nexport function go() {}\n", + "README.md": "# w\n\nSet `FORGE_TEST_KNOB` to configure the widget.\n", + }); + write("src/app.js", "export function go() {}\n"); // remove the env read + const changed = changedEntities(root, { base: "HEAD" }); + assert.ok( + changed.some((c) => c.type === "env" && c.name === "FORGE_TEST_KNOB" && c.removed), + "env var detected as removed", + ); + const r = docsImpact(root, { base: "HEAD" }); + const env = r.impacted.find((e) => e.type === "env" && e.name === "FORGE_TEST_KNOB"); + assert.ok(env, "env var is in the impacted set"); + assert.ok(env.occurrences.some((o) => o.file === "README.md")); +}); + +test("impact: a version bump flags docs that hardcode the OLD version", () => { + const { root, write } = repo({ + "package.json": JSON.stringify({ name: "w", version: "1.0.0" }, null, 2), + "src/app.js": "export function go() {}\n", + "README.md": "# w — v1.0.0\n\nUpgrade guide for v1.0.0.\n", + }); + write("package.json", JSON.stringify({ name: "w", version: "2.0.0" }, null, 2)); + const r = docsImpact(root, { base: "HEAD" }); + const oldV = r.impacted.find((e) => e.type === "version" && e.name === "1.0.0"); + assert.ok(oldV, "the OLD version is a changed (removed) entity"); + assert.ok( + oldV.occurrences.some((o) => o.file === "README.md"), + "README's v1.0.0 flagged", + ); + assert.ok( + r.impacted.some((e) => e.type === "version" && e.name === "2.0.0"), + "new version too", + ); +}); + +test("impact: a renamed command (commands.js diff) flags its old doc mentions", () => { + // The command extractor discovers table keys in a changed src/commands.js. A key that + // no longer exists in forge's COMMANDS registry is treated as removed/renamed. + const { root, write } = repo({ + "package.json": JSON.stringify({ name: "w", version: "1.0.0" }), + "src/commands.js": "export const COMMANDS = {\n frobnicate: 'do a thing',\n};\n", + "README.md": "# w\n\nRun `forge frobnicate` to do the thing.\n", + }); + write("src/commands.js", "export const COMMANDS = {\n frobulate: 'do a thing',\n};\n"); + const changed = changedEntities(root, { base: "HEAD" }); + assert.ok( + changed.some((c) => c.type === "command" && c.name === "frobnicate" && c.removed), + "the removed command key is detected", + ); + const r = docsImpact(root, { base: "HEAD" }); + const cmd = r.impacted.find((e) => e.type === "command" && e.name === "frobnicate"); + assert.ok(cmd?.occurrences.some((o) => o.file === "README.md")); +}); + +test("impact: a diff that changes no documented entity yields nothing to review", () => { + const { root, write } = repo({ + "package.json": JSON.stringify({ name: "w", version: "1.0.0" }), + "src/app.js": "export function go() {}\n", + "README.md": "# w\n\nHello.\n", + }); + write("notes.txt", "just some scratch prose, no entities\n"); + const r = docsImpact(root, { base: "HEAD" }); + assert.equal(r.impacted.length, 0); +}); + +test("impact: a changed entity mentioned ONLY in CHANGELOG is not reported (history)", () => { + const { root, write } = repo({ + "package.json": JSON.stringify({ name: "w", version: "1.0.0" }), + "src/app.js": "const m = process.env.FORGE_TEST_KNOB;\nexport function go() {}\n", + "CHANGELOG.md": "# Changelog\n\n## [1.0.0]\n\n- added FORGE_TEST_KNOB\n", + }); + write("src/app.js", "export function go() {}\n"); + const r = docsImpact(root, { base: "HEAD" }); + assert.ok( + !r.impacted.some((e) => e.occurrences.some((o) => o.file === "CHANGELOG.md")), + "CHANGELOG mentions are exempt", + ); +}); + +test("docsImpact: --staged reads the index, and the report shape is stable", () => { + const { root, g, write } = repo({ + "package.json": JSON.stringify({ name: "w", version: "1.0.0" }), + "src/app.js": "const m = process.env.FORGE_TEST_KNOB;\nexport function go() {}\n", + "README.md": "# w\n\nSet `FORGE_TEST_KNOB`.\n", + }); + write("src/app.js", "export function go() {}\n"); + g("add", "src/app.js"); + const r = docsImpact(root, { staged: true }); + assert.equal(r.base, "--staged"); + assert.ok("summary" in r && "surfaces" in r && Array.isArray(r.changed)); + assert.ok(r.impacted.some((e) => e.type === "env" && e.name === "FORGE_TEST_KNOB")); +});