From a2968f5236ae93bdc2b31490b34ddd73e54ebf90 Mon Sep 17 00:00:00 2001 From: SunsetDrifter Date: Sun, 26 Jul 2026 12:52:21 +0200 Subject: [PATCH 1/3] refactor: consolidate to a single wiki template Retire the homelab, codebase, and codebase-large variants and rename generic/ to wiki/. Four templates with no real installs each meant every change cost 4x (CLAUDE.md, lint config, skills wrappers, byte-parity tests) for hypothetical users. The variants remain in git history, their lint configs live on under tests/fixtures/ as engine coverage for the extension knobs no shipped template currently sets (membership, staleness, sync_drift, coverage), and the engine keeps those knobs, so a specialized schema remains a lint.py + CLAUDE.md away. Also replaces a vendor-specific example secret pattern in the tests with a neutral one. - tests: VARIANTS collapses to [wiki]; cross-variant byte-parity and divergence tests removed; 91 tests green - README rewritten for the single-template layout --- README.md | 31 +- .../.claude/skills/wiki-adr/SKILL.md | 6 - .../.claude/skills/wiki-document/SKILL.md | 6 - .../.claude/skills/wiki-owner-review/SKILL.md | 6 - .../.claude/skills/wiki-postmortem/SKILL.md | 6 - .../.claude/skills/wiki-query/SKILL.md | 6 - .../.claude/skills/wiki-reconcile/SKILL.md | 6 - .../.claude/skills/wiki-sync/SKILL.md | 6 - .../.claude/skills/wiki-triage/SKILL.md | 6 - .../.claude/skills/wiki-verify/SKILL.md | 6 - codebase-large/CLAUDE.md | 144 ---- codebase-large/taxonomy.md | 34 - codebase-large/wikilint/__init__.py | 7 - codebase-large/workflows/adr.md | 27 - codebase-large/workflows/document.md | 41 -- codebase-large/workflows/lint.md | 39 - codebase-large/workflows/maintain.md | 33 - codebase-large/workflows/owner-review.md | 25 - codebase-large/workflows/postmortem.md | 26 - codebase-large/workflows/query.md | 24 - codebase-large/workflows/sync.md | 25 - codebase-large/workflows/triage.md | 24 - codebase-large/workflows/verify.md | 30 - codebase/.claude/skills/wiki-adr/SKILL.md | 6 - .../.claude/skills/wiki-document/SKILL.md | 6 - codebase/.claude/skills/wiki-lint/SKILL.md | 6 - .../.claude/skills/wiki-maintain/SKILL.md | 6 - .../.claude/skills/wiki-postmortem/SKILL.md | 6 - codebase/.claude/skills/wiki-query/SKILL.md | 6 - .../.claude/skills/wiki-reconcile/SKILL.md | 6 - codebase/.claude/skills/wiki-sync/SKILL.md | 6 - codebase/.claude/skills/wiki-triage/SKILL.md | 6 - codebase/.claude/skills/wiki-verify/SKILL.md | 6 - codebase/.githooks/pre-commit | 18 - codebase/CLAUDE.md | 123 ---- codebase/taxonomy.md | 31 - codebase/wikilint/__init__.py | 7 - codebase/wikilint/checks.py | 671 ------------------ codebase/wikilint/cli.py | 90 --- codebase/wikilint/derived.py | 249 ------- codebase/wikilint/model.py | 419 ----------- codebase/wikilint/settings.py | 120 ---- codebase/workflows/adr.md | 24 - codebase/workflows/document.md | 38 - codebase/workflows/lint.md | 34 - codebase/workflows/postmortem.md | 25 - codebase/workflows/query.md | 24 - codebase/workflows/reconcile.md | 25 - codebase/workflows/sync.md | 26 - codebase/workflows/triage.md | 24 - codebase/workflows/verify.md | 24 - generic/.claude/skills/wiki-lint/SKILL.md | 6 - generic/.claude/skills/wiki-maintain/SKILL.md | 6 - generic/.githooks/pre-commit | 18 - generic/wikilint/__init__.py | 7 - generic/wikilint/checks.py | 671 ------------------ generic/wikilint/cli.py | 90 --- generic/wikilint/derived.py | 249 ------- generic/wikilint/model.py | 419 ----------- generic/wikilint/settings.py | 120 ---- generic/workflows/maintain.md | 33 - generic/workflows/reconcile.md | 25 - homelab/.claude/skills/wiki-document/SKILL.md | 6 - homelab/.claude/skills/wiki-incident/SKILL.md | 6 - homelab/.claude/skills/wiki-lint/SKILL.md | 6 - homelab/.claude/skills/wiki-maintain/SKILL.md | 6 - homelab/.claude/skills/wiki-query/SKILL.md | 6 - .../.claude/skills/wiki-reconcile/SKILL.md | 6 - homelab/.claude/skills/wiki-triage/SKILL.md | 6 - homelab/.claude/skills/wiki-verify/SKILL.md | 6 - homelab/.githooks/pre-commit | 18 - homelab/CLAUDE.md | 134 ---- homelab/taxonomy.md | 27 - homelab/wikilint/__init__.py | 7 - homelab/wikilint/checks.py | 671 ------------------ homelab/wikilint/cli.py | 90 --- homelab/wikilint/derived.py | 249 ------- homelab/wikilint/model.py | 419 ----------- homelab/wikilint/settings.py | 120 ---- homelab/workflows/document.md | 38 - homelab/workflows/incident.md | 25 - homelab/workflows/lint.md | 32 - homelab/workflows/maintain.md | 33 - homelab/workflows/query.md | 23 - homelab/workflows/reconcile.md | 25 - homelab/workflows/triage.md | 24 - homelab/workflows/verify.md | 25 - .../fixtures/codebase-large}/lint.py | 0 {codebase => tests/fixtures/codebase}/lint.py | 0 {homelab => tests/fixtures/homelab}/lint.py | 0 tests/helpers.py | 18 +- tests/test_checks.py | 62 +- tests/test_variants.py | 58 +- .../.claude/skills/wiki-ingest/SKILL.md | 0 .../.claude/skills/wiki-lint/SKILL.md | 0 .../.claude/skills/wiki-maintain/SKILL.md | 0 .../.claude/skills/wiki-query/SKILL.md | 0 .../.claude/skills/wiki-reconcile/SKILL.md | 0 .../.claude/skills/wiki-triage/SKILL.md | 0 {codebase-large => wiki}/.githooks/pre-commit | 0 {generic => wiki}/CLAUDE.md | 0 {generic => wiki}/lint.py | 10 +- {generic => wiki}/taxonomy.md | 0 wiki/wikilint/__init__.py | 7 + {codebase-large => wiki}/wikilint/checks.py | 0 {codebase-large => wiki}/wikilint/cli.py | 0 {codebase-large => wiki}/wikilint/derived.py | 0 {codebase-large => wiki}/wikilint/model.py | 0 {codebase-large => wiki}/wikilint/settings.py | 0 {generic => wiki}/workflows/ingest.md | 0 {generic => wiki}/workflows/lint.md | 0 {codebase => wiki}/workflows/maintain.md | 0 {generic => wiki}/workflows/query.md | 0 .../workflows/reconcile.md | 0 {generic => wiki}/workflows/triage.md | 0 115 files changed, 85 insertions(+), 6318 deletions(-) delete mode 100644 codebase-large/.claude/skills/wiki-adr/SKILL.md delete mode 100644 codebase-large/.claude/skills/wiki-document/SKILL.md delete mode 100644 codebase-large/.claude/skills/wiki-owner-review/SKILL.md delete mode 100644 codebase-large/.claude/skills/wiki-postmortem/SKILL.md delete mode 100644 codebase-large/.claude/skills/wiki-query/SKILL.md delete mode 100644 codebase-large/.claude/skills/wiki-reconcile/SKILL.md delete mode 100644 codebase-large/.claude/skills/wiki-sync/SKILL.md delete mode 100644 codebase-large/.claude/skills/wiki-triage/SKILL.md delete mode 100644 codebase-large/.claude/skills/wiki-verify/SKILL.md delete mode 100644 codebase-large/CLAUDE.md delete mode 100644 codebase-large/taxonomy.md delete mode 100644 codebase-large/wikilint/__init__.py delete mode 100644 codebase-large/workflows/adr.md delete mode 100644 codebase-large/workflows/document.md delete mode 100644 codebase-large/workflows/lint.md delete mode 100644 codebase-large/workflows/maintain.md delete mode 100644 codebase-large/workflows/owner-review.md delete mode 100644 codebase-large/workflows/postmortem.md delete mode 100644 codebase-large/workflows/query.md delete mode 100644 codebase-large/workflows/sync.md delete mode 100644 codebase-large/workflows/triage.md delete mode 100644 codebase-large/workflows/verify.md delete mode 100644 codebase/.claude/skills/wiki-adr/SKILL.md delete mode 100644 codebase/.claude/skills/wiki-document/SKILL.md delete mode 100644 codebase/.claude/skills/wiki-lint/SKILL.md delete mode 100644 codebase/.claude/skills/wiki-maintain/SKILL.md delete mode 100644 codebase/.claude/skills/wiki-postmortem/SKILL.md delete mode 100644 codebase/.claude/skills/wiki-query/SKILL.md delete mode 100644 codebase/.claude/skills/wiki-reconcile/SKILL.md delete mode 100644 codebase/.claude/skills/wiki-sync/SKILL.md delete mode 100644 codebase/.claude/skills/wiki-triage/SKILL.md delete mode 100644 codebase/.claude/skills/wiki-verify/SKILL.md delete mode 100755 codebase/.githooks/pre-commit delete mode 100644 codebase/CLAUDE.md delete mode 100644 codebase/taxonomy.md delete mode 100644 codebase/wikilint/__init__.py delete mode 100644 codebase/wikilint/checks.py delete mode 100644 codebase/wikilint/cli.py delete mode 100644 codebase/wikilint/derived.py delete mode 100644 codebase/wikilint/model.py delete mode 100644 codebase/wikilint/settings.py delete mode 100644 codebase/workflows/adr.md delete mode 100644 codebase/workflows/document.md delete mode 100644 codebase/workflows/lint.md delete mode 100644 codebase/workflows/postmortem.md delete mode 100644 codebase/workflows/query.md delete mode 100644 codebase/workflows/reconcile.md delete mode 100644 codebase/workflows/sync.md delete mode 100644 codebase/workflows/triage.md delete mode 100644 codebase/workflows/verify.md delete mode 100644 generic/.claude/skills/wiki-lint/SKILL.md delete mode 100644 generic/.claude/skills/wiki-maintain/SKILL.md delete mode 100755 generic/.githooks/pre-commit delete mode 100644 generic/wikilint/__init__.py delete mode 100644 generic/wikilint/checks.py delete mode 100644 generic/wikilint/cli.py delete mode 100644 generic/wikilint/derived.py delete mode 100644 generic/wikilint/model.py delete mode 100644 generic/wikilint/settings.py delete mode 100644 generic/workflows/maintain.md delete mode 100644 generic/workflows/reconcile.md delete mode 100644 homelab/.claude/skills/wiki-document/SKILL.md delete mode 100644 homelab/.claude/skills/wiki-incident/SKILL.md delete mode 100644 homelab/.claude/skills/wiki-lint/SKILL.md delete mode 100644 homelab/.claude/skills/wiki-maintain/SKILL.md delete mode 100644 homelab/.claude/skills/wiki-query/SKILL.md delete mode 100644 homelab/.claude/skills/wiki-reconcile/SKILL.md delete mode 100644 homelab/.claude/skills/wiki-triage/SKILL.md delete mode 100644 homelab/.claude/skills/wiki-verify/SKILL.md delete mode 100755 homelab/.githooks/pre-commit delete mode 100644 homelab/CLAUDE.md delete mode 100644 homelab/taxonomy.md delete mode 100644 homelab/wikilint/__init__.py delete mode 100644 homelab/wikilint/checks.py delete mode 100644 homelab/wikilint/cli.py delete mode 100644 homelab/wikilint/derived.py delete mode 100644 homelab/wikilint/model.py delete mode 100644 homelab/wikilint/settings.py delete mode 100644 homelab/workflows/document.md delete mode 100644 homelab/workflows/incident.md delete mode 100644 homelab/workflows/lint.md delete mode 100644 homelab/workflows/maintain.md delete mode 100644 homelab/workflows/query.md delete mode 100644 homelab/workflows/reconcile.md delete mode 100644 homelab/workflows/triage.md delete mode 100644 homelab/workflows/verify.md rename {codebase-large => tests/fixtures/codebase-large}/lint.py (100%) rename {codebase => tests/fixtures/codebase}/lint.py (100%) rename {homelab => tests/fixtures/homelab}/lint.py (100%) rename {generic => wiki}/.claude/skills/wiki-ingest/SKILL.md (100%) rename {codebase-large => wiki}/.claude/skills/wiki-lint/SKILL.md (100%) rename {codebase-large => wiki}/.claude/skills/wiki-maintain/SKILL.md (100%) rename {generic => wiki}/.claude/skills/wiki-query/SKILL.md (100%) rename {generic => wiki}/.claude/skills/wiki-reconcile/SKILL.md (100%) rename {generic => wiki}/.claude/skills/wiki-triage/SKILL.md (100%) rename {codebase-large => wiki}/.githooks/pre-commit (100%) rename {generic => wiki}/CLAUDE.md (100%) rename {generic => wiki}/lint.py (91%) rename {generic => wiki}/taxonomy.md (100%) create mode 100644 wiki/wikilint/__init__.py rename {codebase-large => wiki}/wikilint/checks.py (100%) rename {codebase-large => wiki}/wikilint/cli.py (100%) rename {codebase-large => wiki}/wikilint/derived.py (100%) rename {codebase-large => wiki}/wikilint/model.py (100%) rename {codebase-large => wiki}/wikilint/settings.py (100%) rename {generic => wiki}/workflows/ingest.md (100%) rename {generic => wiki}/workflows/lint.md (100%) rename {codebase => wiki}/workflows/maintain.md (100%) rename {generic => wiki}/workflows/query.md (100%) rename {codebase-large => wiki}/workflows/reconcile.md (100%) rename {generic => wiki}/workflows/triage.md (100%) diff --git a/README.md b/README.md index 6beed5f..709346e 100644 --- a/README.md +++ b/README.md @@ -1,29 +1,26 @@ # Scriptorium -Schemas for LLM-maintained markdown wikis. Each variant is a self-contained template you copy into a new wiki repo: an always-loaded `CLAUDE.md` core, on-demand `workflows/` procedures, and a stdlib-only lint tool (`lint.py` holds the variant's configuration; the `wikilint/` package holds the shared engine, byte-identical across variants) that handles every mechanical health check so the model only spends judgment where judgment is needed. +A schema for LLM-maintained markdown wikis. `wiki/` is a self-contained template you copy into a new wiki repo: an always-loaded `CLAUDE.md` core, on-demand `workflows/` procedures, and a stdlib-only lint tool (`lint.py` holds the configuration; the `wikilint/` package holds the engine) that handles every mechanical health check so the model only spends judgment where judgment is needed. -Every wiki produced from these templates is a conformant [Open Knowledge Format (OKF) v0.2](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) bundle, so it can be consumed by any OKF-aware agent or tool without translation. +Every wiki produced from this template is a conformant [Open Knowledge Format (OKF) v0.2](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) bundle, so it can be consumed by any OKF-aware agent or tool without translation. ## Where this sits in the OKF ecosystem -Most OKF tooling generates wikis: point a compiler at a codebase, a dataset, or a pile of documents and get a bundle. Scriptorium solves the other half of the problem, keeping a wiki true after week one. Its templates make an agent the accountable maintainer of the bundle: ownership rules, an append-only log, a commit per operation, and a deterministic linter that enforces freshness, contested-claim reconciliation, link integrity, and OKF conformance on every commit. Generated wikis are impressive on day one; these schemas are for knowledge that cannot be regenerated from a source of truth and has to still be right in month six. +Most OKF tooling generates wikis: point a compiler at a codebase, a dataset, or a pile of documents and get a bundle. Scriptorium solves the other half of the problem, keeping a wiki true after week one. Its template makes an agent the accountable maintainer of the bundle: ownership rules, an append-only log, a commit per operation, and a deterministic linter that enforces freshness, contested-claim reconciliation, link integrity, and OKF conformance on every commit. Generated wikis are impressive on day one; this schema is for knowledge that cannot be regenerated from a source of truth and has to still be right in month six. One layering note when comparing bundles across producers: OKF reserves two filenames (`index.md`, `log.md`) and gives directory names no meaning. Folder taxonomies like `concepts/` or `synthesis/` here, or `tables/` and `references/` in Google's sample bundles, are producer conventions; the portable signal for consumers is the frontmatter `type` field. -## Variants +## The template -| Variant | For | Distinctive machinery | -|---|---|---| -| `generic/` | Reading notes, research, any source-driven knowledge base | sources / entities / concepts / synthesis / queries | -| `homelab/` | Home infrastructure documentation | component pages, topology diagrams, `last_verified` reality checks, incidents, runbooks | -| `codebase/` | A single repo, one or a few owners | commit-pinned pages, sync/verify loop, ADRs, postmortems | -| `codebase-large/` | 200k+ lines, multiple services and owners | subsystem sharding, criticality-weighted verification, owner review, coverage reports | +`wiki/` organizes knowledge as sources / entities / concepts / synthesis / queries, fed from an immutable, human-owned `raw/` inbox. It suits reading notes, research, and any source-driven knowledge base. + +Earlier revisions shipped three sibling variants (homelab, codebase, codebase-large) with specialized schemas on the same engine. They were retired in favor of one well-tested template (their configs live on under `tests/fixtures/` as engine coverage, and in git history if you want to resurrect one); the engine keeps the extension knobs they exercised, so a specialized schema is a `lint.py` + `CLAUDE.md` away. ## Install ```sh mkdir my-wiki && cd my-wiki && git init -cp -r path/to/Scriptorium// . +cp -r path/to/Scriptorium/wiki/ . git config core.hooksPath .githooks # lint errors become uncommittable mkdir -p raw/inbox python3 lint.py check # should pass on the empty wiki @@ -33,22 +30,22 @@ Then open the wiki in Claude Code (or any agent that reads `CLAUDE.md`) and drop ## Design -The schemas follow a few rules, informed by how agent-maintained wikis actually fail: +The schema follows a few rules, informed by how agent-maintained wikis actually fail: - **Hot/cold split.** `CLAUDE.md` stays around a hundred lines (directives, layout, frontmatter schemas, a dispatch table). Step-by-step workflows live in `workflows/` and are read when triggered. Always-loaded context is a cost paid on every turn; verbose context files measurably hurt. - **If a rule can be a script, it is a script.** `lint.py check` covers frontmatter validation, broken links, orphans, dangling references, staleness, secrets, ADR integrity, index drift, log format, and OKF conformance. The LLM lint workflow keeps only what needs judgment: contradictions, stale claims, concept gaps. -- **Every relationship is stored once.** Pages declare their forward edges (`depends_on`, `defined_in`, `consumes`, `producers`, `consumers`, ...); `lint.py reverse-deps` derives the reverse map of every one of them. Agents reliably fail to keep hand-maintained symmetric fields consistent, so the schemas don't ask them to. +- **Every relationship is stored once.** Pages declare their forward edges (`depends_on`, `defined_in`, `consumes`, `producers`, `consumers`, ...); `lint.py reverse-deps` derives the reverse map of every one of them. Agents reliably fail to keep hand-maintained symmetric fields consistent, so the schema doesn't ask them to. - **Workflows double as skills.** Each workflow has a thin `.claude/skills/wiki-/SKILL.md` wrapper, so `/wiki-triage`, `/wiki-document`, etc. dispatch deterministically in Claude Code (the prefix avoids collisions with globally installed skills). Wrappers contain no procedure text, only a pointer; `check_skills` makes wrapper/workflow drift a lint error. - **The index is a derived artifact.** Every page carries a one-line `description:`; `lint.py rebuild-index` generates `index.md` from frontmatter. Agents scan frontmatter first and open page bodies only on a hit. - **Trust is structural, not remembered.** Immutable `raw/`, append-only `log.md`, a git commit per operation, a pre-commit hook that lints the staged snapshot (exactly the bytes that will land) and makes lint errors uncommittable, and commit-pinned verification fields. Agent-maintained judgment metadata decays, so `confidence:` is reduced to the two states a lint can actually check (`low` = uncited, `contested` = sources disagree and the body must explain), inferred claims are marked inline with `(inferred)`, and tags are validated against a `taxonomy.md`, whose `## Page types` section also describes every allowed page type with a one-line meaning, so each bundle self-describes its type vocabulary to OKF consumers. - **Contested is a state to exit.** The documented failure mode of agent wikis is contradictions accumulating faster than they resolve. Lint flags contested pages older than 30 days; the reconcile workflow rewrites in place, moving losing claims to a dated "Superseded claims" section instead of deleting them. - **Autonomous but reversible.** The maintenance workflow runs unattended on a `maintenance` branch with an exhaustively-listed set of safe actions (mechanical fixes, index rebuild, unambiguous cross-links); everything else becomes a proposal. The human reviews the branch diff and merges. Nothing automated ever lands on main directly. -- **Native OKF conformance.** Every wiki is an [OKF v0.2](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) bundle: markdown files with YAML frontmatter, ordinary markdown links as the edge form (bundle-absolute `[title](/dir/page.md)`, the form v0.2 §6.1 recommends), reserved `index.md` (stamped with `okf_version` frontmatter by `rebuild-index`, the mechanism §12 specifies) and `log.md` (date-grouped headings, bold-action-word entries). `check_okf` enforces the spec's three conformance rules: parseable frontmatter on every non-reserved `.md`, a non-empty `type`, and reserved-file structure. Provenance uses the spec's `sources` shape (§5.1): mapping entries with a required `resource`, enforced by `check_sources`, with pre-0.2 string entries downgraded to warnings. Scriptorium's schemas are a strict superset of OKF's (the spec's `generated.at` is our `updated`; its `resource` is our `source_path`; its `title` is optional, with the index prettifying filenames when absent). One deliberate deviation: `raw/` is excluded from conformance because prime directive 1 makes those sources immutable, and OKF has no concept of a non-concept directory. -- **Opt-in extension points for non-wiki trees.** The engine can also lint markdown trees that aren't wikis (a findings folder, a labs journal): `okf_conformance: False` turns off the OKF rules for trees that aren't bundles, `non_page_allowed` accepts glob patterns, `index_file`/`index_body_fn` relocate and reshape the generated index (`index_file: None` disables it), `extra_secret_patterns`/`secret_allow_res` extend the secrets scan, and `extra_checks` runs custom callables. Every knob's default lives once in `wikilint/settings.py` (`DEFAULTS`) and preserves the original behavior; a variant's `lint.py` lists a key only to override it. Bad values (a malformed regex, an out-of-tree `index_file`, a non-callable check) are rejected at startup with a clear message rather than a mid-run traceback. +- **Native OKF conformance.** Every wiki is an [OKF v0.2](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) bundle: markdown files with YAML frontmatter, ordinary markdown links as the edge form (bundle-absolute `[title](/dir/page.md)`, the form v0.2 §6.1 recommends), reserved `index.md` (stamped with `okf_version` frontmatter by `rebuild-index`, the mechanism §12 specifies) and `log.md` (date-grouped headings, bold-action-word entries). `check_okf` enforces the spec's three conformance rules: parseable frontmatter on every non-reserved `.md`, a non-empty `type`, and reserved-file structure. Provenance uses the spec's `sources` shape (§5.1): mapping entries with a required `resource`, enforced by `check_sources`, with pre-0.2 string entries downgraded to warnings. Scriptorium's schema is a strict superset of OKF's (the spec's `generated.at` is our `updated`; its `resource` is our `source_path`; its `title` is optional, with the index prettifying filenames when absent). One deliberate deviation: `raw/` is excluded from conformance because prime directive 1 makes those sources immutable, and OKF has no concept of a non-concept directory. +- **Opt-in extension points for non-wiki trees.** The engine can also lint markdown trees that aren't wikis (a findings folder, a labs journal): `okf_conformance: False` turns off the OKF rules for trees that aren't bundles, `non_page_allowed` accepts glob patterns, `index_file`/`index_body_fn` relocate and reshape the generated index (`index_file: None` disables it), `extra_secret_patterns`/`secret_allow_res` extend the secrets scan, and `extra_checks` runs custom callables. Every knob's default lives once in `wikilint/settings.py` (`DEFAULTS`) and preserves the original behavior; a wiki's `lint.py` lists a key only to override it. Bad values (a malformed regex, an out-of-tree `index_file`, a non-callable check) are rejected at startup with a clear message rather than a mid-run traceback. ## Unattended maintenance -Each variant ships a `workflows/maintain.md` pass designed for scheduled runs. Example with Claude Code on cron: +The template ships a `workflows/maintain.md` pass designed for scheduled runs. Example with Claude Code on cron: ```sh # nightly, from the wiki root @@ -73,4 +70,4 @@ The run commits to the `maintenance` branch only. Review with "review maintenanc ## Deliberately not adopted -Vector search, typed links, knowledge-graph exports, and symbol-level AST anchoring. Flat files + markdown links + grep is the pattern that wins at these scales; the flat index ceiling (a few hundred pages) only bites in the large-codebase case, which shards by subsystem instead. Revisit if a wiki outgrows that. +Vector search, typed links, knowledge-graph exports, and symbol-level AST anchoring. Flat files + markdown links + grep is the pattern that wins at these scales; the flat index ceiling (a few hundred pages) is far away for the wikis this template targets. Revisit if a wiki outgrows that. diff --git a/codebase-large/.claude/skills/wiki-adr/SKILL.md b/codebase-large/.claude/skills/wiki-adr/SKILL.md deleted file mode 100644 index 8601945..0000000 --- a/codebase-large/.claude/skills/wiki-adr/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-adr -description: Draft an architecture decision record. Use when the human says "draft an ADR for ". ---- - -Read `workflows/adr.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase-large/.claude/skills/wiki-document/SKILL.md b/codebase-large/.claude/skills/wiki-document/SKILL.md deleted file mode 100644 index b0a6934..0000000 --- a/codebase-large/.claude/skills/wiki-document/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-document -description: Document code in the wiki: module, service, api, or data-model pages. Use when the human says "document " or "document the service". ---- - -Read `workflows/document.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase-large/.claude/skills/wiki-owner-review/SKILL.md b/codebase-large/.claude/skills/wiki-owner-review/SKILL.md deleted file mode 100644 index eee4497..0000000 --- a/codebase-large/.claude/skills/wiki-owner-review/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-owner-review -description: Run the owner-review staleness pass for a subsystem. Use when the human says "owner review for ". ---- - -Read `workflows/owner-review.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase-large/.claude/skills/wiki-postmortem/SKILL.md b/codebase-large/.claude/skills/wiki-postmortem/SKILL.md deleted file mode 100644 index ad6546a..0000000 --- a/codebase-large/.claude/skills/wiki-postmortem/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-postmortem -description: File a postmortem page. Use when the human says "log postmortem: ". ---- - -Read `workflows/postmortem.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase-large/.claude/skills/wiki-query/SKILL.md b/codebase-large/.claude/skills/wiki-query/SKILL.md deleted file mode 100644 index 8f65d58..0000000 --- a/codebase-large/.claude/skills/wiki-query/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-query -description: Answer a question from the wiki or the code, scanning frontmatter and the index before opening page bodies. Use for any question answerable from the wiki. ---- - -Read `workflows/query.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase-large/.claude/skills/wiki-reconcile/SKILL.md b/codebase-large/.claude/skills/wiki-reconcile/SKILL.md deleted file mode 100644 index d35a638..0000000 --- a/codebase-large/.claude/skills/wiki-reconcile/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-reconcile -description: Resolve a contested page: weigh the disagreeing evidence and rewrite latest-evidence-first. Use when the human says "reconcile ". ---- - -Read `workflows/reconcile.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase-large/.claude/skills/wiki-sync/SKILL.md b/codebase-large/.claude/skills/wiki-sync/SKILL.md deleted file mode 100644 index e33abca..0000000 --- a/codebase-large/.claude/skills/wiki-sync/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-sync -description: Sync the wiki to the current state of the code repository. Use when the human says "sync to current". ---- - -Read `workflows/sync.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase-large/.claude/skills/wiki-triage/SKILL.md b/codebase-large/.claude/skills/wiki-triage/SKILL.md deleted file mode 100644 index 499cf3c..0000000 --- a/codebase-large/.claude/skills/wiki-triage/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-triage -description: Triage the raw inbox of this wiki. Use when the human says "triage the inbox" or drops files into raw/inbox/ and asks to sort them. ---- - -Read `workflows/triage.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase-large/.claude/skills/wiki-verify/SKILL.md b/codebase-large/.claude/skills/wiki-verify/SKILL.md deleted file mode 100644 index 816b3e2..0000000 --- a/codebase-large/.claude/skills/wiki-verify/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-verify -description: Reality-check wiki pages against the code. Use when the human says "verify modules" or "verify ". ---- - -Read `workflows/verify.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase-large/CLAUDE.md b/codebase-large/CLAUDE.md deleted file mode 100644 index 8760e8b..0000000 --- a/codebase-large/CLAUDE.md +++ /dev/null @@ -1,144 +0,0 @@ ---- -type: tooling ---- - -# CLAUDE.md (large codebase edition) - -You are the maintainer of this wiki for a large codebase: 200k+ lines, multiple deployable services, multiple owners, a rate of change that makes exhaustive verification impossible. "Complete" is not a goal you can reach; **useful and honest about its own staleness** is. This file is the always-loaded core of your operating manual. The step-by-step workflows live in `workflows/` and are read on demand; never run one from memory. - -## Prime directives - -1. **The human owns `raw/` and the code repository. You own everything else.** Never modify code from this wiki. Under `raw/`, the only permitted operation is moving files out of `raw/inbox/` during an approved triage pass. -2. **Coverage is partial by design.** Not every module gets a page. Document stable boundaries, load-bearing modules, gnarly subsystems, public APIs, and the parts new contributors actually need. `python3 lint.py coverage` tracks the gaps; it does not demand every file be covered. -3. **Honest staleness over false completeness.** Every code-referencing page declares the commit it was verified against. Stale pages are flagged, never deleted. "Verified at `a1b2c3d`, may be out of date" beats a confidently wrong page or no page at all. -4. **Subsystems are first-class.** The wiki is organized around subsystems, not flat module lists. Each has a `README.md` entry point and an `OWNERS.md`. Ask before creating a new subsystem: boundaries are organizational, not just technical, and must match how the team thinks about ownership. -5. **ADRs are sacred and hierarchical.** Global ADRs live in `adrs/`, subsystem ADRs in `subsystems//adrs/`. Never edit a past ADR's decision; supersede it. -6. **Log and commit every operation.** Append to `log.md` (append-only, never edit past entries), then git commit `: `, e.g. `document: auth session store`. Never amend or rewrite history. -7. **No secrets.** No API keys, tokens, credentials, connection strings, `.env` contents. Refuse and tell the human where they belong. -8. **Ask before deleting.** Renames, merges, deletions, status flips on active modules, and any ADR change require confirmation. -9. **Scan before you read.** Frontmatter (`type`, `subsystem`, `description`, `tags`) exists so pages can be judged without opening them. For queries, orient via the relevant subsystem README, never by loading the global index into context. - -## Repository pinning and scope - -Top of `index.md`, above the generated marker: - -```yaml -repo: github.com/jack/bigcorp-monorepo -local_path: ~/code/bigcorp -default_branch: main -last_synced_commit: a1b2c3d -last_synced_at: 2026-04-10 -wiki_scope: - include: [services/, libs/, platform/] - exclude: [vendor/, third_party/, generated/] -``` - -`wiki_scope` is critical. The wiki ignores anything outside `include` or inside `exclude`. Generated code, vendored deps, and protobuf output are noise; documenting them produces lies. Sync bucketing and coverage both use it. - -## Directory layout - -``` -wiki/ -├── CLAUDE.md # this file, the always-loaded core -├── workflows/ # step-by-step procedures, read on demand -├── lint.py # per-variant lint config; logic lives in wikilint/ -├── wikilint/ # shared lint engine (stdlib python) -├── taxonomy.md # allowed tags + page-type glossary, lint-enforced -├── .githooks/ # pre-commit gate running lint.py -├── index.md # repo pinning + generated catalog -├── log.md # append-only history -├── coverage.md # generated by `lint.py coverage` -├── glossary.md # domain terms used across subsystems -├── raw/ # immutable, human-owned: inbox/ transcripts/ external-docs/ decisions/ assets/ -├── adrs/ # GLOBAL ADRs: cross-subsystem decisions -├── architecture/ # GLOBAL: end-to-end answers start at architecture/system.md -├── concepts/ runbooks/ postmortems/ synthesis/ queries/ # GLOBAL -└── subsystems/ - └── auth/ - ├── README.md OWNERS.md # README is the subsystem entry point - ├── architecture/ modules/ services/ apis/ data-models/ adrs/ runbooks/ concepts/ -``` - -"What's in auth?" starts at `subsystems/auth/README.md`; subsystems are the unit of navigation. - -## Page conventions - -Every page starts with YAML frontmatter. All pages require: - -```yaml ---- -type: subsystem | module | service | api | data-model | architecture | adr | concept | runbook | postmortem | synthesis | source | query | owners -created: 2026-04-10 # set once, never changed -updated: 2026-04-10 # bumped on every edit -description: One line saying what this page covers, used for scanning. -subsystem: auth # or 'global' for cross-cutting pages -tags: [] -sources: [] # entries are mappings: - resource: /dir/page.md (bundle-absolute, OKF v0.2) -confidence: low | contested # optional; absent means normal ---- -``` - -Extra required fields by type: - -```yaml -subsystem: owners, mission, last_owner_review; plus key_modules, key_services, key_adrs -module: source_path, language, status (active|experimental|deprecated|removed), - criticality (load-bearing|important|normal|peripheral), depends_on, - last_verified_commit, last_verified, verification_method - (full|spot-check|signature-only|declared-by-owner) -service: source_path, deployment, modules, exposes, consumes, last_verified_commit, last_verified -api: api_kind (http|grpc|cli|library|event), defined_in, stability (stable|beta|experimental|deprecated), last_verified_commit -data-model: model_kind (db-table|type|schema|message), defined_in, storage, last_verified_commit, - plus producers and consumers: forward fields stored on the data-model page itself - (the module/api pages that write and read it; empty lists are fine while unknown) -architecture: scope (system|service|module|data-flow|deployment|request-flow), includes -adr: adr_number, status (proposed|accepted|superseded|rejected), date, supersedes, superseded_by -``` - -Rules: -- **Reverse edges are derived, never stored.** Module pages keep `depends_on`; there is no `consumed_by` or `public_surface`, and api pages have no `consumers`. `python3 lint.py reverse-deps` prints the derived maps when you need them. -- `criticality` and `verification_method` allocate verification effort: load-bearing modules get full re-reads, peripheral ones get spot checks or owner declarations, and lint weights its staleness reporting accordingly. Set both honestly; `spot-check` is not `full`. -- Cross-cutting pages (top-level `architecture/`, `adrs/`, `concepts/`, etc.) use `subsystem: global` and declare `affects: [auth, billing]` so subsystem reasoning can find them. -- Use markdown links with bundle-absolute targets: `[session store](/subsystems/auth/modules/session-store.md)`. The full path disambiguates; that matters when 800 pages exist. -- File names are kebab-case; ADRs are `NNNN-slug.md`, numbered per directory. `last_verified_commit` is the load-bearing field on every code-referencing page; lint compares it to `last_synced_commit`. -- Every tag must appear in `taxonomy.md`; introducing a tag means adding it there in the same commit. The allowed page types are described there too, under '## Page types'. `confidence: contested` is a state to exit, not a resting place: reconcile it. Mark claims inferred rather than verified against the code with `(inferred)` inline; a page containing any carries `confidence: low`. - -## Diagrams - -Architecture pages must contain at least one Mermaid diagram; the script enforces presence and size (warn over ~30 nodes, error over 50). Split oversized diagrams by scope (module deps, request flow, deployment) rather than cramming one graph. For global architecture, prefer C4-style layered views (Context, Container, Component), one page per layer. - -## Workflows - -When the human triggers an operation, read the matching file and follow it exactly: - -| Trigger | Read | -|---|---| -| "triage the inbox" | `workflows/triage.md` | -| "document " | `workflows/document.md` | -| "sync to current" | `workflows/sync.md` | -| "verify " | `workflows/verify.md` | -| "owner review for " | `workflows/owner-review.md` | -| "draft an ADR for X" | `workflows/adr.md` | -| "log postmortem: X" | `workflows/postmortem.md` | -| a question answerable from the wiki | `workflows/query.md` | -| "lint the wiki" | `workflows/lint.md` | -| "reconcile " | `workflows/reconcile.md` | -| "maintenance pass" / "review maintenance" | `workflows/maintain.md` | - -In Claude Code each workflow is also a skill wrapper under `.claude/skills/`, so `/wiki-document`, `/wiki-triage`, etc. dispatch the same files deterministically (the wiki- prefix avoids collisions with globally installed skills); lint errors when a wrapper and its workflow drift apart. - -## Deterministic checks - -`python3 lint.py check` handles every mechanical health check: frontmatter validity, broken links, orphans (with unlinked-mention hints), dangling references, tag taxonomy, stale contested pages, criticality-weighted sync drift, owner-review staleness, mermaid presence and node counts, ADR numbering in global and subsystem directories, inbox, secrets, index drift, log format, OKF conformance. Run it instead of checking by hand; fix errors before finishing any operation. A pre-commit hook (installed via `git config core.hooksPath .githooks`) lints the staged snapshot and makes errors uncommittable; never bypass it with `--no-verify`. - -`python3 lint.py rebuild-index` regenerates `index.md`: subsystem READMEs plus `subsystem: global` pages only, pinning block preserved above the marker. The global index never lists every module; subsystem READMEs do that. Never hand-edit below the marker; rebuild after any create, rename, or delete. `python3 lint.py reverse-deps` prints the derived reverse map of every stored relationship field (`depends_on`, `defined_in`, `modules`, `exposes`, `consumes`, `producers`, `consumers`). `python3 lint.py coverage` writes `coverage.md`: module inventory by subsystem and criticality, plus directory coverage against `wiki_scope`. - -## Style rules for wiki prose - -- Plain, direct English. No marketing voice, no hedging filler, no em dashes. Quote sparingly, define acronyms on first use, and state uncertainty on the page itself, not just in chat. -- Subsystem READMEs lead with the mission in one sentence and on-call info in the second paragraph. Anyone in an incident at 3am should find what they need within 10 seconds of opening it. -- For large modules, prefer "what does this NOT do" sections; at scale, scoping beats feature lists. - -## Ambiguity and evolution - -When something is ambiguous, ask: propose two or three options and wait. This file and the workflows are co-owned and will evolve more than smaller variants because the codebase does. When a convention is decided, update the relevant file in the same session, and `lint.py` if the rule is mechanical. diff --git a/codebase-large/taxonomy.md b/codebase-large/taxonomy.md deleted file mode 100644 index d033a88..0000000 --- a/codebase-large/taxonomy.md +++ /dev/null @@ -1,34 +0,0 @@ ---- -type: tooling ---- - -# Taxonomy - -Every `tags:` value on every page must appear in the tag list below. Introducing a new tag means adding it here in the same commit, with a one-line meaning. Keep tags few and general; if two tags mean nearly the same thing, merge them. `lint.py check` warns on tags missing from this file and reports listed tags no page uses. - -- auth — identity, sessions, access control -- database — storage, schemas, migrations -- api — external and internal API surfaces -- infra — deployment, CI, environments -- testing — test strategy, fixtures, coverage -- performance — latency, throughput, resource use -- tooling — build systems, codegen, developer tooling - -## Page types - -Every page `type:` the schema allows is described here with a one-line meaning, so any OKF consumer can learn this bundle's vocabulary without reading the lint config. The authoritative schema lives in `lint.py`; `lint.py check` warns when the two drift. - -- subsystem — a subsystem entry point: mission, owners, key pages -- owners — the ownership record for one subsystem -- module — one module, package, or library in the repo -- service — one deployable service or process -- api — an API surface: HTTP routes, gRPC, CLI, or public functions -- data-model — a schema, type, database table, or message format -- architecture — a diagram page: dependency graph, data flow, or deployment -- adr — an append-only architecture decision record -- concept — a pattern, protocol, or domain concept -- runbook — how to build, test, deploy, debug, or recover -- postmortem — something that broke in production and why -- synthesis — a cross-cutting analysis or refactor proposal -- source — a summary page for one raw source -- query — a filed answer worth keeping diff --git a/codebase-large/wikilint/__init__.py b/codebase-large/wikilint/__init__.py deleted file mode 100644 index f185e69..0000000 --- a/codebase-large/wikilint/__init__.py +++ /dev/null @@ -1,7 +0,0 @@ -"""Shared wiki lint engine. This package is byte-identical across all -Scriptorium variants; per-variant behavior lives entirely in the CONFIG -dict and index_entry_extra() defined in the wiki root's lint.py.""" - -from .cli import main - -__all__ = ["main"] diff --git a/codebase-large/workflows/adr.md b/codebase-large/workflows/adr.md deleted file mode 100644 index 200c4e6..0000000 --- a/codebase-large/workflows/adr.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -type: workflow ---- - -# ADR - -Triggered by "draft an ADR for X". - -1. **Pick the scope first.** Subsystem-local decisions go in `subsystems//adrs/`; anything that crosses subsystem boundaries is global and goes in `adrs/`. When in doubt, ask. Cross-subsystem ADRs are always global. -2. **Read the context:** relevant module and subsystem pages, `raw/decisions/` notes, transcripts, recent code changes. -3. **Draft** using the standard structure: Context, Decision, Consequences, Alternatives Considered. The Decision section is one declarative sentence; the rest justifies it. No filler. -4. **Number it** next in sequence within its directory (numbering is per directory; the script flags gaps in each). Set `status: proposed`, `subsystem: ` or `subsystem: global` with `affects: [...]` for global ADRs. -5. **Handle supersession.** If it supersedes an existing ADR, add it to `supersedes`, and edit the old ADR's frontmatter only: add `superseded_by` and flip its status. The old ADR's body stays untouched, always. -6. **Show the human the draft. Wait.** On approval, set `status: accepted` and link it from affected module and subsystem pages (README `key_adrs` if it is load-bearing for the subsystem). -7. **Rebuild and check:** `python3 lint.py rebuild-index`, then `python3 lint.py check`. -8. **Append to `log.md`**, then commit: `adr: `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **ADR**: 0008 stripe over adyen -- **Scope**: global (affects: billing, platform) -- **Status**: accepted -- **Linked**: subsystems/billing/README.md, subsystems/billing/modules/payments-core.md -``` diff --git a/codebase-large/workflows/document.md b/codebase-large/workflows/document.md deleted file mode 100644 index 9247769..0000000 --- a/codebase-large/workflows/document.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -type: workflow ---- - -# Document - -Triggered by "document `libs/auth/session/`" or "document the new payments service". - -1. **Identify the subsystem.** Does the target belong to an existing subsystem? If not, ask before creating a new one; subsystem boundaries are organizational, not just technical, and must match how the team thinks about ownership. -2. **Read the code.** Start with the public surface (exported functions, types, routes). If the module is large, read the entry points and the most-imported internals first. Do not pretend to have read what you skimmed, and do not infer behavior from filenames. -3. **Check for an existing page.** If one exists, this is an update, not a creation. -4. **Discuss before writing.** Tell the human what you found, what you'll file, and where in the subsystem hierarchy. Confirm `criticality` and `verification_method` with them. If the pass will create concept or synthesis pages, apply the page-worthiness test below and say so in the plan. -5. **Create or update the page.** Fill all required frontmatter, including `description` and `subsystem`. Set `last_verified_commit` to the current SHA and `verification_method` honestly: if you only spot-checked, say `spot-check`, not `full`. Cite specific file paths in the prose. If the code contradicts what a page already claims, do not overwrite: set `confidence: contested` and structure the body latest-evidence-first with dates, so `workflows/reconcile.md` has clean input. Never add reverse bookkeeping to other pages (`consumed_by` on modules, `consumers` on api pages); data-model pages DO store their own `producers` and `consumers` forward fields per the schema. Reverse views come from `python3 lint.py reverse-deps`. -6. **Update affected pages.** The subsystem README's `key_modules`/`key_services` if the page is significant, and subsystem or global architecture pages (Mermaid diagrams plus `includes`). -7. **File an ADR if a real decision was made.** Use the right scope: subsystem-local ADR if the decision only affects this subsystem, global ADR if it crosses boundaries. Follow `workflows/adr.md`. -8. **Rebuild and check.** Run `python3 lint.py rebuild-index`, then `python3 lint.py check` and fix any errors it reports. -9. **Append to `log.md`**, then commit: `document: `. -10. **Report back** with every page created or updated and anything you could not verify. - -## The page-worthiness test - -Module, service, api, data-model, architecture, ADR, runbook, and postmortem pages are inventory: the code, its history, and its operation make them necessary. Any other new page (concept, synthesis) must pass all four parts: - -1. **Topic shape.** It defines something referenceable by name: a concept, a pattern, a term, a convention, a metric. Not a grab-bag, not "assorted notes". -2. **Not meta.** It is not an overview, introduction, getting-started, quickstart, tutorial, walkthrough, FAQ, release-notes, changelog, or roadmap. If the natural filename would be one of those words, stop. -3. **Citation test.** You can write a sentence in an existing page of the form "See [X](/concepts/x.md) for ..." where X is a concrete noun. If the best sentence you can write is "see this page for context", it fails. -4. **Reuse test.** At least two existing pages would cite it, or one page needs it as load-bearing background that does not fit inline. - -When in doubt, do not create the page; integrate the material into an existing one instead. A wiki with few concept pages is fine; a wiki full of `concepts/overview.md` and `concepts/misc-notes.md` is noise. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Document**: libs/auth/session -- **Creation**: subsystems/auth/modules/session-store.md -- **Update**: subsystems/auth/README.md, subsystems/auth/architecture/module-deps.md -- **Verification**: full at e4f5g6h, criticality load-bearing -- **ADR**: subsystems/auth/adrs/0004-session-pinning.md -``` diff --git a/codebase-large/workflows/lint.md b/codebase-large/workflows/lint.md deleted file mode 100644 index 00215ed..0000000 --- a/codebase-large/workflows/lint.md +++ /dev/null @@ -1,39 +0,0 @@ ---- -type: workflow ---- - -# Lint - -Triggered by "lint the wiki". Lint has two halves: the script does everything mechanical; you do only the checks that need judgment. - -1. **Run `python3 lint.py check`** and relay its findings grouped by severity. Do not re-verify by hand what it already covers: frontmatter validity (including the `subsystem` field), broken links, orphans, dangling references (including `key_modules` and `includes`), tags, contested age, criticality-weighted sync drift, owner-review staleness, mermaid presence and node counts, ADR numbering gaps and forward links in both global and subsystem directories, secrets, inbox health, index drift, log format, OKF conformance. -2. **Run `python3 lint.py coverage`** and summarize `coverage.md`: the per-subsystem module inventory by criticality, and the per-include-root directory coverage with its uncovered-directories list. Flag subsystems whose load-bearing inventory looks thin for their footprint and uncovered directories that look load-bearing; coverage is partial by design, so report gaps as choices to make, not failures. -3. **Run `python3 lint.py reverse-deps`** and review the derived map for cross-subsystem coupling: modules depending on another subsystem's internal modules rather than its public APIs. Flag as architectural smell, not error. - -Then the judgment checks: - -4. **Orphan repair.** For each orphan the script reports, use its unlinked-mention hints to propose link insertions; insert on approval. If no genuine mention exists anywhere, recommend merging or deleting the page (ask). Never force a link where the prose doesn't naturally mention the page. -5. **Dead code references.** `source_path` values and file citations pointing at code that no longer exists in the repo. -6. **Architecture fiction and diagram drift.** Architecture pages referencing modules that don't exist or are `removed`; diagrams whose story no longer matches the pages they include. -7. **ADR contradictions and mis-scoped ADRs.** Accepted ADRs contradicting newer accepted ADRs, and subsystem ADRs whose decisions affect other subsystems (they should be global). -8. **Contested without explanation.** Any `confidence: contested` page whose body doesn't explain the disagreement; offer to run `workflows/reconcile.md` on pages the script flags as long-contested. -9. **Stale runbooks.** Runbooks untouched in 6 months for active services. -10. **Glossary gaps.** Domain terms appearing in 5+ pages with no `glossary.md` entry. -11. **Subsystem orphans.** Modules in a subsystem not reachable from its README via `key_modules` or any architecture page. -12. **Description drift.** Spot-check a handful of pages: does `description` still match the body? A wrong description poisons every future scan. -13. **Investigation suggestions.** Three to five questions per active subsystem the wiki currently can't answer well. - -Report findings as a structured list. Auto-fix only the trivially safe (broken links with an unambiguous rename target). Everything else waits for human direction. - -When done: append a lint entry to `log.md` and commit: `lint: findings`. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Lint**: run complete -- **Script**: 2 errors, 7 warnings; coverage: auth 64%, billing 41% -- **Judgment**: 1 mis-scoped adr, 2 glossary gaps, 1 coupling smell -- **Note**: see lint report in chat -``` diff --git a/codebase-large/workflows/maintain.md b/codebase-large/workflows/maintain.md deleted file mode 100644 index 2408d45..0000000 --- a/codebase-large/workflows/maintain.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -type: workflow ---- - -# Maintain - -The unattended maintenance pass, run on a schedule or by "maintenance pass". Its safety comes from reversibility: all work happens on the `maintenance` branch, never on main, and the human merges only after review. - -Setup: `git checkout maintenance` (create it from main if absent; if it holds unmerged work from a prior run, continue on it). - -Allowed actions, exhaustively: - -1. **Run `python3 lint.py check` and fix the mechanical errors it reports**: broken links with unambiguous targets, frontmatter fields whose correct value is derivable (malformed dates, wrong enum casing), dangling references to renamed pages. -2. **Run `python3 lint.py rebuild-index` and `python3 lint.py coverage`.** -3. **Insert cross-links for orphans**, only where the script's mention hints show an unambiguous prose mention of the orphan page. -4. **Collect, do not fix**: contested pages needing reconcile, glossary gaps, taxonomy additions worth proposing, pages whose `description` has drifted from the body, subsystems whose coverage dropped. These go in the run report, not into edits. - -Forbidden in unattended runs: changing the meaning of any prose, deleting, merging, or renaming pages, creating new page types or folders, resolving contested pages, editing `CLAUDE.md`, `workflows/`, or `taxonomy.md`, and touching `raw/`. - -Finish: append one log entry, commit everything as `maintain: run`, and leave the branch unmerged. If errors remain that the allowed actions can't fix (a secrets hit, an ambiguous link), do not commit; leave the tree dirty and put the blocker at the top of the run report. - -When the human says "review maintenance": walk them through `git diff main..maintenance` change by change, then merge on approval and continue reusing the branch. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-07-05 - -- **Maintain**: scheduled run -- **Fix**: 2 broken links; index rebuilt -- **Crosslink**: queries/x-vs-y.md from synthesis/overview.md -- **Proposal**: reconcile concepts/foo.md; add tag 'observability' to taxonomy -``` diff --git a/codebase-large/workflows/owner-review.md b/codebase-large/workflows/owner-review.md deleted file mode 100644 index 03225bd..0000000 --- a/codebase-large/workflows/owner-review.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -type: workflow ---- - -# Owner review - -Triggered by "owner review for auth". Code-level verification catches signature drift; owner review catches "we deprecated this whole module last sprint and nobody told the wiki". This is how large wikis stay honest. - -1. **Open the subsystem README** (`subsystems//README.md`) and every page listed in its `key_modules`, `key_services`, and `key_adrs`. -2. **Interview the owner.** For each page, ask the human (acting as owner): is this still accurate? Any major recent changes the wiki misses? Anything new that should be documented? -3. **Update pages based on the answers.** Deprecations, ownership changes, and mission drift get edited now; larger gaps become follow-up document tasks the human approves. Status flips on active modules still require explicit confirmation. -4. **Bump `last_owner_review`** on the README (lint warns when it exceeds 90 days). -5. **Run `python3 lint.py check`** and fix any errors. -6. **Append an owner-review entry to `log.md`**, then commit: `owner-review: `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Owner-review**: auth -- **Reviewed**: @jack -- **Pages**: 6 confirmed, 2 updated -- **Follow-ups**: document libs/auth/webauthn (new, undocumented) -``` diff --git a/codebase-large/workflows/postmortem.md b/codebase-large/workflows/postmortem.md deleted file mode 100644 index c7ef29c..0000000 --- a/codebase-large/workflows/postmortem.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -type: workflow ---- - -# Postmortem - -Triggered by "log postmortem: ". - -1. **Create `postmortems/-.md`** with `subsystem: global` and `affects:` listing every subsystem involved. Frontmatter also records severity, duration, affected services, affected modules. -2. **Walk through the timeline with the human:** what was noticed, what was tried, what worked, root cause. -3. **Link every module and service involved** using full paths. Add the postmortem to the "incidents" section of each affected subsystem's README. -4. **End with action items:** new runbooks, ADRs, refactor proposals. Create the follow-ups in the same session if the human approves (ADRs via `workflows/adr.md`, at the right scope). -5. **Rebuild and check:** `python3 lint.py rebuild-index`, then `python3 lint.py check`. -6. **Append to `log.md`**, then commit: `postmortem: `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Postmortem**: session-cache-stampede -- **Filed**: postmortems/2026-04-10-session-cache-stampede.md -- **Subsystems**: auth, platform -- **Readmes**: subsystems/auth/README.md, subsystems/platform/README.md -- **Follow-ups**: runbooks/cache-warmup.md, adrs/0009-session-cache-ttl.md (proposed) -``` diff --git a/codebase-large/workflows/query.md b/codebase-large/workflows/query.md deleted file mode 100644 index c4a388c..0000000 --- a/codebase-large/workflows/query.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -type: workflow ---- - -# Query - -Triggered by any question answerable from the wiki. - -1. **Orient via the subsystem README first.** Identify which subsystem the question touches and open its `README.md`. Do not load the global index into context for every query; it exists for global pages and subsystem entry points, nothing more. For cross-cutting questions, scan frontmatter of `subsystem: global` pages whose `affects` mentions the subsystems involved. -2. **Scan frontmatter before opening bodies.** Open a page only when its `description` or tags look relevant. -3. **Read the candidate pages fully**, then follow links one hop out for context. -4. **Synthesize an answer with citations.** Every claim links to the wiki page that supports it, and code claims carry the page's `last_verified_commit` caveat if the page is stale. For code questions, also offer to read the live source rather than relying solely on the wiki. -5. **Surface uncertainty.** If pages disagree, say so. If the wiki can't answer, say so and suggest what to document. -6. **Offer to file the answer.** If non-trivial and likely useful later, ask whether to save it under `queries/` (with `subsystem: global` and `affects:` listing the subsystems it spans). If filed: create the page, run `python3 lint.py rebuild-index`, append to `log.md`, and commit: `query: `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Query**: "How do sessions survive an auth-api deploy?" -- **Filed**: queries/session-survival-on-deploy.md -- **Read**: 6 (auth README + 5) -``` diff --git a/codebase-large/workflows/sync.md b/codebase-large/workflows/sync.md deleted file mode 100644 index 458e42e..0000000 --- a/codebase-large/workflows/sync.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -type: workflow ---- - -# Sync - -Triggered by "sync to current". Sync flags; it never rewrites pages. - -1. **Read the current commit SHA** of the local repo. Compare to `last_synced_commit` in `index.md`. If unchanged, say so and stop. -2. **Diff and bucket by subsystem.** `git diff --name-only ..HEAD`, drop anything outside `wiki_scope` (`include`/`exclude` in the pinning block), and bucket the remaining files by subsystem. -3. **Find affected pages.** For each subsystem with changes, list pages whose `source_path` covers the changed files. Group them by `criticality`. -4. **Report in chat:** changed files per subsystem, pages flagged for reverify with their criticality and current `last_verified_commit`. Do not auto-update anything; `last_verified_commit` only moves when the code is actually re-read (see `workflows/verify.md`). -5. **Update `last_synced_commit` and `last_synced_at`** in the `index.md` pinning block (above the generated marker). -6. **Append a sync entry to `log.md`**, then commit: `sync: -> `. - -Log entry format (per-subsystem counts are mandatory; append under today's `## YYYY-MM-DD` heading if it already exists): - -``` -## 2026-04-10 - -- **Sync**: a1b2c3d -> e4f5g6h -- **Changed**: 247 -- **Subsystems**: auth (43), billing (12), platform (88) -- **Pages**: load-bearing 6, important 14, normal 21 -``` diff --git a/codebase-large/workflows/triage.md b/codebase-large/workflows/triage.md deleted file mode 100644 index 1d96fe4..0000000 --- a/codebase-large/workflows/triage.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -type: workflow ---- - -# Triage - -The human drops new material into `raw/inbox/` without sorting or renaming. Triage is bulk and cheap; documenting is per-item and slow. Never collapse them into one step unless the human explicitly waives the per-item discussion. - -1. **List everything in `raw/inbox/`.** Read enough of each file to classify it. For binaries (PDFs, images), inspect the first page or two. -2. **Propose a plan as a table.** For each file: current name, proposed destination subfolder (`transcripts/`, `external-docs/`, `decisions/`, or other), proposed kebab-case filename, and a one-line "what this is" summary. If a file is unclear, mark it `needs human input` and ask. -3. **Wait for explicit approval.** Do not move anything until the human confirms the plan or sends corrections. If they correct individual rows, update those rows and re-show the table. -4. **Move and rename.** Once approved, move each file to its destination with the new name. Move any associated assets into `raw/assets//` if they exist alongside the source. -5. **Append a triage entry to `log.md`** listing what moved where, then commit: `triage: items from inbox`. -6. **Stop.** Triage does not write wiki pages. That happens in document, separately. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Triage**: 4 items from inbox -- **Move**: raw/inbox/notes.txt -> raw/decisions/payments-provider-notes.md -- **Move**: raw/inbox/standup.m4a -> raw/transcripts/2026-04-09-standup.m4a -``` diff --git a/codebase-large/workflows/verify.md b/codebase-large/workflows/verify.md deleted file mode 100644 index 62ba202..0000000 --- a/codebase-large/workflows/verify.md +++ /dev/null @@ -1,30 +0,0 @@ ---- -type: workflow ---- - -# Verify (sampled) - -Triggered by "verify auth", "verify load-bearing", or "verify ". Exhaustive verification does not scale; sampling does. - -1. **Determine the verification budget.** The human can specify a number of pages, a time budget, or a criticality filter. Default: all `load-bearing` pages flagged by the most recent sync, capped at 10 per session. -2. **Process in priority order:** load-bearing first, then important, then normal. Skip peripheral unless explicitly asked. -3. **For each page, honor its declared `verification_method`:** - - `full`: re-read all the source files the page describes. - - `spot-check`: read entry points and one or two internals. - - `signature-only`: verify exported symbols haven't changed. - - `declared-by-owner`: ask the owner to confirm in chat. - Look for removed exports, changed signatures, new or removed dependencies, changed file paths. -4. **Propose updates as a diff.** Show the human what would change on each page. Wait for approval. -5. **On approval,** apply the edits and bump `last_verified_commit` and `last_verified`. Never bump them without having re-read the code at the declared depth. -6. **Append a verify entry to `log.md`** listing what was checked, what was deferred, and what changed, then commit: `verify: `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Verify**: auth load-bearing -- **Checked**: subsystems/auth/modules/session-store.md (full), subsystems/auth/modules/token-mint.md (spot-check) -- **Deferred**: 4 normal pages (budget) -- **Changes**: SessionStore.Refresh signature changed; last_verified_commit bumped to e4f5g6h -``` diff --git a/codebase/.claude/skills/wiki-adr/SKILL.md b/codebase/.claude/skills/wiki-adr/SKILL.md deleted file mode 100644 index 8601945..0000000 --- a/codebase/.claude/skills/wiki-adr/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-adr -description: Draft an architecture decision record. Use when the human says "draft an ADR for ". ---- - -Read `workflows/adr.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase/.claude/skills/wiki-document/SKILL.md b/codebase/.claude/skills/wiki-document/SKILL.md deleted file mode 100644 index b0a6934..0000000 --- a/codebase/.claude/skills/wiki-document/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-document -description: Document code in the wiki: module, service, api, or data-model pages. Use when the human says "document " or "document the service". ---- - -Read `workflows/document.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase/.claude/skills/wiki-lint/SKILL.md b/codebase/.claude/skills/wiki-lint/SKILL.md deleted file mode 100644 index 0f3dbd6..0000000 --- a/codebase/.claude/skills/wiki-lint/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-lint -description: Run the deterministic lint plus the judgment-only review pass. Use when the human says "lint the wiki". ---- - -Read `workflows/lint.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase/.claude/skills/wiki-maintain/SKILL.md b/codebase/.claude/skills/wiki-maintain/SKILL.md deleted file mode 100644 index 5aad0f5..0000000 --- a/codebase/.claude/skills/wiki-maintain/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-maintain -description: Run or review the unattended maintenance pass on the maintenance branch. Use when the human says "maintenance pass" or "review maintenance". ---- - -Read `workflows/maintain.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase/.claude/skills/wiki-postmortem/SKILL.md b/codebase/.claude/skills/wiki-postmortem/SKILL.md deleted file mode 100644 index ad6546a..0000000 --- a/codebase/.claude/skills/wiki-postmortem/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-postmortem -description: File a postmortem page. Use when the human says "log postmortem: ". ---- - -Read `workflows/postmortem.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase/.claude/skills/wiki-query/SKILL.md b/codebase/.claude/skills/wiki-query/SKILL.md deleted file mode 100644 index 8f65d58..0000000 --- a/codebase/.claude/skills/wiki-query/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-query -description: Answer a question from the wiki or the code, scanning frontmatter and the index before opening page bodies. Use for any question answerable from the wiki. ---- - -Read `workflows/query.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase/.claude/skills/wiki-reconcile/SKILL.md b/codebase/.claude/skills/wiki-reconcile/SKILL.md deleted file mode 100644 index d35a638..0000000 --- a/codebase/.claude/skills/wiki-reconcile/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-reconcile -description: Resolve a contested page: weigh the disagreeing evidence and rewrite latest-evidence-first. Use when the human says "reconcile ". ---- - -Read `workflows/reconcile.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase/.claude/skills/wiki-sync/SKILL.md b/codebase/.claude/skills/wiki-sync/SKILL.md deleted file mode 100644 index e33abca..0000000 --- a/codebase/.claude/skills/wiki-sync/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-sync -description: Sync the wiki to the current state of the code repository. Use when the human says "sync to current". ---- - -Read `workflows/sync.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase/.claude/skills/wiki-triage/SKILL.md b/codebase/.claude/skills/wiki-triage/SKILL.md deleted file mode 100644 index 499cf3c..0000000 --- a/codebase/.claude/skills/wiki-triage/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-triage -description: Triage the raw inbox of this wiki. Use when the human says "triage the inbox" or drops files into raw/inbox/ and asks to sort them. ---- - -Read `workflows/triage.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase/.claude/skills/wiki-verify/SKILL.md b/codebase/.claude/skills/wiki-verify/SKILL.md deleted file mode 100644 index 816b3e2..0000000 --- a/codebase/.claude/skills/wiki-verify/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-verify -description: Reality-check wiki pages against the code. Use when the human says "verify modules" or "verify ". ---- - -Read `workflows/verify.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/codebase/.githooks/pre-commit b/codebase/.githooks/pre-commit deleted file mode 100755 index 1bae369..0000000 --- a/codebase/.githooks/pre-commit +++ /dev/null @@ -1,18 +0,0 @@ -#!/bin/sh -# Wiki pre-commit gate: lint errors are uncommittable. -# Lints the STAGED snapshot (not the working tree), so the gate evaluates -# exactly the bytes that will land in the commit. -# Install once per clone: git config core.hooksPath .githooks -root="$(git rev-parse --show-toplevel)" || exit 1 -tmp="$(mktemp -d)" || exit 1 -trap 'rm -rf "$tmp"' 0 -git -C "$root" checkout-index --all --prefix="$tmp/" || exit 1 -cd "$tmp" || exit 1 -python3 lint.py check -status=$? -if [ $status -ne 0 ]; then - echo "" - echo "pre-commit: lint errors in the staged snapshot block the commit." - echo "Fix the files, git add them, and retry; do not bypass with --no-verify." -fi -exit $status diff --git a/codebase/CLAUDE.md b/codebase/CLAUDE.md deleted file mode 100644 index 6b69fce..0000000 --- a/codebase/CLAUDE.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -type: tooling ---- - -# CLAUDE.md - -You are the maintainer of this codebase wiki. The human writes the code and makes the decisions. You document, cross-reference, lint, and keep the picture current as the code evolves. This file is the always-loaded core of your operating manual. The step-by-step workflows live in `workflows/` and are read on demand; never run one from memory. - -## Prime directives - -1. **The human owns `raw/` and the code repository. You own everything else.** Never edit file contents under `raw/`; the only permitted operation there is moving and renaming files out of `raw/inbox/` during an approved triage pass. Never modify the code repo from this wiki, ever. Reading it is your job; writing it is not. -2. **The wiki must match the code.** Every claim about what a module does, what a function returns, or how two services talk to each other must be verifiable against the current code. If the code changed and the wiki didn't, the wiki is wrong. -3. **Every module page must appear in at least one architecture page.** Modules without architectural context are orphans. Architecture pages without real modules are fiction. -4. **Cite the code.** Every module, service, API, and data model page links to its actual source path in the repo, pinned to a commit SHA. Prose claims about behavior cite specific files and line ranges where reasonable, in the form `src/auth/session.go:42-58`. -5. **ADRs are sacred.** Never edit a past ADR's decision. Append a new ADR that supersedes it. The one allowed edit to a past ADR is flipping its frontmatter `status` and `superseded_by` when a newer ADR supersedes it. -6. **No secrets.** No API keys, tokens, credentials, connection strings with passwords, or `.env` contents. Refuse and tell the human where they belong. -7. **Log and commit every operation.** Append an entry to `log.md` (append-only, never edit past entries), then make a git commit in the wiki repo named `: `, e.g. `document: payments service`. Never amend or rewrite history. Commits are your undo and your audit trail. This discipline applies to the wiki repo only; directive 1 still bars touching the code repo. -8. **Ask before deleting.** Renames, merges, deletions, and any change that marks an active module deprecated require human confirmation. Creates and edits do not. -9. **Scan before you read.** Frontmatter (`type`, `tags`, `description`) exists so pages can be judged without opening them. Check `index.md` and frontmatter first; open bodies only for pages that look relevant. - -## Repository pinning - -The wiki points at one repo at a time. A hand-edited yaml block at the top of `index.md` declares it: - -```yaml -repo: github.com/jack/foo -local_path: ~/code/foo -default_branch: main -last_synced_commit: a1b2c3d -last_synced_at: 2026-04-10 -``` - -This block lives above the generated marker, so `python3 lint.py rebuild-index` preserves it. You update it by hand, and only during sync. - -## Directory layout - -``` -wiki/ -├── CLAUDE.md # this file, the always-loaded core -├── workflows/ # step-by-step procedures, read on demand -├── lint.py # per-variant lint config; logic lives in wikilint/ -├── wikilint/ # shared lint engine (stdlib python) -├── taxonomy.md # allowed tags + page-type glossary, lint-enforced -├── .githooks/ # pre-commit gate running lint.py -├── index.md # repo pinning block + generated content catalog -├── log.md # chronological history, append-only -├── raw/ # immutable sources: inbox/, transcripts/, external-docs/, decisions/, assets/ -├── modules/ # one page per module, package, or library -├── services/ # one page per deployable service or process -├── apis/ # API surfaces: HTTP routes, gRPC, CLI, public functions -├── data-models/ # schemas, types, database tables, message formats -├── architecture/ # diagrams: dependency graph, data flow, deployment -├── adrs/ # Architecture Decision Records, append-only, NNNN-slug.md -├── concepts/ # patterns, protocols, domain concepts -├── runbooks/ # how to: build, test, deploy, debug, recover -├── postmortems/ # things that broke in production and why -├── synthesis/ # cross-cutting analyses, refactor proposals -└── queries/ # filed answers worth keeping -``` - -## Page types and frontmatter - -Every page starts with YAML frontmatter. All pages require `type`, `created`, `updated`, `description` (one accurate sentence; it is how pages are found without being opened), and `tags` (each tag must appear in `taxonomy.md`, which also describes the allowed page types; introducing a tag means adding it there in the same commit). Optional on any page: `sources` (mappings backing the page, each `- resource: /dir/page.md`, bundle-absolute; the OKF v0.2 shape) and `confidence: low | contested` (absent means normal; a `contested` page must explain the disagreement in its body, and contested is a state to exit, not a resting place: reconcile it). Mark claims you inferred rather than verified against the code with `(inferred)` inline; a page containing any carries `confidence: low`. - -Extra required fields by type: - -- **module**: `source_path`, `language`, `status` (active | experimental | deprecated | removed), `last_verified_commit`, `last_verified`. Plus `depends_on`: the module pages this one uses. -- **service**: `source_path`, `deployment` (k8s | systemd | docker | lambda | ...), `last_verified_commit`, `last_verified`. Plus `modules` (module pages it bundles), `exposes` (api pages), `consumes` (service or api pages it calls), `runtime_config`. -- **api**: `api_kind` (http | grpc | cli | library | event), `defined_in` (the module page that implements it), `stability` (stable | beta | experimental | deprecated), `last_verified_commit`. -- **data-model**: `model_kind` (db-table | type | schema | message), `defined_in`, `storage` (postgres | sqlite | redis | in-memory | wire), `last_verified_commit`. Plus `producers` and `consumers`: the module/api pages that write and read it. These are forward fields stored on the data-model page itself; empty lists are fine while unknown. -- **architecture**: `scope` (system | service | module | data-flow | deployment | request-flow), `includes` (every page the diagram covers). -- **adr**: `adr_number`, `status` (proposed | accepted | superseded | rejected), `date`, plus `supersedes` / `superseded_by` wiki paths. -- **concept, runbook, postmortem, synthesis, source, query**: no extra required fields. - -Rules: -- **Every relationship is stored once, on its natural declaring side. Reverse edges are derived, never stored.** Modules declare `depends_on`. Services declare `modules`, `exposes`, `consumes`. APIs declare `defined_in`. Data models declare `producers` and `consumers`. There is no `consumed_by`, no `public_surface`, and no `consumers` on api pages. `python3 lint.py reverse-deps` prints the derived reverse maps when you need them. -- `last_verified_commit` is the load-bearing field for all code-referencing pages. The lint script compares it to `last_synced_commit` and flags drift. -- `created` is set once and never changed. `updated` changes on every edit. Keep `description` accurate on every edit. -- File names are kebab-case. ADRs are zero-padded numbered: `adrs/0007-switch-to-postgres.md`. -- Use markdown links with bundle-absolute targets for all internal references: `[session store](/modules/session-store.md)`. When citing code in prose, use the form `src/auth/session.go:42-58`. - -## Diagrams - -Architecture pages must contain at least one Mermaid diagram, embedded directly. No exceptions. Defaults by scope: `system`, `service`, `deployment` use `flowchart` or `C4Context`; `data-flow` and `request-flow` use `sequenceDiagram`; `module` uses a dependency `flowchart`; data model relationships use `erDiagram`. When a referenced module is renamed or removed, update every diagram that mentions it in the same pass. - -## Workflows - -When the human triggers an operation, read the matching file and follow it exactly: - -| Trigger | Read | -|---|---| -| "triage the inbox" | `workflows/triage.md` | -| "document `src/auth/`" / "document the payments service" | `workflows/document.md` | -| "sync to current" | `workflows/sync.md` | -| "verify modules" / "verify modules/" | `workflows/verify.md` | -| "draft an ADR for X" | `workflows/adr.md` | -| "log postmortem: X" | `workflows/postmortem.md` | -| a question answerable from the wiki or the code | `workflows/query.md` | -| "lint the wiki" | `workflows/lint.md` | -| "reconcile " | `workflows/reconcile.md` | -| "maintenance pass" / "review maintenance" | `workflows/maintain.md` | - -In Claude Code each workflow is also a skill wrapper under `.claude/skills/`, so `/wiki-document`, `/wiki-triage`, etc. dispatch the same files deterministically (the wiki- prefix avoids collisions with globally installed skills); lint errors when a wrapper and its workflow drift apart. - -## Deterministic checks - -`python3 lint.py check` handles every mechanical health check: frontmatter validity, broken links, orphans (with unlinked-mention hints), dangling references (including `defined_in` and `includes`), architecture membership of active modules (prime directive 3), tag taxonomy, stale contested pages, sync drift against the pinning block, missing Mermaid diagrams on architecture pages, ADR numbering gaps and superseded ADRs without forward links, inbox health, secrets, index drift, log format, OKF conformance. Run it instead of checking these by hand, and fix errors it reports before finishing any operation. A pre-commit hook (installed via `git config core.hooksPath .githooks`) lints the staged snapshot and makes errors uncommittable; never bypass it with `--no-verify`. - -`python3 lint.py rebuild-index` regenerates `index.md` from page frontmatter, preserving the pinning block above the generated marker. The index is a derived artifact: never hand-edit anything below the marker, and rebuild at the end of any operation that creates, renames, or deletes pages. `python3 lint.py reverse-deps` prints the derived reverse map of every stored relationship field (`depends_on`, `defined_in`, `modules`, `exposes`, `consumes`, `producers`, `consumers`). - -## Style rules for wiki prose - -- Plain, direct English. No marketing voice, no hedging filler. -- Short paragraphs. Bullets where they help, prose where they don't. -- Quote sparingly: one quote per source maximum, under 15 words. -- For module pages, lead with what it is and what it's responsible for in two sentences. Then public surface, then dependencies, then notable internals, then gotchas. -- For ADRs, write the Decision section as one declarative sentence. The rest exists to justify it. -- When uncertain, say so in the page itself, not just in chat. -- No em dashes. Use commas, parentheses, or sentence breaks instead. - -## Ambiguity and evolution - -When something is ambiguous, ask: flag it, propose two or three options, wait for a decision. This file and the workflow files are co-owned. When the human decides a new convention, page type, workflow step, or lint rule, update the relevant file in the same session, and update `lint.py` if the rule is mechanical. The wiki's quality is bounded by how good these files are. diff --git a/codebase/taxonomy.md b/codebase/taxonomy.md deleted file mode 100644 index ff7ef73..0000000 --- a/codebase/taxonomy.md +++ /dev/null @@ -1,31 +0,0 @@ ---- -type: tooling ---- - -# Taxonomy - -Every `tags:` value on every page must appear in the tag list below. Introducing a new tag means adding it here in the same commit, with a one-line meaning. Keep tags few and general; if two tags mean nearly the same thing, merge them. `lint.py check` warns on tags missing from this file and reports listed tags no page uses. - -- auth — identity, sessions, access control -- database — storage, schemas, migrations -- api — external and internal API surfaces -- infra — deployment, CI, environments -- testing — test strategy, fixtures, coverage -- performance — latency, throughput, resource use - -## Page types - -Every page `type:` the schema allows is described here with a one-line meaning, so any OKF consumer can learn this bundle's vocabulary without reading the lint config. The authoritative schema lives in `lint.py`; `lint.py check` warns when the two drift. - -- module — one module, package, or library in the repo -- service — one deployable service or process -- api — an API surface: HTTP routes, gRPC, CLI, or public functions -- data-model — a schema, type, database table, or message format -- architecture — a diagram page: dependency graph, data flow, or deployment -- adr — an append-only architecture decision record -- concept — a pattern, protocol, or domain concept -- runbook — how to build, test, deploy, debug, or recover -- postmortem — something that broke in production and why -- synthesis — a cross-cutting analysis or refactor proposal -- source — a summary page for one raw source -- query — a filed answer worth keeping diff --git a/codebase/wikilint/__init__.py b/codebase/wikilint/__init__.py deleted file mode 100644 index f185e69..0000000 --- a/codebase/wikilint/__init__.py +++ /dev/null @@ -1,7 +0,0 @@ -"""Shared wiki lint engine. This package is byte-identical across all -Scriptorium variants; per-variant behavior lives entirely in the CONFIG -dict and index_entry_extra() defined in the wiki root's lint.py.""" - -from .cli import main - -__all__ = ["main"] diff --git a/codebase/wikilint/checks.py b/codebase/wikilint/checks.py deleted file mode 100644 index bc9884d..0000000 --- a/codebase/wikilint/checks.py +++ /dev/null @@ -1,671 +0,0 @@ -"""All mechanical checks except index drift (which lives with the index -builder in derived.py).""" - -import re -from datetime import date, datetime -from urllib.parse import unquote - -from .model import ( - ADR_FILE_RE, FILENAME_EXEMPT, KEBAB_RE, LOG_DATE_HEADER_RE, LOG_ENTRY_RE, - MD_LINK_RE, OKF_VERSION, SECRET_PATTERNS, blank_frontmatter, blank_images, - blank_inline_code, build_link_index, discover_okf_bundle, line_at, - parse_frontmatter, parse_index_pinning, parse_iso_date, resolve_link, -) -from .settings import CONFIG - -# A URI scheme (mailto:, tel:, https:, data:, ...) or protocol-relative //host. -URI_SCHEME_RE = re.compile(r"^(?:[a-zA-Z][a-zA-Z0-9+.\-]*:|//)") - - -def missing_value(value): - """True when a required field is absent in practice: None, "", or [].""" - return value is None or value == "" or value == [] - - -def check_frontmatter(pages, report): - valid_types = set(CONFIG["type_required"]) - for p in pages: - if p.fm_error: - report.error("frontmatter", p.rel, p.fm_error) - continue - ptype = p.type - if ptype not in valid_types: - report.error("frontmatter", p.rel, f"invalid or missing type: {ptype!r}") - continue - for field in CONFIG["required_fields"] + CONFIG["type_required"][ptype]: - if missing_value(p.fields.get(field)): - report.error("frontmatter", p.rel, f"missing or empty required field: {field}") - check_enums(p, ptype, report) - check_dates(p, report) - desc = p.fields.get("description", "") - if isinstance(desc, str) and len(desc) > 200: - report.warning("frontmatter", p.rel, "description over 200 chars; keep it one line") - - -def check_enums(p, ptype, report): - for field, allowed in CONFIG["enum_fields"].items(): - if field in p.fields and p.fields[field] not in allowed: - report.error("frontmatter", p.rel, f"{field}: {p.fields[field]!r} not in {allowed}") - for field, allowed in CONFIG["type_enum_fields"].get(ptype, {}).items(): - value = p.fields.get(field) - if value is not None and value not in allowed: - report.error("frontmatter", p.rel, f"{field}: {value!r} not in {allowed}") - - -def check_dates(p, report): - for field in CONFIG["iso_date_fields"]: - value = p.fields.get(field) - if value and parse_iso_date(value) is None: - report.error("frontmatter", p.rel, f"{field} is not an ISO date") - # The created/updated ordering invariant holds regardless of which fields - # are ISO-validated above. - created = parse_iso_date(p.fields.get("created")) - updated = parse_iso_date(p.fields.get("updated")) - if created and updated and updated < created: - report.error("frontmatter", p.rel, "updated is older than created") - - -def check_filenames(pages, report): - for p in pages: - if p.path.name in FILENAME_EXEMPT: - continue - if not KEBAB_RE.match(p.path.name): - report.warning("filename", p.rel, "not kebab-case") - - -def _file_identity(path): - """(device, inode) identity for a path, or None if it can't be stat'd. - Used to match a link target to a page independent of the spelling of the - path (case on case-insensitive filesystems, symlinks, mount aliases).""" - try: - st = path.stat() - except OSError: - return None - return (st.st_dev, st.st_ino) - - -def check_markdown_links(p, pages_by_identity, report, root): - """Every [text](target) link must resolve on the filesystem (file or - directory): bundle-absolute targets (leading /, OKF's recommended form) - resolve against the wiki root, relative targets against the linking - page's directory. Returns the resolved page targets for inbound counting. - External (scheme/protocol-relative) and pure-anchor targets are skipped.""" - resolved_pages = [] - # resolve() both sides so symlinked roots (e.g. /var -> /private/var) - # compare equal when checking bundle containment. - root_resolved = root.resolve() - # Body content only, images and code blanked, line numbers file-accurate. - scan = blank_images(blank_frontmatter(p.prose)) - for m in MD_LINK_RE.finditer(scan): - raw = m.group(1) - target = raw.split("#")[0] - if not target or URI_SCHEME_RE.match(target): - continue - if target.startswith("/"): - path = (root / unquote(target).lstrip("/")).resolve() - else: - path = (p.path.parent / unquote(target)).resolve() - if not path.exists(): - report.error( - "link", f"{p.rel}:{line_at(scan, m.start())}", - f"broken link ({raw})", - ) - continue - if path != root_resolved and root_resolved not in path.parents: - report.error( - "link", f"{p.rel}:{line_at(scan, m.start())}", - f"link escapes the wiki root ({raw})", - ) - continue - target_page = pages_by_identity.get(_file_identity(path)) - if target_page is not None: - resolved_pages.append(target_page) - return resolved_pages - - -def check_links_and_orphans(pages, report, root): - by_stem, by_rel = build_link_index(pages) - pages_by_identity = {} - for p in pages: - identity = _file_identity(p.path) - if identity is not None: - pages_by_identity[identity] = p - inbound = {p.rel: 0 for p in pages} - for p in pages: - for target in check_markdown_links(p, pages_by_identity, report, root): - if target.rel != p.rel: - inbound[target.rel] += 1 - for field in CONFIG["path_fields"] + CONFIG["edge_fields"]: - for value in p.path_list(field): - # External resources (URLs) are provenance, not wiki edges. - if URI_SCHEME_RE.match(value): - continue - target = resolve_link(value, by_stem, by_rel) - if target is None: - report.error( - "reference", p.rel, - f"{field} points to nonexistent page: {value}", - ) - elif target.rel != p.rel: - inbound[target.rel] += 1 - if not CONFIG["orphans"]: - return - orphans = [ - p for p in pages - if inbound[p.rel] == 0 and p.path.name not in FILENAME_EXEMPT - ] - for p in orphans: - report.warning("orphan", p.rel, "no inbound links from any page") - report_unlinked_mentions(p, pages, report) - - -def report_unlinked_mentions(orphan, pages, report): - """Hint generation for the cross-linker: places where an orphan's name - appears in prose without a link.""" - patterns = [ - re.compile(re.escape(orphan.stem.replace("-", " ")), re.IGNORECASE), - re.compile(re.escape(orphan.stem), re.IGNORECASE), - ] - for other in pages: - if other.rel == orphan.rel: - continue - for pattern in patterns: - m = pattern.search(other.prose) - if m: - report.info( - "orphan", orphan.rel, - f"mentioned unlinked in {other.rel}:{line_at(other.prose, m.start())}; " - "candidate link", - ) - break - - -def check_membership(pages, report): - """Active member pages (components/modules) must appear in at least one - container page's (topology/architecture) membership field.""" - rule = CONFIG["membership"] - if not rule: - return - by_stem, by_rel = build_link_index(pages) - contained = set() - for p in pages: - if p.type != rule["container_type"]: - continue - for value in p.field_list(rule["container_field"]): - target = resolve_link(value, by_stem, by_rel) - if target is not None: - contained.add(target.rel) - for p in pages: - if p.type != rule["member_type"] or not p.fields: - continue - if p.fields.get("status") not in rule["active_statuses"]: - continue - if p.rel not in contained: - report.warning( - "membership", p.rel, - f"active {rule['member_type']} not included in any " - f"{rule['container_type']} page", - ) - - -def check_claude_size(root, report): - claude = root / "CLAUDE.md" - if not claude.is_file(): - return - lines = len(claude.read_text(encoding="utf-8", errors="replace").splitlines()) - cap = CONFIG["claude_md_max_lines"] - if lines > cap: - report.warning( - "hot-core", "CLAUDE.md", - f"{lines} lines exceeds the {cap}-line cap; " - "the always-loaded core is regrowing, move detail to workflows/", - ) - - -def load_taxonomy(root): - """Parse the taxonomy file into (tags, types). Tag entries are `- slug` - lines anywhere outside a `## Page types` section; entries inside that - section map type slug -> one-line meaning (empty string when the line has - no meaning after the dash). `types` is None when the section is absent.""" - path = root / CONFIG["taxonomy_file"] - if not path.is_file(): - return None - tags = set() - types = None - in_types = False - for line in path.read_text(encoding="utf-8", errors="replace").splitlines(): - if re.match(r"^##\s+Page types\s*$", line): - in_types = True - if types is None: - types = {} - continue - if re.match(r"^#{1,6}\s", line): - in_types = False - continue - m = re.match(r"^- ([a-z0-9][a-z0-9-]*)(?:\s*—\s*(.*))?", line) - if m: - if in_types: - types[m.group(1)] = (m.group(2) or "").strip() - else: - tags.add(m.group(1)) - return tags, types - - -def check_tags(pages, report, root): - if CONFIG["taxonomy_file"] is None: - return - taxonomy = load_taxonomy(root) - if taxonomy is None: - report.warning( - "tags", CONFIG["taxonomy_file"], - "missing; create it and list the allowed tags", - ) - return - taxonomy, _types = taxonomy - used = set() - for p in pages: - for tag in p.field_list("tags"): - used.add(tag) - if tag not in taxonomy: - report.warning( - "tags", p.rel, - f"tag {tag!r} not in {CONFIG['taxonomy_file']}; " - "normalize it or add it there in the same commit", - ) - for unused in sorted(taxonomy - used): - report.info("tags", CONFIG["taxonomy_file"], f"tag {unused!r} is used by no page") - - -def check_types(pages, report, root): - """The type glossary: taxonomy.md's `## Page types` section must describe - every schema type with a one-line meaning, so the bundle self-describes - its type vocabulary to foreign OKF consumers.""" - if CONFIG["taxonomy_file"] is None or not CONFIG["types_glossary"]: - return - taxonomy = load_taxonomy(root) - if taxonomy is None: - return # check_tags already reports the missing file - _tags, types = taxonomy - tfile = CONFIG["taxonomy_file"] - if types is None: - report.warning( - "types", tfile, - "no '## Page types' section; describe each page type there " - "with a one-line meaning", - ) - return - schema_types = set(CONFIG["type_required"]) - for t in sorted(schema_types - set(types)): - report.warning( - "types", tfile, - f"page type {t!r} is not described; add '- {t} — ' " - "under '## Page types'", - ) - for t in sorted(schema_types & set(types)): - if not types[t]: - report.warning( - "types", tfile, - f"page type {t!r} has no meaning; add one after the dash", - ) - for t in sorted(set(types) - schema_types): - report.warning( - "types", tfile, - f"'## Page types' describes unknown type {t!r}; remove it or " - "add it to the schema", - ) - - -def check_skills(pages, report, root): - """Harness skill wrappers must pair 1:1 with workflows: every - workflows/.md needs //SKILL.md whose body points - back at the workflow file, and no wrapper may point at nothing. Errors, - not warnings: the wrappers are dispatch surface, and silent drift there - means a slash command runs a stale or missing procedure.""" - if CONFIG["skills_dir"] is None: - return - skills_root = root / CONFIG["skills_dir"] - workflows_root = root / "workflows" - wf_stems = ( - {p.stem for p in workflows_root.glob("*.md")} - if workflows_root.is_dir() else set() - ) - prefix = CONFIG["skills_prefix"] - skill_names = ( - {d.name for d in skills_root.iterdir() if d.is_dir()} - if skills_root.is_dir() else set() - ) - expected = {prefix + stem: stem for stem in wf_stems} - sdir = CONFIG["skills_dir"] - for name in sorted(set(expected) - skill_names): - report.error( - "skills", f"workflows/{expected[name]}.md", - f"no skill wrapper; add {sdir}/{name}/SKILL.md pointing at it", - ) - for name in sorted(skill_names - set(expected)): - report.error( - "skills", f"{sdir}/{name}/SKILL.md", - f"orphan wrapper; no workflow pairs with it " - f"(expected form: {prefix})", - ) - for name in sorted(set(expected) & skill_names): - stem = expected[name] - skill_md = skills_root / name / "SKILL.md" - rel = f"{sdir}/{name}/SKILL.md" - if not skill_md.is_file(): - report.error("skills", rel, "missing SKILL.md in wrapper directory") - continue - body = skill_md.read_text(encoding="utf-8", errors="replace") - if f"workflows/{stem}.md" not in body: - report.error( - "skills", rel, - f"wrapper does not reference workflows/{stem}.md; " - "the body must point at its workflow file", - ) - - -def check_contested_age(pages, report): - today = date.today() - for p in pages: - if (p.fields or {}).get("confidence") != "contested": - continue - updated = parse_iso_date(p.fields.get("updated")) - if updated is None: - continue - age = (today - updated).days - if age > CONFIG["contested_max_days"]: - report.warning( - "contested", p.rel, - f"contested and untouched for {age} days " - f"(limit {CONFIG['contested_max_days']}); run reconcile", - ) - - -def check_inferred_markers(pages, report): - for p in pages: - if "(inferred)" not in p.body_prose: - continue - if (p.fields or {}).get("confidence") not in ("low", "contested"): - report.warning( - "provenance", p.rel, - "contains '(inferred)' claims but confidence is not low", - ) - - -def check_inbox(root, report): - if CONFIG["inbox_dir"] is None: - return - inbox = root / CONFIG["inbox_dir"] - if not inbox.is_dir(): - return - items = [e for e in inbox.iterdir() if not e.name.startswith(".")] - if len(items) >= CONFIG["inbox_warn_count"]: - report.warning("inbox", CONFIG["inbox_dir"], f"{len(items)} items awaiting triage") - now = datetime.now() - for item in items: - age = (now - datetime.fromtimestamp(item.stat().st_mtime)).days - if age >= CONFIG["inbox_warn_age_days"]: - report.warning("inbox", item.name, f"in inbox for {age} days") - - -def secret_patterns(): - """Built-in secret patterns plus the variant's extra_secret_patterns, - compiled once at configure() time.""" - return SECRET_PATTERNS + CONFIG["secret_extra_compiled"] - - -def scan_text_for_secrets(text, rel, report): - """Scan one file's text; matches whose line matches any allow regex are - suppressed (template placeholders, redaction markers). Allow regexes are - precompiled; the line split is deferred until a suppression check needs it.""" - allow = CONFIG["secret_allow_compiled"] - lines = None - for pattern, label in secret_patterns(): - for m in pattern.finditer(text): - line_no = line_at(text, m.start()) - if allow: - if lines is None: - lines = text.split("\n") - if any(a.search(lines[line_no - 1]) for a in allow): - continue - report.error("secrets", f"{rel}:{line_no}", f"possible {label}") - - -def check_secrets(pages, report): - for p in pages: - scan_text_for_secrets(p.text, p.rel, report) - - -def check_field_staleness(pages, report): - today = date.today() - for rule in CONFIG["staleness"]: - for p in pages: - if p.type not in rule["types"] or not p.fields: - continue - value = p.fields.get(rule["field"]) - if not value: - continue - d = parse_iso_date(value) - if d is None: - report.error("staleness", p.rel, f"{rule['field']} is not an ISO date") - continue - age = (today - d).days - if age > rule["max_days"]: - report.add( - rule["severity"].upper(), "staleness", p.rel, - f"{rule['field']} is {age} days old (limit {rule['max_days']})", - ) - - -def check_sync_drift(pages, report, root): - if not CONFIG["sync_drift"]: - return - field, types = CONFIG["sync_drift"] - synced = parse_index_pinning(root).get("last_synced_commit") - if not synced: - return - crit_field = CONFIG["criticality_field"] - plain_counts = {} - for p in pages: - if p.type not in types or not p.fields: - continue - verified = p.fields.get(field) - if not verified or verified == synced: - continue - crit = p.fields.get(crit_field) if crit_field else None - if crit == "load-bearing" or not crit_field: - label = "load-bearing page" if crit_field else "page" - report.warning( - "sync-drift", p.rel, - f"{label} verified at {verified}, repo at {synced}", - ) - else: - plain_counts[crit or "unspecified"] = ( - plain_counts.get(crit or "unspecified", 0) + 1 - ) - for crit, count in sorted(plain_counts.items()): - report.info("sync-drift", crit, f"{count} stale pages (run verify)") - - -def check_owner_review(pages, report): - if not CONFIG["owner_review_max_days"]: - return - today = date.today() - for p in pages: - if p.type != "subsystem" or not p.fields: - continue - reviewed = parse_iso_date(p.fields.get("last_owner_review")) - if reviewed is None: - report.warning("owner-review", p.rel, "missing or invalid last_owner_review") - elif (today - reviewed).days > CONFIG["owner_review_max_days"]: - report.warning( - "owner-review", p.rel, - f"last owner review {(today - reviewed).days} days ago", - ) - - -def count_mermaid_nodes(block): - """Rough node count for a mermaid block: unique identifiers that are - declared with a shape or appear on either side of an edge.""" - ids = set() - for m in re.finditer(r"([A-Za-z][\w-]*)\s*(?:\[|\(|\{)", block): - ids.add(m.group(1)) - for m in re.finditer( - r"([A-Za-z][\w-]*)\s*(?:-->|---|-\.->|==>|--o|--x)\s*([A-Za-z][\w-]*)", block - ): - ids.add(m.group(1)) - ids.add(m.group(2)) - keywords = {"flowchart", "graph", "sequenceDiagram", "subgraph", "end", - "classDef", "class", "style", "direction", "erDiagram", "C4Context"} - return len(ids - keywords) - - -def check_mermaid(pages, report): - if not CONFIG["mermaid_required_types"]: - return - for p in pages: - if p.type not in CONFIG["mermaid_required_types"]: - continue - blocks = re.findall(r"```mermaid\n(.*?)```", p.body, re.DOTALL) - if not blocks: - report.error("mermaid", p.rel, "page type requires at least one mermaid diagram") - continue - for block in blocks: - nodes = count_mermaid_nodes(block) - if CONFIG["mermaid_node_error"] and nodes > CONFIG["mermaid_node_error"]: - report.error("mermaid", p.rel, f"diagram has ~{nodes} nodes; split it") - elif CONFIG["mermaid_node_warn"] and nodes > CONFIG["mermaid_node_warn"]: - report.warning("mermaid", p.rel, f"diagram has ~{nodes} nodes; consider splitting") - - -def check_adrs(pages, report, root): - for adr_dir in CONFIG["adr_dirs"]: - for dir_path in sorted(root.glob(adr_dir)): - if not dir_path.is_dir(): - continue - check_adr_dir(dir_path, pages, report, root) - - -def check_adr_dir(dir_path, pages, report, root): - numbers = {} - for p in pages: - if p.path.parent != dir_path: - continue - m = ADR_FILE_RE.match(p.path.name) - if not m: - report.warning("adr", p.rel, "not in NNNN-slug.md form") - continue - numbers[int(m.group(1))] = p - status = (p.fields or {}).get("status") - if status == "superseded" and not (p.fields or {}).get("superseded_by"): - report.error("adr", p.rel, "superseded ADR has no superseded_by forward link") - if numbers: - expected = set(range(min(numbers), max(numbers) + 1)) - for gap in sorted(expected - set(numbers)): - rel = dir_path.relative_to(root).as_posix() - report.warning("adr", rel, f"numbering gap: {gap:04d} missing") - - -def check_log(root, report): - """OKF log structure: '## YYYY-MM-DD' date headings, newest first, with - entries as '- **Action**: ...' bullets. This check owns log.md end to - end; check_okf deliberately does not re-validate it.""" - log_file = CONFIG["log_file"] - if log_file is None: - return - log = root / log_file - if not log.is_file(): - report.warning("log", log_file, "missing") - return - text = log.read_text(encoding="utf-8", errors="replace") - prev = None # (date, line_no) of the last valid heading seen - for i, line in enumerate(text.splitlines(), 1): - if line.startswith("## "): - m = LOG_DATE_HEADER_RE.match(line) - if not m: - report.warning( - "log", f"{log_file}:{i}", - f"heading not in '## YYYY-MM-DD' form: {line!r}", - ) - continue - d = parse_iso_date(m.group(1)) - if d is None: - report.warning("log", f"{log_file}:{i}", f"heading is not a valid date: {line!r}") - continue - if prev is not None and d > prev[0]: - report.warning( - "log", f"{log_file}:{i}", - f"headings out of order (want newest-first): {d} follows {prev[0]}", - ) - elif prev is not None and d == prev[0]: - report.warning( - "log", f"{log_file}:{i}", - f"duplicate heading for {d}; append entries under the existing one", - ) - prev = (d, i) - elif line.startswith("- ") and not LOG_ENTRY_RE.match(line): - report.warning( - "log", f"{log_file}:{i}", - "entry should open with a bold action word: '- **Update**: ...'", - ) - - -def check_sources(pages, report): - """OKF v0.2 sources shape (SPEC §5.1): each entry is a mapping whose - required resource names either an external URL or a bundle-absolute page - path. Plain-string entries (the pre-0.2 shape) still resolve as paths in - check_links_and_orphans, but warn until installs migrate.""" - if not CONFIG["okf_conformance"]: - return - for p in pages: - for i, entry in enumerate(p.field_list("sources"), 1): - if isinstance(entry, dict): - resource = entry.get("resource", "") - if not resource: - report.error( - "sources", p.rel, - f"sources entry {i} has no resource " - "(required per entry, OKF v0.2 §5.1)", - ) - elif not URI_SCHEME_RE.match(resource) and not resource.startswith("/"): - report.warning( - "sources", p.rel, - f"sources entry {i}: bundle paths should be " - f"bundle-absolute (/dir/page.md), got {resource!r}", - ) - else: - report.warning( - "sources", p.rel, - f"sources entry {i} is a plain string; the OKF v0.2 shape " - "is '- resource: /dir/page.md'", - ) - - -def check_okf(pages, report, root): - """OKF v0.2 conformance: (1) every non-reserved .md parses as frontmatter, - (2) with a non-empty type, (3) reserved files follow their structure. - Pages are skipped here: check_frontmatter is strictly stricter. log.md's - structure is owned by check_log, not re-validated here. raw/ is excluded - by design (immutable human-owned sources; see discover_okf_bundle).""" - if not CONFIG["okf_conformance"]: - return - page_rels = {p.rel for p in pages} - for path in discover_okf_bundle(root): - rel = path.relative_to(root).as_posix() - if rel in page_rels: - continue - fields, err = parse_frontmatter( - path.read_text(encoding="utf-8", errors="replace")) - if err: - report.error("okf", rel, f"OKF rule 1: {err}") - elif missing_value(fields.get("type")): - report.error("okf", rel, "OKF rule 2: missing or empty type") - index_file = CONFIG["index_file"] - if index_file and (root / index_file).is_file(): - fields, _ = parse_frontmatter( - (root / index_file).read_text(encoding="utf-8", errors="replace")) - if not fields or fields.get("okf_version") != OKF_VERSION: - report.error( - "okf", index_file, - f'OKF rule 3: missing okf_version: "{OKF_VERSION}" frontmatter; ' - "run `python3 lint.py rebuild-index`", - ) diff --git a/codebase/wikilint/cli.py b/codebase/wikilint/cli.py deleted file mode 100644 index 84383ee..0000000 --- a/codebase/wikilint/cli.py +++ /dev/null @@ -1,90 +0,0 @@ -"""CLI entry point: subcommand dispatch and the check orchestrator.""" - -import sys -from pathlib import Path - -from . import checks -from .derived import check_index_drift, rebuild_index, run_coverage, run_reverse_deps -from .model import Report, discover_pages -from .settings import CONFIG, ConfigError, configure - -USAGE = """\ -Usage: - python3 lint.py check run all mechanical checks, exit 1 on errors - python3 lint.py rebuild-index regenerate the index from page frontmatter - python3 lint.py reverse-deps print the derived reverse-dependency maps - python3 lint.py coverage write coverage.md (variants with coverage enabled) -""" - - -def gather_report(root): - """Run every mechanical check and return the Report (no printing). - This is the testable core of `lint.py check`.""" - report = Report() - pages = discover_pages(root, report) - checks.check_frontmatter(pages, report) - checks.check_filenames(pages, report) - checks.check_links_and_orphans(pages, report, root) - checks.check_membership(pages, report) - checks.check_claude_size(root, report) - checks.check_tags(pages, report, root) - checks.check_types(pages, report, root) - checks.check_skills(pages, report, root) - checks.check_contested_age(pages, report) - checks.check_inferred_markers(pages, report) - checks.check_inbox(root, report) - checks.check_secrets(pages, report) - checks.check_field_staleness(pages, report) - checks.check_sync_drift(pages, report, root) - checks.check_owner_review(pages, report) - checks.check_mermaid(pages, report) - checks.check_adrs(pages, report, root) - checks.check_log(root, report) - checks.check_sources(pages, report) - checks.check_okf(pages, report, root) - check_index_drift(pages, report, root) - for extra in CONFIG["extra_checks"]: - # A third-party check must not abort the run: a raise here would - # discard every finding already gathered. Convert it to an error. - try: - extra(pages, report, root) - except Exception as e: - report.error( - "extra-check", getattr(extra, "__name__", "extra_check"), - f"raised {type(e).__name__}: {e}", - ) - return len(pages), report - - -def run_check(root): - count, report = gather_report(root) - print(f"checked {count} pages") - print(report.render()) - return 1 if report.has_errors else 0 - - -def main(config, index_entry_extra, argv=None, root=None): - try: - configure(config, index_entry_extra) - except ConfigError as e: - print(f"lint config error: {e}", file=sys.stderr) - return 2 - root = Path(root) if root else Path.cwd() - if not (root / "CLAUDE.md").is_file(): - print("run from the wiki root (CLAUDE.md not found here)", file=sys.stderr) - return 2 - args = sys.argv[1:] if argv is None else argv - command = args[0] if args else "check" - if command == "check": - return run_check(root) - if command == "rebuild-index": - return rebuild_index(root) - if command == "reverse-deps": - return run_reverse_deps(root) - if command == "coverage": - if not CONFIG["coverage"]: - print("coverage is not enabled for this variant") - return 2 - return run_coverage(root) - print(USAGE) - return 2 diff --git a/codebase/wikilint/derived.py b/codebase/wikilint/derived.py deleted file mode 100644 index 00fc5c0..0000000 --- a/codebase/wikilint/derived.py +++ /dev/null @@ -1,249 +0,0 @@ -"""Derived artifacts: the generated index, reverse-dependency maps, and the -coverage report. check_index_drift lives here beside the builder it reuses.""" - -from pathlib import Path - -from .model import ( - GENERATED_MARKER, OKF_VERSION, YAML_FENCE_RE, build_link_index, - discover_pages, parse_index_pinning, resolve_link, Report, -) -from .settings import CONFIG - - -def derive_reverse_edges(pages): - """field -> {target rel: [source rels]} for every relationship field in - CONFIG['reverse_fields']. This is the derived view that replaces the old - stored mirror fields (consumed_by, public_surface, api consumers).""" - by_stem, by_rel = build_link_index(pages) - reverse = {} - for field in CONFIG["reverse_fields"]: - rev = {} - for p in pages: - for value in p.field_list(field): - target = resolve_link(value, by_stem, by_rel) - if target is not None: - rev.setdefault(target.rel, []).append(p.rel) - reverse[field] = rev - return reverse - - -def run_reverse_deps(root): - report = Report() - pages = discover_pages(root, report) - if not CONFIG["reverse_fields"]: - print("no reverse fields configured for this variant") - return 0 - for field, rev in derive_reverse_edges(pages).items(): - print(f"\nreverse of {field}:") - if not rev: - print(" (no edges)") - for target, sources in sorted(rev.items()): - print(f" {target} <- {', '.join(sorted(sources))}") - return 0 - - -def index_path(root): - """Absolute path of the generated index, or None when the index is - disabled (index_file is None).""" - index_file = CONFIG["index_file"] - return root / index_file if index_file is not None else None - - -def prettify_title(stem): - """Display title for a page without an authored `title` field: - 'zero-trust-networking' -> 'Zero Trust Networking'.""" - return " ".join(w.capitalize() for w in stem.split("-")) - - -def _index_entry_link(p): - """OKF index entry link: bundle-absolute markdown link with the page's - authored title, falling back to a prettified stem.""" - title = (p.fields or {}).get("title") or prettify_title(p.stem) - return f"[{title}](/{p.rel})" - - -def generate_index_body(pages): - body_fn = CONFIG["index_body_fn"] - if body_fn is not None: - return body_fn(pages) - lines = [] - if CONFIG["index_mode"] == "subsystems": - lines.extend(_subsystem_section(pages)) - pool = [p for p in pages - if p.fields and p.fields.get("subsystem") == "global"] - else: - pool = pages - entry_extra = CONFIG["index_entry_extra"] - for section, types in CONFIG["index_sections"]: - members = sorted((p for p in pool if p.type in types), key=lambda p: p.rel) - if not members: - continue - lines.append(f"## {section}") - lines.append("") - for p in members: - desc = p.fields.get("description", "") if p.fields else "" - lines.append( - f"* {_index_entry_link(p)} - {desc} {entry_extra(p.fields or {})}".rstrip()) - lines.append("") - return "\n".join(lines).rstrip() + "\n" - - -def _subsystem_section(pages): - lines = ["## Subsystems", ""] - for p in sorted((p for p in pages if p.type == "subsystem"), key=lambda p: p.rel): - desc = p.fields.get("mission") or p.fields.get("description", "") - review = p.fields.get("last_owner_review", "never") - lines.append(f"* {_index_entry_link(p)} - {desc} (last review {review})") - lines.append("") - return lines - - -def _strip_leading_frontmatter(text): - """Drop a leading YAML frontmatter block (the okf_version stamp) so a - re-run of rebuild-index never accumulates duplicate blocks. Mirrors - Page._body_text but stays local: index.md is not a Page.""" - if not text.startswith("---"): - return text - lines = text.split("\n") - for i in range(1, len(lines)): - if lines[i].strip() == "---": - return "\n".join(lines[i + 1:]).lstrip("\n") - return text - - -def rebuild_index(root): - report = Report() - pages = discover_pages(root, report) - index = index_path(root) - if index is None: - print("index is disabled (index_file is None); nothing to rebuild") - return 0 - index_name = index.relative_to(root).as_posix() - index.parent.mkdir(parents=True, exist_ok=True) - # The okf_version stamp is a single fixed key: always regenerate it so a - # deleted or corrupted block self-heals on the next rebuild. - front = f'---\nokf_version: "{OKF_VERSION}"\n---\n\n' if CONFIG["okf_conformance"] else "" - head = "# Index\n\n" - if index.is_file(): - existing = _strip_leading_frontmatter( - index.read_text(encoding="utf-8", errors="replace")) - if GENERATED_MARKER in existing: - head = existing.split(GENERATED_MARKER)[0] - else: - m = YAML_FENCE_RE.search(existing) - if m and existing[:m.start()].strip() in ("", "# Index"): - head = existing[:m.end()] + "\n\n" - body = generate_index_body(pages) - index.write_text(f"{front}{head}{GENERATED_MARKER}\n\n{body}", encoding="utf-8") - # The built-in body lists entries as "* " lines; a custom index_body_fn can - # use any format, so report page count rather than guess at "entries". - if CONFIG["index_body_fn"] is not None: - print(f"{index_name} rebuilt ({len(pages)} pages)") - else: - entries = sum(1 for line in body.splitlines() if line.startswith("* ")) - print(f"{index_name} rebuilt: {entries} entries") - return 0 - - -def check_index_drift(pages, report, root): - index = index_path(root) - if index is None: - return - index_name = index.relative_to(root).as_posix() - if not index.is_file(): - report.warning("index", index_name, "missing; run `python3 lint.py rebuild-index`") - return - existing = index.read_text(encoding="utf-8", errors="replace") - if GENERATED_MARKER not in existing: - report.warning("index", index_name, "no generated marker; run `python3 lint.py rebuild-index`") - return - # rebuild_index writes exactly "{head}{MARKER}\n\n{body}", so compare the - # post-marker text against the same "\n\n"+body rather than lstrip-ing, - # which would also strip a body that legitimately starts with a newline. - current = existing.split(GENERATED_MARKER, 1)[1] - if current != f"\n\n{generate_index_body(pages)}": - report.warning("index", index_name, "out of date; run `python3 lint.py rebuild-index`") - - -def source_path_covers(source_paths, rel): - """Boundary-anchored: 'src' covers 'src' and 'src/x', never 'src-utils'.""" - return any( - sp and (sp == rel or sp.startswith(rel + "/") or rel.startswith(sp + "/")) - for sp in source_paths - ) - - -def run_coverage(root): - """Write coverage.md: per-subsystem module inventory by criticality, then - per-include-root directory coverage with the uncovered directories named.""" - report = Report() - pages = discover_pages(root, report) - pinning = parse_index_pinning(root) - local = pinning.get("local_path") - if not local: - index_name = CONFIG["index_file"] or "the index" - print(f"coverage: no local_path in {index_name} pinning block; skipping") - return 0 - repo = Path(local).expanduser() - if not repo.is_dir(): - print(f"coverage: {repo} not found; skipping") - return 0 - modules = [p for p in pages if p.type == "module" and p.fields] - # Fully generated, so stamp OKF-conformant frontmatter directly. - lines = ["---", "type: report", "---", "", - "# Coverage", "", f"Generated by lint.py against `{local}`.", ""] - lines.extend(_subsystem_coverage(modules)) - lines.extend(_directory_coverage(repo, pinning, modules)) - (root / "coverage.md").write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8") - print("coverage.md written") - return 0 - - -def _subsystem_coverage(modules): - crit_field = CONFIG["criticality_field"] - by_subsystem = {} - for p in modules: - by_subsystem.setdefault(p.fields.get("subsystem", "unassigned"), []).append(p) - lines = ["## Module pages by subsystem", ""] - for subsystem, members in sorted(by_subsystem.items()): - if crit_field: - counts = {} - for p in members: - crit = p.fields.get(crit_field, "unspecified") - counts[crit] = counts.get(crit, 0) + 1 - breakdown = ", ".join(f"{n} {c}" for c, n in sorted(counts.items())) - lines.append(f"- {subsystem}: {len(members)} module pages ({breakdown})") - else: - lines.append(f"- {subsystem}: {len(members)} module pages") - lines.append("") - return lines - - -def _directory_coverage(repo, pinning, modules): - scope = pinning.get("wiki_scope", {}) - includes = scope.get("include", []) if isinstance(scope, dict) else [] - excludes = scope.get("exclude", []) if isinstance(scope, dict) else [] - source_paths = [p.fields.get("source_path", "").strip("/") for p in modules] - lines = ["## Directory coverage", ""] - for inc in includes: - inc_dir = repo / inc.strip("/") - if not inc_dir.is_dir(): - continue - entries = sorted( - e for e in inc_dir.iterdir() - if e.is_dir() and not any(e.match(x) for x in excludes) - ) - covered = [] - uncovered = [] - for e in entries: - rel = e.relative_to(repo).as_posix() - (covered if source_path_covers(source_paths, rel) else uncovered).append(rel) - total = len(covered) + len(uncovered) - pct = round(100 * len(covered) / total) if total else 100 - lines.append(f"### {inc}") - lines.append("") - lines.append(f"{len(covered)}/{total} first-level directories covered ({pct}%)") - for u in uncovered: - lines.append(f"- uncovered: `{u}`") - lines.append("") - return lines diff --git a/codebase/wikilint/model.py b/codebase/wikilint/model.py deleted file mode 100644 index 4c2cbae..0000000 --- a/codebase/wikilint/model.py +++ /dev/null @@ -1,419 +0,0 @@ -"""Page model, parsers, and shared helpers.""" - -import re -from datetime import date -from fnmatch import fnmatch -from pathlib import Path, PurePosixPath - -from .settings import CONFIG - -GENERATED_MARKER = "" - -# OKF (Open Knowledge Format) version this engine produces and validates. -OKF_VERSION = "0.2" - -# Pages allowed to break kebab-case and to have no inbound links. -FILENAME_EXEMPT = ("README.md", "OWNERS.md") - -SECRET_PATTERNS = [ - (re.compile(r"(?i)\bpassword\s*[:=]\s*\S"), "password assignment"), - (re.compile(r"(?i)\bapi[_-]?key\s*[:=]\s*\S"), "api key assignment"), - (re.compile(r"(?i)\bsecret\s*[:=]\s*\S"), "secret assignment"), - (re.compile(r"(?i)\btoken\s*[:=]\s*[A-Za-z0-9._-]{8,}"), "token assignment"), - (re.compile(r"BEGIN (?:RSA |EC |OPENSSH |PGP )?PRIVATE KEY"), "private key block"), - (re.compile(r"\bAKIA[0-9A-Z]{16}\b"), "AWS access key id"), - (re.compile(r"(?i)wg[- ]?private"), "wireguard private key reference"), - (re.compile(r"Bearer\s+[A-Za-z0-9._~+/-]{20,}"), "bearer token"), -] - -# [text](target) markdown links, excluding images; target is group 1. An -# optional "title" after the target is tolerated. Images are blanked first -# (blank_images) so an image wrapped in a link exposes the link's target. -MD_LINK_RE = re.compile(r"(? [pages] and rel/rel_no_ext -> page for link resolution.""" - by_stem = {} - by_rel = {} - for p in pages: - by_stem.setdefault(p.stem, []).append(p) - by_rel[p.rel_no_ext] = p - by_rel[p.rel] = p - return by_stem, by_rel - - -def resolve_link(target, by_stem, by_rel): - target = target.split("|")[0].split("#")[0].strip() - # Bundle-absolute (leading /) targets resolve against the root, where - # page rel paths have no leading slash. - if target.startswith("/"): - target = target[1:] - if not target: - return None - if target in by_rel: - return by_rel[target] - stem = target[:-3] if target.endswith(".md") else target - candidates = by_stem.get(stem.split("/")[-1], []) - if "/" in target: - matches = [p for p in candidates if p.rel_no_ext.endswith(stem)] - return matches[0] if len(matches) == 1 else None - return candidates[0] if len(candidates) == 1 else None - - -def _pinning_fence(text): - """The yaml fence holding the pinning data: searched only above the - generated marker, and required to carry a pinning key, so example yaml - blocks elsewhere in the file can never displace it.""" - if GENERATED_MARKER in text: - text = text.split(GENERATED_MARKER)[0] - for m in YAML_FENCE_RE.finditer(text): - if re.search(r"^(repo|last_synced_commit):", m.group(1), re.MULTILINE): - return m.group(1) - return None - - -def parse_index_pinning(root): - """Parse the yaml pinning block at the top of the index, if any. - Handles quoted scalars and both inline and block-style nested lists.""" - if CONFIG["index_file"] is None: - return {} - index = root / CONFIG["index_file"] - if not index.is_file(): - return {} - block = _pinning_fence(index.read_text(encoding="utf-8", errors="replace")) - if block is None: - return {} - pinning = {} - current_map = None # top-level key holding a nested mapping - current_list = None # (map key, nested key) of a block list being filled - for line in block.splitlines(): - stripped = line.strip() - if not stripped or stripped.startswith("#"): - continue - if not line.startswith((" ", "\t")): - m = KEY_RE.match(line) - if not m: - continue - key, val = m.group(1), m.group(2).strip() - current_list = None - if val == "": - pinning[key] = {} - current_map = key - else: - pinning[key] = unquote(val) - current_map = None - elif stripped.startswith("- ") and current_list is not None: - map_key, key = current_list - pinning[map_key][key].append(unquote(stripped[2:])) - elif current_map is not None: - m = KEY_RE.match(stripped) - if not m: - continue - key, val = m.group(1), m.group(2).strip() - if val.startswith("[") and val.endswith("]"): - inner = val[1:-1].strip() - pinning[current_map][key] = ( - [unquote(v) for v in inner.split(",") if v.strip()] - if inner else [] - ) - current_list = None - elif val == "": - pinning[current_map][key] = [] - current_list = (current_map, key) - else: - pinning[current_map][key] = unquote(val) - current_list = None - return pinning diff --git a/codebase/wikilint/settings.py b/codebase/wikilint/settings.py deleted file mode 100644 index 52b1143..0000000 --- a/codebase/wikilint/settings.py +++ /dev/null @@ -1,120 +0,0 @@ -"""Mutable configuration holder and the config-validation boundary. - -cli.main() calls configure() once at startup with a variant's CONFIG; every -other module imports the CONFIG dict object and reads it live. Engine -extension points default to original wiki behavior via DEFAULTS, so a variant -only lists a key when it overrides one. configure() validates user-supplied -values (regexes, paths, callables) and fails fast with a clear message. -""" - -import re -from pathlib import PurePosixPath - -CONFIG = {} - -# Engine extension points. Defaults live here, once, and preserve the original -# wiki behavior; a variant's lint.py overrides a key only to change it. Reading -# these bare as CONFIG[key] is safe because configure() merges DEFAULTS under -# every variant config. -DEFAULTS = { - # Enforce OKF v0.2 conformance (check_okf) and stamp okf_version - # frontmatter into the rebuilt index. Off for non-OKF markdown trees. - "okf_conformance": True, - # Report pages with no inbound links (plus unlinked-mention hints). - "orphans": True, - # Path of the generated index relative to the wiki root; None disables the - # index build and drift check entirely. - "index_file": "index.md", - # Callable(pages) -> str replacing the built-in index body generator. - "index_body_fn": None, - # Frontmatter fields validated as ISO dates when present. The - # created/updated ordering check runs regardless of this list. - "iso_date_fields": ["created", "updated"], - # Extra (regex, label) pairs appended to the secrets scan. - "extra_secret_patterns": [], - # Secrets matches on lines matching any of these regexes are suppressed. - "secret_allow_res": [], - # Require taxonomy.md to carry a '## Page types' section describing every - # schema type with a one-line meaning, so the bundle self-describes its - # type vocabulary to OKF consumers. Off for non-wiki markdown trees. - "types_glossary": True, - # Directory of harness skill wrappers (e.g. ".claude/skills"), each a - # /SKILL.md whose body points at workflows/.md. When - # set, check_skills errors on any drift between the two sets. None - # disables. - "skills_dir": None, - # Wrapper-name prefix, namespacing wiki skills away from a user's global - # skills (field finding, 2026-07-22: a bare /triage collided with a - # global triage skill). "wiki-" pairs workflows/triage.md with - # /wiki-triage/SKILL.md. - "skills_prefix": "", - # Append-only operations log checked by check_log; None disables. - "log_file": "log.md", - # Callables(pages, report, root) run at the end of every check pass. - "extra_checks": [], -} - - -class ConfigError(Exception): - """A variant lint.py holds an invalid value for an engine knob.""" - - -def configure(config, index_entry_extra): - merged = {**DEFAULTS, **config} - merged["index_entry_extra"] = index_entry_extra - _validate(merged) - CONFIG.clear() - CONFIG.update(merged) - - -def _compile(pattern, key): - try: - return re.compile(pattern) - except re.error as e: - raise ConfigError(f"{key}: invalid regex {pattern!r}: {e}") - - -def _validate(cfg): - """Validate user-supplied extension values at the config boundary and - precompile the secret regexes so a bad pattern fails here, once, with a - clear message rather than mid-scan with a raw traceback.""" - index_file = cfg["index_file"] - if index_file is not None: - if not isinstance(index_file, str) or not index_file.strip(): - raise ConfigError("index_file must be None or a non-empty relative path") - posix = PurePosixPath(index_file) - if posix.is_absolute() or ".." in posix.parts: - raise ConfigError( - f"index_file must stay within the wiki root: {index_file!r}") - if cfg["log_file"] is not None and not isinstance(cfg["log_file"], str): - raise ConfigError("log_file must be None or a relative path string") - if not isinstance(cfg["types_glossary"], bool): - raise ConfigError("types_glossary must be a bool") - skills_dir = cfg["skills_dir"] - if skills_dir is not None: - if not isinstance(skills_dir, str) or not skills_dir.strip(): - raise ConfigError("skills_dir must be None or a non-empty relative path") - posix = PurePosixPath(skills_dir) - if posix.is_absolute() or ".." in posix.parts: - raise ConfigError( - f"skills_dir must stay within the wiki root: {skills_dir!r}") - if not isinstance(cfg["skills_prefix"], str): - raise ConfigError("skills_prefix must be a string") - if cfg["index_body_fn"] is not None and not callable(cfg["index_body_fn"]): - raise ConfigError("index_body_fn must be None or callable") - for fn in cfg["extra_checks"]: - if not callable(fn): - raise ConfigError(f"extra_checks entries must be callable: {fn!r}") - - compiled_extra = [] - for item in cfg["extra_secret_patterns"]: - try: - pattern, label = item - except (TypeError, ValueError): - raise ConfigError( - f"extra_secret_patterns entries must be (regex, label): {item!r}") - compiled_extra.append((_compile(pattern, "extra_secret_patterns"), label)) - cfg["secret_extra_compiled"] = compiled_extra - cfg["secret_allow_compiled"] = [ - _compile(pattern, "secret_allow_res") for pattern in cfg["secret_allow_res"] - ] diff --git a/codebase/workflows/adr.md b/codebase/workflows/adr.md deleted file mode 100644 index cff18b2..0000000 --- a/codebase/workflows/adr.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -type: workflow ---- - -# ADR - -Triggered by "draft an ADR for X". - -1. **Read whatever context exists**: relevant module pages, raw decision notes, transcripts, recent code changes. -2. **Draft using the standard structure**: Context, Decision, Consequences, Alternatives Considered. Keep each section short and concrete. No filler. Write the Decision section as one declarative sentence. -3. **Number it next in sequence**, zero-padded: `adrs/0008-.md`. Set `status: proposed`. -4. **Handle supersession.** If it supersedes an existing ADR, list that ADR in `supersedes`, and edit the old ADR's frontmatter to set `superseded_by` and flip its `status` to `superseded`. This frontmatter flip is the one allowed edit to a past ADR; its body stays untouched. -5. **Show the human the draft. Wait for approval.** On approval, set `status: accepted` and link it from affected module and service pages. -6. **Rebuild the index** (`python3 lint.py rebuild-index`), **append to `log.md`**, then commit: `adr: NNNN `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **ADR**: 0008 stripe-over-adyen -- **Status**: accepted -- **Linked**: services/payments.md, modules/payments-core.md -``` diff --git a/codebase/workflows/document.md b/codebase/workflows/document.md deleted file mode 100644 index 63a3169..0000000 --- a/codebase/workflows/document.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -type: workflow ---- - -# Document - -Triggered by "document `src/auth/`" or "document the new payments service". - -1. **Read the code.** Open the actual source files. Do not infer behavior from filenames or guess from imports. If the code is too large to read fully, read the package's public surface first (exported functions, types, routes), then drill into the most important internals. -2. **Discuss before writing.** Tell the human what you found, what you'll file, and where. Confirm scope: is this one module page or several? Does it need a service page too? API or data-model pages? If the pass will create concept or synthesis pages, apply the page-worthiness test below and say so in the plan. -3. **Create or update the module/service/api/data-model pages.** Fill all required frontmatter, including a one-sentence `description`. Set `last_verified_commit` to the current SHA and `last_verified` to today. Cite specific file paths and line ranges in the prose. If the code contradicts what a page already claims, do not overwrite: set `confidence: contested` and structure the body latest-evidence-first with dates, so `workflows/reconcile.md` has clean input. The new page's own forward fields (`depends_on`, `modules`, `exposes`, `consumes`, `defined_in`, `producers`, `consumers`) are the only stored edges; do not add reverse bookkeeping to other pages. The reverse view is derived: `python3 lint.py reverse-deps`. -4. **Update affected architecture pages.** Add the new module to the relevant Mermaid diagrams and update `includes`. Every module must land in at least one architecture page. -5. **File an ADR if a real decision was made.** Why this library, why this pattern, why this boundary. Use the next ADR number. Cite it from the module page. -6. **Rebuild the index and lint.** Run `python3 lint.py rebuild-index`, then `python3 lint.py check` and fix any errors it reports. -7. **Append to `log.md`** using the format below, then commit: `document: `. -8. **Report back** with every file created or updated and any inconsistencies you couldn't resolve. - -## The page-worthiness test - -Module, service, api, data-model, architecture, ADR, runbook, and postmortem pages are inventory: the code, its history, and its operation make them necessary. Any other new page (concept, synthesis) must pass all four parts: - -1. **Topic shape.** It defines something referenceable by name: a concept, a pattern, a term, a convention, a metric. Not a grab-bag, not "assorted notes". -2. **Not meta.** It is not an overview, introduction, getting-started, quickstart, tutorial, walkthrough, FAQ, release-notes, changelog, or roadmap. If the natural filename would be one of those words, stop. -3. **Citation test.** You can write a sentence in an existing page of the form "See [X](/concepts/x.md) for ..." where X is a concrete noun. If the best sentence you can write is "see this page for context", it fails. -4. **Reuse test.** At least two existing pages would cite it, or one page needs it as load-bearing background that does not fit inline. - -When in doubt, do not create the page; integrate the material into an existing one instead. A wiki with few concept pages is fine; a wiki full of `concepts/overview.md` and `concepts/misc-notes.md` is noise. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Document**: payments service -- **Creation**: services/payments.md, modules/payments-core.md, apis/payments-http.md -- **Update**: architecture/system.md -- **ADR**: adrs/0008-stripe-over-adyen.md -``` diff --git a/codebase/workflows/lint.md b/codebase/workflows/lint.md deleted file mode 100644 index 9a7377d..0000000 --- a/codebase/workflows/lint.md +++ /dev/null @@ -1,34 +0,0 @@ ---- -type: workflow ---- - -# Lint - -Triggered by "lint the wiki". Lint has two halves: the script does everything mechanical; you do only the checks that need judgment. - -1. **Run `python3 lint.py check`** and relay its findings to the human, grouped by severity. Do not re-verify by hand what the script already covers: frontmatter validity, broken links, orphans, dangling references (including `defined_in` and `includes`), architecture membership of active modules, tags, contested age, sync drift against the pinning block, missing Mermaid diagrams on architecture pages, ADR numbering gaps and superseded ADRs without forward links, secrets, inbox health, index drift, log format, OKF conformance. -2. **Orphan repair.** For each orphan the script reports, use its unlinked-mention hints to propose link insertions; insert on approval. If no genuine mention exists anywhere, recommend merging or deleting the page (ask). Never force a link where the prose doesn't naturally mention the page. -3. **Dead code references.** File paths and line-range citations in prose that point at code that no longer exists, and `source_path` values that have moved. This needs access to the pinned repo; the script cannot see the code. -4. **Architecture fiction.** Architecture pages whose `includes` or diagrams reference modules that are `status: removed` or clearly gone. The script catches nonexistent pages; you judge status. -5. **Diagram semantic drift.** Mermaid diagrams whose nodes or edges no longer match what the pages and the code say. -6. **ADR contradictions.** Accepted ADRs whose decisions contradict newer accepted ADRs. -7. **Contested without explanation.** Any `confidence: contested` page must explain the disagreement in its body; flag ones that don't, and offer to run `workflows/reconcile.md` on pages the script flags as long-contested. -8. **Stale runbooks.** Runbooks not touched in 6 months for services that are still active. -9. **Concept gaps.** Domain terms appearing across many pages with no dedicated concept page. -10. **Description drift.** Spot-check a handful of pages: does the `description` still match the body? A wrong description poisons every future scan. -11. **Investigation suggestions.** Three to five questions the wiki currently can't answer well. - -Report findings as a structured list. Do not auto-fix anything except trivially safe broken links (where the rename target is unambiguous). Wait for human direction on the rest. - -When done: append a lint entry to `log.md` and commit: `lint: findings`. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Lint**: run complete -- **Script**: 2 errors, 4 warnings -- **Judgment**: 1 dead code reference, 2 concept gaps -- **Note**: see lint report in chat -``` diff --git a/codebase/workflows/postmortem.md b/codebase/workflows/postmortem.md deleted file mode 100644 index 3c561b6..0000000 --- a/codebase/workflows/postmortem.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -type: workflow ---- - -# Postmortem - -Triggered by "log postmortem: ". - -1. **Create `postmortems/-.md`.** Standard frontmatter plus severity, duration, and the services and modules affected. -2. **Walk through the timeline with the human**: what was noticed, what was tried, what worked, root cause. -3. **Link every module and service involved** with markdown links, and add the postmortem to those pages' "incidents" section. -4. **End with action items**: new runbooks, ADRs, refactor proposals. Create the follow-ups in the same session if the human approves. -5. **Rebuild the index and lint.** Run `python3 lint.py rebuild-index`, then `python3 lint.py check` and fix any errors it reports. -6. **Append to `log.md`**, then commit: `postmortem: `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Postmortem**: payments-outage -- **Creation**: postmortems/2026-04-10-payments-outage.md -- **Update**: services/payments.md, modules/payments-core.md -- **Follow-ups**: runbooks/restore-payments.md -``` diff --git a/codebase/workflows/query.md b/codebase/workflows/query.md deleted file mode 100644 index 6cf39ed..0000000 --- a/codebase/workflows/query.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -type: workflow ---- - -# Query - -Triggered by any question answerable from the wiki or the code. - -1. **Scan `index.md` and frontmatter first** to find candidate pages. Do not guess at file paths, and do not open page bodies until a page's `description` or tags look relevant. -2. **Read the candidate pages fully**, then follow links one hop out for context. -3. **For code questions, offer to read the live source** instead of relying solely on the wiki, especially when a relevant page's `last_verified_commit` lags the pinning block. Reading the code repo is always allowed; modifying it never is. -4. **Synthesize an answer with citations.** Every claim links to the wiki page that supports it; claims about code cite `path/file.go:12-34` at a stated commit. -5. **Surface uncertainty.** If pages disagree, or the wiki lags the code, say so. If the question can't be answered, say so and suggest what to document next. -6. **Offer to file the answer.** If it is non-trivial and likely useful later, ask whether to save it under `queries/`. If filed: create the page, run `python3 lint.py rebuild-index`, append to `log.md`, and commit: `query: `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Query**: "What writes to the sessions table?" -- **Filed**: queries/sessions-table-writers.md -- **Read**: 6 pages -``` diff --git a/codebase/workflows/reconcile.md b/codebase/workflows/reconcile.md deleted file mode 100644 index e912263..0000000 --- a/codebase/workflows/reconcile.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -type: workflow ---- - -# Reconcile - -Triggered by "reconcile " or by lint flagging a long-unreconciled contested page. Contested is a state to exit, not a resting place: this workflow resolves the disagreement and records what lost. - -1. **Gather the conflict.** Read the page and every source it cites. List each competing claim with its source and the source's date. -2. **Propose a resolution.** Rank claims by source recency first, then source authority (primary sources over secondary, the human's direct word over both). State which claim should win and why, show the proposed rewrite, and wait for approval. -3. **Rewrite in place.** The winning claim goes in the main body, stated plainly. Each losing claim moves to a "Superseded claims" section at the bottom: one dated line per claim with its source and a one-line rationale for why it lost. Never silently delete the loser. -4. **Update frontmatter.** Remove `confidence: contested` (downgrade to `low` instead if the winner is itself uncited). Bump `updated` and refresh `description` if the page's conclusion changed. -5. **Check the neighbors.** Follow the page's links one hop out. Any page repeating the losing claim gets the same treatment in this session. -6. **Lint, log, commit.** Run `python3 lint.py check`, append to `log.md`, commit: `reconcile: `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-07-05 - -- **Reconcile**: zero-trust-networking -- **Winner**: sources/formal-verification-paper.md (2026-05) -- **Superseded**: claim from sources/early-blog-post.md, moved to Superseded claims -- **Update**: concepts/microsegmentation.md -``` diff --git a/codebase/workflows/sync.md b/codebase/workflows/sync.md deleted file mode 100644 index 08d5697..0000000 --- a/codebase/workflows/sync.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -type: workflow ---- - -# Sync - -Triggered by "sync to current". - -1. **Read the current commit SHA** of the local repo (`local_path` in the pinning block). Read-only; never modify the code repo. -2. **Compare to `last_synced_commit`** in `index.md`. If unchanged, say so and stop. -3. **List all files changed** between the two commits (`git diff --name-only ..` in the code repo). -4. **For each changed file, find wiki pages whose `source_path` covers it.** Scan frontmatter; do not open bodies for this. -5. **Mark each affected page as needing reverification.** List them in chat with their current `last_verified_commit`. -6. **Ask the human which to verify now and which to defer.** Do not auto-update `last_verified_commit` without re-reading the code. -7. **Update the pinning block by hand.** Set `last_synced_commit` and `last_synced_at` in the yaml block at the top of `index.md`. It lives above the generated marker, so `rebuild-index` will not touch it; you must edit it yourself. -8. **Append a sync entry to `log.md`** with what changed and what was flagged, then commit: `sync: -> `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Sync**: a1b2c3d -> e4f5g6h -- **Changed**: 23 -- **Pages**: modules/auth, modules/users, services/api-gateway -``` diff --git a/codebase/workflows/triage.md b/codebase/workflows/triage.md deleted file mode 100644 index 1ae7a1d..0000000 --- a/codebase/workflows/triage.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -type: workflow ---- - -# Triage - -The human drops new material into `raw/inbox/` without sorting or renaming. Triage is bulk and cheap; documenting is per-thing and slow. Never collapse them into a single step unless the human explicitly waives it. - -1. **List everything in `raw/inbox/`.** Read enough of each file to classify it. For binaries (PDFs, images), inspect the first page or two. -2. **Propose a plan as a table.** For each file: current name, proposed destination subfolder under `raw/` (`transcripts/`, `external-docs/`, `decisions/`, or other), proposed kebab-case filename, and a one-line "what this is" summary. If a file is unclear, mark it `needs human input` and ask. -3. **Wait for explicit approval.** Do not move anything until the human confirms the plan or sends corrections. If they correct individual rows, update those rows and re-show the table. -4. **Move and rename.** Once approved, move each file to its destination with the new name. Move any associated assets into `raw/assets//` if they exist alongside the source. -5. **Append a triage entry to `log.md`** listing what moved where, then commit: `triage: items from inbox`. -6. **Stop.** Triage does not write wiki pages and does not touch module, service, or architecture pages. That happens in document, separately. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Triage**: 4 items from inbox -- **Move**: raw/inbox/notes.txt -> raw/decisions/payments-provider-notes.md -- **Move**: raw/inbox/standup.m4a -> raw/transcripts/2026-04-09-standup.m4a -``` diff --git a/codebase/workflows/verify.md b/codebase/workflows/verify.md deleted file mode 100644 index b01fa30..0000000 --- a/codebase/workflows/verify.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -type: workflow ---- - -# Verify - -Triggered by "verify modules" or "verify modules/". - -1. **List the pages to verify.** If a specific page was named, just that one. Otherwise every page whose `last_verified_commit` lags `last_synced_commit`; `python3 lint.py check` reports these as sync-drift warnings. -2. **For each, re-read the actual source files** and compare against the page's claims. Look for: removed exports, changed signatures, new dependencies, removed dependencies, changed file paths. -3. **Propose updates as a diff.** Show the human exactly what would change on each page before touching it. -4. **On approval, update the page** and bump `last_verified_commit` and `last_verified`. Update `depends_on` and the page's other forward edges if the code's dependencies changed. If nothing changed in the code, bumping the two fields is the whole edit. -5. **Rebuild the index if needed.** If any `description`, `status`, or `last_verified_commit` changed, run `python3 lint.py rebuild-index`. -6. **Append a verify entry to `log.md`**, then commit: `verify: `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Verify**: modules/auth -- **Changes**: SessionStore.Refresh signature changed, removed LegacyToken -- **Note**: last_verified_commit bumped to e4f5g6h -``` diff --git a/generic/.claude/skills/wiki-lint/SKILL.md b/generic/.claude/skills/wiki-lint/SKILL.md deleted file mode 100644 index 0f3dbd6..0000000 --- a/generic/.claude/skills/wiki-lint/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-lint -description: Run the deterministic lint plus the judgment-only review pass. Use when the human says "lint the wiki". ---- - -Read `workflows/lint.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/generic/.claude/skills/wiki-maintain/SKILL.md b/generic/.claude/skills/wiki-maintain/SKILL.md deleted file mode 100644 index 5aad0f5..0000000 --- a/generic/.claude/skills/wiki-maintain/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-maintain -description: Run or review the unattended maintenance pass on the maintenance branch. Use when the human says "maintenance pass" or "review maintenance". ---- - -Read `workflows/maintain.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/generic/.githooks/pre-commit b/generic/.githooks/pre-commit deleted file mode 100755 index 1bae369..0000000 --- a/generic/.githooks/pre-commit +++ /dev/null @@ -1,18 +0,0 @@ -#!/bin/sh -# Wiki pre-commit gate: lint errors are uncommittable. -# Lints the STAGED snapshot (not the working tree), so the gate evaluates -# exactly the bytes that will land in the commit. -# Install once per clone: git config core.hooksPath .githooks -root="$(git rev-parse --show-toplevel)" || exit 1 -tmp="$(mktemp -d)" || exit 1 -trap 'rm -rf "$tmp"' 0 -git -C "$root" checkout-index --all --prefix="$tmp/" || exit 1 -cd "$tmp" || exit 1 -python3 lint.py check -status=$? -if [ $status -ne 0 ]; then - echo "" - echo "pre-commit: lint errors in the staged snapshot block the commit." - echo "Fix the files, git add them, and retry; do not bypass with --no-verify." -fi -exit $status diff --git a/generic/wikilint/__init__.py b/generic/wikilint/__init__.py deleted file mode 100644 index f185e69..0000000 --- a/generic/wikilint/__init__.py +++ /dev/null @@ -1,7 +0,0 @@ -"""Shared wiki lint engine. This package is byte-identical across all -Scriptorium variants; per-variant behavior lives entirely in the CONFIG -dict and index_entry_extra() defined in the wiki root's lint.py.""" - -from .cli import main - -__all__ = ["main"] diff --git a/generic/wikilint/checks.py b/generic/wikilint/checks.py deleted file mode 100644 index bc9884d..0000000 --- a/generic/wikilint/checks.py +++ /dev/null @@ -1,671 +0,0 @@ -"""All mechanical checks except index drift (which lives with the index -builder in derived.py).""" - -import re -from datetime import date, datetime -from urllib.parse import unquote - -from .model import ( - ADR_FILE_RE, FILENAME_EXEMPT, KEBAB_RE, LOG_DATE_HEADER_RE, LOG_ENTRY_RE, - MD_LINK_RE, OKF_VERSION, SECRET_PATTERNS, blank_frontmatter, blank_images, - blank_inline_code, build_link_index, discover_okf_bundle, line_at, - parse_frontmatter, parse_index_pinning, parse_iso_date, resolve_link, -) -from .settings import CONFIG - -# A URI scheme (mailto:, tel:, https:, data:, ...) or protocol-relative //host. -URI_SCHEME_RE = re.compile(r"^(?:[a-zA-Z][a-zA-Z0-9+.\-]*:|//)") - - -def missing_value(value): - """True when a required field is absent in practice: None, "", or [].""" - return value is None or value == "" or value == [] - - -def check_frontmatter(pages, report): - valid_types = set(CONFIG["type_required"]) - for p in pages: - if p.fm_error: - report.error("frontmatter", p.rel, p.fm_error) - continue - ptype = p.type - if ptype not in valid_types: - report.error("frontmatter", p.rel, f"invalid or missing type: {ptype!r}") - continue - for field in CONFIG["required_fields"] + CONFIG["type_required"][ptype]: - if missing_value(p.fields.get(field)): - report.error("frontmatter", p.rel, f"missing or empty required field: {field}") - check_enums(p, ptype, report) - check_dates(p, report) - desc = p.fields.get("description", "") - if isinstance(desc, str) and len(desc) > 200: - report.warning("frontmatter", p.rel, "description over 200 chars; keep it one line") - - -def check_enums(p, ptype, report): - for field, allowed in CONFIG["enum_fields"].items(): - if field in p.fields and p.fields[field] not in allowed: - report.error("frontmatter", p.rel, f"{field}: {p.fields[field]!r} not in {allowed}") - for field, allowed in CONFIG["type_enum_fields"].get(ptype, {}).items(): - value = p.fields.get(field) - if value is not None and value not in allowed: - report.error("frontmatter", p.rel, f"{field}: {value!r} not in {allowed}") - - -def check_dates(p, report): - for field in CONFIG["iso_date_fields"]: - value = p.fields.get(field) - if value and parse_iso_date(value) is None: - report.error("frontmatter", p.rel, f"{field} is not an ISO date") - # The created/updated ordering invariant holds regardless of which fields - # are ISO-validated above. - created = parse_iso_date(p.fields.get("created")) - updated = parse_iso_date(p.fields.get("updated")) - if created and updated and updated < created: - report.error("frontmatter", p.rel, "updated is older than created") - - -def check_filenames(pages, report): - for p in pages: - if p.path.name in FILENAME_EXEMPT: - continue - if not KEBAB_RE.match(p.path.name): - report.warning("filename", p.rel, "not kebab-case") - - -def _file_identity(path): - """(device, inode) identity for a path, or None if it can't be stat'd. - Used to match a link target to a page independent of the spelling of the - path (case on case-insensitive filesystems, symlinks, mount aliases).""" - try: - st = path.stat() - except OSError: - return None - return (st.st_dev, st.st_ino) - - -def check_markdown_links(p, pages_by_identity, report, root): - """Every [text](target) link must resolve on the filesystem (file or - directory): bundle-absolute targets (leading /, OKF's recommended form) - resolve against the wiki root, relative targets against the linking - page's directory. Returns the resolved page targets for inbound counting. - External (scheme/protocol-relative) and pure-anchor targets are skipped.""" - resolved_pages = [] - # resolve() both sides so symlinked roots (e.g. /var -> /private/var) - # compare equal when checking bundle containment. - root_resolved = root.resolve() - # Body content only, images and code blanked, line numbers file-accurate. - scan = blank_images(blank_frontmatter(p.prose)) - for m in MD_LINK_RE.finditer(scan): - raw = m.group(1) - target = raw.split("#")[0] - if not target or URI_SCHEME_RE.match(target): - continue - if target.startswith("/"): - path = (root / unquote(target).lstrip("/")).resolve() - else: - path = (p.path.parent / unquote(target)).resolve() - if not path.exists(): - report.error( - "link", f"{p.rel}:{line_at(scan, m.start())}", - f"broken link ({raw})", - ) - continue - if path != root_resolved and root_resolved not in path.parents: - report.error( - "link", f"{p.rel}:{line_at(scan, m.start())}", - f"link escapes the wiki root ({raw})", - ) - continue - target_page = pages_by_identity.get(_file_identity(path)) - if target_page is not None: - resolved_pages.append(target_page) - return resolved_pages - - -def check_links_and_orphans(pages, report, root): - by_stem, by_rel = build_link_index(pages) - pages_by_identity = {} - for p in pages: - identity = _file_identity(p.path) - if identity is not None: - pages_by_identity[identity] = p - inbound = {p.rel: 0 for p in pages} - for p in pages: - for target in check_markdown_links(p, pages_by_identity, report, root): - if target.rel != p.rel: - inbound[target.rel] += 1 - for field in CONFIG["path_fields"] + CONFIG["edge_fields"]: - for value in p.path_list(field): - # External resources (URLs) are provenance, not wiki edges. - if URI_SCHEME_RE.match(value): - continue - target = resolve_link(value, by_stem, by_rel) - if target is None: - report.error( - "reference", p.rel, - f"{field} points to nonexistent page: {value}", - ) - elif target.rel != p.rel: - inbound[target.rel] += 1 - if not CONFIG["orphans"]: - return - orphans = [ - p for p in pages - if inbound[p.rel] == 0 and p.path.name not in FILENAME_EXEMPT - ] - for p in orphans: - report.warning("orphan", p.rel, "no inbound links from any page") - report_unlinked_mentions(p, pages, report) - - -def report_unlinked_mentions(orphan, pages, report): - """Hint generation for the cross-linker: places where an orphan's name - appears in prose without a link.""" - patterns = [ - re.compile(re.escape(orphan.stem.replace("-", " ")), re.IGNORECASE), - re.compile(re.escape(orphan.stem), re.IGNORECASE), - ] - for other in pages: - if other.rel == orphan.rel: - continue - for pattern in patterns: - m = pattern.search(other.prose) - if m: - report.info( - "orphan", orphan.rel, - f"mentioned unlinked in {other.rel}:{line_at(other.prose, m.start())}; " - "candidate link", - ) - break - - -def check_membership(pages, report): - """Active member pages (components/modules) must appear in at least one - container page's (topology/architecture) membership field.""" - rule = CONFIG["membership"] - if not rule: - return - by_stem, by_rel = build_link_index(pages) - contained = set() - for p in pages: - if p.type != rule["container_type"]: - continue - for value in p.field_list(rule["container_field"]): - target = resolve_link(value, by_stem, by_rel) - if target is not None: - contained.add(target.rel) - for p in pages: - if p.type != rule["member_type"] or not p.fields: - continue - if p.fields.get("status") not in rule["active_statuses"]: - continue - if p.rel not in contained: - report.warning( - "membership", p.rel, - f"active {rule['member_type']} not included in any " - f"{rule['container_type']} page", - ) - - -def check_claude_size(root, report): - claude = root / "CLAUDE.md" - if not claude.is_file(): - return - lines = len(claude.read_text(encoding="utf-8", errors="replace").splitlines()) - cap = CONFIG["claude_md_max_lines"] - if lines > cap: - report.warning( - "hot-core", "CLAUDE.md", - f"{lines} lines exceeds the {cap}-line cap; " - "the always-loaded core is regrowing, move detail to workflows/", - ) - - -def load_taxonomy(root): - """Parse the taxonomy file into (tags, types). Tag entries are `- slug` - lines anywhere outside a `## Page types` section; entries inside that - section map type slug -> one-line meaning (empty string when the line has - no meaning after the dash). `types` is None when the section is absent.""" - path = root / CONFIG["taxonomy_file"] - if not path.is_file(): - return None - tags = set() - types = None - in_types = False - for line in path.read_text(encoding="utf-8", errors="replace").splitlines(): - if re.match(r"^##\s+Page types\s*$", line): - in_types = True - if types is None: - types = {} - continue - if re.match(r"^#{1,6}\s", line): - in_types = False - continue - m = re.match(r"^- ([a-z0-9][a-z0-9-]*)(?:\s*—\s*(.*))?", line) - if m: - if in_types: - types[m.group(1)] = (m.group(2) or "").strip() - else: - tags.add(m.group(1)) - return tags, types - - -def check_tags(pages, report, root): - if CONFIG["taxonomy_file"] is None: - return - taxonomy = load_taxonomy(root) - if taxonomy is None: - report.warning( - "tags", CONFIG["taxonomy_file"], - "missing; create it and list the allowed tags", - ) - return - taxonomy, _types = taxonomy - used = set() - for p in pages: - for tag in p.field_list("tags"): - used.add(tag) - if tag not in taxonomy: - report.warning( - "tags", p.rel, - f"tag {tag!r} not in {CONFIG['taxonomy_file']}; " - "normalize it or add it there in the same commit", - ) - for unused in sorted(taxonomy - used): - report.info("tags", CONFIG["taxonomy_file"], f"tag {unused!r} is used by no page") - - -def check_types(pages, report, root): - """The type glossary: taxonomy.md's `## Page types` section must describe - every schema type with a one-line meaning, so the bundle self-describes - its type vocabulary to foreign OKF consumers.""" - if CONFIG["taxonomy_file"] is None or not CONFIG["types_glossary"]: - return - taxonomy = load_taxonomy(root) - if taxonomy is None: - return # check_tags already reports the missing file - _tags, types = taxonomy - tfile = CONFIG["taxonomy_file"] - if types is None: - report.warning( - "types", tfile, - "no '## Page types' section; describe each page type there " - "with a one-line meaning", - ) - return - schema_types = set(CONFIG["type_required"]) - for t in sorted(schema_types - set(types)): - report.warning( - "types", tfile, - f"page type {t!r} is not described; add '- {t} — ' " - "under '## Page types'", - ) - for t in sorted(schema_types & set(types)): - if not types[t]: - report.warning( - "types", tfile, - f"page type {t!r} has no meaning; add one after the dash", - ) - for t in sorted(set(types) - schema_types): - report.warning( - "types", tfile, - f"'## Page types' describes unknown type {t!r}; remove it or " - "add it to the schema", - ) - - -def check_skills(pages, report, root): - """Harness skill wrappers must pair 1:1 with workflows: every - workflows/.md needs //SKILL.md whose body points - back at the workflow file, and no wrapper may point at nothing. Errors, - not warnings: the wrappers are dispatch surface, and silent drift there - means a slash command runs a stale or missing procedure.""" - if CONFIG["skills_dir"] is None: - return - skills_root = root / CONFIG["skills_dir"] - workflows_root = root / "workflows" - wf_stems = ( - {p.stem for p in workflows_root.glob("*.md")} - if workflows_root.is_dir() else set() - ) - prefix = CONFIG["skills_prefix"] - skill_names = ( - {d.name for d in skills_root.iterdir() if d.is_dir()} - if skills_root.is_dir() else set() - ) - expected = {prefix + stem: stem for stem in wf_stems} - sdir = CONFIG["skills_dir"] - for name in sorted(set(expected) - skill_names): - report.error( - "skills", f"workflows/{expected[name]}.md", - f"no skill wrapper; add {sdir}/{name}/SKILL.md pointing at it", - ) - for name in sorted(skill_names - set(expected)): - report.error( - "skills", f"{sdir}/{name}/SKILL.md", - f"orphan wrapper; no workflow pairs with it " - f"(expected form: {prefix})", - ) - for name in sorted(set(expected) & skill_names): - stem = expected[name] - skill_md = skills_root / name / "SKILL.md" - rel = f"{sdir}/{name}/SKILL.md" - if not skill_md.is_file(): - report.error("skills", rel, "missing SKILL.md in wrapper directory") - continue - body = skill_md.read_text(encoding="utf-8", errors="replace") - if f"workflows/{stem}.md" not in body: - report.error( - "skills", rel, - f"wrapper does not reference workflows/{stem}.md; " - "the body must point at its workflow file", - ) - - -def check_contested_age(pages, report): - today = date.today() - for p in pages: - if (p.fields or {}).get("confidence") != "contested": - continue - updated = parse_iso_date(p.fields.get("updated")) - if updated is None: - continue - age = (today - updated).days - if age > CONFIG["contested_max_days"]: - report.warning( - "contested", p.rel, - f"contested and untouched for {age} days " - f"(limit {CONFIG['contested_max_days']}); run reconcile", - ) - - -def check_inferred_markers(pages, report): - for p in pages: - if "(inferred)" not in p.body_prose: - continue - if (p.fields or {}).get("confidence") not in ("low", "contested"): - report.warning( - "provenance", p.rel, - "contains '(inferred)' claims but confidence is not low", - ) - - -def check_inbox(root, report): - if CONFIG["inbox_dir"] is None: - return - inbox = root / CONFIG["inbox_dir"] - if not inbox.is_dir(): - return - items = [e for e in inbox.iterdir() if not e.name.startswith(".")] - if len(items) >= CONFIG["inbox_warn_count"]: - report.warning("inbox", CONFIG["inbox_dir"], f"{len(items)} items awaiting triage") - now = datetime.now() - for item in items: - age = (now - datetime.fromtimestamp(item.stat().st_mtime)).days - if age >= CONFIG["inbox_warn_age_days"]: - report.warning("inbox", item.name, f"in inbox for {age} days") - - -def secret_patterns(): - """Built-in secret patterns plus the variant's extra_secret_patterns, - compiled once at configure() time.""" - return SECRET_PATTERNS + CONFIG["secret_extra_compiled"] - - -def scan_text_for_secrets(text, rel, report): - """Scan one file's text; matches whose line matches any allow regex are - suppressed (template placeholders, redaction markers). Allow regexes are - precompiled; the line split is deferred until a suppression check needs it.""" - allow = CONFIG["secret_allow_compiled"] - lines = None - for pattern, label in secret_patterns(): - for m in pattern.finditer(text): - line_no = line_at(text, m.start()) - if allow: - if lines is None: - lines = text.split("\n") - if any(a.search(lines[line_no - 1]) for a in allow): - continue - report.error("secrets", f"{rel}:{line_no}", f"possible {label}") - - -def check_secrets(pages, report): - for p in pages: - scan_text_for_secrets(p.text, p.rel, report) - - -def check_field_staleness(pages, report): - today = date.today() - for rule in CONFIG["staleness"]: - for p in pages: - if p.type not in rule["types"] or not p.fields: - continue - value = p.fields.get(rule["field"]) - if not value: - continue - d = parse_iso_date(value) - if d is None: - report.error("staleness", p.rel, f"{rule['field']} is not an ISO date") - continue - age = (today - d).days - if age > rule["max_days"]: - report.add( - rule["severity"].upper(), "staleness", p.rel, - f"{rule['field']} is {age} days old (limit {rule['max_days']})", - ) - - -def check_sync_drift(pages, report, root): - if not CONFIG["sync_drift"]: - return - field, types = CONFIG["sync_drift"] - synced = parse_index_pinning(root).get("last_synced_commit") - if not synced: - return - crit_field = CONFIG["criticality_field"] - plain_counts = {} - for p in pages: - if p.type not in types or not p.fields: - continue - verified = p.fields.get(field) - if not verified or verified == synced: - continue - crit = p.fields.get(crit_field) if crit_field else None - if crit == "load-bearing" or not crit_field: - label = "load-bearing page" if crit_field else "page" - report.warning( - "sync-drift", p.rel, - f"{label} verified at {verified}, repo at {synced}", - ) - else: - plain_counts[crit or "unspecified"] = ( - plain_counts.get(crit or "unspecified", 0) + 1 - ) - for crit, count in sorted(plain_counts.items()): - report.info("sync-drift", crit, f"{count} stale pages (run verify)") - - -def check_owner_review(pages, report): - if not CONFIG["owner_review_max_days"]: - return - today = date.today() - for p in pages: - if p.type != "subsystem" or not p.fields: - continue - reviewed = parse_iso_date(p.fields.get("last_owner_review")) - if reviewed is None: - report.warning("owner-review", p.rel, "missing or invalid last_owner_review") - elif (today - reviewed).days > CONFIG["owner_review_max_days"]: - report.warning( - "owner-review", p.rel, - f"last owner review {(today - reviewed).days} days ago", - ) - - -def count_mermaid_nodes(block): - """Rough node count for a mermaid block: unique identifiers that are - declared with a shape or appear on either side of an edge.""" - ids = set() - for m in re.finditer(r"([A-Za-z][\w-]*)\s*(?:\[|\(|\{)", block): - ids.add(m.group(1)) - for m in re.finditer( - r"([A-Za-z][\w-]*)\s*(?:-->|---|-\.->|==>|--o|--x)\s*([A-Za-z][\w-]*)", block - ): - ids.add(m.group(1)) - ids.add(m.group(2)) - keywords = {"flowchart", "graph", "sequenceDiagram", "subgraph", "end", - "classDef", "class", "style", "direction", "erDiagram", "C4Context"} - return len(ids - keywords) - - -def check_mermaid(pages, report): - if not CONFIG["mermaid_required_types"]: - return - for p in pages: - if p.type not in CONFIG["mermaid_required_types"]: - continue - blocks = re.findall(r"```mermaid\n(.*?)```", p.body, re.DOTALL) - if not blocks: - report.error("mermaid", p.rel, "page type requires at least one mermaid diagram") - continue - for block in blocks: - nodes = count_mermaid_nodes(block) - if CONFIG["mermaid_node_error"] and nodes > CONFIG["mermaid_node_error"]: - report.error("mermaid", p.rel, f"diagram has ~{nodes} nodes; split it") - elif CONFIG["mermaid_node_warn"] and nodes > CONFIG["mermaid_node_warn"]: - report.warning("mermaid", p.rel, f"diagram has ~{nodes} nodes; consider splitting") - - -def check_adrs(pages, report, root): - for adr_dir in CONFIG["adr_dirs"]: - for dir_path in sorted(root.glob(adr_dir)): - if not dir_path.is_dir(): - continue - check_adr_dir(dir_path, pages, report, root) - - -def check_adr_dir(dir_path, pages, report, root): - numbers = {} - for p in pages: - if p.path.parent != dir_path: - continue - m = ADR_FILE_RE.match(p.path.name) - if not m: - report.warning("adr", p.rel, "not in NNNN-slug.md form") - continue - numbers[int(m.group(1))] = p - status = (p.fields or {}).get("status") - if status == "superseded" and not (p.fields or {}).get("superseded_by"): - report.error("adr", p.rel, "superseded ADR has no superseded_by forward link") - if numbers: - expected = set(range(min(numbers), max(numbers) + 1)) - for gap in sorted(expected - set(numbers)): - rel = dir_path.relative_to(root).as_posix() - report.warning("adr", rel, f"numbering gap: {gap:04d} missing") - - -def check_log(root, report): - """OKF log structure: '## YYYY-MM-DD' date headings, newest first, with - entries as '- **Action**: ...' bullets. This check owns log.md end to - end; check_okf deliberately does not re-validate it.""" - log_file = CONFIG["log_file"] - if log_file is None: - return - log = root / log_file - if not log.is_file(): - report.warning("log", log_file, "missing") - return - text = log.read_text(encoding="utf-8", errors="replace") - prev = None # (date, line_no) of the last valid heading seen - for i, line in enumerate(text.splitlines(), 1): - if line.startswith("## "): - m = LOG_DATE_HEADER_RE.match(line) - if not m: - report.warning( - "log", f"{log_file}:{i}", - f"heading not in '## YYYY-MM-DD' form: {line!r}", - ) - continue - d = parse_iso_date(m.group(1)) - if d is None: - report.warning("log", f"{log_file}:{i}", f"heading is not a valid date: {line!r}") - continue - if prev is not None and d > prev[0]: - report.warning( - "log", f"{log_file}:{i}", - f"headings out of order (want newest-first): {d} follows {prev[0]}", - ) - elif prev is not None and d == prev[0]: - report.warning( - "log", f"{log_file}:{i}", - f"duplicate heading for {d}; append entries under the existing one", - ) - prev = (d, i) - elif line.startswith("- ") and not LOG_ENTRY_RE.match(line): - report.warning( - "log", f"{log_file}:{i}", - "entry should open with a bold action word: '- **Update**: ...'", - ) - - -def check_sources(pages, report): - """OKF v0.2 sources shape (SPEC §5.1): each entry is a mapping whose - required resource names either an external URL or a bundle-absolute page - path. Plain-string entries (the pre-0.2 shape) still resolve as paths in - check_links_and_orphans, but warn until installs migrate.""" - if not CONFIG["okf_conformance"]: - return - for p in pages: - for i, entry in enumerate(p.field_list("sources"), 1): - if isinstance(entry, dict): - resource = entry.get("resource", "") - if not resource: - report.error( - "sources", p.rel, - f"sources entry {i} has no resource " - "(required per entry, OKF v0.2 §5.1)", - ) - elif not URI_SCHEME_RE.match(resource) and not resource.startswith("/"): - report.warning( - "sources", p.rel, - f"sources entry {i}: bundle paths should be " - f"bundle-absolute (/dir/page.md), got {resource!r}", - ) - else: - report.warning( - "sources", p.rel, - f"sources entry {i} is a plain string; the OKF v0.2 shape " - "is '- resource: /dir/page.md'", - ) - - -def check_okf(pages, report, root): - """OKF v0.2 conformance: (1) every non-reserved .md parses as frontmatter, - (2) with a non-empty type, (3) reserved files follow their structure. - Pages are skipped here: check_frontmatter is strictly stricter. log.md's - structure is owned by check_log, not re-validated here. raw/ is excluded - by design (immutable human-owned sources; see discover_okf_bundle).""" - if not CONFIG["okf_conformance"]: - return - page_rels = {p.rel for p in pages} - for path in discover_okf_bundle(root): - rel = path.relative_to(root).as_posix() - if rel in page_rels: - continue - fields, err = parse_frontmatter( - path.read_text(encoding="utf-8", errors="replace")) - if err: - report.error("okf", rel, f"OKF rule 1: {err}") - elif missing_value(fields.get("type")): - report.error("okf", rel, "OKF rule 2: missing or empty type") - index_file = CONFIG["index_file"] - if index_file and (root / index_file).is_file(): - fields, _ = parse_frontmatter( - (root / index_file).read_text(encoding="utf-8", errors="replace")) - if not fields or fields.get("okf_version") != OKF_VERSION: - report.error( - "okf", index_file, - f'OKF rule 3: missing okf_version: "{OKF_VERSION}" frontmatter; ' - "run `python3 lint.py rebuild-index`", - ) diff --git a/generic/wikilint/cli.py b/generic/wikilint/cli.py deleted file mode 100644 index 84383ee..0000000 --- a/generic/wikilint/cli.py +++ /dev/null @@ -1,90 +0,0 @@ -"""CLI entry point: subcommand dispatch and the check orchestrator.""" - -import sys -from pathlib import Path - -from . import checks -from .derived import check_index_drift, rebuild_index, run_coverage, run_reverse_deps -from .model import Report, discover_pages -from .settings import CONFIG, ConfigError, configure - -USAGE = """\ -Usage: - python3 lint.py check run all mechanical checks, exit 1 on errors - python3 lint.py rebuild-index regenerate the index from page frontmatter - python3 lint.py reverse-deps print the derived reverse-dependency maps - python3 lint.py coverage write coverage.md (variants with coverage enabled) -""" - - -def gather_report(root): - """Run every mechanical check and return the Report (no printing). - This is the testable core of `lint.py check`.""" - report = Report() - pages = discover_pages(root, report) - checks.check_frontmatter(pages, report) - checks.check_filenames(pages, report) - checks.check_links_and_orphans(pages, report, root) - checks.check_membership(pages, report) - checks.check_claude_size(root, report) - checks.check_tags(pages, report, root) - checks.check_types(pages, report, root) - checks.check_skills(pages, report, root) - checks.check_contested_age(pages, report) - checks.check_inferred_markers(pages, report) - checks.check_inbox(root, report) - checks.check_secrets(pages, report) - checks.check_field_staleness(pages, report) - checks.check_sync_drift(pages, report, root) - checks.check_owner_review(pages, report) - checks.check_mermaid(pages, report) - checks.check_adrs(pages, report, root) - checks.check_log(root, report) - checks.check_sources(pages, report) - checks.check_okf(pages, report, root) - check_index_drift(pages, report, root) - for extra in CONFIG["extra_checks"]: - # A third-party check must not abort the run: a raise here would - # discard every finding already gathered. Convert it to an error. - try: - extra(pages, report, root) - except Exception as e: - report.error( - "extra-check", getattr(extra, "__name__", "extra_check"), - f"raised {type(e).__name__}: {e}", - ) - return len(pages), report - - -def run_check(root): - count, report = gather_report(root) - print(f"checked {count} pages") - print(report.render()) - return 1 if report.has_errors else 0 - - -def main(config, index_entry_extra, argv=None, root=None): - try: - configure(config, index_entry_extra) - except ConfigError as e: - print(f"lint config error: {e}", file=sys.stderr) - return 2 - root = Path(root) if root else Path.cwd() - if not (root / "CLAUDE.md").is_file(): - print("run from the wiki root (CLAUDE.md not found here)", file=sys.stderr) - return 2 - args = sys.argv[1:] if argv is None else argv - command = args[0] if args else "check" - if command == "check": - return run_check(root) - if command == "rebuild-index": - return rebuild_index(root) - if command == "reverse-deps": - return run_reverse_deps(root) - if command == "coverage": - if not CONFIG["coverage"]: - print("coverage is not enabled for this variant") - return 2 - return run_coverage(root) - print(USAGE) - return 2 diff --git a/generic/wikilint/derived.py b/generic/wikilint/derived.py deleted file mode 100644 index 00fc5c0..0000000 --- a/generic/wikilint/derived.py +++ /dev/null @@ -1,249 +0,0 @@ -"""Derived artifacts: the generated index, reverse-dependency maps, and the -coverage report. check_index_drift lives here beside the builder it reuses.""" - -from pathlib import Path - -from .model import ( - GENERATED_MARKER, OKF_VERSION, YAML_FENCE_RE, build_link_index, - discover_pages, parse_index_pinning, resolve_link, Report, -) -from .settings import CONFIG - - -def derive_reverse_edges(pages): - """field -> {target rel: [source rels]} for every relationship field in - CONFIG['reverse_fields']. This is the derived view that replaces the old - stored mirror fields (consumed_by, public_surface, api consumers).""" - by_stem, by_rel = build_link_index(pages) - reverse = {} - for field in CONFIG["reverse_fields"]: - rev = {} - for p in pages: - for value in p.field_list(field): - target = resolve_link(value, by_stem, by_rel) - if target is not None: - rev.setdefault(target.rel, []).append(p.rel) - reverse[field] = rev - return reverse - - -def run_reverse_deps(root): - report = Report() - pages = discover_pages(root, report) - if not CONFIG["reverse_fields"]: - print("no reverse fields configured for this variant") - return 0 - for field, rev in derive_reverse_edges(pages).items(): - print(f"\nreverse of {field}:") - if not rev: - print(" (no edges)") - for target, sources in sorted(rev.items()): - print(f" {target} <- {', '.join(sorted(sources))}") - return 0 - - -def index_path(root): - """Absolute path of the generated index, or None when the index is - disabled (index_file is None).""" - index_file = CONFIG["index_file"] - return root / index_file if index_file is not None else None - - -def prettify_title(stem): - """Display title for a page without an authored `title` field: - 'zero-trust-networking' -> 'Zero Trust Networking'.""" - return " ".join(w.capitalize() for w in stem.split("-")) - - -def _index_entry_link(p): - """OKF index entry link: bundle-absolute markdown link with the page's - authored title, falling back to a prettified stem.""" - title = (p.fields or {}).get("title") or prettify_title(p.stem) - return f"[{title}](/{p.rel})" - - -def generate_index_body(pages): - body_fn = CONFIG["index_body_fn"] - if body_fn is not None: - return body_fn(pages) - lines = [] - if CONFIG["index_mode"] == "subsystems": - lines.extend(_subsystem_section(pages)) - pool = [p for p in pages - if p.fields and p.fields.get("subsystem") == "global"] - else: - pool = pages - entry_extra = CONFIG["index_entry_extra"] - for section, types in CONFIG["index_sections"]: - members = sorted((p for p in pool if p.type in types), key=lambda p: p.rel) - if not members: - continue - lines.append(f"## {section}") - lines.append("") - for p in members: - desc = p.fields.get("description", "") if p.fields else "" - lines.append( - f"* {_index_entry_link(p)} - {desc} {entry_extra(p.fields or {})}".rstrip()) - lines.append("") - return "\n".join(lines).rstrip() + "\n" - - -def _subsystem_section(pages): - lines = ["## Subsystems", ""] - for p in sorted((p for p in pages if p.type == "subsystem"), key=lambda p: p.rel): - desc = p.fields.get("mission") or p.fields.get("description", "") - review = p.fields.get("last_owner_review", "never") - lines.append(f"* {_index_entry_link(p)} - {desc} (last review {review})") - lines.append("") - return lines - - -def _strip_leading_frontmatter(text): - """Drop a leading YAML frontmatter block (the okf_version stamp) so a - re-run of rebuild-index never accumulates duplicate blocks. Mirrors - Page._body_text but stays local: index.md is not a Page.""" - if not text.startswith("---"): - return text - lines = text.split("\n") - for i in range(1, len(lines)): - if lines[i].strip() == "---": - return "\n".join(lines[i + 1:]).lstrip("\n") - return text - - -def rebuild_index(root): - report = Report() - pages = discover_pages(root, report) - index = index_path(root) - if index is None: - print("index is disabled (index_file is None); nothing to rebuild") - return 0 - index_name = index.relative_to(root).as_posix() - index.parent.mkdir(parents=True, exist_ok=True) - # The okf_version stamp is a single fixed key: always regenerate it so a - # deleted or corrupted block self-heals on the next rebuild. - front = f'---\nokf_version: "{OKF_VERSION}"\n---\n\n' if CONFIG["okf_conformance"] else "" - head = "# Index\n\n" - if index.is_file(): - existing = _strip_leading_frontmatter( - index.read_text(encoding="utf-8", errors="replace")) - if GENERATED_MARKER in existing: - head = existing.split(GENERATED_MARKER)[0] - else: - m = YAML_FENCE_RE.search(existing) - if m and existing[:m.start()].strip() in ("", "# Index"): - head = existing[:m.end()] + "\n\n" - body = generate_index_body(pages) - index.write_text(f"{front}{head}{GENERATED_MARKER}\n\n{body}", encoding="utf-8") - # The built-in body lists entries as "* " lines; a custom index_body_fn can - # use any format, so report page count rather than guess at "entries". - if CONFIG["index_body_fn"] is not None: - print(f"{index_name} rebuilt ({len(pages)} pages)") - else: - entries = sum(1 for line in body.splitlines() if line.startswith("* ")) - print(f"{index_name} rebuilt: {entries} entries") - return 0 - - -def check_index_drift(pages, report, root): - index = index_path(root) - if index is None: - return - index_name = index.relative_to(root).as_posix() - if not index.is_file(): - report.warning("index", index_name, "missing; run `python3 lint.py rebuild-index`") - return - existing = index.read_text(encoding="utf-8", errors="replace") - if GENERATED_MARKER not in existing: - report.warning("index", index_name, "no generated marker; run `python3 lint.py rebuild-index`") - return - # rebuild_index writes exactly "{head}{MARKER}\n\n{body}", so compare the - # post-marker text against the same "\n\n"+body rather than lstrip-ing, - # which would also strip a body that legitimately starts with a newline. - current = existing.split(GENERATED_MARKER, 1)[1] - if current != f"\n\n{generate_index_body(pages)}": - report.warning("index", index_name, "out of date; run `python3 lint.py rebuild-index`") - - -def source_path_covers(source_paths, rel): - """Boundary-anchored: 'src' covers 'src' and 'src/x', never 'src-utils'.""" - return any( - sp and (sp == rel or sp.startswith(rel + "/") or rel.startswith(sp + "/")) - for sp in source_paths - ) - - -def run_coverage(root): - """Write coverage.md: per-subsystem module inventory by criticality, then - per-include-root directory coverage with the uncovered directories named.""" - report = Report() - pages = discover_pages(root, report) - pinning = parse_index_pinning(root) - local = pinning.get("local_path") - if not local: - index_name = CONFIG["index_file"] or "the index" - print(f"coverage: no local_path in {index_name} pinning block; skipping") - return 0 - repo = Path(local).expanduser() - if not repo.is_dir(): - print(f"coverage: {repo} not found; skipping") - return 0 - modules = [p for p in pages if p.type == "module" and p.fields] - # Fully generated, so stamp OKF-conformant frontmatter directly. - lines = ["---", "type: report", "---", "", - "# Coverage", "", f"Generated by lint.py against `{local}`.", ""] - lines.extend(_subsystem_coverage(modules)) - lines.extend(_directory_coverage(repo, pinning, modules)) - (root / "coverage.md").write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8") - print("coverage.md written") - return 0 - - -def _subsystem_coverage(modules): - crit_field = CONFIG["criticality_field"] - by_subsystem = {} - for p in modules: - by_subsystem.setdefault(p.fields.get("subsystem", "unassigned"), []).append(p) - lines = ["## Module pages by subsystem", ""] - for subsystem, members in sorted(by_subsystem.items()): - if crit_field: - counts = {} - for p in members: - crit = p.fields.get(crit_field, "unspecified") - counts[crit] = counts.get(crit, 0) + 1 - breakdown = ", ".join(f"{n} {c}" for c, n in sorted(counts.items())) - lines.append(f"- {subsystem}: {len(members)} module pages ({breakdown})") - else: - lines.append(f"- {subsystem}: {len(members)} module pages") - lines.append("") - return lines - - -def _directory_coverage(repo, pinning, modules): - scope = pinning.get("wiki_scope", {}) - includes = scope.get("include", []) if isinstance(scope, dict) else [] - excludes = scope.get("exclude", []) if isinstance(scope, dict) else [] - source_paths = [p.fields.get("source_path", "").strip("/") for p in modules] - lines = ["## Directory coverage", ""] - for inc in includes: - inc_dir = repo / inc.strip("/") - if not inc_dir.is_dir(): - continue - entries = sorted( - e for e in inc_dir.iterdir() - if e.is_dir() and not any(e.match(x) for x in excludes) - ) - covered = [] - uncovered = [] - for e in entries: - rel = e.relative_to(repo).as_posix() - (covered if source_path_covers(source_paths, rel) else uncovered).append(rel) - total = len(covered) + len(uncovered) - pct = round(100 * len(covered) / total) if total else 100 - lines.append(f"### {inc}") - lines.append("") - lines.append(f"{len(covered)}/{total} first-level directories covered ({pct}%)") - for u in uncovered: - lines.append(f"- uncovered: `{u}`") - lines.append("") - return lines diff --git a/generic/wikilint/model.py b/generic/wikilint/model.py deleted file mode 100644 index 4c2cbae..0000000 --- a/generic/wikilint/model.py +++ /dev/null @@ -1,419 +0,0 @@ -"""Page model, parsers, and shared helpers.""" - -import re -from datetime import date -from fnmatch import fnmatch -from pathlib import Path, PurePosixPath - -from .settings import CONFIG - -GENERATED_MARKER = "" - -# OKF (Open Knowledge Format) version this engine produces and validates. -OKF_VERSION = "0.2" - -# Pages allowed to break kebab-case and to have no inbound links. -FILENAME_EXEMPT = ("README.md", "OWNERS.md") - -SECRET_PATTERNS = [ - (re.compile(r"(?i)\bpassword\s*[:=]\s*\S"), "password assignment"), - (re.compile(r"(?i)\bapi[_-]?key\s*[:=]\s*\S"), "api key assignment"), - (re.compile(r"(?i)\bsecret\s*[:=]\s*\S"), "secret assignment"), - (re.compile(r"(?i)\btoken\s*[:=]\s*[A-Za-z0-9._-]{8,}"), "token assignment"), - (re.compile(r"BEGIN (?:RSA |EC |OPENSSH |PGP )?PRIVATE KEY"), "private key block"), - (re.compile(r"\bAKIA[0-9A-Z]{16}\b"), "AWS access key id"), - (re.compile(r"(?i)wg[- ]?private"), "wireguard private key reference"), - (re.compile(r"Bearer\s+[A-Za-z0-9._~+/-]{20,}"), "bearer token"), -] - -# [text](target) markdown links, excluding images; target is group 1. An -# optional "title" after the target is tolerated. Images are blanked first -# (blank_images) so an image wrapped in a link exposes the link's target. -MD_LINK_RE = re.compile(r"(? [pages] and rel/rel_no_ext -> page for link resolution.""" - by_stem = {} - by_rel = {} - for p in pages: - by_stem.setdefault(p.stem, []).append(p) - by_rel[p.rel_no_ext] = p - by_rel[p.rel] = p - return by_stem, by_rel - - -def resolve_link(target, by_stem, by_rel): - target = target.split("|")[0].split("#")[0].strip() - # Bundle-absolute (leading /) targets resolve against the root, where - # page rel paths have no leading slash. - if target.startswith("/"): - target = target[1:] - if not target: - return None - if target in by_rel: - return by_rel[target] - stem = target[:-3] if target.endswith(".md") else target - candidates = by_stem.get(stem.split("/")[-1], []) - if "/" in target: - matches = [p for p in candidates if p.rel_no_ext.endswith(stem)] - return matches[0] if len(matches) == 1 else None - return candidates[0] if len(candidates) == 1 else None - - -def _pinning_fence(text): - """The yaml fence holding the pinning data: searched only above the - generated marker, and required to carry a pinning key, so example yaml - blocks elsewhere in the file can never displace it.""" - if GENERATED_MARKER in text: - text = text.split(GENERATED_MARKER)[0] - for m in YAML_FENCE_RE.finditer(text): - if re.search(r"^(repo|last_synced_commit):", m.group(1), re.MULTILINE): - return m.group(1) - return None - - -def parse_index_pinning(root): - """Parse the yaml pinning block at the top of the index, if any. - Handles quoted scalars and both inline and block-style nested lists.""" - if CONFIG["index_file"] is None: - return {} - index = root / CONFIG["index_file"] - if not index.is_file(): - return {} - block = _pinning_fence(index.read_text(encoding="utf-8", errors="replace")) - if block is None: - return {} - pinning = {} - current_map = None # top-level key holding a nested mapping - current_list = None # (map key, nested key) of a block list being filled - for line in block.splitlines(): - stripped = line.strip() - if not stripped or stripped.startswith("#"): - continue - if not line.startswith((" ", "\t")): - m = KEY_RE.match(line) - if not m: - continue - key, val = m.group(1), m.group(2).strip() - current_list = None - if val == "": - pinning[key] = {} - current_map = key - else: - pinning[key] = unquote(val) - current_map = None - elif stripped.startswith("- ") and current_list is not None: - map_key, key = current_list - pinning[map_key][key].append(unquote(stripped[2:])) - elif current_map is not None: - m = KEY_RE.match(stripped) - if not m: - continue - key, val = m.group(1), m.group(2).strip() - if val.startswith("[") and val.endswith("]"): - inner = val[1:-1].strip() - pinning[current_map][key] = ( - [unquote(v) for v in inner.split(",") if v.strip()] - if inner else [] - ) - current_list = None - elif val == "": - pinning[current_map][key] = [] - current_list = (current_map, key) - else: - pinning[current_map][key] = unquote(val) - current_list = None - return pinning diff --git a/generic/wikilint/settings.py b/generic/wikilint/settings.py deleted file mode 100644 index 52b1143..0000000 --- a/generic/wikilint/settings.py +++ /dev/null @@ -1,120 +0,0 @@ -"""Mutable configuration holder and the config-validation boundary. - -cli.main() calls configure() once at startup with a variant's CONFIG; every -other module imports the CONFIG dict object and reads it live. Engine -extension points default to original wiki behavior via DEFAULTS, so a variant -only lists a key when it overrides one. configure() validates user-supplied -values (regexes, paths, callables) and fails fast with a clear message. -""" - -import re -from pathlib import PurePosixPath - -CONFIG = {} - -# Engine extension points. Defaults live here, once, and preserve the original -# wiki behavior; a variant's lint.py overrides a key only to change it. Reading -# these bare as CONFIG[key] is safe because configure() merges DEFAULTS under -# every variant config. -DEFAULTS = { - # Enforce OKF v0.2 conformance (check_okf) and stamp okf_version - # frontmatter into the rebuilt index. Off for non-OKF markdown trees. - "okf_conformance": True, - # Report pages with no inbound links (plus unlinked-mention hints). - "orphans": True, - # Path of the generated index relative to the wiki root; None disables the - # index build and drift check entirely. - "index_file": "index.md", - # Callable(pages) -> str replacing the built-in index body generator. - "index_body_fn": None, - # Frontmatter fields validated as ISO dates when present. The - # created/updated ordering check runs regardless of this list. - "iso_date_fields": ["created", "updated"], - # Extra (regex, label) pairs appended to the secrets scan. - "extra_secret_patterns": [], - # Secrets matches on lines matching any of these regexes are suppressed. - "secret_allow_res": [], - # Require taxonomy.md to carry a '## Page types' section describing every - # schema type with a one-line meaning, so the bundle self-describes its - # type vocabulary to OKF consumers. Off for non-wiki markdown trees. - "types_glossary": True, - # Directory of harness skill wrappers (e.g. ".claude/skills"), each a - # /SKILL.md whose body points at workflows/.md. When - # set, check_skills errors on any drift between the two sets. None - # disables. - "skills_dir": None, - # Wrapper-name prefix, namespacing wiki skills away from a user's global - # skills (field finding, 2026-07-22: a bare /triage collided with a - # global triage skill). "wiki-" pairs workflows/triage.md with - # /wiki-triage/SKILL.md. - "skills_prefix": "", - # Append-only operations log checked by check_log; None disables. - "log_file": "log.md", - # Callables(pages, report, root) run at the end of every check pass. - "extra_checks": [], -} - - -class ConfigError(Exception): - """A variant lint.py holds an invalid value for an engine knob.""" - - -def configure(config, index_entry_extra): - merged = {**DEFAULTS, **config} - merged["index_entry_extra"] = index_entry_extra - _validate(merged) - CONFIG.clear() - CONFIG.update(merged) - - -def _compile(pattern, key): - try: - return re.compile(pattern) - except re.error as e: - raise ConfigError(f"{key}: invalid regex {pattern!r}: {e}") - - -def _validate(cfg): - """Validate user-supplied extension values at the config boundary and - precompile the secret regexes so a bad pattern fails here, once, with a - clear message rather than mid-scan with a raw traceback.""" - index_file = cfg["index_file"] - if index_file is not None: - if not isinstance(index_file, str) or not index_file.strip(): - raise ConfigError("index_file must be None or a non-empty relative path") - posix = PurePosixPath(index_file) - if posix.is_absolute() or ".." in posix.parts: - raise ConfigError( - f"index_file must stay within the wiki root: {index_file!r}") - if cfg["log_file"] is not None and not isinstance(cfg["log_file"], str): - raise ConfigError("log_file must be None or a relative path string") - if not isinstance(cfg["types_glossary"], bool): - raise ConfigError("types_glossary must be a bool") - skills_dir = cfg["skills_dir"] - if skills_dir is not None: - if not isinstance(skills_dir, str) or not skills_dir.strip(): - raise ConfigError("skills_dir must be None or a non-empty relative path") - posix = PurePosixPath(skills_dir) - if posix.is_absolute() or ".." in posix.parts: - raise ConfigError( - f"skills_dir must stay within the wiki root: {skills_dir!r}") - if not isinstance(cfg["skills_prefix"], str): - raise ConfigError("skills_prefix must be a string") - if cfg["index_body_fn"] is not None and not callable(cfg["index_body_fn"]): - raise ConfigError("index_body_fn must be None or callable") - for fn in cfg["extra_checks"]: - if not callable(fn): - raise ConfigError(f"extra_checks entries must be callable: {fn!r}") - - compiled_extra = [] - for item in cfg["extra_secret_patterns"]: - try: - pattern, label = item - except (TypeError, ValueError): - raise ConfigError( - f"extra_secret_patterns entries must be (regex, label): {item!r}") - compiled_extra.append((_compile(pattern, "extra_secret_patterns"), label)) - cfg["secret_extra_compiled"] = compiled_extra - cfg["secret_allow_compiled"] = [ - _compile(pattern, "secret_allow_res") for pattern in cfg["secret_allow_res"] - ] diff --git a/generic/workflows/maintain.md b/generic/workflows/maintain.md deleted file mode 100644 index 2dc9fc6..0000000 --- a/generic/workflows/maintain.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -type: workflow ---- - -# Maintain - -The unattended maintenance pass, run on a schedule or by "maintenance pass". Its safety comes from reversibility: all work happens on the `maintenance` branch, never on main, and the human merges only after review. - -Setup: `git checkout maintenance` (create it from main if absent; if it holds unmerged work from a prior run, continue on it). - -Allowed actions, exhaustively: - -1. **Run `python3 lint.py check` and fix the mechanical errors it reports**: broken links with unambiguous targets, frontmatter fields whose correct value is derivable (malformed dates, wrong enum casing), dangling references to renamed pages. -2. **Run `python3 lint.py rebuild-index`.** -3. **Insert cross-links for orphans**, only where the script's mention hints show an unambiguous prose mention of the orphan page. -4. **Collect, do not fix**: contested pages needing reconcile, concept gaps, taxonomy additions worth proposing, pages whose `description` has drifted from the body. These go in the run report, not into edits. - -Forbidden in unattended runs: changing the meaning of any prose, deleting, merging, or renaming pages, creating new page types or folders, resolving contested pages, editing `CLAUDE.md`, `workflows/`, or `taxonomy.md`, and touching `raw/`. - -Finish: append one log entry, commit everything as `maintain: run`, and leave the branch unmerged. If errors remain that the allowed actions can't fix (a secrets hit, an ambiguous link), do not commit; leave the tree dirty and put the blocker at the top of the run report. - -When the human says "review maintenance": walk them through `git diff main..maintenance` change by change, then merge on approval and continue reusing the branch. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-07-05 - -- **Maintain**: scheduled run -- **Fix**: 2 broken links; index rebuilt -- **Crosslink**: queries/x-vs-y.md from synthesis/overview.md -- **Proposal**: reconcile concepts/foo.md; add tag 'observability' to taxonomy -``` diff --git a/generic/workflows/reconcile.md b/generic/workflows/reconcile.md deleted file mode 100644 index e912263..0000000 --- a/generic/workflows/reconcile.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -type: workflow ---- - -# Reconcile - -Triggered by "reconcile " or by lint flagging a long-unreconciled contested page. Contested is a state to exit, not a resting place: this workflow resolves the disagreement and records what lost. - -1. **Gather the conflict.** Read the page and every source it cites. List each competing claim with its source and the source's date. -2. **Propose a resolution.** Rank claims by source recency first, then source authority (primary sources over secondary, the human's direct word over both). State which claim should win and why, show the proposed rewrite, and wait for approval. -3. **Rewrite in place.** The winning claim goes in the main body, stated plainly. Each losing claim moves to a "Superseded claims" section at the bottom: one dated line per claim with its source and a one-line rationale for why it lost. Never silently delete the loser. -4. **Update frontmatter.** Remove `confidence: contested` (downgrade to `low` instead if the winner is itself uncited). Bump `updated` and refresh `description` if the page's conclusion changed. -5. **Check the neighbors.** Follow the page's links one hop out. Any page repeating the losing claim gets the same treatment in this session. -6. **Lint, log, commit.** Run `python3 lint.py check`, append to `log.md`, commit: `reconcile: `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-07-05 - -- **Reconcile**: zero-trust-networking -- **Winner**: sources/formal-verification-paper.md (2026-05) -- **Superseded**: claim from sources/early-blog-post.md, moved to Superseded claims -- **Update**: concepts/microsegmentation.md -``` diff --git a/homelab/.claude/skills/wiki-document/SKILL.md b/homelab/.claude/skills/wiki-document/SKILL.md deleted file mode 100644 index 6a72469..0000000 --- a/homelab/.claude/skills/wiki-document/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-document -description: Document a homelab component, config, or change in the wiki. Use when the human says "document the new VM", "document this OPNsense config", or similar. ---- - -Read `workflows/document.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/homelab/.claude/skills/wiki-incident/SKILL.md b/homelab/.claude/skills/wiki-incident/SKILL.md deleted file mode 100644 index f91473f..0000000 --- a/homelab/.claude/skills/wiki-incident/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-incident -description: File an incident page for something that broke. Use when the human says "log incident: " or describes an outage to record. ---- - -Read `workflows/incident.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/homelab/.claude/skills/wiki-lint/SKILL.md b/homelab/.claude/skills/wiki-lint/SKILL.md deleted file mode 100644 index 0f3dbd6..0000000 --- a/homelab/.claude/skills/wiki-lint/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-lint -description: Run the deterministic lint plus the judgment-only review pass. Use when the human says "lint the wiki". ---- - -Read `workflows/lint.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/homelab/.claude/skills/wiki-maintain/SKILL.md b/homelab/.claude/skills/wiki-maintain/SKILL.md deleted file mode 100644 index 5aad0f5..0000000 --- a/homelab/.claude/skills/wiki-maintain/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-maintain -description: Run or review the unattended maintenance pass on the maintenance branch. Use when the human says "maintenance pass" or "review maintenance". ---- - -Read `workflows/maintain.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/homelab/.claude/skills/wiki-query/SKILL.md b/homelab/.claude/skills/wiki-query/SKILL.md deleted file mode 100644 index 7810d2d..0000000 --- a/homelab/.claude/skills/wiki-query/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-query -description: Answer a question from the wiki, scanning frontmatter and the index before opening page bodies. Use for any question answerable from the wiki. ---- - -Read `workflows/query.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/homelab/.claude/skills/wiki-reconcile/SKILL.md b/homelab/.claude/skills/wiki-reconcile/SKILL.md deleted file mode 100644 index 8c4e35d..0000000 --- a/homelab/.claude/skills/wiki-reconcile/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-reconcile -description: Resolve a contested page: weigh the disagreeing sources and rewrite latest-evidence-first. Use when the human says "reconcile ". ---- - -Read `workflows/reconcile.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/homelab/.claude/skills/wiki-triage/SKILL.md b/homelab/.claude/skills/wiki-triage/SKILL.md deleted file mode 100644 index 7adbd9c..0000000 --- a/homelab/.claude/skills/wiki-triage/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-triage -description: Triage the raw inbox of this homelab wiki. Use when the human says "triage the inbox" or drops files into raw/inbox/ and asks to sort them. ---- - -Read `workflows/triage.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/homelab/.claude/skills/wiki-verify/SKILL.md b/homelab/.claude/skills/wiki-verify/SKILL.md deleted file mode 100644 index f2a4c7b..0000000 --- a/homelab/.claude/skills/wiki-verify/SKILL.md +++ /dev/null @@ -1,6 +0,0 @@ ---- -name: wiki-verify -description: Reality-check wiki pages against the actual infrastructure. Use when the human says "verify components" or "verify ". ---- - -Read `workflows/verify.md` from the wiki root and follow it exactly, step by step. Do not run the procedure from memory; the workflow file is the source of truth and may have changed since you last read it. `python3 lint.py check` verifies this wrapper stays paired with its workflow. diff --git a/homelab/.githooks/pre-commit b/homelab/.githooks/pre-commit deleted file mode 100755 index 1bae369..0000000 --- a/homelab/.githooks/pre-commit +++ /dev/null @@ -1,18 +0,0 @@ -#!/bin/sh -# Wiki pre-commit gate: lint errors are uncommittable. -# Lints the STAGED snapshot (not the working tree), so the gate evaluates -# exactly the bytes that will land in the commit. -# Install once per clone: git config core.hooksPath .githooks -root="$(git rev-parse --show-toplevel)" || exit 1 -tmp="$(mktemp -d)" || exit 1 -trap 'rm -rf "$tmp"' 0 -git -C "$root" checkout-index --all --prefix="$tmp/" || exit 1 -cd "$tmp" || exit 1 -python3 lint.py check -status=$? -if [ $status -ne 0 ]; then - echo "" - echo "pre-commit: lint errors in the staged snapshot block the commit." - echo "Fix the files, git add them, and retry; do not bypass with --no-verify." -fi -exit $status diff --git a/homelab/CLAUDE.md b/homelab/CLAUDE.md deleted file mode 100644 index 04f2be8..0000000 --- a/homelab/CLAUDE.md +++ /dev/null @@ -1,134 +0,0 @@ ---- -type: tooling ---- - -# CLAUDE.md - -You are the maintainer of this homelab wiki. The human owns the infrastructure and the decisions. You document, cross-reference, lint, and keep the picture current. This file is the always-loaded core of your operating manual. The step-by-step workflows live in `workflows/` and are read on demand; never run one from memory. - -## Prime directives - -1. **The human owns `raw/` and the real infrastructure. You own everything else.** Never modify files under `raw/` except during an approved triage pass, which only moves and renames files out of `raw/inbox/`. Treat sources as immutable. -2. **The wiki must match reality.** If the human tells you a component changed, every page that references it gets updated in the same session. Stale infra docs are worse than no docs. -3. **Every component page must appear in at least one topology page.** Components without topology context are orphans. Topology pages without real components are fiction. -4. **Cite everything non-obvious.** Configuration choices, IP assignments, firmware versions, decisions: all need a source, even if the source is "the human told me on 2026-04-10". File those conversations as decision pages under `sources/decisions/`. -5. **No secrets in the wiki, ever.** No passwords, API keys, private keys, WireGuard private keys, recovery codes, or BIOS passwords. If the human pastes one, refuse to file it and tell them where it should actually go (password manager, secrets vault). -6. **Log and commit every operation.** Append an entry to `log.md` (append-only, never edit past entries), then make a git commit named `: `, e.g. `document: grafana VM`. Never amend or rewrite history. Commits are your undo and your audit trail. -7. **Ask before deleting.** Renames, merges, deletions, and any change to `status: active` components require human confirmation. -8. **Scan before you read.** Frontmatter (`type`, `tags`, `description`) exists so pages can be judged without opening them. Check `index.md` and frontmatter first; open bodies only for pages that look relevant. - -## Directory layout - -``` -wiki/ -├── CLAUDE.md # this file, the always-loaded core -├── workflows/ # step-by-step procedures, read on demand -├── lint.py # per-variant lint config; logic lives in wikilint/ -├── wikilint/ # shared lint engine (stdlib python) -├── taxonomy.md # allowed tags + page-type glossary, lint-enforced -├── .githooks/ # pre-commit gate running lint.py -├── index.md # content catalog, generated from frontmatter -├── log.md # chronological history, append-only -├── raw/ # immutable sources, human-owned -│ ├── inbox/ # unsorted drop zone -│ ├── configs/ # config snapshots, exports, dumps -│ ├── vendor-docs/ # manuals, datasheets, vendor PDFs -│ ├── screenshots/ # UI screenshots of routers, dashboards, etc. -│ ├── decisions/ # raw transcripts of decision conversations -│ └── assets/ # images and binaries referenced from wiki pages -├── components/ # one page per host, service, device, subnet, peer -├── topology/ # diagrams and connection maps -├── concepts/ # protocols, patterns, techniques (zero-trust, NAT traversal) -├── runbooks/ # how to do things: bootstrap, recover, upgrade, debug -├── incidents/ # when things broke, what happened, how it was fixed -├── synthesis/ # cross-cutting analyses, comparisons, evaluations -├── sources/ # citable source pages you write -│ └── decisions/ # decision records: what was decided and why -└── queries/ # filed answers worth keeping -``` - -`raw/decisions/` holds raw transcripts; `sources/decisions/` holds the citable decision pages you write from them. If a page doesn't fit, ask the human before inventing a new top-level folder. - -## Page types and frontmatter - -Every wiki page (everything outside `raw/`) starts with YAML frontmatter: - -```yaml ---- -type: component | topology | concept | runbook | incident | synthesis | source | query -created: 2026-04-10 -updated: 2026-04-10 -description: One line saying what this page covers, used for scanning. -tags: [networking, vlan] -sources: - - resource: /sources/decisions/2026-04-10-vlan-split.md -confidence: low | contested # optional; absent means normal ---- -``` - -**Component pages additionally need:** -```yaml -component_kind: host | vm | container | service | device | subnet | peer | volume -status: active | planned | deprecated | retired -depends_on: [components/proxmox-01.md, components/vlan-10-mgmt.md] -host: proxmox-01 # for VMs and containers -ip: 10.0.10.42 # if it has one -interface: eth0 # if relevant -firmware: 7.2.1 # for devices -last_verified: 2026-04-10 # last time the human or you confirmed this matches reality -``` - -**Topology pages additionally need:** -```yaml -scope: physical | l2 | l3 | service | trust | data-flow -includes: [components/proxmox-01.md, components/opnsense.md] -``` - -Rules: -- `sources` entries are mappings whose `resource` is the bundle-absolute page path (OKF v0.2 shape); external URLs are allowed for material with no source page. -- `last_verified` on components is the most important field in the schema. Older than 60 days gets flagged by lint. -- `depends_on` is the only stored dependency edge. Reverse edges (what consumes a component) are derived, never stored: `python3 lint.py reverse-deps` prints the map. -- `confidence` is absent on normal pages. `low` means claims lack citations. `contested` means sources or the human disagree, and the page body must explain the disagreement. Contested is a state to exit, not a resting place: reconcile it. -- Every tag must appear in `taxonomy.md`; introducing a tag means adding it there, with a one-line meaning, in the same commit. The allowed page types are described there too, under '## Page types'. -- Mark claims you inferred rather than confirmed with `(inferred)` inline; a page containing any carries `confidence: low`. -- File names are kebab-case: `proxmox-01.md`, `vlan-10-mgmt.md`. Use markdown links with bundle-absolute targets for all internal references: `[proxmox-01](/components/proxmox-01.md)`. - -## Diagrams - -Topology pages must contain at least one Mermaid diagram; a topology page without one is just a list, and lists drift faster. Default to `flowchart`/`graph` for network topology, `sequenceDiagram` for request flows, `C4Context` for service architecture. When a component in a diagram changes name or status, update the diagram in the same pass. Larger diagrams may be rendered as SVG into `raw/assets/topology/` and embedded, but the Mermaid source still lives in the page. Never embed an image without keeping the source. - -## Workflows - -When the human triggers an operation, read the matching file and follow it exactly: - -| Trigger | Read | -|---|---| -| "triage the inbox" | `workflows/triage.md` | -| "document " | `workflows/document.md` | -| "verify components" / "verify " | `workflows/verify.md` | -| "log incident: " | `workflows/incident.md` | -| a question answerable from the wiki | `workflows/query.md` | -| "lint the wiki" | `workflows/lint.md` | -| "reconcile " | `workflows/reconcile.md` | -| "maintenance pass" / "review maintenance" | `workflows/maintain.md` | - -In Claude Code each workflow is also a skill wrapper under `.claude/skills/`, so `/wiki-document`, `/wiki-triage`, etc. dispatch the same files deterministically (the wiki- prefix avoids collisions with globally installed skills); lint errors when a wrapper and its workflow drift apart. - -## Deterministic checks - -`python3 lint.py check` handles every mechanical health check: frontmatter validity, broken links, orphans (with unlinked-mention hints), dangling references, topology membership of active components (prime directive 3), tag taxonomy, stale contested pages, inbox health, secrets, `last_verified` staleness, Mermaid presence on topology pages, index drift, log format, OKF conformance. Run it instead of checking these by hand, and fix errors it reports before finishing any operation. A pre-commit hook (installed via `git config core.hooksPath .githooks`) lints the staged snapshot and makes errors uncommittable; never bypass it with `--no-verify`. - -`python3 lint.py rebuild-index` regenerates `index.md` from page frontmatter. The index is a derived artifact: never hand-edit anything below its generated marker, and rebuild it at the end of any operation that creates, renames, or deletes pages (or bumps a component's `status`/`last_verified`). - -## Style rules for wiki prose - -- Plain, direct English. No marketing voice, no hedging filler. -- Short paragraphs. Bullets where they help, prose where they don't. -- Component pages lead with what it is and why it exists in two sentences. Then specs, then dependencies, then notes. -- Quote sparingly: one quote per source maximum, under 15 words. -- When uncertain, say so in the page itself, not just in chat. -- No em dashes. Use commas, parentheses, or sentence breaks instead. - -## Ambiguity and evolution - -When something is ambiguous, ask: flag it, propose two or three options, wait for a decision. This file and the workflow files are co-owned. When the human decides a new convention, page type, workflow step, or lint rule, update the relevant file in the same session, and update `lint.py` if the rule is mechanical. The wiki's quality is bounded by how good these files are. diff --git a/homelab/taxonomy.md b/homelab/taxonomy.md deleted file mode 100644 index f3d2946..0000000 --- a/homelab/taxonomy.md +++ /dev/null @@ -1,27 +0,0 @@ ---- -type: tooling ---- - -# Taxonomy - -Every `tags:` value on every page must appear in the tag list below. Introducing a new tag means adding it here in the same commit, with a one-line meaning. Keep tags few and general; if two tags mean nearly the same thing, merge them. `lint.py check` warns on tags missing from this file and reports listed tags no page uses. - -- networking — switches, routing, VLANs, DNS, firewalls -- compute — hypervisors, VMs, containers, bare metal -- storage — disks, pools, NAS, volumes -- backup — backup jobs, replication, recovery -- monitoring — metrics, logs, alerting, dashboards -- security — access control, certificates, VPN, hardening - -## Page types - -Every page `type:` the schema allows is described here with a one-line meaning, so any OKF consumer can learn this bundle's vocabulary without reading the lint config. The authoritative schema lives in `lint.py`; `lint.py check` warns when the two drift. - -- component — one host, VM, container, service, device, subnet, peer, or volume -- topology — a connection map of part of the network, with a Mermaid diagram -- concept — a protocol, pattern, or technique -- runbook — how to do something: bootstrap, recover, upgrade, debug -- incident — something that broke, what happened, and how it was fixed -- synthesis — a cross-cutting analysis, comparison, or evaluation -- source — a citable source page, including decision records -- query — a filed answer worth keeping diff --git a/homelab/wikilint/__init__.py b/homelab/wikilint/__init__.py deleted file mode 100644 index f185e69..0000000 --- a/homelab/wikilint/__init__.py +++ /dev/null @@ -1,7 +0,0 @@ -"""Shared wiki lint engine. This package is byte-identical across all -Scriptorium variants; per-variant behavior lives entirely in the CONFIG -dict and index_entry_extra() defined in the wiki root's lint.py.""" - -from .cli import main - -__all__ = ["main"] diff --git a/homelab/wikilint/checks.py b/homelab/wikilint/checks.py deleted file mode 100644 index bc9884d..0000000 --- a/homelab/wikilint/checks.py +++ /dev/null @@ -1,671 +0,0 @@ -"""All mechanical checks except index drift (which lives with the index -builder in derived.py).""" - -import re -from datetime import date, datetime -from urllib.parse import unquote - -from .model import ( - ADR_FILE_RE, FILENAME_EXEMPT, KEBAB_RE, LOG_DATE_HEADER_RE, LOG_ENTRY_RE, - MD_LINK_RE, OKF_VERSION, SECRET_PATTERNS, blank_frontmatter, blank_images, - blank_inline_code, build_link_index, discover_okf_bundle, line_at, - parse_frontmatter, parse_index_pinning, parse_iso_date, resolve_link, -) -from .settings import CONFIG - -# A URI scheme (mailto:, tel:, https:, data:, ...) or protocol-relative //host. -URI_SCHEME_RE = re.compile(r"^(?:[a-zA-Z][a-zA-Z0-9+.\-]*:|//)") - - -def missing_value(value): - """True when a required field is absent in practice: None, "", or [].""" - return value is None or value == "" or value == [] - - -def check_frontmatter(pages, report): - valid_types = set(CONFIG["type_required"]) - for p in pages: - if p.fm_error: - report.error("frontmatter", p.rel, p.fm_error) - continue - ptype = p.type - if ptype not in valid_types: - report.error("frontmatter", p.rel, f"invalid or missing type: {ptype!r}") - continue - for field in CONFIG["required_fields"] + CONFIG["type_required"][ptype]: - if missing_value(p.fields.get(field)): - report.error("frontmatter", p.rel, f"missing or empty required field: {field}") - check_enums(p, ptype, report) - check_dates(p, report) - desc = p.fields.get("description", "") - if isinstance(desc, str) and len(desc) > 200: - report.warning("frontmatter", p.rel, "description over 200 chars; keep it one line") - - -def check_enums(p, ptype, report): - for field, allowed in CONFIG["enum_fields"].items(): - if field in p.fields and p.fields[field] not in allowed: - report.error("frontmatter", p.rel, f"{field}: {p.fields[field]!r} not in {allowed}") - for field, allowed in CONFIG["type_enum_fields"].get(ptype, {}).items(): - value = p.fields.get(field) - if value is not None and value not in allowed: - report.error("frontmatter", p.rel, f"{field}: {value!r} not in {allowed}") - - -def check_dates(p, report): - for field in CONFIG["iso_date_fields"]: - value = p.fields.get(field) - if value and parse_iso_date(value) is None: - report.error("frontmatter", p.rel, f"{field} is not an ISO date") - # The created/updated ordering invariant holds regardless of which fields - # are ISO-validated above. - created = parse_iso_date(p.fields.get("created")) - updated = parse_iso_date(p.fields.get("updated")) - if created and updated and updated < created: - report.error("frontmatter", p.rel, "updated is older than created") - - -def check_filenames(pages, report): - for p in pages: - if p.path.name in FILENAME_EXEMPT: - continue - if not KEBAB_RE.match(p.path.name): - report.warning("filename", p.rel, "not kebab-case") - - -def _file_identity(path): - """(device, inode) identity for a path, or None if it can't be stat'd. - Used to match a link target to a page independent of the spelling of the - path (case on case-insensitive filesystems, symlinks, mount aliases).""" - try: - st = path.stat() - except OSError: - return None - return (st.st_dev, st.st_ino) - - -def check_markdown_links(p, pages_by_identity, report, root): - """Every [text](target) link must resolve on the filesystem (file or - directory): bundle-absolute targets (leading /, OKF's recommended form) - resolve against the wiki root, relative targets against the linking - page's directory. Returns the resolved page targets for inbound counting. - External (scheme/protocol-relative) and pure-anchor targets are skipped.""" - resolved_pages = [] - # resolve() both sides so symlinked roots (e.g. /var -> /private/var) - # compare equal when checking bundle containment. - root_resolved = root.resolve() - # Body content only, images and code blanked, line numbers file-accurate. - scan = blank_images(blank_frontmatter(p.prose)) - for m in MD_LINK_RE.finditer(scan): - raw = m.group(1) - target = raw.split("#")[0] - if not target or URI_SCHEME_RE.match(target): - continue - if target.startswith("/"): - path = (root / unquote(target).lstrip("/")).resolve() - else: - path = (p.path.parent / unquote(target)).resolve() - if not path.exists(): - report.error( - "link", f"{p.rel}:{line_at(scan, m.start())}", - f"broken link ({raw})", - ) - continue - if path != root_resolved and root_resolved not in path.parents: - report.error( - "link", f"{p.rel}:{line_at(scan, m.start())}", - f"link escapes the wiki root ({raw})", - ) - continue - target_page = pages_by_identity.get(_file_identity(path)) - if target_page is not None: - resolved_pages.append(target_page) - return resolved_pages - - -def check_links_and_orphans(pages, report, root): - by_stem, by_rel = build_link_index(pages) - pages_by_identity = {} - for p in pages: - identity = _file_identity(p.path) - if identity is not None: - pages_by_identity[identity] = p - inbound = {p.rel: 0 for p in pages} - for p in pages: - for target in check_markdown_links(p, pages_by_identity, report, root): - if target.rel != p.rel: - inbound[target.rel] += 1 - for field in CONFIG["path_fields"] + CONFIG["edge_fields"]: - for value in p.path_list(field): - # External resources (URLs) are provenance, not wiki edges. - if URI_SCHEME_RE.match(value): - continue - target = resolve_link(value, by_stem, by_rel) - if target is None: - report.error( - "reference", p.rel, - f"{field} points to nonexistent page: {value}", - ) - elif target.rel != p.rel: - inbound[target.rel] += 1 - if not CONFIG["orphans"]: - return - orphans = [ - p for p in pages - if inbound[p.rel] == 0 and p.path.name not in FILENAME_EXEMPT - ] - for p in orphans: - report.warning("orphan", p.rel, "no inbound links from any page") - report_unlinked_mentions(p, pages, report) - - -def report_unlinked_mentions(orphan, pages, report): - """Hint generation for the cross-linker: places where an orphan's name - appears in prose without a link.""" - patterns = [ - re.compile(re.escape(orphan.stem.replace("-", " ")), re.IGNORECASE), - re.compile(re.escape(orphan.stem), re.IGNORECASE), - ] - for other in pages: - if other.rel == orphan.rel: - continue - for pattern in patterns: - m = pattern.search(other.prose) - if m: - report.info( - "orphan", orphan.rel, - f"mentioned unlinked in {other.rel}:{line_at(other.prose, m.start())}; " - "candidate link", - ) - break - - -def check_membership(pages, report): - """Active member pages (components/modules) must appear in at least one - container page's (topology/architecture) membership field.""" - rule = CONFIG["membership"] - if not rule: - return - by_stem, by_rel = build_link_index(pages) - contained = set() - for p in pages: - if p.type != rule["container_type"]: - continue - for value in p.field_list(rule["container_field"]): - target = resolve_link(value, by_stem, by_rel) - if target is not None: - contained.add(target.rel) - for p in pages: - if p.type != rule["member_type"] or not p.fields: - continue - if p.fields.get("status") not in rule["active_statuses"]: - continue - if p.rel not in contained: - report.warning( - "membership", p.rel, - f"active {rule['member_type']} not included in any " - f"{rule['container_type']} page", - ) - - -def check_claude_size(root, report): - claude = root / "CLAUDE.md" - if not claude.is_file(): - return - lines = len(claude.read_text(encoding="utf-8", errors="replace").splitlines()) - cap = CONFIG["claude_md_max_lines"] - if lines > cap: - report.warning( - "hot-core", "CLAUDE.md", - f"{lines} lines exceeds the {cap}-line cap; " - "the always-loaded core is regrowing, move detail to workflows/", - ) - - -def load_taxonomy(root): - """Parse the taxonomy file into (tags, types). Tag entries are `- slug` - lines anywhere outside a `## Page types` section; entries inside that - section map type slug -> one-line meaning (empty string when the line has - no meaning after the dash). `types` is None when the section is absent.""" - path = root / CONFIG["taxonomy_file"] - if not path.is_file(): - return None - tags = set() - types = None - in_types = False - for line in path.read_text(encoding="utf-8", errors="replace").splitlines(): - if re.match(r"^##\s+Page types\s*$", line): - in_types = True - if types is None: - types = {} - continue - if re.match(r"^#{1,6}\s", line): - in_types = False - continue - m = re.match(r"^- ([a-z0-9][a-z0-9-]*)(?:\s*—\s*(.*))?", line) - if m: - if in_types: - types[m.group(1)] = (m.group(2) or "").strip() - else: - tags.add(m.group(1)) - return tags, types - - -def check_tags(pages, report, root): - if CONFIG["taxonomy_file"] is None: - return - taxonomy = load_taxonomy(root) - if taxonomy is None: - report.warning( - "tags", CONFIG["taxonomy_file"], - "missing; create it and list the allowed tags", - ) - return - taxonomy, _types = taxonomy - used = set() - for p in pages: - for tag in p.field_list("tags"): - used.add(tag) - if tag not in taxonomy: - report.warning( - "tags", p.rel, - f"tag {tag!r} not in {CONFIG['taxonomy_file']}; " - "normalize it or add it there in the same commit", - ) - for unused in sorted(taxonomy - used): - report.info("tags", CONFIG["taxonomy_file"], f"tag {unused!r} is used by no page") - - -def check_types(pages, report, root): - """The type glossary: taxonomy.md's `## Page types` section must describe - every schema type with a one-line meaning, so the bundle self-describes - its type vocabulary to foreign OKF consumers.""" - if CONFIG["taxonomy_file"] is None or not CONFIG["types_glossary"]: - return - taxonomy = load_taxonomy(root) - if taxonomy is None: - return # check_tags already reports the missing file - _tags, types = taxonomy - tfile = CONFIG["taxonomy_file"] - if types is None: - report.warning( - "types", tfile, - "no '## Page types' section; describe each page type there " - "with a one-line meaning", - ) - return - schema_types = set(CONFIG["type_required"]) - for t in sorted(schema_types - set(types)): - report.warning( - "types", tfile, - f"page type {t!r} is not described; add '- {t} — ' " - "under '## Page types'", - ) - for t in sorted(schema_types & set(types)): - if not types[t]: - report.warning( - "types", tfile, - f"page type {t!r} has no meaning; add one after the dash", - ) - for t in sorted(set(types) - schema_types): - report.warning( - "types", tfile, - f"'## Page types' describes unknown type {t!r}; remove it or " - "add it to the schema", - ) - - -def check_skills(pages, report, root): - """Harness skill wrappers must pair 1:1 with workflows: every - workflows/.md needs //SKILL.md whose body points - back at the workflow file, and no wrapper may point at nothing. Errors, - not warnings: the wrappers are dispatch surface, and silent drift there - means a slash command runs a stale or missing procedure.""" - if CONFIG["skills_dir"] is None: - return - skills_root = root / CONFIG["skills_dir"] - workflows_root = root / "workflows" - wf_stems = ( - {p.stem for p in workflows_root.glob("*.md")} - if workflows_root.is_dir() else set() - ) - prefix = CONFIG["skills_prefix"] - skill_names = ( - {d.name for d in skills_root.iterdir() if d.is_dir()} - if skills_root.is_dir() else set() - ) - expected = {prefix + stem: stem for stem in wf_stems} - sdir = CONFIG["skills_dir"] - for name in sorted(set(expected) - skill_names): - report.error( - "skills", f"workflows/{expected[name]}.md", - f"no skill wrapper; add {sdir}/{name}/SKILL.md pointing at it", - ) - for name in sorted(skill_names - set(expected)): - report.error( - "skills", f"{sdir}/{name}/SKILL.md", - f"orphan wrapper; no workflow pairs with it " - f"(expected form: {prefix})", - ) - for name in sorted(set(expected) & skill_names): - stem = expected[name] - skill_md = skills_root / name / "SKILL.md" - rel = f"{sdir}/{name}/SKILL.md" - if not skill_md.is_file(): - report.error("skills", rel, "missing SKILL.md in wrapper directory") - continue - body = skill_md.read_text(encoding="utf-8", errors="replace") - if f"workflows/{stem}.md" not in body: - report.error( - "skills", rel, - f"wrapper does not reference workflows/{stem}.md; " - "the body must point at its workflow file", - ) - - -def check_contested_age(pages, report): - today = date.today() - for p in pages: - if (p.fields or {}).get("confidence") != "contested": - continue - updated = parse_iso_date(p.fields.get("updated")) - if updated is None: - continue - age = (today - updated).days - if age > CONFIG["contested_max_days"]: - report.warning( - "contested", p.rel, - f"contested and untouched for {age} days " - f"(limit {CONFIG['contested_max_days']}); run reconcile", - ) - - -def check_inferred_markers(pages, report): - for p in pages: - if "(inferred)" not in p.body_prose: - continue - if (p.fields or {}).get("confidence") not in ("low", "contested"): - report.warning( - "provenance", p.rel, - "contains '(inferred)' claims but confidence is not low", - ) - - -def check_inbox(root, report): - if CONFIG["inbox_dir"] is None: - return - inbox = root / CONFIG["inbox_dir"] - if not inbox.is_dir(): - return - items = [e for e in inbox.iterdir() if not e.name.startswith(".")] - if len(items) >= CONFIG["inbox_warn_count"]: - report.warning("inbox", CONFIG["inbox_dir"], f"{len(items)} items awaiting triage") - now = datetime.now() - for item in items: - age = (now - datetime.fromtimestamp(item.stat().st_mtime)).days - if age >= CONFIG["inbox_warn_age_days"]: - report.warning("inbox", item.name, f"in inbox for {age} days") - - -def secret_patterns(): - """Built-in secret patterns plus the variant's extra_secret_patterns, - compiled once at configure() time.""" - return SECRET_PATTERNS + CONFIG["secret_extra_compiled"] - - -def scan_text_for_secrets(text, rel, report): - """Scan one file's text; matches whose line matches any allow regex are - suppressed (template placeholders, redaction markers). Allow regexes are - precompiled; the line split is deferred until a suppression check needs it.""" - allow = CONFIG["secret_allow_compiled"] - lines = None - for pattern, label in secret_patterns(): - for m in pattern.finditer(text): - line_no = line_at(text, m.start()) - if allow: - if lines is None: - lines = text.split("\n") - if any(a.search(lines[line_no - 1]) for a in allow): - continue - report.error("secrets", f"{rel}:{line_no}", f"possible {label}") - - -def check_secrets(pages, report): - for p in pages: - scan_text_for_secrets(p.text, p.rel, report) - - -def check_field_staleness(pages, report): - today = date.today() - for rule in CONFIG["staleness"]: - for p in pages: - if p.type not in rule["types"] or not p.fields: - continue - value = p.fields.get(rule["field"]) - if not value: - continue - d = parse_iso_date(value) - if d is None: - report.error("staleness", p.rel, f"{rule['field']} is not an ISO date") - continue - age = (today - d).days - if age > rule["max_days"]: - report.add( - rule["severity"].upper(), "staleness", p.rel, - f"{rule['field']} is {age} days old (limit {rule['max_days']})", - ) - - -def check_sync_drift(pages, report, root): - if not CONFIG["sync_drift"]: - return - field, types = CONFIG["sync_drift"] - synced = parse_index_pinning(root).get("last_synced_commit") - if not synced: - return - crit_field = CONFIG["criticality_field"] - plain_counts = {} - for p in pages: - if p.type not in types or not p.fields: - continue - verified = p.fields.get(field) - if not verified or verified == synced: - continue - crit = p.fields.get(crit_field) if crit_field else None - if crit == "load-bearing" or not crit_field: - label = "load-bearing page" if crit_field else "page" - report.warning( - "sync-drift", p.rel, - f"{label} verified at {verified}, repo at {synced}", - ) - else: - plain_counts[crit or "unspecified"] = ( - plain_counts.get(crit or "unspecified", 0) + 1 - ) - for crit, count in sorted(plain_counts.items()): - report.info("sync-drift", crit, f"{count} stale pages (run verify)") - - -def check_owner_review(pages, report): - if not CONFIG["owner_review_max_days"]: - return - today = date.today() - for p in pages: - if p.type != "subsystem" or not p.fields: - continue - reviewed = parse_iso_date(p.fields.get("last_owner_review")) - if reviewed is None: - report.warning("owner-review", p.rel, "missing or invalid last_owner_review") - elif (today - reviewed).days > CONFIG["owner_review_max_days"]: - report.warning( - "owner-review", p.rel, - f"last owner review {(today - reviewed).days} days ago", - ) - - -def count_mermaid_nodes(block): - """Rough node count for a mermaid block: unique identifiers that are - declared with a shape or appear on either side of an edge.""" - ids = set() - for m in re.finditer(r"([A-Za-z][\w-]*)\s*(?:\[|\(|\{)", block): - ids.add(m.group(1)) - for m in re.finditer( - r"([A-Za-z][\w-]*)\s*(?:-->|---|-\.->|==>|--o|--x)\s*([A-Za-z][\w-]*)", block - ): - ids.add(m.group(1)) - ids.add(m.group(2)) - keywords = {"flowchart", "graph", "sequenceDiagram", "subgraph", "end", - "classDef", "class", "style", "direction", "erDiagram", "C4Context"} - return len(ids - keywords) - - -def check_mermaid(pages, report): - if not CONFIG["mermaid_required_types"]: - return - for p in pages: - if p.type not in CONFIG["mermaid_required_types"]: - continue - blocks = re.findall(r"```mermaid\n(.*?)```", p.body, re.DOTALL) - if not blocks: - report.error("mermaid", p.rel, "page type requires at least one mermaid diagram") - continue - for block in blocks: - nodes = count_mermaid_nodes(block) - if CONFIG["mermaid_node_error"] and nodes > CONFIG["mermaid_node_error"]: - report.error("mermaid", p.rel, f"diagram has ~{nodes} nodes; split it") - elif CONFIG["mermaid_node_warn"] and nodes > CONFIG["mermaid_node_warn"]: - report.warning("mermaid", p.rel, f"diagram has ~{nodes} nodes; consider splitting") - - -def check_adrs(pages, report, root): - for adr_dir in CONFIG["adr_dirs"]: - for dir_path in sorted(root.glob(adr_dir)): - if not dir_path.is_dir(): - continue - check_adr_dir(dir_path, pages, report, root) - - -def check_adr_dir(dir_path, pages, report, root): - numbers = {} - for p in pages: - if p.path.parent != dir_path: - continue - m = ADR_FILE_RE.match(p.path.name) - if not m: - report.warning("adr", p.rel, "not in NNNN-slug.md form") - continue - numbers[int(m.group(1))] = p - status = (p.fields or {}).get("status") - if status == "superseded" and not (p.fields or {}).get("superseded_by"): - report.error("adr", p.rel, "superseded ADR has no superseded_by forward link") - if numbers: - expected = set(range(min(numbers), max(numbers) + 1)) - for gap in sorted(expected - set(numbers)): - rel = dir_path.relative_to(root).as_posix() - report.warning("adr", rel, f"numbering gap: {gap:04d} missing") - - -def check_log(root, report): - """OKF log structure: '## YYYY-MM-DD' date headings, newest first, with - entries as '- **Action**: ...' bullets. This check owns log.md end to - end; check_okf deliberately does not re-validate it.""" - log_file = CONFIG["log_file"] - if log_file is None: - return - log = root / log_file - if not log.is_file(): - report.warning("log", log_file, "missing") - return - text = log.read_text(encoding="utf-8", errors="replace") - prev = None # (date, line_no) of the last valid heading seen - for i, line in enumerate(text.splitlines(), 1): - if line.startswith("## "): - m = LOG_DATE_HEADER_RE.match(line) - if not m: - report.warning( - "log", f"{log_file}:{i}", - f"heading not in '## YYYY-MM-DD' form: {line!r}", - ) - continue - d = parse_iso_date(m.group(1)) - if d is None: - report.warning("log", f"{log_file}:{i}", f"heading is not a valid date: {line!r}") - continue - if prev is not None and d > prev[0]: - report.warning( - "log", f"{log_file}:{i}", - f"headings out of order (want newest-first): {d} follows {prev[0]}", - ) - elif prev is not None and d == prev[0]: - report.warning( - "log", f"{log_file}:{i}", - f"duplicate heading for {d}; append entries under the existing one", - ) - prev = (d, i) - elif line.startswith("- ") and not LOG_ENTRY_RE.match(line): - report.warning( - "log", f"{log_file}:{i}", - "entry should open with a bold action word: '- **Update**: ...'", - ) - - -def check_sources(pages, report): - """OKF v0.2 sources shape (SPEC §5.1): each entry is a mapping whose - required resource names either an external URL or a bundle-absolute page - path. Plain-string entries (the pre-0.2 shape) still resolve as paths in - check_links_and_orphans, but warn until installs migrate.""" - if not CONFIG["okf_conformance"]: - return - for p in pages: - for i, entry in enumerate(p.field_list("sources"), 1): - if isinstance(entry, dict): - resource = entry.get("resource", "") - if not resource: - report.error( - "sources", p.rel, - f"sources entry {i} has no resource " - "(required per entry, OKF v0.2 §5.1)", - ) - elif not URI_SCHEME_RE.match(resource) and not resource.startswith("/"): - report.warning( - "sources", p.rel, - f"sources entry {i}: bundle paths should be " - f"bundle-absolute (/dir/page.md), got {resource!r}", - ) - else: - report.warning( - "sources", p.rel, - f"sources entry {i} is a plain string; the OKF v0.2 shape " - "is '- resource: /dir/page.md'", - ) - - -def check_okf(pages, report, root): - """OKF v0.2 conformance: (1) every non-reserved .md parses as frontmatter, - (2) with a non-empty type, (3) reserved files follow their structure. - Pages are skipped here: check_frontmatter is strictly stricter. log.md's - structure is owned by check_log, not re-validated here. raw/ is excluded - by design (immutable human-owned sources; see discover_okf_bundle).""" - if not CONFIG["okf_conformance"]: - return - page_rels = {p.rel for p in pages} - for path in discover_okf_bundle(root): - rel = path.relative_to(root).as_posix() - if rel in page_rels: - continue - fields, err = parse_frontmatter( - path.read_text(encoding="utf-8", errors="replace")) - if err: - report.error("okf", rel, f"OKF rule 1: {err}") - elif missing_value(fields.get("type")): - report.error("okf", rel, "OKF rule 2: missing or empty type") - index_file = CONFIG["index_file"] - if index_file and (root / index_file).is_file(): - fields, _ = parse_frontmatter( - (root / index_file).read_text(encoding="utf-8", errors="replace")) - if not fields or fields.get("okf_version") != OKF_VERSION: - report.error( - "okf", index_file, - f'OKF rule 3: missing okf_version: "{OKF_VERSION}" frontmatter; ' - "run `python3 lint.py rebuild-index`", - ) diff --git a/homelab/wikilint/cli.py b/homelab/wikilint/cli.py deleted file mode 100644 index 84383ee..0000000 --- a/homelab/wikilint/cli.py +++ /dev/null @@ -1,90 +0,0 @@ -"""CLI entry point: subcommand dispatch and the check orchestrator.""" - -import sys -from pathlib import Path - -from . import checks -from .derived import check_index_drift, rebuild_index, run_coverage, run_reverse_deps -from .model import Report, discover_pages -from .settings import CONFIG, ConfigError, configure - -USAGE = """\ -Usage: - python3 lint.py check run all mechanical checks, exit 1 on errors - python3 lint.py rebuild-index regenerate the index from page frontmatter - python3 lint.py reverse-deps print the derived reverse-dependency maps - python3 lint.py coverage write coverage.md (variants with coverage enabled) -""" - - -def gather_report(root): - """Run every mechanical check and return the Report (no printing). - This is the testable core of `lint.py check`.""" - report = Report() - pages = discover_pages(root, report) - checks.check_frontmatter(pages, report) - checks.check_filenames(pages, report) - checks.check_links_and_orphans(pages, report, root) - checks.check_membership(pages, report) - checks.check_claude_size(root, report) - checks.check_tags(pages, report, root) - checks.check_types(pages, report, root) - checks.check_skills(pages, report, root) - checks.check_contested_age(pages, report) - checks.check_inferred_markers(pages, report) - checks.check_inbox(root, report) - checks.check_secrets(pages, report) - checks.check_field_staleness(pages, report) - checks.check_sync_drift(pages, report, root) - checks.check_owner_review(pages, report) - checks.check_mermaid(pages, report) - checks.check_adrs(pages, report, root) - checks.check_log(root, report) - checks.check_sources(pages, report) - checks.check_okf(pages, report, root) - check_index_drift(pages, report, root) - for extra in CONFIG["extra_checks"]: - # A third-party check must not abort the run: a raise here would - # discard every finding already gathered. Convert it to an error. - try: - extra(pages, report, root) - except Exception as e: - report.error( - "extra-check", getattr(extra, "__name__", "extra_check"), - f"raised {type(e).__name__}: {e}", - ) - return len(pages), report - - -def run_check(root): - count, report = gather_report(root) - print(f"checked {count} pages") - print(report.render()) - return 1 if report.has_errors else 0 - - -def main(config, index_entry_extra, argv=None, root=None): - try: - configure(config, index_entry_extra) - except ConfigError as e: - print(f"lint config error: {e}", file=sys.stderr) - return 2 - root = Path(root) if root else Path.cwd() - if not (root / "CLAUDE.md").is_file(): - print("run from the wiki root (CLAUDE.md not found here)", file=sys.stderr) - return 2 - args = sys.argv[1:] if argv is None else argv - command = args[0] if args else "check" - if command == "check": - return run_check(root) - if command == "rebuild-index": - return rebuild_index(root) - if command == "reverse-deps": - return run_reverse_deps(root) - if command == "coverage": - if not CONFIG["coverage"]: - print("coverage is not enabled for this variant") - return 2 - return run_coverage(root) - print(USAGE) - return 2 diff --git a/homelab/wikilint/derived.py b/homelab/wikilint/derived.py deleted file mode 100644 index 00fc5c0..0000000 --- a/homelab/wikilint/derived.py +++ /dev/null @@ -1,249 +0,0 @@ -"""Derived artifacts: the generated index, reverse-dependency maps, and the -coverage report. check_index_drift lives here beside the builder it reuses.""" - -from pathlib import Path - -from .model import ( - GENERATED_MARKER, OKF_VERSION, YAML_FENCE_RE, build_link_index, - discover_pages, parse_index_pinning, resolve_link, Report, -) -from .settings import CONFIG - - -def derive_reverse_edges(pages): - """field -> {target rel: [source rels]} for every relationship field in - CONFIG['reverse_fields']. This is the derived view that replaces the old - stored mirror fields (consumed_by, public_surface, api consumers).""" - by_stem, by_rel = build_link_index(pages) - reverse = {} - for field in CONFIG["reverse_fields"]: - rev = {} - for p in pages: - for value in p.field_list(field): - target = resolve_link(value, by_stem, by_rel) - if target is not None: - rev.setdefault(target.rel, []).append(p.rel) - reverse[field] = rev - return reverse - - -def run_reverse_deps(root): - report = Report() - pages = discover_pages(root, report) - if not CONFIG["reverse_fields"]: - print("no reverse fields configured for this variant") - return 0 - for field, rev in derive_reverse_edges(pages).items(): - print(f"\nreverse of {field}:") - if not rev: - print(" (no edges)") - for target, sources in sorted(rev.items()): - print(f" {target} <- {', '.join(sorted(sources))}") - return 0 - - -def index_path(root): - """Absolute path of the generated index, or None when the index is - disabled (index_file is None).""" - index_file = CONFIG["index_file"] - return root / index_file if index_file is not None else None - - -def prettify_title(stem): - """Display title for a page without an authored `title` field: - 'zero-trust-networking' -> 'Zero Trust Networking'.""" - return " ".join(w.capitalize() for w in stem.split("-")) - - -def _index_entry_link(p): - """OKF index entry link: bundle-absolute markdown link with the page's - authored title, falling back to a prettified stem.""" - title = (p.fields or {}).get("title") or prettify_title(p.stem) - return f"[{title}](/{p.rel})" - - -def generate_index_body(pages): - body_fn = CONFIG["index_body_fn"] - if body_fn is not None: - return body_fn(pages) - lines = [] - if CONFIG["index_mode"] == "subsystems": - lines.extend(_subsystem_section(pages)) - pool = [p for p in pages - if p.fields and p.fields.get("subsystem") == "global"] - else: - pool = pages - entry_extra = CONFIG["index_entry_extra"] - for section, types in CONFIG["index_sections"]: - members = sorted((p for p in pool if p.type in types), key=lambda p: p.rel) - if not members: - continue - lines.append(f"## {section}") - lines.append("") - for p in members: - desc = p.fields.get("description", "") if p.fields else "" - lines.append( - f"* {_index_entry_link(p)} - {desc} {entry_extra(p.fields or {})}".rstrip()) - lines.append("") - return "\n".join(lines).rstrip() + "\n" - - -def _subsystem_section(pages): - lines = ["## Subsystems", ""] - for p in sorted((p for p in pages if p.type == "subsystem"), key=lambda p: p.rel): - desc = p.fields.get("mission") or p.fields.get("description", "") - review = p.fields.get("last_owner_review", "never") - lines.append(f"* {_index_entry_link(p)} - {desc} (last review {review})") - lines.append("") - return lines - - -def _strip_leading_frontmatter(text): - """Drop a leading YAML frontmatter block (the okf_version stamp) so a - re-run of rebuild-index never accumulates duplicate blocks. Mirrors - Page._body_text but stays local: index.md is not a Page.""" - if not text.startswith("---"): - return text - lines = text.split("\n") - for i in range(1, len(lines)): - if lines[i].strip() == "---": - return "\n".join(lines[i + 1:]).lstrip("\n") - return text - - -def rebuild_index(root): - report = Report() - pages = discover_pages(root, report) - index = index_path(root) - if index is None: - print("index is disabled (index_file is None); nothing to rebuild") - return 0 - index_name = index.relative_to(root).as_posix() - index.parent.mkdir(parents=True, exist_ok=True) - # The okf_version stamp is a single fixed key: always regenerate it so a - # deleted or corrupted block self-heals on the next rebuild. - front = f'---\nokf_version: "{OKF_VERSION}"\n---\n\n' if CONFIG["okf_conformance"] else "" - head = "# Index\n\n" - if index.is_file(): - existing = _strip_leading_frontmatter( - index.read_text(encoding="utf-8", errors="replace")) - if GENERATED_MARKER in existing: - head = existing.split(GENERATED_MARKER)[0] - else: - m = YAML_FENCE_RE.search(existing) - if m and existing[:m.start()].strip() in ("", "# Index"): - head = existing[:m.end()] + "\n\n" - body = generate_index_body(pages) - index.write_text(f"{front}{head}{GENERATED_MARKER}\n\n{body}", encoding="utf-8") - # The built-in body lists entries as "* " lines; a custom index_body_fn can - # use any format, so report page count rather than guess at "entries". - if CONFIG["index_body_fn"] is not None: - print(f"{index_name} rebuilt ({len(pages)} pages)") - else: - entries = sum(1 for line in body.splitlines() if line.startswith("* ")) - print(f"{index_name} rebuilt: {entries} entries") - return 0 - - -def check_index_drift(pages, report, root): - index = index_path(root) - if index is None: - return - index_name = index.relative_to(root).as_posix() - if not index.is_file(): - report.warning("index", index_name, "missing; run `python3 lint.py rebuild-index`") - return - existing = index.read_text(encoding="utf-8", errors="replace") - if GENERATED_MARKER not in existing: - report.warning("index", index_name, "no generated marker; run `python3 lint.py rebuild-index`") - return - # rebuild_index writes exactly "{head}{MARKER}\n\n{body}", so compare the - # post-marker text against the same "\n\n"+body rather than lstrip-ing, - # which would also strip a body that legitimately starts with a newline. - current = existing.split(GENERATED_MARKER, 1)[1] - if current != f"\n\n{generate_index_body(pages)}": - report.warning("index", index_name, "out of date; run `python3 lint.py rebuild-index`") - - -def source_path_covers(source_paths, rel): - """Boundary-anchored: 'src' covers 'src' and 'src/x', never 'src-utils'.""" - return any( - sp and (sp == rel or sp.startswith(rel + "/") or rel.startswith(sp + "/")) - for sp in source_paths - ) - - -def run_coverage(root): - """Write coverage.md: per-subsystem module inventory by criticality, then - per-include-root directory coverage with the uncovered directories named.""" - report = Report() - pages = discover_pages(root, report) - pinning = parse_index_pinning(root) - local = pinning.get("local_path") - if not local: - index_name = CONFIG["index_file"] or "the index" - print(f"coverage: no local_path in {index_name} pinning block; skipping") - return 0 - repo = Path(local).expanduser() - if not repo.is_dir(): - print(f"coverage: {repo} not found; skipping") - return 0 - modules = [p for p in pages if p.type == "module" and p.fields] - # Fully generated, so stamp OKF-conformant frontmatter directly. - lines = ["---", "type: report", "---", "", - "# Coverage", "", f"Generated by lint.py against `{local}`.", ""] - lines.extend(_subsystem_coverage(modules)) - lines.extend(_directory_coverage(repo, pinning, modules)) - (root / "coverage.md").write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8") - print("coverage.md written") - return 0 - - -def _subsystem_coverage(modules): - crit_field = CONFIG["criticality_field"] - by_subsystem = {} - for p in modules: - by_subsystem.setdefault(p.fields.get("subsystem", "unassigned"), []).append(p) - lines = ["## Module pages by subsystem", ""] - for subsystem, members in sorted(by_subsystem.items()): - if crit_field: - counts = {} - for p in members: - crit = p.fields.get(crit_field, "unspecified") - counts[crit] = counts.get(crit, 0) + 1 - breakdown = ", ".join(f"{n} {c}" for c, n in sorted(counts.items())) - lines.append(f"- {subsystem}: {len(members)} module pages ({breakdown})") - else: - lines.append(f"- {subsystem}: {len(members)} module pages") - lines.append("") - return lines - - -def _directory_coverage(repo, pinning, modules): - scope = pinning.get("wiki_scope", {}) - includes = scope.get("include", []) if isinstance(scope, dict) else [] - excludes = scope.get("exclude", []) if isinstance(scope, dict) else [] - source_paths = [p.fields.get("source_path", "").strip("/") for p in modules] - lines = ["## Directory coverage", ""] - for inc in includes: - inc_dir = repo / inc.strip("/") - if not inc_dir.is_dir(): - continue - entries = sorted( - e for e in inc_dir.iterdir() - if e.is_dir() and not any(e.match(x) for x in excludes) - ) - covered = [] - uncovered = [] - for e in entries: - rel = e.relative_to(repo).as_posix() - (covered if source_path_covers(source_paths, rel) else uncovered).append(rel) - total = len(covered) + len(uncovered) - pct = round(100 * len(covered) / total) if total else 100 - lines.append(f"### {inc}") - lines.append("") - lines.append(f"{len(covered)}/{total} first-level directories covered ({pct}%)") - for u in uncovered: - lines.append(f"- uncovered: `{u}`") - lines.append("") - return lines diff --git a/homelab/wikilint/model.py b/homelab/wikilint/model.py deleted file mode 100644 index 4c2cbae..0000000 --- a/homelab/wikilint/model.py +++ /dev/null @@ -1,419 +0,0 @@ -"""Page model, parsers, and shared helpers.""" - -import re -from datetime import date -from fnmatch import fnmatch -from pathlib import Path, PurePosixPath - -from .settings import CONFIG - -GENERATED_MARKER = "" - -# OKF (Open Knowledge Format) version this engine produces and validates. -OKF_VERSION = "0.2" - -# Pages allowed to break kebab-case and to have no inbound links. -FILENAME_EXEMPT = ("README.md", "OWNERS.md") - -SECRET_PATTERNS = [ - (re.compile(r"(?i)\bpassword\s*[:=]\s*\S"), "password assignment"), - (re.compile(r"(?i)\bapi[_-]?key\s*[:=]\s*\S"), "api key assignment"), - (re.compile(r"(?i)\bsecret\s*[:=]\s*\S"), "secret assignment"), - (re.compile(r"(?i)\btoken\s*[:=]\s*[A-Za-z0-9._-]{8,}"), "token assignment"), - (re.compile(r"BEGIN (?:RSA |EC |OPENSSH |PGP )?PRIVATE KEY"), "private key block"), - (re.compile(r"\bAKIA[0-9A-Z]{16}\b"), "AWS access key id"), - (re.compile(r"(?i)wg[- ]?private"), "wireguard private key reference"), - (re.compile(r"Bearer\s+[A-Za-z0-9._~+/-]{20,}"), "bearer token"), -] - -# [text](target) markdown links, excluding images; target is group 1. An -# optional "title" after the target is tolerated. Images are blanked first -# (blank_images) so an image wrapped in a link exposes the link's target. -MD_LINK_RE = re.compile(r"(? [pages] and rel/rel_no_ext -> page for link resolution.""" - by_stem = {} - by_rel = {} - for p in pages: - by_stem.setdefault(p.stem, []).append(p) - by_rel[p.rel_no_ext] = p - by_rel[p.rel] = p - return by_stem, by_rel - - -def resolve_link(target, by_stem, by_rel): - target = target.split("|")[0].split("#")[0].strip() - # Bundle-absolute (leading /) targets resolve against the root, where - # page rel paths have no leading slash. - if target.startswith("/"): - target = target[1:] - if not target: - return None - if target in by_rel: - return by_rel[target] - stem = target[:-3] if target.endswith(".md") else target - candidates = by_stem.get(stem.split("/")[-1], []) - if "/" in target: - matches = [p for p in candidates if p.rel_no_ext.endswith(stem)] - return matches[0] if len(matches) == 1 else None - return candidates[0] if len(candidates) == 1 else None - - -def _pinning_fence(text): - """The yaml fence holding the pinning data: searched only above the - generated marker, and required to carry a pinning key, so example yaml - blocks elsewhere in the file can never displace it.""" - if GENERATED_MARKER in text: - text = text.split(GENERATED_MARKER)[0] - for m in YAML_FENCE_RE.finditer(text): - if re.search(r"^(repo|last_synced_commit):", m.group(1), re.MULTILINE): - return m.group(1) - return None - - -def parse_index_pinning(root): - """Parse the yaml pinning block at the top of the index, if any. - Handles quoted scalars and both inline and block-style nested lists.""" - if CONFIG["index_file"] is None: - return {} - index = root / CONFIG["index_file"] - if not index.is_file(): - return {} - block = _pinning_fence(index.read_text(encoding="utf-8", errors="replace")) - if block is None: - return {} - pinning = {} - current_map = None # top-level key holding a nested mapping - current_list = None # (map key, nested key) of a block list being filled - for line in block.splitlines(): - stripped = line.strip() - if not stripped or stripped.startswith("#"): - continue - if not line.startswith((" ", "\t")): - m = KEY_RE.match(line) - if not m: - continue - key, val = m.group(1), m.group(2).strip() - current_list = None - if val == "": - pinning[key] = {} - current_map = key - else: - pinning[key] = unquote(val) - current_map = None - elif stripped.startswith("- ") and current_list is not None: - map_key, key = current_list - pinning[map_key][key].append(unquote(stripped[2:])) - elif current_map is not None: - m = KEY_RE.match(stripped) - if not m: - continue - key, val = m.group(1), m.group(2).strip() - if val.startswith("[") and val.endswith("]"): - inner = val[1:-1].strip() - pinning[current_map][key] = ( - [unquote(v) for v in inner.split(",") if v.strip()] - if inner else [] - ) - current_list = None - elif val == "": - pinning[current_map][key] = [] - current_list = (current_map, key) - else: - pinning[current_map][key] = unquote(val) - current_list = None - return pinning diff --git a/homelab/wikilint/settings.py b/homelab/wikilint/settings.py deleted file mode 100644 index 52b1143..0000000 --- a/homelab/wikilint/settings.py +++ /dev/null @@ -1,120 +0,0 @@ -"""Mutable configuration holder and the config-validation boundary. - -cli.main() calls configure() once at startup with a variant's CONFIG; every -other module imports the CONFIG dict object and reads it live. Engine -extension points default to original wiki behavior via DEFAULTS, so a variant -only lists a key when it overrides one. configure() validates user-supplied -values (regexes, paths, callables) and fails fast with a clear message. -""" - -import re -from pathlib import PurePosixPath - -CONFIG = {} - -# Engine extension points. Defaults live here, once, and preserve the original -# wiki behavior; a variant's lint.py overrides a key only to change it. Reading -# these bare as CONFIG[key] is safe because configure() merges DEFAULTS under -# every variant config. -DEFAULTS = { - # Enforce OKF v0.2 conformance (check_okf) and stamp okf_version - # frontmatter into the rebuilt index. Off for non-OKF markdown trees. - "okf_conformance": True, - # Report pages with no inbound links (plus unlinked-mention hints). - "orphans": True, - # Path of the generated index relative to the wiki root; None disables the - # index build and drift check entirely. - "index_file": "index.md", - # Callable(pages) -> str replacing the built-in index body generator. - "index_body_fn": None, - # Frontmatter fields validated as ISO dates when present. The - # created/updated ordering check runs regardless of this list. - "iso_date_fields": ["created", "updated"], - # Extra (regex, label) pairs appended to the secrets scan. - "extra_secret_patterns": [], - # Secrets matches on lines matching any of these regexes are suppressed. - "secret_allow_res": [], - # Require taxonomy.md to carry a '## Page types' section describing every - # schema type with a one-line meaning, so the bundle self-describes its - # type vocabulary to OKF consumers. Off for non-wiki markdown trees. - "types_glossary": True, - # Directory of harness skill wrappers (e.g. ".claude/skills"), each a - # /SKILL.md whose body points at workflows/.md. When - # set, check_skills errors on any drift between the two sets. None - # disables. - "skills_dir": None, - # Wrapper-name prefix, namespacing wiki skills away from a user's global - # skills (field finding, 2026-07-22: a bare /triage collided with a - # global triage skill). "wiki-" pairs workflows/triage.md with - # /wiki-triage/SKILL.md. - "skills_prefix": "", - # Append-only operations log checked by check_log; None disables. - "log_file": "log.md", - # Callables(pages, report, root) run at the end of every check pass. - "extra_checks": [], -} - - -class ConfigError(Exception): - """A variant lint.py holds an invalid value for an engine knob.""" - - -def configure(config, index_entry_extra): - merged = {**DEFAULTS, **config} - merged["index_entry_extra"] = index_entry_extra - _validate(merged) - CONFIG.clear() - CONFIG.update(merged) - - -def _compile(pattern, key): - try: - return re.compile(pattern) - except re.error as e: - raise ConfigError(f"{key}: invalid regex {pattern!r}: {e}") - - -def _validate(cfg): - """Validate user-supplied extension values at the config boundary and - precompile the secret regexes so a bad pattern fails here, once, with a - clear message rather than mid-scan with a raw traceback.""" - index_file = cfg["index_file"] - if index_file is not None: - if not isinstance(index_file, str) or not index_file.strip(): - raise ConfigError("index_file must be None or a non-empty relative path") - posix = PurePosixPath(index_file) - if posix.is_absolute() or ".." in posix.parts: - raise ConfigError( - f"index_file must stay within the wiki root: {index_file!r}") - if cfg["log_file"] is not None and not isinstance(cfg["log_file"], str): - raise ConfigError("log_file must be None or a relative path string") - if not isinstance(cfg["types_glossary"], bool): - raise ConfigError("types_glossary must be a bool") - skills_dir = cfg["skills_dir"] - if skills_dir is not None: - if not isinstance(skills_dir, str) or not skills_dir.strip(): - raise ConfigError("skills_dir must be None or a non-empty relative path") - posix = PurePosixPath(skills_dir) - if posix.is_absolute() or ".." in posix.parts: - raise ConfigError( - f"skills_dir must stay within the wiki root: {skills_dir!r}") - if not isinstance(cfg["skills_prefix"], str): - raise ConfigError("skills_prefix must be a string") - if cfg["index_body_fn"] is not None and not callable(cfg["index_body_fn"]): - raise ConfigError("index_body_fn must be None or callable") - for fn in cfg["extra_checks"]: - if not callable(fn): - raise ConfigError(f"extra_checks entries must be callable: {fn!r}") - - compiled_extra = [] - for item in cfg["extra_secret_patterns"]: - try: - pattern, label = item - except (TypeError, ValueError): - raise ConfigError( - f"extra_secret_patterns entries must be (regex, label): {item!r}") - compiled_extra.append((_compile(pattern, "extra_secret_patterns"), label)) - cfg["secret_extra_compiled"] = compiled_extra - cfg["secret_allow_compiled"] = [ - _compile(pattern, "secret_allow_res") for pattern in cfg["secret_allow_res"] - ] diff --git a/homelab/workflows/document.md b/homelab/workflows/document.md deleted file mode 100644 index c6d2db8..0000000 --- a/homelab/workflows/document.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -type: workflow ---- - -# Document - -Triggered by "document the new VM", "document this OPNsense config", or similar. Default to one component at a time with the human in the loop. - -1. **Gather context.** Read the raw source (config dump, screenshot, vendor doc) if there is one. Ask the human anything you need: hostname, IP, purpose, dependencies, who consumes it. -2. **Discuss before writing.** Tell the human what you're going to file and where. Confirm component kind, dependencies, and which topology pages this touches. Wait for their response before touching the wiki. If the pass will create concept or synthesis pages, apply the page-worthiness test below and say so in the plan. -3. **Create or update the component page** under `components/`. Fill all required frontmatter, including a one-line `description`. Set `last_verified` to today. If new information contradicts what a page already claims, do not overwrite: set `confidence: contested` and structure the body latest-evidence-first with dates, so `workflows/reconcile.md` has clean input. List everything it depends on in its own `depends_on`; that is the only stored edge. Do not touch other components' frontmatter to record the reverse direction, it is derived (`python3 lint.py reverse-deps`). -4. **Update affected topology pages.** Add the component to the relevant Mermaid diagrams and to `includes` frontmatter. Every component must appear in at least one topology page. -5. **Create or append a decision record.** If this involved a real decision (why VLAN 20 and not VLAN 30, why Caddy over Traefik), file the reasoning under `sources/decisions/-.md` and cite it from the component page's `sources`. -6. **Rebuild the index and lint.** Run `python3 lint.py rebuild-index`, then `python3 lint.py check` and fix any errors it reports. -7. **Append to `log.md`** using the format below, then commit: `document: `. -8. **Report back.** List every file created or updated. Call out any cross-reference inconsistencies you couldn't resolve. - -## The page-worthiness test - -Component, topology, incident, runbook, and decision pages are inventory: they record what exists, what happened, and how to operate it. Any other new page (concept, synthesis) must pass all four parts: - -1. **Topic shape.** It defines something referenceable by name: a concept, a pattern, a term, a convention, a metric. Not a grab-bag, not "assorted notes". -2. **Not meta.** It is not an overview, introduction, getting-started, quickstart, tutorial, walkthrough, FAQ, release-notes, changelog, or roadmap. If the natural filename would be one of those words, stop. -3. **Citation test.** You can write a sentence in an existing page of the form "See [X](/concepts/x.md) for ..." where X is a concrete noun. If the best sentence you can write is "see this page for context", it fails. -4. **Reuse test.** At least two existing pages would cite it, or one page needs it as load-bearing background that does not fit inline. - -When in doubt, do not create the page; integrate the material into an existing one instead. A wiki with few concept pages is fine; a wiki full of `concepts/overview.md` and `concepts/misc-notes.md` is noise. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Document**: grafana VM -- **Creation**: components/grafana.md -- **Update**: topology/services.md (diagram + includes) -- **Decision**: sources/decisions/2026-04-10-grafana-on-proxmox-01.md -``` diff --git a/homelab/workflows/incident.md b/homelab/workflows/incident.md deleted file mode 100644 index 5d1cb53..0000000 --- a/homelab/workflows/incident.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -type: workflow ---- - -# Incident - -When something broke. Triggered by "log incident: ". - -1. **Create `incidents/-.md`** with frontmatter `type: incident`, a one-line `description`, plus severity, affected components, and duration in the body or frontmatter. -2. **Walk through the timeline with the human:** what was noticed, what was tried, what worked. Write it up as a timeline, not a narrative. -3. **Link every component involved** with markdown links. Add the incident to each affected component page's "Incidents" section and bump those pages' `updated`. -4. **Ask about follow-ups.** Does this suggest a new runbook, a topology change, or a concept page? If yes, create the follow-ups in the same session. -5. **Rebuild the index and lint.** Run `python3 lint.py rebuild-index`, then `python3 lint.py check` and fix any errors it reports. -6. **Append to `log.md`** using the format below, then commit: `incident: `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Incident**: grafana down 45min -- **Filed**: incidents/2026-04-10-grafana-oom.md -- **Update**: components/grafana.md, components/proxmox-01.md -- **Runbook**: runbooks/grafana-oom-recovery.md -``` diff --git a/homelab/workflows/lint.md b/homelab/workflows/lint.md deleted file mode 100644 index d3aaf32..0000000 --- a/homelab/workflows/lint.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -type: workflow ---- - -# Lint - -Triggered by "lint the wiki". Lint has two halves: the script does everything mechanical; you do only the checks that need judgment. - -1. **Run `python3 lint.py check`** and relay its findings to the human, grouped by severity. Do not re-verify by hand what the script already checks (frontmatter, links, orphans, dangling references, topology membership of active components, tags, contested age, inbox health, secrets, `last_verified` staleness, Mermaid presence on topology pages, index drift, log format, OKF conformance). -2. **Orphan repair.** For each orphan the script reports, use its unlinked-mention hints to propose link insertions; insert on approval. If no genuine mention exists anywhere, recommend merging or deleting the page (ask). Never force a link where the prose doesn't naturally mention the page. -3. **Topology fiction.** The script verifies that `includes` paths exist; you check that no topology page includes a component whose `status` is `retired`, and that diagrams don't depict links or devices that no longer exist. -4. **Diagram semantic drift.** Mermaid diagrams whose node names no longer match the current component page titles, or that show a component under an old hostname. -5. **Contested pages.** Any `confidence: contested` page must explain the disagreement in its body; flag ones that don't, and offer to run `workflows/reconcile.md` on pages the script flags as long-contested. -6. **Concept gaps.** Terms that appear repeatedly across pages (a protocol, a pattern) but have no dedicated concept page. -7. **Stale runbooks.** Runbooks not touched in over 6 months whose components are still `status: active`. Procedures rot as fast as firmware. -8. **Description drift.** Spot-check a handful of pages: does the `description` still match the body? A wrong description poisons every future scan. -9. **Investigation suggestions.** Three to five questions the wiki currently can't answer well, and what to document or verify to fix that. - -Report findings as a structured list. Do not auto-fix anything except trivially safe broken links (where the rename target is unambiguous). Wait for human direction on the rest. - -When done: append a lint entry to `log.md` and commit: `lint: findings`. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Lint**: run complete -- **Script**: 2 errors, 4 warnings -- **Judgment**: 1 topology fiction, 2 concept gaps -- **Note**: see lint report in chat -``` diff --git a/homelab/workflows/maintain.md b/homelab/workflows/maintain.md deleted file mode 100644 index 2dc9fc6..0000000 --- a/homelab/workflows/maintain.md +++ /dev/null @@ -1,33 +0,0 @@ ---- -type: workflow ---- - -# Maintain - -The unattended maintenance pass, run on a schedule or by "maintenance pass". Its safety comes from reversibility: all work happens on the `maintenance` branch, never on main, and the human merges only after review. - -Setup: `git checkout maintenance` (create it from main if absent; if it holds unmerged work from a prior run, continue on it). - -Allowed actions, exhaustively: - -1. **Run `python3 lint.py check` and fix the mechanical errors it reports**: broken links with unambiguous targets, frontmatter fields whose correct value is derivable (malformed dates, wrong enum casing), dangling references to renamed pages. -2. **Run `python3 lint.py rebuild-index`.** -3. **Insert cross-links for orphans**, only where the script's mention hints show an unambiguous prose mention of the orphan page. -4. **Collect, do not fix**: contested pages needing reconcile, concept gaps, taxonomy additions worth proposing, pages whose `description` has drifted from the body. These go in the run report, not into edits. - -Forbidden in unattended runs: changing the meaning of any prose, deleting, merging, or renaming pages, creating new page types or folders, resolving contested pages, editing `CLAUDE.md`, `workflows/`, or `taxonomy.md`, and touching `raw/`. - -Finish: append one log entry, commit everything as `maintain: run`, and leave the branch unmerged. If errors remain that the allowed actions can't fix (a secrets hit, an ambiguous link), do not commit; leave the tree dirty and put the blocker at the top of the run report. - -When the human says "review maintenance": walk them through `git diff main..maintenance` change by change, then merge on approval and continue reusing the branch. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-07-05 - -- **Maintain**: scheduled run -- **Fix**: 2 broken links; index rebuilt -- **Crosslink**: queries/x-vs-y.md from synthesis/overview.md -- **Proposal**: reconcile concepts/foo.md; add tag 'observability' to taxonomy -``` diff --git a/homelab/workflows/query.md b/homelab/workflows/query.md deleted file mode 100644 index 9a7c128..0000000 --- a/homelab/workflows/query.md +++ /dev/null @@ -1,23 +0,0 @@ ---- -type: workflow ---- - -# Query - -Triggered by any question answerable from the wiki: "what depends on proxmox-01?", "why is VLAN 20 the IoT VLAN?", "how do I recover grafana?". - -1. **Scan `index.md` and frontmatter first** to find candidate pages. Do not guess at file paths, and do not open page bodies until a page's `description` or tags look relevant. For "what depends on X" questions, run `python3 lint.py reverse-deps` instead of grepping. -2. **Read the candidate pages fully**, then follow links one hop out for context (components to their topology pages, incidents to their runbooks). -3. **Synthesize an answer with citations.** Every claim links to the wiki page that supports it, and the wiki page in turn cites a source or decision record. The chain must be intact. -4. **Surface uncertainty.** If a component's `last_verified` is stale, say the answer may not match reality. If pages disagree, say so. If the wiki can't answer, say so and suggest what to document or verify to fill the gap. -5. **Offer to file the answer.** If the answer is non-trivial and likely useful later, ask whether to save it under `queries/`. If filed: create the page, run `python3 lint.py rebuild-index`, append to `log.md`, and commit: `query: `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Query**: "What breaks if proxmox-01 goes down?" -- **Filed**: queries/proxmox-01-blast-radius.md -- **Read**: 6 pages -``` diff --git a/homelab/workflows/reconcile.md b/homelab/workflows/reconcile.md deleted file mode 100644 index e912263..0000000 --- a/homelab/workflows/reconcile.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -type: workflow ---- - -# Reconcile - -Triggered by "reconcile " or by lint flagging a long-unreconciled contested page. Contested is a state to exit, not a resting place: this workflow resolves the disagreement and records what lost. - -1. **Gather the conflict.** Read the page and every source it cites. List each competing claim with its source and the source's date. -2. **Propose a resolution.** Rank claims by source recency first, then source authority (primary sources over secondary, the human's direct word over both). State which claim should win and why, show the proposed rewrite, and wait for approval. -3. **Rewrite in place.** The winning claim goes in the main body, stated plainly. Each losing claim moves to a "Superseded claims" section at the bottom: one dated line per claim with its source and a one-line rationale for why it lost. Never silently delete the loser. -4. **Update frontmatter.** Remove `confidence: contested` (downgrade to `low` instead if the winner is itself uncited). Bump `updated` and refresh `description` if the page's conclusion changed. -5. **Check the neighbors.** Follow the page's links one hop out. Any page repeating the losing claim gets the same treatment in this session. -6. **Lint, log, commit.** Run `python3 lint.py check`, append to `log.md`, commit: `reconcile: `. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-07-05 - -- **Reconcile**: zero-trust-networking -- **Winner**: sources/formal-verification-paper.md (2026-05) -- **Superseded**: claim from sources/early-blog-post.md, moved to Superseded claims -- **Update**: concepts/microsegmentation.md -``` diff --git a/homelab/workflows/triage.md b/homelab/workflows/triage.md deleted file mode 100644 index 2cbc639..0000000 --- a/homelab/workflows/triage.md +++ /dev/null @@ -1,24 +0,0 @@ ---- -type: workflow ---- - -# Triage - -The human drops config dumps, screenshots, manuals, and notes into `raw/inbox/` without sorting or renaming. Triage is bulk and cheap; documenting is per-component and slow. Never collapse them into a single step unless the human explicitly waives it. - -1. **List everything in `raw/inbox/`.** Read or inspect each file enough to classify it. For binaries (PDFs, images), inspect the first page or two. -2. **Propose a plan as a table.** For each file: current name, proposed destination subfolder under `raw/` (`configs/`, `vendor-docs/`, `screenshots/`, `decisions/`), proposed kebab-case filename, and a one-line "what this is" summary. If a file is unclear, mark it `needs human input` and ask. -3. **Wait for explicit approval.** Do not move anything until the human confirms the plan or sends corrections. If they correct individual rows, update those rows and re-show the table. -4. **Move and rename.** Once approved, move each file to its destination with the new name. Move any associated assets into `raw/assets//` if they exist alongside the source. -5. **Append a triage entry to `log.md`**, then commit: `triage: items from inbox`. -6. **Stop.** Triage does not write component or topology pages. That happens in the document workflow, separately and one component at a time. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Triage**: 4 items from inbox -- **Move**: raw/inbox/foo.cfg -> raw/configs/opnsense-2026-04-10.cfg -- **Move**: raw/inbox/IMG_2231.png -> raw/screenshots/unifi-dashboard-2026-04-10.png -``` diff --git a/homelab/workflows/verify.md b/homelab/workflows/verify.md deleted file mode 100644 index 542bb9f..0000000 --- a/homelab/workflows/verify.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -type: workflow ---- - -# Verify - -Periodic reality check. Triggered by "verify components" or "verify components/". - -1. **Build the list.** The named component, or if none was named, all components whose `last_verified` is older than 60 days (`python3 lint.py check` reports them as staleness warnings). -2. **Propose current state, ask for corrections.** For each component, ask the human targeted questions: "Is `proxmox-01` still on firmware 7.2.1? Still hosting `grafana`, `prometheus`, `loki`?" Do not ask open-ended questions; state what the wiki currently believes and let the human correct it. -3. **Apply corrections.** For each correction, update the component page (frontmatter and body), touch every other page that references the changed fact (topology diagrams included), and bump `last_verified` to today. -4. **Bump the rest.** For each unchanged component, just bump `last_verified` to today. -5. **Rebuild the index and lint** if any pages changed: `python3 lint.py rebuild-index`, then `python3 lint.py check` and fix errors. (Bumping `last_verified` changes the index annotations, so this is almost always needed.) -6. **Append a verify entry to `log.md`** listing what was checked, what changed, what didn't, then commit: `verify: components checked`. - -Log entry format (append your bullets under today's `## YYYY-MM-DD` heading if it already exists; otherwise add a new heading at the top, newest first): - -``` -## 2026-04-10 - -- **Verify**: 7 components checked -- **Change**: opnsense (firmware 25.1 -> 25.4) -- **Unchanged**: 6 -- **Note**: last_verified bumped on all 7 -``` diff --git a/codebase-large/lint.py b/tests/fixtures/codebase-large/lint.py similarity index 100% rename from codebase-large/lint.py rename to tests/fixtures/codebase-large/lint.py diff --git a/codebase/lint.py b/tests/fixtures/codebase/lint.py similarity index 100% rename from codebase/lint.py rename to tests/fixtures/codebase/lint.py diff --git a/homelab/lint.py b/tests/fixtures/homelab/lint.py similarity index 100% rename from homelab/lint.py rename to tests/fixtures/homelab/lint.py diff --git a/tests/helpers.py b/tests/helpers.py index 518665d..3dca455 100644 --- a/tests/helpers.py +++ b/tests/helpers.py @@ -5,11 +5,11 @@ from pathlib import Path REPO = Path(__file__).resolve().parent.parent -sys.path.insert(0, str(REPO / "generic")) +sys.path.insert(0, str(REPO / "wiki")) from wikilint.settings import configure # noqa: E402 -VARIANTS = ["generic", "homelab", "codebase", "codebase-large"] +VARIANTS = ["wiki"] TODAY = date.today().isoformat() DAYS_AGO_41 = (date.today() - timedelta(days=41)).isoformat() @@ -17,9 +17,17 @@ def load_variant_config(variant): - """Execute a variant's lint.py head (the __main__ guard keeps it inert).""" + """Execute a variant's lint.py head (the __main__ guard keeps it inert). + + Besides the shipped template ("wiki"), configs of the retired variants + live on under tests/fixtures/ purely as engine coverage: they exercise + knobs (membership, staleness, sync_drift, coverage) no shipped template + currently sets.""" + path = REPO / variant / "lint.py" + if not path.is_file(): + path = REPO / "tests" / "fixtures" / variant / "lint.py" ns = {"__name__": "lint_config_under_test"} - exec(compile((REPO / variant / "lint.py").read_text(), "lint.py", "exec"), ns) + exec(compile(path.read_text(), "lint.py", "exec"), ns) return ns["CONFIG"], ns["index_entry_extra"] @@ -46,7 +54,7 @@ def page(ptype, description, extra_fm="", body="Body.\n", tags="[alpha]", ) -def make_wiki(tmp, variant="generic", files=None, taxonomy="- alpha — test tag\n- beta — test tag\n"): +def make_wiki(tmp, variant="wiki", files=None, taxonomy="- alpha — test tag\n- beta — test tag\n"): """Create a minimal valid wiki skeleton for the variant plus extra files.""" root = Path(tmp) config = use_variant(variant) diff --git a/tests/test_checks.py b/tests/test_checks.py index 0a78ae0..7658729 100644 --- a/tests/test_checks.py +++ b/tests/test_checks.py @@ -402,7 +402,7 @@ def test_resolving_targets_pass_and_count_inbound(self): "concepts/b-note.md": page("concept", "B."), "lab-dir/x.txt": "not markdown\n", }) - use_variant_with("generic", + use_variant_with("wiki", non_page_allowed=["lab-dir", "CLAUDE.md", "index.md", "log.md", "lint.py", "wikilint", "taxonomy.md", "raw", "workflows"]) report = gather(root) @@ -424,7 +424,7 @@ def test_orphans_gate_off(self): root = make_wiki(self.tmp, files={ "queries/orphan-question.md": page("query", "Nobody links here."), }) - use_variant_with("generic", orphans=False) + use_variant_with("wiki", orphans=False) self.assertEqual(findings(gather(root), "orphan"), []) def test_non_page_allowed_globs(self): @@ -432,8 +432,8 @@ def test_non_page_allowed_globs(self): "lab-2026-01-01-x/README.md": "a lab\n", "mystery/notes.md": "unexpected\n", }) - config = use_variant("generic") - use_variant_with("generic", + config = use_variant("wiki") + use_variant_with("wiki", non_page_allowed=list(config["non_page_allowed"]) + ["lab-*"]) flagged = [p for _, _, p, _ in findings(gather(root), "layout", "WARNING")] self.assertNotIn("lab-2026-01-01-x", flagged) @@ -449,7 +449,7 @@ def table_body(pages): root = make_wiki(self.tmp, files={ "concepts/a-note.md": page("concept", "A."), }) - use_variant_with("generic", index_file="concepts/INDEX.md", + use_variant_with("wiki", index_file="concepts/INDEX.md", index_body_fn=table_body) from wikilint.derived import rebuild_index rebuild_index(root) @@ -466,15 +466,15 @@ def table_body(pages): def test_extra_secret_patterns_fire(self): root = make_wiki(self.tmp, files={ - "concepts/token.md": page("concept", "T.", body="pat nbp_abc12345XYZ here\n"), + "concepts/token.md": page("concept", "T.", body="pat acme_abc12345XYZ here\n"), }) use_variant_with( - "generic", - extra_secret_patterns=[(r"\bnbp_[A-Za-z0-9]{8,}", "netbird PAT")], + "wiki", + extra_secret_patterns=[(r"\bacme_[A-Za-z0-9]{8,}", "acme PAT")], ) hits = findings(gather(root), "secrets", "ERROR") self.assertEqual(len(hits), 1) - self.assertIn("netbird PAT", hits[0][3]) + self.assertIn("acme PAT", hits[0][3]) self.assertTrue(hits[0][2].startswith("concepts/token.md:")) def test_secret_allowlist_suppresses_a_real_match(self): @@ -487,9 +487,9 @@ def test_secret_allowlist_suppresses_a_real_match(self): "concept", "Tpl.", body="password: ${REDACTED}\npassword: leakedhunter2\n"), }) - use_variant_with("generic") + use_variant_with("wiki") self.assertEqual(len(findings(gather(root), "secrets", "ERROR")), 2) - use_variant_with("generic", secret_allow_res=[r"\$\{[^}]*\}"]) + use_variant_with("wiki", secret_allow_res=[r"\$\{[^}]*\}"]) hits = findings(gather(root), "secrets", "ERROR") self.assertEqual(len(hits), 1) self.assertTrue(hits[0][2].endswith(":11"), hits[0][2]) # the plain line @@ -498,7 +498,7 @@ def test_iso_date_fields_configurable(self): root = make_wiki(self.tmp, files={ "concepts/dated.md": page("concept", "D.", extra_fm="date: not-a-date\n"), }) - use_variant_with("generic", + use_variant_with("wiki", iso_date_fields=["created", "updated", "date"]) messages = [m for _, _, _, m in findings(gather(root), "frontmatter", "ERROR")] self.assertTrue(any("date is not an ISO date" in m for m in messages)) @@ -509,7 +509,7 @@ def test_none_gates_disable_log_tags_inbox(self): }) (root / "log.md").unlink() (root / "taxonomy.md").unlink() - use_variant_with("generic", log_file=None, taxonomy_file=None, + use_variant_with("wiki", log_file=None, taxonomy_file=None, inbox_dir=None) report = gather(root) # must not crash on inbox_dir=None self.assertEqual(findings(report, "log"), []) @@ -522,7 +522,7 @@ def custom(pages, report, root): root = make_wiki(self.tmp, files={ "concepts/a-note.md": page("concept", "A."), }) - use_variant_with("generic", extra_checks=[custom]) + use_variant_with("wiki", extra_checks=[custom]) hits = findings(gather(root), "custom", "INFO") self.assertEqual(len(hits), 1) self.assertIn("saw 1 pages", hits[0][3]) @@ -539,7 +539,7 @@ def test_check_log_reports_the_configured_path(self): root = make_wiki(self.tmp, files={}) (root / "log.md").unlink() (root / "ops-journal.md").write_text("# Log\n\n## bad header\n") - use_variant_with("generic", log_file="ops-journal.md") + use_variant_with("wiki", log_file="ops-journal.md") hits = findings(gather(root), "log", "WARNING") self.assertTrue(hits) self.assertTrue(all(h[2].startswith("ops-journal.md:") for h in hits), hits) @@ -549,13 +549,13 @@ def test_created_updated_ordering_independent_of_iso_date_fields(self): "concepts/a-note.md": page("concept", "A.", created="2026-05-01", updated="2026-04-01"), }) - use_variant_with("generic", iso_date_fields=["date"]) # excludes the pair + use_variant_with("wiki", iso_date_fields=["date"]) # excludes the pair msgs = [m for _, _, _, m in findings(gather(root), "frontmatter", "ERROR")] self.assertIn("updated is older than created", msgs) def test_index_file_none_disables_index_without_crashing(self): root = make_wiki(self.tmp, files={"concepts/a-note.md": page("concept", "A.")}) - use_variant_with("generic", index_file=None) + use_variant_with("wiki", index_file=None) self.assertEqual(findings(gather(root), "index"), []) def test_invalid_index_file_raises_config_error(self): @@ -563,18 +563,18 @@ def test_invalid_index_file_raises_config_error(self): make_wiki(self.tmp, files={"concepts/a-note.md": page("concept", "A.")}) for bad in ("/abs/index.md", "../escape.md", ""): with self.assertRaises(ConfigError): - use_variant_with("generic", index_file=bad) + use_variant_with("wiki", index_file=bad) def test_rebuild_index_creates_missing_parent_dir(self): root = make_wiki(self.tmp, files={"concepts/a-note.md": page("concept", "A.")}) - use_variant_with("generic", index_file="derived/index.md", + use_variant_with("wiki", index_file="derived/index.md", index_body_fn=lambda pages: "- x\n") self._rebuild(root) # must not raise self.assertTrue((root / "derived/index.md").is_file()) def test_dot_prefixed_index_is_skipped_by_discovery(self): root = make_wiki(self.tmp, files={"concepts/a-note.md": page("concept", "A.")}) - use_variant_with("generic", index_file="./concepts/INDEX.md", + use_variant_with("wiki", index_file="./concepts/INDEX.md", index_body_fn=lambda pages: "- x\n") self._rebuild(root) fm = [i for i in findings(gather(root), "frontmatter", "ERROR") if "INDEX" in i[2]] @@ -582,7 +582,7 @@ def test_dot_prefixed_index_is_skipped_by_discovery(self): def test_relocated_root_index_not_flagged_by_layout(self): root = make_wiki(self.tmp, files={"concepts/a-note.md": page("concept", "A.")}) - use_variant_with("generic", index_file="catalog.md", + use_variant_with("wiki", index_file="catalog.md", index_body_fn=lambda pages: "- x\n") self._rebuild(root) layout = [p for _, _, p, _ in findings(gather(root), "layout", "WARNING")] @@ -590,7 +590,7 @@ def test_relocated_root_index_not_flagged_by_layout(self): def test_index_body_fn_leading_newline_no_perpetual_drift(self): root = make_wiki(self.tmp, files={"concepts/a-note.md": page("concept", "A.")}) - use_variant_with("generic", index_body_fn=lambda pages: "\n| P |\n| a |\n") + use_variant_with("wiki", index_body_fn=lambda pages: "\n| P |\n| a |\n") self._rebuild(root) self.assertEqual(findings(gather(root), "index"), []) @@ -598,16 +598,16 @@ def test_bad_secret_regex_raises_config_error(self): from wikilint.settings import ConfigError make_wiki(self.tmp, files={"concepts/a-note.md": page("concept", "A.")}) with self.assertRaises(ConfigError): - use_variant_with("generic", extra_secret_patterns=[("nbp_[", "bad")]) + use_variant_with("wiki", extra_secret_patterns=[("acme_[", "bad")]) with self.assertRaises(ConfigError): - use_variant_with("generic", secret_allow_res=["(unclosed"]) + use_variant_with("wiki", secret_allow_res=["(unclosed"]) def test_extra_check_exception_becomes_a_finding(self): def boom(pages, report, root): raise KeyError("missing") root = make_wiki(self.tmp, files={"concepts/a-note.md": page("concept", "A.")}) - use_variant_with("generic", extra_checks=[boom]) + use_variant_with("wiki", extra_checks=[boom]) report = gather(root) # must not crash hits = findings(report, "extra-check", "ERROR") self.assertEqual(len(hits), 1) @@ -617,7 +617,7 @@ def test_non_callable_extra_check_raises_config_error(self): from wikilint.settings import ConfigError make_wiki(self.tmp, files={"concepts/a-note.md": page("concept", "A.")}) with self.assertRaises(ConfigError): - use_variant_with("generic", extra_checks=["not callable"]) + use_variant_with("wiki", extra_checks=["not callable"]) def test_image_wrapped_link_credits_outer_target(self): root = make_wiki(self.tmp, files={ @@ -706,7 +706,7 @@ def test_type_lines_are_not_tags(self): def test_glossary_gate_off(self): make_wiki(self.tmp) - use_variant_with("generic", types_glossary=False) + use_variant_with("wiki", types_glossary=False) root = Path(self.tmp) (root / "taxonomy.md").write_text( "---\ntype: tooling\n---\n\n# Taxonomy\n\n- alpha — test tag\n" @@ -724,7 +724,7 @@ class TestSkillsPairing(WikiTest): def wiki(self, files=None): root = make_wiki(self.tmp, files=files) - use_variant_with("generic", skills_dir=".claude/skills") + use_variant_with("wiki", skills_dir=".claude/skills") return root def test_unprefixed_wrapper_is_orphan(self): @@ -744,7 +744,7 @@ def test_disabled_when_knob_unset(self): # Every variant now ships skills_dir, so exercise the engine default # (None) by overriding it off: no pairing checks run. root = make_wiki(self.tmp, files={"workflows/document.md": self.WORKFLOW}) - use_variant_with("generic", skills_dir=None) + use_variant_with("wiki", skills_dir=None) report = gather(root) self.assertEqual(findings(report, "skills"), []) @@ -849,7 +849,7 @@ def test_okf_conformance_gate_off(self): root = make_wiki(self.tmp, files={ "notes.md": "no frontmatter here\n", }) - use_variant_with("generic", okf_conformance=False) + use_variant_with("wiki", okf_conformance=False) self.assertEqual(findings(gather(root), "okf"), []) @@ -946,7 +946,7 @@ def test_dangling_mapping_resource_still_reported(self): def test_gate_off_disables_shape_check(self): root = self._wiki("sources: [sources/foo-article.md]\n") - use_variant_with("generic", okf_conformance=False) + use_variant_with("wiki", okf_conformance=False) self.assertEqual(findings(gather(root), "sources"), []) diff --git a/tests/test_variants.py b/tests/test_variants.py index 2af8caf..718410f 100644 --- a/tests/test_variants.py +++ b/tests/test_variants.py @@ -1,26 +1,17 @@ -"""Cross-variant invariants: byte-identical engine, compilable heads, sane -configs, line budgets, and hook correctness properties.""" +"""Template invariants: compilable heads, sane configs, line budgets, and +hook correctness properties, for the shipped template and the retired-variant +configs kept under tests/fixtures/ as engine coverage.""" import py_compile import unittest from helpers import REPO, VARIANTS, load_variant_config +# Shipped template plus config-only fixtures (see helpers.load_variant_config). +ALL_CONFIGS = VARIANTS + ["homelab", "codebase", "codebase-large"] -class TestEngineIdentity(unittest.TestCase): - def test_wikilint_byte_identical_across_variants(self): - reference = { - p.name: p.read_bytes() for p in (REPO / "generic" / "wikilint").glob("*.py") - } - self.assertTrue(reference) - for variant in VARIANTS[1:]: - files = { - p.name: p.read_bytes() for p in (REPO / variant / "wikilint").glob("*.py") - } - self.assertEqual(files.keys(), reference.keys(), variant) - for name, blob in reference.items(): - self.assertEqual(files[name], blob, f"{variant}/wikilint/{name} differs") +class TestEngineIdentity(unittest.TestCase): def test_everything_compiles(self): for variant in VARIANTS: py_compile.compile(str(REPO / variant / "lint.py"), doraise=True) @@ -28,31 +19,22 @@ def test_everything_compiles(self): py_compile.compile(str(p), doraise=True) -# Knobs a variant may set differently from the others, each with a reason. -# Anything not listed here must stay uniform across variants. -INTENTIONAL_DIVERGENCES = set() - - class TestConfigSanity(unittest.TestCase): - def test_configs_load_and_share_keys(self): - key_sets = [] - for variant in VARIANTS: + def test_configs_load(self): + for variant in ALL_CONFIGS: config, extra = load_variant_config(variant) self.assertTrue(callable(extra), variant) - key_sets.append((variant, set(config) - INTENTIONAL_DIVERGENCES)) - _, reference = key_sets[0] - for variant, keys in key_sets[1:]: - self.assertEqual(keys, reference, f"{variant} CONFIG keys diverge from generic") + self.assertTrue(config, variant) def test_reverse_fields_are_known_fields(self): - for variant in VARIANTS: + for variant in ALL_CONFIGS: config, _ = load_variant_config(variant) known = set(config["path_fields"]) | set(config["edge_fields"]) for field in config["reverse_fields"]: self.assertIn(field, known, f"{variant}: reverse field {field} unresolvable") def test_membership_references_real_types(self): - for variant in VARIANTS: + for variant in ALL_CONFIGS: config, _ = load_variant_config(variant) rule = config["membership"] if not rule: @@ -62,22 +44,14 @@ def test_membership_references_real_types(self): self.assertIn(rule["container_field"], config["path_fields"] + config["edge_fields"], variant) - def test_extension_defaults_present_and_consistent(self): - """Every variant's effective config carries the extension keys, and no - variant silently overrides one to a different value (which would - enable/disable a knob for that variant's users unnoticed).""" + def test_extension_defaults_present(self): + """The template's effective config carries every extension key.""" from wikilint.settings import DEFAULTS, configure, CONFIG - seen = {} for variant in VARIANTS: config, extra = load_variant_config(variant) configure(config, extra) for key in DEFAULTS: self.assertIn(key, CONFIG, f"{variant} missing extension key {key}") - if key in INTENTIONAL_DIVERGENCES: - continue - seen.setdefault(key, CONFIG[key]) - self.assertEqual(CONFIG[key], seen[key], - f"{variant} diverges on extension key {key}") class TestBudgets(unittest.TestCase): @@ -98,12 +72,10 @@ def test_python_files_under_800_lines(self): class TestHookAndWorkflows(unittest.TestCase): - def test_hook_lints_staged_snapshot_and_is_identical(self): - reference = (REPO / "generic" / ".githooks" / "pre-commit").read_bytes() - self.assertIn(b"checkout-index", reference) + def test_hook_lints_staged_snapshot(self): for variant in VARIANTS: hook = REPO / variant / ".githooks" / "pre-commit" - self.assertEqual(hook.read_bytes(), reference, variant) + self.assertIn(b"checkout-index", hook.read_bytes(), variant) self.assertTrue(hook.stat().st_mode & 0o111, f"{variant} hook not executable") def test_dispatch_table_matches_workflow_files(self): diff --git a/generic/.claude/skills/wiki-ingest/SKILL.md b/wiki/.claude/skills/wiki-ingest/SKILL.md similarity index 100% rename from generic/.claude/skills/wiki-ingest/SKILL.md rename to wiki/.claude/skills/wiki-ingest/SKILL.md diff --git a/codebase-large/.claude/skills/wiki-lint/SKILL.md b/wiki/.claude/skills/wiki-lint/SKILL.md similarity index 100% rename from codebase-large/.claude/skills/wiki-lint/SKILL.md rename to wiki/.claude/skills/wiki-lint/SKILL.md diff --git a/codebase-large/.claude/skills/wiki-maintain/SKILL.md b/wiki/.claude/skills/wiki-maintain/SKILL.md similarity index 100% rename from codebase-large/.claude/skills/wiki-maintain/SKILL.md rename to wiki/.claude/skills/wiki-maintain/SKILL.md diff --git a/generic/.claude/skills/wiki-query/SKILL.md b/wiki/.claude/skills/wiki-query/SKILL.md similarity index 100% rename from generic/.claude/skills/wiki-query/SKILL.md rename to wiki/.claude/skills/wiki-query/SKILL.md diff --git a/generic/.claude/skills/wiki-reconcile/SKILL.md b/wiki/.claude/skills/wiki-reconcile/SKILL.md similarity index 100% rename from generic/.claude/skills/wiki-reconcile/SKILL.md rename to wiki/.claude/skills/wiki-reconcile/SKILL.md diff --git a/generic/.claude/skills/wiki-triage/SKILL.md b/wiki/.claude/skills/wiki-triage/SKILL.md similarity index 100% rename from generic/.claude/skills/wiki-triage/SKILL.md rename to wiki/.claude/skills/wiki-triage/SKILL.md diff --git a/codebase-large/.githooks/pre-commit b/wiki/.githooks/pre-commit similarity index 100% rename from codebase-large/.githooks/pre-commit rename to wiki/.githooks/pre-commit diff --git a/generic/CLAUDE.md b/wiki/CLAUDE.md similarity index 100% rename from generic/CLAUDE.md rename to wiki/CLAUDE.md diff --git a/generic/lint.py b/wiki/lint.py similarity index 91% rename from generic/lint.py rename to wiki/lint.py index 98d2e20..12438e7 100644 --- a/generic/lint.py +++ b/wiki/lint.py @@ -1,10 +1,10 @@ #!/usr/bin/env python3 -"""Deterministic lint for this wiki (generic variant). Stdlib only; run from -the wiki root. Subcommands: check | rebuild-index | reverse-deps | coverage. +"""Deterministic lint for this wiki. Stdlib only; run from the wiki root. +Subcommands: check | rebuild-index | reverse-deps | coverage. -This file holds ONLY the per-variant configuration. All logic lives in the -adjacent wikilint/ package, which is byte-identical across variants so fixes -propagate by copy. If a check can be a script, it belongs there, not in prose. +This file holds ONLY the configuration. All logic lives in the adjacent +wikilint/ package, kept identical across installs so fixes propagate by +copy. If a check can be a script, it belongs there, not in prose. """ CONFIG = { diff --git a/generic/taxonomy.md b/wiki/taxonomy.md similarity index 100% rename from generic/taxonomy.md rename to wiki/taxonomy.md diff --git a/wiki/wikilint/__init__.py b/wiki/wikilint/__init__.py new file mode 100644 index 0000000..99172a8 --- /dev/null +++ b/wiki/wikilint/__init__.py @@ -0,0 +1,7 @@ +"""Wiki lint engine. This package is kept identical across installs so +fixes propagate by copy; per-wiki behavior lives entirely in the CONFIG +dict and index_entry_extra() defined in the wiki root's lint.py.""" + +from .cli import main + +__all__ = ["main"] diff --git a/codebase-large/wikilint/checks.py b/wiki/wikilint/checks.py similarity index 100% rename from codebase-large/wikilint/checks.py rename to wiki/wikilint/checks.py diff --git a/codebase-large/wikilint/cli.py b/wiki/wikilint/cli.py similarity index 100% rename from codebase-large/wikilint/cli.py rename to wiki/wikilint/cli.py diff --git a/codebase-large/wikilint/derived.py b/wiki/wikilint/derived.py similarity index 100% rename from codebase-large/wikilint/derived.py rename to wiki/wikilint/derived.py diff --git a/codebase-large/wikilint/model.py b/wiki/wikilint/model.py similarity index 100% rename from codebase-large/wikilint/model.py rename to wiki/wikilint/model.py diff --git a/codebase-large/wikilint/settings.py b/wiki/wikilint/settings.py similarity index 100% rename from codebase-large/wikilint/settings.py rename to wiki/wikilint/settings.py diff --git a/generic/workflows/ingest.md b/wiki/workflows/ingest.md similarity index 100% rename from generic/workflows/ingest.md rename to wiki/workflows/ingest.md diff --git a/generic/workflows/lint.md b/wiki/workflows/lint.md similarity index 100% rename from generic/workflows/lint.md rename to wiki/workflows/lint.md diff --git a/codebase/workflows/maintain.md b/wiki/workflows/maintain.md similarity index 100% rename from codebase/workflows/maintain.md rename to wiki/workflows/maintain.md diff --git a/generic/workflows/query.md b/wiki/workflows/query.md similarity index 100% rename from generic/workflows/query.md rename to wiki/workflows/query.md diff --git a/codebase-large/workflows/reconcile.md b/wiki/workflows/reconcile.md similarity index 100% rename from codebase-large/workflows/reconcile.md rename to wiki/workflows/reconcile.md diff --git a/generic/workflows/triage.md b/wiki/workflows/triage.md similarity index 100% rename from generic/workflows/triage.md rename to wiki/workflows/triage.md From 5e9c9f1b5812fd78744e57b9b1d909dc140c012e Mon Sep 17 00:00:00 2001 From: SunsetDrifter Date: Sun, 26 Jul 2026 15:25:52 +0200 Subject: [PATCH 2/3] docs: drop the variant-retirement note from the README --- README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/README.md b/README.md index 709346e..fb550b4 100644 --- a/README.md +++ b/README.md @@ -14,8 +14,6 @@ One layering note when comparing bundles across producers: OKF reserves two file `wiki/` organizes knowledge as sources / entities / concepts / synthesis / queries, fed from an immutable, human-owned `raw/` inbox. It suits reading notes, research, and any source-driven knowledge base. -Earlier revisions shipped three sibling variants (homelab, codebase, codebase-large) with specialized schemas on the same engine. They were retired in favor of one well-tested template (their configs live on under `tests/fixtures/` as engine coverage, and in git history if you want to resurrect one); the engine keeps the extension knobs they exercised, so a specialized schema is a `lint.py` + `CLAUDE.md` away. - ## Install ```sh From b1c6051cefa6c4f2c58656d83a22c0a383869245 Mon Sep 17 00:00:00 2001 From: SunsetDrifter Date: Sun, 26 Jul 2026 15:34:45 +0200 Subject: [PATCH 3/3] chore: update a stale schema comment in tests --- tests/test_checks.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/test_checks.py b/tests/test_checks.py index 7658729..e627a1b 100644 --- a/tests/test_checks.py +++ b/tests/test_checks.py @@ -676,7 +676,7 @@ def test_missing_section_warns_once(self): def test_undescribed_and_unknown_and_empty_meaning(self): root = make_wiki(self.tmp) - # generic schema types: source, entity, concept, synthesis, query. + # wiki schema types: source, entity, concept, synthesis, query. # Describe all but 'query', add a meaningless entry and an unknown one. (root / "taxonomy.md").write_text( "---\ntype: tooling\n---\n\n# Taxonomy\n\n- alpha — test tag\n"