diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e54b0f5..ace6f05 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -34,7 +34,7 @@ jobs: run: mypy src/ - name: Test with coverage - run: pytest --cov=matledger --cov-report=term-missing --cov-fail-under=80 -q + run: pytest --cov=nilvar --cov-report=term-missing --cov-fail-under=80 -q security: runs-on: ubuntu-latest @@ -60,5 +60,5 @@ jobs: - uses: actions/checkout@v4 - name: Build Docker image - run: docker build -t matledger:ci . + run: docker build -t nilvar:ci . diff --git a/CHANGELOG.md b/CHANGELOG.md index 228dd2f..2aef292 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,6 @@ # Changelog -All notable changes to matledger are documented here. +All notable changes to nilvar are documented here. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [0.9.0] - 2026-07-01 @@ -33,11 +33,11 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). `/analytics/explore-exploit`, `/analytics/data-quality` - CLI: all 11 analytics now included in `report` command (JSON, markdown, HTML) - Field-relevance classification in README analytics table -- `matledger export` CLI command: portable JSON evidence bundle with all events, +- `nilvar export` CLI command: portable JSON evidence bundle with all events, hash chain verification, event counts, schema version, and timestamps -- Enhanced `matledger audit`: now shows hash version, event range, time range, +- Enhanced `nilvar audit`: now shows hash version, event range, time range, event breakdown by type, actors, and iteration range (text and JSON modes) -- Error hierarchy: `MatledgerError` base with `LifecycleError`, `CausalLinkError`, +- Error hierarchy: `NilvarError` base with `LifecycleError`, `CausalLinkError`, `DuplicateEventError`, `ProjectionError`, `IngestError` subclasses - `--verbose` / `-v` and `--quiet` / `-q` CLI flags wiring Python logging level - `SchemaError` and `ConfigError` exception subclasses for structured error handling @@ -46,14 +46,14 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). - Structured logging across handler, store, projections, and API middleware - Bandit (ruff S) security linting with justified `noqa` for false positives - API request logging middleware with timing -- `MatledgerError` exception handler in API returning structured error codes +- `NilvarError` exception handler in API returning structured error codes - Trajectory plots (proxy-gaming, diversity, budget, discovery rate) - HTML report with embedded charts - SDK plotting helpers - `GET /health` endpoint returning event count and status - `--version` / `-V` flag on CLI - `--format json` for `report` command; `--output json` for `audit` command -- `pydantic-settings` typed configuration (`MATLEDGER_DB`, `MATLEDGER_LOG_LEVEL`, `MATLEDGER_CORS_ORIGINS`) +- `pydantic-settings` typed configuration (`NILVAR_DB`, `NILVAR_LOG_LEVEL`, `NILVAR_CORS_ORIGINS`) - Multi-stage Dockerfile with non-root user - Docker Compose with health check, named volume, and `.env` support - `.env.example` configuration reference diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e6e82e4..8a0cdbf 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,12 +1,12 @@ -# Contributing to matledger +# Contributing to nilvar -Thank you for considering a contribution. matledger is an event-sourced campaign ledger for AI-driven scientific discovery, and contributions that strengthen its core invariants are welcome. +Thank you for considering a contribution. nilvar is an event-sourced campaign ledger for AI-driven scientific discovery, and contributions that strengthen its core invariants are welcome. ## Quick start ```bash -git clone https://github.com/jaysonmulwa/matledger.git -cd matledger +git clone https://github.com/jaysonmulwa/nilvar.git +cd nilvar make dev # install with all dependencies make check # run the full quality gate ``` @@ -28,7 +28,7 @@ Every change must pass before merging: | Lint | `ruff check src/ tests/` | Zero errors | | Format | `ruff format --check src/ tests/` | Zero diffs | | Type check | `mypy src/` | Strict mode, zero errors | -| Tests | `pytest --cov=matledger --cov-fail-under=80` | 80%+ coverage | +| Tests | `pytest --cov=nilvar --cov-fail-under=80` | 80%+ coverage | Run all four at once with `make check`. @@ -51,13 +51,13 @@ type(scope): description ## Architecture constraints -matledger uses event sourcing with CQRS. These invariants must not be violated: +nilvar uses event sourcing with CQRS. These invariants must not be violated: - **CQRS import boundaries.** `events/` imports nothing else. `store/` and `commands/` import from `events/` only. `projections/` imports from `events/` only. `analytics/` imports from `events/` and `projections/`. See [CLAUDE.md](CLAUDE.md) for the full import map. - **Append-only immutability.** Events are never updated or deleted after appending. - **Hash chain integrity.** Every event is SHA-256 hash-chained to its predecessor. Changes must not break the chain. - **Content-hash deduplication.** Duplicate events are rejected by content hash. The hash covers type, payload, iteration, and replicate. -- **Structured exceptions.** Use `MatledgerError` subclasses, not bare `ValueError` or `AssertionError`. +- **Structured exceptions.** Use `NilvarError` subclasses, not bare `ValueError` or `AssertionError`. ## Testing guidelines @@ -72,7 +72,7 @@ Every new dependency must justify its weight. Do not add a dependency that dupli ## Compliance awareness -matledger claims alignment (not compliance) with ALCOA+, 21 CFR Part 11, FAIR, and GAMP 5. When proposing changes, flag anything that would weaken the compliance position: breaking append-only immutability, removing actor/timestamp fields, bypassing the hash chain, or exposing event data without access boundaries. +nilvar claims alignment (not compliance) with ALCOA+, 21 CFR Part 11, FAIR, and GAMP 5. When proposing changes, flag anything that would weaken the compliance position: breaking append-only immutability, removing actor/timestamp fields, bypassing the hash chain, or exposing event data without access boundaries. ## Security diff --git a/README.md b/README.md index 73aad2a..0b5658b 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# matledger +# nilvar [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/downloads/) [![Tests](https://img.shields.io/badge/tests-374%20passed-brightgreen.svg)](#g-development--contributing) @@ -16,21 +16,21 @@ Event-sourced campaign ledger for AI-driven scientific discovery — 11 trajecto AI-driven discovery loops (materials, chemistry, drug candidates) generate and screen candidates per iteration. The loop is fragile in ways current tooling can't surface: surrogate-model exploitation, diversity collapse, repeated failed experiments, and poor budget visibility. -Existing tools record **independent runs**. A discovery campaign is a **causal loop** (propose → test → learn → decide). The questions that matter — *is the surrogate decoupling from reality? is the search collapsing?* — are functions of the **trajectory**, not properties of current state. matledger models the campaign as a causal process and mines it for discovery-health signals. +Existing tools record **independent runs**. A discovery campaign is a **causal loop** (propose → test → learn → decide). The questions that matter — *is the surrogate decoupling from reality? is the search collapsing?* — are functions of the **trajectory**, not properties of current state. nilvar models the campaign as a causal process and mines it for discovery-health signals. ## How it compares -Researchers already track experiments with other tools. matledger occupies a different niche: it models the campaign as a **causal trajectory**, not a collection of independent runs. +Researchers already track experiments with other tools. nilvar occupies a different niche: it models the campaign as a **causal trajectory**, not a collection of independent runs. -| Tool | What it tracks | Where matledger differs | +| Tool | What it tracks | Where nilvar differs | |------|---------------|------------------------| -| **MLflow / W&B** | Model metrics, hyperparameters, individual experiment runs | matledger tracks the full propose-test-learn-decide loop across iterations; its analytics detect trajectory-level pathologies (proxy-gaming, diversity collapse) that per-run trackers cannot surface | -| **Electronic lab notebooks** | Lab procedures, observations, results in human-readable form | matledger is machine-readable, event-sourced, and hash-chained for tamper detection; events are structured data, not prose | +| **MLflow / W&B** | Model metrics, hyperparameters, individual experiment runs | nilvar tracks the full propose-test-learn-decide loop across iterations; its analytics detect trajectory-level pathologies (proxy-gaming, diversity collapse) that per-run trackers cannot surface | +| **Electronic lab notebooks** | Lab procedures, observations, results in human-readable form | nilvar is machine-readable, event-sourced, and hash-chained for tamper detection; events are structured data, not prose | | **Spreadsheets / AirTable** | Ad-hoc campaign tracking, manual entry | No immutability guarantees, no audit trail, no built-in analytics; a shared spreadsheet cannot prove it was not edited after the fact | -| **Custom scripts** | Whatever you build | matledger ships 11 trajectory analytics grounded in the Bayesian optimization literature, plus counterfactual replay, out of the box | -| **Ax / BoTorch / Dragonfly** | Surrogate models and acquisition functions (the optimization engine) | matledger does not replace the engine; it sits alongside it, recording what the engine proposed and what reality returned, then auditing the gap | +| **Custom scripts** | Whatever you build | nilvar ships 11 trajectory analytics grounded in the Bayesian optimization literature, plus counterfactual replay, out of the box | +| **Ax / BoTorch / Dragonfly** | Surrogate models and acquisition functions (the optimization engine) | nilvar does not replace the engine; it sits alongside it, recording what the engine proposed and what reality returned, then auditing the gap | -matledger is complementary to most of these. It ingests predictions from your surrogate model of choice and records outcomes from your lab or simulation, then mines the trajectory for signals that no individual tool above is designed to catch. +nilvar is complementary to most of these. It ingests predictions from your surrogate model of choice and records outcomes from your lab or simulation, then mines the trajectory for signals that no individual tool above is designed to catch. ## Table of Contents @@ -60,7 +60,7 @@ This installs the CLI, API server, analytics (FAISS), trajectory charts, and YAM | Extra | What it adds | |-------|-------------| -| `cli` | Typer CLI (`matledger` command) | +| `cli` | Typer CLI (`nilvar` command) | | `api` | FastAPI server + uvicorn | | `analytics` | FAISS for vector similarity search | | `viz` | Matplotlib trajectory charts | @@ -81,27 +81,27 @@ pip install -e ".[dev]" ```bash # Initialize a campaign -matledger init --campaign-id perovskite-opt --goal "maximize bandgap" --budget-cap 50000.0 +nilvar init --campaign-id perovskite-opt --goal "maximize bandgap" --budget-cap 50000.0 # Ingest candidate data from a file -matledger ingest --file candidates.json +nilvar ingest --file candidates.json # Record a validated lab result -matledger record-result --candidate-id CsPbI3 --value 1.68 --uncertainty 0.05 --iteration 1 +nilvar record-result --candidate-id CsPbI3 --value 1.68 --uncertainty 0.05 --iteration 1 # Generate a campaign report -matledger report -matledger report --format html --output report.html +nilvar report +nilvar report --format html --output report.html # Run analytics -matledger counterfactual --policy greedy --top-k 1 +nilvar counterfactual --policy greedy --top-k 1 ```
SDK Quick Start (notebooks/scripts) ```python -from matledger.sdk import CampaignClient +from nilvar.sdk import CampaignClient client = CampaignClient("my_campaign.db") client.create_campaign("perovskite-opt", "maximize bandgap", 50000.0) @@ -126,7 +126,7 @@ print(client.report()) ``` See [`examples/basic_campaign.py`](examples/basic_campaign.py) for a complete 3-iteration campaign, or [`examples/notebook_demo.ipynb`](examples/notebook_demo.ipynb) for an interactive Jupyter walkthrough. -You can also [run the notebook directly in Google Colab](https://colab.research.google.com/github/jaysonmulwa/matledger/blob/main/examples/notebook_demo.ipynb) — no local install required. +You can also [run the notebook directly in Google Colab](https://colab.research.google.com/github/jaysonmulwa/nilvar/blob/main/examples/notebook_demo.ipynb) — no local install required. For a realistic 5-iteration perovskite campaign that exercises every CLI command (CSV ingestion, failures, counterfactuals, reports), see the [smoke test fixtures](fixtures/smoke_test/README.md). The accompanying [analytics guide](fixtures/smoke_test/analytics_guide.html) explains how proxy gaming happens, what corrections exist, and how counterfactual replay quantifies the cost. @@ -217,7 +217,7 @@ Every event is hash-chained (SHA-256) for tamper-evidence and content-hash dedup ### Project Structure ``` -src/matledger/ +src/nilvar/ ├── events/ # Event envelope, catalog, failure taxonomy (WRITE SIDE) ├── store/ # SQLite append-only event store (WRITE SIDE) ├── commands/ # Command handler with validation + dedup (WRITE SIDE) @@ -234,9 +234,9 @@ src/matledger/ ## C. Analytics -matledger's differentiators — trajectory analytics that CRUD experiment trackers structurally cannot provide. +nilvar's differentiators — trajectory analytics that CRUD experiment trackers structurally cannot provide. -*BO = Bayesian optimisation — the closed-loop surrogate-driven search that matledger records.* +*BO = Bayesian optimisation — the closed-loop surrogate-driven search that nilvar records.* | Analytic | What it detects | Signal | Field relevance | |----------|----------------|--------|----------------| @@ -261,7 +261,7 @@ Install the `viz` extra for matplotlib-based trajectory visualisation: pip install -e ".[viz]" ``` -Charts are embedded automatically in HTML reports (`matledger report --format html`). In notebooks, use the SDK plotting helpers ([try them in Colab](https://colab.research.google.com/github/jaysonmulwa/matledger/blob/main/examples/notebook_demo.ipynb)): +Charts are embedded automatically in HTML reports (`nilvar report --format html`). In notebooks, use the SDK plotting helpers ([try them in Colab](https://colab.research.google.com/github/jaysonmulwa/nilvar/blob/main/examples/notebook_demo.ipynb)): ```python fig = client.plot_proxy_gaming() # predicted vs validated divergence @@ -290,18 +290,18 @@ Replays the recorded campaign through alternate decision policies (greedy or ran ## D. Compliance Alignment -matledger's event-sourcing architecture aligns with scientific data integrity standards by design. These properties are structural consequences of event sourcing, not bolted-on features. +nilvar's event-sourcing architecture aligns with scientific data integrity standards by design. These properties are structural consequences of event sourcing, not bolted-on features. -| Standard | How matledger aligns | What's not covered | +| Standard | How nilvar aligns | What's not covered | |----------|---------------------|-------------------| -| **ALCOA+** (data integrity) | Attributable (actor field), Contemporaneous (ts + server-side recorded_at), Original (append-only log + raw_data_ref), Accurate (hash chain + unit fields), Legible (`matledger export` evidence bundle) | Enduring (archival/WORM), Available (access controls) | +| **ALCOA+** (data integrity) | Attributable (actor field), Contemporaneous (ts + server-side recorded_at), Original (append-only log + raw_data_ref), Accurate (hash chain + unit fields), Legible (`nilvar export` evidence bundle) | Enduring (archival/WORM), Available (access controls) | | **21 CFR Part 11** (FDA electronic records) | Immutable audit trail with tamper-evident hash chain | E-signatures, access controls, system validation (organizational controls beyond a local-first tool) | | **FAIR** (scientific data) | Findable (campaign URN), Accessible (REST API), Interoperable (JSON schema), Reusable (license, provenance, replay from seq 0) | Persistent DOIs, standard ontologies, JSON-LD export | | **GAMP 5** (pharma system validation) | Category 5 classification with validation pack: IQ/OQ/PQ protocols, traceability matrix | Executed validation is the adopting organization's responsibility | -matledger provides the technical substrate for compliance; the adopting organization layers identity, access controls, SOPs, and validation execution on top. +nilvar provides the technical substrate for compliance; the adopting organization layers identity, access controls, SOPs, and validation execution on top. -**GAMP 5 validation pack:** `docs/gamp/` contains the system classification, traceability matrix, and executed IQ/OQ/PQ protocols with actual results (53 steps passed, 16 Docker-dependent steps deferred to CI). Organizations adopting matledger in GxP environments should re-execute these protocols in their target environment. +**GAMP 5 validation pack:** `docs/gamp/` contains the system classification, traceability matrix, and executed IQ/OQ/PQ protocols with actual results (53 steps passed, 16 Docker-dependent steps deferred to CI). Organizations adopting nilvar in GxP environments should re-execute these protocols in their target environment. Standards assessed and documented as not applicable: EU GMP Annex 11, ICH GCP E6, HIPAA, GDPR, ISO 27001/SOC 2, ISO 42001/AI governance, export controls. See [`docs/compliance_reference.md`](docs/compliance_reference.md) for rationale on each. @@ -312,52 +312,52 @@ Standards assessed and documented as not applicable: EU GMP Annex 11, ICH GCP E6 ### CLI ```bash -matledger --version # print version and exit -matledger --verbose ... # enable debug logging -matledger --quiet ... # suppress info, show warnings/errors only +nilvar --version # print version and exit +nilvar --verbose ... # enable debug logging +nilvar --quiet ... # suppress info, show warnings/errors only ``` **Write commands:** ```bash # Initialize a campaign database -matledger init --campaign-id perovskite-opt --goal "maximize bandgap" --budget-cap 50000.0 +nilvar init --campaign-id perovskite-opt --goal "maximize bandgap" --budget-cap 50000.0 # Ingest events from JSON/CSV/YAML -matledger ingest --file data.json +nilvar ingest --file data.json # Record a validated result -matledger record-result --candidate-id CsPbI3 --value 1.68 --uncertainty 0.05 --iteration 1 +nilvar record-result --candidate-id CsPbI3 --value 1.68 --uncertainty 0.05 --iteration 1 # Record a failure -matledger record-failure --candidate-id CsPbBr3 --failure-class synthesizability --iteration 1 --detail "Precursor unavailable" +nilvar record-failure --candidate-id CsPbBr3 --failure-class synthesizability --iteration 1 --detail "Precursor unavailable" ``` **Read commands:** ```bash # Campaign report (markdown, HTML, or JSON) -matledger report -matledger report --format html --output report.html -matledger report --format json +nilvar report +nilvar report --format html --output report.html +nilvar report --format json # Verify hash chain integrity (shows event breakdown, actors, time range) -matledger audit -matledger audit --output json +nilvar audit +nilvar audit --output json # Export portable evidence bundle for auditors -matledger export --output bundle.json +nilvar export --output bundle.json # Counterfactual replay -matledger counterfactual --policy greedy --top-k 1 -matledger counterfactual --policy random --seed 42 +nilvar counterfactual --policy greedy --top-k 1 +nilvar counterfactual --policy random --seed 42 ```
SDK Reference ```python -from matledger.sdk import CampaignClient +from nilvar.sdk import CampaignClient client = CampaignClient("campaign.db") ``` @@ -425,7 +425,7 @@ The API runs via Docker Compose (see [Docker](#docker) below). For local develop ```bash pip install -e ".[api]" -uvicorn matledger.api.app:app --reload +uvicorn nilvar.api.app:app --reload ``` Interactive docs at `http://localhost:8000/docs` (Swagger UI). @@ -501,20 +501,20 @@ curl http://localhost:8000/health | Variable | Default | Description | |----------|---------|-------------| -| `MATLEDGER_DB` | `/data/matledger.db` | SQLite database path inside the container | -| `MATLEDGER_LOG_LEVEL` | `INFO` | Python logging level | -| `MATLEDGER_CORS_ORIGINS` | `[]` | JSON array of allowed CORS origins | +| `NILVAR_DB` | `/data/nilvar.db` | SQLite database path inside the container | +| `NILVAR_LOG_LEVEL` | `INFO` | Python logging level | +| `NILVAR_CORS_ORIGINS` | `[]` | JSON array of allowed CORS origins | -Set variables in `.env` (loaded automatically by Compose) or inline: `MATLEDGER_LOG_LEVEL=DEBUG docker compose up`. See [`.env.example`](.env.example) for the full reference. +Set variables in `.env` (loaded automatically by Compose) or inline: `NILVAR_LOG_LEVEL=DEBUG docker compose up`. See [`.env.example`](.env.example) for the full reference. -**Data persistence:** The SQLite database is stored in a Docker named volume (`matledger-data`), mounted at `/data` inside the container. Data survives container restarts and image rebuilds. +**Data persistence:** The SQLite database is stored in a Docker named volume (`nilvar-data`), mounted at `/data` inside the container. Data survives container restarts and image rebuilds. ```bash # Back up the database -docker compose cp matledger-api:/data/matledger.db ./backup.db +docker compose cp nilvar-api:/data/nilvar.db ./backup.db ``` -**Production:** For deployments behind a reverse proxy, configure rate limiting and TLS at the proxy layer (nginx, Caddy, Cloudflare). The matledger API is stateless HTTP. +**Production:** For deployments behind a reverse proxy, configure rate limiting and TLS at the proxy layer (nginx, Caddy, Cloudflare). The nilvar API is stateless HTTP.
@@ -522,7 +522,7 @@ docker compose cp matledger-api:/data/matledger.db ./backup.db ## F. Literature -matledger's trajectory analytics are grounded in the Bayesian optimization, self-driving lab, and autonomous experimentation literature. +nilvar's trajectory analytics are grounded in the Bayesian optimization, self-driving lab, and autonomous experimentation literature. ### Foundational @@ -573,7 +573,7 @@ Benchmarked on a 1,000-event in-memory SQLite store (100 iterations, 10 candidat | Operation | Throughput | Latency | |-----------|-----------|---------| -| Cold import (`from matledger`) | | ~310 ms | +| Cold import (`from nilvar`) | | ~310 ms | | Event writes (hash-chained appends) | ~175 events/sec | ~6 ms/event | | Full replay (4 projectors from seq 0) | ~55,000 events/sec | ~18 ms / 1k events | | Report generation (markdown) | | < 1 ms | diff --git a/docker-compose.yml b/docker-compose.yml index 23812eb..5c84f8a 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,16 +1,16 @@ services: - matledger-api: + nilvar-api: build: . ports: - "8000:8000" volumes: - - matledger-data:/data + - nilvar-data:/data env_file: - path: .env required: false environment: - MATLEDGER_DB: /data/matledger.db - MATLEDGER_LOG_LEVEL: INFO + NILVAR_DB: /data/nilvar.db + NILVAR_LOG_LEVEL: INFO healthcheck: test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"] interval: 30s @@ -19,4 +19,4 @@ services: start_period: 10s volumes: - matledger-data: + nilvar-data: diff --git a/docs/DESIGN_DOC.md b/docs/DESIGN_DOC.md index dd7a9a7..b97405f 100644 --- a/docs/DESIGN_DOC.md +++ b/docs/DESIGN_DOC.md @@ -1,6 +1,6 @@ -# matledger — Design Doc (RFC) +# nilvar — Design Doc (RFC) -**Status:** v0.9.0 — Implemented · **Owner:** Jayson · **Companion:** `matledger — PRD` (the *what/who/why*). This doc covers the *how*, and argues the consequential, contestable decisions with their tradeoffs. +**Status:** v0.9.0 — Implemented · **Owner:** Jayson · **Companion:** `nilvar — PRD` (the *what/who/why*). This doc covers the *how*, and argues the consequential, contestable decisions with their tradeoffs. **Reading guide:** this doc is deliberately weighted. The plumbing (event store, API, CLI) is short because it is well-understood and low-risk. The trajectory analytics (§6) and the event/failure model (§5) are long because they are the hard, hard-to-reverse, value-bearing decisions. *Write longest where least certain.* @@ -8,11 +8,11 @@ ## 1. Context & the one decision everything hangs on -matledger records an AI-driven discovery campaign — the loop **propose → test → learn → decide** — and mines it for discovery-health signals (proxy-gaming, diversity collapse) that experiment trackers structurally cannot produce. +nilvar records an AI-driven discovery campaign — the loop **propose → test → learn → decide** — and mines it for discovery-health signals (proxy-gaming, diversity collapse) that experiment trackers structurally cannot produce. The defining outputs are **functions of the trajectory**, not properties of current state. "Is the surrogate decoupling from validated reality?" is only visible across a *sequence* of iterations; it cannot be read from a snapshot. This single fact drives the entire architecture: the system of record must be the *sequence of events*, not the *current state*. Everything below follows from that. -The test that justifies the architecture, stated once: **does answering the core question require replaying a sequence, or just looking up a value?** For matledger it is the former → event sourcing. (For MLflow's "what was run 47's F1?" it is the latter → CRUD. Same reason, opposite answer.) +The test that justifies the architecture, stated once: **does answering the core question require replaying a sequence, or just looking up a value?** For nilvar it is the former → event sourcing. (For MLflow's "what was run 47's F1?" it is the latter → CRUD. Same reason, opposite answer.) --- @@ -60,24 +60,24 @@ The **write** pattern (append one validated fact) and the **read** patterns (fol - **MLflow/W&B + a custom script.** Rejected: trackers model independent runs, not a causal loop; no failure/decision-rationale primitives; no replay. - **Graph DB.** Rejected for v0: the causal links are real but the dominant access pattern is *sequential fold*, which a log serves more simply; revisit only if cross-candidate graph queries dominate. -#### 3.3.2 Domain incumbents — and the exact line matledger draws +#### 3.3.2 Domain incumbents — and the exact line nilvar draws -The discovery-tooling space is **not empty**; it is densely populated by orchestrators, schedulers, and event fabrics. matledger's novelty is therefore *not* a mechanism nobody could build — every component below exists — but the **assembly** none of them provides: a retained, immutable, replayable campaign-health ledger. The competitive map, by layer: +The discovery-tooling space is **not empty**; it is densely populated by orchestrators, schedulers, and event fabrics. nilvar's novelty is therefore *not* a mechanism nobody could build — every component below exists — but the **assembly** none of them provides: a retained, immutable, replayable campaign-health ledger. The competitive map, by layer: -| Layer | Incumbent(s) | Does it do matledger's job? | +| Layer | Incumbent(s) | Does it do nilvar's job? | |---|---|---| | Event **transport** | OCTOPUS / *Globus event fabric* (Kafka-as-a-service, ~4–9M events/s) | **No** — a message bus that *transports* events; it does not retain them as a replayable system-of-record | | Job **scheduling** | OCTOPUS / *Yoo et al.* (task-optimization scheduler), AlabOS (resource reservation) | **No** — parallelizes and reserves; does not audit whether the *search* is healthy | | **Orchestration + storage** | ChemOS 2.0, AlabOS, UniLabOS | Stores data to *run* the loop (current-state SQL); not an immutable causal log to *audit* it | | Discovery **analyses** | ChemOS (computes explore/exploit parameter-distance distributions), the optimizers themselves | Computed **post-hoc, per-run, as paper figures** — not standing, replayable, cross-campaign projections | -| **Replayable campaign-health ledger** | *— none —* | **This is matledger.** | +| **Replayable campaign-health ledger** | *— none —* | **This is nilvar.** | Two clarifications that pre-empt the obvious reviewer objections: -- **"Isn't OCTOPUS already event-driven — isn't this the same thing?"** No. There are two distinct tools named OCTOPUS: a Globus **event fabric** (literally managed Kafka — *messaging/transport*) and a Yoo et al. **scheduler** (*job parallelization*). Neither is event sourcing as a *system of record*. Notably, the Globus fabric's events already carry instrument, timestamp, experiment id, action, and results — strikingly close to matledger's envelope — but the fabric *forwards and discards*; matledger *retains the ordered log and folds analytics over the sequence*. That retention-and-replay is the entire difference, and it is the line to state in one breath. -- **"Doesn't ChemOS already measure diversity?"** Yes — ChemOS interprets parameter-distance distributions as the explore/exploit signature of the acquisition function. But it does so as a **one-off plot over a single run**, not as a standing projection rebuildable across campaigns. The analysis is known; the *replayable, cross-campaign, governance-grade instrumentation of it* is not. matledger's contribution is the instrument, not the metric. +- **"Isn't OCTOPUS already event-driven — isn't this the same thing?"** No. There are two distinct tools named OCTOPUS: a Globus **event fabric** (literally managed Kafka — *messaging/transport*) and a Yoo et al. **scheduler** (*job parallelization*). Neither is event sourcing as a *system of record*. Notably, the Globus fabric's events already carry instrument, timestamp, experiment id, action, and results — strikingly close to nilvar's envelope — but the fabric *forwards and discards*; nilvar *retains the ordered log and folds analytics over the sequence*. That retention-and-replay is the entire difference, and it is the line to state in one breath. +- **"Doesn't ChemOS already measure diversity?"** Yes — ChemOS interprets parameter-distance distributions as the explore/exploit signature of the acquisition function. But it does so as a **one-off plot over a single run**, not as a standing projection rebuildable across campaigns. The analysis is known; the *replayable, cross-campaign, governance-grade instrumentation of it* is not. nilvar's contribution is the instrument, not the metric. -**Integration, not competition.** Because the Globus event fabric carries exactly the events matledger wants, matledger can *consume an OCTOPUS stream as a source* and become the durable, analytical **sink** on the end of it — a cleaner position than "another orchestrator." matledger does not replace the orchestration/transport stack; it is the campaign-health layer none of that stack provides. +**Integration, not competition.** Because the Globus event fabric carries exactly the events nilvar wants, nilvar can *consume an OCTOPUS stream as a source* and become the durable, analytical **sink** on the end of it — a cleaner position than "another orchestrator." nilvar does not replace the orchestration/transport stack; it is the campaign-health layer none of that stack provides. --- @@ -165,7 +165,7 @@ These are the only parts that answer the question CRUD trackers can't. Each: sig Flag proxy-gaming when `gap(i)` **widens monotonically** over a window of *W* iterations while `val_best(i)` is flat or declining. The divergence *trajectory* is the signal, not any single iteration. -**Verdict:** `{verdict: GAMING|OK, window, gap_trend, val_trend, since_iteration, evidence_cids}` — evidence-bearing, like every matledger verdict. +**Verdict:** `{verdict: GAMING|OK, window, gap_trend, val_trend, since_iteration, evidence_cids}` — evidence-bearing, like every nilvar verdict. **Open problems (the part that proves you see the hard parts):** - *Window size W.* Too short → false alarms on noise; too long → slow detection. Likely campaign-dependent; expose as config with a defensible default, and validate against synthetic fixtures. @@ -288,7 +288,7 @@ Periodically persists a projection's materialized state at `seq = N`; replay the ### 7.3 Pluggable policy engine ✅ Implemented -A `Policy` protocol that reads projections (untested space, budget left, failure ledger, diversity state) and proposes what to select. Three built-in policies: Greedy, Random, AvoidFailures. The `--avoid-known-failures` policy reads straight from the failure ledger to exclude candidates matching prior failure classes. matledger *hosts* the interface; it does not author the policy. +A `Policy` protocol that reads projections (untested space, budget left, failure ledger, diversity state) and proposes what to select. Three built-in policies: Greedy, Random, AvoidFailures. The `--avoid-known-failures` policy reads straight from the failure ledger to exclude candidates matching prior failure classes. nilvar *hosts* the interface; it does not author the policy. --- diff --git a/docs/PRD.md b/docs/PRD.md index 733acb0..1e14b56 100644 --- a/docs/PRD.md +++ b/docs/PRD.md @@ -1,8 +1,8 @@ -# matledger — Product Requirements Document (PRD) +# nilvar — Product Requirements Document (PRD) **Status:** v0.9.0 — Implemented (M0-M9 complete) **Owner:** Jayson -**Companion doc:** `matledger — Design Doc (RFC)` (the *how*: event-sourcing/CQRS spine, event model, analytics internals). This PRD covers the *what / who / why / success*. +**Companion doc:** `nilvar — Design Doc (RFC)` (the *how*: event-sourcing/CQRS spine, event model, analytics internals). This PRD covers the *what / who / why / success*. **One-line:** An event-sourced campaign ledger for AI-driven scientific discovery that answers the question experiment trackers structurally cannot — *"are we actually discovering useful candidates, or gaming a weak proxy?"* --- @@ -40,7 +40,7 @@ Existing tools record the wrong thing. Experiment trackers (MLflow, Weights & Bi ## 3. Target Users & Personas -| Persona | Role in the loop | What they need from matledger | +| Persona | Role in the loop | What they need from nilvar | |---|---|---| | **AI optimizer / agent** | Proposes candidates, asks "what next?" | A clean ingest + `suggest-next` API; programmatic, idempotent. | | **Research scientist** | Reviews, overrides, interprets | A per-candidate timeline, decision rationale, failure ledger; the "why." | @@ -158,7 +158,7 @@ Priority: **M** = Must (v0), **S** = Should (v0.1), **C** = Could (v1). Status: |---|---|---|---| | D1 | `suggest-next` reads projections (untested space, budget, failures) and returns candidates to test. | S | ⊘ Superseded by D4 | | D2 | `--avoid-known-failures` excludes regions/candidates matching the failure ledger. | S | ✅ AvoidFailuresPolicy | -| D3 | Pluggable active-learning policy interface (matledger hosts, does not implement, the policy). | C | ✅ Policy protocol + GreedyPolicy, RandomPolicy | +| D3 | Pluggable active-learning policy interface (nilvar hosts, does not implement, the policy). | C | ✅ Policy protocol + GreedyPolicy, RandomPolicy | | D4 | **Counterfactual replay**: re-run an alternate decision policy over recorded history; report what it would have found. | C | ✅ | ### 6.4 I/O & reporting @@ -182,7 +182,7 @@ Priority: **M** = Must (v0), **S** = Should (v0.1), **C** = Could (v1). Status: - **Consistency model:** write side strongly consistent (append); read side eventually consistent (projections) — documented, not hidden. - **Portability:** local-first (SQLite, single binary/process) with zero external infra for OSS; Postgres + object store for hosted. - **Security (commercial):** SSO/RBAC, tenant isolation, audit export. -- **No hardware dependency:** fully runnable on a laptop; results are ingested, never produced by matledger. +- **No hardware dependency:** fully runnable on a laptop; results are ingested, never produced by nilvar. --- diff --git a/docs/adr/001-event-sourcing.md b/docs/adr/001-event-sourcing.md index e3832c1..5de335e 100644 --- a/docs/adr/001-event-sourcing.md +++ b/docs/adr/001-event-sourcing.md @@ -4,7 +4,7 @@ Accepted ## Context -matledger records the propose-test-learn-decide loop for AI-driven scientific +nilvar records the propose-test-learn-decide loop for AI-driven scientific discovery campaigns. The core requirement is an auditable, immutable record that supports retroactive analysis: a new analytics question (e.g., "was the model proxy-gaming?") must be answerable over historical data without schema diff --git a/docs/adr/002-sqlite-event-store.md b/docs/adr/002-sqlite-event-store.md index d83c6af..fe789e6 100644 --- a/docs/adr/002-sqlite-event-store.md +++ b/docs/adr/002-sqlite-event-store.md @@ -5,7 +5,7 @@ Accepted ## Context The event store needs to support append-only writes with hash chaining, -concurrent read/write access, and zero external infrastructure. matledger +concurrent read/write access, and zero external infrastructure. nilvar targets single-machine scientific workflows where campaigns produce thousands to low hundreds of thousands of events. @@ -28,7 +28,7 @@ database via `file:...?mode=memory&cache=shared` URI. ## Consequences ### Benefits -- Zero infrastructure: `pip install matledger` is all you need +- Zero infrastructure: `pip install nilvar` is all you need - WAL mode gives concurrent read/write without locking readers - SQLite is battle-tested and available everywhere Python runs - Content-hash unique index provides O(1) deduplication checks diff --git a/docs/adr/004-compliance-envelope-extensions.md b/docs/adr/004-compliance-envelope-extensions.md index eda63a3..92b65eb 100644 --- a/docs/adr/004-compliance-envelope-extensions.md +++ b/docs/adr/004-compliance-envelope-extensions.md @@ -6,7 +6,7 @@ Accepted ## Context -External review against ALCOA+, 21 CFR Part 11, and FAIR data principles identified gaps in the event envelope and payload schemas. matledger's event-sourcing architecture provides a strong audit-trail foundation, but several fields needed for scientific data integrity are missing. +External review against ALCOA+, 21 CFR Part 11, and FAIR data principles identified gaps in the event envelope and payload schemas. nilvar's event-sourcing architecture provides a strong audit-trail foundation, but several fields needed for scientific data integrity are missing. The event envelope (frozen in M0 as the "hardest to reverse" decision) needs careful extension. Changes here affect the hash chain, serialization, and every consumer of events. diff --git a/docs/design_walkthrough.md b/docs/design_walkthrough.md index 2111707..456ea1d 100644 --- a/docs/design_walkthrough.md +++ b/docs/design_walkthrough.md @@ -1,6 +1,6 @@ -# matledger — Design Walkthrough +# nilvar — Design Walkthrough -A narrative walkthrough of the architectural decisions behind matledger: why each exists, what it costs, and what breaks if you remove it. Written for engineers interested in the system's design tradeoffs, not its feature count. +A narrative walkthrough of the architectural decisions behind nilvar: why each exists, what it costs, and what breaks if you remove it. Written for engineers interested in the system's design tradeoffs, not its feature count. --- @@ -12,7 +12,7 @@ The critical insight: these failure modes are **properties of the trajectory**, This single observation drives the entire architecture. If the defining questions require replaying a sequence, the system of record must *be* the sequence. Event sourcing isn't a stylistic choice here — it's a structural requirement. -The litmus test I used throughout: **does answering this question require replaying a sequence, or just looking up a value?** For matledger, it's the former. For MLflow's "what was run 47's F1?" it's the latter. Same reasoning, opposite answers. The architecture is justified by what it unlocks, not by the pattern's reputation. +The litmus test I used throughout: **does answering this question require replaying a sequence, or just looking up a value?** For nilvar, it's the former. For MLflow's "what was run 47's F1?" it's the latter. Same reasoning, opposite answers. The architecture is justified by what it unlocks, not by the pattern's reputation. --- @@ -53,7 +53,7 @@ This means a change to the analytics layer cannot accidentally break the write p Every event's hash covers its seq, type, payload, and the prior event's hash — the same structure as a Merkle chain in certificate transparency logs (RFC 6962). If someone edits event #5, every subsequent event's `prev_hash` no longer matches. -**Why:** tamper-evidence, not tamper-prevention. A malicious actor could rewrite the entire chain. The goal is catching accidental data corruption and auditing that no events were silently modified post-recording. `matledger audit` walks the chain and reports exactly where it breaks. +**Why:** tamper-evidence, not tamper-prevention. A malicious actor could rewrite the entire chain. The goal is catching accidental data corruption and auditing that no events were silently modified post-recording. `nilvar audit` walks the chain and reports exactly where it breaks. **Content-hash deduplication** is the companion mechanism: SHA-256 of (type + payload), ignoring seq and timestamp. Two events with identical type and payload produce the same content-hash, enabling idempotent ingest. If you accidentally ingest the same CSV twice, duplicates are rejected rather than polluting the log. @@ -71,7 +71,7 @@ This field is the one people forget and the one that matters most. It's a DAG (d ## Decision 5: The failure taxonomy -Most systems treat failures as a boolean. matledger classifies them into six categories, each revealing something different about where the surrogate model is blind: +Most systems treat failures as a boolean. nilvar classifies them into six categories, each revealing something different about where the surrogate model is blind: | Class | What it tells you about the proxy | |-------|----------------------------------| @@ -139,7 +139,7 @@ Pruning (`keep_latest=N`) prevents snapshot accumulation. The default keeps the ## What I deliberately did not build -- **Surrogate model integration.** matledger records what the surrogate predicted and what actually happened. It does not run the surrogate. Adding adapter code for BoTorch or Ax would be integration plumbing, not architectural judgment. The interface is the event vocabulary — any system that can emit `CandidateProposed` events with `predicted_value` and `model_version` is compatible. +- **Surrogate model integration.** nilvar records what the surrogate predicted and what actually happened. It does not run the surrogate. Adding adapter code for BoTorch or Ax would be integration plumbing, not architectural judgment. The interface is the event vocabulary — any system that can emit `CandidateProposed` events with `predicted_value` and `model_version` is compatible. - **Multi-campaign queries.** v0 is single-campaign. Cross-campaign analytics (e.g., "which surrogate architecture games the proxy least across all our campaigns?") would require a campaign registry and cross-log projections. The event model supports it — campaigns are identified by `campaign_id` — but the read infrastructure isn't built because the single-campaign case needs to work first. @@ -159,7 +159,7 @@ Pruning (`keep_latest=N`) prevents snapshot accumulation. The default keeps the ## Literature grounding -matledger is a campaign health monitor — it detects when the discovery loop is going wrong. The trajectory analytics it implements (and the eight more planned in M9) are grounded in peer-reviewed work from the Bayesian optimization, self-driving lab, and autonomous experimentation communities. This is the short list of papers that validate the existence of this project — that these failure modes are real, measurable, and worth detecting. +nilvar is a campaign health monitor — it detects when the discovery loop is going wrong. The trajectory analytics it implements (and the eight more planned in M9) are grounded in peer-reviewed work from the Bayesian optimization, self-driving lab, and autonomous experimentation communities. This is the short list of papers that validate the existence of this project — that these failure modes are real, measurable, and worth detecting. **Foundational BO theory:** - Srinivas et al. 2010, ["Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design"](https://doi.org/10.48550/arXiv.0912.3995) (ICML) — regret bounds and the explore-exploit framework that underpins cumulative regret tracking diff --git a/docs/gamp/classification.md b/docs/gamp/classification.md index 0388ff0..a35bce4 100644 --- a/docs/gamp/classification.md +++ b/docs/gamp/classification.md @@ -1,17 +1,17 @@ -# GAMP 5 System Classification -- matledger +# GAMP 5 System Classification -- nilvar ## Software Category **Category 5: Custom Application** -matledger is a bespoke event-sourced campaign ledger built for AI-driven scientific discovery. It is not a commercial off-the-shelf product (Category 3/4). All source code is custom-written and maintained by the development team. +nilvar is a bespoke event-sourced campaign ledger built for AI-driven scientific discovery. It is not a commercial off-the-shelf product (Category 3/4). All source code is custom-written and maintained by the development team. ## System Description | Attribute | Value | |-----------|-------| -| System name | matledger | -| Version | See `src/matledger/__init__.py` | +| System name | nilvar | +| Version | See `src/nilvar/__init__.py` | | Purpose | Record and audit propose/test/learn/decide loops in AI-driven discovery campaigns | | Technology | Python 3.11+, SQLite, Pydantic v2 | | Deployment | Local CLI tool, Docker Compose for API surface | @@ -22,7 +22,7 @@ matledger is a bespoke event-sourced campaign ledger built for AI-driven scienti | Risk | Severity | Likelihood | Mitigation | |------|----------|------------|------------| -| Data integrity loss (event tampering) | High | Low | SHA-256 hash chain with audit verification (`matledger audit`) | +| Data integrity loss (event tampering) | High | Low | SHA-256 hash chain with audit verification (`nilvar audit`) | | Data loss (database corruption) | High | Low | WAL mode, Docker volume persistence, backup guidance | | Incorrect measurement recording | High | Medium | Pydantic schema validation, unit/uncertainty fields, calibration references | | Incomplete experiment tracking | Medium | Medium | `incomplete_experiments()` advisory check | @@ -44,7 +44,7 @@ Per GAMP 5 risk-based approach, Category 5 software requires: | Role | Responsibility | |------|---------------| -| **matledger maintainers** | Provide validation pack (this directory), maintain test suite, document changes | +| **nilvar maintainers** | Provide validation pack (this directory), maintain test suite, document changes | | **Adopting organization** | Execute IQ/OQ/PQ in their environment, maintain SOPs, conduct periodic reviews, own change control | -matledger provides the validation documentation and test evidence. The adopting organization owns execution, site-specific risk assessment, and ongoing compliance. +nilvar provides the validation documentation and test evidence. The adopting organization owns execution, site-specific risk assessment, and ongoing compliance. diff --git a/docs/gamp/iq_protocol.md b/docs/gamp/iq_protocol.md index 66a4f64..ce3f12a 100644 --- a/docs/gamp/iq_protocol.md +++ b/docs/gamp/iq_protocol.md @@ -1,12 +1,12 @@ -# Installation Qualification (IQ) Protocol -- matledger +# Installation Qualification (IQ) Protocol -- nilvar ## Purpose -Verify that matledger is installed correctly in the target environment with all dependencies resolved and the system ready for operational testing. +Verify that nilvar is installed correctly in the target environment with all dependencies resolved and the system ready for operational testing. ## Scope -This protocol covers installation of matledger via pip (development/CLI use) and Docker Compose (API deployment). The adopting organization executes this protocol in their specific environment. +This protocol covers installation of nilvar via pip (development/CLI use) and Docker Compose (API deployment). The adopting organization executes this protocol in their specific environment. ## Prerequisites @@ -27,24 +27,24 @@ This protocol covers installation of matledger via pip (development/CLI use) and | Step | Action | Expected Result | Actual Result | Pass/Fail | |------|--------|-----------------|---------------|-----------| -| 1 | Run `git clone ` | Repository cloned | Repository present at C:\Users\Lenovo\matledger | PASS | -| 2 | Run `pip install -e ".[dev]"` | Installation completes without errors | matledger 0.1.0 installed (editable) | PASS | -| 3 | Run `python -c "import matledger; print(matledger.__version__)"` | Version string printed | `0.1.0` | PASS | +| 1 | Run `git clone ` | Repository cloned | Repository present at C:\Users\Lenovo\nilvar | PASS | +| 2 | Run `pip install -e ".[dev]"` | Installation completes without errors | nilvar 0.1.0 installed (editable) | PASS | +| 3 | Run `python -c "import nilvar; print(nilvar.__version__)"` | Version string printed | `0.1.0` | PASS | ### IQ-003: CLI Installation Verification | Step | Action | Expected Result | Actual Result | Pass/Fail | |------|--------|-----------------|---------------|-----------| | 1 | Run `pip install -e ".[cli]"` | Installation completes | Included in dev extras | PASS | -| 2 | Run `matledger --version` | Version string printed | `matledger 0.1.0` | PASS | -| 3 | Run `matledger --help` | Help text displayed with all commands listed | 14 commands listed (init, ingest, record-result, record-failure, propose, start-experiment, select, reject, model-update, close-iteration, consume-budget, report, audit, counterfactual) | PASS | +| 2 | Run `nilvar --version` | Version string printed | `nilvar 0.1.0` | PASS | +| 3 | Run `nilvar --help` | Help text displayed with all commands listed | 14 commands listed (init, ingest, record-result, record-failure, propose, start-experiment, select, reject, model-update, close-iteration, consume-budget, report, audit, counterfactual) | PASS | ### IQ-004: Docker Image Build | Step | Action | Expected Result | Actual Result | Pass/Fail | |------|--------|-----------------|---------------|-----------| -| 1 | Run `docker build -t matledger:iq .` | Image builds without errors | N/A -- Docker not available in this environment; validated in CI (docker job in ci.yml) | DEFERRED | -| 2 | Run `docker images matledger:iq` | Image listed with size and creation time | N/A -- see above | DEFERRED | +| 1 | Run `docker build -t nilvar:iq .` | Image builds without errors | N/A -- Docker not available in this environment; validated in CI (docker job in ci.yml) | DEFERRED | +| 2 | Run `docker images nilvar:iq` | Image listed with size and creation time | N/A -- see above | DEFERRED | ### IQ-005: Docker Compose Deployment diff --git a/docs/gamp/oq_protocol.md b/docs/gamp/oq_protocol.md index 68a05eb..0151361 100644 --- a/docs/gamp/oq_protocol.md +++ b/docs/gamp/oq_protocol.md @@ -1,8 +1,8 @@ -# Operational Qualification (OQ) Protocol -- matledger +# Operational Qualification (OQ) Protocol -- nilvar ## Purpose -Verify that matledger operates correctly under normal conditions, producing expected results for all core functions. OQ validates functional requirements against the design specification. +Verify that nilvar operates correctly under normal conditions, producing expected results for all core functions. OQ validates functional requirements against the design specification. ## Scope @@ -11,7 +11,7 @@ This protocol covers the core event-sourcing pipeline: event creation, storage, ## Prerequisites - IQ protocol completed and signed off -- matledger installed with dev extras: `pip install -e ".[dev]"` +- nilvar installed with dev extras: `pip install -e ".[dev]"` ## Test Execution @@ -21,16 +21,16 @@ The primary OQ evidence is the automated test suite. All tests must pass. | Step | Action | Expected Result | Actual Result | Pass/Fail | |------|--------|-----------------|---------------|-----------| -| 1 | Run `pytest --cov=matledger --cov-report=term-missing -q` | All tests pass, coverage >= 80% | 313 passed in 21.93s | PASS | +| 1 | Run `pytest --cov=nilvar --cov-report=term-missing -q` | All tests pass, coverage >= 80% | 313 passed in 21.93s | PASS | | 2 | Record test count and coverage percentage | Document in results | 313 tests, 93.56% coverage (threshold: 80%) | PASS | ### OQ-002: Hash Chain Integrity | Step | Action | Expected Result | Actual Result | Pass/Fail | |------|--------|-----------------|---------------|-----------| -| 1 | Run `matledger init oq-001 --goal "OQ test" --budget-cap 10000 --db oq_test.db` | Campaign created | Campaign created: oq-001 (seq=0) | PASS | +| 1 | Run `nilvar init oq-001 --goal "OQ test" --budget-cap 10000 --db oq_test.db` | Campaign created | Campaign created: oq-001 (seq=0) | PASS | | 2 | Record 49 events via Python API (12 proposals, 10 experiments, 6 results, 4 failures, 2 selections, 1 rejection, 3 budget consumptions, 3 iteration closes, 1 additional proposal+experiment+result+selection+close) | Events appended | 49 events appended across 3 iterations | PASS | -| 3 | Run `matledger audit --db oq_test.db` | "Hash chain verified" with zero breaks | "Hash chain intact (49 events verified)", 0 breaks | PASS | +| 3 | Run `nilvar audit --db oq_test.db` | "Hash chain verified" with zero breaks | "Hash chain intact (49 events verified)", 0 breaks | PASS | ### OQ-003: Lifecycle Enforcement @@ -52,8 +52,8 @@ The primary OQ evidence is the automated test suite. All tests must pass. | Step | Action | Expected Result | Actual Result | Pass/Fail | |------|--------|-----------------|---------------|-----------| | 1 | Record a multi-iteration campaign (3+ iterations) | Events stored correctly | 49 events across 3 iterations, 13 candidates | PASS | -| 2 | Run `matledger report --db oq_test.db` | Markdown report with timeline, budget, failures | 1,515-char report with Summary, Candidates (13), Failure Ledger, Budget by Iteration, Iteration Summary, Campaign Health | PASS | -| 3 | Run `matledger report --db oq_test.db --format html` | HTML report with trajectory charts | 7,735-char HTML report generated | PASS | +| 2 | Run `nilvar report --db oq_test.db` | Markdown report with timeline, budget, failures | 1,515-char report with Summary, Candidates (13), Failure Ledger, Budget by Iteration, Iteration Summary, Campaign Health | PASS | +| 3 | Run `nilvar report --db oq_test.db --format html` | HTML report with trajectory charts | 7,735-char HTML report generated | PASS | | 4 | Verify proxy-gaming analytic returns a verdict | Verdict is "OK" or "GAMING" with evidence | Verdict: OK (Window=0, no divergence) | PASS | | 5 | Verify diversity analytic returns a verdict | Verdict is "OK" or "COLLAPSING" with evidence | Verdict: OK (Window=0, spread stable) | PASS | diff --git a/docs/gamp/pq_protocol.md b/docs/gamp/pq_protocol.md index 15bd982..d8bbe79 100644 --- a/docs/gamp/pq_protocol.md +++ b/docs/gamp/pq_protocol.md @@ -1,8 +1,8 @@ -# Performance Qualification (PQ) Protocol -- matledger +# Performance Qualification (PQ) Protocol -- nilvar ## Purpose -Verify that matledger performs correctly under conditions representative of actual use, with real-world data volumes and workflows. PQ validates that the system meets user requirements in the production environment. +Verify that nilvar performs correctly under conditions representative of actual use, with real-world data volumes and workflows. PQ validates that the system meets user requirements in the production environment. ## Scope @@ -11,7 +11,7 @@ This protocol covers end-to-end campaign workflows: campaign creation through an ## Prerequisites - IQ and OQ protocols completed and signed off -- matledger installed in the target production environment +- nilvar installed in the target production environment - Representative test data prepared (campaign with 3+ iterations, 10+ candidates) ## Test Cases @@ -27,7 +27,7 @@ Execute a complete campaign lifecycle and verify all outputs. | 3 | Start experiments and record results/failures | All outcomes linked to experiments | 10 experiments started iter 0 (6 results, 4 failures), 2 experiments iter 1 (2 results), 1 experiment iter 2 (1 result with raw_data_ref) | PASS | | 4 | Select and reject candidates with rationale | Decisions recorded with causal links | cand-001 selected ("Best bandgap"), cand-002 rejected ("Too expensive"), cand-012 selected ("Optimal bandgap") | PASS | | 5 | Close iterations with summaries | Iteration boundaries recorded | 3 iterations closed with summaries: "Initial screening complete", "Second iteration", "Final iteration" | PASS | -| 6 | Run `matledger audit` | Hash chain verified, zero breaks | "Hash chain intact (49 events verified)", 0 breaks | PASS | +| 6 | Run `nilvar audit` | Hash chain verified, zero breaks | "Hash chain intact (49 events verified)", 0 breaks | PASS | | 7 | Generate markdown report | Report includes all sections | 1,515-char report with Summary, Candidates, Failure Ledger, Budget, Iteration Summary, Campaign Health | PASS | | 8 | Generate HTML report | Report includes trajectory charts | 7,735-char HTML report generated with embedded analytics | PASS | | 9 | Verify all analytics produce verdicts | proxy-gaming, diversity, discovery-rate return results | Proxy-gaming: OK, Diversity: OK, Discovery rate: cost_per_result=$144.44, not flatlined | PASS | @@ -38,7 +38,7 @@ Execute a complete campaign lifecycle and verify all outputs. | Step | Action | Expected Result | Actual Result | Pass/Fail | |------|--------|-----------------|---------------|-----------| | 1 | Prepare a JSON file with 15 candidates | File created | pq_batch_test.json created with 15 candidates (flat feature keys) | PASS | -| 2 | Run `matledger ingest --file ` | All events ingested | "Ingested 15 candidates from pq_batch_test.json" | PASS | +| 2 | Run `nilvar ingest --file ` | All events ingested | "Ingested 15 candidates from pq_batch_test.json" | PASS | | 3 | Run ingest again with same file | All duplicates rejected (idempotent) | "Skipped duplicate events (idempotent ingest)" | PASS | | 4 | Verify hash chain integrity | Zero breaks | "Hash chain intact (16 events verified)" | PASS | @@ -63,7 +63,7 @@ Execute a complete campaign lifecycle and verify all outputs. | Step | Action | Expected Result | Actual Result | Pass/Fail | |------|--------|-----------------|---------------|-----------| -| 1 | Back up database: `docker compose cp matledger-api:/data/matledger.db ./backup.db` | Backup created | N/A -- Docker not available; SQLite file-copy backup is a documented standard operation | DEFERRED | +| 1 | Back up database: `docker compose cp nilvar-api:/data/nilvar.db ./backup.db` | Backup created | N/A -- Docker not available; SQLite file-copy backup is a documented standard operation | DEFERRED | | 2 | Add more events after backup | Events appended | N/A | DEFERRED | | 3 | Restore backup to new container | Backup loads, events up to backup point present | N/A | DEFERRED | | 4 | Verify hash chain on restored database | Zero breaks | N/A | DEFERRED | diff --git a/docs/gamp/traceability_matrix.md b/docs/gamp/traceability_matrix.md index e014d06..17ec686 100644 --- a/docs/gamp/traceability_matrix.md +++ b/docs/gamp/traceability_matrix.md @@ -1,9 +1,9 @@ -# GAMP 5 Traceability Matrix -- matledger +# GAMP 5 Traceability Matrix -- nilvar **Document ID:** TM-001 **Version:** 1.0 **Date:** 2026-07-01 -**System:** matledger -- Event-sourced campaign ledger for AI-driven scientific discovery +**System:** nilvar -- Event-sourced campaign ledger for AI-driven scientific discovery **Classification:** GAMP 5 Category 5 (Custom Application) --- @@ -16,8 +16,8 @@ This traceability matrix maps User Requirements (from the PRD) through Functiona | Doc ID | Title | Location | |--------|-------|----------| -| PRD | matledger Product Requirements Document | `docs/PRD.md` | -| DD | matledger Design Doc (RFC) | `docs/DESIGN_DOC.md` | +| PRD | nilvar Product Requirements Document | `docs/PRD.md` | +| DD | nilvar Design Doc (RFC) | `docs/DESIGN_DOC.md` | ## Legend @@ -31,13 +31,13 @@ This traceability matrix maps User Requirements (from the PRD) through Functiona | Req ID | Requirement (from PRD) | Design Ref | Implementation | Test Evidence | Status | |--------|------------------------|------------|----------------|---------------|--------| -| C2-ENV | Event envelope carries seq, type, payload, prev_hash, iteration, caused_by, ts, actor | DD SS 5.1 | `src/matledger/events/envelope.py` | `tests/test_events/test_envelope.py` | Verified | -| C2-CT | Content hash for idempotent dedup (SHA-256 of type + payload + iteration) | DD SS 5.1 | `src/matledger/events/envelope.py` | `tests/test_events/test_envelope.py` | Verified | -| C2-ET | Event type catalog: CampaignCreated, CandidateProposed, ExperimentStarted, ResultRecorded, FailureRecorded, CandidateSelected, CandidateRejected, BudgetConsumed, ModelUpdated, IterationClosed | DD SS 5.2 | `src/matledger/events/catalog.py` | `tests/test_events/test_catalog.py` | Verified | -| C3 | Results capture value + uncertainty + calibration_ref, not a bare number | DD SS 5.2 (ResultRecorded) | `src/matledger/events/catalog.py` (ResultRecorded) | `tests/test_events/test_catalog.py` | Verified | -| C4 | Failures captured under typed failure taxonomy (synthesizability, stability, out_of_spec, non_novel, measurement_error, process_error) | DD SS 5.3 | `src/matledger/events/failures.py` | `tests/test_events/test_failures.py` | Verified | -| C5 | Selection/rejection events capture rationale + model_version | DD SS 5.2 (CandidateSelected/Rejected) | `src/matledger/events/catalog.py` (CandidateSelected, CandidateRejected) | `tests/test_events/test_catalog.py` | Verified | -| C2-PV | CandidateProposed carries predicted_value + model_version for proxy-gaming detection | DD SS 5.2 | `src/matledger/events/catalog.py` (CandidateProposed) | `tests/test_events/test_catalog.py` | Verified | +| C2-ENV | Event envelope carries seq, type, payload, prev_hash, iteration, caused_by, ts, actor | DD SS 5.1 | `src/nilvar/events/envelope.py` | `tests/test_events/test_envelope.py` | Verified | +| C2-CT | Content hash for idempotent dedup (SHA-256 of type + payload + iteration) | DD SS 5.1 | `src/nilvar/events/envelope.py` | `tests/test_events/test_envelope.py` | Verified | +| C2-ET | Event type catalog: CampaignCreated, CandidateProposed, ExperimentStarted, ResultRecorded, FailureRecorded, CandidateSelected, CandidateRejected, BudgetConsumed, ModelUpdated, IterationClosed | DD SS 5.2 | `src/nilvar/events/catalog.py` | `tests/test_events/test_catalog.py` | Verified | +| C3 | Results capture value + uncertainty + calibration_ref, not a bare number | DD SS 5.2 (ResultRecorded) | `src/nilvar/events/catalog.py` (ResultRecorded) | `tests/test_events/test_catalog.py` | Verified | +| C4 | Failures captured under typed failure taxonomy (synthesizability, stability, out_of_spec, non_novel, measurement_error, process_error) | DD SS 5.3 | `src/nilvar/events/failures.py` | `tests/test_events/test_failures.py` | Verified | +| C5 | Selection/rejection events capture rationale + model_version | DD SS 5.2 (CandidateSelected/Rejected) | `src/nilvar/events/catalog.py` (CandidateSelected, CandidateRejected) | `tests/test_events/test_catalog.py` | Verified | +| C2-PV | CandidateProposed carries predicted_value + model_version for proxy-gaming detection | DD SS 5.2 | `src/nilvar/events/catalog.py` (CandidateProposed) | `tests/test_events/test_catalog.py` | Verified | --- @@ -45,11 +45,11 @@ This traceability matrix maps User Requirements (from the PRD) through Functiona | Req ID | Requirement (from PRD) | Design Ref | Implementation | Test Evidence | Status | |--------|------------------------|------------|----------------|---------------|--------| -| C1 | Append events to an immutable, ordered, hash-chained log; events are never mutated or deleted | DD SS 3, SS 4 | `src/matledger/store/sqlite.py` | `tests/test_store/test_sqlite.py` | Verified | -| C1-HC | Hash chain: each event's prev_hash is SHA-256 of prior event; tamper-evident | DD SS 5.1 | `src/matledger/store/hash_chain.py` | `tests/test_store/test_hash_chain.py` | Verified | -| C7 | audit verify validates hash chain and reports any break at row N | DD SS 10 | `src/matledger/store/hash_chain.py` (verify_chain) | `tests/test_store/test_hash_chain.py` | Verified | -| C1-WAL | SQLite event store with WAL mode for concurrent reads | DD SS 4, SS 8 | `src/matledger/store/sqlite.py` | `tests/test_store/test_concurrency.py` | Verified | -| C1-PB | Property-based invariants (hash chains, serialization round-trip, ordering) | DD SS 3.1 | `src/matledger/store/sqlite.py`, `src/matledger/events/envelope.py` | `tests/test_store/test_property_based.py` | Verified | +| C1 | Append events to an immutable, ordered, hash-chained log; events are never mutated or deleted | DD SS 3, SS 4 | `src/nilvar/store/sqlite.py` | `tests/test_store/test_sqlite.py` | Verified | +| C1-HC | Hash chain: each event's prev_hash is SHA-256 of prior event; tamper-evident | DD SS 5.1 | `src/nilvar/store/hash_chain.py` | `tests/test_store/test_hash_chain.py` | Verified | +| C7 | audit verify validates hash chain and reports any break at row N | DD SS 10 | `src/nilvar/store/hash_chain.py` (verify_chain) | `tests/test_store/test_hash_chain.py` | Verified | +| C1-WAL | SQLite event store with WAL mode for concurrent reads | DD SS 4, SS 8 | `src/nilvar/store/sqlite.py` | `tests/test_store/test_concurrency.py` | Verified | +| C1-PB | Property-based invariants (hash chains, serialization round-trip, ordering) | DD SS 3.1 | `src/nilvar/store/sqlite.py`, `src/nilvar/events/envelope.py` | `tests/test_store/test_property_based.py` | Verified | --- @@ -57,10 +57,10 @@ This traceability matrix maps User Requirements (from the PRD) through Functiona | Req ID | Requirement (from PRD) | Design Ref | Implementation | Test Evidence | Status | |--------|------------------------|------------|----------------|---------------|--------| -| C6 | Ingestion is idempotent (content-hash dedupe); optimizer/LIMS retries don't pollute the log | DD SS 4 | `src/matledger/commands/handler.py` | `tests/test_commands/test_handler.py` | Verified | -| C2-CB | caused_by causal links assigned by command handler | DD SS 5.1 | `src/matledger/commands/handler.py` | `tests/test_commands/test_handler.py` | Verified | -| C1-LV | Lifecycle validation: commands validated before append (type checks, field constraints) | DD SS 4 | `src/matledger/commands/handler.py` | `tests/test_commands/test_handler.py` | Verified | -| I1 | Import candidates/results from CSV/JSON/YAML | DD SS 4 (API/CLI) | `src/matledger/commands/ingest.py` | `tests/test_commands/test_ingest.py` | Verified | +| C6 | Ingestion is idempotent (content-hash dedupe); optimizer/LIMS retries don't pollute the log | DD SS 4 | `src/nilvar/commands/handler.py` | `tests/test_commands/test_handler.py` | Verified | +| C2-CB | caused_by causal links assigned by command handler | DD SS 5.1 | `src/nilvar/commands/handler.py` | `tests/test_commands/test_handler.py` | Verified | +| C1-LV | Lifecycle validation: commands validated before append (type checks, field constraints) | DD SS 4 | `src/nilvar/commands/handler.py` | `tests/test_commands/test_handler.py` | Verified | +| I1 | Import candidates/results from CSV/JSON/YAML | DD SS 4 (API/CLI) | `src/nilvar/commands/ingest.py` | `tests/test_commands/test_ingest.py` | Verified | --- @@ -68,12 +68,12 @@ This traceability matrix maps User Requirements (from the PRD) through Functiona | Req ID | Requirement (from PRD) | Design Ref | Implementation | Test Evidence | Status | |--------|------------------------|------------|----------------|---------------|--------| -| A7 | Any projection is rebuildable from seq 0; new analytics run retroactively | DD SS 3.1, SS 4 | `src/matledger/projections/framework.py` | `tests/test_projections/test_framework.py` | Verified | -| A1 | Per-candidate timeline projection (full lifecycle of any candidate) | DD SS 4 (read models) | `src/matledger/projections/timeline.py` | `tests/test_projections/test_timeline.py` | Verified | -| A2 | Failure-ledger projection (negledger) queryable by class/iteration/campaign | DD SS 4 (read models), SS 5.3 | `src/matledger/projections/negledger.py` | `tests/test_projections/test_negledger.py` | Verified | -| A3 | Budget projection: cumulative cost vs cap | DD SS 6.3 | `src/matledger/projections/budget.py` | `tests/test_projections/test_budget.py` | Verified | -| A7-IS | Iteration summary projection: per-iteration aggregate statistics | DD SS 5.2 (IterationClosed) | `src/matledger/projections/iteration_summary.py` | `tests/test_projections/test_iteration_summary.py` | Verified | -| A7-SN | Snapshotting: persist projection state at seq N for replay-at-scale | DD SS 7.2 | `src/matledger/projections/snapshots.py` | `tests/test_projections/test_snapshots.py` | Verified | +| A7 | Any projection is rebuildable from seq 0; new analytics run retroactively | DD SS 3.1, SS 4 | `src/nilvar/projections/framework.py` | `tests/test_projections/test_framework.py` | Verified | +| A1 | Per-candidate timeline projection (full lifecycle of any candidate) | DD SS 4 (read models) | `src/nilvar/projections/timeline.py` | `tests/test_projections/test_timeline.py` | Verified | +| A2 | Failure-ledger projection (negledger) queryable by class/iteration/campaign | DD SS 4 (read models), SS 5.3 | `src/nilvar/projections/negledger.py` | `tests/test_projections/test_negledger.py` | Verified | +| A3 | Budget projection: cumulative cost vs cap | DD SS 6.3 | `src/nilvar/projections/budget.py` | `tests/test_projections/test_budget.py` | Verified | +| A7-IS | Iteration summary projection: per-iteration aggregate statistics | DD SS 5.2 (IterationClosed) | `src/nilvar/projections/iteration_summary.py` | `tests/test_projections/test_iteration_summary.py` | Verified | +| A7-SN | Snapshotting: persist projection state at seq N for replay-at-scale | DD SS 7.2 | `src/nilvar/projections/snapshots.py` | `tests/test_projections/test_snapshots.py` | Verified | --- @@ -81,12 +81,12 @@ This traceability matrix maps User Requirements (from the PRD) through Functiona | Req ID | Requirement (from PRD) | Design Ref | Implementation | Test Evidence | Status | |--------|------------------------|------------|----------------|---------------|--------| -| A4 | Proxy-gaming detector: flags sustained divergence between predicted and validated values | DD SS 6.1 | `src/matledger/analytics/proxy_gaming.py` | `tests/test_analytics/test_proxy_gaming.py` | Verified | -| A5 | Diversity monitor: per-iteration spread metric over candidate features; flags sustained contraction | DD SS 6.2 | `src/matledger/analytics/diversity.py` | `tests/test_analytics/test_diversity.py` | Verified | -| A5-FAISS | Diversity monitor with FAISS ANN for high-dimensional spread computation | DD SS 6.2 | `src/matledger/analytics/diversity.py` | `tests/test_analytics/test_diversity_faiss.py` | Verified | -| A6 | Discovery-rate analytics: cost-per-validated-hit, marginal discovery rate, failure-class trends | DD SS 6.3 | `src/matledger/analytics/discovery_rate.py` | `tests/test_analytics/test_discovery_rate.py` | Verified | -| D4 | Counterfactual replay: re-run alternate decision policy over recorded history | DD SS 7.1 | `src/matledger/decisions/counterfactual.py` | `tests/test_decisions/test_counterfactual.py` | Verified | -| D3 | Pluggable policy interface (greedy, random baselines) | DD SS 7.3 | `src/matledger/decisions/policy.py` | `tests/test_decisions/test_policy.py` | Verified | +| A4 | Proxy-gaming detector: flags sustained divergence between predicted and validated values | DD SS 6.1 | `src/nilvar/analytics/proxy_gaming.py` | `tests/test_analytics/test_proxy_gaming.py` | Verified | +| A5 | Diversity monitor: per-iteration spread metric over candidate features; flags sustained contraction | DD SS 6.2 | `src/nilvar/analytics/diversity.py` | `tests/test_analytics/test_diversity.py` | Verified | +| A5-FAISS | Diversity monitor with FAISS ANN for high-dimensional spread computation | DD SS 6.2 | `src/nilvar/analytics/diversity.py` | `tests/test_analytics/test_diversity_faiss.py` | Verified | +| A6 | Discovery-rate analytics: cost-per-validated-hit, marginal discovery rate, failure-class trends | DD SS 6.3 | `src/nilvar/analytics/discovery_rate.py` | `tests/test_analytics/test_discovery_rate.py` | Verified | +| D4 | Counterfactual replay: re-run alternate decision policy over recorded history | DD SS 7.1 | `src/nilvar/decisions/counterfactual.py` | `tests/test_decisions/test_counterfactual.py` | Verified | +| D3 | Pluggable policy interface (greedy, random baselines) | DD SS 7.3 | `src/nilvar/decisions/policy.py` | `tests/test_decisions/test_policy.py` | Verified | --- @@ -94,12 +94,12 @@ This traceability matrix maps User Requirements (from the PRD) through Functiona | Req ID | Requirement (from PRD) | Design Ref | Implementation | Test Evidence | Status | |--------|------------------------|------------|----------------|---------------|--------| -| I3-CLI | CLI wraps same API surface: init, ingest, propose, record-result, record-failure, select, reject, model-update, close-iteration, consume-budget, report, audit, counterfactual | DD SS 4, SS 8 | `src/matledger/cli/main.py` | `tests/test_cli/test_main.py` | Verified | -| I3-INIT | `init` command creates a new campaign | DD SS 4 | `src/matledger/cli/main.py` (init) | `tests/test_cli/test_main.py` | Verified | -| I3-ING | `ingest` command imports from CSV/JSON/YAML files | DD SS 4 | `src/matledger/cli/main.py` (ingest) | `tests/test_cli/test_main.py` | Verified | -| I3-RPT | `report` command generates markdown/HTML/JSON reports with analytics verdicts | DD SS 4 | `src/matledger/cli/main.py` (report) | `tests/test_cli/test_main.py` | Verified | -| I3-AUD | `audit` command verifies hash chain integrity | DD SS 10 | `src/matledger/cli/main.py` (audit_verify) | `tests/test_cli/test_main.py` | Verified | -| I3-CF | `counterfactual` command runs alternate-policy replay | DD SS 7.1 | `src/matledger/cli/main.py` (counterfactual) | `tests/test_cli/test_main.py` | Verified | +| I3-CLI | CLI wraps same API surface: init, ingest, propose, record-result, record-failure, select, reject, model-update, close-iteration, consume-budget, report, audit, counterfactual | DD SS 4, SS 8 | `src/nilvar/cli/main.py` | `tests/test_cli/test_main.py` | Verified | +| I3-INIT | `init` command creates a new campaign | DD SS 4 | `src/nilvar/cli/main.py` (init) | `tests/test_cli/test_main.py` | Verified | +| I3-ING | `ingest` command imports from CSV/JSON/YAML files | DD SS 4 | `src/nilvar/cli/main.py` (ingest) | `tests/test_cli/test_main.py` | Verified | +| I3-RPT | `report` command generates markdown/HTML/JSON reports with analytics verdicts | DD SS 4 | `src/nilvar/cli/main.py` (report) | `tests/test_cli/test_main.py` | Verified | +| I3-AUD | `audit` command verifies hash chain integrity | DD SS 10 | `src/nilvar/cli/main.py` (audit_verify) | `tests/test_cli/test_main.py` | Verified | +| I3-CF | `counterfactual` command runs alternate-policy replay | DD SS 7.1 | `src/nilvar/cli/main.py` (counterfactual) | `tests/test_cli/test_main.py` | Verified | --- @@ -107,11 +107,11 @@ This traceability matrix maps User Requirements (from the PRD) through Functiona | Req ID | Requirement (from PRD) | Design Ref | Implementation | Test Evidence | Status | |--------|------------------------|------------|----------------|---------------|--------| -| I3-API | REST API: one surface for agent, script, and human; FastAPI | DD SS 4, SS 8 | `src/matledger/api/app.py` | `tests/test_api/test_app.py` | Verified | -| I3-HLT | Health check endpoint (GET /health) | DD SS 4 | `src/matledger/api/app.py` (/health) | `tests/test_api/test_app.py` | Verified | -| I3-WR | Write endpoints: POST /campaigns, /candidates/propose, /experiments/start, /results, /failures, /candidates/select, /candidates/reject, /model/update, /iterations/close | DD SS 4 | `src/matledger/api/app.py` | `tests/test_api/test_app.py` | Verified | -| I3-RD | Read endpoints: GET /timeline, /negledger, /budget, /iterations, /analytics/proxy-gaming, /analytics/diversity, /analytics/discovery-rate, /analytics/counterfactual, /report, /audit | DD SS 4 | `src/matledger/api/app.py` | `tests/test_api/test_app.py` | Verified | -| I3-ERR | Error handling: MatledgerError (400), DuplicateEventError (409), ValidationError (422), ValueError (400) | DD SS 4 | `src/matledger/api/app.py` | `tests/test_api/test_app.py` | Verified | +| I3-API | REST API: one surface for agent, script, and human; FastAPI | DD SS 4, SS 8 | `src/nilvar/api/app.py` | `tests/test_api/test_app.py` | Verified | +| I3-HLT | Health check endpoint (GET /health) | DD SS 4 | `src/nilvar/api/app.py` (/health) | `tests/test_api/test_app.py` | Verified | +| I3-WR | Write endpoints: POST /campaigns, /candidates/propose, /experiments/start, /results, /failures, /candidates/select, /candidates/reject, /model/update, /iterations/close | DD SS 4 | `src/nilvar/api/app.py` | `tests/test_api/test_app.py` | Verified | +| I3-RD | Read endpoints: GET /timeline, /negledger, /budget, /iterations, /analytics/proxy-gaming, /analytics/diversity, /analytics/discovery-rate, /analytics/counterfactual, /report, /audit | DD SS 4 | `src/nilvar/api/app.py` | `tests/test_api/test_app.py` | Verified | +| I3-ERR | Error handling: NilvarError (400), DuplicateEventError (409), ValidationError (422), ValueError (400) | DD SS 4 | `src/nilvar/api/app.py` | `tests/test_api/test_app.py` | Verified | --- @@ -119,11 +119,11 @@ This traceability matrix maps User Requirements (from the PRD) through Functiona | Req ID | Requirement (from PRD) | Design Ref | Implementation | Test Evidence | Status | |--------|------------------------|------------|----------------|---------------|--------| -| I3-SDK | CampaignClient: high-level Python client wrapping same API surface | DD SS 4, SS 8 | `src/matledger/sdk/client.py` | `tests/test_sdk/test_client.py` | Verified | -| I3-SDK-W | Write operations: create_campaign, propose, start_experiment, record_result, record_failure, select, reject, update_model, close_iteration | DD SS 4 | `src/matledger/sdk/client.py` | `tests/test_sdk/test_client.py` | Verified | -| I3-SDK-R | Read operations: timeline, negledger, budget, iterations, report | DD SS 4 | `src/matledger/sdk/client.py` | `tests/test_sdk/test_client.py` | Verified | -| I3-SDK-A | Analytics operations: proxy_gaming, diversity, discovery_rate, counterfactual_replay | DD SS 6, SS 7.1 | `src/matledger/sdk/client.py` | `tests/test_sdk/test_client.py` | Verified | -| I3-SDK-V | Audit: verify() validates hash chain via SDK | DD SS 10 | `src/matledger/sdk/client.py` | `tests/test_sdk/test_client.py` | Verified | +| I3-SDK | CampaignClient: high-level Python client wrapping same API surface | DD SS 4, SS 8 | `src/nilvar/sdk/client.py` | `tests/test_sdk/test_client.py` | Verified | +| I3-SDK-W | Write operations: create_campaign, propose, start_experiment, record_result, record_failure, select, reject, update_model, close_iteration | DD SS 4 | `src/nilvar/sdk/client.py` | `tests/test_sdk/test_client.py` | Verified | +| I3-SDK-R | Read operations: timeline, negledger, budget, iterations, report | DD SS 4 | `src/nilvar/sdk/client.py` | `tests/test_sdk/test_client.py` | Verified | +| I3-SDK-A | Analytics operations: proxy_gaming, diversity, discovery_rate, counterfactual_replay | DD SS 6, SS 7.1 | `src/nilvar/sdk/client.py` | `tests/test_sdk/test_client.py` | Verified | +| I3-SDK-V | Audit: verify() validates hash chain via SDK | DD SS 10 | `src/nilvar/sdk/client.py` | `tests/test_sdk/test_client.py` | Verified | --- @@ -131,13 +131,13 @@ This traceability matrix maps User Requirements (from the PRD) through Functiona | Req ID | Requirement (from PRD) | Design Ref | Implementation | Test Evidence | Status | |--------|------------------------|------------|----------------|---------------|--------| -| ALCOA-A | Attributable: every event carries `actor` field (agent, scientist, LIMS) | DD SS 5.1 | `src/matledger/events/envelope.py` (actor field) | `tests/test_compliance/test_alcoa.py` | Verified | -| ALCOA-L | Legible: typed event catalog with Pydantic validation; payload schemas enforce structure | DD SS 5.2 | `src/matledger/events/catalog.py` | `tests/test_compliance/test_alcoa.py` | Verified | -| ALCOA-C | Contemporaneous: `ts` (observation time) and `recorded_at` (server-side append time) are distinct timestamps | DD SS 5.1 | `src/matledger/events/envelope.py` (ts, recorded_at) | `tests/test_compliance/test_alcoa.py` | Verified | -| ALCOA-O | Original: `raw_data_ref` and `raw_data_checksum` on ResultRecorded trace back to raw instrument data | DD SS 5.2 | `src/matledger/events/catalog.py` (ResultRecorded) | `tests/test_compliance/test_alcoa.py` | Verified | -| ALCOA-AC | Accurate: `unit` and `calibration_ref` on ResultRecorded provide measurement traceability | DD SS 5.2 | `src/matledger/events/catalog.py` (ResultRecorded) | `tests/test_compliance/test_alcoa.py` | Verified | -| ALCOA-E | Enduring: append-only, hash-chained event log; events are never mutated or deleted | DD SS 3, SS 5.1 | `src/matledger/store/sqlite.py`, `src/matledger/store/hash_chain.py` | `tests/test_compliance/test_alcoa.py` | Verified | -| ALCOA-AV | Available: projections are rebuildable from seq 0; data never lost | DD SS 3.1 | `src/matledger/projections/framework.py` | `tests/test_compliance/test_alcoa.py` | Verified | +| ALCOA-A | Attributable: every event carries `actor` field (agent, scientist, LIMS) | DD SS 5.1 | `src/nilvar/events/envelope.py` (actor field) | `tests/test_compliance/test_alcoa.py` | Verified | +| ALCOA-L | Legible: typed event catalog with Pydantic validation; payload schemas enforce structure | DD SS 5.2 | `src/nilvar/events/catalog.py` | `tests/test_compliance/test_alcoa.py` | Verified | +| ALCOA-C | Contemporaneous: `ts` (observation time) and `recorded_at` (server-side append time) are distinct timestamps | DD SS 5.1 | `src/nilvar/events/envelope.py` (ts, recorded_at) | `tests/test_compliance/test_alcoa.py` | Verified | +| ALCOA-O | Original: `raw_data_ref` and `raw_data_checksum` on ResultRecorded trace back to raw instrument data | DD SS 5.2 | `src/nilvar/events/catalog.py` (ResultRecorded) | `tests/test_compliance/test_alcoa.py` | Verified | +| ALCOA-AC | Accurate: `unit` and `calibration_ref` on ResultRecorded provide measurement traceability | DD SS 5.2 | `src/nilvar/events/catalog.py` (ResultRecorded) | `tests/test_compliance/test_alcoa.py` | Verified | +| ALCOA-E | Enduring: append-only, hash-chained event log; events are never mutated or deleted | DD SS 3, SS 5.1 | `src/nilvar/store/sqlite.py`, `src/nilvar/store/hash_chain.py` | `tests/test_compliance/test_alcoa.py` | Verified | +| ALCOA-AV | Available: projections are rebuildable from seq 0; data never lost | DD SS 3.1 | `src/nilvar/projections/framework.py` | `tests/test_compliance/test_alcoa.py` | Verified | --- @@ -145,9 +145,9 @@ This traceability matrix maps User Requirements (from the PRD) through Functiona | Req ID | Requirement (from PRD) | Design Ref | Implementation | Test Evidence | Status | |--------|------------------------|------------|----------------|---------------|--------| -| I2-MD | Markdown campaign report: timeline, failures, budget, analytics verdicts | DD SS 4 | `src/matledger/report/markdown.py` | `tests/test_cli/test_main.py` | Verified | -| I2-HTML | HTML campaign report with embedded analytics verdicts and chart images | DD SS 4 | `src/matledger/report/html.py` | `tests/test_report/test_html.py` | Verified | -| I2-PLOT | Trajectory charts: proxy-gaming, diversity, budget burn, discovery rate (matplotlib) | DD SS 6 | `src/matledger/report/plots.py` | `tests/test_report/test_plots.py` | Verified | +| I2-MD | Markdown campaign report: timeline, failures, budget, analytics verdicts | DD SS 4 | `src/nilvar/report/markdown.py` | `tests/test_cli/test_main.py` | Verified | +| I2-HTML | HTML campaign report with embedded analytics verdicts and chart images | DD SS 4 | `src/nilvar/report/html.py` | `tests/test_report/test_html.py` | Verified | +| I2-PLOT | Trajectory charts: proxy-gaming, diversity, budget burn, discovery rate (matplotlib) | DD SS 6 | `src/nilvar/report/plots.py` | `tests/test_report/test_plots.py` | Verified | --- diff --git a/examples/basic_campaign.py b/examples/basic_campaign.py index 233ccaf..becb762 100644 --- a/examples/basic_campaign.py +++ b/examples/basic_campaign.py @@ -4,7 +4,7 @@ python examples/basic_campaign.py """ -from matledger.sdk import CampaignClient +from nilvar.sdk import CampaignClient client = CampaignClient(":memory:") diff --git a/fixtures/README.md b/fixtures/README.md index 45014c6..edac20c 100644 --- a/fixtures/README.md +++ b/fixtures/README.md @@ -1,6 +1,6 @@ # Synthetic Known-Answer Fixtures -These fixtures validate that matledger's analytics produce correct verdicts +These fixtures validate that nilvar's analytics produce correct verdicts against campaigns where the answer is known by construction. This is the "validate the ruler before you measure" discipline (Design Doc §10). diff --git a/fixtures/smoke_test/README.md b/fixtures/smoke_test/README.md index 5aaa956..9df8900 100644 --- a/fixtures/smoke_test/README.md +++ b/fixtures/smoke_test/README.md @@ -1,6 +1,6 @@ # Smoke Test: Perovskite Bandgap Optimisation Campaign -End-to-end smoke test that exercises every matledger CLI command against a realistic +End-to-end smoke test that exercises every nilvar CLI command against a realistic materials science campaign. Run it to verify the full system works before release. ```bash @@ -152,7 +152,7 @@ For a visual walkthrough of how proxy gaming happens, what corrections exist, and how counterfactual replay applies to this campaign, see the [Analytics Guide](analytics_guide.html). It includes diagrams of the feedback loop, the divergence chart, correction strategies, and product direction for -matledger's evolution from diagnostic ledger to advisory system. +nilvar's evolution from diagnostic ledger to advisory system. ### References diff --git a/fixtures/smoke_test/analytics_guide.html b/fixtures/smoke_test/analytics_guide.html index d786766..3a43b2f 100644 --- a/fixtures/smoke_test/analytics_guide.html +++ b/fixtures/smoke_test/analytics_guide.html @@ -3,7 +3,7 @@ -matledger Analytics Guide — Proxy Gaming, Corrections & Counterfactual Replay +nilvar Analytics Guide — Proxy Gaming, Corrections & Counterfactual Replay