Develop

.gitignore

@@ -26,6 +26,9 @@ htmlcov/
 
 # matledger data
 *.db
+smoke_report.*
+smoke_bundle.json
+smoke_test.db
 *.sqlite
 *.sqlite3
 *.parquet

README.md

@@ -128,6 +128,8 @@ print(client.report())
 See [`examples/basic_campaign.py`](examples/basic_campaign.py) for a complete 3-iteration campaign, or [`examples/notebook_demo.ipynb`](examples/notebook_demo.ipynb) for an interactive Jupyter walkthrough.
 You can also [run the notebook directly in Google Colab](https://colab.research.google.com/github/jaysonmulwa/matledger/blob/main/examples/notebook_demo.ipynb) — no local install required.
 
+For a realistic 5-iteration perovskite campaign that exercises every CLI command (CSV ingestion, failures, counterfactuals, reports), see the [smoke test fixtures](fixtures/smoke_test/README.md). The accompanying [analytics guide](fixtures/smoke_test/analytics_guide.html) explains how proxy gaming happens, what corrections exist, and how counterfactual replay quantifies the cost.
+
 </details>
 
 ---
@@ -603,6 +605,7 @@ All available targets:
 | `make coverage` | Run tests with coverage report (93%, 80% gate) |
 | `make audit` | Audit dependencies for vulnerabilities |
 | `make clean` | Remove build artifacts |
+| `bash fixtures/smoke_test/run_smoke_test.sh` | Run end-to-end smoke test ([details](fixtures/smoke_test/README.md)) |
 
 See [CONTRIBUTING.md](CONTRIBUTING.md) for the full development workflow, architecture constraints, and commit format.
 

fixtures/smoke_test/README.md

@@ -0,0 +1,162 @@
+# Smoke Test: Perovskite Bandgap Optimisation Campaign
+
+End-to-end smoke test that exercises every matledger CLI command against a realistic
+materials science campaign. Run it to verify the full system works before release.
+
+```bash
+bash fixtures/smoke_test/run_smoke_test.sh
+```
+
+Produces: `smoke_test.db`, `smoke_report.{md,html,json}`, `smoke_bundle.json`
+
+---
+
+## What is being optimised?
+
+**Perovskite solar cell bandgap.** Perovskites are a family of crystalline materials
+used in next-generation solar cells. Their optical bandgap (measured in electron-volts,
+eV) determines how efficiently they convert sunlight to electricity. The ideal bandgap
+for a single-junction solar cell is around 1.1-1.5 eV, but for tandem cells stacked on
+top of silicon, a wider bandgap of 1.7-2.0 eV is desirable.
+
+**Campaign goal:** find a perovskite composition with bandgap > 1.8 eV and thermal
+stability above 300 degrees Celsius, within a $100k budget.
+
+**Was the goal achieved?** Partially. The best validated result was **1.98 eV**
+(pvk-034, iteration 5), exceeding the 1.8 eV target. However, the campaign shows
+clear signs of **proxy gaming** (surrogate model predictions inflating to 2.0-2.2 eV
+while validated results plateau around 1.93-1.98) and **diversity collapse** (all
+late-campaign candidates cluster in the high-Br region). A real PI would pause the
+campaign and retrain the surrogate with a corrected acquisition function.
+
+---
+
+## Campaign structure
+
+The campaign runs 5 iterations of a Bayesian optimisation loop:
+
+1. **Propose** -- a Gaussian process surrogate model (GP) predicts which compositions
+   are worth testing, ranked by predicted bandgap.
+2. **Test** -- a robotic lab (LIMS) synthesises each composition and starts experiments.
+3. **Learn** -- an XRD autoloader measures the bandgap of successfully synthesised films.
+   Some experiments fail (synthesis, stability, equipment errors).
+4. **Decide** -- the PI (dr-kimani) selects the best candidates and rejects poor ones.
+5. **Repeat** -- the surrogate model retrains on new data and proposes the next batch.
+
+### Actors
+
+| Actor | Role | Real-world analogue |
+|-------|------|---------------------|
+| `dr-kimani` | Principal investigator; creates campaign, selects/rejects, closes iterations | Lab PI or team lead |
+| `surrogate-gp` | Gaussian process surrogate model; proposes candidates and retrains | BoTorch, Ax, or GPyTorch model |
+| `lims` | Laboratory information management system; starts experiments | Automated scheduling system |
+| `lab-tech-ochieng` | Lab technician; records failures with physical detail | Bench scientist |
+| `xrd-autoloader` | X-ray diffraction instrument; records validated measurements | Bruker D8 or similar diffractometer |
+
+---
+
+## CSV column reference
+
+### Candidate files (`iter*_candidates.csv`)
+
+These are the compositions the surrogate model proposes for synthesis and testing.
+
+| Column | Type | Unit | Description |
+|--------|------|------|-------------|
+| `candidate_id` | string | -- | Unique identifier (pvk-NNN) |
+| `predicted_value` | float | eV | Surrogate model's predicted bandgap at proposal time |
+| `Cs` | float | molar fraction | Caesium content in the A-site of the perovskite lattice |
+| `Pb` | float | molar fraction | Lead content in the B-site |
+| `I` | float | molar fraction | Iodide content in the X-site |
+| `Br` | float | molar fraction | Bromide content in the X-site |
+| `Sn` | float | molar fraction | Tin content (partial B-site substitution for lead) |
+
+The general perovskite formula is ABX3 where A=Cs, B=Pb/Sn, X=I/Br. Higher bromide
+content generally increases bandgap but reduces stability -- the fundamental tradeoff
+the campaign explores.
+
+### Result files (`iter*_results.csv`)
+
+These are the validated experimental measurements from XRD characterisation.
+
+| Column | Type | Unit | Description |
+|--------|------|------|-------------|
+| `candidate_id` | string | -- | Which candidate was measured |
+| `value` | float | eV | Measured optical bandgap |
+| `uncertainty` | float | eV | Measurement uncertainty (1-sigma standard deviation) |
+| `calibration_ref` | string | -- | Calibration standard used (e.g. XRD-CAL-2024-07) |
+
+Not every candidate appears in the results file. Missing candidates failed during
+synthesis, stability testing, or due to equipment errors -- those are recorded as
+`FailureRecorded` events via the CLI.
+
+### Failure classes used
+
+| Class | Count | Meaning |
+|-------|-------|---------|
+| `synthesizability` | 7 | Composition did not form the target crystal phase |
+| `stability` | 4 | Film decomposed during accelerated aging (85C/85% relative humidity) |
+| `process_error` | 3 | Equipment malfunction (furnace, glovebox); not the candidate's fault |
+| `measurement_error` | 1 | XRD sample holder misaligned; measurement discarded |
+
+---
+
+## Iteration-by-iteration narrative
+
+| Iter | Proposed | Tested | Failed | Best result | Surrogate predictions | Signal |
+|------|----------|--------|--------|-------------|----------------------|--------|
+| 1 | 8 | 5 | 3 | 1.85 eV (pvk-003) | 1.55 - 1.91 eV | Initial exploration; high failure rate (37%) |
+| 2 | 8 | 5 | 3 | 1.89 eV (pvk-015) | 1.67 - 2.01 eV | Model starts exploiting Br-enrichment |
+| 3 | 8 | 5 | 3 | 1.95 eV (pvk-022) | 1.85 - 2.10 eV | Predictions inflating; Br fraction rising |
+| 4 | 8 | 5 | 3 | 1.97 eV (pvk-030) | 1.93 - 2.18 eV | Prediction-reality gap widening |
+| 5 | 8 | 5 | 3 | 1.98 eV (pvk-034) | 2.00 - 2.22 eV | Proxy gaming confirmed; diversity collapsed |
+
+**Totals:** 40 candidates, 25 validated results, 15 failures, $100k spent, 142 events.
+
+---
+
+## How realistic is this data?
+
+This fixture is calibrated against published self-driving lab (SDL) campaigns. The table
+below compares our fixture parameters against real campaigns from the literature.
+
+| Parameter | This fixture | Literature range | Reference |
+|-----------|-------------|-----------------|-----------|
+| Iterations | 5 | 5-50 typical | Perovskite BO (Nature 2026): 5 rounds of 33 total experiments |
+| Batch size | 8 per iteration | 5-8 (wet lab), up to 96 (high-throughput screening) | Christensen et al. (Commun. Chem. 2021): 8 parallel reaction loops |
+| Total candidates | 40 | 30-200 typical | A-Lab, Berkeley (Nature 2023): 58 targets over 17 days |
+| Failure rate | 37% (15/40) | 30-50% is normal | A-Lab: 29% target failure, 70% recipe failure; MF-BO drug discovery (ACS 2025): 34-51% synthesis failure |
+| Cost per experiment | $2,500 | $500-$5,000 | Typical materials synthesis + XRD characterisation |
+| Total budget | $100,000 | $25k-$250k | Mid-range academic lab campaign |
+| Actors | 5 | 3-10 | Realistic lab team (PI, tech, instruments, model) |
+
+### What we simplified
+
+- **Uniform batch size.** Real campaigns often vary batch size across iterations (larger
+  early batches for exploration, smaller late batches for exploitation). We use a fixed
+  8 for simplicity.
+- **Uniform cost.** Real experiment costs vary by composition and characterisation method.
+  We use a flat $2,500 per experiment.
+- **Fixed failure rate.** Real failure rates often increase as the model pushes into
+  unexplored regions of composition space. Our fixture has a constant 3/8 failure rate
+  per iteration, though a real campaign would likely see failure rates climb in later
+  iterations.
+- **Single objective.** Real perovskite campaigns often optimise multiple objectives
+  simultaneously (bandgap, stability, defect density). We optimise bandgap only.
+- **No replicate measurements.** Real labs run 2-3 replicates per candidate to assess
+  reproducibility. We record one measurement per candidate.
+
+### Understanding the analytics
+
+For a visual walkthrough of how proxy gaming happens, what corrections exist,
+and how counterfactual replay applies to this campaign, see the
+[Analytics Guide](analytics_guide.html). It includes diagrams of the feedback
+loop, the divergence chart, correction strategies, and product direction for
+matledger's evolution from diagnostic ledger to advisory system.
+
+### References
+
+- Szymanski et al. 2023, "An autonomous laboratory for the accelerated synthesis of novel materials" (Nature 624). A-Lab at Berkeley: 353 synthesis attempts on 58 targets, 71% success rate.
+- Autonomous closed-loop framework for reproducible perovskite solar cells (Nature 2026). 33 experiments across 5 AL rounds, batch size 5-8.
+- Christensen et al. 2021, "Data-science driven autonomous process optimization" (Commun. Chem.). 120-192 experiments, 8 parallel reaction loops.
+- Bayesian optimization over multiple experimental fidelities (ACS Central Science 2025). Drug discovery: 3,500+ docked, 120+ synthesised, 34-51% synthesis failure rate.