Skip to content

Add --curated-dir to soma reproduce to reuse a pre-curated manifest#274

Merged
clemsgrs merged 1 commit into
mainfrom
add-reproduce-curated-dir
Jul 9, 2026
Merged

Add --curated-dir to soma reproduce to reuse a pre-curated manifest#274
clemsgrs merged 1 commit into
mainfrom
add-reproduce-curated-dir

Conversation

@clemsgrs

@clemsgrs clemsgrs commented Jul 9, 2026

Copy link
Copy Markdown
Owner

What

Adds --curated-dir DIR to soma reproduce as a third manifest source alongside --raw-root (curate from raw) and --from-run-dir (re-score an existing run). When given, it points the pipeline straight at an existing dataset.csv + splits.csv and skips curation entirely.

Why

Curation is a deterministic raw -> manifest step that can cost minutes — the HEST-bench IDC task explodes ~35k spots to lossless PNGs — yet soma reproduce re-ran it on every invocation. This bit during the manual hest/IDC uni2 reproduction, where the same validated manifest was re-derived on each run. --curated-dir lets you curate once and reuse across seeds/encoders.

Design

Chosen over silently reusing an already-populated --out-dir: an explicit flag keeps the "this manifest is good, don't re-derive it" assertion with the caller and never picks up a stale manifest without a signal. The flag is:

  • dataset-type-agnostic — the spatial_expression targets.npy / genes.json sidecars are resolved by the Manifest loader relative to dataset.csv, so it needs no per-benchmark knowledge;
  • family-aware — nests per-member under fan-out (eva, hest);
  • fail-fast — errors clearly when the dir is not a manifest;
  • deferential to --from-run-dir when both are given.

Tests

  • tests/test_benchmarks_cli.py: +3 (skip-curation, missing-manifest error, three-mode error message)
  • docs/cli.rst regenerated from build_cli_rst(); docs-sync + Sphinx build green
  • Full test_benchmarks_cli.py + test_docs.py: 41 passed

Curation is a deterministic raw -> manifest step that can cost minutes
(HEST-bench IDC explodes ~35k spots to lossless PNGs), yet `soma reproduce`
re-ran it on every invocation. Add `--curated-dir DIR` as a third manifest
source alongside `--raw-root` (curate from raw) and `--from-run-dir`
(re-score an existing run): it points the pipeline straight at an existing
dataset.csv + splits.csv and skips curation entirely.

Chosen over silently reusing an already-populated --out-dir: an explicit
flag keeps the "this manifest is good, don't re-derive it" assertion with
the caller and never picks up a stale manifest without a signal. It is
dataset-type-agnostic (the spatial_expression targets.npy / genes.json
sidecars are resolved by the Manifest loader relative to dataset.csv), nests
per-member under family fan-out, fails fast when the dir is not a manifest,
and yields to --from-run-dir when both are given.
@clemsgrs
clemsgrs merged commit 0d48ecd into main Jul 9, 2026
3 checks passed
@clemsgrs
clemsgrs deleted the add-reproduce-curated-dir branch July 9, 2026 23:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant