Update
Open problems integration <https://github.com/r-sayar/task_expression_reconstruction>
Benchmark for gene expression reconstruction from single-cell latent representations, covering observational and perturbational tasks.
Fig 1. (a) Reconstructing latent cell representations. (b) Latent space modeling under various conditions. (c) Two reconstruction schemes: stand-alone reconstruction (end-to-end & foundation-model) and latent-shift reconstruction (perturbation prediction). (d) Experiment space spans three datasets, three out-of-distribution levels and four hyperparameter axes. (e) Three metric families: statistical, biological, perturbational.
Full documentation, API reference and rendered tutorials at reconeval.readthedocs.io.
Latent representations
- End-to-end: PCA, AE, VAE across latent dims
{10, 32, 128, 512, 2048}and library size handling (None, Modeled, Observed). - Foundation model embeddings: SE from STATE (2058-d), scGPT (512-d), scConcept (512-d), SCimilarity (128-d)
Decoders
- MLP, Transformer, KNN
Datasets
Out-of-distribution levels — 3 level of splitting by cell type / cell line, perturbation, condition.
Metric families — see Computing metrics on your own data below for the API.
- Statistical — R², MMD-RBF, energy distance
- Biological — DEG recovery, coexpression structure, cell-cycle composition, cytokine response, pathway activity
- Perturbational — KNN purity
- Linux (Rocky Linux 9.6 tested); Python 3.12; PyTorch 2.5 + CUDA 12.4.
Full pins per env in
envs/*.yaml. - An NVIDIA GPU is required for training. Metrics + tutorials run on CPU.
Install time: ~2 min (metrics only), ~30 min (full training env).
pip install -r envs/requirements-min.txt # metrics only
conda env create -f envs/cstm_scvi_env.yaml # full training envRuntime: ~5 min on CPU.
Before running, fetch the small demo fixtures from Hugging Face
(luca_demo.h5ad, cytokine_act_merged.csv,
regev_lab_cell_cycle_genes.txt) into analysis/data/frozen/ — see
the Reproducibility section below.
jupyter lab tutorials/metrics.ipynbExpected output: per-metric scores + a funky_heatmap figure.
Metrics on your own (true, reconstructed) AnnData pair:
from sc_reconstruction.metrics import compute_all_metrics
scores = compute_all_metrics(adata_true, adata_pred)For training: see experiments/{01_end_to_end, 02_foundation_model,
03_latent_shift}/README.md.
Reproduction of paper figures: see Reproducibility below.
The metrics notebook walks through each metric on a single
(true, reconstructed) AnnData pair, then shows the rank-percentile
aggregation used to compare methods. The same API applies to all three
benchmark settings in Fig 1c.
The analysis notebooks under Reproducibility run the same recipe against the cached paper artefacts.
YAML configs and SLURM submission scripts for each benchmark setting are
in experiments/, organised by task:
| Folder | What it contains |
|---|---|
experiments/preprocessing/ |
PBMC / LuCA / Tahoe data-preparation scripts. |
experiments/01_end_to_end/ |
PCA / AE / VAE (scVI, nlscVI, mlscVI) reconstruction. |
experiments/02_foundation_model/ |
FM (SE, scGPT, scConcept, SCimilarity) embed + decoder train. |
experiments/03_latent_shift/ |
CellFlow / STATE latent-shift reconstruction. |
Three notebooks under analysis/data/plots/ reproduce the paper's
figures from cached metric CSVs and lookup tables hosted on
huggingface.co/datasets/theislab/ReconEval.
| Setting (Fig 1c) | Notebook | Figures produced |
|---|---|---|
| End-to-end reconstruction (PCA / AE / VAE) | analysis/data/plots/fig2_clean.ipynb |
Fig 2 (qualitative + summary + scaling) |
| Foundation-model reconstruction (frozen FM + decoder) | analysis/data/plots/fig3_clean.ipynb |
Fig 3 (FM × decoder × metrics panels) |
| Latent-shift reconstruction (CellFlow + STATE) | analysis/data/plots/fig4_clean.ipynb |
Fig 4 (ST/CF scaling + B-cell spotlight) |
- Reproducibility data — huggingface.co/datasets/theislab/ReconEval
Update: Model weights uploaded
Preprint: available here!
@article{Fu2026.06.15.731445,
author = {Fu, Xiaotong and Klein, Dominik and Antipov, Egor and Palma, Alessandro and Tejada-Lapuerta, Alejandro and Bahrami, Mojtaba and K{\"u}mmerle, Louis B. and Lubetzki, Manuel and Casale, Francesco Paolo and Luecken, Malte D. and Theis, Fabian J.},
title = {Benchmarking gene expression reconstruction from single-cell latent representations},
elocation-id = {2026.06.15.731445},
year = {2026},
doi = {10.64898/2026.06.15.731445},
publisher = {Cold Spring Harbor Laboratory},
URL = {https://www.biorxiv.org/content/early/2026/06/18/2026.06.15.731445},
eprint = {https://www.biorxiv.org/content/early/2026/06/18/2026.06.15.731445.full.pdf},
journal = {bioRxiv}
}
MIT — see LICENSE.