The Enformer module encapsulates the human and mouse head of the large-scale multi-task Enformer model and can make predictions for sequences of any length. Its training data includes chromatin accessibility, DNA binding and expression.

Agarwal et al. (2025) MPRA evaluator first requests log-scale predictions for ~56k probes across WTC11, K562, and HepG2 cells. It then calculates Pearson correlations between predicted and measured (log2(rna/dna)). It also assesses cell-type specificity by correlating predicted vs. observed expression differences.

Martyn et al. dataset measured the effects of 783 variants in Jurkat T cells and THP-1 monocytes cells. This Evaluator is used to evaluate models' abilities to predict variant effects. We extract 2000bp around the indel and request log scale expression predictions for the reference and mutated sequence.

GAME was designed for the functional genomics community to create seamless communication across pre-trained models and genomics datasets. It is a product of the feedback from many model and dataset experts and our hope is that it allows for long-lasting benchmarking of models.

As most of the sample Evaluators request point-based predictions, we incorporated an Evaluator that explicitly requests and evaluates track-based predictions. To construct this Evaluator, we intersected and merged genomics coordinates from the Enformer and Borzoi test datasets.

To evaluate models’ abilities to predict long range interactions for synthetic modifications we included the Fulco et al. CRISPRi dataset. Endogenous enhancers are synthetically repressed and their effects on 30 unique genes are measured in K562 cells.

We designed three Evaluators requesting H1 cell interaction matrices for Chr 8/9 (Orca test set) and Chr 10 (validation set). They tile chromosomes in 1Mb steps, extracting hg38 sequences at runtime. Evaluation computes Pearson correlations between predicted and measured Hi-C matrices. These modules also demonstrate msgpack-numpy response support.

To evaluate whether models were internally consistent in their predictions, we created Consistency Evaluators.

This synthetic MPRA (Gosai et al. 2024) Evaluator requests expression predictions for ~51k synthetic probes in K562 (erythroid precursors), HepG2 (hepatocytes) and SK-N-SH (neuroblastoma). The Pearson r is calculated between the measured and predicted log2FC values.