GOLDMARK — Manuscript Reference Pipeline - https://artificialintelligencepathology.org/

This repository is in support of the manuscript GOLDMARK: Governed Outcome-Linked Diagnostic Model Assessment Reference Kit and is composed of the end-to-end benchmarking pipeline:

Target construction (TCGA via GDC download + variant labeling / OncoKB annotation)
Tiling (20x/40x coordinate generation)
Feature extraction (canonical pathology foundation models + custom encoders) with QC metadata
Training (gated-attention MIL reference head) on predefined patient-level splits
Inference + external inference and attention export (best checkpoint + fixed late epoch)

The design goal is reproducibility and clarity: cross-validation alone is not sufficient—the pipeline is built around reciprocal external testing (e.g., TCGA→external and external→TCGA) under identical preprocessing and evaluation criteria.

Manuscript Figures (Fig 1-3)

Fig 1 (workflow): images/fig1_workflow.pdf
Fig 2 (cross-validation + reciprocal external): images/fig2_cv_reciprocal_external.pdf
Fig 3 (top eight task grid): images/fig3_top8_task_grid.pdf

Start here (real GDC download + end-to-end pipeline)

Downloads and labels real TCGA SVS slides via GDC and runs:

download → tiling → features → training → inference

This is a real-data run (WSI deps + OpenSlide required). Expect multiple TB of downloads and non-trivial runtime.

Preflight checklist (avoid the common failure modes)

Most encoders are hosted on Hugging Face; approval + a valid HF token are required to run this pipeline. Without access you will hit 401 Unauthorized during feature extraction.
Ensure configs/secrets.env exists and is sourced for non-interactive runs (SLURM/nohup). Example: set -a; source configs/secrets.env; set +a
OncoKB is required for TCGA mutation labeling in this pipeline. Set ONCOKB_TOKEN in configs/secrets.env (or provide precomputed OncoKB labels). The pipeline will refuse to run without it.
If you use h-optimus-0, your HUGGINGFACE_HUB_TOKEN (or HF_TOKEN) must have explicit access to bioptimus/H-optimus-0. A 401 Unauthorized means the token is missing or lacks access.
gdc-client requires newer glibc on some clusters. If it fails, GOLDMARK falls back to the GDC API (slower). For full cohorts, prefer a compatible gdc-client or a newer node.
Some clusters do not provide python. Use python3 or set PYTHON_BIN in the SLURM script.
The LUAD KRAS end-to-end run defaults to the full cohort (--per-class 0). Set --per-class to a small value for a quick sanity check.

git clone https://github.com/chadvanderbilt/GOLDMARK.git
cd GOLDMARK

# Put tokens in one place (never commit the filled file).
# GOLDMARK commands will auto-load `configs/secrets.env` if present.
cp configs/secrets.env.example configs/secrets.env
# edit configs/secrets.env (GDC token file path, ONCOKB_TOKEN, HF_TOKEN, ...)

# Option A (recommended on HPC): conda
source /home/vanderbc/.bashrc
bash scripts/setup_conda_goldmark_env.sh
conda activate goldmark

# Option B (sudo/venv): installs system deps + creates venv
bash scripts/setup_goldmark_sudo.sh

# Installs gdc-client into bin/ (ignored by git)
python scripts/install_gdc_client.py --dest bin/gdc-client
# Note: some HPC nodes ship older glibc; if gdc-client fails to run, the pipeline
# will fall back to downloading via the GDC API (sufficient for small runs).

# Downloads 2 tumor + 2 normal slides (smallest by file size) and runs the full pipeline on CPU.
python scripts/gdc_smoke_test_tcga.py --project-id TCGA-COAD --per-class 2 --device cpu --force

Outputs land in runs/gdc_smoke_test/ (including runs/gdc_smoke_test/inference/inference/inference_results.csv).

Reciprocal TCGA→external end-to-end runs (mutation labels + external inference)

This repo ships two end-to-end scripts that exercise the full mutation-label → MIL training → attention export pathway and validate reciprocal external inference (TCGA→external):

scripts/tcga_to_external_smoke_test.py (fast, minimal; good for “does anything run?”)
scripts/tcga_cv_to_external_full_run.py (full run by default; 5-fold CV, attention exports, external inference)

Recommended: TCGA-LUAD KRAS (5×70/30 splits) → external LUAD inference

What it does:

Downloads TCGA-LUAD diagnostic slides via GDC (filters to -00-DX unless --allow-non-dx)
Only downloads slides with targets: the script labels cases using MAFs, then builds a subset manifest and downloads only those labeled slides.
Labels each patient as KRAS positive/negative from the GDC Masked Somatic Mutation MAF
Builds 5 independent 70/30 splits with per-split val assignments (train/val/test stored as columns)
Trains for a full cohort by default and writes a cv_summary.csv
Runs held-out test inference per split and exports attention vectors
Runs external inference on the external LUAD cohort using the best split checkpoint (and links it from all split dirs)

python scripts/tcga_cv_to_external_full_run.py \
  --run-name TCGA-LUAD \
  --per-class 0 \
  --device cuda \
  --epochs 25 \
  --patience 50 \
  --force

Generic SLURM + nohup launchers (configurable project/gene/encoder)

For GitHub users, we provide generic launchers that accept options via environment variables. These are not hard-coded to a specific project or gene.

Make sure the goldmark conda environment is active (or set CONDA_ENV=goldmark before running the scripts).

SLURM (GPU example) — examples/slurm/submit_tcga_cv_to_external.sh

# Example: minimal resume (20x+40x, no external)
export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=0.25
sbatch examples/slurm/submit_tcga_cv_to_external.sh

20x only (no external)

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=
sbatch examples/slurm/submit_tcga_cv_to_external.sh

40x only (no external)

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export TARGET_MPP=0.25
export EXTRA_TARGET_MPP=
sbatch examples/slurm/submit_tcga_cv_to_external.sh

External inference (20x+40x) — examples/slurm/submit_tcga_luad_EGFR_cv_to_external.sh

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export EXTERNAL_PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=0.25
export EXTERNAL_MANIFEST=/path/to/external_manifest.csv
export EXTERNAL_ROOT=/path/to/foundation_model_training_images/EXTERNAL

sbatch examples/slurm/submit_tcga_luad_EGFR_cv_to_external.sh

Notes for SLURM users:

You must set a GPU-capable partition in the script (#SBATCH -p ...) and match your cluster’s GPU request syntax (e.g., --gres=gpu:1).
Some clusters require different SBATCH fields (account/QoS/time/memory). Adjust the header in examples/slurm/submit_tcga_cv_to_external.sh to match local policy.

External inference only runs if EXTERNAL_MANIFEST (and optionally EXTERNAL_ROOT) are set.

Minimum required columns in EXTERNAL_MANIFEST:

Label column: one of <GENE> (e.g., EGFR), label_index, label, or target
Slide ID column: one of slide_id, DMP_ASSAY_ID, sample_id, case_id, or slide

Optional columns:

feature_path (absolute/relative path to features per slide; overrides EXTERNAL_ROOT)
OncoTree_Code (used to resolve features under EXTERNAL_ROOT)
scanned_slides_exist (rows with 0/false are skipped)

nohup (interactive node) — scripts/nohup_tcga_cv_to_external.sh

# Example: minimal resume (20x+40x, no external)
export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=0.25
bash scripts/nohup_tcga_cv_to_external.sh

20x only (no external)

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=
bash scripts/nohup_tcga_cv_to_external.sh

40x only (no external)

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export TARGET_MPP=0.25
export EXTRA_TARGET_MPP=
bash scripts/nohup_tcga_cv_to_external.sh

External inference (20x+40x)

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=resume
export PER_CLASS=0
export EXTERNAL_PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=0.25
export EXTERNAL_MANIFEST=/path/to/external_manifest.csv
export EXTERNAL_ROOT=/path/to/foundation_model_training_images/EXTERNAL
bash scripts/nohup_tcga_cv_to_external.sh

Watch logs:

tail -f runs/<run-name>/logs/nohup.out

End-to-end example (foreground) — examples/run_tcga_luad_egfr_end_to_end.sh

# Example: minimal force (20x+40x, no external)
export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=force
export PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=0.25
bash examples/run_tcga_luad_egfr_end_to_end.sh

20x only (no external)

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=force
export PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=
bash examples/run_tcga_luad_egfr_end_to_end.sh

40x only (no external)

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=force
export PER_CLASS=0
export TARGET_MPP=0.25
export EXTRA_TARGET_MPP=
bash examples/run_tcga_luad_egfr_end_to_end.sh

External inference (20x+40x)

export PROJECT_ID=TCGA-LUAD
export GENE=EGFR
export ENCODER=h-optimus-0
export RUN_NAME=TCGA-LUAD
export RUN_MODE=force
export PER_CLASS=0
export EXTERNAL_PER_CLASS=0
export TARGET_MPP=0.5
export EXTRA_TARGET_MPP=0.25
export EXTERNAL_MANIFEST=/path/to/external_manifest.csv
export EXTERNAL_ROOT=/path/to/foundation_model_training_images/EXTERNAL
bash examples/run_tcga_luad_egfr_end_to_end.sh

Options (env vars)

PROJECT_ID (default: TCGA-LUAD)
GENE (default: KRAS)
ENCODER (default: h-optimus-0)
DEVICE (default: cuda)
TARGET_MPP (default: 0.5)
EXTRA_TARGET_MPP (default: 0.25; comma-separated MPP buckets — each slide is assigned to the nearest bucket)
RUN_NAME (default: ${PROJECT_ID})
RUN_MODE in {force|resume|rebuild} (default: force)
- force: reset derived outputs only (tiling/features/training/inference/etc.) and preserve gdc_downloads/; re-downloads happen only if checksum/size validation fails.
- resume: keep all existing outputs; only run missing stages (downloads/tiling/features/training/inference).
- rebuild: keep downloads + derived labels, but re-run tiling, features, training, inference, external_inference, plots.
PER_CLASS, EXTERNAL_PER_CLASS, LIMIT_TILES, EPOCHS, PATIENCE, VAL_PER_CLASS (default: 0 = use full test split for validation)

By default, RUN_NAME is set to the project id so all outputs live under runs/<project-id>/.

Output layout (important files and where to find them)

All outputs land under runs/<project-id>/ (the default RUN_NAME is the project id).

A) Project inputs (download + derived labels)

runs/<project-id>/gdc_downloads/<PROJECT>_svs_manifest.tsv — full GDC SVS manifest for the project
runs/<project-id>/gdc_downloads/svs/**/**.svs — downloaded TCGA SVS files (GDC UUID folders)
runs/<project-id>/gdc_downloads/maf/**/**.maf.gz — downloaded GDC mutation calls used for labels
runs/<project-id>/checkpoints/<TARGET>/manifests/<PROJECT>_<TARGET>_svs_manifest_subset.tsv — selected SVS rows
runs/<project-id>/checkpoints/<TARGET>/manifests/<PROJECT>_<TARGET>_slides.csv — selected slide list + labels (schema below)
runs/<project-id>/checkpoints/<TARGET>/manifests/external_manifest_<TARGET>_<encoder>.csv — external subset manifest (schema below)

B) Tiling

runs/<project-id>/tiling/tiles_20x/manifests/<slide_id>_tiles.csv — per-slide tile coordinate manifest (schema below)
runs/<project-id>/tiling/tiles_20x/cases/<case_id>_tiles.csv — per-case tile coordinates aggregated across slides
runs/<project-id>/tiling/tiles_20x/cases/cases_index.csv — case_id → case manifest path index
runs/<project-id>/tiling/tiles — alias pointing at tiles_20x for compatibility
runs/<project-id>/tiling/tile_coords_20x.csv — aggregated tile coords (x,y,slide,sample_id,target)
runs/<project-id>/tiling/tiling_manifest.csv — slide inventory (file_name,sample_id,target,slide_path)

When EXTRA_TARGET_MPP is set (e.g., 0.25), slides are bucketed by native MPP:

runs/<project-id>/tiling/tiles_20x/... for slides closer to 0.5
runs/<project-id>/tiling/tiles_40x/... for slides closer to 0.25
runs/<project-id>/tiling/tile_coords_20x.csv and tile_coords_40x.csv each contain only the slides assigned to that bucket

Tiles keep the same physical size across buckets by scaling pixel size (e.g., 224px at 0.5 mpp ⇔ 448px at 0.25 mpp).

C) Features + QC

runs/<project-id>/features/<encoder>/features_<slide_id>.pt — per-slide feature tensor (N tiles × D; tile size set by the slide’s MPP bucket)
runs/<project-id>/features/<encoder>/features_<slide_id>.json — QC metadata + checksums (schema below)
If feature extraction fails QC, tensors are renamed:
- features_<slide_id>.FAILED_tile_count_mismatch.pt
- features_<slide_id>.FAILED_degenerate_embeddings.pt

Feature tensors are saved as PyTorch .pt files with rows aligned to the slide’s tile manifest order, so each row can be joined back to tile coordinates using the corresponding tile manifest CSV.

Performance note: the feature extractor auto-tunes GPU batch size based on available VRAM and uses CPU data loader workers to keep the GPU fed. You can override both via --batch-size and --num-workers. As a rule of thumb, start with --num-workers 4 and increase to 8–16 on beefy GPU nodes if CPU tile decoding becomes the bottleneck.

When EXTRA_TARGET_MPP is set, features are still written once per slide (no suffix). Slides assigned to 40x are extracted with the larger pixel tile size, but the output path remains:

runs/<project-id>/features/<encoder>/features_<slide_id>.pt

D) Training (5-fold CV)

AGGREGATOR is the upper-case aggregation method (default: GMA).
runs/<project-id>/training/checkpoints/<GENE>/versioned_split_manifest/<GENE>_all_splits_latest.csv — shared split manifest used for all encoders (schema below)
runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/classification_report/cv_summary.csv — per-split best epoch + metrics (schema below)
runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/split_1_set/checkpoint/checkpoint_best.pt — best checkpoint for that split (and similarly for split_2_set..split_5_set)

E) Per-split held-out test inference (attention export + ROC/PR plots)

Written under each split so split context is self-contained:

runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/split_1_set/inference/test/inference_results.csv
runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/split_1_set/inference/test/attention/<slide_id>_attention.csv
runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/split_1_set/inference/test/plots/roc_pr_curves.png

Note: the training stage also writes probability exports under the same directory (e.g. probabilities_test_set.csv).

F) External inference (external LUAD)

External inference runs for each split using the best AUC epoch from CV plus any configured epochs (e.g., the final epoch), and evaluates every slide in the external cohort. Results are written under each split:

runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/split_1_set/external_inference/external/ckpt_best_<epoch>/inference_results.csv
runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/split_1_set/external_inference/external/ckpt_best_<epoch>/attention/<slide_id>_attention.csv
runs/<project-id>/training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/split_1_set/external_inference/external/ckpt_best_<epoch>/plots/roc_pr_curves.png

Additional epochs (e.g., final epoch) appear under:

.../external_inference/external/ckpt_<epoch>/...

File schemas (column names)

TCGA slide selection manifest (training/checkpoints/<GENE>/manifests/<PROJECT>_<GENE>_slides.csv)

slide_id (e.g. TCGA-05-4244-01Z-00-DX1)
slide_path (downloaded .svs path)
label_index (0/1 mutation label for the chosen gene)
patient_id (TCGA case/patient barcode prefix)

Versioned split manifest (training/checkpoints/<GENE>/versioned_split_manifest/<GENE>_all_splits_latest.csv)

slide_path, slide_id, label_index
target (legacy column retained for compatibility)
split_1_set … split_5_set with values in {train,val,test}

Tile manifest (tiling/tiles_20x/manifests/<slide_id>_tiles.csv)

slide_id
tile_id (stable tile identifier)
x, y (level-0 pixel coordinates)
level (OpenSlide level used for extraction / overlays)
width, height (tile size in pixels at level)
tissue_fraction (fraction of tile kept after tissue mask filtering)
tile_path (optional; blank when only coordinates are generated)

Tile manifest with attention (.../attention/<slide_id>_tiles_with_attention.csv)

Same columns as the tile manifest, plus attention for that slide.

Feature metadata JSON (features/<encoder>/features_<slide_id>.json)

slide_id, encoder
tile_manifest
num_tiles, num_features, feature_dim
embedding_stats (degenerate/variance checks)
feature_sha256, feature_bytes
status in {ok,failed} and failure_reason when failed Metadata is written once per slide (no suffix), regardless of its MPP bucket.

CV summary (training/checkpoints/<GENE>/<ENCODER>/<AGGREGATOR>/classification_report/cv_summary.csv)

split (e.g. split_1_set)
best_epoch
val_* and test_* metrics, including *_roc_auc, *_accuracy, *_precision, *_recall, *_f1, *_balanced_error_rate

Inference results (.../inference_results.csv)

slide_id
probability (predicted P(class=1))
prediction (thresholded at 0.5 by default)
target (ground-truth label, from label_index)

Attention exports (.../attention/<slide_id>_attention.csv)

slide_id
tile_index (0-based)
tile_id, x, y, level (when tile manifest metadata is available)
attention (raw attention weight)
probability (slide-level probability repeated for convenience)

External inference manifest (training/checkpoints/<GENE>/manifests/external_manifest_<GENE>_<encoder>.csv)

Minimum required columns for external inference with InferenceRunner:

slide_id (string id used to resolve features / label rows)
label_index (0/1 ground-truth label; used as target_column)

Strongly recommended columns (enables overlays + clearer provenance):

slide_path (absolute .svs path for overlays; optional if you do not generate overlays)
feature_path (absolute .pt path; optional if you pass --feature-dir and follow features_<slide_id>.pt naming)

Optional columns:

split (only needed if you want to run inference on a subset via InferenceConfig.split_column/split_value)

Docs:

docs/targets.md
docs/pipeline.md
SLURM (generic): examples/slurm/submit_tcga_cv_to_external.sh
SLURM (EGFR example): examples/slurm/submit_tcga_luad_EGFR_cv_to_external.sh

Canonical pathology foundation model encoders

Each encoder below reflects the implementation in goldmark/features/canonical_sources.py, which follows the recommended preprocessing from the original model repositories. Access typically requires a Hugging Face token.

prov-gigapath

Access

Hugging Face access (may require approval); set HF_TOKEN/HUGGINGFACE_HUB_TOKEN or use huggingface-cli login.

Hugging Face (more details)

Repo: prov-gigapath/prov-gigapath
URL: https://huggingface.co/prov-gigapath/prov-gigapath

Implementation

Loader: timm.create_model("hf_hub:prov-gigapath/prov-gigapath", pretrained=True)
Transform: resize 256 → center crop 224 → ImageNet normalize (timm defaults)
Tile size: 224
Feature dim: 1536

EAGLE

Access

Public on Hugging Face; set HF_TOKEN/HUGGINGFACE_HUB_TOKEN if needed.

Hugging Face (more details)

Repo: MCCPBR/EAGLE
URL: https://huggingface.co/MCCPBR/EAGLE

Implementation

Loader: use the custom encoder hook (--custom-encoder / --custom-encoder-script) to point at the EAGLE implementation.
Transform: follow the EAGLE repo’s preprocessing exactly (see internal docs).
Feature dim: defined by the EAGLE encoder (see internal docs).

EAGLE repo reference (example)

git clone --no-checkout https://huggingface.co/MCCPBR/EAGLE && cd EAGLE

from PIL import Image
import numpy as np
import eagle
import torch
import torchvision.transforms as transforms

# Load model
model = eagle.EAGLE()

# Set up transform
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225))
])

# Image
img = np.random.randint(0, 256, size=224*224*3).reshape(224,224,3).astype(np.uint8)
img = Image.fromarray(img)
img = transform(img).unsqueeze(0)

# Inference
with torch.no_grad():
    h, att, p = model(img)

UNI

Access

Hugging Face access (gated); set HF_TOKEN/HUGGINGFACE_HUB_TOKEN or use huggingface-cli login.
If you already have weights locally, point MIL_UNI_CHECKPOINT to the file (fallback: weights/uni_checkpoint.pth).

Hugging Face (more details)

Repo: MahmoodLab/UNI
URL: https://huggingface.co/MahmoodLab/UNI

Implementation

Loader: timm.create_model("vit_large_patch16_224", ...) + load checkpoint
Transform: resize 224 → ImageNet normalize (mean=(0.485,0.456,0.406), std=(0.229,0.224,0.225))
Tile size: 224
Feature dim: 1024

Virchow

Access

Hugging Face access (may require approval); set HF_TOKEN/HUGGINGFACE_HUB_TOKEN or use huggingface-cli login.

Hugging Face (more details)

Repo: paige-ai/Virchow
URL: https://huggingface.co/paige-ai/Virchow

Implementation

Loader: timm.create_model("hf-hub:paige-ai/Virchow", ...) with SwiGLUPacked + SiLU
Transform: timm.create_transform from model config
Tile size: 224 (0.5 mpp recommended by model card)
Feature dim: 2560 (class token + pooled patch tokens)

Virchow2

Access

Hugging Face access (may require approval); set HF_TOKEN/HUGGINGFACE_HUB_TOKEN or use huggingface-cli login.

Hugging Face (more details)

Repo: paige-ai/Virchow2
URL: https://huggingface.co/paige-ai/Virchow2

Implementation

Loader: timm.create_model("hf-hub:paige-ai/Virchow2", ...) with SwiGLUPacked + SiLU
Transform: timm.create_transform from model config
Tile size: 224 (0.5 mpp recommended by model card)
Feature dim: 2560 (class token + pooled patch tokens; ignores 4 register tokens)

h-optimus-0

Access

Hugging Face access (gated; requires explicit approval).

Hugging Face (more details)

Repo: bioptimus/H-optimus-0
URL: https://huggingface.co/bioptimus/H-optimus-0

Implementation

Loader: timm.create_model("hf-hub:bioptimus/H-optimus-0", pretrained=True)
Transform: resize 224 → normalize Bioptimus mean/std (mean=(0.707223,0.578729,0.703617), std=(0.211883,0.230117,0.177517))
Tile size: 224 (0.5 mpp recommended by model card)
Feature dim: 1536

Repository layout

Path	What it contains
`targets/`	Public, token-safe scripts for GDC download + mutation target tables
`goldmark/`	Minimal Python package: tiling, feature extraction, training, inference
`scripts/`	Manuscript-scale runners (CV scanning, inference plans, attention export)
`examples/`	Example manifest headers + plan templates
`paper/`	Select manuscript figures/tables for reference

Installation

Preferred (conda):

source /home/vanderbc/.bashrc
bash scripts/setup_conda_goldmark_env.sh

If you have sudo and prefer a venv:

bash scripts/setup_goldmark_sudo.sh

Quickstart (typical flow)

Set a repo root (recommended):

export REPO_ROOT="$PWD"

1) Targets (TCGA example)

Generate a GDC manifest (SVS example):

python -m goldmark gdc-manifest svs \
  --project-id TCGA-COAD \
  --out tcga_coad_svs_manifest.tsv

Download data from GDC:

targets/tcga/gdc_download.sh --manifest tcga_coad_svs_manifest.tsv --out data/gdc_download
# If needed for controlled-access data, set GDC_TOKEN_FILE in configs/secrets.env
# (or pass --token /path/to/gdc_token.txt).

Annotate MAF with OncoKB (token via env var):

# export ONCOKB_TOKEN="..."  # or set in configs/secrets.env
python targets/variants/annotate_maf_oncokb_by_hgvsg.py \
  --maf-glob "data/gdc_download/**/**.maf.gz" \
  --oncotree-code <ONCOTREE_CODE> \
  --output data/oncokb/oncokb_annotations.csv

--oncotree-code ensures OncoKB levels are computed for the correct tumor type (e.g., LUAD for TCGA-LUAD).

Summarize a single gene’s patient-level mutation status:

python targets/variants/summarize_gene_status.py \
  --annotations data/oncokb/oncokb_annotations.csv \
  --gene <GENE> \
  --output data/targets/<GENE>_patient_labels.csv

The output includes per-patient p. changes and OncoKB level summaries for spot-checking:

p_changes (joined with | for multiple mutations; no_gene_changes if none)
oncokb_levels (joined with |)
oncokb_positive / has_gene_mutation / label_reason

label_index is positive if any mutation is OncoKB level 1/2/3 for the target gene.

Build a slide-level manifest by joining SVS paths to patient labels:

python targets/tcga/build_slide_manifest_from_svs_and_mutations.py \
  --svs-root data/gdc_download \
  --labels data/targets/<GENE>_patient_labels.csv \
  --output data/manifests/TCGA_<GENE>_slide_manifest.csv

Generate the manuscript-style split manifest:

PYTHONPATH=. python scripts/generate_versioned_split_manifest.py \
  --manifest data/manifests/TCGA_<GENE>_slide_manifest.csv \
  --target <GENE> \
  --label-column label_index \
  --target-dir data/checkpoints/<GENE>

See manifest header examples:

TCGA split manifest schema: examples/manifests/tcga_split_manifest_header.csv
External manifest schema (header only): examples/manifests/external_LUAD_manifest_header.csv

2) Tiling (20x/40x coordinates)

The canonical tiler is goldmark/tiling/extractor.py. For convenience, this repo also exposes a small CLI:

PYTHONPATH=. python -m goldmark tiling data/manifests/TCGA_<GENE>_slide_manifest.csv \
  --output runs \
  --run-name tcga_<GENE>_demo \
  --tile-size 224 \
  --stride 224 \
  --target-mpp 0.5

Tile manifests are written under runs/<run-name>/tiling/tiles/manifests/ as <slide_id>_tiles.csv.

3) Feature extraction + QC metadata

Feature extraction writes:

features_<slide_id>.pt (per-slide tensor)
features_<slide_id>.json (QC metadata + checksums)

If tile/feature counts mismatch or embeddings are degenerate (low variance), the tensor is renamed: features_<slide_id>.FAILED_<reason>.pt and training/inference will treat the slide as missing.

Example:

PYTHONPATH=. python -m goldmark features data/manifests/TCGA_<GENE>_slide_manifest.csv \
  --tile-manifests runs/tcga_<GENE>_demo/tiling/tiles \
  --output runs \
  --run-name tcga_<GENE>_demo \
  --encoder prov-gigapath \
  --device cuda \
  --num-workers 4

Optional integrity audit (tile counts vs feature lengths):

python scripts/check_tile_feature_counts.py \
  --tile-manifest-dir runs/tcga_<GENE>_demo/tiling/tiles/manifests \
  --feature-dir runs/tcga_<GENE>_demo/features/prov-gigapath

4) Training (reference gated-attention MIL head)

Manuscript-scale training is orchestrated by:

scripts/run_training_scan_target.py (scan encoders + submit missing runs)
scripts/train_task_v2.py (train one target across 5 splits)

5) Inference + external inference + attention export

The manuscript pipeline exports attention vectors for (i) best checkpoint and (ii) a fixed late epoch to show how checkpoint selection affects results and to support consistent downstream overlays.

See:

scripts/run_inference_from_plan.py
scripts/gma_inference_pipeline.py
SLURM template: examples/slurm/submit_attn_external_BLCA_ERBB2.sh

Reference figures/tables

Workflow: paper/figures/workflow_schematic.pdf
Tile/feature QC: paper/figures/qc_tile_feature_integrity.pdf
Table 1 (counts): paper/tables/table1_gene_mutation_counts.tex

Manuscript tasks (Table 1)

Show tumor/target tasks and cohort counts

Tumor Code	Tumor Type	Gene Mutation	MSKCC Total	MSKCC Positive	MSKCC Negative	TCGA Total	TCGA Positive	TCGA Negative
BLCA	Bladder Urothelial Carcinoma	PIK3CA	2031	337	1694	386	77	309
BLCA	Bladder Urothelial Carcinoma	FGFR3	2031	393	1638	386	34	352
BLCA	Bladder Urothelial Carcinoma	ERBB2	2031	207	1824	386	37	349
BLCA	Bladder Urothelial Carcinoma	TSC1	2031	131	1900	386	27	359
BLCA	Bladder Urothelial Carcinoma	ERCC2	2031	141	1890	386	28	358
BRCA	Breast Carcinoma	PIK3CA	2735	966	1769	1000	354	646
CESC	Cervical Squamous Cell Carcinoma	PIK3CA	171	62	109	266	82	184
COAD	Colon Adenocarcinoma	PIK3CA	2959	605	2354	550	130	420
COAD	Colon Adenocarcinoma	BRAF	2959	318	2641	550	59	491
COAD	Colon Adenocarcinoma	ATM	2959	153	2806	550	63	487
COAD	Colon Adenocarcinoma	PTEN	2959	183	2776	550	38	512
COAD	Colon Adenocarcinoma	MSI	2959	350	2609	405	70	335
GBM	Glioblastoma Multiforme	PTEN	816	263	553	244	77	167
UCEC	Endometrial Cancer	PTEN	2062	972	1090	499	319	180
UCEC	Endometrial Cancer	PIK3CA	2062	904	1158	499	254	245
UCEC	Endometrial Cancer	FBXW7	2062	279	1783	499	94	405
UCEC	Endometrial Cancer	ATM	2062	142	1920	499	73	426
UCEC	Endometrial Cancer	POLE	2062	114	1948	499	49	450
LGG	Glioma	IDH1	449	216	233	491	382	109
LGG	Glioma	PIK3CA	449	41	408	491	42	449
HNSC	Head and Neck Carcinoma	PIK3CA	485	84	401	431	66	365
HNSC	Head and Neck Carcinoma	HRAS	485	11	474	431	25	406
LUAD	Lung Adenocarcinoma	KRAS	923	273	650	465	171	294
LUAD	Lung Adenocarcinoma	EGFR	923	273	650	465	51	414
SKCM	Melanoma	BRAF	886	213	673	432	233	199
SKCM	Melanoma	NRAS	886	143	743	432	112	320
SKCM	Melanoma	PTEN	886	63	823	432	50	382
SKCM	Melanoma	MAP2K1	886	38	848	432	26	406
PAAD	Pancreatic Adenocarcinoma	KRAS	3018	2674	344	181	136	45
PCPG	Pheochromocytoma/Paraganglioma	HRAS	18	2	16	176	19	157
STAD	Stomach Adenocarcinoma	PIK3CA	742	69	673	374	60	314
THCA	Thyroid Cancer	BRAF	920	399	521	495	296	199
THCA	Thyroid Cancer	NRAS	920	130	790	495	41	454

Launch all manuscript tasks (optional)

This repo includes a convenience launcher that loops over the manuscript task list and calls scripts/run_training_scan_target.py for each tumor/target.

export MIL_DATA_ROOT="/path/to/foundation_model_training_images"

# dry-run (prints commands)
python scripts/launch_manuscript_tasks.py

# execute (runs training scans)
python scripts/launch_manuscript_tasks.py --execute

Note: this wrapper runs commands sequentially. On HPC clusters you will typically wrap each printed command in sbatch (or your scheduler of choice).

Run a full project workflows

1) Run a full project with slides from GDC (default)

This path starts from GDC slide download and runs:

download -> tiling -> features -> training -> inference

For concrete command examples, use the detailed sections above:

### Recommended: TCGA-LUAD KRAS (5×70/30 splits) → external LUAD inference
SLURM (GPU example) accordion (examples/slurm/submit_tcga_cv_to_external.sh)
nohup (interactive node) accordion (scripts/nohup_tcga_cv_to_external.sh)
End-to-end example (foreground) accordion (examples/run_tcga_luad_egfr_end_to_end.sh)

Primary orchestration script:

scripts/tcga_cv_to_external_full_run.py

Run outputs:

runs/<RUN_NAME>/ (full artifacts)
runs/<RUN_NAME>/training/checkpoints/ (trained checkpoints)
runs/<RUN_NAME>/inference/ (inference outputs)

Public GOLDMARK reference links:

Landing page: https://artificialintelligencepathology.org/
Runs API: https://artificialintelligencepathology.org/api/runs
Example TCGA run: https://artificialintelligencepathology.org/runs/rf207d22e2c0c~TCGA-BLCA_svs

2) Run a full project from pre-extracted features with GOLDMARK API

This path reuses existing GOLDMARK training code (scripts/train_task_v2.py) and only adds API orchestration. The helper script:

discovers downloadable artifacts from https://artificialintelligencepathology.org/api/runs/{slug}/downloads
downloads the sanitized gene split manifest bundle
resolves feature tensors (local extracted dir when available, or downloaded feature zip)
launches scripts/train_task_v2.py with AGGREGATION_METHOD=gma

Script:

scripts/api_demo_from_preextracted_features.py

BLCA + FGFR3 + UNI demo:

python scripts/api_demo_from_preextracted_features.py \
  --base-url https://artificialintelligencepathology.org \
  --slug rf207d22e2c0c~TCGA-BLCA_svs \
  --gene FGFR3 \
  --encoder uni \
  --work-dir runs/api_demo_blca_fgfr3_uni \
  --sample-per-class 40 \
  --epochs 5 \
  --device cuda

Useful API links for this workflow:

Runs list: https://artificialintelligencepathology.org/api/runs?dataset=tcga
Download catalog (BLCA example): https://artificialintelligencepathology.org/api/runs/rf207d22e2c0c~TCGA-BLCA_svs/downloads?download_source=postgres&results_mode=publication&epoch_view=best
Category-specific API instructions (manifest): https://artificialintelligencepathology.org/download-api-instructions?slug=rf207d22e2c0c~TCGA-BLCA_svs&category=manifest&results_mode=publication&epoch_view=best&download_source=postgres
Category-specific API instructions (features): https://artificialintelligencepathology.org/download-api-instructions?slug=rf207d22e2c0c~TCGA-BLCA_svs&category=features&results_mode=publication&epoch_view=best&download_source=postgres

Download scheme for direct artifacts:

Fetch catalogs from https://artificialintelligencepathology.org. Catalog item download_url values may be absolute URLs on https://downloads.artificialintelligencepathology.org, so do not blindly prepend BASE_URL.
Derive the local filename from path or relative_path in the catalog and pass it explicitly with -o. Do not combine curl --continue-at - with -J/--remote-header-name; curl rejects that combination.
For large feature archives, use resumable curl with retries and a local log. If a transfer drops, rerun the same command from the same directory; --continue-at - resumes the partial file.

Example: list feature bundle names and URLs for BLCA:

BASE_URL="https://artificialintelligencepathology.org"
RUN_SLUG="rf207d22e2c0c~TCGA-BLCA_svs"
CATALOG_URL="${BASE_URL}/api/runs/${RUN_SLUG}/downloads?download_source=postgres&results_mode=publication&epoch_view=best"

curl -fsS "${CATALOG_URL}" | jq -r --arg base "${BASE_URL}" '
  .categories[]
  | select(.key=="features")
  | .items[]
  | select(.download_url != null)
  | [
      .label,
      ((.path // .relative_path // (.label + ".download")) | split("/") | .[-1]),
      (if (.download_url | startswith("http")) then .download_url else ($base + .download_url) end)
    ]
  | @tsv
'

Example: resumable download of the first item in a category:

BASE_URL="https://artificialintelligencepathology.org"
RUN_SLUG="rf207d22e2c0c~TCGA-BLCA_svs"
CATEGORY="manifest"
CATALOG_URL="${BASE_URL}/api/runs/${RUN_SLUG}/downloads?download_source=postgres&results_mode=publication&epoch_view=best"
LOG="goldmark_download_$(date -u +%Y%m%dT%H%M%SZ).log"

FIRST_ROW=$(curl -fsS "${CATALOG_URL}" | jq -r --arg base "${BASE_URL}" --arg category "${CATEGORY}" '
  .categories[]
  | select(.key==$category)
  | .items[]
  | select(.download_url != null)
  | [
      ((.path // .relative_path // (.label + ".download")) | split("/") | .[-1]),
      (if (.download_url | startswith("http")) then .download_url else ($base + .download_url) end)
    ]
  | @tsv
' | head -n 1)

FILENAME=$(printf '%s\n' "${FIRST_ROW}" | cut -f1)
DOWNLOAD_URL=$(printf '%s\n' "${FIRST_ROW}" | cut -f2-)

{
  printf 'started_utc=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
  printf 'filename=%s\nurl=%s\n' "${FILENAME}" "${DOWNLOAD_URL}"
  curl --http1.1 -fL --continue-at - -o "${FILENAME}" \
    --retry 1000 --retry-all-errors --retry-delay 30 --connect-timeout 30 \
    --speed-time 300 --speed-limit 1024 \
    --write-out '\nhttp_code=%{http_code}\nremote_ip=%{remote_ip}\nsize_download=%{size_download}\ntime_total=%{time_total}\nurl_effective=%{url_effective}\n' \
    "${DOWNLOAD_URL}"
  printf 'finished_utc=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
} 2>&1 | tee -a "${LOG}"

Notes:

On the same host where /data3/vanderbc/foundation_model_training_images/... is mounted, keep --prefer-local-feature-dir (default) to avoid re-downloading very large feature zips.
On a remote host, use --no-prefer-local-feature-dir and optionally --extract-all-features when you need the full feature archive.

3) Run Inference on User Slides (requires mutations curation)

This workflow is for external engineers running GOLDMARK on their own .svs slides.

Important constraints:

You must curate mutation-derived labels outside this repo and provide binary labels (Positive/Negative -> 1/0) per task.
We do not provide one canonical OncoKB labeling script for external data because source formats differ across institutions.
Given a labeled manifest + slides, GOLDMARK can run tiling -> features -> inference and consume slide-level checkpoints from the public API.

A) Build a labeled user manifest

Minimum recommended columns:

slide_id (stable per-slide id)
slide_path (absolute path to .svs)
label_index (0 or 1)
split (set all rows to test for pure inference)

Example:

slide_id,slide_path,label_index,split
CASE_001,/absolute/path/CASE_001.svs,1,test
CASE_002,/absolute/path/CASE_002.svs,0,test

B) Tiling + feature extraction (canonical encoders)

PYTHONPATH=. python -m goldmark tiling user_manifest.csv \
  --output runs \
  --run-name user_slides_fgfr3 \
  --tile-size 224 \
  --stride 224 \
  --target-mpp 0.5

PYTHONPATH=. python -m goldmark features user_manifest.csv \
  --tile-manifests runs/user_slides_fgfr3/tiling/tiles \
  --output runs \
  --run-name user_slides_fgfr3 \
  --encoder uni \
  --device cuda \
  --num-workers 4

You may switch --encoder to any canonical encoder (gigapath_ft, h-optimus-0, prov-gigapath, uni, virchow, virchow2).

C) Pull a checkpoint from GOLDMARK API (default: best external inference AUC)

The checkpoint catalog is exposed from:

https://artificialintelligencepathology.org/api/runs/rf207d22e2c0c~TCGA-BLCA_svs/downloads?download_source=postgres&results_mode=publication&epoch_view=best

The command below selects the checkpoint with the highest external_auc for one (gene, encoder) pair (fallback to cv_auc if external_auc is missing):

BASE_URL="https://artificialintelligencepathology.org"
RUN_SLUG="rf207d22e2c0c~TCGA-BLCA_svs"
GENE="FGFR3"
ENCODER="uni"
export GENE ENCODER

DOWNLOADS_JSON="$(mktemp)"
export DOWNLOADS_JSON
curl -sS "${BASE_URL}/api/runs/${RUN_SLUG}/downloads?download_source=postgres&results_mode=publication&epoch_view=best" > "${DOWNLOADS_JSON}"

BEST_JSON="$(python - <<'PY'
import json,sys,re,os
obj=json.load(open(os.environ["DOWNLOADS_JSON"]))
gene=os.environ["GENE"].upper()
enc=os.environ["ENCODER"].lower()
def norm(x): return re.sub(r'[^a-z0-9]+','',str(x).lower())
rows=[]
for cat in obj.get("categories",[]):
    if cat.get("key")!="slide_models":
        continue
    for it in cat.get("items",[]):
        m=it.get("metadata") or {}
        if str(m.get("gene","")).upper()!=gene:
            continue
        if norm(m.get("encoder",""))!=norm(enc):
            continue
        score=m.get("external_auc")
        if score is None:
            score=m.get("cv_auc")
        rows.append((float(score) if score is not None else -1.0, it))
if not rows:
    raise SystemExit("No matching slide_models found.")
rows.sort(key=lambda t:t[0], reverse=True)
print(json.dumps(rows[0][1]))
PY)"
export BEST_JSON

CKPT_URL="$(python -c 'import json,os,urllib.parse; u=json.loads(os.environ["BEST_JSON"])["download_url"]; b=os.environ["BASE_URL"].rstrip("/") + "/"; print(u if u.startswith(("http://","https://")) else urllib.parse.urljoin(b, u.lstrip("/")))')"
mkdir -p models
curl --http1.1 -fL --continue-at - -o "models/${GENE}_${ENCODER}_best_external_auc.pt" "${CKPT_URL}"
rm -f "${DOWNLOADS_JSON}"

D) Run inference (GMA) on user slides

GENE="FGFR3"
ENCODER="uni"

PYTHONPATH=. python -m goldmark inference user_manifest.csv \
  --feature-dir "runs/user_slides_fgfr3/features/${ENCODER}" \
  --checkpoint "models/${GENE}_${ENCODER}_best_external_auc.pt" \
  --output runs \
  --run-name "user_slides_fgfr3_${ENCODER}_inference" \
  --target label_index \
  --split-column split \
  --split-value test \
  --export-attention \
  --no-overlays

E) Option: run all available checkpoints for `(gene, encoder)`

If you want split-by-split checkpoint sweeps (instead of default “best external AUC” only):

BASE_URL="https://artificialintelligencepathology.org"
RUN_SLUG="rf207d22e2c0c~TCGA-BLCA_svs"
GENE="FGFR3"
ENCODER="uni"
export GENE ENCODER

DOWNLOADS_JSON="$(mktemp)"
export DOWNLOADS_JSON
curl -sS "${BASE_URL}/api/runs/${RUN_SLUG}/downloads?download_source=postgres&results_mode=publication&epoch_view=best" > "${DOWNLOADS_JSON}"

python - <<'PY' > /tmp/checkpoints.tsv
import json,sys,re,os
obj=json.load(open(os.environ["DOWNLOADS_JSON"]))
gene=os.environ["GENE"].upper()
enc=os.environ["ENCODER"].lower()
def norm(x): return re.sub(r'[^a-z0-9]+','',str(x).lower())
for cat in obj.get("categories",[]):
    if cat.get("key")!="slide_models":
        continue
    for it in cat.get("items",[]):
        m=it.get("metadata") or {}
        if str(m.get("gene","")).upper()!=gene:
            continue
        if norm(m.get("encoder",""))!=norm(enc):
            continue
        split=str(m.get("split","na"))
        ext=m.get("external_auc")
        cv=m.get("cv_auc")
        url=it['download_url']
        base=os.environ.get("BASE_URL", "https://artificialintelligencepathology.org").rstrip("/") + "/"
        if not str(url).startswith(("http://", "https://")):
            import urllib.parse
            url=urllib.parse.urljoin(base, str(url).lstrip("/"))
        print(f"{split}\t{ext}\t{cv}\t{url}")
PY
rm -f "${DOWNLOADS_JSON}"

while IFS=$'\t' read -r SPLIT EXT_AUC CV_AUC URL; do
  OUT_CKPT="models/${GENE}_${ENCODER}_split${SPLIT}.pt"
  curl --http1.1 -fL --continue-at - -o "${OUT_CKPT}" "${URL}"
  PYTHONPATH=. python -m goldmark inference user_manifest.csv \
    --feature-dir runs/user_slides_fgfr3/features/uni \
    --checkpoint "${OUT_CKPT}" \
    --output runs \
    --run-name "user_slides_fgfr3_uni_split${SPLIT}" \
    --target label_index \
    --split-column split \
    --split-value test \
    --no-overlays
done < /tmp/checkpoints.tsv

Notes

This repo intentionally does not ship raw WSIs or protected clinical data.
OncoKB and GDC access are governed by their respective terms; keep tokens out of git history.
The codebase is built around downloadable artifacts and reproducible workflows exposed via the GOLDMARK API: https://artificialintelligencepathology.org/

Name		Name	Last commit message	Last commit date
Latest commit History 54 Commits
configs		configs
docs		docs
examples		examples
goldmark		goldmark
images		images
paper		paper
scripts		scripts
targets		targets
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
environment.yml		environment.yml
requirements-encoders.txt		requirements-encoders.txt
requirements-wsi.txt		requirements-wsi.txt
requirements.txt		requirements.txt

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

GOLDMARK — Manuscript Reference Pipeline - https://artificialintelligencepathology.org/

Manuscript Figures (Fig 1-3)

Start here (real GDC download + end-to-end pipeline)

Reciprocal TCGA→external end-to-end runs (mutation labels + external inference)

Recommended: TCGA-LUAD KRAS (5×70/30 splits) → external LUAD inference

Generic SLURM + nohup launchers (configurable project/gene/encoder)

Output layout (important files and where to find them)

File schemas (column names)

Canonical pathology foundation model encoders

Repository layout

Installation

Quickstart (typical flow)

1) Targets (TCGA example)

2) Tiling (20x/40x coordinates)

3) Feature extraction + QC metadata

4) Training (reference gated-attention MIL head)

5) Inference + external inference + attention export

Reference figures/tables

Manuscript tasks (Table 1)

Launch all manuscript tasks (optional)

Run a full project workflows

A) Build a labeled user manifest

B) Tiling + feature extraction (canonical encoders)

C) Pull a checkpoint from GOLDMARK API (default: best external inference AUC)

D) Run inference (GMA) on user slides

E) Option: run all available checkpoints for `(gene, encoder)`

Notes

About

Releases

Packages

Contributors

Languages

Folders and files

Latest commit

History

Repository files navigation

GOLDMARK — Manuscript Reference Pipeline - https://artificialintelligencepathology.org/

Manuscript Figures (Fig 1-3)

Start here (real GDC download + end-to-end pipeline)

Reciprocal TCGA→external end-to-end runs (mutation labels + external inference)

Recommended: TCGA-LUAD KRAS (5×70/30 splits) → external LUAD inference

Generic SLURM + nohup launchers (configurable project/gene/encoder)

Output layout (important files and where to find them)

File schemas (column names)

Canonical pathology foundation model encoders

Repository layout

Installation

Quickstart (typical flow)

1) Targets (TCGA example)

2) Tiling (20x/40x coordinates)

3) Feature extraction + QC metadata

4) Training (reference gated-attention MIL head)

5) Inference + external inference + attention export

Reference figures/tables

Manuscript tasks (Table 1)

Launch all manuscript tasks (optional)

Run a full project workflows

A) Build a labeled user manifest

B) Tiling + feature extraction (canonical encoders)

C) Pull a checkpoint from GOLDMARK API (default: best external inference AUC)

D) Run inference (GMA) on user slides

E) Option: run all available checkpoints for (gene, encoder)

Notes

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

E) Option: run all available checkpoints for `(gene, encoder)`