This repository contains code for processing long-read sequencing cohorts end-to-end - from variant calling and callset integration through phasing and annotation. The pipeline was run on two cohorts: a combined HPRC/HGSVC set (292 samples) and All of Us Phase 1 (1027 samples).
The pipeline covers three broad stages.
- Callset Generation: Per-sample variant calls from DeepVariant (for SNVs/indels), a series of SV callers and TRGT (for tandem repeats) are integrated into cohort-level VCFs and filtered.
- Callset Processing: A series of preprocessing steps convert these VCFs into the desired format. Then, variants are physically phased per sample using HiPhase, and these phase blocks are then linked via backbone phasing.
- Callset Annotation: A suite of annotation workflows characterizes each variant - calling mobile element insertions (MEIs) and deletions (MEDs) using PALMER, SVAN and L1ME-AID, identifying duplications and NUMTs, annotating functional effects via VEP and SVAnnotate, computing in-silico predictor scores, assigning allele frequencies, linking variants to external databases (dbSNP, dbVaR, gnomAD) and more.
Stack: All workflows are written in WDL and executed via Cromwell on Terra, which is built on top of GCP. The code logic is a mixture of Python, R, Bash and Hail.
- Annotations - VCF INFO fields, FORMAT fields and filter definitions.
- Cohort - sample cohorts, sizes and metadata sources.
- Conventions - WDL and Python style conventions.
- Pipeline - end-to-end pipeline description covering callset generation, preprocessing and annotation.
- References - all reference files and their GCS locations.
- Repository Structure - directory layout, Dockerfile build/push conventions, scripts overview and CI/CD.
- Workflows - annotation workflows, annotation utilities and tool wrappers with inputs and outputs.