GhostVariant

GhostVariant is a command-line tool for finding repeat-prone sequence contexts and annotating VCF/BCF variants that overlap them.

It is aimed at variant interpretation, benchmarking, and filtering workflows where low-complexity sequence, tandem repeats, and homopolymers are useful stratification features.

Current CLI version: 0.1.0

Features

• scan: detect low-complexity regions, tandem repeats, and homopolymers from a FASTA reference and write a BED file
• annotate: add repeat-region annotations to VCF/BCF records
• Supports FASTA input compressed with gzip
• Supports VCF, VCF.GZ/BGZF, BCF, and BCF.GZ/BGZF through HTSlib
• Optional --filter to write repeat type labels into the VCF FILTER field
• Optional --pad to widen variant overlap queries
• Optional --stats summary TSV for annotate

Repeat sources:

• lowcomplexity: longdust-derived low-complexity regions
• tandem: ATR-style tandem repeats
• homopolymer: perfect or one-base interrupted homopolymer runs
• bed: generic BED3 regions when annotate is run with BED3 input

Quick Start

These examples assume the ghostvariant binary is available after building from source or by using Docker.

Create repeat regions from a reference FASTA:

./ghostvariant scan --fasta ref.fa.gz --out repeats.bed --threads 8 --header

Annotate variants:

./ghostvariant annotate \
  --vcf input.vcf.gz \
  --bed repeats.bed \
  --out annotated.vcf.gz \
  --threads 4 \
  --output-format vcf.gz \
  --stats annotated.stats.tsv

Add repeat types to FILTER as well as INFO:

./ghostvariant annotate \
  --vcf input.vcf.gz \
  --bed repeats.bed \
  --out annotated.filter.vcf.gz \
  --filter

Input and Output

Input FASTA:

• Plain or gzip-compressed FASTA is accepted.
• Use - to read FASTA from stdin.
• Sequence names are taken from the FASTA header up to the first whitespace.
• Lowercase bases are accepted.

Input VCF/BCF:

• Plain VCF, compressed VCF, BCF, and compressed BCF are accepted.
• Use - to read from stdin.
• Chromosome/contig names must match the BED names exactly.

Input BED:

• annotate accepts either full GhostVariant BED output or simple BED3.
• BED coordinates must satisfy 0 <= start <= end.
• BED3 input is treated as generic bed context. Metadata-derived INFO fields are emitted as missing values (.).

Output VCF/BCF:

• Output format is inferred from --out when possible.
• Use --output-format vcf, vcf.gz, bcf, or bcf.gz to force a format.
• Use - to write to stdout. When writing binary output to stdout, set --output-format bcf or --output-format bcf.gz.

Coordinates:

• BED uses 0-based, half-open intervals: [start, end).
• VCF POS is 1-based.
• GV_REPEAT_REGION is reported as 1-based inclusive source_chrom_start_end.

scan

ghostvariant scan detects repeat-prone regions from FASTA input and writes a BED file. FASTA input order is preserved.

Example:

./ghostvariant scan -i ref.fa.gz -o repeats.bed \
  --threads 8 --chunk-size 5000000 --overlap 10000 --header

Options:

• -i, --fasta PATH: input FASTA (gzip supported; - for stdin)
• -o, --out PATH: output BED (- for stdout)
• --threads INT: number of threads (default: 1)
• --chunk-size INT: chunk size in bp (default: 5,000,000)
• --overlap INT: overlap size in bp (default: 10,000; recommended >= 2*max(ws, atr-x); <=0 for auto)
• --header: write BED header
• --min-run INT: homopolymer min run (default: 7)
• --merge-gap INT: homopolymer merge gap (default: 1)
• --atr-m INT, --atr-M INT, --atr-r INT, --atr-l INT, --atr-e INT, --atr-p FLOAT, --atr-x INT: tandem repeat parameters
• --ld-kmer INT, --ld-ws INT, --ld-thres FLOAT, --ld-xdrop INT, --ld-min-start INT, --ld-gc FLOAT, --ld-approx: longdust parameters

BED columns:

chrom start end repeat_type gc_content unit_seq unit_len repeat_len copy_number n_count purity

Column notes:

• repeat_type is lowcomplexity, tandem, or homopolymer.
• gc_content uses A/C/G/T bases as the denominator.
• unit_seq and unit_len describe the repeat unit when applicable.
• repeat_len is the interval length used for the reported repeat.
• copy_number is repeat_len / unit_len for tandem repeats and the number of matching homopolymer bases after gap-merge for homopolymers.
• purity is reported for tandem and homopolymer as the percentage of bases matching the reported repeat unit from the interval start.
• lowcomplexity has purity=unknown.

Chunk boundary policy:

• lowcomplexity and homopolymer intervals are retained when they intersect the chunk core.
• tandem intervals are retained when their start lies in the chunk core.
• This lowcomplexity policy was validated against original longdust on the CHM13 genome.

Homopolymer defaults:

• The default homopolymer threshold (--min-run 7) is consistent with the GIAB/NIST low-complexity stratification convention for perfect homopolymers greater than 6 bp.
• The --merge-gap 1 option is intended to capture one-base interrupted same-base runs, conceptually similar to GIAB imperfect homopolymer stratifications greater than 10 bp with a 1-bp interruption. GhostVariant does not attempt to exactly reproduce GIAB stratification BEDs.
• See the GIAB/NIST genome stratifications for the underlying stratification convention: https://github.com/genome-in-a-bottle/genome-stratifications

ATR input normalization:

• Lowercase bases are uppercased.
• Non-ACGT bases, including IUPAC ambiguous bases and N, are normalized to N.
• N is skipped, not treated as a hard boundary.

Validated longdust-consistency setting:

• For CHM13v2 whole-genome validation against original longdust, --chunk-size 20000000 --overlap 5000000 with the lowcomplexity core-intersection policy reproduced the original longdust BED exactly.
• The CLI defaults remain smaller for general runtime.

annotate

ghostvariant annotate uses a BED file to determine whether each VCF/BCF variant overlaps repeat regions. It adds GhostVariant-managed INFO fields and, optionally, FILTER labels.

Example:

./ghostvariant annotate -i input.vcf.gz -b repeats.bed -o annotated.vcf.gz \
  --threads 4 --buffer-size 10000 --output-format vcf.gz --stats stats.tsv

Options:

• -i, --vcf PATH: input VCF/BCF (- for stdin)
• -b, --bed PATH: input BED (required; GhostVariant BED or BED3)
• -o, --out PATH: output VCF/BCF (- for stdout)
• --threads INT: number of threads (default: 1)
• --buffer-size INT: buffer size (default: 10,000; must be >= 1)
• --info-prefix STR: INFO prefix (default: GV)
• --output-format STR: vcf, vcf.gz, bcf, or bcf.gz
• --stats PATH: write one-row record-level summary TSV
• --filter: write repeat type into FILTER when overlapping. PASS is replaced; existing non-PASS labels are preserved and appended with ;. BED3-derived overlaps use bed.
• --pad INT: symmetric padding in bp applied to the overlap query (default: 0)

INFO fields:

• GV_REPEAT_REGION: overlapping regions as source_chrom_start_end
• GV_TYPE: repeat type
• GV_GC: GC percentage
• GV_UNIT: repeat unit
• GV_UNIT_LENGTH: repeat unit length
• GV_REPEAT_LEN: repeat length
• GV_COPY_NUMBER: copy number
• GV_N_COUNT: number of N bases
• GV_PURITY: repeat purity percentage

Variant footprint rules:

• SNVs and MNVs are evaluated over their reference footprint.
• Indels use POS-1 .. POS-1+len(REF).
• DEL, DUP, INV, and CNV-like alleles use a reference span derived from END and/or allele-specific SVLEN.
• INS and breakend-like alleles are treated as length 1 at POS.
• Multi-allelic records are annotated at record level using the union of ALT-specific footprints.

Re-running annotation:

• Re-running annotate with the same INFO prefix replaces existing {prefix}_* annotations on each record.
• GhostVariant-managed FILTER labels (lowcomplexity, tandem, homopolymer, bed) are recomputed on each run.
• Unrelated FILTER labels are preserved.

Stats output:

--stats writes a one-row TSV with these columns:

total_records annotated_records annotated_frac filtered_records filtered_frac total_snv total_mnv total_indel total_sv total_other annotated_snv annotated_mnv annotated_indel annotated_sv annotated_other overlap_lowcomplexity overlap_tandem overlap_homopolymer overlap_bed multi_overlap_records multi_type_records

--stats - is not supported because stdout may already be used for VCF/BCF output.

Install From Source

Requirements

Build requirements:

• C++17 compiler
• GNU Make
• zlib, bzip2, liblzma, and libcurl development headers

Runtime requirements:

• zlib, bzip2, liblzma, libcurl, and the C++ standard library

Development and validation:

• Python 3 for the test scripts
• Docker, optional, for containerized builds

HTSlib and longdust source snapshots are vendored under third_party/.

On Ubuntu-like systems, install the build dependencies:

sudo apt-get update
sudo apt-get install -y build-essential libbz2-dev libcurl4-openssl-dev liblzma-dev zlib1g-dev

Build GhostVariant:

git clone <repository-url>
cd ghostvariant
make

Check the CLI:

./ghostvariant --version
./ghostvariant --help
./ghostvariant scan --help
./ghostvariant annotate --help

Clean local build outputs:

make clean

Docker

Build the runtime image:

docker build -t ghostvariant:latest .

Show command help:

docker run --rm ghostvariant:latest ghostvariant --version
docker run --rm ghostvariant:latest ghostvariant scan --help
docker run --rm ghostvariant:latest ghostvariant annotate --help

Run scan and annotate with the current directory mounted at /data:

docker run --rm -v "$PWD:/data" ghostvariant:latest \
  ghostvariant scan --fasta /data/input.fasta --out /data/repeats.bed --header

docker run --rm -v "$PWD:/data" ghostvariant:latest \
  ghostvariant annotate --vcf /data/input.vcf.gz --bed /data/repeats.bed \
  --out /data/annotated.vcf.gz --stats /data/annotated.stats.tsv

Interpretation Notes and Limits

• GhostVariant reports sequence-context annotations. It does not classify variants as pathogenic, benign, or sequencing artifacts.
• Output depends on the reference FASTA, repeat parameters, and BED/VCF contig naming. Use the same reference naming convention throughout a workflow.
• BED3 input to annotate is intentionally generic; use scan output when repeat metadata such as GC, unit, copy number, and purity are needed.
• The default scan parameters are intended for general use. For exact longdust-consistency validation, use the setting documented above.
• scan results may differ slightly when --chunk-size, --overlap, or thread scheduling changes, especially for repeats near chunk boundaries. Use fixed scan parameters when comparing outputs across runs.
• Whole-genome scans can be CPU intensive. Increase --threads and use a suitable --chunk-size for production references.

Validation

Build the binary before running the development test suite:

make

The GitHub Actions workflow runs the regression and edge-case tests on Linux and macOS. For local development, see tests/README.md for the maintained test commands and fixture notes.

The test fixtures are intentionally small and are kept under tests/data/ and tests/expected/.

Citation and Methods

If GhostVariant is used in a publication or benchmark, cite the GhostVariant repository or release once a DOI/release archive is available. Also cite the underlying methods and libraries as appropriate for your workflow:

• longdust for low-complexity region detection
• HTSlib for VCF/BCF I/O
• pytrf/ATR-style tandem repeat detection, for the tandem repeat algorithmic reference
• GIAB/NIST genome stratifications when comparing against those conventions

See third_party/longdust-1.4/README.md and third_party/htslib-1.22.2/README for upstream citation details.

Licenses

GhostVariant is released under the MIT License. See LICENSE.

Third-party components are documented in NOTICE and LICENSES/. The source tree currently includes:

• longdust 1.4 source under third_party/longdust-1.4 (MIT)
• kseq.h bundled with longdust (MIT)
• pytrf 1.4.2 algorithmic reference reimplemented in src/tandem/atr.cpp (MIT)
• htslib 1.22.2 under third_party/htslib-1.22.2 (MIT/Expat for most files; Modified 3-Clause BSD for cram/)
• htscodecs bundled under htslib (third_party/htslib-1.22.2/htscodecs) with BSD-style, public-domain, and CC0 notices as described by upstream

When redistributing source or binaries, include LICENSE, NOTICE, and the relevant files under LICENSES/.