Skip to content

Add missing pseudosequences for specific training alleles (~6.6k rows dropped) #294

Description

@iskandr

Summary

A small set of real, specific alleles present in the curated training data are dropped at training time because they have no pseudosequence in allele_sequences.csv (so they don't match a key and can't be encoded for the pan-allele model). These are genuine alleles (mostly non-human species), not serotypes or junk — adding their pseudosequences would let the dropped measurements train.

Data (from curated_training_data, 994,948 rows)

16 unique alleles / 6,663 rows (~0.67%) are dropped because they resolve to no pseudosequence key. Of these, 12 parse as a specific Allele (recoverable with a pseudosequence); 4 are gene-level labels (RT1-A, Mamu-E, H2-D, H2-K) that are inherently too coarse and would stay dropped.

allele rows parses as
Trvu-UB*01:01 5,607 Allele
Mamu-A*07 681 Allele
RT1-A 139 Gene (too coarse)
Ptal-N*01:01 74 Allele
SLA-3*02:02 41 Allele
FLA-E*018:01 36 Allele
Gaga-BF2*13:01 30 Allele
Mamu-E 16 Gene (too coarse)
Eqca-N*006:02 13 Allele
SLA-1*03:02 7 Allele
H2-D 5 Gene (too coarse)
SLA-2*01:06 4 Allele
H2-K*wm7 4 Allele
H2-K*bm8 2 Allele
Mamu-B*04 2 Allele
H2-K 2 Gene (too coarse)

Trvu-UB*01:01 (a trout/fish MHC allele) alone is 5,607 rows — most of the recoverable volume.

Why it matters

These are the actual training rows currently lost to the pan-allele model (not serotypes — serotypes are ≈0% of the curated data; see sibling issue). Recovery requires authoring/adding pseudosequences, not new code:

  • Add pseudosequence rows for the 12 specific alleles to allele_sequences.csv (where structural/sequence data exists), OR
  • Decide they're out of scope (rare non-human species) and document that the drop is expected.

Proposed work

  1. For each of the 12 specific alleles, determine whether a pseudosequence can be derived (the make_allele_sequences.py / filter_sequences.py pipeline under downloads-generation/allele_sequences/).
  2. Regenerate allele_sequences.csv including them; retrain picks them up automatically (training ingestion now canonicalizes against whatever keys exist).
  3. Leave the 4 gene-level labels dropped (correctly too coarse).

Effort

Small-to-medium — mostly data work in the allele_sequences generation pipeline, no model-code changes. Low volume overall (~6.6k rows, dominated by one fish allele), so prioritize accordingly.

How to reproduce the list

Classify curated_training_data.csv.bz2 alleles against allele_sequences.csv keys via mhcflurry.common.canonicalize_allele_series; the unresolved ones are the table above.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions