🧬 PfamIE

The entire Pfam universe as an on-device inference engine: paste a sequence, get a family, offline.

🌐 Website

✉️ Contact

🐙 GitHub

PfamIE turns all 30,031 Pfam families into an inference engine that fits in your pocket. A quantised ESM-2 protein language model runs on the Apple Neural Engine, embeds any sequence you paste, and classifies it against a pre-baked matrix of family centroid embeddings: family assignment with no HMM, no server, and no network. Around that core sit five views, from a 3D flythrough of the whole Pfam universe to an offline atlas you can search in plain English.

Why it matters: every existing route to a Pfam assignment goes through a server. HMMER against Pfam-A means a web queue or a local install and a 1.5 GB HMM library, and neither works on a train, in a hospital, at a conference, or anywhere the sequence in front of you is not yours to upload. PfamIE puts the whole search space on the device, answers a 536-residue protein in about a second, and states how often it is wrong rather than leaving you to guess. It is useful for anyone who wants a first-pass family call, a domain architecture, or a functional lead for an uncharacterised protein: bench scientists triaging a hit, structural biologists sizing up a construct, and teachers who want the shape of protein sequence space on a screen they can spin.

📸 What it looks like


Galaxy. Every family, coloured by clan, the dark proteome drawn dim.	Oracle. Human SRC: the kinase domain at 91%, read from residues 281 to 536 (InterPro says 271 to 518).	Field Guide. "breaks down plastic" finds PETase, with no network.

Grammarian. SH2 and the domains it travels with, across 1,896 architectures.	Prospector. 6,925 families with no assigned function, largest first.	visionOS. The same five views as a window in the room.

🥽 The Pfam universe in a volume

On visionOS the Galaxy leaves the window. All 30,031 families render as a RealityKit volume you can walk around and lean into, clans holding their colour, the dark proteome dim, with the family card following as an ornament below.

Two things make that work at all:

Batched geometry. RealityKit has no point primitive, and the obvious implementation (one ModelEntity per family) is 30,031 entities and 30,031 draw calls, which never reaches frame rate. Families are batched into one generated mesh per clan, so a scene of 30,031 objects becomes at most 892 draws.
Tetrahedra, not billboards. A flat quad has to be reoriented every frame in a volume the viewer walks around, and vanishes edge-on when it is not. A four-triangle tetrahedron per family is cheaper than the reorientation and correct from every angle.

The other four views open as ordinary windows reusing the shared SwiftUI code untouched. There is no visionOS fork of the app: one RootView adapts by size class across iPhone, iPad, Mac and Vision Pro.

⚡ On the Neural Engine

The whole point of the app is that a transformer runs locally, fast enough that classification feels like a lookup. Measured on an M1 Max:

Reproduce with ./Tools/benchmark.sh. They are excluded from the normal test run on purpose: see the note below the table.

Operation	Time	Notes
ESM-2 t12-35M embedding, 512 tokens, Neural Engine	31.3 ms
the same on CPU	76.1 ms	2.4x slower
Top-20 search over 30,031 families	1.8 ms	Accelerate `sgemv` over a memory-mapped int8 matrix
Field Guide query over 30,031 descriptions	8.6 ms	MiniLM embed plus a second `sgemv`
Full classification of a 536-residue protein	~1.2 s	33 windows across four scales, end to end

The smaller t6-8M tier runs at 6.0 ms per window and classifies the same protein in 247 ms, four times faster. It was the shipped model until the measurements below said otherwise.

Benchmarks only mean something run alone, which took two goes to get right. Left in the normal test run, the other suites drive Core ML concurrently, the Neural Engine queue saturates, and a 31 ms embedding measures at 494 ms while the CPU path stays flat: the benchmark concluded the ANE was five times slower. Interleaving the two paths to equalise the load made it worse again, because alternating between an ANE-resident model and a CPU one reloads the Neural Engine context every call. Consecutive blocks, in an opt-in suite that runs on its own, is the only arrangement that measures the model.

How the models are shaped for the ANE

Fixed input length. ESM-2 is converted at exactly 512 tokens and MiniLM at 256. Flexible shapes make the Neural Engine ineligible and, on Metal, force a graph recompilation per distinct shape.
8-bit palettisation for both models. It halves ESM-2 t12 from 69 MB to 34 MB and MiniLM from 45 MB to 23 MB, and costs nothing measurable: identical Neural Engine latency, and end-to-end top-1 unchanged at 0.78 on held-out sequences through the converted model.
The whole recipe is compiled into the graph. Masked mean pooling, the whitening transform and L2 normalisation are all graph operations, so the model takes tokens and returns a finished unit vector. Two benefits: Swift never has to index a multi-dimensional MLMultiArray (Core ML pads rows, and indexing by index * width instead of strides silently shifts every value after the first), and the on-device vector is bit-comparable with the one the forge computed.

The float16 trap that ships silently

Baking the whitening into the graph made the Neural Engine return all zeros while the CPU path looked perfect. The whitening matrix has entries up to 200, so the whitened vector's norm runs into the hundreds and its sum of squares exceeds float16's 65,504. The ANE computes in float16, so the norm overflowed to infinity and the normalisation divided by it.

The output is L2 normalised, so scaling the whitening matrix is mathematically free. The forge now scales it so a typical sequence lands near unit length, and asserts parity on ComputeUnit.ALL as well as CPU, failing the build rather than shipping a model that returns zeros on device. A check that has only ever run on the CPU is not a check.

🔬 The science

Classification without an HMM

Pfam assignment is conventionally a profile HMM search. PfamIE does something different: it embeds the query with a protein language model and finds the nearest family centroid in that embedding space. The centroids are built once, in the forge, from Pfam's own seed alignments.

Two findings shaped the result, and both are the kind that only appear when you measure.

1. Mean-pooled ESM-2 embeddings are strongly anisotropic. Raw, every family sits at cosine ~0.97 to every other, and short families collapse into a single hub. The geometry is nearly meaningless. Whitening the centroid covariance fixes it:

	top-1	top-5	mean nearest-neighbour cosine
Raw mean-pooled	0.514	0.641	0.968
Whitened	0.715	0.810	0.677

A shrinkage sweep put the plateau at eps = 1e-5, which also caps eigenvalue amplification at a conservative 34x. Without whitening the Galaxy is a featureless ball and the Oracle is close to a coin toss.

2. How you choose the centroid's sequences matters more than how many. Picking the seed sequences closest to the median length looks sensible and does the opposite: it selects near-duplicates. PF00062 (Lys) covers both c-type lysozymes and the alpha-lactalbumins, and typicality chose six lactalbumins out of eight, leaving a centroid that could not recognise hen lysozyme (rank 42 of 30,031). Seed alignments are ordered roughly phylogenetically, so an even sweep through them costs nothing and spans the family. Hen lysozyme moved to rank 2.

A real protein is not a trimmed domain

The single most important measurement in this project is the gap between two benchmarks:

Benchmark	top-1	What it measures
Held-out Pfam seed sequences	0.75	The index against its own kind
Real UniProt proteins	0.49	What the app is actually handed

Seed sequences are trimmed to domain boundaries and drawn from the alignments the centroids were built from. Real proteins carry signal peptides, linkers, disordered tails and other domains, and embedding one end to end averages all of that into the answer. Quoting the seed number would have been comfortable and wrong, and every figure in this README is the real-protein one.

Acting on it changed the engine. Scanning at several window widths, measured on 400 real single-domain proteins:

Approach	top-1	top-5
Whole sequence embedded end to end	0.285	0.435
One 160-residue sliding window	0.425	0.548
Four widths: 96 / 160 / 256 / 384	0.535	0.635

No single width fits Pfam, whose domains run from about 30 residues to several hundred. The headline answer now comes from the best-reading window and says which residues it read, which is both more accurate and a more useful thing to tell a user than a whole-sequence guess.

Confidence you can act on

A ranking without a calibrated confidence is an invitation to over-trust it. The softmax temperature and the band thresholds are fitted on 2,500 real UniProt proteins, with thresholds chosen on 1,875 and every figure below measured on the 625 held back:

Band	Share of queries	Correct
High (p ≥ 0.85)	19.7%	93.5%
Moderate (p ≥ 0.45)	31.2%	69.7%
Low (p ≥ 0.20)	30.6%	23.6%
No confident family (p < 0.20)	18.6%	5.2%

Every result carries the measured accuracy of its own band, and the bottom band exists so the app can say "no confident family" instead of naming the least-bad of 30,031 options. A test fails the build if the shipped calibration ever matches the seed-fitted one, because that would overstate accuracy by about 25 points.

Domain grammar is real and strongly conserved

Across 71,573 domain pairs sharing at least ten proteins, 97.7% have an invariant N-to-C order. SH3 is always N-terminal to SH2; the tyrosine kinase domain is always C-terminal to it. The Grammarian says so in words for the invariant majority and keeps a percentage for the 2.3% that genuinely vary, because those are the interesting ones.

Honest limits

ESM-2 t12-35M is 33.5 million parameters. 0.49 top-1 on real proteins against 30,031 classes is respectable for that size and is not an HMM replacement. The next tier up (t30-150M) is untested here.
Domain detection is the binding limit, not localisation: in a multi-domain protein the scanner finds 47.8% of the true domains, at 0.84 precision. Boundaries for the domains it does find are median 61 residues out.
Nearest-neighbour proximity in the Prospector is a reason to look, not evidence of function, and the wording throughout that tab says so.
There is no hub pathology: the most-connected family is the nearest neighbour of only 19 of 30,031, and the top 15 absorb 0.6% of nearest-neighbour slots.

✨ The five views

Tab	What it is for
Galaxy	All 30,031 families as a 3D point cloud, clans as coloured regions, the dark proteome drawn dim. Tap a star, open its card, or watch your last query drop in as an amber comet.
Oracle	Paste a sequence or open a FASTA. Multi-scale scanning returns the family, the clan, the N-to-C architecture and a calibrated confidence.
Grammarian	Which domains travel with which, in what order, and how often: co-occurrence over 151,818 real architectures, and "what else is built like mine?"
Prospector	The 6,925 families with no known function, each with its nearest annotated neighbours as an explicitly hypothesis-flavoured lead.
Field Guide	The offline Pfam atlas. Plain-English queries work with no network, through a bundled MiniLM, alongside FTS5 for names and accessions.

Every family reference anywhere carries the same four actions (Open card · Show in Galaxy · Similar architectures · View structure), so no tab is a dead end. Any family opens an AlphaFold model with its Pfam domain highlighted: AlphaFold uses UniProt numbering, so Pfam boundaries map onto the structure with no residue-mapping step.

🧱 How it is built

PfamIE/
├── forge/                Python: builds every asset the app ships (Phase 0)
├── PfamIEKit/            SwiftPM package: engine, data, and all shared views
│   └── Sources/PfamIEKit/
│       ├── Engine/       tokenisers, Core ML embedders, centroid index, domain scanner
│       ├── Data/         SQLite store, AlphaFold client, models
│       ├── Core/         engine facade, router, theme, app environment
│       └── Views/        Galaxy, Oracle, Grammarian, Prospector, Field Guide, Structure
├── Apps/                 thin per-platform targets
│   ├── iOS/              iPhone tabs and iPad three-column, one target
│   ├── macOS/            native SwiftUI, sidebar, menu commands, FASTA on the dock icon
│   ├── visionOS/         volumetric Galaxy plus windowed tabs
│   └── watchOS/          companion glance over WatchConnectivity
├── Tools/                project generation and bundle verification
└── project.yml           XcodeGen spec: the .xcodeproj is derived, never hand-edited

Component	Notes
`ProteinEmbedder`	ESM-2 t12-35M as Core ML, 8-bit palettised, fixed 512 tokens, Neural Engine eligible, returns a finished unit vector.
`Int8Matrix`	Memory-mapped int8 with a per-row scale and a chunked Accelerate `sgemv`. The scale is applied to the result, not to every element.
`DomainScanner`	Multi-scale windows, per-scale merging, then greedy non-overlapping selection by confidence.
`SemanticSearch`	MiniLM as Core ML plus a Swift WordPiece tokeniser over 30,031 description embeddings.
`PfamStore`	Raw sqlite3, no wrapper library. FTS5 alongside the semantic search, not instead of it.
`Router`	One closed `Destination` enum. Every context menu and card action resolves to a case of it.

All five platforms share one engine. PfamIEKit imports no UIKit or AppKit except the two files behind the structure viewer, which need a web view and say so. The watch carries no assets and no model: the phone classifies and sends a summary.

⚗️ The forge

forge/ builds everything the app ships, from the Pfam 38.2 release flatfiles and the InterPro API. Reading the flatfiles rather than the API replaces 30,031 requests with one 52 MB download.

python3.12 -m venv .venv
.venv/bin/pip install -r forge/requirements.txt

mkdir -p assets/raw && cd assets/raw
for f in Pfam-A.clans.tsv.gz Pfam-C.gz Pfam-A.hmm.dat.gz Pfam-A.seed.gz; do
  curl -sLO "https://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/$f"
done
cd ../..

.venv/bin/python forge/stage_metadata.py        # families, clans, representative sequences
.venv/bin/python forge/stage_interpro.py ida    # domain architectures (about 100 min, resumable)
.venv/bin/python forge/stage_interpro.py counters
.venv/bin/python forge/stage_embed.py           # ESM-2 over 368,451 sequences
.venv/bin/python forge/stage_transform.py       # whitening and the seed reference calibration
.venv/bin/python forge/stage_project.py         # 3D UMAP for the Galaxy
.venv/bin/python forge/stage_descemb.py         # MiniLM over every abstract
.venv/bin/python forge/stage_coreml.py          # Core ML conversion and parity checks
.venv/bin/python forge/stage_calibrate_real.py  # confidence, fitted on real proteins
.venv/bin/python forge/stage_sqlite.py
.venv/bin/python forge/stage_emit.py
.venv/bin/python forge/make_icon.py             # the app icon, drawn from the real map

Stage	Produces
`stage_metadata`	30,031 families, 891 clans, 6,925 unknown-function, a UniProt structural representative for every family
`stage_interpro`	151,818 distinct N-to-C architectures and 151,719 co-occurrence edges
`stage_embed`	403,367 ESM-2 embeddings, 16 stratified seed sequences per centroid
`stage_transform`	The 480 x 480 whitening transform
`stage_coreml`	Two `.mlpackage` models, parity asserted on the Neural Engine and end to end against the index
`stage_calibrate_real`	The shipped temperature and confidence bands
`stage_emit`	`centroids.bin`, `umap3d.bin`, `desc_emb.bin`, `manifest.json`

🔧 Building the app

brew install xcodegen
./Tools/generate-project.sh                 # regenerates PfamIE.xcodeproj from project.yml
open PfamIE.xcodeproj

xcodebuild -project PfamIE.xcodeproj -scheme PfamIE-macOS -destination 'platform=macOS' build
swift test -c release --package-path PfamIEKit

Tests need the forge output. Without it the asset-dependent suites skip rather than fail, so a fresh clone still runs green.

Always verify a built bundle. BUILD SUCCEEDED says nothing about the contents, and an app missing its Core ML models builds and signs perfectly:

./Tools/verify-bundle.sh ~/Library/Developer/Xcode/DerivedData/PfamIE-*/Build/Products/Debug/PfamIE.app

The script is negative-tested: strip a model from a copy of the bundle and it must exit 1.

📋 Requirements


Xcode	26.6 or later
Deployment	iOS 18, macOS 15, visionOS 2, watchOS 11
Forge	Python 3.12 (numba and coremltools are not yet reliable on 3.14)
Bundle	About 148 MB: 58 MB database, 26 MB matrices (int8), 57 MB models, 5 MB Mol*
Network	Only for AlphaFold structures, and for InterProScan verification if you opt in. Classification, architecture and search are entirely offline.

✅ To Do

Roadmap for PfamIE, in dependency order. Suggestions welcome.

🔬 Data sources and licences

PfamIE is MIT licensed. Every bundled component is MIT, Apache-2.0 or CC0, so nothing here carries a copyleft or non-commercial term. Full attribution is in THIRD-PARTY-NOTICES.md.

Source	Licence	Used for
Pfam 38.2	CC0 1.0	Family metadata, clans, abstracts, seed alignments
InterPro	EMBL-EBI, freely available	Domain architectures, family sizes, taxonomic breadth
ESM-2	MIT	Protein sequence embeddings
all-MiniLM-L6-v2	Apache-2.0	Description embeddings for offline semantic search
Mol*	MIT	Structure rendering
AlphaFold DB	CC-BY 4.0	Predicted structures, fetched on demand, never redistributed
UniProt	CC-BY 4.0	Benchmark and calibration sequences (forge only)

If PfamIE contributes to published work, please cite the underlying resources rather than this app. The citations are listed in THIRD-PARTY-NOTICES.md.

👤 Author

Marc C. Deller, D.Phil.
Structural biologist & drug discovery scientist

🌐

marcdeller.com

✉️

[email protected]

🐙

github.com/bellcheddar/PfamIE

Name		Name	Last commit message	Last commit date
Apps		Apps
PfamIEKit		PfamIEKit
Tools		Tools
docs		docs
forge		forge
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
LICENSE		LICENSE
README.md		README.md
THIRD-PARTY-NOTICES.md		THIRD-PARTY-NOTICES.md
pfamie_build_plan_v2.md		pfamie_build_plan_v2.md
project.yml		project.yml

Navigation Menu

Repository files navigation

🧬 PfamIE

📸 What it looks like

🥽 The Pfam universe in a volume

⚡ On the Neural Engine

How the models are shaped for the ANE

The float16 trap that ships silently

🔬 The science

Classification without an HMM

A real protein is not a trimmed domain

Confidence you can act on

Domain grammar is real and strongly conserved

Honest limits

✨ The five views

🧱 How it is built

⚗️ The forge

🔧 Building the app

📋 Requirements

✅ To Do

🔬 Data sources and licences

👤 Author

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

🧬 PfamIE

📸 What it looks like

🥽 The Pfam universe in a volume

⚡ On the Neural Engine

How the models are shaped for the ANE

The float16 trap that ships silently

🔬 The science

Classification without an HMM

A real protein is not a trimmed domain

Confidence you can act on

Domain grammar is real and strongly conserved

Honest limits

✨ The five views

🧱 How it is built

⚗️ The forge

🔧 Building the app

📋 Requirements

✅ To Do

🔬 Data sources and licences

👤 Author

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages