Repro (macOS, vaxrank with --input-lens)
```
vaxrank --input-lens HugoLo_IPRES_2016-Pt02....tsv --output-peptide x.fasta
```
Process aborts with:
```
OMP: Error #15: Initializing libomp.dylib, but found libomp.dylib already initialized.
[1] PID abort vaxrank --input-lens
```
OR (with `KMP_DUPLICATE_LIB_OK=TRUE` set) silently exits with code 139 (SIGSEGV) without running pepsickle / construct writers.
Cause
Pepsickle pulls in torch. macOS torch ships its own `libomp.dylib`. By the time `_load_default_predictor` is called, pandas / numpy / pyarrow have already loaded a different `libomp` via OpenBLAS / MKL. The second OpenMP runtime to initialize either:
- aborts the process at init time (the loud failure), OR
- continues with two OpenMP runtimes coexisting and segfaults later when their internal state collides during inference (the silent failure).
`KMP_DUPLICATE_LIB_OK=TRUE` (the LLVM-documented workaround) silences the abort but doesn't prevent the segfault.
Current state (PR #262, post-this-fix)
Pepsickle credibility tagging is now off by default (`--processing-aware-annotation=False`). Users opt in when their environment has a single OpenMP runtime. Linux installs typically work; macOS users can try `MKL_THREADING_LAYER=GNU` or run pepsickle separately.
Robust fix options
-
Subprocess isolation: spawn a fresh Python process for pepsickle inference (no pandas / numpy in the child). Pickle source sequences in, pickle cleavage probabilities out. Adds ~1s subprocess-launch overhead per run; fully solves the libomp clash. Probably the right move.
-
Use ONNX runtime instead of torch: pepsickle's epitope model could be exported to ONNX and run via `onnxruntime` (single OpenMP, no torch). Requires upstream work in pepsickle.
-
Pin a single libomp: ship vaxrank with a wheel that links every OpenMP-using dep against the same libomp. Brittle; couples vaxrank to torch's libomp version.
-
Use a non-OpenMP backend for inference: torch.set_num_threads(1) + force CPU mode + disable parallelism. Doesn't fully solve the dual-runtime issue but mitigates some failure modes.
Acceptance
- Pepsickle credibility tagging reliable on macOS without env-var dancing.
- Default returns to on (currently off as a defensive measure).
- CI test that exercises the LENS + pepsickle path end-to-end on macOS — currently we opt out via `--no-processing-aware-annotation` because of this issue.
Repro (macOS, vaxrank with --input-lens)
```
vaxrank --input-lens HugoLo_IPRES_2016-Pt02....tsv --output-peptide x.fasta
```
Process aborts with:
```
OMP: Error #15: Initializing libomp.dylib, but found libomp.dylib already initialized.
[1] PID abort vaxrank --input-lens
```
OR (with `KMP_DUPLICATE_LIB_OK=TRUE` set) silently exits with code 139 (SIGSEGV) without running pepsickle / construct writers.
Cause
Pepsickle pulls in torch. macOS torch ships its own `libomp.dylib`. By the time `_load_default_predictor` is called, pandas / numpy / pyarrow have already loaded a different `libomp` via OpenBLAS / MKL. The second OpenMP runtime to initialize either:
`KMP_DUPLICATE_LIB_OK=TRUE` (the LLVM-documented workaround) silences the abort but doesn't prevent the segfault.
Current state (PR #262, post-this-fix)
Pepsickle credibility tagging is now off by default (`--processing-aware-annotation=False`). Users opt in when their environment has a single OpenMP runtime. Linux installs typically work; macOS users can try `MKL_THREADING_LAYER=GNU` or run pepsickle separately.
Robust fix options
Subprocess isolation: spawn a fresh Python process for pepsickle inference (no pandas / numpy in the child). Pickle source sequences in, pickle cleavage probabilities out. Adds ~1s subprocess-launch overhead per run; fully solves the libomp clash. Probably the right move.
Use ONNX runtime instead of torch: pepsickle's epitope model could be exported to ONNX and run via `onnxruntime` (single OpenMP, no torch). Requires upstream work in pepsickle.
Pin a single libomp: ship vaxrank with a wheel that links every OpenMP-using dep against the same libomp. Brittle; couples vaxrank to torch's libomp version.
Use a non-OpenMP backend for inference: torch.set_num_threads(1) + force CPU mode + disable parallelism. Doesn't fully solve the dual-runtime issue but mitigates some failure modes.
Acceptance