Context
The current hardware detection (scripts/config/hardware.ts) maps all macOS machines to mlDevice: 'cpu' since CUDA is unavailable on Apple Silicon. As a result, transcribe-audio.js runs faster-whisper on CPU on M2/M3 Macs.
Apple Silicon GPUs expose Metal Performance Shaders (MPS) which can accelerate PyTorch workloads. The original OpenAI Whisper library supports --device mps. However, faster-whisper (CTranslate2) does not currently support MPS — see openai/whisper#2504.
Options to investigate
- Switch to
openai-whisper on Apple Silicon — uses PyTorch MPS backend, supports --device mps. Trade-off: slower than faster-whisper on CUDA, no built-in VAD filter.
- Switch to
mlx-whisper — Apple's MLX framework, natively targets Apple Silicon GPU (both CPU+GPU unified memory). Fastest option on M2/M3. Requires a new Python script.
- Wait for CTranslate2 MPS support — track CTranslate2 issue tracker for MPS backend addition.
Proposed change
- Add
supportsMps: boolean and update mlDevice to 'cuda' | 'mps' | 'cpu' in HardwareProfile
- Map
darwin + arm64 → mlDevice: 'mps' once a supported backend is chosen
- Add a new
transcribe-mps.py script (mlx-whisper) or update transcribe-gpu.py to handle --device mps
Acceptance criteria
Reference
Context
The current hardware detection (
scripts/config/hardware.ts) maps all macOS machines tomlDevice: 'cpu'since CUDA is unavailable on Apple Silicon. As a result,transcribe-audio.jsrunsfaster-whisperon CPU on M2/M3 Macs.Apple Silicon GPUs expose Metal Performance Shaders (MPS) which can accelerate PyTorch workloads. The original OpenAI Whisper library supports
--device mps. However, faster-whisper (CTranslate2) does not currently support MPS — see openai/whisper#2504.Options to investigate
openai-whisperon Apple Silicon — uses PyTorch MPS backend, supports--device mps. Trade-off: slower than faster-whisper on CUDA, no built-in VAD filter.mlx-whisper— Apple's MLX framework, natively targets Apple Silicon GPU (both CPU+GPU unified memory). Fastest option on M2/M3. Requires a new Python script.Proposed change
supportsMps: booleanand updatemlDeviceto'cuda' | 'mps' | 'cpu'inHardwareProfiledarwin + arm64 → mlDevice: 'mps'once a supported backend is chosentranscribe-mps.pyscript (mlx-whisper) or updatetranscribe-gpu.pyto handle--device mpsAcceptance criteria
detectHardware()returnsmlDevice: 'mps'ondarwin/arm64mlDevicevaluesReference