Skip to content

feat(transcribe): add MPS device support for Apple Silicon GPU acceleration #88

Description

@missabawse

Context

The current hardware detection (scripts/config/hardware.ts) maps all macOS machines to mlDevice: 'cpu' since CUDA is unavailable on Apple Silicon. As a result, transcribe-audio.js runs faster-whisper on CPU on M2/M3 Macs.

Apple Silicon GPUs expose Metal Performance Shaders (MPS) which can accelerate PyTorch workloads. The original OpenAI Whisper library supports --device mps. However, faster-whisper (CTranslate2) does not currently support MPS — see openai/whisper#2504.

Options to investigate

  1. Switch to openai-whisper on Apple Silicon — uses PyTorch MPS backend, supports --device mps. Trade-off: slower than faster-whisper on CUDA, no built-in VAD filter.
  2. Switch to mlx-whisper — Apple's MLX framework, natively targets Apple Silicon GPU (both CPU+GPU unified memory). Fastest option on M2/M3. Requires a new Python script.
  3. Wait for CTranslate2 MPS support — track CTranslate2 issue tracker for MPS backend addition.

Proposed change

  • Add supportsMps: boolean and update mlDevice to 'cuda' | 'mps' | 'cpu' in HardwareProfile
  • Map darwin + arm64 → mlDevice: 'mps' once a supported backend is chosen
  • Add a new transcribe-mps.py script (mlx-whisper) or update transcribe-gpu.py to handle --device mps

Acceptance criteria

  • detectHardware() returns mlDevice: 'mps' on darwin/arm64
  • Transcription uses Apple Silicon GPU on M2/M3
  • VAD filter equivalent is available on the chosen backend
  • Unit tests updated for the new mlDevice values

Reference

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions