GPU-accelerated video encoding via ffmpeg (NVENC) as alternative to VideoWriter

[tabedzki]

Summary

Add an optional ffmpeg-based video encoding path as an alternative to MATLAB's VideoWriter (Motion JPEG 2000), enabling GPU-accelerated H.264/H.265 encoding via NVENC on Windows rigs with NVIDIA GPUs.

Motivation

MATLAB's VideoWriter is CPU-only and produces large .mj2 files. For labs recording at higher frame rates (>60fps) or wanting better storage efficiency, H.264 via NVENC offers:

• ~10–50× smaller files vs. Motion JPEG 2000 at comparable quality
• Encoding offloaded to GPU, keeping CPU free for ViRMEn's render loop
• Flexible quality/bitrate control

Note: This is a secondary enhancement. The primary video acquisition feature (using VideoWriter) is tracked in a separate issue. This should be built on top of that foundation.

Design

RigParameters additions

% Set useFFmpeg = true and fill in ONE of the two encoder blocks below.
% ffmpeg must be on the system PATH. Falls back to VideoWriter with a warning if unavailable.
useFFmpeg       = false

% -- NVENC (NVIDIA GPU, primary target for Windows rigs) --
%    ffmpegEncoder   = 'h264_nvenc'
%    ffmpegPreset    = 'fast'      % 'fast' | 'medium' | 'slow' | 'lossless'
%    ffmpegBitrate   = '5M'        % target bitrate; higher = better quality
%    ffmpegCRF       = []          % leave empty (NVENC uses bitrate, not CRF)

% -- libx264 (CPU fallback, no GPU required) --
%    ffmpegEncoder   = 'libx264'
%    ffmpegPreset    = 'fast'      % 'ultrafast' | 'fast' | 'medium' | 'slow'
%    ffmpegCRF       = 23          % 0-51, lower = better; 18 ≈ visually lossless
%    ffmpegBitrate   = []          % leave empty (libx264 uses CRF, not bitrate)

% Power-user escape hatch — appended verbatim to ffmpeg command:
%    ffmpegExtraArgs = ''

Timing challenge (must solve before merging)

The key difficulty with ffmpeg is accurately capturing when video acquisition begins. With VideoWriter, vr.timeElapsedVideoStart = toc(vr.preTic) reliably reflects frame 0. With ffmpeg launched as a subprocess, there is a 100–500ms startup delay before the process begins consuming frames, introducing an unknown offset.

Proposed solution — pre-warm ffmpeg:

1. Launch the ffmpeg process during initialization(), before capturing vr.timeElapsedVideoStart
2. Poll ffmpeg's stderr until it signals readiness (blocking on stdin)
3. Only then capture the timestamp and call start(vr.v)

This moves ffmpeg startup latency into the pre-experiment phase where timing doesn't matter, and reduces jitter between "timestamp captured" and "first frame consumed" to <5ms.

Alternative — hardware TTL sync: FLIR cameras can emit a GPIO pulse at frame 0. Recording this on a NI-DAQ digital input gives microsecond-accurate sync independent of software latency. Worth documenting as an option for labs that have the wiring.

Implementation sketch

configureSingleCamera.m gains an ffmpeg branch: opens a background ffmpeg process via a pipe, returns a struct {type='ffmpeg', pipe=..., filename=...} instead of a videoinput object. startVideoAcquisition / stopVideoAcquisition detect the type and handle accordingly.

Requirements

• ffmpeg on system PATH (check with system('ffmpeg -version'))
• NVIDIA GPU with NVENC support (for h264_nvenc path)
• Built on top of the VideoWriter-based acquisition (see related issue)

Out of scope

• Apple VideoToolbox (macOS) — PureVirmen is Windows-only
• AMD AMF encoder
• Screen capture (this is hardware frame grab from a FLIR camera, not screen recording)