Skip to content

feat: ffmpeg GPU-accelerated encoding path for video acquisition#6

Open
tabedzki wants to merge 1 commit into
feature/video-acquisitionfrom
feature/video-acquisition-ffmpeg
Open

feat: ffmpeg GPU-accelerated encoding path for video acquisition#6
tabedzki wants to merge 1 commit into
feature/video-acquisitionfrom
feature/video-acquisition-ffmpeg

Conversation

@tabedzki

@tabedzki tabedzki commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

🤖 AI text below 🤖

Adds an optional ffmpeg recording backend to the camera path (#5), selected by RigParameters.useFFmpeg, supporting NVENC (NVIDIA GPU) and libx264 (CPU). Falls back to the VideoWriter .mj2 path with a warning if ffmpeg isn't on PATH. Stacked on feature/video-acquisition (#5) — merge that first.

How it works

FLIR camera ─IMAQ buffer─► FramesAcquiredFcn ─raw gray bytes─► ffmpeg stdin ─H.264─► .mp4
                           (pushFramesToFFmpeg)               (NVENC or libx264)
  • configureSingleCamera.m: launches ffmpeg via java.lang.ProcessBuilder for a real handle on its stdin; builds the command as a token list; drains ffmpeg's merged stdout/stderr on a timer so its pipe buffer can't deadlock encoding; forces -pix_fmt yuv420p on output for broad player compatibility with the grayscale source; sets LoggingMode='memory' so frames are buffered for the callback. NVENC uses bitrate control, libx264 uses CRF.
  • startVideoAcquisition.m: registers a FramesAcquiredFcn that drains the IMAQ buffer and streams raw frames to ffmpeg's stdin; resolves the actual .mp4 output filename.
  • pushFramesToFFmpeg.m: shared frame-drain helper; transposes each frame so column-major serialisation matches ffmpeg rawvideo gray row-major byte order; warns (doesn't crash the experiment loop) on a broken pipe.
  • stopVideoAcquisition.m: flushes remaining frames, closes stdin (EOF → ffmpeg finalizes the file), waits for the process, tears down the timer.
  • checkCameraRigParameters.m / RigParameters.m.example: validation + NVENC/libx264 config blocks.

Tests (hardware-free — no Image Acquisition Toolbox, no camera)

sensors/camera/tests/ drives the real transport through a real ffmpeg subprocess with a mocked camera, and asserts a readable .mp4 is produced with a marker pixel at the correct coordinate (catches transpose / byte-order regressions).

Run it:

# headless / CI (exits non-zero on failure)
matlab -batch "addpath('sensors/camera/tests'); runCameraTests"

or from the MATLAB command window: cd sensors/camera/tests; smokeTestFFmpegTransport.

Verified passing under MATLAB R2023b + ffmpeg 8.1 (libx264):

PASS: ffmpeg produced a non-empty file
PASS: file is readable, 10 frames, 64x48
PASS: dimensions preserved (64x48)
PASS: marker at (11,37) read back at (11,37) -- byte order correct
ALL CHECKS PASSED

See sensors/camera/tests/README.md for details and ffmpeg install hints per platform. The live FLIR/GenTL hardware calls still require the Windows rig and are not covered by these tests.

Adds an optional ffmpeg recording backend for the camera path, selected by
RigParameters.useFFmpeg, supporting NVENC (NVIDIA GPU) and libx264 (CPU).
Falls back to the VideoWriter (Motion JPEG 2000) path with a warning if
ffmpeg is not on the system PATH.

How it works:

    FLIR camera -> IMAQ buffer -> FramesAcquiredFcn -> ffmpeg stdin -> .mp4
                                  (pushFramesToFFmpeg)  (NVENC/libx264)

- configureSingleCamera.m: launches ffmpeg via java.lang.ProcessBuilder to
  obtain a real handle on its stdin; builds the command as a token list;
  drains ffmpeg's merged stdout/stderr on a timer so its pipe buffer cannot
  deadlock encoding; forces -pix_fmt yuv420p on output for broad player
  compatibility with the grayscale source; sets LoggingMode='memory' so
  frames are buffered for the callback. NVENC uses bitrate control, libx264
  uses CRF.
- startVideoAcquisition.m: registers a FramesAcquiredFcn that drains the IMAQ
  buffer and streams raw frames to ffmpeg's stdin; resolves the actual output
  filename (ffmpeg writes .mp4).
- pushFramesToFFmpeg.m: shared frame-drain helper; transposes each frame so
  column-major serialisation matches ffmpeg rawvideo 'gray' row-major byte
  order; warns (does not crash the experiment loop) on a broken pipe.
- stopVideoAcquisition.m: flushes remaining frames, closes stdin (EOF lets
  ffmpeg finalize the file), waits for the process, and tears down the timer.
- checkCameraRigParameters.m: validates ffmpegEncoder when useFFmpeg=true;
  warns (not errors) if ffmpeg is not on PATH.
- RigParameters.m.example: NVENC and libx264 configuration blocks.

Tests (hardware-free, no Image Acquisition Toolbox or camera required):
- sensors/camera/tests/smokeTestFFmpegTransport.m drives the real transport
  through a real ffmpeg subprocess with a mocked camera and verifies a
  readable .mp4 is produced with a marker pixel at the correct coordinate
  (catches transpose / byte-order regressions).
- FakeImaqCamera.m mocks the IMAQ object; runCameraTests.m is a headless/CI
  runner; README.md documents how to run them.

Run: matlab -batch "addpath('sensors/camera/tests'); runCameraTests"

Assisted-by: ClaudeCode:claude-opus-4.8
@tabedzki
tabedzki force-pushed the feature/video-acquisition-ffmpeg branch from 85941d4 to aa3c98e Compare June 18, 2026 20:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant