feat: ffmpeg GPU-accelerated encoding path for video acquisition#6
Open
tabedzki wants to merge 1 commit into
Open
feat: ffmpeg GPU-accelerated encoding path for video acquisition#6tabedzki wants to merge 1 commit into
tabedzki wants to merge 1 commit into
Conversation
Adds an optional ffmpeg recording backend for the camera path, selected by
RigParameters.useFFmpeg, supporting NVENC (NVIDIA GPU) and libx264 (CPU).
Falls back to the VideoWriter (Motion JPEG 2000) path with a warning if
ffmpeg is not on the system PATH.
How it works:
FLIR camera -> IMAQ buffer -> FramesAcquiredFcn -> ffmpeg stdin -> .mp4
(pushFramesToFFmpeg) (NVENC/libx264)
- configureSingleCamera.m: launches ffmpeg via java.lang.ProcessBuilder to
obtain a real handle on its stdin; builds the command as a token list;
drains ffmpeg's merged stdout/stderr on a timer so its pipe buffer cannot
deadlock encoding; forces -pix_fmt yuv420p on output for broad player
compatibility with the grayscale source; sets LoggingMode='memory' so
frames are buffered for the callback. NVENC uses bitrate control, libx264
uses CRF.
- startVideoAcquisition.m: registers a FramesAcquiredFcn that drains the IMAQ
buffer and streams raw frames to ffmpeg's stdin; resolves the actual output
filename (ffmpeg writes .mp4).
- pushFramesToFFmpeg.m: shared frame-drain helper; transposes each frame so
column-major serialisation matches ffmpeg rawvideo 'gray' row-major byte
order; warns (does not crash the experiment loop) on a broken pipe.
- stopVideoAcquisition.m: flushes remaining frames, closes stdin (EOF lets
ffmpeg finalize the file), waits for the process, and tears down the timer.
- checkCameraRigParameters.m: validates ffmpegEncoder when useFFmpeg=true;
warns (not errors) if ffmpeg is not on PATH.
- RigParameters.m.example: NVENC and libx264 configuration blocks.
Tests (hardware-free, no Image Acquisition Toolbox or camera required):
- sensors/camera/tests/smokeTestFFmpegTransport.m drives the real transport
through a real ffmpeg subprocess with a mocked camera and verifies a
readable .mp4 is produced with a marker pixel at the correct coordinate
(catches transpose / byte-order regressions).
- FakeImaqCamera.m mocks the IMAQ object; runCameraTests.m is a headless/CI
runner; README.md documents how to run them.
Run: matlab -batch "addpath('sensors/camera/tests'); runCameraTests"
Assisted-by: ClaudeCode:claude-opus-4.8
tabedzki
force-pushed
the
feature/video-acquisition-ffmpeg
branch
from
June 18, 2026 20:42
85941d4 to
aa3c98e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Adds an optional ffmpeg recording backend to the camera path (#5), selected by
RigParameters.useFFmpeg, supporting NVENC (NVIDIA GPU) and libx264 (CPU). Falls back to the VideoWriter.mj2path with a warning if ffmpeg isn't on PATH. Stacked onfeature/video-acquisition(#5) — merge that first.How it works
configureSingleCamera.m: launches ffmpeg viajava.lang.ProcessBuilderfor a real handle on its stdin; builds the command as a token list; drains ffmpeg's merged stdout/stderr on a timer so its pipe buffer can't deadlock encoding; forces-pix_fmt yuv420pon output for broad player compatibility with the grayscale source; setsLoggingMode='memory'so frames are buffered for the callback. NVENC uses bitrate control, libx264 uses CRF.startVideoAcquisition.m: registers aFramesAcquiredFcnthat drains the IMAQ buffer and streams raw frames to ffmpeg's stdin; resolves the actual.mp4output filename.pushFramesToFFmpeg.m: shared frame-drain helper; transposes each frame so column-major serialisation matches ffmpeg rawvideograyrow-major byte order; warns (doesn't crash the experiment loop) on a broken pipe.stopVideoAcquisition.m: flushes remaining frames, closes stdin (EOF → ffmpeg finalizes the file), waits for the process, tears down the timer.checkCameraRigParameters.m/RigParameters.m.example: validation + NVENC/libx264 config blocks.Tests (hardware-free — no Image Acquisition Toolbox, no camera)
sensors/camera/tests/drives the real transport through a real ffmpeg subprocess with a mocked camera, and asserts a readable.mp4is produced with a marker pixel at the correct coordinate (catches transpose / byte-order regressions).Run it:
or from the MATLAB command window:
cd sensors/camera/tests; smokeTestFFmpegTransport.Verified passing under MATLAB R2023b + ffmpeg 8.1 (libx264):
See
sensors/camera/tests/README.mdfor details and ffmpeg install hints per platform. The live FLIR/GenTL hardware calls still require the Windows rig and are not covered by these tests.