spike(rust/decode): FFmpeg vs gstreamer-rs binding decision (#108)

.gitignore

@@ -111,3 +111,5 @@ scripts/diarize/__pycache__/
 
 # Rust spike build artifacts
 spike/audio-sync/target/
+spike/rust-decode-spike/ffmpeg-candidate/target/
+spike/rust-decode-spike/gstreamer-candidate/target/

CLAUDE.md

@@ -256,6 +256,8 @@ Core problems being fixed: non-deterministic output, no project file, no pipelin
 
 Phases 0–4 (scripts) and 5–6 (Remotion) can proceed on separate branches in parallel.
 
+**Native rewrite spikes** (separate from the phases above — see `docs/rfcs/0001-native-desktop-rewrite.md`): throwaway Rust crates under `spike/`, deleted once each spike concludes. `spike/audio-sync/` validated the agent-driven Rust dev loop; `spike/rust-decode-spike/` (issue #108) is evaluating `ffmpeg-the-third` vs `gstreamer-rs` for the render engine's decode/encode binding — see its `FINDINGS.md`.
+
 ### Target directory additions (post-refactor)
 
 **`.gitignore` rule:** Every directory under `.ragtech/` (and any other runtime-generated directory such as `runs/`, `cache/`, output artifact dirs) must be listed in `.gitignore`. These hold generated/binary data, not source code, and must never be committed.

docs/implementation-guides/issue-108-rust-decode-spike.md

@@ -0,0 +1,87 @@
+# Issue #108 — spike(rust/decode): FFmpeg vs gstreamer-rs binding decision
+
+## Goal
+
+Build the minimal Rust harness at `spike/rust-decode-spike/` needed to answer the five evaluation
+criteria in [issue #108](https://github.com/ragTechDev/deckcreate/issues/108): can `ffmpeg-the-third`
+or `gstreamer-rs` reach hardware codec paths (VideoToolbox / NVENC/NVDEC), decode frame-accurately,
+build cleanly cross-platform, and (for GStreamer) avoid proprietary-codec redistribution risk.
+
+Not a working compositor — two standalone binary crates (one per candidate), a committed test
+fixture, and a `FINDINGS.md` that gets filled in as results land from each platform. This machine
+is an Apple M3, so the Mac/VideoToolbox rows get real results in this branch. Mac M2 and
+Windows/NVIDIA rows are scaffolded and stay open for the contributors who own that hardware to fill
+in via the PR.
+
+Follows the established spike convention from #93–#99 (`spike/<name>/`, standalone `Cargo.toml`
+per crate, no root-level workspace, `spike(rust/<scope>):` commit prefix).
+
+## Steps
+
+### Step 1 — Scaffold layout, fixtures, and .gitignore
+
+Create `spike/rust-decode-spike/` with:
+- `fixtures/generate_fixtures.sh` — regenerates the test clip and reference frames deterministically
+  via the `ffmpeg` CLI (`testsrc2` pattern, so pixel content per frame is reproducible).
+- `fixtures/test-clip.mp4` — small (320×240, ~3s, H.264) committed fixture.
+- `fixtures/reference/*.ppm` — one or two reference frames (raw RGB24 PPM) extracted at known
+  timestamps, used as ground truth for the pixel-accuracy check.
+- Append `spike/rust-decode-spike/*/target/` to the root `.gitignore`.
+
+**Status check:** `test -f spike/rust-decode-spike/fixtures/test-clip.mp4 && grep -q 'spike/rust-decode-spike' .gitignore`
+
+### Step 2 — ffmpeg-the-third candidate: decode, seek, pixel verify (software + VideoToolbox)
+
+Create `spike/rust-decode-spike/ffmpeg-candidate/` (standalone `Cargo.toml`, depends on
+`ffmpeg-the-third`). CLI: `ffmpeg-candidate <clip> <timestamp_secs> <reference.ppm> [--hw]`.
+Software path: open input, seek near the target timestamp, decode forward to the exact frame,
+convert to RGB24 via the software scaler, compare against the reference PPM (mean/max abs diff),
+print PASS/FAIL. `--hw` path: raw FFI through `ffmpeg_the_third::ffi` to set `hw_device_ctx`
+(`av_hwdevice_ctx_create` with `AVHWDeviceType::VIDEOTOOLBOX`) and a `get_format` callback on the
+decoder's *unopened* `AVCodecContext`, then `av_hwframe_transfer_data` to copy each decoded frame
+back to a software buffer before the same RGB24 compare. No high-level wrapper exists for this in
+the crate (checked: no hwaccel/hwdevice/hwcontext module) — this hand-written unsafe path, and how
+much of it there is, is itself the "without extra shims" answer for criterion #1. Software and
+hardware share one seek/decode/compare flow rather than splitting into a second commit, since the
+`--hw` branch is threaded through the same loop, not a separable unit.
+
+**Status check:** `cargo run --manifest-path spike/rust-decode-spike/ffmpeg-candidate/Cargo.toml -- spike/rust-decode-spike/fixtures/test-clip.mp4 1.5 spike/rust-decode-spike/fixtures/reference/frame_at_1.5s.ppm --hw | grep -q "HARDWARE PATH: active"`
+
+### Step 3 — gstreamer-rs candidate: decode, seek, pixel verify (software + VideoToolbox)
+
+Create `spike/rust-decode-spike/gstreamer-candidate/` (standalone `Cargo.toml`, depends on
+`gstreamer`, `gstreamer-app`, `gstreamer-video`). CLI mirrors the ffmpeg candidate. Software path
+uses `avdec_h264` (explicit software decoder, not `decodebin`, so the software/hardware choice
+stays deliberate rather than autonegotiated); hardware path pins the pipeline to the `vtdec_hw`
+element (hardware-only — pipeline fails to reach PAUSED if VideoToolbox isn't actually engaged,
+which is itself the criterion #1 answer for this candidate). Pull the frame via `appsink`, compare
+against the same reference PPM. Note for the Windows/NVIDIA contributor: swap `vtdec_hw` for
+`nvh264dec` (nvcodec plugin) behind a platform check to complete criterion #1 on that hardware —
+not attempted here since it cannot be verified on this machine.
+
+**Status check:** `cargo run --manifest-path spike/rust-decode-spike/gstreamer-candidate/Cargo.toml -- spike/rust-decode-spike/fixtures/test-clip.mp4 1.5 spike/rust-decode-spike/fixtures/reference/frame_at_1.5s.ppm | grep -q PASS`
+
+### Step 4 — FINDINGS.md and contributor README
+
+Write `spike/rust-decode-spike/FINDINGS.md` covering all five evaluation criteria, with real
+results filled in for Mac M3 (this machine) and open template rows for Mac M2 / Windows+NVIDIA.
+Include the GStreamer plugin licensing research for criterion #5 (which plugin ships which codec,
+under what license). Write `spike/rust-decode-spike/README.md` with exact per-platform setup +
+run commands for the contributors who'll fill in the remaining rows.
+
+**Status check:** `test -f spike/rust-decode-spike/FINDINGS.md && grep -q "Mac M2" spike/rust-decode-spike/FINDINGS.md && grep -q "Windows" spike/rust-decode-spike/FINDINGS.md`
+
+### Step 5 — Update CLAUDE.md
+
+Add a one-line pointer to the spike under a relevant section (Refactor Plan) so future agents know
+it exists and where.
+
+**Status check:** `grep -q 'rust-decode-spike' CLAUDE.md`
+
+## Out of scope
+
+- A working compositor or any production integration
+- Full verification on Mac M2 or Windows/NVIDIA hardware (scaffolded for contributors, not run here)
+- `ac-ffmpeg` or `ffmpeg-next` fallback candidates (only pursued if `ffmpeg-the-third` fails)
+- The CLI-subprocess (`ffmpeg-sidecar`) alternative — explicitly deferred per issue #108
+- Any change to files outside `spike/rust-decode-spike/`, `CLAUDE.md`, and `.gitignore`