[Bug]: VAAPI transcode config uses libx264 (software) encoder instead of h264_vaapi

[mark-zalewski]

This issue respects the following points:

• This is a bug, not a question or a configuration issue; Please visit our Discord first to troubleshoot with volunteers, before creating a report.
• This issue is not already reported on GitHub (I've searched it).
• I'm using an up to date version of Tunarr; We generally do not support previous older versions. If possible, please update to the latest version before opening an issue.

Description of the bug

Describe the bug

A transcode config with hardwareAccelerationMode: vaapi selects the VAAPI builder (Builder: FfmpegStreamFactory (vaapi)) and Tunarr reports HW Accel: vaapi, but the generated FFmpeg command uses the software -c:v libx264 encoder instead of -c:v h264_vaapi. The -vaapi_device is initialized and used for filters (e.g. hwupload/hwdownload for deinterlace), but the video is never hardware-encoded, so transcoding runs almost entirely on the CPU (~330% / 3.3 cores per 1080p stream).

Hardware H.264 encoding is fully functional on the host — a manual ffmpeg … -c:v h264_vaapi encode succeeds and lights up the GPU video engine — so this looks like an encoder-selection bug in the VAAPI pipeline builder, not a driver/capability problem.

Reproduction steps

Steps to reproduce

1. Configure a transcode config with hardwareAccelerationMode: vaapi, vaapiDriver: ihd, vaapiDevice: /dev/dri/renderD128, videoFormat: h264, videoBitDepth: 8, target 1920x1080.
2. Add a Plex (or Jellyfin) media source and a channel; play a program that requires a video transcode (e.g. a 720p 8-bit H.264 source upscaled to 1080p).
3. Watch the stream (or use System → Troubleshoot → Run Troubleshooter) and inspect the generated FFmpeg command.

Result: the command ends with -c:v libx264 and uses a software scale= filter; there is no -c:v h264_vaapi and no scale_vaapi.

What is the current bug behavior?

GPU should be used for transcoding

What is the expected correct behavior?

Expected behavior

With hardwareAccelerationMode: vaapi and a working VAAPI device, the pipeline should use -c:v h264_vaapi for the video encoder (hardware encode), not libx264.

FFmpeg command (from the built-in Troubleshooter)

Builder reported: FfmpegStreamFactory (vaapi), HW Accel: vaapi. Generated command:

-nostdin -hide_banner -nostats -loglevel warning -fflags +genpts+discardcorrupt+igndts \
-vaapi_device /dev/dri/renderD128 -ss 1206799ms -c:v h264 -readrate 1 -readrate_initial_burst 60 \
-i "/mnt/tv/.../S12E05.720p.AMZN.WEB-DL.DDP5.1.H.264.mkv" \
-filter_complex [0:0]scale=1920:1080:flags=bicubic,setsar=1[v];[0:1]aresample=async=1,apad[a] \
-map [v] -map [a] -muxdelay 0 -muxpreload 0 -flags cgop -movflags +faststart -t 30000ms \
-video_track_timescale 90000 -b:v 6000k -maxrate:v 6000k -bufsize:v 4000k \
-c:v libx264 -sc_threshold 1000000000 \
-c:a aac -ac 2 -b:a 192k -maxrate:a 192k -bufsize:a 384k -ar 48k \
-g 96 -keyint_min 96 -force_key_frames expr:gte(t,n_forced*4) \
-f hls -hls_time 4 -hls_list_size 0 -segment_list_flags +live -hls_segment_type mpegts …

Note: -vaapi_device is set, but decode is -c:v h264 (software), scaling is software scale=, and the encoder is -c:v libx264. (Same result in both the MPEG-TS and HLS pipelines.)

Source media (from Troubleshooter Media Info): Video h264 (high) 1280x720 23.98fps 8-bit bt709; Audio eac3 6ch. Target h264 @ 1920x1080 / aac — both streams require a real transcode, so the encoder is exercised.

Evidence the GPU is idle for encode (but works otherwise)

intel_gpu_top on the host while a channel streams — the Video (VCS) engine stays at 0% (only the deinterlace/video-enhance engine is briefly active):

Freq 0 MHz | RC6 100% | Power 0.00 W | RCS 0% | VCS 0.00% | VECS 0%

A manual h264_vaapi encode on the same device immediately drives the video engine, proving hardware encode works:

ffmpeg -vaapi_device /dev/dri/renderD128 -f lavfi -i testsrc=1920x1080:rate=30 \
  -vf format=nv12,hwupload -c:v h264_vaapi -t 10 -f null -
# intel_gpu_top: Freq 1250 MHz | RC6 0% | RCS 84% | VCS 26% | Power 2.75 W

vainfo confirms the encode entrypoint is present:

VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointEncSlice
Driver: Intel iHD driver for Intel(R) Gen Graphics - 25.4.6

Per-process /proc/<pid>/fdinfo for the Tunarr ffmpeg shows drm-engine-video: 0 ns (no hw encode) with only drm-engine-video-enhance active (the deinterlace filter).

Things ruled out

• vaapiDriver: system and ihd — no change.
• deinterlaceVideo: true and false — no change.
• Source type: remote Plex HTTP and local file (via pathReplacements, -i /mnt/...) — both produce libx264.
• FFmpeg binary: wrapped /usr/local/bin/ffmpeg and the raw /usr/lib/jellyfin-ffmpeg/ffmpeg — both produce libx264.
• Tunarr's own capability detection lists vaapi and h264_vaapi (/api/ffmpeg-info).

Environment

• Tunarr: 1.3.9
• FFmpeg: 7.1.4 (jellyfin-ffmpeg7)
• Node.js: v22.20.0
• Platform: Linux x64, Debian 13 LXC on Proxmox VE 9 (kernel 7.x-pve)
• GPU: Intel iGPU, VAAPI via Intel iHD driver 25.4.6 (/dev/dri/renderD128), native LXC device passthrough
• HW Acceleration reported: cuda, vaapi, qsv, drm, opencl, vulkan

Tunarr version

Latest

FFMPEG encoder type

VAAPI

Specify full Edge version id

No response

Specify Alpha release number

No response

Specify version number

No response

What browser did you experience the the problem on? Please provide if the issue is related to the Web UI and definitely not the server.

No response

Deployment Type

None

What operating system are you using?

Proxmox LXC running Tunarr

Full server logs

# ============================================================================
# Tunarr VAAPI bug — encoder uses libx264 instead of h264_vaapi
# ============================================================================
# Tunarr 1.3.9 | FFmpeg 7.1.4 (jellyfin-ffmpeg7) | Node.js v22.20.0
# Debian 13 LXC on Proxmox VE 9 (kernel 7.x-pve) | x64
# Intel iGPU, VAAPI via iHD driver 25.4.6, /dev/dri/renderD128 (LXC dev passthrough)
#
# Transcode config: hardwareAccelerationMode=vaapi, vaapiDriver=ihd, videoFormat=h264,
#                   videoBitDepth=8, target 1920x1080
#
# SYMPTOM: builder is 'FfmpegStreamFactory (vaapi)' and -vaapi_device is set, but the
# generated command uses '-c:v libx264' (software) instead of '-c:v h264_vaapi'.
# Decode is also software ('-c:v h264'), scaling is software ('scale='). The GPU video
# engine stays at 0% during the stream; a manual '-c:v h264_vaapi' encode on the same
# device works (video engine ~26%). See the ffmpeg command below (note '-c:v libx264').
# ============================================================================

===== LIVE FFMPEG COMMAND (from ps, mid-stream) =====
/usr/lib/jellyfin-ffmpeg/ffmpeg -nostdin -hide_banner -nostats -loglevel warning -fflags +genpts+discardcorrupt+igndts -vaapi_device /dev/dri/renderD128 -ss 1647220ms -c:v h264 -readrate 1 -i /mnt/tv/CSI Crime Scene Investigation/CSI.S10E17.Irradiator.720p.Bluray.DD5.1.x264-RZF.mkv -filter_complex [0:0]scale=1920:1080:flags=bicubic,setsar=1[v];[0:1]aresample=async=1,apad[a] -map [v] -map [a] -muxdelay 0 -muxpreload 0 -flags cgop -movflags +faststart -t 1019020ms -video_track_timescale 90000 -b:v 6000k -maxrate:v 6000k -bufsize:v 6000k -c:v libx264 -sc_threshold 1000000000 -c:a aac -ac 2 -b:a 192k -maxrate:a 192k -bufsize:a 384k -ar 48k -f mpegts pipe:1

===== TUNARR JOURNAL (last 35s, transcode session) =====
Jul 25 01:44:41 tunarr tunarr-v1.3.9-linux-x64[94]: 2026-07-25T01:44:41.607Z [info]: channel-5-transcode still running after SIGTERM. Sending SIGKILL
Jul 25 01:44:41 tunarr tunarr-v1.3.9-linux-x64[94]: 2026-07-25T01:44:41.623Z [info]: channel-5-transcode exited. (signal=SIGKILL, code=-1, expected?=true) {"args":"-nostdin -hide_banner -nostats -loglevel warning -fflags +genpts+discardcorrupt+igndts -vaapi_device /dev/dri/renderD128 -ss 3244ms -readrate 1 -i /mnt/tv/Stargate SG-1/Stargate SG-1 Season 5/Stargate.SG-1.S05E20.DVDRip.DivX-SFM.avi -filter_complex [0:0]scale=1920:1080:flags=bicubic:force_original_aspect_ratio=decrease,setsar=1,format=nv12|p010le|vaapi,hwupload,pad_vaapi=w=1920:h=1080:x=-1:y=-1:color=black[v];[0:1]aresample=async=1,apad[a];[v]hwdownload,format=nv12[vpf] -map [vpf] -map [a] -muxdelay 0 -muxpreload 0 -flags cgop -movflags +faststart -t 2541316ms -video_track_timescale 90000 -b:v 6000k -maxrate:v 6000k -bufsize:v 6000k -c:v libx264 -sc_threshold 1000000000 -c:a aac -b:a 192k -maxrate:a 192k -bufsize:a 384k -ar 48k -f mpegts pipe:1"}