Configure HKSV acceleration per camera and forward RTP limits

[JxnLexn]

Summary

• add an explicit per-camera Use Hardware Acceleration for HKSV setting
• keep hardware acceleration enabled by default while allowing it to be disabled for an affected camera only
• forward HomeKit’s negotiated RTP height and bitrate to the streaming session

Scope

The automatic software fallback, HomeKit-specific Go2RTC source selection, failure-report function, and automatic FMP4 session restart have been removed following review. This PR no longer attempts to recover HKSV by restarting recording sessions.

The underlying split-fragment issue discovered during the investigation is fixed generically in cameraui/camera.ui#521.

The RTP bitrate option used here is exposed by cameraui/sdk#37 and applied by cameraui/camera.ui#519.

[seydx]

instead of implementing an automatic software fallback, we should look into why these hksv errors happen in the first place and fix them.

we shouldn't add an automatic software fallback here, instead we could solve it through the camera settings schema with a flag...

are you running into hksv errors yourself?

[JxnLexn]

Thanks for the feedback. Yes, I am seeing this on a TP-Link Tapo D235. Its main RTSP stream is HEVC, while my other cameras provide H.264. In production I observed 734 HKSV recording starts and 571 Unexpected Failure closures, usually after only 3–4 fragments. The HomeKit process then grew to roughly 2.5 GiB anonymous RSS and eventually made the LXC unresponsive. The decoder logs also contained Error constructing the frame RPS, which points at the HEVC decode path.

I agree that silently switching decoding modes is not appropriate. I have removed the automatic software fallback and the plugin-specific Go2RTC source selection. Instead, the branch now adds an explicit per-camera Use Hardware Acceleration for HKSV setting (enabled by default). Disabling it affects only that camera’s HKSV FMP4 session; live streaming and other cameras remain hardware accelerated.

The per-camera failure isolation remains: after repeated Unexpected Failure results, only that camera’s FMP4 session is restarted. It no longer changes the configured decoder automatically. The focused circuit-breaker test still passes.

Separately, I found a concrete live-stream issue: HomeKit negotiated 1280x720 at 299 kbit/s, but the negotiated height and bitrate were not forwarded to node-av. That produced 2,372 reported lost RTP packets in about 11 seconds. I split that correction into the corresponding SDK/core changes rather than solving it with a Go2RTC workaround.

[JxnLexn]

The RTP bitrate option used by this branch is exposed in cameraui/sdk#37 and applied by cameraui/camera.ui#519.

[seydx]

yeah that direction is much better, though i'm not too happy with implementing a report function and restarting the fmp4 session...

there seems to be a bug here where resources aren't being released on the FMP4Session and/or node-av side, which i want to look into first

and it would be important to understand what your camera produces for hksv

would it be possible to record the stream with ffmpeg, and maybe also what the data looks like after FMP4Session

[JxnLexn]

Thanks — I was able to capture both the raw D235 stream and the output produced through FMP4Stream, and this exposed a concrete issue outside the plugin.

The D235 source is HEVC Main, 2560x1920 at 20 fps with mono PCMA at 8 kHz. The raw RTSP capture also reports unset packet timestamps. When decoding the HEVC source, node-av/FFmpeg repeatedly logs:

Could not find ref with POC
Error constructing the frame RPS
Skipping invalid undecodable NALU

More importantly, I found why HomeKit receives the suspiciously small fragments. Depending on the stream path, node-av delivers moof and mdat either together or in two consecutive onData callbacks. Fmp4Session currently forwards only callbacks containing a moof, so the following standalone mdat — containing the actual media data — is discarded.

This exactly matches the production behavior:

Fragment 1: 1319 bytes
Fragment 2: 804 bytes
Fragment 3: 824 bytes
Stream 1 closed: Unexpected Failure

The 804–824 byte values correspond to standalone moof boxes. In a local sample, the following mdat boxes were between roughly 2.2 MB and 3.6 MB. Once moof and mdat are combined, FFmpeg can decode the resulting fragmented MP4.

I implemented a generic core fix that assembles split moof/mdat callbacks before exposing them through Fmp4Session. Combined callbacks are passed through unchanged. The branch is:

https://github.com/JxnLexn/camera.ui/tree/dev/fix/fmp4-split-box-assembly

It includes regression tests for combined fragments, split fragments, orphaned boxes, and incomplete-fragment replacement.

I also removed reportRecordingFailure() and the automatic FMP4 session restart from this plugin branch. The explicit per-camera HKSV hardware acceleration setting remains.

[JxnLexn]

The generic core fix is now available as cameraui/camera.ui#521.

That PR assembles split moof and mdat callbacks into complete fMP4 media fragments before exposing them to consumers. It includes focused regression tests and does not introduce any HomeKit- or plugin-specific stream routing.

As noted above, I removed reportRecordingFailure() and the automatic FMP4 session restart from this plugin branch. PR #257 now retains the explicit per-camera HKSV hardware acceleration setting and the separately negotiated RTP parameters.

[seydx]

if we pass the height as well, streams with an aspect ratio of e.g. 8:3 get forced into a 16:9 aspect ratio and look "stretched"