feat(cmcd): upgrade to CMCD v2 using @svta/cml-cmcd CmcdReporter#7725
feat(cmcd): upgrade to CMCD v2 using @svta/cml-cmcd CmcdReporter#7725littlespex wants to merge 26 commits intovideo-dev:masterfrom
Conversation
…d v2 version support Replace the monolithic @svta/common-media-library package with the scoped @svta/[email protected] and @svta/[email protected] packages for CMCD functionality. The old package is retained for non-CMCD imports (ID3, UTF8). Add a `version` option to CMCDControllerConfig (defaults to 1 for backwards compatibility) that controls CMCD encoding version. When set to 2, the controller uses CMCD v2 Structured Field Value encoding via the new library. Key changes: - Update all CMCD imports to use @svta/cml-cmcd single entry point - Update uuid import to use @svta/cml-utils - Update tsconfig moduleResolution to "bundler" for exports field support - Adapt CMCD data fields (br, bl, mtp, tb, nor) to v2 array types - Pass version through to CmcdEncodeOptions for version-aware encoding https://claude.ai/code/session_01FmnN6xNSm9Qo17tp52ag3U
Phase 3: Add new CMCD v2 data fields to the controller: - Stream type (st): Detect VOD/LIVE/LOW_LATENCY from level details based on live flag, canBlockReload, and canSkipUntil properties - Player state (sta): Track player state transitions via media element events (waiting, playing, pause, seeking, ended) and hls.js ERROR events for fatal errors. Maps to CmcdPlayerState enum values. - Both fields are only included when version >= 2 Phase 4: Integrate CmcdReporter for event-mode reporting: - Add eventTargets config option (CmcdEventReportConfig[]) for v2 event reporting endpoints - Instantiate CmcdReporter when version >= 2 and eventTargets are configured, with session/content ID and transmission mode - Record PLAY_STATE events on player state transitions - Record ERROR events on fatal hls.js errors - Record BITRATE_CHANGE events on level switches - Stop and flush reporter on controller destroy Add unit tests for v2 version encoding, stream type detection (VOD, LIVE, LOW_LATENCY), and player state inclusion. https://claude.ai/code/session_01FmnN6xNSm9Qo17tp52ag3U
Phase 6: Re-export CMCD types and constants from exports-named.ts for ESM consumers: CmcdObjectType, CmcdStreamType, CmcdStreamingFormat, CmcdPlayerState, CmcdEventType, CmcdHeaderField, CMCD_V1, CMCD_V2, and type exports for Cmcd, CmcdEncodeOptions, CmcdEventReportConfig, CmcdVersion. Add @svta/cml-cmcd, @svta/cml-utils, and @svta/cml-structured-field-values to api-extractor bundledPackages so external types are inlined in the rolled-up dist/hls.d.ts. Phase 7: Add tests for: - v2 fragment data includes version, stream type, and player state - v2 headers mode includes v2 fields in CMCD headers - Reporter is not created without eventTargets or for v1 - Reporter is created with v2 + eventTargets - Reporter.stop(true) is called on destroy - Play state events are recorded on state transitions - Error events are recorded on fatal hls.js errors - Duplicate player state events are deduplicated Phase 8: Verified TypeScript type-check, all Rollup build configs (full, fullEsm, light), and api-extractor declaration bundling. https://claude.ai/code/session_01FmnN6xNSm9Qo17tp52ag3U
| // TODO: Is this the best way to determine the low-latency stream type? | ||
| if (details.canBlockReload || details.canSkipUntil) { | ||
| return CmcdStreamType.LOW_LATENCY; | ||
| } |
There was a problem hiding this comment.
@robwalch I'm not sure about this check. Is there a better way to determine if a stream is low latency?
There was a problem hiding this comment.
Good question!
Only checking EXT-X-SERVER-CONTROL for blocking reload or delta playlist capabilities is not enough. Both of these features can be used without partial segments (details.partList) for live playback at a standard three target duration hold back.
If we follow the requirements of a "Low-Latency Server Configuration Profile":
if (
// Playlist has EXT-X-PROGRAM-DATE-TIME
details.hasProgramDateTime &&
// Segments have at least one key-frame (EXT-X-INDEPENDENT-SEGMENTS or track violations)
// skip this requirement - something like `couldBacktrack` on LevelDetails could be nice to have
// Playlist has EXT-X-PART(s)
!!details.partList &&
// Playlist has EXT-X-PRELOAD-HINT
// skip this requirement - support will be added with #6356
// Playlists has at least one EXT-X-RENDITION-REPORT or only one variant
(!!details.renditionReports || this.hls.levels.length === 1) &&
// Blocking Playlist Reload is required
details.canBlockReload
) {
return CmcdStreamType.LOW_LATENCY;
}Delta playlists are not required because playlist can be short enough that skipping segments doesn't add much if any benefit. Blocking reload is critical - servers that don't implement it, and worse only signal they do, perform miserably. HLS.js today may still use parts without PDT and Rendition Reports (I'll have to check), but it should not. I'll file an issue to follow up with other scheduled LL-HLS enhancements.
There was a problem hiding this comment.
…ooks CmcdReporter sends CMCD v2 event reports via a requester function that defaults to bare fetch(), bypassing customer auth headers and credentials configured via xhrSetup/fetchSetup. Add a createCmcdRequester adapter that routes through the same setup hooks applied to media/playlist requests. Co-Authored-By: Claude Opus 4.6 <[email protected]>
Add end-to-end tests for CMCD v2 covering query mode, header mode, event mode, key filtering, and version comparison (v1 vs v2). Fix eventTargets enabledKeys mapping bug in CMCDController where includeKeys was not being mapped to the library's enabledKeys parameter, causing event reports to have no enabled keys. Co-Authored-By: Claude Opus 4.6 <[email protected]>
There was a problem hiding this comment.
I'm not sure if these should just be merged with the tests/unit/controller/cmcd-controller.ts tests. They are different in that they actually load media and inspect the outgoing traffic for correct CMCD data.
There was a problem hiding this comment.
This helper monkey patches fetch so that outgoing requests can be inspected. It also acts as an "event" server for CMCD event mode, generating the 204 response.
… option Remove createCmcdRequester function and allow users to pass a custom loader via cmcd.loader config. When not provided, CmcdReporter uses its default fetch-based requester. Co-Authored-By: Claude Opus 4.5 <[email protected]>
robwalch
left a comment
There was a problem hiding this comment.
Preliminary review. I still need to run some tests before I can sign off.
| if (this.version >= CMCD_V2) { | ||
| hls.off(Events.ERROR, this.onError, this); | ||
| hls.off(Events.LEVEL_SWITCHING, this.onLevelSwitching, this); | ||
| } |
There was a problem hiding this comment.
We don't need to guard here. It's safe to call off whether or not the listener was registered.
There was a problem hiding this comment.
The if (this.version >= CMCD_V2) guard was removed from both registerListeners and unregisterListeners in c148687.
| if (this.version >= CMCD_V2) { | ||
| this.media.addEventListener('pause', this.onPause); | ||
| this.media.addEventListener('seeking', this.onSeeking); | ||
| this.media.addEventListener('ended', this.onEnded); |
There was a problem hiding this comment.
You might consider using Events.MEDIA_ENDED. The event has a stalled flag that signals whether a stall (waiting) was reached near the end or "ended" was received.
There was a problem hiding this comment.
Done in c10b1f5, replaced the native 'ended' listener with hls.on(Events.MEDIA_ENDED, this.onMediaEnded). Both stalled and non-stalled variants map to CmcdPlayerState.ENDED; REBUFFERING is already set by the 'waiting' handler when a pre-end stall occurs.
| this.reporter?.update({ br: [data.bitrate / 1000] }); | ||
| this.reporter?.recordEvent(CmcdEventType.BITRATE_CHANGE); |
There was a problem hiding this comment.
Prefer if (this.reporter) { block to multiple optional operators.
There was a problem hiding this comment.
Done in c10b1f5, onError, onLevelSwitching, setPlayerState, and apply now guard with if (this.reporter) { ... } blocks instead of ! assertions.
| version?: CmcdVersion; | ||
| eventTargets?: (Omit<CmcdEventReportConfig, 'enabledKeys'> & { | ||
| includeKeys?: CmcdKey[]; | ||
| })[]; | ||
| loader?: (request: { |
There was a problem hiding this comment.
Can you add these new options to docs/API.md#cmcd?
There was a problem hiding this comment.
Done in c10b1f5, added version, eventTargets (with a child list covering url, events, interval, batchSize, and per-target includeKeys), and loader to the CMCD section in docs/API.md. Also extended the existing includeKeys line to note the version-aware default.
| private initialized: boolean = false; | ||
| private starved: boolean = false; | ||
| private buffering: boolean = true; | ||
| private playerState: CmcdPlayerState = CmcdPlayerState.STARTING; |
There was a problem hiding this comment.
How is CMCD supposed to behave when another item is loaded using the same player instance? Same as if a new instance was used?
Maybe the player state should also get set to STARTING on Events.MANIFEST_LOADING? We use this event to reset controllers whenever hls.loadSource(<m3u8>) is called to load a new item.
There was a problem hiding this comment.
Done in c10b1f5, added an onManifestLoading handler that stops and recreates the CmcdReporter, resets playerState to STARTING, and clears the initialized/starved/buffering flags. The reporter is recreated with the same config so sid / cid persist across loadSource() calls on the same Hls instance (same player session = same CMCD session).
| const loadLevel = this.hls.loadLevel; | ||
| const details = | ||
| loadLevel >= 0 ? this.hls.levels[loadLevel]?.details : undefined; |
There was a problem hiding this comment.
Use top-level API to get latest loaded details:
| const loadLevel = this.hls.loadLevel; | |
| const details = | |
| loadLevel >= 0 ? this.hls.levels[loadLevel]?.details : undefined; | |
| const details = this.hls.latestLevelDetails; |
| // TODO: Is this the best way to determine the low-latency stream type? | ||
| if (details.canBlockReload || details.canSkipUntil) { |
There was a problem hiding this comment.
Until #7729 do
| // TODO: Is this the best way to determine the low-latency stream type? | |
| if (details.canBlockReload || details.canSkipUntil) { | |
| // TODO: Replace with an `isLowLatency` check in #7729 | |
| if (!!details.partList && details.canBlockReload) { |
It's not low-latency without parts. The potential to deliver playlist deltas (canSkipUntil) does not make the playlist low-latency.
- Swap native 'ended' listener for hls Events.MEDIA_ENDED - Reset + recreate reporter on MANIFEST_LOADING so loadSource() on an existing Hls instance behaves like a fresh CMCD session - Replace this.reporter! assertions with if (this.reporter) blocks - Use hls.latestLevelDetails in getStreamType - Tighten low-latency detection to require details.partList (TODO video-dev#7729) - Document version, eventTargets (with child properties), and loader config options in docs/API.md - Extend setupEach test harness with a streamController.getLevelDetails mock to support latestLevelDetails lookup Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
Summary
Implements CMCD v2 support by migrating from
@svta/common-media-libraryto the dedicated@svta/cml-*packages and fully delegating CMCD encoding toCmcdReporterfrom@svta/cml-cmcd.Resolves #7723.
Changes
@svta/common-media-librarywith@svta/cml-cmcd,@svta/cml-id3,@svta/cml-utils, and@svta/cml-structured-field-valuesversion,st(stream type),sta(player state),sn(sequence number) to CMCD output. Inner list encoding forbr,tb,bl,mtp,norper v2 specCmcdReporter.createRequestReport(), removing manualappendCmcdHeaders/appendCmcdQuerycalls and thecreateData()methodPLAY_STATE), fatal errors (ERROR), and bitrate changes (BITRATE_CHANGE)version,enabledKeys, andeventTargetstoCMCDControllerConfig. DefaultenabledKeystoCMCD_V1_KEYS(v1) orCMCD_KEYS(v2)CmcdVersion,CmcdPlayerState,CmcdStreamType, etc.) fromexports-named.tsBehavioral notes
v=1is omitted from output per CMCD spec (it is the default)sn(sequence number) is automatically included in v2 output, filtered out for v1noruses root-relative paths (viaurl.originas baseUrl) instead of path-relativebl,tb) are guarded againstNaNto prevent Structured Fields serialization errorsTest plan