Skip to content

feat: pipeline DAV2 startup with bounded read-ahead#104

Merged
knzeng-e merged 2 commits into
mainfrom
feat/dav2-read-ahead-pipeline
Jul 22, 2026
Merged

feat: pipeline DAV2 startup with bounded read-ahead#104
knzeng-e merged 2 commits into
mainfrom
feat/dav2-read-ahead-pipeline

Conversation

@knzeng-e

@knzeng-e knzeng-e commented Jul 18, 2026

Copy link
Copy Markdown
Owner

Why this PR exists

DAV2 already avoids downloading and decrypting a whole protected track before playback, but its MSE pump was still serial:

range chunk 0 -> decrypt 0 -> append 0 -> range chunk 1 -> decrypt 1 -> append 1

That leaves the network and Web Crypto idle while SourceBuffer appends, and it makes later playback more exposed to public-gateway latency. This PR implements the next production slice of the Project 5 streaming gate without changing who may receive a content key.

Backlog / Project 5: Refs #88

What changes

  • Adds a bounded DAV2 pipeline that prepares the current chunk plus two future chunks.
  • Keeps SourceBuffer appends strictly ordered even when fetch/decrypt work finishes out of order.
  • Cancels in-flight read-ahead when the selected track changes or the pipeline fails.
  • Imports the AES-GCM CryptoKey once per playback instead of once per chunk.
  • Validates known chunk byte counts and exposed Content-Range values before decryption, so a truncated gateway response can retry elsewhere instead of looking like ciphertext tampering.
  • Cleans up synchronous appendBuffer() failure listeners.
  • Isolates and tests the fail-closed MSE recovery policy: authentication failures retire playback, while recoverable transport/parser failures may use authenticated full-file recovery.
  • Documents the current DAV2/MSE/WebRTC boundary and the remaining real-device evidence gates.

The steady-state shape is:

prepare chunk 0 ───────────────> append 0
prepare chunk 1 (read-ahead) ─────────────> append 1
prepare chunk 2 (read-ahead) ─────────────────────> append 2

There are never more than three chunks in the preparation window, about 1.5 MiB with today’s 512 KiB container default.

Technical context for reviewers

DAV2 encryption chunks and MSE media segments solve different problems. DAV2 chunks are independent AES-256-GCM authentication units. MSE sees the decrypted appends as one logical byte stream and retains incomplete parser input between calls, so an encryption boundary may split a media frame. The reconstructed bytes must still conform to the MSE byte-stream format for their MIME type.

That is why this PR keeps both checks:

  1. MediaSource.isTypeSupported(mediaMime) decides whether streaming is worth attempting.
  2. Recoverable gateway or parser failures may use the authenticated full-file path, while AES-GCM authentication failures stop playback and never fall through.

Useful references:

Product and security invariants

  • The artist runtime and key service remain authoritative for access.
  • Individual listeners or room hosts may receive a temporary key only after the existing access path approves them.
  • Room guests still receive only the ephemeral WebRTC stream: no wallet requirement, encrypted source, clear source, or content key.
  • Chunk authentication failures remain fail-closed.
  • The optimization is bounded; it does not fan out over every gateway or buffer a whole track.
  • DAV3/media normalization is still evidence-led, not introduced speculatively.

This preserves Dotify’s north star: reduce dead air in the shared listening moment while keeping artist sovereignty and infrastructure complexity below the surface.

Reviewer map

  • web/src/features/catalog/audioV2Pipeline.ts: concurrency bound, cancellation, and ordered append invariant.
  • web/src/hooks/useCatalog.ts: DAV2/MSE integration, first-chunk metrics, and authentication error boundary.
  • web/src/features/catalog/audioV2Gateway.ts: HTTP range response validation and gateway retry classification.
  • web/src/features/catalog/audioV2Recovery.ts: the security-relevant fail-closed versus recoverable fallback decision used by useCatalog.
  • web/src/shared/utils/audioV2.ts: one imported AES key per playback/container.
  • Tests next to those modules: overlap, append ordering, abort, fail-closed recovery routing, truncation, and Content-Range mismatch.
  • docs/design/dotify-v2-access-and-streaming.md: standards review, room-stream coherence, and remaining [Streaming][P0] Pipeline DAV2 startup for sub-2-second first sound #88/[Rooms][P1] Productionize WebRTC with TURN, quality metrics, and an SFU path #89 decisions.

Validation

  • cd web && npm run fmt
  • cd web && npm run fmt:check
  • cd web && npm run test:unit - 170 passing
  • cd web && npm run lint - passes with 3 pre-existing hook dependency warnings
  • cd web && npm run build
  • cd web && npm run test:e2e -- e2e/room-join.spec.ts - 6 passing
  • node scripts/backlog-sync.mjs --check --offline - passes with the existing ticket-24 mapping and duplicate ticket-08 warnings
  • git diff --check

What this PR does not prove

The #88 latency budgets are not claimed yet. Release evidence still needs real DAV2 assets and public gateways across Chrome, Firefox, Safari, iOS, and Android, followed by the backend read-through decision. Worker decryption, first-chunk sizing, and telemetry export also remain in #88. TURN, ICE restart/quality telemetry, and SFU scaling remain room-reliability work in #89.

@netlify

netlify Bot commented Jul 18, 2026

Copy link
Copy Markdown

Deploy Preview for muzinga ready!

Name Link
🔨 Latest commit 1c82da8
🔍 Latest deploy log https://app.netlify.com/projects/muzinga/deploys/6a5bf5dcd66faf0008bedeb5
😎 Deploy Preview https://deploy-preview-104--muzinga.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

@knzeng-e
knzeng-e marked this pull request as ready for review July 18, 2026 21:24
@knzeng-e

Copy link
Copy Markdown
Owner Author

@claude please review this

@claude

claude Bot commented Jul 18, 2026

Copy link
Copy Markdown

Claude finished @knzeng-e's task in 3m 52s —— View job


Review complete

  • Gather context (PR body, changed files list)
  • Read CLAUDE.md / AGENTS.md guidance
  • Inspect diff for reviewer-mapped files (audioV2Pipeline, useCatalog, audioV2Gateway, audioV2.ts)
  • Check tests for the new pipeline
  • Post review findings

Reviewed web/src/features/catalog/audioV2Pipeline.ts, audioV2Gateway.ts, useCatalog.ts, shared/utils/audioV2.ts, and the accompanying tests/docs against origin/main. Could not execute npm run test:unit / lint / build myself (no node_modules in this checkout and installing would need network access I didn't attempt), so the validation numbers in the PR description are unverified by me; the analysis below is from static reading.

Findings

1. (Minor / test-coverage gap) No unit test exercises the fail-closed vs. fallback branch in useCatalog.ts

web/src/hooks/useCatalog.ts:735-755 (createAudioV2MseSource's .catch) is where the security-relevant decision actually lives: an AudioV2ChunkAuthenticationError must retire playback and never fall through, while any other recoverable error (gateway/parser) must fall back to the authenticated full-file path. This is exactly the invariant the PR calls out ("AES-GCM authentication failures stop playback and never fall through"), but it's only covered indirectly, through audioV2Pipeline.test.ts (which tests the generic pipeline, not this branching) and audioV2Gateway.test.ts (which tests the fetch layer). There's no useCatalog.test.ts (or equivalent) asserting that a chunk-decrypt failure retires the object URL without calling the full-file fallback, versus a gateway/parse failure that does. The reviewer map in the PR body lists useCatalog.ts among the modules with tests "next to" it, but I don't see one added for this file. Given this is the actual authorization/fail-closed boundary described in the product invariants, a regression here (e.g. someone later "simplifies" the else if chain) wouldn't be caught by CI.

Suggested smallest fix: a focused test around fetchAndDecryptAudio/createAudioV2MseSource that stubs MediaSource/SourceBuffer, forces a decryptAudioV2Chunk rejection, and asserts the object URL is retired with no fallback fetch, plus a companion case where a gateway error triggers the full-file fallback.

2. (Nit) audio-v2-container.md's Content-Range wording is ambiguous versus the implementation

docs/reference/audio-v2-container.md (new "Range and MSE Boundary" section) reads: "When Content-Range is visible through CORS, its start, end, and returned length must also match; otherwise the client tries another configured gateway before decrypting." In audioV2Gateway.ts:84-93 (validateRangeBytes), when the content-range header is absent, the function just returns early — it does not retry another gateway, it silently accepts the response (byte-count check only). The doc's "otherwise" reads naturally as covering the "header absent" case too, which isn't what the code does (only an actual start/end/length mismatch, when the header is present, triggers a retry). Not a functional bug — GCM auth on each chunk still fail-closed catches real tampering/truncation regardless — but worth a wording tweak so the doc doesn't overstate gateway-side validation coverage, per this repo's "claims in docs must reflect implemented and verified behavior" rule.

What looks solid

  • audioV2Pipeline.ts: the sliding-window read-ahead (prepareThrough) correctly bounds in-flight preparation to current + readAheadChunks, keeps preparedByIndex consumption strictly ordered via the for loop awaiting preparedByIndex.get(index), and aborts+rethrows on any failure. Cancellation propagation (parent signal -> internal controller) and cleanup (removeEventListener in finally) look correct. Tests cover ordering under out-of-order resolution, abort, and auth-failure-cancels-future-work.
  • audioV2Gateway.ts: hedging is correctly bounded (MAX_PARALLEL_HEDGED_RANGES = 2), timers are cleared on both race outcomes, and the byte-count check (bytes.length !== expectedLength for non-header phases) means Content-Range absence doesn't create a real security gap — GCM tag verification is the actual authenticity boundary.
  • shared/utils/audioV2.ts: importAudioV2ContentKey is now shared across all chunk decrypts in both decryptAudioV2Container and the new pipeline path, matching the "one imported key per playback" claim, and is asserted via the importKeySpy count in audioV2.test.ts.
  • useCatalog.ts correctly threads the pipeline's internal per-chunk AbortSignal into fetchAudioV2Range/appendSourceBuffer rather than the outer track-selection signal, and appendSourceBuffer's synchronous appendBuffer() throw path now cleans up its listeners before rejecting (previously it did not).
  • Product/security invariants preserved: room guests still never touch DAV2 chunks/keys (only the host path calls this code); key resolution still goes through the existing server-authorized resolveServerContentKey/resolveFreeContentKey paths untouched by this diff.
  • Docs (docs/design/dotify-v2-access-and-streaming.md, docs/reference/audio-v2-container.md, docs/explanation/content-protection.md) were updated in the same PR and accurately describe the new pipeline shape, MSE byte-stream boundary, and remaining [Streaming][P0] Pipeline DAV2 startup for sub-2-second first sound #88/[Rooms][P1] Productionize WebRTC with TURN, quality metrics, and an SFU path #89 gates.

Nothing here blocks merge in my read; #1 is worth a follow-up test before this becomes the default streaming path, and #2 is a one-line doc clarification.

@knzeng-e knzeng-e self-assigned this Jul 18, 2026
@knzeng-e
knzeng-e merged commit eec0f79 into main Jul 22, 2026
11 checks passed
@knzeng-e
knzeng-e deleted the feat/dav2-read-ahead-pipeline branch July 22, 2026 16:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant