Ultrasound cine#878
Open
PaulHax wants to merge 31 commits into
Open
Conversation
Lands a self-contained cine pipeline for single-file ultrasound DICOMs (SOP Class UID 1.2.840.10008.5.1.4.1.1.3 / .3.1, NumberOfFrames > 1) alongside the existing volume pipeline. Multi-chunk volume imports never match the cine router, so CT/MR streaming and 3D volume behavior are unchanged. Core additions under src/core/cine/: - parseCineDicom.ts wraps cornerstonejs/dicom-parser to extract the header (transfer syntax, geometry, FrameTime, patient/study/series, SequenceOfUltrasoundRegions) and per-frame byte views — zero-copy for native PixelData, fragment-aware for encapsulated JPEG-Baseline (with populated BOT, empty BOT, and JPEG-SOI scan fallbacks). Supports Implicit + Explicit VR LE and JPEG-Baseline. - DicomCineImage extends BaseProgressiveImage, owns one 2D vtkImageData (extent [0,cols-1, 0,rows-1, 0,0], 3-component RGB uint8), and swaps scalars in-place when the selected frame changes. setFrame() bumps a decode token unconditionally so any new request — cached or decode — invalidates in-flight decodes. - frameCache.ts: byte-budgeted LRU keyed by frame index; decodeJpegFrame via createImageBitmap + OffscreenCanvas; decodeNativeFrame for native RGB / MONOCHROME2. - isCineImage / getCineImage helpers and getRenderSlice that returns 0 for cine so the VTK mapper renders slice 0 while the semantic slice is the frame index. Import routing in src/store/datasets-dicom.ts: when a chunk group has a single chunk, an UltrasoundMultiframe SOP UID (current or retired), and NumberOfFrames > 1, it diverts to _importCineChunk before the legacy DicomChunkImage path. View integration: - VtkBaseSliceRepresentation.vue: render-slice helper pins VTK slice to 0 for cine; conditional W/L sync (cine pixels are display-encoded, so wlConfig defaults don't clobber them); slice watch is immediate so restored sessions paint the saved frame on first mount. - SliceViewerOverlay shows "Frame: N/M" and hosts a new CineTransport component (play/pause/loop/fps via useIntervalFn, FPS seeded from FrameTime). - Ruler/Rectangle/Polygon widgets use getRenderSlice for their plane manipulator origin so annotations still scope to a frame. - view-configs/slicing.ts overrides the slice range to [0, numberOfFrames - 1] for cine. - image-cache.removeImage now calls dispose() before delete; cine's dispose clears the LRU, drops compressed frame refs, and deletes the vtkImageData. - image-stats early-returns for cine ids — histogram/auto-range is meaningless on display-encoded data. Testing: - 3 vitest tests build a synthetic DICOM in-memory (Explicit VR LE native, encapsulated with populated BOT, encapsulated with empty BOT) to exercise the parser without external fixtures. - New tests/specs/cine-rendering.e2e.ts loads US-MONO2-8-8x-execho.dcm from the BSD-licensed GDCM corpus on SourceForge (cached via the existing wdio onPrepare hook into .tmp/), asserts the cine transport renders with "1 / 8", and asserts the counter advances on ArrowUp. Adds dicom-parser ^1.8.21 (MIT, 0 deps, ~6.9 KB gzipped) to devDependencies.
…meInfo.kind Promote getThumbnail(strategy) to the ProgressiveImage interface with a default null implementation on BaseProgressiveImage. Cine images and LoadedVtkImage inherit the null thumbnail automatically, so the data browser falls back to modality text instead of spinning forever when a cine DICOM is selected. Replace every `instanceof DicomChunkImage|DicomCineImage` check with a read of useDICOMStore().volumeInfo[id]?.kind === 'cine': - isCineImage / getCineImage now branch on the store tag. - datasets-dicom guards both the cine bail and the chunk-volume reuse against the same kind. - segmentGroups skips the SEG-decoding branch for cine ids so a cine image can't reach chunkImage.getModality() and crash. - PatientStudyVolumeBrowser just calls image.getThumbnail() — the workaround added for cine in the previous commit is gone. ThumbnailStrategy moves to progressiveImage.ts; chunkImage.ts re-exports it for back-compat. DicomChunkImage.getThumbnail return type tightens from Promise<any> to Promise<string | null>.
Each 2D view now renders cine from its own local vtkImageData, so two views can hold different frames or play independently without overwriting each other's pixels. The canonical cine vtkImageData stays as a compatibility surface for metadata and older consumers. - DicomCineImage exposes getFrame(n) backed by FrameCache plus an inFlightFrames map so concurrent views share a single decode. - VtkBaseSliceRepresentation builds a per-component CineRenderImage (vtkImageData + RGB scalars), binds the mapper to it for cine, and copies decoded frames into it with stale-token guards. - startLoad() now seeds the canonical scalar buffer via getFrame(0); the public setFrame/currentFrame/getCurrentFrame/decodeToken API is removed. - Scalar probe is unmounted and cleared for cine since it samples the canonical (frame-0) image.
getThumbnail() only ever supported MiddleSlice, so callers always passed the same value and DicomChunkImage threw on anything else. Remove the parameter and the enum.
The slice manipulators set the active view from a watcher on a ref that is bidirectionally synced with sliceConfig.slice. Cine playback writes to that ref every frame, so two playing views fought over which was active and the green selection ring flickered. Fire setActiveView only from the manipulator's user-input callback so the active view changes on real wheel/drag input, not on programmatic writes that come back through syncRef.
Cine images report dimensions [cols, rows, 1], so tools placed on any frame past index 0 had their frame-of-reference resolution fail the bounds check and jumpToTool returned early. Pass allowOutOfBoundsSlice so we still get the axis and can drive the view's slice config.
✅ Deploy Preview for volview-dev ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Tools and rendering panels resolve currentImageID through activeView and viewByID. Select the first startup view directly, then enforce a visible active view when data binds. Session restore reuses that binding path when manifest data IDs are rebound. Added store coverage for startup, normal data load, and session rebinding. Relaxed the ultrasound spacing e2e assertion around the physical-spacing range. Verified with focused Vitest, lint, and full Chrome e2e.
- guard _importCineChunk against an existing non-cine cached image - prune imageErrors[id] in removeImage - defer per-view render-buffer disposal so the mapper drops its reference before the vtkImageData is freed - surface JPEG dimension mismatch via reportError instead of console - type PlayControls.spec wrapper so setProps accepts component props
getRenderSlice now takes an optional toolSlice param; the three tool widgets that previously computed `tool?.slice ?? slice.value` inline just pass both to the helper instead.
Adds three e2e cases on top of the existing scrub test: - play/pause advances and halts the frame counter via DOM label - two cine views playing keeps the active-view ring on the clicked view (regression where playing cine stole active-view focus) - repeated maximize toggles during playback keeps the canvas painting non-black pixels (regression where remount + async JPEG decode left the buffer empty) The remount test uses pydicom's color3d_jpeg_baseline.dcm because the failure mode only repros under the async JPEG decode path; the GDCM MONOCHROME2 corpus is too fast to trigger it. Adds the dataset constant to configTestUtils. All assertions stay on DOM/CSS/canvas pixels — no Pinia or VTK internals.
Consolidate the two store-write helpers behind a single patchConfig(clipId, patch), fold the two-branch imageId watch into a unified pause-both-clips block, drop the clampFpsValue wrapper and the redundant period guards.
Coronal/Sagittal 2D slots kept their orientation when a cine image was dropped on them, and the view-type select was still offered. Force the slice viewer to render Axial for cine images, swap the bottom-right switcher for play controls, and clean up FPS label styling to match surrounding overlay text.
- SliceViewer: inline effectiveOrientation; reuse isCine.value in windowingManipulatorProps - SliceViewerOverlay: extract LOCKED_ORIENTATION_SUFFIXES list and a dedicated isLockedOrientationView computed; the previous showViewTypeSwitcher mixed cine and orientation concerns - PlayControls: drop dead !viewId.value guard (non-nullable string prop)
- Fold applyDecodedFrame and markComplete into startLoad; each was
called exactly once and the post-conditions ('status' === 'incomplete',
loading === true) made the markComplete guards tautological at the
call site
- Drop the unused scalars local in the inlined apply path
- Destructure { cols, rows } from header and { physicalDeltaX, physicalDeltaY }
from region for less line noise
On remount (e.g. maximize toggle), the second watchImmediate called resetCameraClippingRange against the default-position camera because usePersistCameraConfig ran afterwards. The clip planes were computed near the origin while the restored camera sits ~789 units away, so the slice fell outside the clip volume and the canvas painted black. Move usePersistCameraConfig above the clipping-range watcher so syncRef writes the saved position/focalPoint/scale into the camera (sync flush, immediate) before the near/far planes are computed.
jumpToTool and paint's cross-plane sync read view.options.orientation directly, but SliceViewer forces every 2D view to render as Axial when the image is a cine. The mismatch meant Reveal Slice updated only views whose configured orientation happened to be Axial (typically the first slot), and paint sync computed the slice index against the wrong LPS axis on cine. Route both through getEffectiveViewAxis so the store layer agrees with what's on screen.
- Trim verbose comments in VtkSliceView, useAnnotationTool, useVtkView to non-obvious WHY only; note why public cancelAnimation is unusable. - Rename cineActiveOnly -> restrictToActiveView so the flag describes the behavior, not the trigger. - Drop speculative `as any` casts and removeListener fallback in the interactor-lifecycle e2e; align BrowserLogEntry with WDIO's LogEntry.
…tiveViewAxis useSliceInfo and crosshairs read view.options.orientation directly, which diverges from what cine views actually render (Axial). Route all three remaining sites — useSliceInfo (powers annotation widget plane math), crosshairs (cross-view slice sync), and SliceViewer (its own inline cine override) — through getEffectiveViewAxis so the cine collapse is applied consistently and the helper is the single source of truth.
Unify the RGBA→RGB frame copy into a single helper in frameCache, route pixel-spacing through the shared unitToMm util, and skip patchDoubleKeyRecord when slice/playback updates wouldn't change state. Tighten narrative comments left over from the prior simplify pass.
Moves JPEG-Baseline decode off the main thread via a small worker pool and speculatively decodes the next frame on every getFrame, so forward playback hits cache even when the clip exceeds the LRU budget. Measured on a 4-view 2x2 layout of color3d_jpeg_baseline.dcm at 30 FPS: per-view FPS rises from ~20 to ~29 and mean decode latency drops from ~50ms to ~12ms.
Have the worker produce a 3-component Uint8Array directly so the transferred buffer is 25% smaller and the main thread publishes frames with a single typed-array copy instead of a per-pixel RGBA→RGB loop. decodeNativeFrame follows suit and writes RGB without the dummy alpha channel.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.