Summary
Reduce redundant JPEG macro-tile decoding in the dense read path by giving it read locality — the
dense counterpart of the pooled path's SuperTileBatchSampler. Follow-up to #216 (which adds
thread-pinning + prefetch but does not address decode amplification).
The cost model
Decode cost is per distinct macro-tile touched, not per read. Measured on a slide with
4096×4096 JPEG tiles: a 512×512 region read costs the same 49 ms as a 256×256 one — cuCIM decodes
the containing 4096² macro-tile either way. Reading the whole macro-tile costs 248 ms, i.e. ~13×
cheaper per pixel. On a 512-px-tiled slide a read is 8 ms and this effect vanishes — so it is
entirely a function of the WSI's internal tiling vs the ROI size.
iter_regions_dense today reads one ROI at a time via wsi.read_region_at_spacing — no grouping —
so it pays a full macro-tile decode per ROI. The pooled path already solves this
(SuperTileBatchSampler + WSITileReader, slide2vec/data/tile_reader.py): group nearby tiles,
read one larger region covering them, crop each out. The dense path does not use any of it.
Options (decide during implementation)
Strict tile-alignment (snapping ROI coords to the macro-tile grid) is not viable — arbitrary
ROI coordinates straddle macro-tile borders. Two workable designs:
- Reuse the supertile approach (recommended first): group a slide's ROIs into clusters, read
one region per cluster, crop. Near-optimal when ROIs are dense/clustered (a supertile over a
contiguous annotated patch decodes each underlying macro-tile ~once). One refinement worth adding:
size the supertile read window to a multiple of the WSI's native TileWidth/TileLength (TIFF
metadata) so border macro-tiles aren't re-decoded across adjacent supertiles — snap the read
window, not the coordinates.
- Decoded-macro-tile LRU: cache at the file's native tile granularity — strictly more general
(robust to sparse, overlapping, and border-straddling coords with no supertile-size knob; a border
ROI just touches two cache entries), at the cost of memory + backend plumbing to address native
tiles.
Inherent floor: neither helps sparse ROIs (isolated coordinates far apart) — an isolated ROI
forces a full macro-tile decode regardless. That's physics, not a code smell. For soma's
slide-manifest segmentation ROIs are typically clustered, so option 1 is likely ~90% of the win at a
fraction of option 2's complexity.
Before committing: measure the real ROI distribution
The single fact that decides whether this is worth wiring in is the spatial distribution of ROIs in
a real soma slide-manifest run (clustered vs sparse). If clustered, supertile grouping pays off; if
sparse, the #216 prefetcher alone is the ceiling and this issue is moot. Worth a quick measurement
first.
Acceptance
- Dense reads on macro-tiled slides decode each touched macro-tile ~once for clustered ROIs, rather
than once per ROI.
- No change to emitted grids beyond float tolerance — reordering reads for locality must not change
B (composition is irrelevant; see docs/adr/0002-features-are-reproducible-to-a-tolerance.md).
References
Summary
Reduce redundant JPEG macro-tile decoding in the dense read path by giving it read locality — the
dense counterpart of the pooled path's
SuperTileBatchSampler. Follow-up to #216 (which addsthread-pinning + prefetch but does not address decode amplification).
The cost model
Decode cost is per distinct macro-tile touched, not per read. Measured on a slide with
4096×4096 JPEG tiles: a 512×512 region read costs the same 49 ms as a 256×256 one — cuCIM decodes
the containing 4096² macro-tile either way. Reading the whole macro-tile costs 248 ms, i.e. ~13×
cheaper per pixel. On a 512-px-tiled slide a read is 8 ms and this effect vanishes — so it is
entirely a function of the WSI's internal tiling vs the ROI size.
iter_regions_densetoday reads one ROI at a time viawsi.read_region_at_spacing— no grouping —so it pays a full macro-tile decode per ROI. The pooled path already solves this
(
SuperTileBatchSampler+WSITileReader,slide2vec/data/tile_reader.py): group nearby tiles,read one larger region covering them, crop each out. The dense path does not use any of it.
Options (decide during implementation)
Strict tile-alignment (snapping ROI coords to the macro-tile grid) is not viable — arbitrary
ROI coordinates straddle macro-tile borders. Two workable designs:
one region per cluster, crop. Near-optimal when ROIs are dense/clustered (a supertile over a
contiguous annotated patch decodes each underlying macro-tile ~once). One refinement worth adding:
size the supertile read window to a multiple of the WSI's native
TileWidth/TileLength(TIFFmetadata) so border macro-tiles aren't re-decoded across adjacent supertiles — snap the read
window, not the coordinates.
(robust to sparse, overlapping, and border-straddling coords with no supertile-size knob; a border
ROI just touches two cache entries), at the cost of memory + backend plumbing to address native
tiles.
Inherent floor: neither helps sparse ROIs (isolated coordinates far apart) — an isolated ROI
forces a full macro-tile decode regardless. That's physics, not a code smell. For soma's
slide-manifest segmentation ROIs are typically clustered, so option 1 is likely ~90% of the win at a
fraction of option 2's complexity.
Before committing: measure the real ROI distribution
The single fact that decides whether this is worth wiring in is the spatial distribution of ROIs in
a real soma slide-manifest run (clustered vs sparse). If clustered, supertile grouping pays off; if
sparse, the #216 prefetcher alone is the ceiling and this issue is moot. Worth a quick measurement
first.
Acceptance
than once per ROI.
B(composition is irrelevant; seedocs/adr/0002-features-are-reproducible-to-a-tolerance.md).References
slide2vec/runtime/dense_regions.py(per-ROIread_region_at_spacing)slide2vec/data/tile_reader.py(SuperTileBatchSampler,WSITileReader, supertile plans)