Skip to content

Converge the dense read path onto the shared WSI reader#222

Merged
clemsgrs merged 2 commits into
mainfrom
converge-dense-read-path
Jul 15, 2026
Merged

Converge the dense read path onto the shared WSI reader#222
clemsgrs merged 2 commits into
mainfrom
converge-dense-read-path

Conversation

@clemsgrs

@clemsgrs clemsgrs commented Jul 15, 2026

Copy link
Copy Markdown
Owner

Closes #216

What this does

Converges the dense extraction read path onto the same batched WSI reader the pooled path uses, driven by an hs2p TilingResult, and retires iter_regions_dense's ad-hoc (coordinates, requested_spacing_um, wsi) contract.

Before this change the dense path had drifted from the pooled path: it took raw coords + a scalar spacing + an injectable wsi and read one region at a time through the high-level per-call wsi.read_region_at_spacing, instead of keeping the TilingResult and reading through the low-level batched WSIRegionReader (cuCIM read_regions(num_workers=…)). That drift was the root cause of the "overlap reads with the forward" problem; convergence removes the need for any bespoke prefetcher (supersedes #220).

Changes

  • iter_regions_dense is now driven by a TilingResult. It drops wsi, coordinates, requested_spacing_um, target_size, tolerance and takes tiling_result (carrying tiles.x/y, read_level, read_tile_size_px, requested_tile_size_px, image_path, backend) plus two reader knobs (backend override, num_workers). All encode-side params are unchanged (model, device, window_size, overlap, feature_kind, attention_*, batch_size, precision, output_dtype, pad_mode, image_pad_value, dense_transform).
  • Reads go through the shared WSIRegionReader (batched cuCIM read_regions; serial read_region for non-cucim backends). The bespoke per-region read_region_at_spacing loop and the torch.stack([_read_padded(loc) …]) serial read are gone. No standalone ThreadPoolExecutor.
  • Area geometry-resize added to the shared reader. WSIRegionReader gains resize_to_px / interpolation (default resize_to_px=None → no resize, so the hierarchical unfold path is untouched). The dense reader passes requested_tile_size_px + "area"; the resize is applied iff read_tile_size_px != requested_tile_size_px, reusing hs2p's own resize_array(..., interpolation="area"). area is preserved (not switched to bilinear — a real, non-tolerance pixel change; see docs/adr/0002).
  • Backend resolved the way the pooled path does (resolve_slide_backend): an explicit backend wins, else it falls back to the TilingResult's own backend ("auto" resolution).
  • get_dense_transform (normalization-only) is untouched — the encoder still receives the full requested_tile_size_px tile; dynamic_img_size handles it. No model-input resize was added to the dense path.
  • Docs (docs/api.rst) and module docstrings updated to the new TilingResult-driven contract.

Output preservation (ADR-0002)

The converged dense pixels are identical to the pre-convergence output:

  • Same level. TilingResult.read_level / read_tile_size_px are set directly from plan_spacing_read(requested_spacing_um=…, target_size_px=(requested_tile_size_px, requested_tile_size_px), tolerance=…) at tiling time (hs2p/tiling/generate.py). That is the exact call read_region_at_spacing runs internally, so the level + read size are the same ones the old dense path would have picked per read.
  • Same resize. The reader reads read_tile_size_px at read_level then area-resizes to requested_tile_size_px via the identical resize_array(..., "area") the old path used (when levels match exactly it is a no-op, lossless).
  • Same photometrics. Each region is fed to the same normalization-only get_dense_transform via Image.fromarray, unchanged.

Same level + same resize_array → identical read pixels → dense grids are preserved within the encode-side ADR-0002 float tolerance (per-grid cosine ≥ 1 − 1e-4). The rewritten test suite pins byte-identity of the streamed grids against a hand-rolled read → (area-resize) → transform → pad → encode reference, for both feature kinds and the whole + sliding-window paths, including the resized (read != requested) case.

Breaking change / downstream

This is a breaking dense-API change (hard replacement, no back-compat shim — the old signature is gone). The required downstream update is soma#279: soma/dense_slide_extraction.py must stop unpacking the TilingResult into coords + spacing and pass it through instead. soma stays on the prior slide2vec until updated. Note for release notes.

Tests

tests/test_dense_regions.py rewritten onto the new offline seam — monkeypatch slide2vec.data.tile_reader._open_wsi_backend to return a fake backend serving canned regions via read_regions / read_region. Preserves the prior behavioural coverage (coordinate order, eager validation before any read, empty input, output_dtype, bfloat16 rejection, one-batch-at-a-time streaming) and adds: the area-resize case (read_tile_size_px != requested_tile_size_px), a serial-backend case (non-cucim read_region), and an explicit batch-invariance check (composition irrelevant, only B matters).

Full suite (CPU, no GPU): pytest -q -m "not heavy" --no-cov tests467 passed, 25 deselected. (--no-cov used because the repo's default --cov sqlite flakes on this network filesystem; no coverage-relevant change.)

Notes for the reviewer

clemsgrs added 2 commits July 15, 2026 15:42
Drive iter_regions_dense from an hs2p TilingResult and read through the
shared batched WSIRegionReader (cuCIM read_regions) instead of the ad-hoc
(coordinates, requested_spacing_um, wsi) contract + per-region
read_region_at_spacing loop.

The TilingResult already resolved spacing->level at tiling time
(read_level / read_tile_size_px / requested_tile_size_px via
plan_spacing_read), so the reader no longer re-plans per read: it reads
read_tile_size_px at read_level and area-resizes to requested_tile_size_px
when they differ, reusing hs2p resize_array(..., "area") -- the identical
op read_region_at_spacing performed, so the read pixels are unchanged.

- WSIRegionReader gains resize_to_px / interpolation (default off, so the
  hierarchical unfold path is unaffected); the dense reader passes
  requested_tile_size_px + "area".
- iter_regions_dense drops wsi/coordinates/requested_spacing_um/target_size
  and takes a TilingResult (+ backend / num_workers knobs); all encode-side
  params are unchanged. Breaking dense-API change (hard replacement).
- Backend resolved the way the pooled path does (resolve_slide_backend).
- get_dense_transform (normalization-only) left untouched.
- Tests rewritten onto the new offline seam (monkeypatch
  tile_reader._open_wsi_backend), preserving coordinate order, eager
  validation, empty input, output_dtype, bf16 rejection, streaming, and
  batch-invariance; adds the area-resize and serial-backend cases.
@clemsgrs
clemsgrs merged commit b3ecd17 into main Jul 15, 2026
3 checks passed
@clemsgrs
clemsgrs deleted the converge-dense-read-path branch July 15, 2026 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Converge the dense read path onto the shared WSI reader (retire the ad-hoc coords+spacing contract)

1 participant