Skip to content

ehrdata_blobs stores the time series in the default 3D .X - #285

Open
sueoglu wants to merge 3 commits into
mainfrom
fix/issue-284
Open

ehrdata_blobs stores the time series in the default 3D .X#285
sueoglu wants to merge 3 commits into
mainfrom
fix/issue-284

Conversation

@sueoglu

@sueoglu sueoglu commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Closes #284

ehrdata_blobs was the last dataset loader still defaulting its tensor to .layers["tem_data"]. It now takes layer: str | None = None and puts the time series in the default 3D .X, matching physionet2012 / physionet2019.

What changed

The 2D static X is gone. The generator built the tensor first, then overwrote X with a per-observation mid-timepoint slice of it. Verified before removal, the old X was reproducible from the tensor to max abs diff: 0.0.

Docstring fixes. layer was documented twice in the Args block, and the first entry claimed "If not specified, uses X", the opposite of what the old default did. Collapsed to one accurate entry. The intermediate X variable is renamed base_values, since nothing named X is returned any more and the old name actively misleads about what .X holds.

Also corrected the to_pandas docstring, which advertised shape of .X: (2, 2) plus a .R field that no longer exists.

Requires anndata >=0.13 to use the default

ehrdata_blobs() with defaults now raises on anndata<0.13, which rejects a >2D X at construction:

ValueError: X needs to be 2-dimensional, not 3-dimensional.

On older anndata the time series has to go in a layer, so ehrdata_blobs(layer="tem_data", ...) remains the way to use it there.

Tests
tests/dt/test_dt.py updated: assertions read the tensor from .X, and a new test_ehrdata_blobs_layer covers the named-layer path, which had no coverage before, it asserts X is None and that both paths produce an identical tensor.

Two assertion groups were deleted rather than adapted, because they pinned the snapshot relationship this PR removes by design:

  • the loop asserting edata.X[i, v] == edata.layers[...][i, v, mid_idx] for categorical variables
  • the check that the tensor at t=0 correlates with X

@sueoglu
sueoglu requested a review from eroell August 6, 2026 14:31
@sueoglu
sueoglu marked this pull request as ready for review August 6, 2026 14:34

@eroell eroell left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good!

If you merge main into this branch, the last CI check should also pass.

Please add an entry in the release notes (Claude sometimes creates a bit overly verbose release note comments, maybe shorten the first suggestion it makes), then this is ready to merge!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ehrdata_blobs still defaults its tensor to .layers["tem_data"] instead of .X

2 participants