Skip to content

Sparse test - #1167

Merged
henrydingliu merged 10 commits into
potarafrom
sparse_test
Jul 29, 2026
Merged

Sparse test#1167
henrydingliu merged 10 commits into
potarafrom
sparse_test

Conversation

@henrydingliu

@henrydingliu henrydingliu commented Jul 29, 2026

Copy link
Copy Markdown
Member

Summary of Changes

Related GitHub Issue(s)

Additional Context for Reviewers

  • I passed tests locally for both code (uv run pytest) and documentation changes (uv run jb build docs --builder=custom --custom-builder=doctest)

Note

Medium Risk
Changes Mack standard-error recursion and sparse reduction helpers used in production paths, not only tests; incorrect backend alignment could affect numerical results on sparse triangles.

Overview
Improves sparse-array test coverage and fixes backend mismatches when triangles mix numpy and sparse after auto_sparse or reductions.

Test harness: Sample fixtures now set backend per triangle via set_backend() instead of toggling global ARRAY_BACKEND. The full prism fixture runs sparse-only; a smaller prism_convert slice backs the auto-sparse test.

Sparse utilities: nan_to_num accepts a configurable NaN fill value. New nanquantile / nanmedian on COO arrays support median/quantile on sparse-backed triangles.

Runtime fixes: Mack recursion and summary_ force matching backends before sums and concatenation (e.g. after sum triggers auto-sparse). DevelopmentML._prep_w_ml densifies weights for scikit-learn. Several tests assert Triangle equality or numpy arrays instead of backend-specific xp.testing.

Test tweaks: Cape Cod zero-triangle uses .at; Mack malformed test uses the raa fixture.

Reviewed by Cursor Bugbot for commit ac9e1b5. Bugbot is set up for automated code reviews on this repo. Configure here.

henrydingliu and others added 10 commits July 25, 2026 22:01
expecting errors
* fixing fixture

expecting errors

* fixing prism

* trying another approach

* some fixes

* more fixes

* more fixes

* fix for learning

* more learning fix

* restoring test

still fails
* further improving prism_convert

* capecod & mack fixes

* fixing test_utilities and removing sparse.testing

* more fixes

* implementing nanquantile and nanmedian

* improving comment, adding tests, fixing bugbot

* fixing test

* removing empty check
@henrydingliu
henrydingliu merged commit c2ad446 into potara Jul 29, 2026
9 of 27 checks passed

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want reviews to match your repository better? Bugbot Learning can learn team-specific rules from PR activity. A team admin can enable Learning in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ac9e1b5. Configure here.


for start, end in zip(starts, ends):
gid = group_ids[start]
out[gid] = np.nanquantile(values[start:end], q)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Empty sparse quantile crashes

Medium Severity

When a multi-dimensional COO has no stored values (nnz == 0), group_ids is empty but the loop still runs with starts=[0], so group_ids[start] raises IndexError. An all-missing reduction should return NaNs, matching np.nanquantile.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit ac9e1b5. Configure here.

@github-actions

Copy link
Copy Markdown

Pyright Type Completeness

View the full pyright --verifytypes output for this commit

Project (full chainladder package, at this PR's head): 15.1% of exported symbols fully typed (195 / 1294)

Known Ambiguous Unknown Total
Project (head) 195 110 989 1294

Other symbols referenced but not exported by chainladder: 13

Known Ambiguous Unknown Total
Other (head) 3 1 9 13

Symbols without documentation:

  • Functions without docstring: 315
  • Functions without default param: 0
  • Classes without docstring: 10

Patch (exported symbols added or changed by this PR): 28.6% fully typed (2 / 7); 2 no longer exported

Known Ambiguous Unknown Total
Patch 2 0 5 7
Patch symbol details
Symbol Status Change
chainladder.utils.sparse.nanmedian ❌ unknown new
chainladder.utils.sparse.nanquantile ❌ unknown new
chainladder.utils.tests.test_sparse.test_1D_nanquantile ✅ known new
chainladder.utils.tests.test_sparse.test_keepdims_nanquantile ✅ known new
chainladder.utils.tests.test_utilities.test_maximum_2 ❌ unknown new
chainladder.utils.tests.test_utilities.test_maximum_minimum_1 ❌ unknown new
chainladder.utils.tests.test_utilities.test_minimum_2 ❌ unknown new
chainladder.utils.tests.test_utilities.test_maximum no longer exported (was ❌ unknown)
chainladder.utils.tests.test_utilities.test_minimum no longer exported (was ❌ unknown)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant