Skip to content

Stay in xarray in do() producers; simplify with native broadcasting#357

Open
drbenvincent wants to merge 3 commits into
mainfrom
fix/356-stay-in-xarray-do-producers
Open

Stay in xarray in do() producers; simplify with native broadcasting#357
drbenvincent wants to merge 3 commits into
mainfrom
fix/356-stay-in-xarray-do-producers

Conversation

@drbenvincent

@drbenvincent drbenvincent commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Closes #356.

The do() producers (run_do_pymc, run_do_panel_unified) now assemble labelled xr.DataArrays end-to-end. The leftover .to_numpy() → reshape → _da_chain_draw* round-trips from the flat-dict era are gone, and so is the adapter machinery that existed only to re-wrap numpy back into labelled arrays.

Rather than swap the old numpy wrappers for new single-use ones, the producers lean on xarray's native broadcasting and named reductions so they read as label algebra:

  • Constants by broadcasting. A coord-carrying ones = _chain_draw_ones(post) turns every intervention/exogenous fill into ones * value — no manual dims/coords or np.full/np.broadcast_to.
  • Mean path inlines inverse_link → isel(subgroup) → .mean(obs_dims); the g-computation standardization is one expression.
  • Predictive path just renames the observation dim to unit; _stack_sample flattens it at the public .draws() boundary.
  • Panel uses _panel_unit_mean (average units → (time, chain, draw)) and _spread_over_time for scalar/per-time interventions.
  • _apply_inverse_link dropped its dead numpy branch and dispatches np.exp/pt.exp on the input type, operating directly on DataArrays (no xr.apply_ufunc).

Helpers removed: _da_chain_draw, _da_chain_draw_unit, _da_time_chain_draw, _const_chain_draw, _const_time_chain_draw, _select_subgroup, _mean_to_chain_draw, _predictive_to_chain_draw, _panel_to_time_chain_draw, _extra_dims, _chain_draw_coords. Kept/added only small label-only helpers used across multiple call sites: _obs_dims, _chain_draw_ones, _spread_over_time, _panel_unit_mean, _fill_missing_posterior_rvs, _predictive_source, _exog_value.

Small cleanups: _model.py prob() / sensitivity() use public .draws(); repr sample counts come from dim sizes.

Size: simulate.py is 79 lines smaller than main (985 vs 1064), with no numpy bridges remaining in the producers.

Test plan

  • uv run pytest tests/test_xarray_storage.py tests/test_estimand_result_api.py tests/test_reprs.py tests/test_panel_by_time.py tests/test_causal_queries.py tests/test_att_atu.py
  • Full make test: all pass, 87.8% coverage
  • make lint (ruff, ruff-format, mypy, license checks)
  • Grep gate: no .to_numpy() / numpy re-wrap on PyMC/ArviZ outputs in producers
  • Added coverage for time-varying (array-valued) panel interventions — the (n_times,) set path through run_do_panel_unified was previously exercised only with scalar values. New tests assert the intervened variable's per-time draws track the supplied array exactly.

Notes for reviewers

  • No tests referenced the deleted helpers or the old xr→np→xr machinery, so none were removed; the inverse-link mean path (bernoulli/poisson) is covered by test_bernoulli.py / test_families.py.
  • _panel_unit_mean and the predictive path identify the observation dims positionally (obs[0] = time, obs[1] = unit) because the scan compiler names them generically (mu_sales_dim_0 / _dim_1). This (time, unit) ordering was confirmed against a live scan model; if the compiler ever changes it, this is the spot that would need a coord-name lookup instead.

Replace numpy extract-and-re-wrap paths in run_do_pymc and
run_do_panel_unified with xarray slice/mean/transpose operations.
Remove _da_chain_draw* helpers and use public draws() at API boundaries.

Closes #356

Co-authored-by: Cursor <[email protected]>
@drbenvincent drbenvincent added the Refactor Refactor, clean up, or improvement with no visible changes to the user label Jun 26, 2026
Replace the single-use numpy-rewrap helpers with broadcasting and named
reductions so producers read as label algebra. Constants become
`ones * value` off a coord-carrying template; the mean path inlines
inverse-link/subgroup/mean; the predictive path just renames the obs dim
to `unit`. Drops `_const_*`, `_select_subgroup`, `_mean_to_chain_draw`,
`_predictive_to_chain_draw`, `_panel_to_time_chain_draw`,
`_chain_draw_coords`, and the dead numpy branch of `_apply_inverse_link`.

Net 79 lines smaller than main with no numpy bridges left in producers.

Co-authored-by: Cursor <[email protected]>
@drbenvincent drbenvincent changed the title Stay in xarray in do() producers; delete numpy re-wrap machinery Stay in xarray in do() producers; simplify with native broadcasting Jun 26, 2026
The (n_times,) intervention path through run_do_panel_unified had no
coverage; every panel simulate_over='time' test used a scalar set value.
Add characterization tests asserting the intervened variable's per-time
draws track the supplied array exactly and the outcome stays finite.

Co-authored-by: Cursor <[email protected]>
@drbenvincent

Copy link
Copy Markdown
Collaborator Author

@BugBot review

@cursor

cursor Bot commented Jun 26, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches core causal do() propagation for cross-sectional and panel models (g-computation standardization, predictive unit flattening, time-varying interventions); behavior is intended to match prior semantics but regressions would affect ATE/CATE/ATT outputs.

Overview
do() simulation (run_do_pymc, run_do_panel_unified) now assembles labelled xr.DataArrays directly instead of .to_numpy() → reshape → manual wrappers. Interventions and exogenous fills use _chain_draw_ones broadcasting; the mean path does inverse link → subgroup isel.mean(obs_dims) on arrays; panel paths use _spread_over_time and _panel_unit_mean.

Removed the old _da_chain_draw* helpers and consolidated shared logic into _fill_missing_posterior_rvs, _predictive_source, _exog_value, and _sample_count (repr draw counts). _apply_inverse_link works on DataArrays and PyTensor via a single dispatch. PathModel.sensitivity() and prob() call public .draws() instead of ._draw().

New panel tests assert array-valued set interventions track the supplied per-time values through by_time().

Reviewed by Cursor Bugbot for commit 858bfb9. Bugbot is set up for automated code reviews on this repo. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 858bfb9. Configure here.

@drbenvincent drbenvincent self-assigned this Jun 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Refactor Refactor, clean up, or improvement with no visible changes to the user

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stay in xarray in do() producers; delete numpy re-wrap machinery

1 participant