Skip to content

to_dataset fails on the pivot fallback when a sql_transform projects a dim away #2048

Description

@ghostiee-11

XArraySQLSource.to_dataset falls back to pivoting the long-form result when the installed xarray-sql has no native to_dataset. A projecting sql_transform drops a dim from the result, and the rows that dim separated
then collapse onto the same index, so the pivot raises:

ValueError: cannot convert a DataFrame with a non-unique MultiIndex into xarray

lumen/tests/sources/test_xarray_sql.py::TestToDataset::test_to_dataset_ignores_projected_away_dim covers this and fails whenever the fallback branch runs. The native path handles it, so whether the test passes depends on the installed xarray-sql
version rather than on lumen.

The fallback has to decide what a projected-away dim means before it can pivot: aggregate the duplicate rows, or refuse and say so. Right now it does neither and fails inside xarray.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions