Propose MosaicAgent as the chart-generating agent

[SuMayaBee]

Summary

@ahuang11 this follows up on the Mosaic/vgplot idea you raised a while back.

I've been contributing to Vega-Lite, and Lumen happens to use Vega-Lite too, through VegaLiteAgent, for chart generation. While working on Vega-Lite stuff I came across Mosaic (the vgplot library) and ended up contributing there as well. The more time I spent in Mosaic's codebase, the more it felt like its architecture lines up almost perfectly with how Lumen is already built.

Lumen is built on DuckDB. Mosaic is also built around DuckDB, and its whole design is basically: keep the data in the database, only send query results to the browser instead of shipping raw rows over. VegaLiteAgent doesn't do that. It pulls the full result out of the database and embeds it as inline JSON in the chart spec.

So I started thinking, since Mosaic and Lumen already line up so well on the data side, why not build a MosaicAgent for Lumen and let it be the agent Lumen reaches for when generating charts?

How it fits in

Lumen already has this pattern for Vega-Lite: an agent that gets the LLM to write a spec, and a view that renders it. I built MosaicAgent to follow the same shape, just targeting Mosaic's spec format instead.

flowchart TD
    U[User asks for a chart] --> A[MosaicAgent]
    A -->|LLM writes a Mosaic spec| S[mosaic-spec YAML]
    S --> V[MosaicView]
    V --> C[Interactive chart]

Loading

That's the version I've built so far. MosaicWidget's Python API actually gives you two ways to hand it data:

• con=<a duckdb connection>: it queries directly against a connection you hand it
• data={name: df}: it makes its own fresh duckdb.connect() internally and loads your DataFrame into that

Right now I'm using data=. There's a further step we could take on top of that: switch to con= and point it at Lumen's existing DuckDB connection directly, so there's no copying involved at all. That's not quite as simple as it sounds though, since a Lumen pipeline can apply filters and SQL transforms on top of the raw table, so we'd need to register the pipeline's actual (filtered/transformed) query as a view rather than just handing over the connection and pointing at the raw table. It also only works when the pipeline is backed by DuckDBSource specifically; plenty of Lumen sources (Snowflake, Intake, plain files, etc.) don't expose a DuckDB connection at all, so there'd need to be a fallback for those. Still very doable, just more design work than what I've built so far.

flowchart TD
    subgraph Built["Built so far"]
        direction LR
        P1[Pipeline data] -->|copy into a DataFrame| D1[data=]
        D1 --> M1[MosaicWidget's own DuckDB]
    end

    subgraph Next["Possible next step, DuckDBSource only"]
        direction LR
        P2[Pipeline's filtered/transformed query] -->|register as a view| V2[View in Lumen's DuckDB]
        V2 -->|con=| M2[Lumen's existing DuckDB connection]
    end

    Built ~~~ Next

Loading

Everything else still falls back to the copy path above, since only DuckDBSource has a connection to hand over in the first place.

What this actually involves

• MosaicAgent: writes a Mosaic spec, same idea as VegaLiteAgent but a different output format
• MosaicView: renders it via the mosaic-widget widget
• some prompt examples so the LLM actually writes valid specs
• mosaic-widget as a dependency

[ahuang11]

I think it's best to have a Mosaic pane in Panel first.