Skip to content

Node selection for dataform compile (compile/output a single model) #2203

Description

@alandrebraga

dataform compile always emits the entire compiled graph. There is no way to ask
the CLI for the compiled SQL of a single model (or a subset of models) the way
--actions / --tags already work for dataform run.

I'd like dataform compile (and dataform compile --json) to accept the existing
selection flags so I can get the compiled SQL for just the action(s) I care about, e.g.:

dataform compile --actions my_model --json
dataform compile --tags daily --include_deps --json

This mirrors dbt's dbt compile --select my_model, which is a feature I rely on for
quickly inspecting the rendered SQL of one model without scrolling through the whole graph
or post-processing the JSON with jq.

Looking at code in this repo i understand that the compiler must evaluate the whole project because ref() resolution and dependency graph require every action to be registered. So this request is not about partial compilation its purely about filtering the compile ouput to the selected actions (similar to what prune does for run) its an UX improvement in CLI.

Possible Implementation

The filtering logic seems to already exists and is reused prune() in
cli/api/commands/prune.ts takes a CompiledGraph + a
RunConfig (actions, tags, includeDependencies, includeDependents) and returns a
filtered CompiledGraph. dataform run/build already call it.

The change is to make the compile command optionally apply prune() before printing:

Open question i have

I would like to work on this but have some questions to discuss

  1. When a selector is present, should compilation
    errors from unselected actions still fail the command / be printed? I lean toward keeping
    graph-level errors as-is (the whole project still had to compile) but suppressing per-action
    errors unrelated to the selection

  2. f the selector matches nothing, should compile warn and
    exit non-zero (like a "no actions" case) or just emit an empty graph?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions