Issues with speed when calling dbt-core (Typer)

[joshuataylor]

Honestly, just have two completions.

hyperfine --warmup 3 'zsh -c "source _dbt; words=(dbt run --se); CURRENT=3; _dbt"' --shell=zsh

Before:

Benchmark 1: zsh -c "source _dbt; words=(dbt run --se); CURRENT=3; _dbt"
  Time (mean ± σ):       6.8 ms ±   0.3 ms    [User: 2.3 ms, System: 3.6 ms]
  Range (min … max):     6.3 ms …   8.6 ms    266 runs

After:

Benchmark 1: zsh -c "source _dbt; words=(dbt run --se); CURRENT=3; _dbt"
  Time (mean ± σ):      2.348 s ±  0.709 s    [User: 2.050 s, System: 0.289 s]
  Range (min … max):    1.716 s …  3.490 s    10 runs

[b-per]

Yep. You found the PR where we changed the behavior to be dynamic (and to add support for Fusion).

To be honest, I am not sure what to do here to provide the best experience ...

click completion is just so slow ; but handrolling all the parameters is not sustainable either

On my side

• for Fusion, I get about 250ms ; which is not instantaneous but feels acceptable
• for dbt Core, I get 1.1s ; which is not great, but still much faster than your 2.3s average

And completion for model names after -s/--select should still be very fast.

[b-per]

Did I get nerd sniped: yes

Let me know if you manage to test the completion from that PR.
You can't use hyperfine though to check the perf as it can't leverage the ootb cache from zsh that we now leverage. Just try to run the completion "by hand"

[joshuataylor]

Amazing, will take a look, I actually was curious if we could generate completions as part of a script, so we could use a github action or something to rebuild, and try and figure out how to handle different dbt-core versions etc (to show flags per version).

Got Claude to spit this out, warning have only ran it, no idea how good/bad the code is (haven't reviewed it, was going to tonight, but you out nerd-sniped me, well done). https://gist.github.com/joshuataylor/5a49cbeed4901b125a1fdc46e967512d

[b-per]

The updated approach in the PR I shared takes care of all the potential differences in different versions.

It uses the click script the first time it calls a given completion (more or less one time per "subcommand" to oversimplify how it works) and then caches it and retrieves it the next time the same completion is requested. If the dbt executable changes ; it invalidates the cache.