perf(cli): experimental bundled distribution — 306MB/21.8s → 26MB/4.3s cold npx

[shapirodaniel]

Description

this PR attempts to cut cold start times on npx --yes @sanity/cli by inlining most dependencies, and loading the rest as they're needed

Benchmarks

Initial bundle setup

cli-rewrite-experiment on  perf/bundled-cli-experiment [?] via  v26.4.0 on ☁️   
❯ node scripts/bundle-experiment/bundle.mjs 
variant: aggressive
entries: 728
worker stubs: 16
  url-ref stub: _chunks/sanitizeIcon.js -> actions/manifest/sanitizeIcon
  url-ref stub: _chunks/iconResolver.js -> actions/manifest/iconResolver
  url-ref stub: manifest/iconResolver.js -> actions/manifest/iconResolver
chunks containing import.meta.url: 34
  audit: _chunks/deleteSchemaAction-CJPnJopW.js
  audit: _chunks/deploySchemas-BEZFqriv.js
  audit: _chunks/deployStudioSchemasAndManifests-CmIJz98n.js
  audit: _chunks/dist-BgVCvmzj.js
  audit: _chunks/dist-CSGna9s6.js
  audit: _chunks/dist-npl98oVv.js
  audit: _chunks/execScript-ocPK8e7F.js
  audit: _chunks/extractCoreAppManifest-B9vxDw6b.js
  audit: _chunks/extractGraphQLAPIs-111wsji9.js
  audit: _chunks/extractManifest-CT_H7kDA.js
  audit: _chunks/extractWorkspaceManifest-BUTk47Z-.js
  audit: _chunks/getCliConfig-BlvRC1SF.js
  audit: _chunks/getCliConfigSync-BOCfzlIt.js
  audit: _chunks/getCliVersion-DDuSoYN7.js
  audit: _chunks/getGraphQLAPIs-5alPUg0g.js
  audit: _chunks/importModule-DGupQX-J.js
  audit: _chunks/isInstalledUsingYarn-DreC02zV.js
  audit: _chunks/listSchemas-0WkjnyRY.js
  audit: _chunks/plugin-module-federation-Md2EYCQ1.js
  audit: _chunks/pluginDts-CGDIZCsD-C5BtPPGF.js
  audit: _chunks/previewServer-q_rbcTid.js
  audit: _chunks/readStudioConfig-DD2PLeYf.js
  audit: _chunks/renderDocument-UpI-RjCe.js
  audit: _chunks/rolldown-runtime-DSjMdhoD.js
  audit: _chunks/runSchemaExtraction-CrFM19pU.js
  audit: _chunks/setupSkills-DZJ4lU6h.js
  audit: _chunks/showNotificationUpdate-CDX6wxdB.js
  audit: _chunks/startStudioDevServer-3IPwxjFi.js
  audit: _chunks/studioWorkerTask-BuFPa1bv.js
  audit: _chunks/tsxWorkerTask-CSkx4Mef.js
  audit: _chunks/updateChecker-DcmzRkld.js
  audit: _chunks/uploadSchemaToLexicon-Ccu10TOr.js
  audit: _chunks/validate-DEh0Fy9W.js
  audit: _chunks/validateAction-QH6FxvQp.js
sanity-cli-7.12.1-bundle-experiment.0.tgz
stage dist:  14M
tarball: /Users/daniel.shapiro/Documents/Personal/cli-rewrite-experiment/.bundle-stage/sanity-cli-7.12.1-bundle-experiment.0.tgz (2.9MB)

Current sanity CLI

cli-rewrite-experiment on  perf/bundled-cli-experiment [?] via  v26.4.0 on ☁️   took 19s 
❯ bash scripts/bundle-experiment/bench-cold-start.sh @sanity/[email protected] 3 --help
spec: @sanity/[email protected] | args: --help | runs: 3
workdir: /var/folders/yt/_34xbc1x6pnbvk1_7j4yvkm00000gp/T//cli-bench.wtuxZ6
cold run 1: 27406ms | installed: 307MB | top-level pkg jsons: 260
warm run  : 1431ms (install cached; measures npx overhead + boot)
cold run 2: 25119ms | installed: 307MB | top-level pkg jsons: 260
cold run 3: 24999ms | installed: 307MB | top-level pkg jsons: 260
RESULT spec=@sanity/[email protected] cold_median_ms=25119 installed_mb=307

Experimental bundled version of CLI

cli-rewrite-experiment on  perf/bundled-cli-experiment [?] via  v26.4.0 on ☁️   
❯ bash scripts/bundle-experiment/bench-cold-start.sh .bundle-stage/sanity-cli-7.12.1-bundle-experiment.0.tgz 3 --help
spec: .bundle-stage/sanity-cli-7.12.1-bundle-experiment.0.tgz | args: --help | runs: 3
workdir: /var/folders/yt/_34xbc1x6pnbvk1_7j4yvkm00000gp/T//cli-bench.VQ1X0A
cold run 1: 5474ms | installed: 26MB | top-level pkg jsons: 20
warm run  : 2396ms (install cached; measures npx overhead + boot)
cold run 2: 5272ms | installed: 26MB | top-level pkg jsons: 20
cold run 3: 5021ms | installed: 26MB | top-level pkg jsons: 20
RESULT spec=.bundle-stage/sanity-cli-7.12.1-bundle-experiment.0.tgz cold_median_ms=5272 installed_mb=26

Details

Most dependencies are compiled into the published package at build time rather than installed individually by npm. The larger development tooling (vite, tsx, jsdom; all used by commands like dev and build) either ships as regular dependencies ("safe" output) or is installed once on first use ("aggressive" output).

The typegen/functions/blueprints command groups similarly install their implementation on first use, while their help text ships with the CLI. Commands that don't use this tooling (login, init, dataset and document commands, --help) don't load or install it.

Source changes

• the env-file loader no longer imports vite on each command invocation
• worker threads can inherit a module-resolution hook when one is active (a no-op otherwise)
• jsdom is resolved by Node rather than going through the vite transform during studio config reads.

Open questions

• is this maintainable / will there be friction when adding new packages?
• should we try to ship an explicitly-tagged experimental build to get early signal

For reviewers

high level, there's a few key mechanisms worth auditing

• scripts/bundle-experiment/README.md: design notes, full results, issues found during validation, and known gaps
• packages/@sanity/cli/src/util/loadEnv.ts (+ tests): replaces vite's loadEnv with a dotenv-based equivalent
• packages/@sanity/cli-core: worker spawn and studio-worker config changes
• scripts/bundle-experiment/: the publish pipeline and validation harnesses

Testing

• Unit suite (3,289 tests), type check, lint, and dependency check pass
• Each module in the installed bundle (1,449 files, both outputs) was imported successfully, including worker files loaded inside worker threads
• All 109 commands were compared against an npm-installed build of the current CLI: --help output matched, and executing each command to its natural stopping point (output, usage error, or auth requirement), in and outside a studio project, showed no differences in the runs performed
• schema extract output was byte-identical; build produced the same artifact list; dev served locally
• Ran on node 22.12 (the supported minimum) via a compatibility fallback, and on node 26
• A live end-to-end run against a real project (login, init, schema deploy, deploy to sanity.studio, cleanup) passed for both outputs
• Known gaps (interactive prompts, Linux, credentialed e2e in CI) are listed in a follow-up comment with possible next steps

Notes for release

N/A

[github-actions]

📦 Bundle Stats — @sanity/cli

Compared against main (efb5705b)

@sanity/cli

Metric	Value	vs main (efb5705)
Internal (raw)	4.7 KB	+2.5 KB, +112.8%
Internal (gzip)	1.7 KB	+946 B, +112.9%
Bundled (raw)	10.11 MB	-1.09 MB, -9.7%
Bundled (gzip)	1.85 MB	-268.8 KB, -12.4%
Import time	818ms	-57ms, -6.6%

bin:sanity

Metric	Value	vs main (efb5705)
Internal (raw)	782 B	-
Internal (gzip)	423 B	-
Bundled (raw)	9.90 MB	-
Bundled (gzip)	1.78 MB	-
Import time	2.19s	+94ms, +4.5%

🗺️ View treemap · Artifacts

Details

• Import time regressions over 10% are flagged with ⚠️
• Sizes shown as raw / gzip 🗜️. Internal bytes = own code only. Total bytes = with all dependencies. Import time = Node.js cold-start median.

📦 Bundle Stats — @sanity/cli-core

Compared against main (efb5705b)

Metric	Value	vs main (efb5705)
Internal (raw)	115.4 KB	+1.3 KB, +1.2%
Internal (gzip)	29.6 KB	+452 B, +1.5%
Bundled (raw)	21.76 MB	+1.1 KB, +0.0%
Bundled (gzip)	3.46 MB	+344 B, +0.0%
Import time	802ms	+15ms, +1.9%

🗺️ View treemap · Artifacts

Details

• Import time regressions over 10% are flagged with ⚠️
• Sizes shown as raw / gzip 🗜️. Internal bytes = own code only. Total bytes = with all dependencies. Import time = Node.js cold-start median.

📦 Bundle Stats — @sanity/cli-build

Compared against main (efb5705b)

@sanity/cli-build/_internal/build

Metric	Value	vs main (efb5705)
Internal (raw)	113.2 KB	-
Internal (gzip)	28.7 KB	-
Bundled (raw)	17.75 MB	+1.1 KB, +0.0%
Bundled (gzip)	3.56 MB	+411 B, +0.0%
Import time	1.16s	+8ms, +0.7%

@sanity/cli-build/_internal/env

Metric	Value	vs main (efb5705)
Internal (raw)	1.8 KB	-
Internal (gzip)	644 B	-
Bundled (raw)	1.31 MB	-
Bundled (gzip)	333.8 KB	-
Import time	128ms	+1ms, +0.7%

@sanity/cli-build/_internal/extract

Metric	Value	vs main (efb5705)
Internal (raw)	8.6 KB	-
Internal (gzip)	2.7 KB	-
Bundled (raw)	156.1 KB	+1.1 KB, +0.7%
Bundled (gzip)	39.9 KB	+429 B, +1.1%
Import time	253ms	+2ms, +0.8%

🗺️ ./_internal/env · ./_internal/extract · @sanity/cli-build:./_internal/build treemap too large to embed · Artifacts

Details

• Import time regressions over 10% are flagged with ⚠️
• Sizes shown as raw / gzip 🗜️. Internal bytes = own code only. Total bytes = with all dependencies. Import time = Node.js cold-start median.

📦 Bundle Stats — create-sanity

Compared against main (efb5705b)

Metric	Value	vs main (efb5705)
Internal (raw)	908 B	-
Internal (gzip)	483 B	-
Bundled (raw)	931 B	-
Bundled (gzip)	491 B	-
Import time	❌ ChildProcess denied: node	-

Details

• Import time regressions over 10% are flagged with ⚠️
• Sizes shown as raw / gzip 🗜️. Internal bytes = own code only. Total bytes = with all dependencies. Import time = Node.js cold-start median.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	npm/​@​sanity/​runtime-cli@​17.1.0 ⏵ 17.2.0	-9		+1	+2

View full report

[github-actions]

Coverage Delta

File	Statements
packages/@sanity/cli-core/src/loaders/studio/studioWorkerTask.ts	100.0% (±0%)
packages/@sanity/cli-core/src/util/jitToolchainExecArgv.ts	80.0% (new)
packages/@sanity/cli-core/src/util/promisifyWorker.ts	100.0% (±0%)
packages/@sanity/cli/src/actions/init/bootstrapLocalTemplate.ts	1.6% (- 0.0%)
packages/@sanity/cli/src/commands/codemod.ts	98.0% (- 2.0%)
packages/@sanity/cli/src/util/loadEnv.ts	100.0% (new)

Comparing 6 changed files against main @ efb5705b0ccde1108a2503f921a9cab4f2aa850e

Overall Coverage

Metric	Coverage
Statements	78.5% (+ 0.0%)
Branches	70.1% (- 0.0%)
Functions	73.7% (+ 0.0%)
Lines	78.9% (+ 0.0%)

[shapirodaniel]

Live UAT report (personal org, fully automated)

Ran a credentialed end-to-end UAT against a throwaway project in a personal org (oztYsFirH), using browser-assisted sanity login --provider google into an isolated sandbox HOME. Everything below ran through the bundled artifacts (safe variant primary, aggressive spot-checked), then was torn down.

Green end-to-end (safe variant): login (SSO + Google provider flows) → init -y creating a real project + dataset + full template scaffold with npm install → documents create/query/delete → datasets create/list/delete → schema deploy (1/1) + schema list + schema extract → manifest extract → typegen generate (through the JIT stub, prettier formatting included) → build → dev (served + curled) → deploy (live at a real *.sanity.studio URL, verified with HTTP) → undeploy → project deleted via the new sanity api command → logout.

Aggressive variant spot-checks against the same real project: schema extract, documents query (returned live document count), typegen generate — all through the JIT toolchain machinery.

One real bug found and fixed (f0713ed): sanity init failed to bootstrap templates — bootstrapLocalTemplate (and codemod) located the package root via fixed-depth import.meta.dirname + '../../..', which points one level too high when the module is hoisted into a shared chunk. This was invisible to the exhaustive link-check (no import involved) and to execution parity (init's project creation needs credentials, so the harness only ever reached the auth boundary). Fixed with package-directory walk-up; bug list in the README updated (10).

Residual UAT gaps: interactive prompt flows (PTY), Linux, and CI e2e with the org's e2e credentials — the org-level SAML account cannot create projects (sanity.organization.projects/attach grant), so team-side UAT needs either personal orgs (as here) or the e2e project token.

[shapirodaniel]

Remaining validation gaps and suggested next steps

Context for what's left untested and some options for closing each item.

Interactive prompts

Prompt flows (sanity init menus and inputs) only render when a real terminal is attached. Automated checks here ran piped, so those code paths didn't execute; the e2e suite emulates terminals via node-pty, which couldn't allocate one in the environment this was built in (the same failure occurs with the current published CLI, so it looks environmental rather than a regression).

Risk seems low — the prompt library is included in the bundle and passes the module link-check — but nobody has confirmed a rendered prompt from the bundled build yet. Closing it: run npx -p <tarball> sanity init in a normal terminal once, and/or let the terminal-based e2e tests run in CI.

Linux

All validation so far ran on macOS/ARM. The bundle relies on npm selecting per-platform native binaries at install time, which has only been observed selecting the macOS ones — and the agent evals that motivated this work run on Linux. Closing it: install the tarball and run a few commands (--help, init, schema extract) in a node:22 container or on any Linux CI runner.

e2e suite with credentials

The cli-e2e suite is the scripted equivalent of the manual UAT above, and needs the existing CI secrets (SANITY_E2E_TOKEN etc.) that aren't available locally. It already supports targeting any binary via E2E_BINARY_PATH, so no test changes are needed — just a CI job that points it at the bundled artifact. This is also the piece that keeps future changes honest, since it would catch the next bundle-breaking change automatically.

One possible consolidation

A single CI job on a Linux runner could cover most of the above: build the tarballs (both variants) → npm-install them → run validate/link-check.mjs → run the e2e suite with E2E_BINARY_PATH set → optionally check bench-cold-start.sh against a budget. The only step that stays manual is a one-time interactive init from the tarball.

Open questions

• Which variant to pursue: safe (91MB, minimal runtime machinery) vs. aggressive (26MB, adds a resolution hook and install-on-demand that would need an integrity/consent story).
• The sanity package-name handover: npx sanity@latest currently resolves to the studio package, so the eval scenario isn't affected until the name or bin points at this CLI.
• Install-on-demand policy for the aggressive variant: version pinning/integrity, offline behavior, whether to prompt.

[shapirodaniel]

Linux e2e results (full suite, both variants)

Follow-up on the "Linux" and "e2e suite with credentials" gaps from the previous comment: the full cli-e2e suite has now run on Linux against both bundled outputs, alongside an unbundled control, using credentials for a throwaway project in a personal org (same setup as the UAT above). Environment: node:22 Linux containers (arm64) via apple/container on a macOS host — one container per run, repo and dependencies baked into a shared image, each variant tarball npm-installed inside its container and pointed at via E2E_BINARY_PATH (mirroring the scheduled registry-mode workflow, including create-sanity@latest).

run	passed	failed	skipped	duration
control (unbundled)	63	8	8	434s
safe	59	8	12	442s
aggressive	59	8	12	440s

No new failures in either variant. The 8 failures are the same 8 tests in all three runs — all in init.nextjs.test.ts, all one root cause: pnpm 10 inside the scaffolded Next.js app refuses to run dependency build scripts without pnpm approve-builds, and the flow exits nonzero. The unbundled control fails identically, and the same tests pass on GitHub-hosted runners, so this reads as a property of the container environment rather than of this PR — though if someone knows why hosted ubuntu behaves differently here, I'd like to understand it.

The count differences are registry-mode bookkeeping, not lost coverage: registry mode skips 6 tests (3 deploy --dry-run, 3 workbench) and un-skips the 2 create sanity@latest package-manager tests (both passed against the variants). Since the skipped 6 exercise the bundled workbench-cli and deploy paths, they were re-run against both variants with registry mode off: 6/6 passed. Net: every test the control executed either passed or failed identically for both variants.

A few things this closes beyond the headline:

• Interactive prompts, now confirmed. The PTY-based tests (node-pty) ran here — the earlier validation environment couldn't allocate terminals. Interactive init flows, prompt navigation, and Ctrl+C handling all passed against the bundled binaries.
• Per-platform native binary selection. npm selected the linux-arm64 binaries for the bundled installs (7.12.1-bundle-experiment.0 linux-arm64 node-v22.23.1), which had only ever been observed picking macOS ones.
• Suite duration is within ~2% of control, and no worker crashes, SIGABRT exits, or module-resolution errors appear in any log.

Caveats and what's still open:

• This is linux-arm64 and node 22 only. linux-x64 and the node 24/26 rows of the CI matrix haven't been run against the bundled outputs; no specific risk is known there, but it's untested.
• These were one-off local containers. The repeatable version of this is the CI job sketched in the previous comment (build tarballs → link-check → e2e with E2E_BINARY_PATH → cold-start budget), which remains the right next step if this PR moves forward.
• The product questions (variant choice, sanity name handover, install-on-demand policy) are unchanged by these results.

Rough repro, for anyone who wants to run it elsewhere: build the tarball (scripts/bundle-experiment/README.md), then in a node:22 container with the repo present and built, npm install --prefix /tmp/cli-bin <tarball> create-sanity@latest, export E2E_BINARY_PATH=/tmp/cli-bin/node_modules/.bin/sanity, E2E_CREATE_SANITY_BINARY_PATH, E2E_REGISTRY_MODE=true, and the SANITY_E2E_* credentials, and run vitest in packages/@sanity/cli-e2e. One repo-agnostic note: turbo fails with "I/O error: Is a directory" if the container's repo copy has no .git (e.g. from git archive) — git init && git commit first.

[shapirodaniel]

Matrix extension: node 26 and linux-x64

Follow-up to the Linux e2e comment above, closing the two cells it flagged as untested. Same setup (one container per run, control + both variants, E2E_BINARY_PATH, personal-org credentials), two new dimensions: a node:26 arm64 image, and node:22 amd64 containers running under Rosetta translation.

cell	control (unbundled)	safe	aggressive
linux-arm64 / node 22 (from previous comment)	63 passed / 8 failed	59 / 8	59 / 8
linux-arm64 / node 26	71 passed / 0 failed	67 / 0	67 / 0
linux-x64 / node 22, Rosetta	62 passed / 9 failed	58 / 9	58 / 9

(Pass-count differences between control and variants are the registry-mode skip bookkeeping explained in the previous comment; the registry-skipped workbench/deploy tests were separately verified green against both variants.)

What the numbers say:

• Node 26 is fully green in all three runs. The init.nextjs pnpm approve-builds failures from the node 22 cell don't occur at all on node 26 — which supports reading them as a property of the node 22 container image rather than of this branch.
• linux-x64 behaves. npm selected the linux-x64 native binaries for the bundled installs (7.12.1-bundle-experiment.0 linux-x64 node-v22.23.1), and the failure set is identical between control and both variants: the same 8 nextjs failures, plus one interactive-init test hitting its 90s PTY exit timeout. Everything runs ~2.4× slower under Rosetta and the unbundled control times out identically, so that one reads as emulation speed, not CLI behavior.
• Across all nine runs in the matrix there is still no test that fails with a bundled variant and passes with the control.

Not covered: node 24 (bracketed by the 22 floor and 26 head — a CI job would cover it as a matrix row), and real x64 hardware (Rosetta is a translation layer; it exercised binary selection and full execution, but a native x64 CI runner is the definitive version). Both fold into the CI job sketched earlier, which is the remaining piece that makes any of this repeatable.