Skip to content

fix(build): resolve SDK runtime + host WIT for registry installs (0.1.1)#11

Open
joshuajbouw wants to merge 2 commits into
mainfrom
fix/build-resolve-sdk-via-require
Open

fix(build): resolve SDK runtime + host WIT for registry installs (0.1.1)#11
joshuajbouw wants to merge 2 commits into
mainfrom
fix/build-resolve-sdk-via-require

Conversation

@joshuajbouw

Copy link
Copy Markdown
Contributor

Summary

@unicity-astrid/[email protected] only worked in workspace (file:-link) mode.
Registry installs (npm install @unicity-astrid/build) failed two ways
that this PR fixes, bundled in @unicity-astrid/[email protected].

Bug 1 — SDK runtime resolution

SDK_PKG_DIR was hardcoded as resolve(HERE, "..", "..", "astrid-sdk").
In workspace mode, HERE resolved through the symlink to the real
sdk-js/packages/astrid-build/src/ so ../../astrid-sdk correctly landed
at sdk-js/packages/astrid-sdk. Under npm install, HERE is
node_modules/@unicity-astrid/build/src/ and ../../astrid-sdk becomes
node_modules/@unicity-astrid/astrid-sdk — wrong scope, doesn't exist.

Fix: import.meta.resolve('@unicity-astrid/sdk/runtime'), which uses the
ESM resolver and honours the SDK's exports.import condition.

createRequire(...).resolve(...) was tried first but uses the CJS
resolver, which only sees exports.require — our exports map is
import-only.

Bug 2 — canonical host WIT shipping

CANONICAL_WIT_DIR was resolve(REPO_ROOT, "contracts", "host")
unavailable on consumer machines (the unicity-astrid/wit submodule
isn't part of the npm tarball). Failed with canonical host WIT missing or pre-split at /…/node_modules/contracts/host.

The original comment at the resolver claimed "the JS SDK has no such
constraint" (vs cargo package's tarball-must-be-self-contained rule) —
that was wrong; npm has the same constraint.

Fix: committed copies of the canonical per-domain host WIT files to
packages/astrid-build/wit-staging/host/ (analogous to what
astrid-sys does for the Rust SDK). The resolver prefers the workspace
submodule when present (so live developer edits land without sync) and
falls back to the staged copy otherwise. Added scripts/sync-build-wit.sh
to keep the staged copy in sync with the canonical source.

Test plan

  • Local sphere-capsule componentize end-to-end via packed tarball
    install (npm install <tgz> then npx astrid-js-build .) —
    previously failed at SDK runtime, then at host WIT, now produces
    a valid .wasm (~47MB, 154 host imports)
  • scripts/sync-build-wit.sh --check confirms wit-staging matches
    the canonical submodule
  • Post-merge: publish @unicity-astrid/[email protected] to npm.
    sphere-capsule and any other downstream consumer references
    ^0.1.1; the published ^0.1.0 is currently broken on registry
    install.

`@unicity-astrid/[email protected]` only worked when installed via `file:` link
in a sibling-package workspace layout. Two bugs surfaced when consuming
from the npm registry:

1. SDK runtime resolution. The build tool hard-coded `../../astrid-sdk`
   relative to its own source. In workspace mode the symlink follow
   resolved that to `sdk-js/packages/astrid-sdk`; from a npm install
   it pointed at the non-existent
   `node_modules/@unicity-astrid/astrid-sdk`. Switched to
   `import.meta.resolve('@unicity-astrid/sdk/runtime')`, which uses the
   ESM resolver and honors the SDK's `exports.import` condition.

2. Canonical host WIT shipping. The build tool read host WIT files from
   `<repo-root>/contracts/host` — only present when working from a
   checkout of unicity-astrid/sdk-js with the wit submodule initialised.
   The tarball had no copy, so registry installs failed with 'canonical
   host WIT missing'. Committed a copy of the per-domain host WIT files
   to `packages/astrid-build/wit-staging/host/` and added
   `scripts/sync-build-wit.sh` to keep them in sync with the canonical
   `contracts/host` submodule. The resolver prefers the workspace
   submodule when present (so developer edits are picked up live) and
   falls back to the staged copy otherwise.

Bump to 0.1.1.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates @unicity-astrid/build to version 0.1.1, introducing fallback resolution paths for WIT files and the SDK runtime to support both workspace builds and published-package installations. It includes committed copies of the host WIT files in a new wit-staging directory and adds a synchronization script sync-build-wit.sh to keep them in sync with the canonical submodule. A review comment identifies a potential failure in the synchronization script when globbing .wit files in an empty directory and suggests using shopt -s nullglob to handle empty matches gracefully.

Comment thread scripts/sync-build-wit.sh Outdated
Comment on lines +41 to +66

if [[ "${1:-}" == "--check" ]]; then
tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT
cp "$SRC_DIR"/*.wit "$tmp/"
if ! diff -rq "$tmp" "$DST_DIR" >/dev/null 2>&1; then
echo "sync-build-wit: $DST_DIR is out of sync with $SRC_DIR" >&2
echo "sync-build-wit: run scripts/sync-build-wit.sh to fix" >&2
diff -r "$tmp" "$DST_DIR" >&2 || true
exit 1
fi
echo "sync-build-wit: in sync"
exit 0
fi

mkdir -p "$DST_DIR"
# Sync exactly: copy all .wit, then remove any stale files in DST that
# the canonical source no longer has.
cp "$SRC_DIR"/*.wit "$DST_DIR/"
for f in "$DST_DIR"/*.wit; do
base=$(basename "$f")
if [[ ! -f "$SRC_DIR/$base" ]]; then
rm "$f"
fi
done
echo "sync-build-wit: $DST_DIR ← $SRC_DIR/*.wit ($(ls "$SRC_DIR" | grep -c '\.wit$' | tr -d ' ') files)"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If the contracts/host directory exists but is empty (e.g., when the git submodule is not initialized), the cp command will fail with a generic error because the glob *.wit does not expand. Additionally, if DST_DIR is empty, the for f in "$DST_DIR"/*.wit loop will execute once with the literal string *.wit and attempt to delete it, causing a script failure under set -e.

Using shopt -s nullglob and checking the array length of matched files provides a robust and clean solution with a helpful error message.

shopt -s nullglob
files=("$SRC_DIR"/*.wit)
if [[ ${#files[@]} -eq 0 ]]; then
  echo "sync-build-wit: no .wit files found in $SRC_DIR" >&2
  echo "sync-build-wit: did you forget 'git submodule update --init --recursive'?" >&2
  exit 1
fi

if [[ "${1:-}" == "--check" ]]; then
  tmp=$(mktemp -d)
  trap 'rm -rf "$tmp"' EXIT
  cp "${files[@]}" "$tmp/"
  if ! diff -rq "$tmp" "$DST_DIR" >/dev/null 2>&1; then
    echo "sync-build-wit: $DST_DIR is out of sync with $SRC_DIR" >&2
    echo "sync-build-wit: run scripts/sync-build-wit.sh to fix" >&2
    diff -r "$tmp" "$DST_DIR" >&2 || true
    exit 1
  fi
  echo "sync-build-wit: in sync"
  exit 0
fi

mkdir -p "$DST_DIR"
# Sync exactly: copy all .wit, then remove any stale files in DST that
# the canonical source no longer has.
cp "${files[@]}" "$DST_DIR/"
for f in "$DST_DIR"/*.wit; do
  base=$(basename "$f")
  if [[ ! -f "$SRC_DIR/$base" ]]; then
    rm "$f"
  fi
done
echo "sync-build-wit: $DST_DIR$SRC_DIR/*.wit (${#files[@]} files)"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed. The script guarded SRC_DIR existence but not whether it actually contained .wit files, so an initialised-but-empty submodule dir would feed a literal *.wit to cp (cryptic failure), and the freshly-mkdir'd empty DST would make the staleness loop run once on literal *.wit and rm it under set -e. Added shopt -s nullglob after set -euo pipefail (the shebang is already #!/usr/bin/env bash), then build src_files=("$SRC_DIR"/*.wit) once after the existing dir guard and bail with the same git submodule update --init hint when the array is empty. Both the --check and sync cp now use "${src_files[@]}", and with nullglob the for f in "$DST_DIR"/*.wit staleness loop simply iterates zero times on an empty DST. Verified: --check still reports "in sync", and an empty source dir now exits 1 with the helpful message.

…ource dir

The existing guard only checked that SRC_DIR exists, not that it holds any
.wit files. With an initialised-but-empty submodule dir, 'cp "$SRC_DIR"/*.wit'
passed a literal '*.wit' to cp (cryptic failure), and the freshly-mkdir'd empty
DST made the staleness loop iterate once on literal '*.wit' and rm it under
set -e.

Enable 'shopt -s nullglob', collect the source files into a validated array
once (with the same 'git submodule update --init' hint when empty), and use
that array on both the --check and sync cp paths. Verified: --check still
reports 'in sync', and an empty source dir now exits 1 with the helpful
message.
joshuajbouw added a commit that referenced this pull request Jun 7, 2026
## Summary

Mirrors the Rust SDK and the host `astrid:[email protected]`
**persistent-process tier** (host: astrid-runtime/astrid#867; Rust SDK:
astrid-runtime/sdk-rust#53; contract: astrid-runtime/wit#12), so JS/TS
capsule authors can spawn a background child that **outlives the pooled,
stateless instance** that started it — unlike
`process.BackgroundProcessHandle`, whose kernel resource is reaped on
instance reset.

## Changes

- **`contracts` submodule → the merged `astrid:[email protected]` persistent
tier** (wit commit `1a06cf4`, the #12 squash). Deliberately the
**#12-only** commit (predates #11), so the `astrid:contracts` events
bundle (`astrid-contracts.wit` / generated `contracts.ts`) is untouched
(`sync-contracts-wit.sh --check` stays green) and unrelated open SDK PRs
are unaffected. On `main`, `astrid-build` reads the host WIT **live from
`contracts/host/`** (`CANONICAL_WIT_DIR`), so the bump is all
ComponentizeJS needs.
- **`wit-imports.d.ts`** (hand-written host ABI types): persistent types
+ free-function declarations on `astrid:process/[email protected]` —
`spawnPersistent`, `attach`, `listProcesses`, `status`, `statusMany`,
`readLogs`, `readSince`, `writeStdin`, `closeStdin`, `signal`, `wait`,
`stop`, `releaseProcess`, `watch`, `unwatch`; `ProcessInfo`,
`ProcessPhase`, `LogStream`, `LogCursor`, `LogChunk`, `OverflowPolicy`,
`ResourceLimits`; `ErrorCode` gains `no-such-process` / `registry-full`
/ `persist-unsupported`; `ProcessSignal` gains `stop` / `cont`;
`SpawnRequest` gains the 8 persistent fields.
- **`process.ts`** ergonomic layer: `spawnPersistent(cmd, args,
options)` + the persistent `SpawnPersistentOptions` knobs;
`PersistentProcess` class (`status` / `readLogs` (drain) / `readSince`
(non-draining cursor → byte-faithful `LogChunkResult`;
`logCursorStart()`) / `writeStdin` / `closeStdin` / `signal` / `wait`
(bounded) / `stop` (consumes — SIGTERM→grace→SIGKILL) / `release`);
module fns `attach(id)` (id-wrapper reattach — works without the host's
deferred `attach` resource fn), `listProcesses`, `statusMany`; ergonomic
types. `buildSpawnRequest` updated for the new `SpawnRequest` fields
(required).
- **`index.ts`**: top-level re-exports for the new public types.

## Not exposed (host-deferred)

`watch` / `unwatch` (open publish-authority RFC question — poll via
`status` + bounded `wait`) and resource-limit enforcement (`limits` is
plumbed through but the host doesn't enforce it yet).

## Test Plan

- [x] `npm run build` (tsc) green across the workspace
- [x] **End-to-end:** the `examples/test-capsule` **componentizes**
against the persistent WIT — ComponentizeJS produced a 12.96 MB WASM
with **169 host imports**, proving the new host functions wire through
the JS build path
- [x] `scripts/sync-contracts-wit.sh --check` passes (events bundle
untouched)
- [x] Git diff scoped to 5 files (contracts pointer + 3 SDK sources +
CHANGELOG); no `contracts.ts` drift
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant