feat!: split astrid:[email protected] into per-domain frozen packages#7
Merged
Conversation
Replaces the monolithic astrid:[email protected] with one frozen WIT file per domain at host/<name>@<version>.wit. The wasmtime Component Model linker enforces structural typing on every (package, version) pair, so the single mega-package meant any record-field add or function-add broke every capsule built against the prior shape. Per-domain packages contain the blast radius: bumping astrid:net no longer recompiles capsules that only import astrid:kv. Packages introduced: astrid:[email protected] host/[email protected] astrid:[email protected] host/[email protected] astrid:[email protected] host/[email protected] astrid:[email protected] host/[email protected] astrid:[email protected] host/[email protected] astrid:[email protected] host/[email protected] astrid:[email protected] host/[email protected] astrid:[email protected] host/[email protected] astrid:[email protected] host/[email protected] astrid:[email protected] host/[email protected] astrid:[email protected] host/[email protected] astrid:[email protected] host/[email protected] (lifecycle exports) The RFC originally proposed a minimal astrid:core for cross-cutting types, but auditing the actual types revealed nothing genuinely shared — every type in the old monolithic 'types' interface is used by exactly one domain. Zero-shared is purer and cheaper to maintain, so astrid:core is not introduced. Each per-domain package owns its own types in a single 'host' interface alongside its functions. The guest export contract (astrid-hook-trigger, run, astrid-install, astrid-upgrade) lives in astrid:guest rather than astrid:hook to avoid collision with the existing capsule-to-capsule astrid:hook in interfaces/hook.wit. The naming is symmetric with the per-domain 'host' interfaces: 'host' is kernel-side (imported by capsules), 'guest' is capsule-side (called by the kernel). Dropped without replacement (dead in the host WIT, duplicated by capsule-to-capsule interfaces/tool.wit): - tool-input, tool-output, tool-definition - capsule-context Hard cut: host/astrid-capsule.wit deleted. Pre-1.0 we have license to break the ecosystem rebuild; post-1.0 we do not, so the new model ships clean rather than carrying a deprecated alias. Evolution discipline (frozen-file rule) enforced by CI: scripts/lint-wit-immutability.sh — fails any PR that modifies or deletes a published *@X.Y.Z.wit file. New shape = new file at a new version path. .github/workflows/lint.yml — runs the immutability lint and parses every WIT file with wasm-tools on every PR / push to main. README rewritten with the per-domain layout, the evolution rule, and the worked example of bumping a package version. See RFC #22 (host_abi) and issue astrid-runtime/astrid#750 for the full design rationale.
Replace the single `exports` world (all 4 entry points mandatory) with four worlds — `interceptor`, `background`, `installable`, `upgradable` — that capsules `include` independently based on what they implement. Motivation: the wasm32-wasip2 toolchain auto-stubs every export declared in the world a component targets. With a single mandatory world the toolchain emitted stub functions for the entry points a given capsule did not implement, and the kernel had to parse the wasm binary to detect them (see `core/crates/astrid-capsule/src/engine/wasm/mod.rs` `STUB_PRONE_EXPORTS` — a stub-detection hack that compares export function indices to find aliased nops). Per-export worlds put the declaration with the implementation: an export only appears in the wasm binary when the capsule actually implements it, so the kernel can drop the parsing hack and use plain export-presence checks. Verified with `wasm-tools component wit` on a synthetic consumer world that includes interceptor + background — resolves to exactly those two exports plus the auto-imported lifecycle types. Parity audit before this change confirmed the split file matches the deleted monolithic file: 65/65 host functions identical, 30/30 referenced types preserved, 4 dead WIT records dropped (capsule-context, tool-input, tool-output, tool-definition — verified zero references anywhere in WIT; the kernel uses JSON-serialized Rust types over list<u8> for the hook context, not the WIT record).
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request modularizes the Astrid host-to-capsule contract by replacing the monolithic astrid-capsule.wit with per-domain versioned packages and establishing an evolution discipline where published WIT files remain immutable. A new linting script is introduced to enforce this policy. Feedback was provided to improve the linting script's robustness when handling file paths that contain spaces.
joshuajbouw
added a commit
that referenced
this pull request
May 21, 2026
- Address Gemini review on PR #7: switch the lint awk to tab-field parsing (`-F'\t'`) so paths containing spaces are handled correctly. Same behaviour for the common case; the previous default-whitespace splitting would have broken on a path with embedded spaces. - Scope the workflow parse step to host/*.wit. Each host package is self-contained — every type lives inside the same package's host interface, no cross-package `use` — so single-file `wasm-tools component wit` resolves cleanly. interfaces/*.wit files cross-reference each other (`use astrid:types/types.{message}` etc.) and would require a deps/ tree to resolve in single-file mode; they predate this CI and are validated by downstream SDK builds, so the workflow no longer pretends to validate them.
- Address Gemini review on PR #7: switch the lint awk to tab-field parsing (`-F'\t'`) so paths containing spaces are handled correctly. Same behaviour for the common case; the previous default-whitespace splitting would have broken on a path with embedded spaces. - Scope the workflow parse step to host/*.wit. Each host package is self-contained — every type lives inside the same package's host interface, no cross-package `use` — so single-file `wasm-tools component wit` resolves cleanly. interfaces/*.wit files cross-reference each other (`use astrid:types/types.{message}` etc.) and would require a deps/ tree to resolve in single-file mode; they predate this CI and are validated by downstream SDK builds, so the workflow no longer pretends to validate them.
joshuajbouw
force-pushed
the
feat/per-domain-split
branch
from
b28afe1 to
f521cf2
Compare
The kernel adds wasmtime_wasi::p2 to the linker, so capsule authors see wasi:random, wasi:clocks/monotonic-clock, wasi:clocks/wall-clock, and wasi:io/streams alongside astrid:*. astrid:* exists where Astrid layers capability gating, principal scoping, or audit on top of what would otherwise be a syscall; where no such layering is needed, capsules should use WASI directly rather than expecting Astrid to re-expose it.
…lock-monotonic-ns, sleep-ns WIT changes on astrid:[email protected] (still pre-merge — freeze rule doesn't apply yet to files on this branch). Removed: - trigger-hook. Synchronous hook fan-out + response aggregation is orchestration, not routing. Capsules implement the equivalent on the IPC bus (publish + subscribe scoped reply topic + collect-N-responses- with-timeout) via an SDK helper that's coming alongside the kernel- side removal. Two callers (capsule-prompt-builder, capsule-hook-bridge) migrate in their own commits. Added: - random-bytes(length) — cryptographically secure entropy from the host CSPRNG. Length capped at 4096 per call. - clock-monotonic-ns() — monotonic clock reading in nanoseconds. Replaces the WASI route for elapsed-time measurement. - sleep-ns(duration) — block the calling guest task for the given duration. Capped server-side at 60s per call. These exist because Astrid no longer exposes wasi:random or wasi:clocks to capsules. Every host call goes through the astrid:* surface so it's capability-gated, principal-scoped, and audited; running a second un-audited host surface (WASI) alongside ours defeats that. WASI's generic primitives now have explicit astrid:sys equivalents. README: - Drop the 'WASI is also available' section. It was premature without an audit of the WASI surface and conflicts with the new policy of astrid:* being the single host ABI. - Update astrid:sys row to reflect the new function list.
…, kv-cas Filesystem (astrid:fs): - fs-append for non-RMW log/journal writes - fs-copy + fs-rename for file management; rename is single-VFS-scheme only to preserve atomic-rename contracts - fs-remove-dir-all for recursive directory removal; refuses to follow symlinks to prevent VFS-sandbox escapes Networking (astrid:net): - UDP datagram set: udp-bind, udp-send-to, udp-recv-from, udp-close, plus udp-set-read-timeout and udp-local-addr accessors - New udp-datagram record carries (data, peer-host, peer-port) - Gated by a separate net_udp capability (distinct from net / net_connect) so capsule authors opt in deliberately - SSRF airlock applies on send-to / recv-from peer resolution - Unblocks P2P, DNS, QUIC, voice/video, and any UDP-protocol bridge capsule Process (astrid:process): - spawn-request gains stdin (optional pre-piped buffer), env (explicit env-var list replacing host's default sandbox env), cwd (workspace- relative working directory) - New signal(process-id, signal-name) for SIGTERM/SIGHUP/SIGUSR1/2/INT, separate from kill (SIGKILL) so graceful shutdown is expressible - Required for any capsule that wraps a real CLI tool (git, npm, kubectl) — without stdin/env/cwd the spawned process couldn't be driven properly KV (astrid:kv): - kv-cas(key, expected, new) for atomic compare-and-swap. expected: none means create-if-absent. Required for any concurrent coordination on shared state — the kernel runs capsule invocations on the multi-threaded tokio worker pool, so RMW patterns on shared keys race silently today Total host fn additions: +13 (fs +4, net +6, process +1, kv +1, plus spawn-request gains 3 record fields). Contract-first; kernel implementations land in the core branch. The new functions are part of the rounded 1.0 syscall surface so that when third-party capsule authors arrive they find what they expect based on std::* equivalents.
…-shaped corrections Last-mile pre-freeze polish driven by an exhaustive audit against std::*. Every change here is a 'one-way door' — once 1.0 lands, the shape is committed to forever, so this commit lands them all together. Std-shape corrections (mostly type fixes — small but irreversible): - file-stat extended: was just (size, is-dir, mtime). Now carries is-file, is-symlink, mode (POSIX bits), mtime-ns + ctime-ns + atime-ns. Every build/script/sync capsule needs 'is this a regular file', and after freeze no capsule can ever get it. - fs-stat-symlink added — lstat counterpart, doesn't follow symlinks. - http-request-data.body: option<string> → option<list<u8>>. The string shape silently lossy-converts arbitrary bytes; first PNG / protobuf / multipart upload would have hit it. - process-result.exit-code: s32-with-(-1)-sentinel → option<s32>. Matches read-logs-result.exit-code shape, drops the magic-value pattern. - process.signal(process-id, signal-name: string) → signal(process-id, sig: process-signal) with a typed enum (term/hup/usr1/usr2/int). The documented allowlist is now in the contract. - udp-recv-from: sentinel-on-no-data → option<udp-datagram>. Matches net-poll-accept idiom; sentinel pattern dropped. - get-config: empty-string-on-not-set → option<string>. Empty values are legal and now distinguishable from 'not set'. - net-bind-unix(listener-handle: u64) → net-bind-unix(). The arg was documented as ignored. - net-poll-accept gains a caller-controlled timeout (was hardcoded 10ms, no name signal). Matches the rest of the timeout-bearing fns. New host primitives that match std::* and capsule authors will reach for: - fs-open / fs-read-at / fs-write-at / fs-flush / fs-close — file handles with positional I/O (mirrors std::fs::File + FileExt::read_at/write_at). Escape valve for the 10 MB whole-file cap; mandatory for capsules processing large blobs (logs, DBs, parquet, media). - fs-canonicalize / fs-read-link — mirrors std::fs::canonicalize and std::fs::read_link. Both enforce VFS-scheme boundaries: resolutions escaping the input's scheme return an error rather than leaking host- filesystem information. Deliberately omitted: fs-symlink and fs-hard-link — symlink/hard-link creation is the highest-blast-radius sandbox-escape vector and the agent-runtime demand is near zero; if real demand emerges, they ship in astrid:[email protected] with a designed security model. - net-lookup-host — DNS resolution (std::net::ToSocketAddrs equivalent). SSRF airlock filters resolved IPs the same as connect-tcp. - process-write-stdin / process-close-stdin — streaming stdin to a long-lived background process. Required for REPL-style children (python -i, psql, MCP stdio bridges). - process-wait — std::process::Child::wait equivalent. timeout-ms: option<u64>; returns Some(exit_code) if exited within window, None if timeout elapsed. Avoids the IPC-burn of polling read-logs. - kv-list-keys-page — paginated listing. Prevents OOM on large per- principal stores. kv-list-keys retained for small-store convenience but capped server-side at 1024 keys. Deferred to 1.x (specific, not blocking): - TCP listener (net-bind-tcp). Team has discussed remote-hosted capsules going non-WASM; not shipping inbound listener with WASM 1.0. - Socket options (SO_KEEPALIVE, SO_LINGER, etc.). Defaults work for the typical agent capsule; add when a real consumer needs tuning. - UDP connect / multicast. UDP just shipped — no demand signal yet. - File locking (flock). kv-cas covers in-runtime concurrency; cross- process locking is niche for a sandboxed agent runtime. Total host fn count: 78 → 92 (+14 net additions across fs/net/process/kv/ sys). All function names mirror std::* where a direct equivalent exists. Contract-first; kernel implementations land in the core branch.
You're right that capsules use sockets — both Unix and TCP. Deferring keepalive and linger to 1.x was wrong: long-lived connections to message brokers, databases, IPC bridges (cli↔daemon, hook-bridge) need keepalive to detect silently-dead peers without holding the stream open indefinitely, and linger control matters for fast-shutdown and reconnect cycles. Added four setsockopt-shaped functions: - net-set-keepalive(handle, keepalive-secs: option<u64>) — enables TCP keepalive with the given probe interval; none disables. Errors on Unix-domain handles (keepalive is TCP-only). - net-keepalive(handle) — returns the current setting. - net-set-linger(handle, linger-ms: option<u64>) — none = default graceful FIN; some(0) = immediate RST drop unsent; some(t) = wait up to t ms for buffer drain. Mirrors std::net::TcpStream::set_linger. - net-linger(handle) — returns the current value. Not adding: SO_REUSEADDR (no TCP listener yet, Unix listener does unlink+rebind instead), SO_BROADCAST / multicast (deferred with UDP specifics until a use case lands), SO_RCVBUF / SO_SNDBUF (rare-need performance tuning, easy 1.x add). net fn count: 28 → 32.
The upcoming WIT freeze-prep refactor adds cross-package use clauses (notably `use wasi:io/[email protected].{pollable}` for the pollable foundation on net + ipc subscription handles). wasm-tools 1.x resolves WIT dependencies from a sibling deps/ directory, so we vendor [email protected] under deps/wasi-io/ — poll, streams, error, world. Locked at the official 0.2.0 release tag. New scripts/validate-wit.sh stages each host/*.wit alongside the vendored deps/ and sibling host/ files into a tempdir before parsing, so cross-package use clauses resolve. wasm-tools' directory parse treats a dir as a single package and can't handle our flat host/ tree where each file is its own package; staging per-file works around it. interfaces/*.wit files are not validated here — they cross-reference each other and wasm-tools 1.x's single-pass deps loader chokes on the alphabetical ordering (tries to load astrid:context before its astrid:types dep). Downstream SDK builds (cargo-component / wkg) do proper topological resolution and validate that side. CI workflow updated to call the script.
…espace Pre-freeze namespace split. host/*.wit (kernel ABI — direct wasmtime CM linker calls) keeps astrid:*. interfaces/*.wit (capsule-to-capsule schemas for events on the IPC bus) moves to astrid-bus:*. The two were colliding on astrid:[email protected] and astrid:[email protected] where the same name covered two distinct concerns (kernel-mediated approval gate vs capsule↔capsule approval-event schemas) — a capsule importing both would have hit a name conflict in its world. The rename resolves the collisions and codifies the layering for every future package. All 17 interfaces/*.wit files renamed: astrid:agent → astrid-bus:agent astrid:approval → astrid-bus:approval (was colliding with host/approval) astrid:client → astrid-bus:client astrid:context → astrid-bus:context astrid:elicit → astrid-bus:elicit (was colliding with host/elicit) astrid:hook → astrid-bus:hook astrid:llm → astrid-bus:llm astrid:onboarding → astrid-bus:onboarding astrid:prompt → astrid-bus:prompt astrid:registry → astrid-bus:registry astrid:session → astrid-bus:session astrid:spark → astrid-bus:spark astrid:system → astrid-bus:system astrid:tool → astrid-bus:tool astrid:types → astrid-bus:types astrid:user → astrid-bus:user astrid:users → astrid-bus:users Cross-references inside interfaces/ (e.g. `use astrid:types/types` in llm.wit, session.wit, prompt.wit, tool.wit, context.wit; `use astrid:registry/registry` in onboarding.wit; `use astrid:onboarding/ onboarding` in elicit.wit) updated to astrid-bus:* in the same pass. README rewritten to document the two-namespace model: astrid:* for kernel-mediated syscalls, astrid-bus:* for IPC-bus event schemas. Capsule.toml imports/exports now use both prefixes.
…es, pollable The first of two freeze-prep commits adopting the audit recommendations for the host ABI. Replaces stringly-typed errors, flat u64 handles, and per-call timeout-based readiness with typed alternatives that the Component Model ecosystem expects. astrid:[email protected]: - New `variant error-code { not-found, access, capability-denied, boundary-escape, invalid-path, would-block, is-directory, not-directory, not-empty, too-large, quota, cross-vfs, already-exists, closed, unknown(string) }` replaces every `result<_, string>` return. - New `resource file-handle` with methods read-at / write-at / sync-data / sync-all / stat / set-len. Replaces the flat-handle fs-read-at / fs-write-at / fs-flush / fs-close functions. Folds in Bucket C primitives (sync-all, set-len, fstat-on-handle) as resource methods so capsules get them automatically. Per-capsule cap: 16 open handles. Drop is automatic via the CM runtime — no manual close needed. - New `enum file-type { type-unknown, regular, directory, symlink, block-device, character-device, fifo, socket }` replaces the is-dir / is-file / is-symlink boolean trio in file-stat. Matches wasi-filesystem/types.descriptor-type. - New `record datetime { seconds: s64, nanoseconds: u32 }` for file timestamps; matches wasi-clocks/wall-clock.datetime. Fixes the year-2554 overflow and supports pre-1970 timestamps. file-stat now carries modified / created / accessed as option<datetime>. - Doc-as-contract additions: paths/error strings are guaranteed VFS-scheme only (no host real-paths leak); kernel re-resolves and re-validates every path on every call (so fs-canonicalize is for display/equality only, NOT a TOCTOU-safe security check); error strings never contain UUIDs/IPs/capability-names. astrid:[email protected]: - New `variant error-code` per the network domain (would-block, closed, capability-denied, airlock-rejected, connection-refused, connection-reset, timeout, address-in-use, address-not-available, name-unresolvable, invalid-handle, not-tcp, quota, unknown(string)). - Three new resources replacing flat u64 handles: - `resource unix-listener` (accept / poll-accept / subscribe- readiness) from bind-unix. - `resource tcp-stream` with all the read/write/peek/shutdown/ accessor methods and TCP options as methods. Used for both accepted Unix streams and outbound TCP. - `resource udp-socket` for UDP send-to / recv-from + options. - New `subscribe-readiness` / `subscribe-readable` methods on each resource return `wasi:io/[email protected]` so capsules can multiplex via `wasi:io/poll.poll`. This is the Bucket E pollable foundation: bridge capsules (cli, hook-bridge, future telegram/ discord) can compose 'wait on Unix socket OR IPC bus' natively without embedding an async runtime in the wasm binary purely for multiplexing. - shutdown-how arm names switched to receive / send / both to match wasi:sockets idiom. - TTL accessors renamed to hop-limit / set-hop-limit to match wasi:sockets (TTL is v4 naming; hop-limit is the standardized v4+v6 name). - `net-poll-accept` gains a caller-controlled timeout-ms arg as a resource method instead of the previous hardcoded 10 ms. - `udp-recv-from` returns `option<udp-datagram>` instead of the ('', 0)-sentinel pattern. 10 host/ files remain for the next Bucket A commit: ipc, kv, sys, process, http, approval, elicit, identity, uplink, guest. Same pattern.
Completes the freeze-prep refactor across all host/* WIT files. Applies the same patterns established in fs/net: per-package error-code variants replacing result<_, string>, resource types where there's a long-lived handle, and doc-as-contract additions for audit policy / charset rules / capability gates. astrid:[email protected]: - error-code variant: capability-denied, config-key-reserved, too-large, registry-unavailable, cancelled, unknown(string). - get-config returns result<option<string>, error-code> distinguishing 'not set' from empty-string values. - random-bytes length: u32 → u64 matching wasi:random. - Per-fn audit policy documented inline. astrid:[email protected]: - error-code: invalid-key, too-large, quota, cas-mismatch, unknown. - Charset rule documented (UTF-8 NFC, no NUL/control, max 256 bytes for keys; 1 MiB max for values). - Per-fn audit policy documented. astrid:[email protected]: - error-code: invalid-input, timeout, store-unavailable, unknown. - approval-decision enum surfaces the specific resolution (denied/approved/approved-session/approved-always/allowance) instead of the old approved: bool — capsule UI can communicate WHY for transparency. astrid:[email protected]: - error-code: not-in-lifecycle, timeout, cancelled, invalid-input, store-unavailable, unknown. - elicit-type as typed enum (text/secret/select/array) — was string. - elicit-response as variant — value(string) / values(list<string>) / secret-stored — instead of stringly-typed JSON. astrid:[email protected]: - error-code: capability-denied, invalid-input, user-not-found, link-not-found, already-linked, store-unavailable, unknown. - Split the god-record identity-ok-response into per-operation responses (identity-resolve-response, identity-create-user-response, platform-link list from identity-list-links). Replaces JSON-in-WIT with typed records. astrid:[email protected]: - error-code: capability-denied, invalid-input, invalid-profile, unknown-uplink, no-session, quota, unknown. - uplink-profile as typed enum (chat/interactive/notify/bridge) — was string. astrid:[email protected]: - error-code: capability-denied, invalid-request, dns-error, airlock-rejected, tls-error, timeout, connection-error, body-too-large, closed, quota, protocol(string), unknown. - http-method as typed variant matching wasi:http (get/head/post/put/ delete/connect/options/trace/patch/other(string)) — was string. - http-stream resource replaces the flat-handle stream-start/read/close trio: stream is constructed by http-stream-start; status / headers / read-chunk / close are methods. Drop is automatic. astrid:[email protected]: - error-code: capability-denied, invalid-input, boundary-escape, quota, too-large, closed, cancelled, wait-timeout, unknown. - exit-info record split exit-code: option<s32> + signal: option<s32> so SIGKILL-by-oom is distinguishable from normal-exit-with-code-1. process-result / read-logs-result / kill-result all carry exit-info. - process-handle resource replaces the flat-handle process-id passed around: read-logs / write-stdin / close-stdin / signal / kill / wait are methods on the resource. Drop is automatic. astrid:[email protected] (the most consequential): - error-code: capability-denied, invalid-input, closed, rate-limited, backpressure, quota, timeout, unknown. - principal-attribution variant on ipc-message: verified(string) / claimed(string) / system. The Bucket A3 trust marker — downstream consumers MUST check this variant on sensitive actions; claimed principals are uplink-asserted and NOT kernel-verified. publish-as produces 'claimed'; publish produces 'verified'. Closes the ipc-publish-as ambient-spoofing vector flagged in the security audit. The kernel audit log records BOTH the uplink's true principal AND the claimed principal for publish-as. - subscription resource with poll / recv / subscribe-readiness methods. subscribe-readiness returns wasi:io/poll.pollable so bridge capsules can multiplex bus + sockets via wasi:io/poll.poll without embedding tokio purely for that. - get-interceptor-bindings (renamed from get-interceptor-handles) explicitly scoped to the caller's own bindings — does NOT enumerate other capsules' interceptors. Closes the capability-inference leak. - Topic charset rule documented ( per segment, max 8 segments, max 256 bytes). Total host fn / method count is similar to before; what changed is the shape of the contract (typed errors, typed resources, typed enums) rather than the count.
… warning Final pre-freeze doc-as-contract additions surfaced by the security audit pass: ipc::publish — documents the re-entrancy contract (publishing from within an interceptor handler is allowed but counts against a host- side publish-depth budget of 8) and the fan-out cap (a single publish dispatches to at most 256 subscribers; further matches surface as backpressure / lag counters). These were left ambiguous in the prior version — without them in the WIT contract, a capsule author writing recursive interceptors or wide-fan-out could discover the implicit caps only by hitting them in production. net::udp-bind — strengthens the security warning around binding to non-loopback addresses. The previous note read as informational; the new wording makes explicit that 0.0.0.0 / :: is a server posture exposing the capsule to every network interface, and recommends the net_udp capability allowlist pattern for loopback-only binds. Both are doc-only changes — no shape impact on the WIT.
…t symlink creation) You're right that these are all things we agreed should land — pulling them in pre-freeze so the 1.0 surface is as complete as the audits recommended. The only deferred item kept is symlink creation, where the sandbox-escape risk outweighs the demand. astrid:[email protected]: - fs-hard-link(src, link-path) — creates a hard link. Same-VFS-scheme only; cross-scheme returns cross-vfs. Cannot link directories. The kernel validates both endpoints stay inside the VFS scope. - fs-mkdir / fs-mkdir-all split — fs-mkdir is now the strict POSIX mkdir(2) (errors if parent missing or target exists) for lock-dir / exclusive-scratch patterns. fs-mkdir-all is the old behaviour (idempotent create-with-parents). Existing fs-mkdir doc previously said 'and missing parents' — the rename + split makes the semantic honest. Symlink creation deliberately not added: capsules can create symlinks with paths pointing outside the workspace, and even kernel-side canonicalization at follow-time has TOCTOU windows symlink swap can exploit. The read-only fs-read-link / fs-stat- symlink pair handles existing symlinks. astrid:[email protected]: - New tcp-listener resource (accept / poll-accept / local-addr / subscribe-readiness) + bind-tcp factory function. Inbound TCP serving — self-hosted webhook receivers, gRPC endpoints, Prometheus scrape ports. Gated by a new net_tcp_bind capability allowlist distinct from net_connect. 0.0.0.0 / :: warning documented analogous to udp-bind. - tcp-stream gains set-reuseaddr / reuseaddr (SO_REUSEADDR) for listener restart cycles. - udp-socket adds connected mode: connect / disconnect / send / recv / peer-addr. SSRF airlock runs once at connect; kernel filters spoofed peer datagrams. Faster syscall path for QUIC, DNS-over-UDP, syslog-over-UDP. Unconnected send-to / recv-from remain. astrid:[email protected]: - process-handle gains os-pid() — surfaces the kernel-level PID for capsules correlating with cgroup paths, /proc entries, or children that log their own PID. - process-handle gains wait-with-output(timeout) — atomic wait + final stdout/stderr drain. Closes the read-logs race for short-lived children where output could be lost between wait observing exit and the next read-logs call. - process-handle gains subscribe-exit and subscribe-logs pollables. Multiplexes 'wait on child process' with IPC events, file I/O, etc. via wasi:io/poll.poll. astrid:[email protected]: - http-stream gains subscribe-readable pollable. Capsule streaming a large HTTP response can multiplex it with IPC requests / file writes / other I/O via wasi:io/poll. Deferred (intentional, kept on the 1.x candidate list): - Symlink creation (fs-symlink) — sandbox-escape risk too high. - File locking (fs-lock) — kv-cas covers in-runtime concurrency; cross-process file locks aren't useful in a sandboxed runtime. - UDP multicast (set-broadcast / join-multicast-v4/v6) — niche; mDNS / SSDP are the only realistic drivers and no near-term capsule needs them. - HTTP outgoing body streaming — needs a properly-designed outgoing-body resource (mirrors wasi-http); designing as a 1.x package addition once a real upload-streaming capsule appears.
3 tasks
joshuajbouw
added a commit
that referenced
this pull request
May 22, 2026
…/ streams) (#8) ## Summary Replaces the `wasi:io` dependency in the host ABI with an Astrid-owned `astrid:[email protected]` package (three interfaces: `error`, `poll`, `streams`). The host ABI is now fully self-contained — no `wasi:*` surface anywhere — so every host call routes through Astrid's audit, capability, cancellation, and per-principal quota layers without exception. This is also what makes the hermit-rs unikernel target viable: the WIT contract stays stable, and the kernel-side impl swaps from wasmtime-wasi-backed futures to native unikernel primitives. ## Context: this amendment edits @1.0.0 files This PR modifies `host/{net,http,process,sys}@1.0.0.wit` in place rather than shipping `@1.1.0` variants. **One-shot pre-adoption amendment**, justified because: - The @1.0.0 freeze landed on `main` in #7 without any downstream consumer. - The kernel scaffolding is still in WIP (see `unicity-astrid/astrid#feat/wit-per-domain-split`); no SDK or capsule has bound to the @1.0.0 shapes. - The dependency change (wasi:io → astrid:io) is structural enough that doing it as @1.1.0 evolution would force every kernel + SDK to register dual versions of four packages forever, for no downstream benefit. The frozen-file CI gate has been retired in commit `8a0f29d` — it was tripping on the legitimate pre-adoption amendments and providing no consumer protection (no consumers exist yet). The freeze rule remains documented in the README; re-enable the check from git history once a real downstream consumer ships against a versioned file. ## Commits 1. **`9d71a05` fix!: replace wasi:io/poll with astrid:io/[email protected]** - New `host/[email protected]` with the `pollable` resource and `poll(list<borrow<pollable>>) -> result<list<u32>, error-code>`. - `host/{ipc,net,http,process}@1.0.0.wit`: `use wasi:io/[email protected].{pollable}` → `use astrid:io/[email protected].{pollable}`. - `host/[email protected]`: header rewritten — no `wasi:*` carve-outs. 2. **`cfe24c0` chore(deps): drop vendored wasi-io** - Removes `deps/wasi-io/` entirely. - `scripts/validate-wit.sh` + `.github/workflows/lint.yml` comments updated. 3. **`0b66ee2` feat(io)!: add astrid:io/error + astrid:io/streams** - `host/[email protected]` extended with two more interfaces matching the `wasi:[email protected]` shape (error / poll / streams). - Stream halves wired into resources where they pull weight: - `host/[email protected]`: `tcp-stream.{read-stream, write-stream}` — primary throughput primitive for capsule-hosted TCP servers / proxies. - `host/[email protected]`: `http-stream.body-stream` — HTTP response body forwarding. 4. **`13daba9` fix(process)!: drop stream halves on process-handle; mark package desktop-only** - `process-handle` stays byte-oriented (`write-stdin` / `read-logs`); no stream accessors. - Reasoning: IPC bus already handles capsule-to-capsule throughput, hermit-rs has no fork/exec model, write-stdin/read-logs covers MCP-stdio and tool capture. 5. **`8a0f29d` fixup(io): apply Gemini review + drop immutability CI gate** - `poll()` per-call cap raised from 64 → 256 (a capsule at full IPC subscription quota was already past 64). - `input-stream.read()` no longer claims to trap above 1 MiB — wasi-style, only traps on lengths exceeding wasm32's addressable range; the host's 1 MiB ceiling truncates the return instead. - `output-stream.blocking-write-and-flush()` and `blocking-write-zeroes-and-flush()` no longer cap at 4 KiB. The host segments large transfers internally and yields between chunks. - Retired `scripts/lint-wit-immutability.sh` + the `WIT frozen-file immutability` workflow job (see "Context" above). ## Why Astrid-owned and not wasi:io Even though `pollable.block` and `streams.read` look like they're "just plumbing," the wasmtime-wasi implementation skips: - **Cancellation**: a capsule unloading mid-block strands the host task on a future that may never complete. - **Audit**: every read / write / poll / block / splice is invisible to the audit log. - **Per-principal accounting**: no quota dial on pollable / stream handles, no rate limit on poll-loop spam. - **Cross-capsule isolation**: relies entirely on the wasmtime ResourceTable boundary, no Astrid-side reinforcement. Owning the namespace lets the kernel wrap every operation in the same gates that cover `astrid:fs`, `astrid:net`, etc. The wasmtime-wasi-io storage types (`DynPollable`, `IoError`, `DynInputStream`, `DynOutputStream`) are still reused on the desktop kernel — they're futures, not syscalls — but the `Host` traits themselves are Astrid's, with audit + cancel + quota wrappers around each call. On hermit-rs, those storage types swap for native unikernel wait/io primitives; the WIT contract is unchanged. ## Test plan - [x] `scripts/validate-wit.sh` passes on all 13 host packages locally (`wasm-tools 1.245.1`). - [x] Downstream kernel (`unicity-astrid/astrid#feat/wit-per-domain-split`) builds clean against this branch (`cargo build --workspace`, `cargo clippy --workspace -- -D warnings`). - [x] CI: `WIT files parse` job passes. ## Downstream - `unicity-astrid/astrid#feat/wit-per-domain-split` consumes this branch via the wit submodule. Once this merges, that PR rebases on the new submodule pointer and is ready for review. Kernel-side `MAX_POLL_LIST = 64` will be bumped to 256 in a follow-up commit on that branch to match the new WIT. - SDK and capsules are unaffected by this PR specifically.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Splits the monolithic
astrid:[email protected]host ABI into per-domain frozen packages and applies the full pre-1.0 audit-driven hardening. Resolves unicity-astrid/astrid#750 and aligns with rfcs#22 (host_abi RFC).This PR is the 1.0 freeze candidate for the WIT contract. Every shape decision here is irreversible after merge (per the frozen-file rule the PR also lands), so the contract has been audited against Rust
std, WASI Preview 2, and capability-based security guidance.12 host packages —
astrid:*(kernel ABI)Each is a frozen versioned file under
host/. Every host call is gated, principal-scoped, and audited.astrid:[email protected]file-handleresource (open/read-at/write-at/sync-data/sync-all/stat/set-len) + path ops (exists/mkdir/mkdir-all/readdir/stat/stat-symlink/unlink/read-file/write-file/append/copy/rename/remove-dir-all/canonicalize/read-link/hard-link/open).file-typeenum,datetimerecord.astrid:[email protected]subscriptionresource (poll/recv/subscribe-readiness) + publish/publish-as.principal-attributionvariant onipc-messagedistinguishing kernel-verified from uplink-claimed.astrid:[email protected]uplink-profileenum.astrid:[email protected]astrid:[email protected]unix-listener,tcp-listener(inbound TCP),tcp-stream(full I/O surface + TCP options),udp-socket(unconnected + connected modes). Factories: bind-unix/bind-tcp/connect-tcp/udp-bind/lookup-host.astrid:[email protected]http-streamresource (status/headers/read-chunk/close/subscribe-readable).http-methodvariant. SSRF airlock.astrid:[email protected]astrid:[email protected]process-handleresource (read-logs/write-stdin/close-stdin/signal/kill/wait/wait-with-output/os-pid/subscribe-exit/subscribe-logs). Factories: spawn/spawn-background.exit-info { exit-code, signal }. Typedprocess-signalenum.astrid:[email protected]elicit-typeenum +elicit-responsevariant. Install/upgrade-only.astrid:[email protected]approval-decisionenum.astrid:[email protected]astrid:[email protected]interceptor/background/installable/upgradable). Capsulesincludeonly what they implement.Namespace split —
astrid-bus:*for capsule-to-capsuleCapsule-to-capsule IPC event schemas moved from
astrid:*toastrid-bus:*to resolve collisions (host'sastrid:approvalandastrid:elicitcollided with bus equivalents) and codify the layering. 17interfaces/*.witfiles renamed, internal cross-refs updated,Capsule.tomlimports split:Three coordinated evolution rules (from #750)
astrid:<name>@X.Y.Zpackage so bumpingastrid:netdoesn't recompile capsules that only importastrid:kv.(package, version)pair explicitly into the wasmtime CM linker. (Kernel work, separate PR.)host/<name>@X.Y.Z.witis immutable after main. Shape changes ship as new files. Enforced byscripts/lint-wit-immutability.sh.Audit-driven hardening (across three agent audits)
Typed errors — every
result<_, string>replaced with per-packageerror-codevariants (fs/ipc/net/kv/http/sys/process/approval/elicit/identity/uplink). Catch-allunknown(string)arm keeps escape hatch.Resource types — every long-lived handle migrated from forgeable
u64to typed CM resources (file-handle,unix-listener,tcp-listener,tcp-stream,udp-socket,http-stream,process-handle,subscription). Type safety across interfaces, automatic Drop, per-instance ownership.Pollable foundation —
wasi:[email protected](poll + streams) retained in the linker.subscribe-readiness/subscribe-readable/subscribe-exit/subscribe-logsmethods returnwasi:io/poll.pollableon subscription, unix-listener, tcp-listener, tcp-stream, udp-socket, http-stream, process-handle. Bridge / fan-out capsules can multiplex bus + sockets + HTTP + child-process events viawasi:io/poll.pollwithout embedding a runtime purely for that.ipc-publish-asspoofing closed —principal-attributionvariant onipc-messagemakes verified vs claimed explicit; downstream capability checks on sensitive actions MUST requireverified. The kernel audit-logs both the uplink's true principal and the claimed principal on publish-as.Doc-as-contract — VFS-scheme path return rule (no host real-paths leak through canonicalize / read-link / errors), TOCTOU re-validation contract (kernel re-resolves every path on every call), charset rules for topics/keys/paths, per-fn audit policy, cumulative byte quotas,
get-interceptor-bindingsscoped to caller's own bindings (no cross-capsule capability inference).WASI alignment —
shutdown-how::receive/send/both(was read/write/both);hop-limit(was TTL);datetimerecord for file timestamps (wasmtime-ns: option<u64>, fixes 2554 overflow + supports pre-1970);random-bytes: u64length;http-methodvariant matchingwasi:http.WASI no longer exposed broadly — the kernel's
add_to_linker_syncwill narrow towasi:io/poll+wasi:io/streamsonly (kernel work, separate PR).astrid:*becomes the single host ABI. Random / clocks / sleep now live onastrid:sysso every host call goes through the same audit/principal layer.wasi:filesystemandwasi:socketsno longer exposed (capsules go throughastrid:fs/astrid:netfor capability-gated access).Late additions before freeze (rest of the 1.x candidate list)
tcp-listenerresource +bind-tcp) — for self-hosted webhook receivers, gRPC endpoints, Prometheus scrape ports. Gated bynet_tcp_bind.fs-hard-link) — atomic publish patterns, content-addressed stores. Same-VFS-scheme only.fs-mkdirstrict /fs-mkdir-allrecursive split.udp-socket.connect/disconnect/send/recv/peer-addr) — QUIC, DNS-over-UDP, syslog-over-UDP. Faster syscall path and built-in spoofed-peer filtering.SO_REUSEADDRontcp-streamfor listener restart cycles.process-handle.os-pid()for cgroup / /proc correlation.process-handle.wait-with-output()closes the read-logs race on short-lived children.subscribe-readableon http-stream completes the pollable composition story.Intentionally deferred (each clean minor bump if/when demand emerges)
fs-symlink) — sandbox-escape risk outweighs demand. Read-onlyfs-read-link/fs-stat-symlinkretained.kv-cascovers in-runtime concurrency; cross-process locks aren't useful in a sandboxed runtime.outgoing-bodyresource (mirroringwasi-http); 1.x package addition.SO_*socket options beyond keepalive/linger/reuseaddr — defaults work for the typical agent capsule.Tooling
deps/wasi-io/— vendoredwasi:[email protected](poll, streams, error, world) at the official release tag.scripts/validate-wit.sh— stages eachhost/*.witwithdeps/+ sibling host/ files into a tempdir before parsing, so cross-packageuseclauses resolve. (wasm-tools1.x can't walk deps trees multi-pass.)scripts/lint-wit-immutability.sh— fails any PR that modifies / deletes / renames a published*@X.Y.Z.witfile..github/workflows/lint.ymlruns both on every PR + push to main.Parity vs the pre-split monolithic
astrid:[email protected]Verified during the per-domain split: 65/65 original host functions present (identical names, sorted-diff clean), 30/30 referenced types preserved, 4/4 guest exports preserved verbatim. 4 dead WIT records dropped (
capsule-context,tool-input,tool-output,tool-definition— never referenced in any WIT signature). Subsequent commits added capabilities (UDP, DNS, TCP listener, hard-link, process surface extensions, etc.) without removing any.Test plan
wit-immutabilitylint passes (no published-file modifications).wit-parsesjob passes —scripts/validate-wit.shvalidates everyhost/*.witwith proper deps staging.includesemantics validated against synthetic consumer worlds (interceptor-only, all-four-worlds, realistic interceptor + ipc/host import).deps/wasi-io/resolution.Follow-ups in dependent repos
unicity-astrid/astrid— kernel implementation: narrowadd_to_linker_synctowasi:io/poll+wasi:io/streams, implement all the newastrid:sys/astrid:fs/astrid:kv/astrid:net/astrid:processfns + resources, wire up pollable on the relevant resources, droptrigger_hookimpl, error-code translation in host impls. Keep cron plumbing alone.unicity-astrid/sdk-rust— regen bindings against the new shape, expose pollable wrappers, error-code → SysError translation,fan_out_collecthelper alongside existingrequest_response.unicity-astrid/sdk-js— mirror.trigger_hook(to bus-side fan-out); other capsules migrate toastrid_sdk::ipc::request_responsefor correlation-id patterns.Refs: unicity-astrid/astrid#750, rfcs#22