feat(net): outbound TCP host fns (full std::net::TcpStream parity) by joshuajbouw · Pull Request #746 · astrid-runtime/astrid

joshuajbouw · 2026-05-19T11:39:14Z

Linked Issue

Closes #745

Summary

Adds outbound TCP to the host ABI: net-connect-tcp plus 14 more host fns that give capsules full std::net::TcpStream parity. Unblocks every capsule that needs persistent TCP — WebSocket clients (Fulcrum, Nostr), MQTT, Discord/Telegram gateways, postgres / redis. Strictly additive to the existing net interface; TLS is not in scope (capsules ship their own crate, matching the std::net model where TLS lives in user-space).

Changes

WIT: syncs canonical unicity-astrid/wit#5 into in-tree wit/astrid-capsule.wit. Adds the shutdown-how enum + 15 new net-* fns (connect-tcp, read-bytes, write-bytes, peek, shutdown, peer-addr, local-addr, set-nodelay / nodelay, set-read-timeout / read-timeout, set-write-timeout / write-timeout, set-ttl / ttl) and brings in ipc-message.principal (canonical PR Hot-Reload of OpenClaw Plugins #4) which had drifted from the in-tree copy.
Manifest: CapabilitiesDef.net_connect: Vec<String> — per-capsule allowlist for outbound TCP destinations. Each entry is "host:port" (exact) or "host:*" (any port). Empty / missing denies all outbound TCP (fail-closed). Independent of net_bind, http, host_process.
Kernel (engine/wasm/host/net/): net_connect_tcp runs capability check → host string validation → DNS resolve → SSRF airlock (is_safe_ip, same gate as http-request) → bounded 10s connect. read_bytes / write_bytes are byte-stream variants honouring per-stream timeouts (empty Vec = EOF unambiguously; timeout fires as would block for SDK-side ErrorKind::WouldBlock mapping). peek uses tokio's native TcpStream::peek. shutdown uses socket2::SockRef for full Read|Write|Both direction support.
Security caps: per-call read/peek buffer capped at MAX_BYTES_PER_CALL = 10 MB (guards against u32::MAX OOM). Host string validation (non-empty, ≤255 bytes per RFC 1035, no null bytes) at the host-fn boundary.
Security gate: new CapsuleSecurityGate::check_net_connect trait method, default-deny. ManifestSecurityGate impl matches host:port against the manifest allowlist (case-insensitive host, exact-or-* port).
Refactors: net.rs split into net/{mod,handshake,stream}.rs and manifest.rs split into manifest/{mod,capabilities,topics}.rs to stay under the 1000-line CI cap. pub use re-exports preserve the public API.

Test Plan

Automated

cargo test --workspace passes (296 lib tests in astrid-capsule — was 285, + 11 new covering net_connect pattern matching, socket2 shutdown round-trip, validate_host gates)
No new clippy warnings (cargo clippy --workspace -- -D warnings clean)
cargo +nightly fmt --all --check clean

Manual

Tag a SDK release (separate PR per CLAUDE.md) so capsule authors can bump
Smoke-test a capsule against a public ws:// endpoint over the new surface (deferred to a follow-up capsule PR)

Refs

Checklist

Linked to an issue
CHANGELOG.md updated under [Unreleased]

Adds one outbound-TCP host fn — astrid:capsule/net.net-connect-tcp(host, port) -> stream-handle — gated by a per-capsule net_connect allowlist in Capsule.toml. Reuses the existing net-read / net-write / net-close-stream plumbing (single new variant on a NetStream enum; framing helpers shared across UnixStream and TcpStream via tokio::io::AsyncRead/AsyncWrite trait bounds). SSRF airlock from http-request runs on the resolved IP after the capability check. 10s connect timeout. Per-capsule active-stream cap (8) shared with inbound net-accept. Unblocks WebSocket clients, MQTT, Discord/Telegram, postgres/redis. The immediate motivator is a Unicity-network capsule wrapping Sphere SDK (Fulcrum + Nostr WebSocket transports). Also splits the now-1000-line manifest.rs into a manifest/ submodule (capabilities.rs, topics.rs) to land under the 1000-line CI cap. Public API preserved via pub use re-exports. And resyncs wit/astrid-capsule.wit from canonical (picks up the principal field on ipc-message that drifted in from canonical PR #4). RFC: astrid-runtime/rfcs#27 WIT: astrid-runtime/wit#5 SDK: unicity-astrid/sdk-rust feat/net-connect-tcp Tracking issue: closes #745

Implements 14 new host fns to give capsules complete std::net::TcpStream parity. All are kernel-side; the SDK wraps them in unicity-astrid/sdk-rust. New fns on net::Host: - net_read_bytes / net_write_bytes: byte-stream read/write (no length-prefix framing). Read returns empty Vec on EOF; timeout surfaces as 'read would block' / 'write would block' to match std::io::ErrorKind::WouldBlock. - net_peek: peek without consuming - net_shutdown(read|write|both): half-close (write supported; read/both return clear errors pending socket2 integration) - net_peer_addr / net_local_addr: address introspection - net_set_nodelay / net_nodelay: TCP_NODELAY toggle - net_set_read_timeout / net_read_timeout (stored per-stream) - net_set_write_timeout / net_write_timeout (stored per-stream) - net_set_ttl / net_ttl: IP TTL Refactor: - NetStream::Tcp now holds TcpStreamSlot { stream, read_timeout, write_timeout } instead of a bare Arc<Mutex<TcpStream>>. Per-stream timeout state lives next to the socket so the byte-stream read/write fns can apply tokio::time::timeout per call without global state. - net.rs split into net/ submodule (handshake.rs + stream.rs + mod.rs) to stay under the 1000-line CI cap. Unix variants reject TCP-only ops (nodelay, ttl, addresses, timeouts) with clear errors — those are TCP socket options that don't apply. RFC: astrid-runtime/rfcs#27 WIT: astrid-runtime/wit#5 Tracking issue: closes #745

Replaces the temporary 'only write half-close supported' error with the real impl. tokio::net::TcpStream only exposes the write half via the AsyncWrite trait, but socket2::SockRef::from(&stream) wraps the borrowed FD and exposes shutdown(std::net::Shutdown) for all three directions — the same OS shutdown(2) syscall std::net::TcpStream uses internally. No FD ownership transfer; the SockRef borrows the tokio stream so subsequent reads/writes on the slot still work (within whatever half-closed state the shutdown produced). Added a unit test that exercises Read/Write/Both against a real loopback TcpStream — guards against a future tokio or socket2 bump breaking the glue. socket2 was already a transitive dep via tokio; promoted to a direct dep on astrid-capsule.

gemini-code-assist · 2026-05-19T11:40:51Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enables outbound TCP connectivity for capsules, allowing them to interface with protocols like WebSockets, MQTT, and various database clients. It implements a comprehensive set of host functions mirroring std::net::TcpStream while ensuring security through a new per-capsule allowlist. The implementation includes significant refactoring of the network and manifest modules to accommodate the expanded API surface and ensure future scalability.

Highlights

Outbound TCP Support: Added net-connect-tcp and 14 related host functions to provide full std::net::TcpStream parity for capsules.
Security Gating: Introduced a net_connect capability allowlist in Capsule.toml to strictly control outbound TCP access.
Refactoring: Split net.rs and manifest.rs into submodules to improve maintainability and stay within file size limits.
WIT Contract Sync: Resynced wit/astrid-capsule.wit with canonical upstream and added the ipc-message.principal field.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

TCP streams now flow with ease, / Capsules connect as they please. / With gates in place and logic sound, / New network paths are finally found.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

This pull request introduces outbound TCP support for capsules, allowing persistent connections via the net.connect-tcp host function, gated by a new net_connect manifest capability. The implementation includes DNS resolution with SSRF protection and several new host functions for raw byte-stream I/O, supported by a refactored networking module and an updated HostState that manages both Unix and TCP streams. Review feedback identified critical security risks, specifically potential OOM vulnerabilities in net_peek and read_bytes_inner due to uncapped buffer allocations from guest input. The reviewer also recommended early validation of host strings to prevent malformed DNS queries and suggested architectural changes to avoid holding Mutex locks across await points, which could impede full-duplex I/O performance. Additionally, the reviewer noted that error handling in read_bytes_inner should be refined to clearly distinguish between timeouts and peer disconnects.

Addresses Gemini code-review feedback on astrid-runtime/sdk-rust#42. Before: - timeout=None falls through to a 50ms internal timeout. - Timeout expiry with no data returned Ok(empty Vec). - Caller can't distinguish 'transient no data' from 'peer disconnected'. After: - timeout=None drops the tokio::time::timeout wrapper entirely. Reads block until data, EOF, or capsule unload (via the outer bounded_block_on_cancellable cancellation token). - Empty Vec is now ALWAYS EOF — matches std::io::Read's Ok(0) contract. - 'read would block' error only surfaces when the caller has explicitly set a read timeout that expired. SDK-side, that maps to std::io::ErrorKind::WouldBlock so std::Read callers can distinguish. The peek and write_bytes paths already had the right shape; only net_read_bytes carried the bug.

Addresses Gemini code review feedback on #746. OOM cap on byte-stream reads (security-critical): - Guest-supplied max_bytes was used directly as the buffer size in read_bytes_inner and net_peek. u32::MAX would allocate 4 GB on the host. New shared constant MAX_BYTES_PER_CALL = 10 MB (matches the framed net-read payload cap) is applied as .min(max_bytes_usize) at both sites. - 10 MB is well above any realistic single-call read for the targeted protocols: TLS records cap at 16 KB, WebSocket frames typically ≤64 KB, MQTT control packets stream in chunks. Host string validation (security-medium): - net_connect_tcp now rejects empty hosts, hosts >255 bytes (RFC 1035 max-name-length), and hosts containing null bytes — at the host-fn boundary, before the resolver / audit log / tracing spans see the input. New validate_host helper + 5 unit tests covering each gate. Already addressed by the prior commit (e12e60b): - read_bytes_inner EOF-vs-timeout ambiguity. timeout=None now blocks indefinitely instead of falling through to a 50ms internal poll; empty Vec is unambiguously EOF. Not addressed in this PR: - Mutex-across-await architectural concern (full-duplex on one handle). Splitting the stream into read/write halves with independent locks would touch every method dispatcher and the HostState layout. Worth doing but distinct from the TCP surface this PR ships — filed for a follow-up issue.

## Summary JS/TS mirror of [astrid-runtime/sdk-rust#42](astrid-runtime/sdk-rust#42). Adds outbound TCP for capsules — `connect()` plus the full set of methods a Rust binary's `std::net::TcpStream` exposes — on the `StreamHandle` class. ## Changes ### Ambient module (`wit-imports.d.ts`) Declares the 14 new host fns + `ShutdownHow` type in the `astrid:capsule/[email protected]` module, matching the canonical WIT in [astrid-runtime/wit#5](astrid-runtime/wit#5). ### `net.ts` - `connect(host, port)` — outbound TCP (already in this branch's tip, now upgraded from a stub). - New `StreamHandle` methods: - `readBytes(maxBytes)` / `writeBytes(data)` — byte-stream read/write (no length-prefix framing). - `peek(maxBytes)` — non-destructive read. - `shutdown(how)` — half-close (`'read' | 'write' | 'both'`). - `peerAddr()` / `localAddr()` — `ip:port` strings. - `setNodelay(b)` / `nodelay()` — TCP_NODELAY. - `setReadTimeout(ms)` / `readTimeout()`. - `setWriteTimeout(ms)` / `writeTimeout()`. - `setTtl(n)` / `ttl()` — IP TTL. ### Submodule `contracts/` bumped to `feat/net-connect-tcp` tip (matches WIT PR head). ### Unchanged The existing framed surface (`recv` / `tryRecv` / `send` / `close` + `AsyncIterable<Uint8Array>`) keeps its semantics — that's the right shape for the inbound CLI proxy. Outbound TCP capsules use the byte-stream pair (`readBytes` / `writeBytes`). ## TLS Out of scope. JS-side capsules ship their own WebSocket / TLS library over `readBytes` / `writeBytes`, same as native Rust binaries do — std::net::TcpStream doesn't ship TLS, and we're holding that line. ## Verification - `npm run -w packages/astrid-sdk build` clean (tsc -b) - 64 contract types regenerate cleanly via the prebuild script ## Refs - RFC: [astrid-runtime/rfcs#27](astrid-runtime/rfcs#27) - WIT: [astrid-runtime/wit#5](astrid-runtime/wit#5) - SDK (rust): [astrid-runtime/sdk-rust#42](astrid-runtime/sdk-rust#42) - Core: [astrid-runtime/astrid#746](astrid-runtime/astrid#746) - Tracking issue: [astrid-runtime/astrid#745](astrid-runtime/astrid#745)

## Linked Issue Closes #757 ## Summary Bump all workspace crates from 0.6.0 to 0.7.0. Big release rolling up the per-domain WIT host ABI migration (#752 — wasi-elimination, every host call routed through audited astrid:* interfaces), outbound TCP for capsules (#746), wasmtime 43 → 45 (closes RUSTSEC-2026-0149), atomic kv_cas, O(1) HostState quota counters, and the Gemini review fixups landed since 0.6.0 was tagged. ## Changes - Workspace version 0.6.0 → 0.7.0 - All 20 workspace dependency versions updated to 0.7.0 - CHANGELOG [Unreleased] rolled into [0.7.0] with consolidated sections — duplicate Added/Changed blocks (from #752 + #746 accumulation) merged into a single canonical set, wasi:* elimination moved from Changed to Breaking, three new entries for the Gemini #752 follow-up commits (tokio::process::Child, strict fs-mkdir, wasmtime 45 bump). Order: Breaking, Added, Changed, Fixed.

…s Closed (#760) ## Linked Issue Closes #759 ## Summary Fix the `TcpStream::read_bytes` / `peek` cancellation hole flagged by Gemini on #758 (`CHANGELOG.md:48` thread), plus a CHANGELOG `[Unreleased]` cleanup that consolidates duplicate Added/Changed blocks from the independent #752 + #746 roll-ups. Goes in before #758 so the 0.7.0 release inherits the corrected behaviour and cleaner CHANGELOG structure. ## Changes ### `crates/astrid-capsule/src/engine/wasm/host/net/tcp_stream.rs` - `read_bytes`: `None => Ok(Vec::new())` → `None => Err(ErrorCode::Closed)` - `peek`: `result.unwrap_or(Ok(Vec::new()))` → `result.unwrap_or(Err(ErrorCode::Closed))` Both methods previously collapsed cancellation into an empty `Vec<u8>`, which is indistinguishable from a clean EOF in byte-stream reads (`std::io::Read::read` / `tokio::io::AsyncReadExt::read` convention). The framed read path (`read_frame`) and the write side (`write_bytes`, `shutdown`) already return `Err(ErrorCode::Closed)` on cancellation. `read_bytes` / `peek` were the outliers. Capsules with EOF-triggered finalizers (write trailers, send last-message IPC, flush log, transition to "stream complete" state) would execute those finalizers under a forced unload when the cancel fires mid-read. The bug surface is narrow (the capsule has milliseconds to live) but it's a real correctness hole. Now `Closed` distinguishes cancellation from EOF; empty `Vec` keeps its "clean EOF" meaning (no behaviour change for the EOF case). ### `CHANGELOG.md` (`[Unreleased]` consolidation) - Merged the two `### Added` blocks (one from #752, one from #746) into a single block. Same for `### Changed`. - New `### Fixed` entry for the `read_bytes` / `peek` change above. - No new entries removed — purely a structural cleanup. ## Test Plan ### Automated - [x] `cargo test -p astrid-capsule --lib` — 272 passed, 0 failed - [x] `cargo clippy --workspace --all-features -- -D warnings` clean ### Manual - [x] N/A — the cancellation path only fires on capsule unload, exercised in integration ## Checklist - [x] Linked to an issue - [x] CHANGELOG.md updated under `[Unreleased]`

## Linked Issue Closes #757 ## Summary Bump all workspace crates from 0.6.0 to 0.7.0. Big release rolling up the per-domain WIT host ABI migration (#752 — wasi-elimination, every host call routed through audited astrid:* interfaces), outbound TCP for capsules (#746), wasmtime 43 → 45 (closes RUSTSEC-2026-0149), atomic kv_cas, O(1) HostState quota counters, the Gemini review fixups for #752, and the TcpStream cancellation hole closed in #760. ## Changes - Workspace version 0.6.0 → 0.7.0 - All 20 workspace dependency versions updated to 0.7.0 - CHANGELOG [Unreleased] rolled into [0.7.0] (consolidation already done in #760 — Breaking, Added, Changed, Fixed all single-section)

## Linked Issue Closes #757 ## Summary Bump all workspace crates from 0.6.0 to 0.7.0. Big release rolling up the per-domain WIT host ABI migration (#752 — wasi-elimination, every host call routed through audited `astrid:*` interfaces), outbound TCP for capsules (#746), wasmtime 43 → 45 (closes RUSTSEC-2026-0149), atomic `kv_cas`, O(1) `HostState` quota counters, and the Gemini review fixups landed since 0.6.0 was tagged. ## Changes - Workspace version 0.6.0 → 0.7.0 - All 20 workspace dependency versions updated to 0.7.0 - CHANGELOG `[Unreleased]` rolled into `[0.7.0]` with consolidated sections — duplicate Added/Changed blocks (from #752 + #746 accumulation) merged into a single canonical set, `wasi:*` elimination moved from Changed to Breaking, three new entries for the Gemini #752 follow-up commits (tokio::process::Child, strict fs-mkdir, wasmtime 45 bump). Order: Breaking, Added, Changed, Fixed. ## Test Plan ### Automated - [x] `cargo check --workspace` passes ### Manual - [ ] Tag `v0.7.0` after merge - [x] Release CI builds cross-platform binaries ## Checklist - [x] Linked to an issue - [x] CHANGELOG.md updated under `[0.7.0]`

joshuajbouw added 3 commits May 19, 2026 14:21

joshuajbouw mentioned this pull request May 19, 2026

feat(sdk,net): full std::net::TcpStream surface mirror astrid-runtime/sdk-js#3

Merged

gemini-code-assist Bot reviewed May 19, 2026

View reviewed changes

joshuajbouw added 2 commits May 19, 2026 16:03

joshuajbouw self-assigned this May 19, 2026

joshuajbouw merged commit 8a3e5ce into main May 19, 2026
17 of 18 checks passed

joshuajbouw deleted the feat/net-connect-tcp branch May 19, 2026 12:50

This was referenced May 21, 2026

feat(wit,kernel)!: pre-1.0 ABI hardening — split astrid:capsule, multi-version registration, frozen-WIT discipline #750

Closed

chore: release v0.7.0 #757

Closed

chore: release 0.7.0 #758

Merged

This was referenced May 25, 2026

fix(capsule/net): TcpStream::{read_bytes,peek} return empty Vec on cancellation, indistinguishable from clean EOF #759

Closed

fix(capsule/net): TcpStream::{read_bytes,peek} surface cancellation as Closed #760

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(net): outbound TCP host fns (full std::net::TcpStream parity)#746

feat(net): outbound TCP host fns (full std::net::TcpStream parity)#746
joshuajbouw merged 5 commits into
mainfrom
feat/net-connect-tcp

joshuajbouw commented May 19, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot commented May 19, 2026

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

joshuajbouw commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Linked Issue

Summary

Changes

Test Plan

Automated

Manual

Refs

Checklist

Uh oh!

gemini-code-assist Bot commented May 19, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

joshuajbouw commented May 19, 2026 •

edited

Loading