Skip to content

Add --firewall: domain-level network isolation via tinyproxy + softnet#48

Open
zachary-s-wiens wants to merge 8 commits into
webcoyote:mainfrom
zachary-s-wiens:feature/firewall
Open

Add --firewall: domain-level network isolation via tinyproxy + softnet#48
zachary-s-wiens wants to merge 8 commits into
webcoyote:mainfrom
zachary-s-wiens:feature/firewall

Conversation

@zachary-s-wiens

@zachary-s-wiens zachary-s-wiens commented May 21, 2026

Copy link
Copy Markdown

Summary

  • Adds --firewall[=FILE] global option for host-side domain-level network isolation
  • Two enforcement layers (both host-side — guest sudo can't bypass):
    1. tinyproxy forward proxy with FilterDefaultDeny — only allowlisted domains pass
    2. Tart softnet — VM can only reach the gateway IP (where the proxy runs)
  • Gateway IP detected dynamically from bridge100 after VM boot (softnet uses 192.168.2.1)
  • Proxy env vars (HTTPS_PROXY, HTTP_PROXY, NO_PROXY) injected into VM SSH session
  • Auto-installs tinyproxy via brew if missing

Usage

clod --firewall shell myvm              # built-in defaults (.github.com, .anthropic.com)
clod --firewall=domains.txt shell myvm  # custom domain allowlist

Domain file format (one per line, leading dot = subdomains):

.github.com
.anthropic.com
firestore.googleapis.com

Files changed

File What
lib/firewall.sh New module — proxy lifecycle, domain→regex conversion, gateway detection
clod --firewall flag parsing, proxy start/stop in shell dispatch
lib/instance.sh Proxy env injection in ssh_into_vm(), softnet flags in vm_shell()
lib/commands.sh Help text
spec/firewall_spec.sh 13 ShellSpec tests (filter regex, gateway detection, proxy URL, softnet args)
spec/help_spec.sh Verify --firewall in help output

Test plan

  • shellspec — 47/47 pass (13 new + 34 existing)
  • Manual: clod --firewall shell <instance> → verify allowed domains work, blocked domains get HTTP 403 from proxy
  • Manual: curl --noproxy '*' <anything> inside VM → verify softnet blocks direct bypass

🤖 Generated with Claude Code

Two-layer host-side enforcement that guest sudo cannot bypass:

1. tinyproxy (explicit forward proxy) — domain allowlist via Filter +
   FilterDefaultDeny. VM's HTTPS_PROXY points here.
2. Tart softnet — hypervisor-level isolation blocks all VM outbound
   except the gateway IP (where the proxy runs).

Usage:
  clod --firewall shell myvm              # built-in defaults (.github.com, .anthropic.com)
  clod --firewall=domains.txt shell myvm  # custom domain allowlist

The domain file uses one domain per line, leading dot for subdomains:
  .github.com        # matches *.github.com and github.com
  exact.example.com  # matches only exact.example.com

Gateway IP is detected dynamically from bridge100 after VM boot
(softnet uses 192.168.2.1, differs from bridged mode's 192.168.64.1).

Includes ShellSpec tests for filter generation, gateway detection,
softnet args, and proxy URL construction.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@zachary-s-wiens

Copy link
Copy Markdown
Author

FYI there's a few improvements I'm adding. Will rebase this PR when ready.

zacsst and others added 7 commits May 25, 2026 11:43
NSURLSession-backed tools (xcodebuild's SwiftPM, system frameworks,
anything reading CFNetworkCopySystemProxySettings) ignore HTTP_PROXY
env vars and read system proxy settings from macOS preferences. Under
--firewall, those tools were bypassing tinyproxy entirely, so SwiftPM
binary XCFramework downloads (Firebase gRPC, Facebook SDK, etc.) hit
the softnet block and timed out, leaving workspace-state.json with
empty "artifacts": [] and silently corrupting the SPM cache.

ssh_into_vm now calls vm_apply_system_proxy on every connect:
  - firewall active  -> setwebproxy + setsecurewebproxy to the same URL
                        already injected via HTTP_PROXY env
  - firewall off     -> setwebproxystate off / setsecurewebproxystate off
                        so a no-firewall session after a firewall session
                        doesn't inherit stale settings

Uses sudo -n; requires NOPASSWD sudo for networksetup on the guest,
which is already provided by ALLOW_SUDO=true at build-base time.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
firewall_start only checked _FIREWALL_PID, an in-process variable that
tracks tinyproxys started by THIS shell. When a sibling tool (e.g. a
devcontainer script sharing the same port to reuse one upstream proxy)
already holds the port, clod's tinyproxy fails to bind, exits with
"Could not create listening sockets", and the startup loop aborts.

Add an nc -z probe ahead of the spawn. If the port is already open,
log that we're reusing and return — _FIREWALL_PID stays empty so the
matching firewall_stop path won't try to kill a process it doesn't
own.

Both sides are expected to use the same allowlist file, so the filter
set is identical and sharing is safe.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
Previous version silently exited 0 when no network service was detected
and swallowed all stderr from networksetup, making it impossible to
diagnose why NSURLSession-backed tools (xcodebuild/SwiftPM) were still
bypassing the firewall.

Now: prints the detected service name, dumps the resulting proxy state
on apply, and surfaces stderr indented under a host-side warning on
failure. Empty service list is treated as a failure rather than a
silent no-op.
The previous 'first non-disabled service' heuristic picked up virtual
network services like com.redhat.spice.0 (SPICE virtio) that don't
carry VM traffic. networksetup happily set the proxy on that service,
NSURLSession ignored it (since real traffic flows through a different
interface), and SwiftPM kept hitting the softnet block with timeouts.

Now we read the default-route interface from 'route get default' and
pick the networksetup service whose '(Hardware Port: ..., Device: enX)'
line matches. Fails loudly if no service maps to the active device.
Lets the guest tunnel native SSH to allowlisted hosts (e.g. github.com:22)
through tinyproxy. The domain Filter still applies — port 22 isn't a
wildcard hole, only allowlisted hosts accept CONNECT on it.

This replaces the older insteadOf+credential-helper chain consumers had
to use to route git through the HTTPS proxy. With ConnectPort 22, guests
can use a plain ProxyCommand and speak SSH directly.
Gradle's parallel artifact resolve (AGP InstrumentationAnalysisTransform
running across the Android/Kotlin classpath) opens >50 concurrent CONNECT
tunnels through tinyproxy and exhausts the previous limit. Symptom: gradle
hangs indefinitely at the resolve step; tinyproxy log shows "Maximum number
of connections reached. Refusing new connections." 200 has comfortable
headroom for parallel resolves without affecting memory footprint.

Co-Authored-By: Claude Opus 4.7 <[email protected]>
@webcoyote webcoyote mentioned this pull request May 28, 2026
@webcoyote

Copy link
Copy Markdown
Owner

I've created a copy of your PR (#49) because I can't push my changes to your branch. I've added quite a lot of changes because I can't quite get things to work on my computer. Would you like to look over that PR and incorporate any changes you'd like into your PR (it's fine if you want to squash out my commits or whatever, I'm not concerned about authorship).

Some of the issues I corrected:

  • listening on 0.0.0.0 is kinda scary, but unfortunately has to be done, so I added the limitation that connections can only be accepted from 127.0.0.1/192.168.2.x
  • fixed case-sensitive domain-name matching
  • fixed multi-IP bridged IP
  • fixed POSIX issues when using BSD vs GNU tools in path
  • the firewall didn't seem to work with the "legacy" path (default unnamed VM instance) so I combined the legacy-unnamed and named-VM paths
  • I removed the special-case "exec" path for calling ssh to simplify the code. This led to discovering that the environment shell scripts can leave a bogus exit code, so I fixed the guest/home/.z* files.

Unfortunately, I tried, but it's just not working yet :(

Thanks for your help!

@zachary-s-wiens

Copy link
Copy Markdown
Author

Hey, I'll have time next week to look in detail. I have a few things going on in the install-extra.sh on my end too, might be something there.

What is breaking for you?

@webcoyote

Copy link
Copy Markdown
Owner

I think I fixed all the issues, and I've pushed updates to the branch. I've done local testing, would you do me the favor of testing on your computer and see if you can spot any issues? Thanks!

@webcoyote

webcoyote commented Jun 5, 2026

Copy link
Copy Markdown
Owner

Don't know if you saw this @zachary-s-wiens:

I think I fixed all the issues, and I've pushed updates to the branch. I've done local testing, would you do me the favor of testing on your computer and see if you can spot any issues? Thanks!

The changes are in #49

@zachary-s-wiens

zachary-s-wiens commented Jun 5, 2026 via email

Copy link
Copy Markdown
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants