Skip to content

Commit 34ceccc

Browse files
mattj-monadclaude
andcommitted
feat(embed): add the TypeScript router + conformance suite
Adds @monad-inc/embed-server — a zero-runtime-dependency /embed router that mounts into Express or bare Node http via createEmbedRouter (core logic + node adapter + Monad API adapter) — and the language-agnostic conformance suite that proves any backend satisfies the contract over HTTP: - conformance/ — Schemathesis response-conformance across all 9 routes plus scripted lifecycle scenarios (mint -> build -> status -> disable -> remove, and egress), driven against a stateful in-memory mock Monad. The harness boots the router under test as a subprocess (ROUTER=ts|go|python); only the ts server ships here — go/python servers land with their router layers. - A CI "Conformance (ts)" job builds the router and runs ROUTER=ts. From here, conformance is the required gate every later router PR must pass. Also ignores the Python harness .venv in prettier and formats two source files that weren't prettier-clean. Local: TS router 13 tests + conformance 14 checks green (ROUTER=ts); full typecheck / lint / format / build clean. Layer 3 (PRO-402), stacked on the browser client (PRO-401). Co-Authored-By: Claude Opus 4.8 <[email protected]> Claude-Session: https://claude.ai/code/session_011ASKA2VrufrNLGGrcFjY7n
1 parent 2cd9b48 commit 34ceccc

23 files changed

Lines changed: 1879 additions & 1 deletion

.github/workflows/ci.yml

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,3 +44,35 @@ jobs:
4444
- run: pnpm run typecheck
4545
- run: pnpm run test
4646
- run: pnpm run build
47+
48+
conformance:
49+
name: Conformance (ts)
50+
runs-on: ubuntu-latest
51+
steps:
52+
- uses: actions/checkout@v4
53+
54+
- uses: pnpm/action-setup@v4
55+
56+
- uses: actions/setup-node@v4
57+
with:
58+
node-version: 22
59+
cache: pnpm
60+
61+
- run: pnpm install --frozen-lockfile
62+
63+
- name: Build the TS router
64+
run: pnpm -C routers/typescript build
65+
66+
- uses: astral-sh/setup-uv@v5
67+
68+
- name: Set up the conformance harness
69+
working-directory: conformance
70+
run: |
71+
uv venv --python 3.10 .venv
72+
uv pip install --python .venv/bin/python schemathesis pytest httpx
73+
74+
- name: Run conformance against the TS router
75+
working-directory: conformance
76+
env:
77+
ROUTER: ts
78+
run: .venv/bin/python -m pytest -q

.prettierignore

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,4 +5,9 @@ pnpm-lock.yaml
55
CHANGELOG.md
66
LICENSE
77
NOTICE
8-
.github/CODEOWNERS
8+
.github/CODEOWNERS
9+
10+
# Python harness tooling (conformance venv + caches) — not ours to format
11+
**/.venv
12+
**/.pytest_cache
13+
**/__pycache__

conformance/.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
.venv/
2+
__pycache__/
3+
.pytest_cache/
4+
.hypothesis/

conformance/README.md

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# `/embed` conformance suite
2+
3+
Proves that every backend router behaves identically and matches the
4+
[`/embed` contract](../packages/embed/openapi/embed.openapi.yaml). It is
5+
**language-agnostic** — it boots a router under test and drives it over HTTP, so
6+
the same suite runs against the TypeScript, Go, and Python routers (and any
7+
future one).
8+
9+
## How it works
10+
11+
```
12+
mock_monad.py ── a stateful in-memory stand-in for the Monad API
13+
14+
│ MONAD_API_BASE
15+
router under test ── booted by conftest.py (ROUTER=python|go|ts), mounted at /embed
16+
17+
│ HTTP
18+
tests ── test_conformance.py (Schemathesis) + test_scenarios.py (lifecycle)
19+
```
20+
21+
- **`test_conformance.py`** — Schemathesis reads the OpenAPI spec, generates
22+
requests for every operation, and asserts each response matches the documented
23+
**status code, response schema, and content type**.
24+
- **`test_scenarios.py`** — the stateful lifecycle Schemathesis's stateless
25+
fuzzing can't cover: mint → build ingress → status → disable → status →
26+
remove → status, plus egress.
27+
28+
The router is booted with a stubbed `getCustomerOrgID` (always the tenant
29+
`org_conf`) and provisioning (`destinationOutputId=out_store`, `sourceInputId=in_source`),
30+
pointed at the mock. Nothing touches real Monad.
31+
32+
## Run
33+
34+
```sh
35+
# one-time: create the harness venv (installs Schemathesis + the Python router)
36+
uv venv --python 3.10 .venv
37+
uv pip install --python .venv/bin/python schemathesis pytest httpx fastapi uvicorn -e ../routers/python
38+
39+
./run.sh # all routers: python go ts
40+
./run.sh python go # a subset
41+
ROUTER=ts .venv/bin/python -m pytest -q # a single router directly
42+
```
43+
44+
Prerequisites per router: **python** — none (uses the harness venv); **go**
45+
a Go toolchain (`go run ./cmd/conformance`); **ts** — the built package
46+
(`run.sh` runs `pnpm -C packages/embed build` automatically).
47+
48+
## Scope note — auth and negative input
49+
50+
The suite runs Schemathesis's **response-conformance** checks
51+
(`status_code_conformance`, `response_schema_conformance`,
52+
`content_type_conformance`). It deliberately does **not** run the
53+
`ignored_auth` or `negative_data_rejection` checks: auth is the host's
54+
responsibility — each router mounts _behind_ the host's auth middleware and
55+
trusts `getCustomerOrgID`, which the harness stubs — so those checks would test the
56+
stub, not the router. A `5xx` is a documented, conformant outcome here
57+
(`500 internal_error`, `502 upstream_error`).

conformance/conftest.py

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
"""Conformance harness fixtures.
2+
3+
Boots the stateful mock Monad plus the router under test (selected by the
4+
``ROUTER`` env var: ``python`` | ``go`` | ``ts``), pointed at the mock, then
5+
yields for the tests. Language-agnostic: every test drives the router over HTTP.
6+
"""
7+
8+
from __future__ import annotations
9+
10+
import os
11+
import signal
12+
import subprocess
13+
import time
14+
import urllib.request
15+
16+
import pytest
17+
18+
import mock_monad
19+
20+
ROUTER = os.environ.get("ROUTER", "python")
21+
MOCK_PORT = int(os.environ.get("MOCK_PORT", "8790"))
22+
ROUTER_PORT = int(os.environ.get("ROUTER_PORT", "8791"))
23+
24+
_HERE = os.path.dirname(os.path.abspath(__file__))
25+
_REPO = os.path.abspath(os.path.join(_HERE, ".."))
26+
27+
28+
def _command() -> tuple[list[str], str, dict[str, str]]:
29+
env = dict(os.environ)
30+
env["MONAD_API_BASE"] = f"http://127.0.0.1:{MOCK_PORT}"
31+
env["PORT"] = str(ROUTER_PORT)
32+
if ROUTER == "python":
33+
return [os.path.join(_HERE, ".venv", "bin", "python"), os.path.join(_HERE, "servers", "py_server.py")], _HERE, env
34+
if ROUTER == "go":
35+
return ["go", "run", "./cmd/conformance"], os.path.join(_REPO, "routers", "go"), env
36+
if ROUTER == "ts":
37+
return ["node", os.path.join(_HERE, "servers", "ts_server.mjs")], _HERE, env
38+
raise ValueError(f"unknown ROUTER={ROUTER!r} (expected python|go|ts)")
39+
40+
41+
def _wait_healthy(url: str, timeout: float = 45.0) -> bool:
42+
deadline = time.time() + timeout
43+
while time.time() < deadline:
44+
try:
45+
with urllib.request.urlopen(url, timeout=1) as resp:
46+
if resp.status < 500:
47+
return True
48+
except Exception: # noqa: BLE001 — server not up yet
49+
time.sleep(0.15)
50+
return False
51+
52+
53+
@pytest.fixture(scope="session", autouse=True)
54+
def servers():
55+
server, _state = mock_monad.start(MOCK_PORT)
56+
cmd, cwd, env = _command()
57+
# New session so we can kill the whole group (e.g. `go run` + its child binary).
58+
proc = subprocess.Popen(cmd, cwd=cwd, env=env, start_new_session=True)
59+
try:
60+
if not _wait_healthy(f"http://127.0.0.1:{ROUTER_PORT}/embed/config"):
61+
raise RuntimeError(f"router '{ROUTER}' did not become healthy on port {ROUTER_PORT}")
62+
yield
63+
finally:
64+
try:
65+
os.killpg(os.getpgid(proc.pid), signal.SIGTERM)
66+
except ProcessLookupError:
67+
pass
68+
try:
69+
proc.wait(timeout=5)
70+
except subprocess.TimeoutExpired:
71+
try:
72+
os.killpg(os.getpgid(proc.pid), signal.SIGKILL)
73+
except ProcessLookupError:
74+
pass
75+
server.shutdown()

conformance/mock_monad.py

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
"""A stateful in-memory stand-in for the Monad API.
2+
3+
The conformance harness boots a router under test pointed at this mock (via
4+
``MONAD_API_BASE``) so the routers run their real logic without touching real
5+
Monad. It is stateful enough for the lifecycle scenario: creating a pipeline
6+
stores it, listing/detail return it, PATCH updates it, DELETE removes it.
7+
"""
8+
9+
from __future__ import annotations
10+
11+
import json
12+
import re
13+
import threading
14+
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
15+
16+
17+
class _State:
18+
def __init__(self) -> None:
19+
self.lock = threading.Lock()
20+
self.pipelines: dict[str, dict] = {}
21+
self._counter = 0
22+
23+
def new_id(self, prefix: str) -> str:
24+
with self.lock:
25+
self._counter += 1
26+
return f"{prefix}_{self._counter}"
27+
28+
29+
def _make_handler(state: _State):
30+
class Handler(BaseHTTPRequestHandler):
31+
def log_message(self, *_args) -> None: # silence per-request logging
32+
pass
33+
34+
def _send(self, code: int, obj=None) -> None:
35+
body = b"" if obj is None else json.dumps(obj).encode()
36+
self.send_response(code)
37+
if obj is not None:
38+
self.send_header("Content-Type", "application/json")
39+
self.send_header("Content-Length", str(len(body)))
40+
self.end_headers()
41+
if body:
42+
self.wfile.write(body)
43+
44+
def _read(self) -> dict:
45+
n = int(self.headers.get("Content-Length") or 0)
46+
if n == 0:
47+
return {}
48+
try:
49+
return json.loads(self.rfile.read(n) or b"{}")
50+
except (ValueError, TypeError):
51+
return {}
52+
53+
def do_GET(self): # noqa: N802
54+
self._route("GET")
55+
56+
def do_POST(self): # noqa: N802
57+
self._route("POST")
58+
59+
def do_PATCH(self): # noqa: N802
60+
self._route("PATCH")
61+
62+
def do_DELETE(self): # noqa: N802
63+
self._route("DELETE")
64+
65+
def _route(self, method: str) -> None:
66+
path = self.path.split("?", 1)[0]
67+
body = self._read() if method in ("POST", "PATCH") else {}
68+
69+
if method == "POST" and path == "/v3/sessions":
70+
return self._send(200, {"session_token": "tok_conf", "expires_at": "2026-12-31T00:00:00Z"})
71+
72+
if method == "GET" and re.fullmatch(r"/v1/(inputs|outputs)", path):
73+
return self._send(200, [
74+
{"type_id": "aws-cloudtrail", "name": "AWS CloudTrail"},
75+
{"type_id": "okta-systemlog", "name": "Okta System Log"},
76+
])
77+
78+
m = re.fullmatch(r"/v1/([^/]+)/(inputs|outputs)", path)
79+
if method == "GET" and m:
80+
kind = m.group(2)
81+
return self._send(200, {kind: [{"id": "cfg_1", "type": "aws-cloudtrail", "name": "Configured"}]})
82+
83+
if method == "POST" and re.fullmatch(r"/v2/([^/]+)/outputs", path):
84+
return self._send(200, {"id": state.new_id("out")})
85+
86+
# pipelines collection (create / list) — check before status/detail
87+
if re.fullmatch(r"/v2/([^/]+)/pipelines/?", path):
88+
if method == "POST":
89+
pid = state.new_id("pipe")
90+
with state.lock:
91+
state.pipelines[pid] = {**body, "id": pid}
92+
return self._send(200, {"id": pid})
93+
if method == "GET":
94+
with state.lock:
95+
items = [{"id": pid, "enabled": bool(p.get("enabled"))} for pid, p in state.pipelines.items()]
96+
return self._send(200, items)
97+
98+
if method == "GET" and re.fullmatch(r"/v2/([^/]+)/pipelines/([^/]+)/status", path):
99+
return self._send(200, {"status": "Running"})
100+
101+
m = re.fullmatch(r"/v2/([^/]+)/pipelines/([^/]+)", path)
102+
if m:
103+
pid = m.group(2)
104+
if method == "GET":
105+
with state.lock:
106+
p = state.pipelines.get(pid)
107+
if p is None: # lenient default so unknown ids still conform
108+
p = {"name": "default", "description": "", "enabled": True, "nodes": [], "edges": []}
109+
return self._send(200, {"config": p})
110+
if method == "PATCH":
111+
with state.lock:
112+
state.pipelines[pid] = {**body, "id": pid}
113+
return self._send(200, {})
114+
if method == "DELETE":
115+
with state.lock:
116+
state.pipelines.pop(pid, None)
117+
return self._send(204)
118+
119+
if method == "DELETE" and re.fullmatch(r"/v1/([^/]+)/(inputs|outputs)/([^/]+)", path):
120+
return self._send(204)
121+
122+
return self._send(404, {"error": f"mock unhandled {method} {path}"})
123+
124+
return Handler
125+
126+
127+
def start(port: int):
128+
"""Start the mock on 127.0.0.1:port in a background thread. Returns (server, state)."""
129+
state = _State()
130+
server = ThreadingHTTPServer(("127.0.0.1", port), _make_handler(state))
131+
threading.Thread(target=server.serve_forever, daemon=True).start()
132+
return server, state

conformance/run.sh

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
#!/usr/bin/env bash
2+
# Run the conformance suite against one or more routers.
3+
# ./run.sh # all: python go ts
4+
# ./run.sh python go # a subset
5+
set -uo pipefail
6+
cd "$(dirname "$0")"
7+
8+
PY=.venv/bin/python
9+
ROUTERS="${*:-python go ts}"
10+
11+
# The TS router runs from its built standalone package.
12+
if [[ " $ROUTERS " == *" ts "* ]]; then
13+
echo "=== building @monad-inc/embed-server (for ts router) ==="
14+
(cd ../routers/typescript && pnpm build >/dev/null)
15+
fi
16+
17+
rc=0
18+
for r in $ROUTERS; do
19+
echo ""
20+
echo "======================== conformance: $r ========================"
21+
if ROUTER="$r" "$PY" -m pytest -q; then
22+
echo "--- $r: PASS ---"
23+
else
24+
echo "--- $r: FAIL ---"
25+
rc=1
26+
fi
27+
done
28+
exit $rc

conformance/servers/ts_server.mjs

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
// Boot the TypeScript router for conformance testing, pointed at the mock
2+
// Monad. Imports the built standalone package (run `pnpm -C routers/typescript
3+
// build` first).
4+
import http from 'node:http';
5+
import { createEmbedRouter } from '../../routers/typescript/dist/index.js';
6+
7+
const router = createEmbedRouter({
8+
apiKey: 'conf-key',
9+
apiBase: process.env.MONAD_API_BASE,
10+
frameOrigin: 'https://app.monad.com/embed',
11+
getCustomerOrgID: () => 'org_conf',
12+
getProvisionedComponents: () => ({
13+
destinationOutputId: 'out_store',
14+
sourceInputId: 'in_source'
15+
}),
16+
catalogAllow: ['aws-cloudtrail', 'okta-systemlog']
17+
});
18+
19+
const port = Number(process.env.PORT || 8791);
20+
http.createServer(router).listen(port, '127.0.0.1');

0 commit comments

Comments
 (0)