diff --git a/apps/web/content/docs/modules/functions.mdx b/apps/web/content/docs/modules/functions.mdx index 4536e82..e4318ce 100644 --- a/apps/web/content/docs/modules/functions.mdx +++ b/apps/web/content/docs/modules/functions.mdx @@ -1,10 +1,12 @@ --- title: Functions description: Admin-defined server-side JavaScript with grpcSdk access. -agent_summary: "Admin-trusted JS — webhooks, event bus handlers, admin routes; grpcSdk access; not a substitute for database custom endpoints." +agent_summary: "Admin-trusted JS — request/webhook/socket/middleware/event/cron types; VM timeout + require allowlist; routes on Client API via router; CRUD on Admin API." --- -The **functions** module runs **admin-defined JavaScript** in-process with full `grpcSdk` access. Treat function code like server configuration — admin-trusted, not user-supplied. +import FunctionsDeepDive from "@/mdx/deep-dives/functions-deep-dive.mdx"; + +The **functions** module runs **admin-defined JavaScript** in-process with full `grpcSdk` access. Operators upload function definitions through the Admin API; HTTP and socket handlers register on the **Client API** (via router), while function CRUD and execution history live on the **Admin API**. Treat function code like server configuration — admin-trusted, not user-supplied. ## Use cases @@ -12,15 +14,23 @@ The **functions** module runs **admin-defined JavaScript** in-process with full items={[ { title: "Webhooks", - outcome: "Receive Stripe/GitHub callbacks and call grpcSdk.database or grpcSdk.email", + outcome: "Receive Stripe/GitHub callbacks at /hook/{name} and call grpcSdk.database or grpcSdk.email", + }, + { + title: "Custom Client API routes", + outcome: "Authenticated request handlers at /{name} with declared params and returns", }, { title: "Event automation", - outcome: "React to bus events — send notifications when records change", + outcome: "React to Redis bus events — send notifications when records change", + }, + { + title: "Scheduled jobs", + outcome: "Cron functions run on a BullMQ schedule (Redis-backed)", }, { - title: "Admin-only routes", - outcome: "Custom HTTP handlers registered on Admin API surface", + title: "Socket handlers", + outcome: "Custom Socket.io event handlers on a dedicated namespace path", }, { title: "Cross-module orchestration", @@ -33,72 +43,100 @@ The **functions** module runs **admin-defined JavaScript** in-process with full -## Example: Webhook handler concept +## Example: Webhook handler + ## How it works -### Trust boundary + + -Functions code is **admin-trusted** — equivalent to a custom module deployed by operators. It is not exposed to end users as arbitrary script upload. For user-facing filtered queries, use [database custom endpoints](/docs/modules/database) at `/database/function/{name}` on the Client API instead. +## Configure -### grpcSdk access +| Key | Default | Meaning | +|-----|---------|---------| +| `active` | `true` | Module serves and registers routes when router is up | -Inside a function, `grpcSdk` provides typed clients for database, authentication, storage, chat, communications, and other registered modules — same as custom ManagedModule services. +Define functions via Admin panel or Admin API when the functions module is enabled. Each document stores `name`, `functionType`, `functionCode`, `inputs`, optional `returns`, and `timeout`. -### When to use Functions vs custom modules +On create/update/delete the module publishes to the `functions` bus channel and **refreshes router registrations**. -| Need | Use | -|------|-----| -| Filtered Client API query | Database custom endpoint | -| Long-running domain service | Custom ManagedModule | -| Webhook, cron, one-off admin automation | Functions | -| System chat message from backend | grpcSdk.chat from Function or custom module | - +## Client API -## Configure +Functions register routes on the router (not Admin API): -Define functions via Admin panel or MCP when the functions module is enabled. Each function specifies trigger type (HTTP route, bus event), source code, and enabled flag. +| Type | Example path | Notes | +|------|--------------|-------| +| `request` | `PUT /myHandler` | When `inputs.method` is `PUT` → UPDATE action | +| `webhook` | `POST /hook/stripeWebhook` | Typically unauthenticated | +| `socket` | Namespace `/{name}` | Event from `inputs.event` | -## Client API +App runtime code calls these URLs on `CLIENT_BASE_URL` like any other Client API route. **`request` functions with `inputs.auth: true` require a bearer token.** + +Functions do **not** replace `/database/function/{name}` — those are database custom endpoints. + +## Admin API + +Function management on `ADMIN_BASE_URL/functions/...`: -Functions do **not** replace Client API routes for app runtime. App code calls `/database/function/{name}` for provisioned queries — those are database custom endpoints, not this module. +| Method | Path | Purpose | +|--------|------|---------| +| POST | `/functions/upload` | Create function | +| GET | `/functions/` | List (`search`, `skip`, `limit`, `sort`) | +| GET | `/functions/:id` | Get function | +| PATCH | `/functions/:id` | Update code, inputs, returns, timeout | +| DELETE | `/functions/:id` | Delete function | +| DELETE | `/functions/` | Bulk delete (`ids[]` query) | +| GET | `/functions/list/executions` | All execution logs | +| GET | `/functions/executions/:functionId` | Per-function executions (`success` filter) | ## MCP -Enable functions module in deployment. Admin tools manage function CRUD alongside other admin routes. +Enable with `?modules=functions` in your MCP server URL. Admin tools manage function CRUD; uploaded functions appear on the Client API after router refresh. diff --git a/apps/web/content/docs/modules/router.mdx b/apps/web/content/docs/modules/router.mdx index f78d726..f3a6a64 100644 --- a/apps/web/content/docs/modules/router.mdx +++ b/apps/web/content/docs/modules/router.mdx @@ -1,10 +1,12 @@ --- title: Router description: Client API gateway — REST, GraphQL, and WebSockets. -agent_summary: "Hermes gateway on CLIENT_HTTP_PORT 3000 and CLIENT_SOCKET_PORT 3001; aggregates module client routes; Admin API on core :3030." +agent_summary: "Hermes gateway on CLIENT_HTTP_PORT 3000 and CLIENT_SOCKET_PORT 3001; proxies module client routes over gRPC; Admin API on core :3030; transports rest/graphql/sockets only." --- -The **router** module (Hermes) exposes the **Client API** by aggregating routes registered by other modules. It is the single entry point applications connect to. +import RouterDeepDive from "@/mdx/deep-dives/router-deep-dive.mdx"; + +The **router** module (Hermes) is the **Client API gateway**. Every module that serves application traffic registers routes with router over gRPC; Hermes terminates HTTP, GraphQL, and Socket.io on the client ports and **proxies** each request to the owning module handler. The **Admin API** lives on **core** (`:3030`) — router does not expose admin routes on the client port. ## Use cases @@ -16,7 +18,7 @@ The **router** module (Hermes) exposes the **Client API** by aggregating routes }, { title: "Realtime apps", - outcome: "REST on :3000, Socket.io on :3001 with shared auth", + outcome: "REST on :3000, Socket.io on :3001 with shared auth middleware", }, { title: "GraphQL queries", @@ -26,6 +28,10 @@ The **router** module (Hermes) exposes the **Client API** by aggregating routes title: "Client credentials", outcome: "Router validates client id/secret context for auth flows", }, + { + title: "Public hook URLs", + outcome: "hostUrl config builds correct /hook/... links in emails and deep links", + }, ]} /> @@ -33,12 +39,15 @@ The **router** module (Hermes) exposes the **Client API** by aggregating routes @@ -47,48 +56,45 @@ The **router** module (Hermes) exposes the **Client API** by aggregating routes ## How it works -### Port layout - -| Port | Env var | Protocol | -|------|---------|----------| -| 3000 | `CLIENT_HTTP_PORT` | REST, GraphQL | -| 3001 | `CLIENT_SOCKET_PORT` | WebSockets / Socket.io | -| 3030 | Admin | Admin API (core module) | - -### Request flow + + -``` -App → Router (Hermes) → Module client routes - → authMiddleware, client credentials - → database, storage, chat, … -``` +## Configure -Each module registers its Client API routes with Hermes at startup. The database module is required by router and most feature modules. +Router requires the [database](/docs/modules/database) module. Enable feature modules in deployment; their Client routes appear automatically when they register with router. -### What router is not +Patch via MCP `patch_config_router` (`?modules=router`): -Router does **not** replace [database custom endpoints](/docs/modules/database) — filtered queries still need provisioned `/database/function/{name}` endpoints. Router also does not expose Admin API routes; use `:3030` or MCP for provisioning. - - -## Configure +| Key | Default | Meaning | +|-----|---------|---------| +| `hostUrl` | `http://localhost:3000` | Public Client API base for generated links | +| `transports.rest` | `true` | Enable REST | +| `transports.graphql` | `true` | Enable GraphQL | +| `transports.sockets` | `true` | Enable Socket.io | +| `cors.enabled` | `true` | CORS middleware | +| `cors.origin` | `*` | Allowed origin(s), comma-separated | +| `security.clientValidation` | `false` | Require registered client credentials on auth routes | +| `captcha.enabled` | `false` | Captcha on applicable routes | +| `rateLimit.maxRequests` | `50` | Max requests per interval per user | +| `rateLimit.resetInterval` | `1` | Reset interval in seconds | -Router is a core dependency — enable other modules in deployment; their Client routes appear automatically. Environment variables: +Environment variables: | Variable | Default | Meaning | |----------|---------|---------| -| `CLIENT_HTTP_PORT` | 3000 | REST/GraphQL port | -| `CLIENT_SOCKET_PORT` | 3001 | WebSocket port | +| `CLIENT_HTTP_PORT` | `3000` | REST/GraphQL port | +| `CLIENT_SOCKET_PORT` | `3001` | WebSocket port | See [Environment variables](/docs/reference/env-vars) and [Architecture](/docs/learn/architecture). @@ -96,20 +102,40 @@ See [Environment variables](/docs/reference/env-vars) and [Architecture](/docs/l All module Client paths are relative to `CLIENT_BASE_URL`: -- `/authentication/*` -- `/database/*` -- `/storage/*` -- `/chat/*` -- `/authorization/*` -- `/graphql` +| Prefix | Module | +|--------|--------| +| `/authentication/*` | Authentication | +| `/database/*` | Database | +| `/storage/*` | Storage | +| `/chat/*` | Chat | +| `/authorization/*` | Authorization | +| `/hook/*` | Functions webhooks (and module hooks) | +| `/{functionName}` | Functions `request` type | +| `/graphql` | GraphQL (when enabled) | + +## Admin API + +Operator routes registered by the router module on `ADMIN_BASE_URL`: + +| Method | Path | Purpose | +|--------|------|---------| +| GET | `/routes` | List all registered client routes by module | +| GET | `/router/middlewares` | List available middleware handlers | +| GET | `/router/route-middlewares` | Query `path` + `action` → middleware chain | +| PATCH | `/router/patch-middleware` | Patch middleware on a client route | +| POST | `/security/client` | Create security client (platform + secret) | +| GET | `/security/client` | List security clients | +| PATCH | `/security/client/:id` | Update security client | +| DELETE | `/security/client/:id` | Delete security client | ## MCP -Router has no separate MCP module — configure via core and enable feature modules (`?modules=database,authentication,…`). +Enable router in deployment for `patch_config_router` and route inspection tools. Feature modules are enabled separately (`?modules=database,authentication,…`). diff --git a/apps/web/src/mdx/deep-dives/functions-deep-dive.mdx b/apps/web/src/mdx/deep-dives/functions-deep-dive.mdx new file mode 100644 index 0000000..057f862 --- /dev/null +++ b/apps/web/src/mdx/deep-dives/functions-deep-dive.mdx @@ -0,0 +1,77 @@ +### Trust boundary + +Function code is **admin-trusted** — equivalent to deploying code to the server. It runs in the **same Node.js process** with full `grpcSdk` privileges. The VM layer applies a per-invocation **timeout** and limits **`require`** to `lodash` and `axios` only. These are guardrails against accidents, **not** a security boundary for malicious code. Do not use this module for tenant-submitted scripts. + +For user-facing filtered queries, use [database custom endpoints](/docs/modules/database) at `/database/function/{name}` on the Client API instead. + +### Handler shape + +Stored `functionCode` is the **body** of: + +```js +function (grpcSdk, req, res) { + // your code +} +``` + +| Parameter | Meaning | +|-----------|---------| +| `grpcSdk` | Conduit gRPC SDK — database, auth, storage, chat, communications, … | +| `req` | Parsed request: HTTP params/body/query for routes; event payload for bus/cron | +| `res` | Callback `(data) => void` — return value mapped through `returns` schema | + +`console.log` output is captured in execution logs. + +### Function types + +| `functionType` | Trigger | Client API surface | +|----------------|---------|-------------------| +| `request` | HTTP route | `/{name}` (+ optional `/:urlParams`) | +| `webhook` | HTTP route (no auth middleware by default) | `/hook/{name}` | +| `middleware` | Global middleware | Registered as named middleware on `/` | +| `socket` | Socket.io event | Namespace path `/{name}`, event from `inputs.event` | +| `event` | Redis bus subscription | `inputs.event` = bus channel name | +| `cron` | BullMQ repeatable job | `inputs.event` = cron expression | + +The module waits for **router** to be serving before registering routes (`watch: router rising`). + +### HTTP methods and PUT mapping + +`inputs.method` accepts `GET`, `POST`, `PUT`, `PATCH`, or `DELETE`. **`PUT` maps to `ConduitRouteActions.UPDATE`** — the same route action Hermes uses for update verbs (not GET). This matters when declaring `returns` and when matching routes in middleware patches. + +Optional `inputs.auth: true` on `request` functions adds `authMiddleware`. + +Declare `inputs.bodyParams`, `inputs.queryParams`, and `inputs.urlParams` to validate incoming data; `urlParams` append `/:key` segments to the route path. + +### VM sandbox + +Each invocation runs inside `node:vm`: + +- **Timeout** — per-function `timeout` in milliseconds (default **180000** / 3 minutes) +- **require allowlist** — only `lodash` (`lodash-es`) and `axios`; anything else throws at runtime +- **Single run** — compile + invoke share one timeout budget + +Syntax is validated on upload (`POST /functions/upload`) and update (`PATCH /functions/:id`). + +### Event functions + +`functionType: event` subscribes to `grpcSdk.bus` on channel `inputs.event`. Payloads are passed as `req` (JSON-parsed when the bus message is a JSON string). Handler errors are logged; they do not block the publisher. + +### Cron functions (BullMQ) + +`functionType: cron` stores a **cron expression** in `inputs.event` (for example `0 */6 * * *`). The functions module registers a **BullMQ** repeatable job on Redis — the same job-queue infrastructure used by database and communications modules. On each tick the handler runs with an empty or minimal `req` payload. + +Cron requires the platform **Redis** connection used by other BullMQ workers (`grpcSdk.redisManager`) — there is no separate `FUNCTIONS_REDIS_*` configuration. Execution history is recorded like other function types. + +### Returns mapping + +If `returns` is omitted, the handler result is wrapped as `{ result: data }`. When `returns` is a schema object, only declared keys are forwarded from the `res(...)` payload. Set `returns: "String"` for a plain string result. + +### When to use Functions vs alternatives + +| Need | Use | +|------|-----| +| Filtered Client API query | Database custom endpoint | +| Long-running domain service | Custom ManagedModule | +| Webhook, cron, bus automation | Functions | +| System chat message from backend | grpcSdk.chat from Function or custom module | diff --git a/apps/web/src/mdx/deep-dives/router-deep-dive.mdx b/apps/web/src/mdx/deep-dives/router-deep-dive.mdx new file mode 100644 index 0000000..50073be --- /dev/null +++ b/apps/web/src/mdx/deep-dives/router-deep-dive.mdx @@ -0,0 +1,75 @@ +### Client vs Admin surfaces + +| Surface | Host | Consumers | What router does | +|---------|------|-----------|------------------| +| **Client API** | `CLIENT_BASE_URL` (`:3000` REST/GraphQL) | Apps, mobile, user-scoped server routes | Proxies registered module client routes | +| **Client sockets** | `SOCKET_BASE_URL` (`:3001`) | Socket.io clients | Proxies module socket namespaces and events | +| **Admin API** | `ADMIN_BASE_URL` (`:3030`, core) | Admin panel, MCP, CI | Router module registers **operator** routes only (`/router/*`, `/routes`, `/security/client`) | + +Hermes powers both the router (client) and the admin package (admin). End-user apps must never call `:3030`. See [Client vs Admin API](/docs/learn/client-vs-admin-api). + +### Port layout + +| Port | Env var | Protocol | +|------|---------|----------| +| 3000 | `CLIENT_HTTP_PORT` | REST, GraphQL, Swagger | +| 3001 | `CLIENT_SOCKET_PORT` | WebSockets / Socket.io | +| 3030 | Admin (core) | Admin API — not router | + +### Request proxy flow + +```text +App → Router (Hermes) → global middleware (CORS, rate limit, client validation, …) + → route middleware (authMiddleware, captcha, …) + → gRPC call to registering module handler + → database, storage, chat, functions, … +``` + +At startup (and on HA recovery), each module calls `registerConduitRoute` over gRPC. Router stores route metadata, mounts paths in Hermes, and publishes route state to the Redis bus so peer router replicas stay in sync. + +### Transports + +Router config toggles three transports — all default to `true`: + +| Key | Default | Effect | +|-----|---------|--------| +| `transports.rest` | `true` | REST + Swagger | +| `transports.graphql` | `true` | `/graphql` | +| `transports.sockets` | `true` | Socket.io on socket port | + +There is **no** `transports.proxy` flag. Older deployments that referenced a proxy transport should use `rest`, `graphql`, and `sockets` instead. + +### WebSockets + +Socket.io listens on `CLIENT_SOCKET_PORT` with **`path: /realtime`**. Each module registers a namespace (for example `/chat/`). Clients connect with bearer auth in headers: + +```typescript +import { io } from "socket.io-client"; + +const socket = io(`${SOCKET_BASE_URL}/chat/`, { + path: "/realtime", + extraHeaders: { authorization: `Bearer ${accessToken}` }, +}); +``` + +Modules push events to connected clients via router's `socketPush` gRPC method (rooms, receivers, namespace). + +### Global middleware + +When at least one transport is enabled, router registers: + +- **rateLimiter** — per-user request cap (`rateLimit.maxRequests` per `rateLimit.resetInterval` seconds) +- **corsMiddleware** — configurable origins, methods, credentials +- **helmetMiddleware** — security headers (relaxed for `/graphql`, `/swagger`, `/reference` GET) +- **clientMiddleware** — optional client id/secret validation when `security.clientValidation` is `true` +- **captchaMiddleware** — optional reCAPTCHA / hCaptcha / Turnstile when `captcha.enabled` is `true` + +Per-route middleware (for example `authMiddleware`) is declared by each module and can be patched at runtime via Admin API. + +### Public base URL (`hostUrl`) + +`hostUrl` defaults to `http://localhost:{CLIENT_HTTP_PORT}`. Set it via `patch_config_router` to your public Client API origin so modules can embed correct absolute URLs — for example [chat invitation hooks](/docs/modules/chat) at `{hostUrl}/hook/chat/invitations/accept/{token}`. + +### What router is not + +Router does **not** replace [database custom endpoints](/docs/modules/database) — filtered queries still need provisioned `/database/function/{name}` endpoints. Router does **not** serve the Admin API on port 3000; provisioning uses `:3030` or MCP.