diff --git a/apps/web/content/docs/deployment/kubernetes-helm.mdx b/apps/web/content/docs/deployment/kubernetes-helm.mdx index 7a17def4..f7176e1c 100644 --- a/apps/web/content/docs/deployment/kubernetes-helm.mdx +++ b/apps/web/content/docs/deployment/kubernetes-helm.mdx @@ -1,10 +1,10 @@ --- title: Kubernetes and Helm -description: Deploy Conduit with the official Helm chart. -agent_summary: "helm install conduit conduit-platform/conduit -f values.yaml from conduit-platform/conduit chart repo." +description: Deploy Conduit with the official Helm chart — probes, readiness, and production layout. +agent_summary: "helm install conduit conduit-platform/conduit; core readiness HTTP :3030/ready; liveness gRPC :5000; READY_REQUIRED_MODULES set by chart." --- -Conduit is **Kubernetes-first**. Official chart: `conduit-platform/conduit`. +Conduit is **Kubernetes-first**. Official chart: `conduit-platform/conduit` ([conduit-helm-charts](https://github.com/ConduitPlatform/conduit-helm-charts)). ```bash helm repo add conduit-platform https://conduitplatform.github.io/helm-charts/ @@ -15,9 +15,55 @@ Starter values live in the Conduit repo under `deploy/k8s/values/`. Configure ingress for: -- **Admin UI** and Admin API -- **Router** (Client API) +- **Admin UI** and Admin API (`:3030`) +- **Router** (Client API, `:3000`) For local clusters see the Conduit repo `deploy/k8s/minikube.md`. Chart defaults target dev/demo — use external DB, Redis, and observability stacks for production. -See also [conduit-helm-charts](https://github.com/ConduitPlatform/conduit-helm-charts) for advanced layouts. +## Health probes (core pod) + +The chart distinguishes **readiness** from **liveness** on the core container: + +| Probe | Mechanism | Target | Why | +|-------|-----------|--------|-----| +| **Readiness** | HTTP `GET` | `:3030/ready` | **Deep** check — core bootstrap, Redis, required modules (`READY_REQUIRED_MODULES`) | +| **Startup** | HTTP `GET` | `:3030/ready` | Gates slow module registration before liveness failures (up to ~150s by default) | +| **Liveness** | gRPC health | `:5000` | **Shallow** — process and gRPC server alive; avoids restarts during module warm-up | + +Readiness uses a **3s timeout** because `/ready` may probe Redis and module health. Liveness stays on gRPC so a temporarily unready platform (e.g. database pod still starting) does not kill core. + +Default `READY_REQUIRED_MODULES` on core: + +- `database` when router is disabled +- `database,router` when the router subchart is enabled + +Override probes and env in `values.yaml`: + +```yaml +core: + adminHttpPort: 3030 + readinessProbe: + httpGet: + path: /ready + port: 3030 + timeoutSeconds: 3 + livenessProbe: + grpc: + port: 5000 + startupProbe: + enabled: true + httpGet: + path: /ready + port: 3030 + env: + - name: READY_REQUIRED_MODULES + value: "database,router,authentication" +``` + +Feature modules (database, router, authentication, etc.) use **gRPC health** for both readiness and liveness on their own pods — only **core** uses the deep HTTP `/ready` endpoint. + +## Shallow liveness endpoint + +`GET /live` on Admin HTTP (`:3030`) returns immediately with `{ status: "alive" }`. Use it for manual smoke tests; prefer the chart's gRPC liveness probe for Kubernetes. + +See also [conduit-helm-charts](https://github.com/ConduitPlatform/conduit-helm-charts) for advanced layouts and multi-tenant experiments. diff --git a/apps/web/content/docs/getting-started/mcp-setup.mdx b/apps/web/content/docs/getting-started/mcp-setup.mdx index 52f77804..ae4223a1 100644 --- a/apps/web/content/docs/getting-started/mcp-setup.mdx +++ b/apps/web/content/docs/getting-started/mcp-setup.mdx @@ -1,7 +1,7 @@ --- title: MCP Setup description: Configure the Conduit Admin MCP server for development-time provisioning. -agent_summary: "Add ADMIN_URL/mcp?modules=... to .cursor/mcp.json with Bearer admin JWT or cdt_ token." +agent_summary: "Add ADMIN_URL/mcp?modules=... to .cursor/mcp.json with Bearer admin JWT or cdt_ token; DB import/introspect excluded from MCP." --- The Conduit **MCP server** exposes Admin API operations as tools so AI agents can provision schemas, users, and config during development. **Never call MCP from application runtime code.** @@ -14,7 +14,7 @@ The Conduit **MCP server** exposes Admin API operations as tools so AI agents ca 1. Running Conduit instance with Admin API reachable (default port **3030**) 2. MCP transport enabled: `transports.mcp: true` in admin config -3. Admin JWT or `cdt_` API token +3. Admin JWT or `cdt_` API token (recommended for automation) ## Cursor configuration @@ -44,7 +44,7 @@ On connect, only `core` and `__meta__` load by default. Enable modules via the U | Schemas, custom endpoints | `database` | | Users, teams | `authentication` | | Files | `storage` | -| ReBAC | `authorization` | +| ReBAC (incl. bulk relations) | `authorization` | | Chat | `chat` | | Email/SMS/push | `communications` (or `email,push,sms`) | @@ -52,14 +52,42 @@ After changing the URL, **reconnect MCP** and call `list_modules` to verify `loa ## Authentication -- **Admin JWT** — from admin login flow -- **API token** — create via Admin API (`cdt_` prefix, shown once) +| Method | Usage | +|--------|--------| +| **Admin JWT** | Short-lived; from admin login (`POST /login`) | +| **`cdt_` API token** | Long-lived; create via `POST /api-tokens` — **shown once**, ideal for MCP and CI | Every request: `Authorization: Bearer `. +### API tokens (`cdt_`) + +Create tokens on the Admin API (or via MCP `post_apitokens` once core tools are loaded): + +```bash +curl -X POST http://localhost:3030/api-tokens \ + -H "Authorization: Bearer " \ + -H "Content-Type: application/json" \ + -d '{"name":"cursor-mcp","expiresInDays":90}' +``` + +Response includes the plaintext `cdt_` token **once**. Store it in your MCP client config or secrets manager. List and revoke tokens with `GET /api-tokens` and `DELETE /api-tokens/:id` — values are never returned after creation. + +API tokens bypass the `masterkey` header requirement and authenticate as the creating admin. Each token is scoped to that admin's privileges. + +## Routes excluded from MCP + +Some Admin routes are marked `mcp: false` and **never** appear as MCP tools — use the Admin API or admin panel directly: + +| Area | Excluded routes | Reason | +|------|-----------------|--------| +| **Database** | `POST /database/schemas/import`, all `/database/introspection/*` | Destructive bulk schema import and DB introspection need explicit operator intent | +| **Auth bootstrap** | `POST /login`, admin user/password/2FA routes | Session and credential flows — not agent-safe | + +Schema **export** (`GET /database/schemas/export`) remains available via MCP. For GitOps-style full platform state, use `GET /state/export` / `POST /state/import` on the Admin API — see [GitOps guide](/docs/guides/gitops-state-export). + ## Tool naming -Admin routes map to tools: `GET /database/schemas` → `get_database_schemas`. +Admin routes map to tools: `GET /database/schemas` → `get_database_schemas`. See [MCP tools reference](/docs/reference/mcp-tools). ." +description: Module config via Admin API, config bus, environment variables, and readiness tuning. +agent_summary: "Modules react to config via Redis config bus; patch via Admin API or MCP patch_config_; READY_* env vars gate /ready." --- Conduit stores module configuration in core. Changes propagate over the **config bus** (Redis pub/sub). Modules implement `onConfig` lifecycle hooks. @@ -20,6 +20,22 @@ Conduit stores module configuration in core. Changes propagate over the **config Never patch production config from application runtime code. +## Readiness configuration + +Core's deep readiness probe (`GET /ready`) evaluates bootstrap state, gRPC health, Redis, and module registration. Tune behavior with `READY_*` environment variables on the **core** process: + +| Variable | Default | Purpose | +|----------|---------|---------| +| `READY_REQUIRED_MODULES` | _(empty)_ | Comma-separated module names that must be registered and serving before `/ready` returns 200 | +| `READY_CHECK_REDIS` | `true` | PING Redis as part of readiness | +| `READY_STRICT` | `false` | When `true`, optional modules that are not serving fail readiness (not just warn) | +| `READY_PROBE_MODULES_ACTIVE` | `false` | When `true`, actively gRPC-health-check each module instead of trusting registry `serving` flag | +| `READY_TOTAL_TIMEOUT_MS` | `3000` | Overall budget for the readiness evaluation | +| `READY_REDIS_TIMEOUT_MS` | `500` | Timeout for Redis PING | +| `READY_MODULE_TIMEOUT_MS` | `1000` | Per-module health probe timeout | + +Helm sets `READY_REQUIRED_MODULES` automatically on the core deployment. Override in `core.env` when running custom topologies. + ## Environment variables Every module process requires: diff --git a/apps/web/content/docs/reference/admin-api.mdx b/apps/web/content/docs/reference/admin-api.mdx index 179090d4..679993c5 100644 --- a/apps/web/content/docs/reference/admin-api.mdx +++ b/apps/web/content/docs/reference/admin-api.mdx @@ -1,7 +1,7 @@ --- title: Admin API -description: REST, GraphQL, and WebSocket admin surface on core — auth, scope, and MCP relationship. -agent_summary: "ADMIN_BASE_URL :3030; masterkey, admin JWT, or cdt_ tokens; provisioning only — not for app runtime." +description: REST, GraphQL, and WebSocket admin surface on core — auth, scope, health, and MCP relationship. +agent_summary: "ADMIN_BASE_URL :3030; masterkey, admin JWT, or cdt_ tokens; GET /ready deep, GET /live shallow; provisioning only." --- The **Admin API** runs on **core** (default `ADMIN_BASE_URL` → port **3030**). It powers the Conduit Admin Panel, CI provisioning scripts, and the [MCP server](/docs/getting-started/mcp-setup). Application code must **not** call it from user-facing paths. @@ -23,14 +23,39 @@ The **Admin API** runs on **core** (default `ADMIN_BASE_URL` → port **3030**). Response formats are **not interchangeable** between Admin and Client APIs. +## Health endpoints + +No authentication required: + +| Method | Path | Purpose | +|--------|------|---------| +| GET | `/live` | **Liveness** — shallow; `{ status: "alive", message: "..." }` | +| GET | `/ready` | **Readiness** — deep; structured report with per-check `pass`/`fail`/`warn` | + +`/ready` returns **503** when not ready (e.g. required module not registered, Redis down). Use `Accept: application/json` for the full report; legacy plain-text clients get `Conduit Core is online!` / `Conduit Core is not ready` (or `?legacy=true`). + +Configure required modules and probe behavior via `READY_*` env vars — see [Environment variables](/docs/reference/env-vars). + ## Authentication | Method | Usage | |--------|--------| | `masterkey` | Header on trusted bootstrap/ops scripts | -| Admin JWT | `Authorization: Bearer ` after admin login | +| Admin JWT | `Authorization: Bearer ` after `POST /login` | | `cdt_` API tokens | Long-lived tokens for automation and MCP clients | +### API tokens (`cdt_`) + +| Method | Path | Notes | +|--------|------|-------| +| POST | `/api-tokens` | Body: `{ name, expiresInDays? }` — returns plaintext `cdt_…` **once** | +| GET | `/api-tokens` | Lists metadata (`tokenPrefix`, `lastUsedAt`) — never the secret | +| DELETE | `/api-tokens/:id` | Revoke; admins can only delete their own tokens | + +Token format: `cdt_` + base64url random bytes. Prefix (`cdt_xxxxxxxx`) is stored for lookup; full value is bcrypt-hashed server-side. + +**v0.17 breaking change:** Client API authentication **service accounts** (`POST /authentication/service`) are removed. Migrate automation and CI from service-account credentials to `cdt_` tokens created here. See [Migration v0.16 → v0.17](/docs/resources/migration-v0.16-to-v0.17). + Never embed admin credentials in application code, browser env vars, or client bundles. ## Surfaces @@ -45,10 +70,10 @@ Interactive reference: Admin API **Swagger** (available on a running instance). ## MCP relationship -The MCP server at `{ADMIN_BASE_URL}/mcp` wraps a **subset** of Admin API operations as tools. Authentication is handled by the MCP connection — tools run with admin privileges. +The MCP server at `{ADMIN_BASE_URL}/mcp` wraps a **subset** of Admin API operations as tools. Routes with `mcp: false` (database import/introspection, login, credential management) are **excluded**. Authentication is handled by the MCP connection — tools run with admin privileges. - Call `list_modules` to see loaded modules -- Enable more tools via `/mcp?modules=authentication,database,storage` +- Enable more tools via `/mcp?modules=authentication,database,storage,authorization` - Tool naming: HTTP method + path with `/` → `_` (see [MCP tools](/docs/reference/mcp-tools)) MCP is for **development and deploy-time provisioning** only. Do not substitute MCP calls for Client API calls in application runtime. @@ -60,7 +85,9 @@ MCP is for **development and deploy-time provisioning** only. Do not substitute | Create or patch schemas, indexes, custom endpoints | Admin API or MCP | | Patch module config (auth providers, storage buckets, email templates) | Admin API or MCP | | Manage admin users, API tokens, teams (operator workflows) | Admin API or MCP | +| Bulk ReBAC tuples (`POST /authorization/relations/many`) | Admin API or MCP (`post_authorization_relations_many`) | | Export/import platform state (GitOps) | `GET /state/export`, `POST /state/import` — [GitOps guide](/docs/guides/gitops-state-export) | +| Database introspection or schema import | Admin API / panel only — **not MCP** | | End-user login, CRUD, permission checks | **Client API** only | | Application file upload/download | **Client API** with user token | diff --git a/apps/web/content/docs/reference/client-api.mdx b/apps/web/content/docs/reference/client-api.mdx index 5e50551d..84f4362a 100644 --- a/apps/web/content/docs/reference/client-api.mdx +++ b/apps/web/content/docs/reference/client-api.mdx @@ -18,6 +18,7 @@ The **Client API** runs on the **router** module (default `CLIENT_BASE_URL` → | **Consumers** | Web/mobile apps, user-scoped server routes | Admin panel, MCP, CI | | **Auth** | User access/refresh tokens | masterkey, admin JWT, `cdt_` tokens | | **Database** | `/database/{Schema}`, `/database/function/{name}` | Schema/endpoint/index admin | +| **Health** | Not exposed | `GET /live`, `GET /ready` on core | | **Realtime** | Socket.io on `:3001` (`/chat/` namespace) | Admin WebSockets where registered | ## Authentication @@ -49,6 +50,8 @@ Store tokens **server-side** (Redis vault + iron-session cookie). See [Next.js g GraphQL: `POST /graphql` when modules register types. +Admin-only database operations (schema import, introspection, bulk schema management) live on the **Admin API** and are excluded from MCP — provision schemas at deploy time. + ## Authorization | Method | Path | @@ -58,6 +61,8 @@ GraphQL: `POST /graphql` when modules register types. Pass `scope` on database **creates** for team-owned records. See [ReBAC guide](/docs/guides/rebac-team-scoping). +Bulk relation creation (`POST /authorization/relations/many` with `subject`, `relation`, `resources[]`) is **Admin API only** — use MCP `post_authorization_relations_many` at provision time, not from app runtime. + ## Storage | Method | Path | Auth | diff --git a/apps/web/content/docs/reference/env-vars.mdx b/apps/web/content/docs/reference/env-vars.mdx index 6bb903dc..8f0cef9f 100644 --- a/apps/web/content/docs/reference/env-vars.mdx +++ b/apps/web/content/docs/reference/env-vars.mdx @@ -1,7 +1,7 @@ --- title: Environment Variables -description: Core and module environment variable reference. -agent_summary: "CONDUIT_SERVER and SERVICE_URL required per module; CLIENT_HTTP_PORT 3000; ADMIN_HTTP_PORT 3030; GRPC_KEY for inter-module auth." +description: Core and module environment variable reference — ports, gRPC, and readiness tuning. +agent_summary: "CONDUIT_SERVER and SERVICE_URL required per module; READY_* on core; ADMIN_HTTP_PORT 3030; CLIENT_HTTP_PORT 3000." --- ## Every module process @@ -17,15 +17,36 @@ agent_summary: "CONDUIT_SERVER and SERVICE_URL required per module; CLIENT_HTTP_ | Variable | Default | Purpose | |----------|---------|---------| -| `ADMIN_HTTP_PORT` | `3030` | Admin API | +| `ADMIN_HTTP_PORT` | `3030` | Admin API, `/ready`, `/live`, MCP (`/mcp`) | | `CLIENT_HTTP_PORT` | `3000` | Client API (router) | | `CLIENT_SOCKET_PORT` | `3001` | Client WebSockets | | `REDIS_HOST` / `REDIS_PORT` | — | Config bus | +## Core readiness (`READY_*`) + +Set on the **core** process. Controls `GET /ready` deep health evaluation: + +| Variable | Default | Purpose | +|----------|---------|---------| +| `READY_REQUIRED_MODULES` | _(empty)_ | Comma-separated modules that must register before ready (Helm sets `database` or `database,router`) | +| `READY_CHECK_REDIS` | `true` | Include Redis PING in readiness | +| `READY_STRICT` | `false` | Fail readiness when optional modules are not serving | +| `READY_PROBE_MODULES_ACTIVE` | `false` | Actively gRPC-health-check each module | +| `READY_TOTAL_TIMEOUT_MS` | `3000` | Overall readiness evaluation budget | +| `READY_REDIS_TIMEOUT_MS` | `500` | Redis PING timeout | +| `READY_MODULE_TIMEOUT_MS` | `1000` | Per-module health probe timeout | + +Example for a minimal compose stack: + +```dotenv +READY_REQUIRED_MODULES=database,router +READY_CHECK_REDIS=true +``` + ## Database module | Variable | Purpose | |----------|---------| | `DB_CONN_URI` | MongoDB or PostgreSQL connection string | -See deployment guides for compose and Helm-specific overrides. +See [Kubernetes and Helm](/docs/deployment/kubernetes-helm) for probe and `READY_REQUIRED_MODULES` defaults in the official chart, and deployment guides for compose-specific overrides. diff --git a/apps/web/content/docs/reference/mcp-tools.mdx b/apps/web/content/docs/reference/mcp-tools.mdx index 24e95bbd..c53e1316 100644 --- a/apps/web/content/docs/reference/mcp-tools.mdx +++ b/apps/web/content/docs/reference/mcp-tools.mdx @@ -1,13 +1,15 @@ --- title: MCP Tools -description: How Admin API routes map to MCP tool names. -agent_summary: "Pattern: {method}_{path_with_underscores}; list_modules always available; enable modules via ?modules= URL." +description: How Admin API routes map to MCP tool names — modules, exclusions, and bulk operations. +agent_summary: "Pattern: {method}_{path_with_underscores}; list_modules always available; DB import/introspect excluded; authz bulk via post_authorization_relations_many." --- ## Discovery Call **`list_modules`** on every new MCP connection. Tools for unloaded modules are unavailable until you add `?modules=` to the MCP URL and reconnect. +Only **`core`** and **`__meta__`** load by default. The `core` module includes config tools, API token management, and health (`get_ready`, `get_live`). + ## Naming pattern Admin routes convert to tools: @@ -16,6 +18,7 @@ Admin routes convert to tools: 2. Replace `/` with `_` 3. Lowercase 4. Prepend HTTP method: `get_`, `post_`, `patch_`, `put_`, `delete_` +5. Remove hyphens and other non-alphanumeric characters from path segments (e.g. `/api-tokens` → `apitokens`, not `api_tokens`) | Admin route | MCP tool | |-------------|----------| @@ -23,6 +26,12 @@ Admin routes convert to tools: | `POST /database/schemas` | `post_database_schemas` | | `PATCH /config/database` | `patch_config_database` | | `GET /ready` | `get_ready` | +| `GET /live` | `get_live` | +| `POST /api-tokens` | `post_apitokens` | +| `GET /api-tokens` | `get_apitokens` | +| `DELETE /api-tokens/:id` | `delete_apitokens_id` | +| `POST /authorization/relations/many` | `post_authorization_relations_many` | +| `POST /authorization/indexer/reconstruct` | `post_authorization_indexer_reconstruct` | ## Config tools @@ -31,6 +40,36 @@ When a module is loaded: - `get_config_{module}` - `patch_config_{module}` +Global: `get_config` returns all module configs. + +## Authorization bulk tools + +With `?modules=authorization` loaded: + +| Tool | Admin route | Body | +|------|-------------|------| +| `post_authorization_relations` | `POST /authorization/relations` | Single tuple: `subject`, `relation`, `resource` | +| `post_authorization_relations_many` | `POST /authorization/relations/many` | Bulk: `subject`, `relation`, `resources[]` (async index jobs per tuple) | +| `post_authorization_indexer_reconstruct` | `POST /authorization/indexer/reconstruct` | Optional `{ soft: true }` — rebuild ReBAC indexes after corruption or bulk import | + +Prefer `post_authorization_relations_many` when seeding many tuples at provision time. Validate `subject`/`relation` against the `ResourceDefinition`; keep batches ≤100 resources when possible. + +## Excluded routes (`mcp: false`) + +These Admin routes **never** register as MCP tools. Call the Admin API or admin panel directly: + +| Module | Routes | Why excluded | +|--------|--------|--------------| +| **database** | `POST /database/schemas/import` | Bulk schema import — operator intent required | +| **database** | `GET/POST /database/introspection`, `GET /database/introspection/schemas`, `POST /database/introspection/schemas/finalize` | DB introspection mutates pending schema state | +| **core** | `POST /login`, admin password/2FA/user bootstrap routes | Credential and session flows | + +`GET /database/schemas/export` **is** available as `get_database_schemas_export`. + ## Resources -MCP exposes `conduit://docs/*` resources including API guides. See [MCP setup](/docs/getting-started/mcp-setup). +MCP exposes `conduit://docs/*` resources including Admin and Client API OpenAPI specs and an API usage guide. See [MCP setup](/docs/getting-started/mcp-setup). + +## Module aliases + +`?modules=communications` expands to `email`, `push`, and `sms` module tools.