Modelgov

Self-hosted AI policy gateway — one config file for AI budgets, model access, safety rules, routing, and usage logs.

AI features can silently burn money, leak sensitive data, and call the wrong model. Modelgov sits between your app and your model provider. Every request must declare a user, user type, and feature. Policy is checked before the model call happens.

Your app decides: is this user allowed to ask? Modelgov decides: is this AI request allowed to run?

Quick start

Run the gateway from this repo:

./setup

That one command checks prerequisites, creates local config, starts Docker, waits for readiness, and runs a real chat smoke test against the built-in demo provider. No OpenAI key, Anthropic key, Ollama model, or compose knowledge is needed for the default local stack.

Successful setup ends with:

ok smoke chat succeeded

✓ Ready — the gateway is running on the built-in demo AI (no key needed to start).
  Opening the console to connect your AI provider… (if it doesn't open, click:)

  http://localhost:5174/login?url=...&token=...

  make status · make stop

Browser setup wizard

After ./setup, open the console link printed in your terminal (it signs you in automatically on localhost). First visit launches a guided setup wizard at /setup:

Path	What it does
Quick start (recommended)	Real provider in ~2 minutes: OpenAI preset, paste one API key, balanced safety, $200/mo cap
Customize step by step	Pick use case, backend (14+ cloud providers / local Ollama), keys, limits
Just exploring (no API key)	Built-in demo AI + support chat template — no keys, no real calls

The wizard writes litellm_config.generated.yaml, saves API keys to your local .env, and restarts the model proxy for you — so real calls go live immediately, no extra command needed. If it couldn't auto-restart (no Docker socket available), it prints a pnpm modelgov reload-providers command for you to run.

To run the wizard again: open the browser console and run localStorage.removeItem('modelgov-setup-v1-complete'), then reload /setup.

After setup:

make status      # containers plus /health and /ready
make stop        # stop the local stack
make start       # start it again

Call Modelgov from your app:

import { createModelgovClient } from "@modelgov/sdk";

const ai = createModelgovClient({
  baseUrl: process.env.MODELGOV_URL!, // ./setup writes this to .env
  apiKey: process.env.MODELGOV_API_KEY!,
});

const res = await ai.chat({
  userId: "user_123",
  userType: "logged_in",
  feature: "support_chat",
  modelClass: "cheap",
  messages: [{ role: "user", content: "Help me reset my password" }],
});

Policy lives in modelgov.yaml:

features:
  support_chat:
    safety: strict
    model_class: cheap
    max_tokens: 500

budgets:
  by_user_type:
    logged_in:
      daily_usd: 0.25
      daily_requests: 50
      models: ["cheap", "standard"]

Debug a decision without spending:

pnpm modelgov explain --local \
  --userType logged_in --feature support_chat --modelClass premium

# Validate production config
pnpm modelgov validate --config modelgov.yaml --production

# Run policy regression tests
pnpm modelgov test-policy --file modelgov.policy-tests.yaml

What you get

Capability	How
Per-user / per-feature budgets	Postgres reservations with row locks
Model class routing	Primary, fallback, budget-aware degrade
Safety	PII mask/block + prompt injection (Presidio)
Usage audit trail	Every request logged with cost and decision
Typed SDK	feature / userType unions generated from your YAML

Every request is checked before it reaches OpenAI, Anthropic, Gemini, Bedrock, or any LiteLLM-supported provider.

Examples

Example	What it shows
support_chat	Chat, PII masking, injection block, daily budget
saas_tiers	Free vs paid model access
document_extraction	Structured extraction — real integration pattern
nextjs_support_chat	Next.js API route — app auth → Modelgov SDK
fastapi_support_chat	Python/FastAPI — the same pattern via the modelgov PyPI SDK
chatbot	Full Next.js chatbot app on a governed gateway
rag_support	RAG with grounded answers (context verification)
ocr_pipeline	Document/OCR extraction pipeline
event_intake_app	Server-side intake workflow with budgets

./setup
# ./setup writes the chosen API port to .env (it picks the first free port in
# 3090-3099). Pass MODELGOV_URL through so the example targets the right one.
MODELGOV_URL="$(grep '^MODELGOV_URL=' .env | cut -d= -f2)" \
MODELGOV_API_KEY=sk-modelgov-api-local \
  pnpm --filter support-chat-example start "How do I reset my password?"

Packages

Package	Role
@modelgov/policy-engine	Pure evaluateAiRequest() — no I/O
@modelgov/api	Fastify API: budgets, LiteLLM, safety
@modelgov/sdk	Typed HTTP client
@modelgov/cli	Setup, ops, config validation, and policy dry-runs
create-modelgov	Scaffold wizard

Deploy modes

Command	Intended use
./setup	One-command local setup with built-in demo provider
make start / make stop	Start or stop the default local stack
make start-cloud	Local dev with real OpenAI/Anthropic keys
make start-full	Local dev + Langfuse
make start-local	Local Ollama eval
make up-prod	Small self-hosted production (not HA)
Helm	Enterprise production — recommended

Current release: v1.7.1 — pin ghcr.io/mml555/modelgov/modelgov-api:v1.7.1 in production.

Command	Stack
./setup	First-run setup + readiness wait + smoke test
make status	Containers plus /health and /ready
make doctor	Local prerequisites and runtime health
pnpm modelgov doctor production	Production env posture check

Documentation

Doc	Audience
Docs index	Everyone
Mental model	Who owns what (start here)
Integration checklist	Add to an existing app in ~20 min
Real app pattern	Embed in a product workflow (event intake)
Getting started	First deploy + first API call
Configuration	modelgov.yaml reference
HTTP API	REST, auth, explain, idempotency
Operations	Production, health, backups
Production deploy	Official Helm path
Operator console	Self-hosted admin UI
Architecture	Engine design, budgets

Develop & test

pnpm install && pnpm build
pnpm test                 # full suite (starts Postgres via Docker when DATABASE_URL is unset)
pnpm test:coverage        # same + coverage gates (requires Postgres — same as CI)
make test-db              # explicit disposable Postgres container
pnpm verify               # build, typecheck, lint, coverage, OpenAPI export

pnpm test and pnpm verify need Postgres. CI sets DATABASE_URL; locally Docker is used automatically when it is unset. Without Docker, export DATABASE_URL or run make test-db.

License

MIT — see LICENSE. Security: SECURITY.md.