WayGate

WayGate is a modular platform for building Generation-Augmented Retrieval (GAR) workflows. It provides a plugin-based runtime for ingesting content via webhooks, storing and managing documents, driving LLM generation pipelines, and scheduling recurring jobs — all wired together through a shared configuration and plugin registry.

Monorepo Structure

apps/
  web/          — Unified FastAPI web app for UI, auth, and mounted webhook ingress
  scheduler/    — Background job runner for cron-style workflows
  worker-app/   — Single transport-agnostic worker app for workflow execution
libs/
  core/         — Shared framework: plugin system, config registry, bootstrap, logging
  webhooks/     — Mountable FastAPI webhook ingress app and OpenAPI helpers
  worker/       — Shared worker runtime helpers and JetStream consumer loop
  workflows/    — Shared workflow entrypoints executed by workers
plugins/
  local-storage/    — StoragePlugin backed by the local filesystem
  provider-ollama/  — LLMProviderPlugin backed by a local Ollama server
  communication-http/ — CommunicationClientPlugin for HTTP worker dispatch
  communication-nats/ — CommunicationClientPlugin for JetStream worker dispatch
  communication-rq/   — CommunicationClientPlugin for RQ worker dispatch
  webhook-generic/  — WebhookPlugin for generic HTTP webhook ingestion

How It Works

1. Bootstrap — call bootstrap_app() from waygate-core. It loads all installed plugins via setuptools entry points, discovers their config schemas, builds a merged Pydantic settings object (populated from environment variables), and instantiates each plugin with its config injected.

2. Configuration — all settings live under the WAYGATE_ env prefix. Core settings use WAYGATE_CORE__*; each plugin gets its own namespace derived from its plugin name (e.g. WAYGATE_LOCAL_STORAGE__BASE_PATH).

3. Plugins — plugins implement one or more pluggy hook specs defined in waygate-core. Installing a plugin package and declaring its entry point is sufficient; no code changes to the core are needed.

Quick Start

Requires Python 3.14+ and uv.

# Install all packages
uv sync --all-packages

# Run the unified web app
uv run waygate-web

# Run the scheduler
uv run waygate-scheduler

Copy env.example to .env and set values appropriate for your environment before starting.

Docker Compose

Use env.compose.example as the template for your local .env.compose file before starting either Compose workflow.

Development Compose

The default compose.yml is the local development path. It builds the images from the workspace, keeps web and worker on the default stack, and runs a one-shot migration container before application services start.

docker compose up -d

Optional development-only profiles:

• scheduler adds the scheduler process.
• switch WAYGATE_CORE__COMMUNICATION_PLUGIN_NAME to communication-rq or communication-http when you want the single worker service to listen on a different transport.

Production-Style Compose

Use compose.prod.yml when you want an explicit runtime shape selected via profiles.

NATS-backed runtime with local Ollama and Chroma:

docker compose -f compose.prod.yml --profile nats --profile chroma --profile ollama up -d web worker

RQ-backed runtime with local Ollama and Chroma:

# First set WAYGATE_CORE__COMMUNICATION_PLUGIN_NAME=communication-rq in .env.compose
docker compose -f compose.prod.yml --profile rq --profile chroma --profile ollama up -d web worker

Add --profile scheduler to either production-style command when you want the cron scheduler in the stack.

Important runtime details:

• env.compose.example defaults to communication-nats, so the nats profile is the matching production-style transport unless you change the env file.
• env.compose.example also points WAYGATE_WEB_AUTH__DEFAULT_DATABASE_URI at the Compose Postgres service so AuthTuna does not fall back to an in-container SQLite path.
• env.compose.example includes WAYGATE_OLLAMAPROVIDER__BASE_URL=http://ollama:11434, so the ollama profile is required unless you point that setting at an external provider.
• The one-shot migrate service now runs alembic upgrade head before the web app and workers start.

Docker Compose Smoke Test

Use the profiled production-style Compose stack when you want to exercise the generic webhook -> web app -> JetStream -> worker pipeline with the Ollama provider.

1. Start the profiled smoke stack.

docker compose -f compose.prod.yml --profile nats --profile chroma --profile ollama up -d db valkey chromadb nats ollama

1. Pull the models required by the local smoke test before starting the worker.

docker compose -f compose.prod.yml --profile nats --profile chroma --profile ollama exec ollama ollama pull qwen3.5:9b
docker compose -f compose.prod.yml --profile nats --profile chroma --profile ollama exec ollama ollama pull hermes3:8b

1. Start the web app and worker service. The migration service runs automatically.

docker compose -f compose.prod.yml --profile nats --profile chroma --profile ollama up -d web worker

1. Post the sample generic webhook payload.

curl -X POST http://127.0.0.1:8080/webhooks/generic-webhook \
  -H "Content-Type: application/json" \
  --data @scripts/fixtures/generic-webhook.sample.json

1. Verify the result in the bind-mounted wiki directory.

Raw webhook artifacts are written under ./wiki/raw/. Successful compile runs write published markdown under ./wiki/published/. If the workflow stops for a human decision, the review record is written under ./wiki/review/.

• The worker validates the configured compile models at startup. If you change WAYGATE_CORE__METADATA_MODEL_NAME, WAYGATE_CORE__DRAFT_MODEL_NAME, or WAYGATE_CORE__REVIEW_MODEL_NAME, pull those models into Ollama before restarting it.
• mise run test:compose:nats-smoke runs the same workflow in an isolated Compose project using the profiled production-style stack and a lightweight Ollama smoke-model profile so end-to-end verification completes faster.

Packages

Package	Description
waygate-core	Plugin system, config registry, bootstrap
waygate-web	Unified FastAPI web app
waygate-scheduler	Cron job runner
waygate-worker-app	Primary transport-agnostic worker app
waygate-webhooks	Mountable webhook ingress app
waygate-worker	Shared worker runtime helpers
waygate-plugin-local-storage	Filesystem storage plugin
waygate-plugin-provider-ollama	Ollama LLM provider plugin
waygate-plugin-communication-http	HTTP communication client plugin
waygate-plugin-communication-nats	JetStream communication client plugin
waygate-plugin-communication-rq	RQ communication client plugin
waygate-workflows	Shared workflow entrypoints
waygate-plugin-webhook-generic	Generic webhook ingestion plugin

Writing a Plugin

Implement the relevant abstract base class from waygate-core, register a waygate_plugin_config hookimpl to expose your config schema, and declare the entry point in your pyproject.toml. See any plugin under plugins/ for a working example.

Contributing

This repository uses Conventional Commits. All commit messages must follow the <type>(<scope>): <summary> format — see .github/copilot-instructions.md for the full convention used here.