Skip to content

gateway-architecture.md

Latest commit

 

History

History
49 lines (35 loc) · 1.31 KB

gateway-architecture.md

File metadata and controls

49 lines (35 loc) · 1.31 KB
Raw

Gateway Architecture

Agentic Provider Gateway is the lowest model-provider layer for agentic applications.

It is not an agent framework. It does not decide goals, tasks, memory, or workflows. It gives those higher layers a stable way to call models across providers.

Main Flow

  1. Application builds a gateway-native request.
  2. Compiler validates input/output/task/capability shape.
  3. Router finds provider lanes that can satisfy the model and capabilities.
  4. Rate limiter inspects and reserves a lane.
  5. Provider runtime converts the request to provider-native API payloads.
  6. Provider response is normalized into gateway output parts or stream events.
  7. Usage and trace receipts are recorded.

Boundaries

The router does not know Gemini, Groq, or OpenAI-specific request details.

Provider runtimes own:

  • API client behavior
  • model id mapping
  • provider-native request schemas
  • response parsing
  • streaming chunks
  • rate-limit policy
  • provider-native tools
  • model sync

The gateway core owns:

  • contracts
  • request validation
  • routing
  • trace receipts
  • provider registry loading
  • capability matching

Runtime Storage

Runtime state defaults to .apg/ in the current working directory:

  • .apg/gateway_traces.sqlite3
  • .apg/gateway_state/*.json

Set APG_STORAGE_DIR to place runtime state elsewhere.