|
1 | 1 | # @quantum-l9/llm-router |
2 | 2 |
|
3 | | -> The shared intelligence routing layer for all L9 bots. One module, all models, zero waste. |
| 3 | +Shared deterministic model routing, budget enforcement, provider resilience, and visual QA for L9 bots. |
4 | 4 |
|
5 | | -## What This Is |
6 | | - |
7 | | -A standalone, reusable TypeScript module that any L9 bot imports to get optimal model selection, budget enforcement, and multi-provider routing — without each bot implementing its own LLM integration. |
8 | | - |
9 | | -```typescript |
10 | | -import { L9LLMRouter, TaskType, TaskComplexity } from '@quantum-l9/llm-router'; |
| 5 | +```ts |
| 6 | +import { L9LLMRouter, TaskComplexity, TaskType } from '@quantum-l9/llm-router'; |
11 | 7 |
|
12 | 8 | const router = new L9LLMRouter({ |
13 | 9 | perplexityApiKey: process.env.PERPLEXITY_API_KEY!, |
14 | 10 | openrouterApiKey: process.env.OPENROUTER_API_KEY!, |
15 | | - appName: 'L9-SEO-Bot', |
16 | 11 | }); |
17 | | - |
18 | | -router.initClient('safehavenrr', { monthlyBudgetPerClient: 200 }); |
19 | | - |
20 | | -const response = await router.execute( |
21 | | - { |
22 | | - clientId: 'safehavenrr', |
23 | | - type: TaskType.CONTENT_GENERATION, |
24 | | - complexity: TaskComplexity.MEDIUM, |
25 | | - description: 'Write blog post about roof repair costs in Houston', |
26 | | - }, |
27 | | - 'You are an expert roofing content writer...', |
28 | | - 'Write a 1200-word blog post about...', |
| 12 | +router.initClient('tenant-a', { monthlyBudgetPerClient: 200 }); |
| 13 | +const result = await router.execute( |
| 14 | + { clientId: 'tenant-a', type: TaskType.CONTENT_GENERATION, complexity: TaskComplexity.MEDIUM }, |
| 15 | + 'You are a careful writer.', |
| 16 | + 'Draft the article.', |
29 | 17 | ); |
30 | 18 | ``` |
31 | 19 |
|
32 | | -## Architecture |
33 | | - |
34 | | -``` |
35 | | -┌─────────────────────────────────────────────────────────────┐ |
36 | | -│ L9LLMRouter.execute() │ |
37 | | -├─────────────────────────────────────────────────────────────┤ |
38 | | -│ 1. Classify task → TaskType × TaskComplexity │ |
39 | | -│ 2. Check budget → BudgetTracker.evaluateTask() │ |
40 | | -│ 3. Route to matrix: │ |
41 | | -│ ├── Search tasks → Perplexity Matrix → Sonar models │ |
42 | | -│ ├── Vision tasks → Vision Matrix → GPT-4o/Claude/Gemini │ |
43 | | -│ └── General tasks → General Matrix → Best model per job │ |
44 | | -│ 4. Execute via provider client │ |
45 | | -│ 5. Record spend + log routing decision │ |
46 | | -└─────────────────────────────────────────────────────────────┘ |
47 | | -``` |
48 | | - |
49 | | -## Three Matrices |
50 | | - |
51 | | -### 1. Perplexity Matrix (Search-Grounded Tasks) |
52 | | - |
53 | | -Ported from the [Enrichment.Inference.Engine](https://github.com/cryptoxdog/Enrichment.Inference.Engine) search optimizer. Maps task complexity to Sonar model + search depth: |
54 | | - |
55 | | -| Complexity | Model | Search Context | Cost/Call | |
56 | | -|---|---|---|---| |
57 | | -| Trivial/Low | `sonar` | `low` | ~$0.001 | |
58 | | -| Medium | `sonar-pro` | `medium` | ~$0.01 | |
59 | | -| High | `sonar-pro` | `high` | ~$0.03 | |
60 | | -| Critical | `sonar-deep-research` | `high` | ~$0.05 | |
61 | | - |
62 | | -Includes consensus mode (multiple variations for high-stakes research). |
63 | | - |
64 | | -### 2. General Matrix (All Other Tasks) |
65 | | - |
66 | | -Maps `TaskType × TaskComplexity` to the optimal model across providers: |
67 | | - |
68 | | -| Task Type | Low | Medium | High | Critical | |
69 | | -|---|---|---|---|---| |
70 | | -| Classification | GPT-4o-mini | GPT-4o-mini | GPT-4o | Claude Sonnet | |
71 | | -| Content Generation | Claude Haiku | Claude Sonnet | Claude Sonnet | Claude Opus | |
72 | | -| Strategic Reasoning | GPT-4o | Claude Sonnet | Claude Sonnet | O3 | |
73 | | -| Code Generation | Claude Haiku | Claude Sonnet | Claude Sonnet | O3 | |
74 | | -| Extraction | GPT-4o-mini | GPT-4o | GPT-4o | Claude Sonnet | |
75 | | - |
76 | | -Each model has a 2-deep fallback chain for resilience. |
77 | | - |
78 | | -### 3. Vision Matrix (Visual QA Tasks) |
79 | | - |
80 | | -| Complexity | Model | Detail | Cost/Call | |
81 | | -|---|---|---|---| |
82 | | -| Low (quick check) | Gemini Flash Vision | `low` | ~$0.001 | |
83 | | -| Medium (layout) | GPT-4o Vision | `auto` | ~$0.015 | |
84 | | -| High (detailed) | GPT-4o Vision | `high` | ~$0.02 | |
85 | | -| Multi-image comparison | Claude Sonnet Vision | `high` | ~$0.03 | |
86 | | - |
87 | | -## Budget Engine |
88 | | - |
89 | | -No daily hard cap. Trajectory-based throttling with surge awareness: |
90 | | - |
91 | | -- **Monthly budget**: $200/client (configurable per-client) |
92 | | -- **Weekly target**: $50/week soft target |
93 | | -- **Weekly ceiling**: $100/week hard safety net |
94 | | -- **Surge**: If week-to-date spend < 60% by Thursday, allow burst up to ceiling |
95 | | -- **Critical override**: CRITICAL tasks ALWAYS proceed regardless of budget |
96 | | -- **Downgrade, don't kill**: Under throttle, tasks get cheaper models instead of being blocked |
97 | | - |
98 | | -## Vision QA (Site Visual Validation) |
99 | | - |
100 | | -The router includes a full Visual QA system that lets bots "see" websites: |
101 | | - |
102 | | -```typescript |
103 | | -// Generate QA plan for a site |
104 | | -const tasks = router.planVisualQA({ |
105 | | - pages: ['https://safehavenrr.com', 'https://safehavenrr.com/services'], |
106 | | - viewports: [VIEWPORTS.desktop_1440, VIEWPORTS.mobile_iphone], |
107 | | - competitorUrl: 'https://competitor.com', |
108 | | - conversionAudit: true, |
109 | | -}); |
110 | | - |
111 | | -// Bot takes screenshots, then executes each task |
112 | | -for (const task of tasks) { |
113 | | - const result = await router.execute( |
114 | | - { clientId: 'safehavenrr', type: TaskType.LAYOUT_VALIDATION, complexity: TaskComplexity.MEDIUM }, |
115 | | - task.prompt, |
116 | | - 'Analyze this screenshot', |
117 | | - { images: [screenshotUrl] }, |
118 | | - ); |
119 | | -} |
120 | | -``` |
121 | | - |
122 | | -Cost: ~$0.40 per full site audit (5 pages × 3 viewports + competitor + conversion). |
123 | | - |
124 | | -## Consuming This Module |
125 | | - |
126 | | -### From L9 SEO Bot |
127 | | - |
128 | | -```typescript |
129 | | -// In l9-seo-bot/package.json |
130 | | -"dependencies": { |
131 | | - "@quantum-l9/llm-router": "file:../l9-llm-router" |
132 | | -} |
133 | | -``` |
134 | | - |
135 | | -### From L9 Website Factory |
136 | | - |
137 | | -```typescript |
138 | | -// In l9-website-factory/package.json |
139 | | -"dependencies": { |
140 | | - "@quantum-l9/llm-router": "file:../l9-llm-router" |
141 | | -} |
142 | | -``` |
143 | | - |
144 | | -### From Any Future Bot |
145 | | - |
146 | | -Same pattern. Import, configure with API keys, call `execute()`. |
147 | | - |
148 | | -## Environment Variables |
149 | | - |
150 | | -```env |
151 | | -OPENROUTER_API_KEY=sk-or-v1-... |
152 | | -PERPLEXITY_API_KEY=pplx-... |
153 | | -``` |
154 | | - |
155 | | -That's it. Two API keys give you access to every model. |
156 | | - |
157 | | -## File Structure |
158 | | - |
159 | | -``` |
160 | | -src/ |
161 | | -├── index.ts # Main router + re-exports |
162 | | -├── types.ts # All types, enums, interfaces |
163 | | -├── matrices/ |
164 | | -│ ├── perplexity-matrix.ts # Search task → Sonar model resolver |
165 | | -│ └── general-matrix.ts # General task → model resolver |
166 | | -├── vision/ |
167 | | -│ └── index.ts # Visual QA engine + prompts |
168 | | -├── budget/ |
169 | | -│ └── index.ts # Budget tracker + throttle engine |
170 | | -└── providers/ |
171 | | - ├── perplexity.ts # Perplexity API client |
172 | | - └── openrouter.ts # OpenRouter API client |
173 | | -``` |
174 | | - |
175 | | -## Design Principles |
176 | | - |
177 | | -1. **Deterministic routing** — No LLM call needed to decide which LLM to call |
178 | | -2. **Budget-aware, not budget-killed** — Downgrade models under pressure, never block critical work |
179 | | -3. **Provider-agnostic** — Bots don't know or care which provider serves the response |
180 | | -4. **Surge-friendly** — Quiet weeks allow burst activity without throttling |
181 | | -5. **Consensus-capable** — High-stakes research runs multiple variations for reliability |
182 | | -6. **Vision-native** — Visual QA is a first-class capability, not an afterthought |
183 | | -7. **Portable** — Any L9 bot imports this module identically |
| 20 | +Direct imports from `./openrouter` or `./perplexity` are retained for 1.x compatibility but are deprecated. They bypass router-level budget and circuit controls. |
0 commit comments