Skip to content

Commit 599a3a1

Browse files
committed
fix(router): harden routing, breaker, and budget invariants
Transplanted-From: 1091f6b (remediation pack 2026-07-20)
1 parent 8e777e3 commit 599a3a1

32 files changed

Lines changed: 1340 additions & 6528 deletions

.github/workflows/ci.yml

Lines changed: 0 additions & 35 deletions
This file was deleted.

.github/workflows/publish.yml

Lines changed: 0 additions & 56 deletions
This file was deleted.

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
11
node_modules/
22
dist/
33
*.log
4+
*.tgz
5+
artifacts/
6+
coverage/

ARCHITECTURE.md

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Architecture
2+
3+
`@quantum-l9/llm-router` is a reusable TypeScript routing library. The root `L9LLMRouter` is the supported execution surface.
4+
5+
## Runtime flow
6+
7+
```text
8+
validated TaskDescriptor
9+
-> pure route resolution
10+
-> atomic process-local budget reservation
11+
-> provider-family-safe downgrade
12+
-> per-provider circuit permit
13+
-> provider dispatch
14+
-> cost reconciliation and audit log
15+
```
16+
17+
Provider clients live under `src/providers/`. Production modules outside `src/index.ts` and `src/providers/` may not import them. Existing provider subpath exports remain available during the 1.x line for compatibility, but direct use is deprecated because it bypasses budget and circuit controls.
18+
19+
## State scope
20+
21+
The built-in budget tracker and circuit breaker are process-local. They prevent concurrent overspend and provider stampedes inside one process. Distributed enforcement requires an external persistence adapter and is intentionally not claimed here.

README.md

Lines changed: 9 additions & 172 deletions
Original file line numberDiff line numberDiff line change
@@ -1,183 +1,20 @@
11
# @quantum-l9/llm-router
22

3-
> The shared intelligence routing layer for all L9 bots. One module, all models, zero waste.
3+
Shared deterministic model routing, budget enforcement, provider resilience, and visual QA for L9 bots.
44

5-
## What This Is
6-
7-
A standalone, reusable TypeScript module that any L9 bot imports to get optimal model selection, budget enforcement, and multi-provider routing — without each bot implementing its own LLM integration.
8-
9-
```typescript
10-
import { L9LLMRouter, TaskType, TaskComplexity } from '@quantum-l9/llm-router';
5+
```ts
6+
import { L9LLMRouter, TaskComplexity, TaskType } from '@quantum-l9/llm-router';
117

128
const router = new L9LLMRouter({
139
perplexityApiKey: process.env.PERPLEXITY_API_KEY!,
1410
openrouterApiKey: process.env.OPENROUTER_API_KEY!,
15-
appName: 'L9-SEO-Bot',
1611
});
17-
18-
router.initClient('safehavenrr', { monthlyBudgetPerClient: 200 });
19-
20-
const response = await router.execute(
21-
{
22-
clientId: 'safehavenrr',
23-
type: TaskType.CONTENT_GENERATION,
24-
complexity: TaskComplexity.MEDIUM,
25-
description: 'Write blog post about roof repair costs in Houston',
26-
},
27-
'You are an expert roofing content writer...',
28-
'Write a 1200-word blog post about...',
12+
router.initClient('tenant-a', { monthlyBudgetPerClient: 200 });
13+
const result = await router.execute(
14+
{ clientId: 'tenant-a', type: TaskType.CONTENT_GENERATION, complexity: TaskComplexity.MEDIUM },
15+
'You are a careful writer.',
16+
'Draft the article.',
2917
);
3018
```
3119

32-
## Architecture
33-
34-
```
35-
┌─────────────────────────────────────────────────────────────┐
36-
│ L9LLMRouter.execute() │
37-
├─────────────────────────────────────────────────────────────┤
38-
│ 1. Classify task → TaskType × TaskComplexity │
39-
│ 2. Check budget → BudgetTracker.evaluateTask() │
40-
│ 3. Route to matrix: │
41-
│ ├── Search tasks → Perplexity Matrix → Sonar models │
42-
│ ├── Vision tasks → Vision Matrix → GPT-4o/Claude/Gemini │
43-
│ └── General tasks → General Matrix → Best model per job │
44-
│ 4. Execute via provider client │
45-
│ 5. Record spend + log routing decision │
46-
└─────────────────────────────────────────────────────────────┘
47-
```
48-
49-
## Three Matrices
50-
51-
### 1. Perplexity Matrix (Search-Grounded Tasks)
52-
53-
Ported from the [Enrichment.Inference.Engine](https://github.com/cryptoxdog/Enrichment.Inference.Engine) search optimizer. Maps task complexity to Sonar model + search depth:
54-
55-
| Complexity | Model | Search Context | Cost/Call |
56-
|---|---|---|---|
57-
| Trivial/Low | `sonar` | `low` | ~$0.001 |
58-
| Medium | `sonar-pro` | `medium` | ~$0.01 |
59-
| High | `sonar-pro` | `high` | ~$0.03 |
60-
| Critical | `sonar-deep-research` | `high` | ~$0.05 |
61-
62-
Includes consensus mode (multiple variations for high-stakes research).
63-
64-
### 2. General Matrix (All Other Tasks)
65-
66-
Maps `TaskType × TaskComplexity` to the optimal model across providers:
67-
68-
| Task Type | Low | Medium | High | Critical |
69-
|---|---|---|---|---|
70-
| Classification | GPT-4o-mini | GPT-4o-mini | GPT-4o | Claude Sonnet |
71-
| Content Generation | Claude Haiku | Claude Sonnet | Claude Sonnet | Claude Opus |
72-
| Strategic Reasoning | GPT-4o | Claude Sonnet | Claude Sonnet | O3 |
73-
| Code Generation | Claude Haiku | Claude Sonnet | Claude Sonnet | O3 |
74-
| Extraction | GPT-4o-mini | GPT-4o | GPT-4o | Claude Sonnet |
75-
76-
Each model has a 2-deep fallback chain for resilience.
77-
78-
### 3. Vision Matrix (Visual QA Tasks)
79-
80-
| Complexity | Model | Detail | Cost/Call |
81-
|---|---|---|---|
82-
| Low (quick check) | Gemini Flash Vision | `low` | ~$0.001 |
83-
| Medium (layout) | GPT-4o Vision | `auto` | ~$0.015 |
84-
| High (detailed) | GPT-4o Vision | `high` | ~$0.02 |
85-
| Multi-image comparison | Claude Sonnet Vision | `high` | ~$0.03 |
86-
87-
## Budget Engine
88-
89-
No daily hard cap. Trajectory-based throttling with surge awareness:
90-
91-
- **Monthly budget**: $200/client (configurable per-client)
92-
- **Weekly target**: $50/week soft target
93-
- **Weekly ceiling**: $100/week hard safety net
94-
- **Surge**: If week-to-date spend < 60% by Thursday, allow burst up to ceiling
95-
- **Critical override**: CRITICAL tasks ALWAYS proceed regardless of budget
96-
- **Downgrade, don't kill**: Under throttle, tasks get cheaper models instead of being blocked
97-
98-
## Vision QA (Site Visual Validation)
99-
100-
The router includes a full Visual QA system that lets bots "see" websites:
101-
102-
```typescript
103-
// Generate QA plan for a site
104-
const tasks = router.planVisualQA({
105-
pages: ['https://safehavenrr.com', 'https://safehavenrr.com/services'],
106-
viewports: [VIEWPORTS.desktop_1440, VIEWPORTS.mobile_iphone],
107-
competitorUrl: 'https://competitor.com',
108-
conversionAudit: true,
109-
});
110-
111-
// Bot takes screenshots, then executes each task
112-
for (const task of tasks) {
113-
const result = await router.execute(
114-
{ clientId: 'safehavenrr', type: TaskType.LAYOUT_VALIDATION, complexity: TaskComplexity.MEDIUM },
115-
task.prompt,
116-
'Analyze this screenshot',
117-
{ images: [screenshotUrl] },
118-
);
119-
}
120-
```
121-
122-
Cost: ~$0.40 per full site audit (5 pages × 3 viewports + competitor + conversion).
123-
124-
## Consuming This Module
125-
126-
### From L9 SEO Bot
127-
128-
```typescript
129-
// In l9-seo-bot/package.json
130-
"dependencies": {
131-
"@quantum-l9/llm-router": "file:../l9-llm-router"
132-
}
133-
```
134-
135-
### From L9 Website Factory
136-
137-
```typescript
138-
// In l9-website-factory/package.json
139-
"dependencies": {
140-
"@quantum-l9/llm-router": "file:../l9-llm-router"
141-
}
142-
```
143-
144-
### From Any Future Bot
145-
146-
Same pattern. Import, configure with API keys, call `execute()`.
147-
148-
## Environment Variables
149-
150-
```env
151-
OPENROUTER_API_KEY=sk-or-v1-...
152-
PERPLEXITY_API_KEY=pplx-...
153-
```
154-
155-
That's it. Two API keys give you access to every model.
156-
157-
## File Structure
158-
159-
```
160-
src/
161-
├── index.ts # Main router + re-exports
162-
├── types.ts # All types, enums, interfaces
163-
├── matrices/
164-
│ ├── perplexity-matrix.ts # Search task → Sonar model resolver
165-
│ └── general-matrix.ts # General task → model resolver
166-
├── vision/
167-
│ └── index.ts # Visual QA engine + prompts
168-
├── budget/
169-
│ └── index.ts # Budget tracker + throttle engine
170-
└── providers/
171-
├── perplexity.ts # Perplexity API client
172-
└── openrouter.ts # OpenRouter API client
173-
```
174-
175-
## Design Principles
176-
177-
1. **Deterministic routing** — No LLM call needed to decide which LLM to call
178-
2. **Budget-aware, not budget-killed** — Downgrade models under pressure, never block critical work
179-
3. **Provider-agnostic** — Bots don't know or care which provider serves the response
180-
4. **Surge-friendly** — Quiet weeks allow burst activity without throttling
181-
5. **Consensus-capable** — High-stakes research runs multiple variations for reliability
182-
6. **Vision-native** — Visual QA is a first-class capability, not an afterthought
183-
7. **Portable** — Any L9 bot imports this module identically
20+
Direct imports from `./openrouter` or `./perplexity` are retained for 1.x compatibility but are deprecated. They bypass router-level budget and circuit controls.

eslint.config.js

Lines changed: 3 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,7 @@
1-
// @ts-check
1+
import eslint from '@eslint/js';
22
import tseslint from 'typescript-eslint';
3-
4-
// ESLint v9 flat config. This repo is ESM (`"type": "module"`), TypeScript-only
5-
// (src/ + tests/), with no runtime bundler config to lint.
63
export default tseslint.config(
7-
{
8-
ignores: ['dist/**', 'node_modules/**', 'src/docs/**'],
9-
},
4+
{ ignores: ['dist/**', 'node_modules/**'] },
5+
eslint.configs.recommended,
106
...tseslint.configs.recommended,
11-
{
12-
files: ['**/*.ts'],
13-
rules: {
14-
// Router config objects intentionally cast a resolved model string back
15-
// onto a typed field when applying a budget downgrade (see src/index.ts).
16-
'@typescript-eslint/no-explicit-any': 'warn',
17-
'@typescript-eslint/no-unused-vars': [
18-
'error',
19-
{ argsIgnorePattern: '^_', varsIgnorePattern: '^_' },
20-
],
21-
},
22-
},
237
);

0 commit comments

Comments
 (0)