feat: model-based routing — wildcard model-to-provider mapping in proxy#277
feat: model-based routing — wildcard model-to-provider mapping in proxy#277zhangyang-crazy-one wants to merge 50 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c3fe23b460
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ) | ||
| .await | ||
| { | ||
| Ok(Some((_route_id, provider))) => (vec![provider], Some("model_route".to_string())), |
There was a problem hiding this comment.
When a model route matches, this replaces the provider list with only the route target, but the later success path still builds SuccessSyncInfo and calls ProxyServerState::sync_successful_provider_selection, which persists set_current_provider and may update the live backup whenever the route target differs from current_provider_id_at_start. In a setup where *sonnet* routes to provider B while the user’s current provider is A, the first successful routed request permanently switches the app to B, so subsequent non-routed traffic and live config no longer use the user-selected provider.
Useful? React with 👍 / 👎.
| let mut regex_str = String::from("(?i)"); | ||
| for (i, segment) in segments.iter().enumerate() { | ||
| if i > 0 { | ||
| regex_str.push_str(".*"); | ||
| } | ||
| regex_str.push_str(®ex::escape(segment)); | ||
| } | ||
|
|
||
| Regex::new(®ex_str) |
There was a problem hiding this comment.
Wildcard patterns are compiled without ^...$, so regex matching can succeed on any substring rather than the whole model id. For example, a user adding the documented prefix route claude-* would also match xclaude-opus, and sonnet* would match claude-sonnet even though no leading wildcard was supplied; this makes route precedence unpredictable when model ids merely contain another route’s prefix.
Useful? React with 👍 / 👎.
| pattern, | ||
| provider_id, | ||
| priority, | ||
| enabled: true, |
There was a problem hiding this comment.
The TUI edit flow always sends enabled: true to update_model_route, so editing a disabled model route’s pattern/provider/priority silently re-enables it. This differs from the CLI update path, which preserves existing.enabled, and it can cause a route the user intentionally disabled to start matching requests again after a harmless edit.
Useful? React with 👍 / 👎.
zhangyang-crazy-one
changed the title
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c4e55a75d9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| json!({ | ||
| "id": id, | ||
| "object": "model", | ||
| "created": 1700000000, | ||
| "owned_by": "cc-switch" | ||
| }) |
There was a problem hiding this comment.
When Claude Code is pointed at this proxy via ANTHROPIC_BASE_URL, its /model flow uses Anthropic's GET /v1/models schema; the Anthropic Models API documents each item with fields like type, display_name, and created_at (plus pagination fields). This handler only emits OpenAI-style entries (object, created, owned_by), so Anthropic clients that deserialize or display ModelInfo can reject the response or show no usable models even though routes exist. Consider emitting the Anthropic fields as well, or a protocol superset.
Useful? React with 👍 / 👎.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 05861bcea0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| let request_model = body | ||
| .get("model") | ||
| .and_then(|value| value.as_str()) | ||
| .unwrap_or("unknown") |
There was a problem hiding this comment.
For standard Gemini proxy calls, the model is in the path (for example /v1beta/models/gemini-2.5-pro:generateContent) rather than in the JSON body, so this always sets request_model to unknown for those requests. Since model-route matching now uses this value before falling back to ProviderRouter, any --app gemini model route created through the new CLI/TUI will never match normal Gemini traffic.
Useful? React with 👍 / 👎.
| self.overlay = Overlay::TextInput(TextInputState { | ||
| title: texts::tui_model_route_add_priority_title().to_string(), | ||
| prompt: texts::tui_model_route_add_priority_prompt().to_string(), | ||
| input: TextInput::new("0".to_string()), |
There was a problem hiding this comment.
When editing an existing route in the TUI, the priority prompt is always prefilled with 0, so accepting the default after changing only the pattern/provider silently rewrites any non-zero priority to highest priority. That changes routing order for existing rules even though the user did not intend to edit priority; the edit flow should prefill or preserve the route's current priority.
Useful? React with 👍 / 👎.
| if version == 12 { | ||
| log::warn!("数据库版本 {version} 高于 SCHEMA_VERSION={SCHEMA_VERSION}，跳过迁移（兼容模式）"); | ||
| conn.execute("RELEASE schema_migration;", []).ok(); | ||
| return Ok(()); |
There was a problem hiding this comment.
This compatibility branch accepts user_version=12 and returns before running the column repairs that this patch relies on for model_routes. If a v12 database already has model_routes but lacks hit_count/last_hit_at (the code comments call out that case), create_tables will not alter the existing table and later list_model_routes SELECTs those columns, causing proxy/TUI/CLI model-route operations to fail with no such column.
Useful? React with 👍 / 👎.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a45cc8b5b0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if version == 12 { | ||
| log::warn!("数据库版本 {version} 高于 SCHEMA_VERSION={SCHEMA_VERSION}，进入兼容模式并补齐列"); | ||
| Self::create_tables_on_conn(conn)?; |
There was a problem hiding this comment.
When opening a database with user_version 12, this compatibility branch returns after create_tables_on_conn, but CREATE TABLE IF NOT EXISTS does not alter an existing upstream model_routes table. The new DAO selects hit_count and last_hit_at, so a v12 database whose table lacks those local tracking columns will make model-route list/TUI/proxy lookups fail with no such column instead of running in compatibility mode. Run the same add_column_if_missing repairs here before returning.
Useful? React with 👍 / 👎.
| self.overlay = Overlay::ModelRouteProviderPicker { | ||
| pattern: raw, | ||
|
|
||
| selected: 0, |
There was a problem hiding this comment.
When editing an existing rule, the provider picker is always initialized at index 0. If the user only changes or accepts the pattern and presses Enter through the provider step, handle_model_route_provider_picker_key uses that selected row as the provider, so the route is silently moved to the first provider instead of preserving its current target. Initialize selected from the route's current provider_id before opening the picker.
Useful? React with 👍 / 👎.
… ModelRoute type - Add ModelRoute struct with camelCase serde, unit test for serialization - Register model_route module and public export in lib.rs - Add schema_migration_v10_adds_model_routes_table test (RED - no migration yet)
…es table - Bump SCHEMA_VERSION from 10 to 11 - Add CREATE TABLE model_routes to create_tables_on_conn (table 17) - Add migrate_v10_to_v11 function with identical schema to upstream - Add version 10 match arm to apply_schema_migrations_on_conn - All existing tests pass (2596 passed, 0 failed)
- Add list_model_routes (ordered by priority ASC, created_at ASC) - Add get_model_route (by id, returns Option) - Add create_model_route (with FK validation for provider_id) - Add update_model_route (with FK validation on provider_id change) - Add delete_model_route (checks changes count) - Add toggle_model_route (flips enabled, uses NOT enabled) - All 6 DAO unit tests pass within crate tests - Full lib test suite: 2602 passed, 0 failed
- Add model_route_dao_crud_roundtrip: tests create, get, FK validation, update, toggle, delete, list ordering by priority, and app_type filtering - Add model_route_cascade_delete_on_provider_removal: verifies ON DELETE CASCADE works when provider is deleted - All tests pass: 2604 lib tests, all integration test targets green - cargo fmt --check passes, no new clippy warnings
- Add proxy/model_router.rs with ModelRouter struct - Wildcard * to regex conversion with meta-character escaping - Priority-based route selection (lowest number wins) - Case-insensitive matching, enabled-only routing - Defensive empty model and missing provider handling - 16 unit tests covering exact, wildcard, priority, disabled, case-insensitive, regex meta-char, empty model, and missing provider
- Add model_router field to ProxyServerState struct and constructor - Add model_router and route_source fields to HandlerContext - HandlerContext::load() now calls model_router.match_route() first - Model route match bypasses failover queue with single provider - Unmatched/error cases fall back to existing ProviderRouter logic - Update all test_state() helpers: server.rs, handler_context.rs, handlers.rs, response_handler/tests.rs
- Add model_route_match_bypasses_failover_queue integration test - Add no_model_route_falls_back_to_provider_router integration test - Apply cargo fmt across all modified files - Remove unused Mutex import in model_router.rs tests - All 2622 tests pass with zero regressions
- Add ModelRouteCommand enum (List, Add, Remove, Toggle, Update) - Add ModelRoute variant to ProxyCommand with #[command(subcommand)] - Wire ModelRoute dispatch in execute() with get_state() - Add stub handle_model_route() function
- 13 tests covering list, add, remove, toggle, update operations - Tests for non-existent provider rejection - Tests for codex app type isolation - Seed helper for test providers
- Add print_model_routes() helper with comfy-table output - Implement handle_model_route() for List, Add, Remove, Toggle, Update - Add calls Phase 1 DAO methods directly - All 13 TDD tests pass
…, and state fields - Add SettingsModelRoutes variant to Route enum - Define ModelRouteRow and ModelRouteSnapshot types with provider name resolution - Add model_routes field to UiData with data loading from DB - Add SettingsItem::ModelRoutes to settings menu - Add model_routes_idx field to App struct with clamp_selections - Add i18n text for model routes title (EN/CN) - Add stub key handler and placeholder render dispatch - Create ui/model_routes.rs with table rendering
…ers, and overlay dispatch - Add Action::ModelRouteAdd, ModelRouteEdit, ModelRouteDelete, ModelRouteToggle variants - Add TextSubmit multi-step flow variants for Add/Edit (Pattern -> Provider -> Priority) - Add ConfirmAction::ModelRouteDelete variant - Create runtime_actions/model_routes.rs with handle_add, handle_edit, handle_delete, handle_toggle - Wire dispatch in handle_action and cache invalidation - Add i18n texts (toasts, overlay titles/prompts/messages) in English and Chinese - Wire TextSubmit handlers for the 3-step Add and Edit overlay flows - Wire ConfirmAction dispatch for ModelRouteDelete
…key bar - Expand on_settings_model_routes_key with a/e/d/Space handlers - 'a' opens 3-step Add overlay (pattern -> provider -> priority) - 'e' opens 3-step Edit overlay with pre-filled pattern value - 'd' opens Confirm overlay for delete - Space dispatches ModelRouteToggle - Update key bar in model_routes rendering: show Add, Toggle, and conditional Edit/Delete
…ata test literal - Add ModelRouteSnapshot to test imports in ui/tests.rs - Add model_routes: ModelRouteSnapshot::default() to manual UiData construction
- Use TEXT PRIMARY KEY + UUID v4 for id (matching cc-switch v12) - Add model_routes indexes (idx_model_routes_lookup, idx_model_routes_provider) - Keep SCHEMA_VERSION=11 for merge compatibility - Add forward-compat detection for v12 DBs (cc-switch already upgraded) - Update ModelRoute.id from Option<i64> to String across all layers - Fix CLI args, TUI actions, TextSubmit/ConfirmAction variants
- Add ProxySwitch item to LocalProxySettingsItem enum - Render enabled/disabled value in proxy settings table - Handle Enter/Space key to toggle proxy via Action::SetProxyEnabled
- Add hit_count, last_hit_at columns to model_routes table - Backward-compat: ALTER TABLE adds columns if missing - record_model_route_hit() called inside ModelRouter::match_route (spawn_blocking) - aggregate_route_hits_by_provider() for dashboard aggregation - TUI dashboard: multi-color route hit legend (cyan/magenta/yellow/green/blue) - Per-provider color coding: name + hit percentage + total count - Total 8-color palette for up to 5 displayed providers in legend
Track per-provider token activity in proxy server state, expose via ProxyStatus, poll in TUI, and render wave chart dots with per-provider colors matching the legend palette. - ProxyServerState: add provider_token_map with record_provider_activity() - Response handler: record provider activity on successful requests - TUI data: load provider_token_map into ProxySnapshot - App state: poll and maintain per-provider activity samples - Rendering: color wave chart dots per-column based on dominant provider
…ovider Add is_model_routed flag to SuccessSyncInfo. When a model route matches and selects a different provider, the success sync path no longer persists it as the current provider or updates the live backup. This prevents a single routed request from silently switching the user's configured provider.
Wildcard patterns were compiled without a ^ anchor, causing prefix matches like claude-* to match xclaude-opus (substring match). Added ^ anchor so patterns must match from the start of the model id.
The TUI edit flow always sent enabled=true, silently re-enabling disabled routes. Now reads the existing route's enabled state before constructing the update.
The v12 compatibility special case allows SCHEMA_VERSION+1 through, so tests that reject future schemas must use +2 to get a truly unsupported version.
The /v1/models handler only returned OpenAI-style fields (object, created, owned_by). Claude Code clients via ANTHROPIC_BASE_URL expect Anthropic fields (type, display_name, created_at, pagination). Now emits a protocol superset with both Anthropic and OpenAI fields so both client types can consume the response.
Caught by rust-analyzer diagnostic. `json` macro was added speculatively during schema v10→v11 development but never used in the final test suite.
…y, v12 repair) - proxy/handler_context: extract model from Gemini URI path (/v1beta/models/gemini-2.5-pro:generateContent) so model-route matching works for Gemini traffic. Pass path into HandlerContext::load. - tui/views: prefill priority with existing route's value when editing, to avoid silently rewriting non-zero priorities to 0. - database/schema: in v12 compatibility branch of apply_schema_migrations_on_conn, still run create_tables_on_conn so hit_count/last_hit_at columns are added to pre-existing model_routes tables from older versions.
- data::refresh_proxy_snapshot: also reload model_routes from DB so the dashboard legend reflects hit_count accumulated during proxy runtime (previously stayed frozen at the initial UiData::load snapshot). - main_page::LEGEND_MIN_HITS: hide providers with fewer than 5 hits from the legend to reduce noise from 0%-frequency entries.
Non-streaming responses have no char_count-based token estimate, so estimated_output_tokens is 0. The previous early-return at tokens == 0 meant those requests never incremented provider_token_map, leaving the dashboard dot matrix blank for any non-streaming traffic. Use tokens.max(1) so every request contributes at least one unit, ensuring the per-provider dot colors reflect real request distribution even when token estimation is unavailable.
zhangyang-crazy-one
force-pushed
the
feat/model-based-routing-pr
branch
from
39675ac to
20feb74
Compare
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 20feb74877
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| regex_str.push_str(®ex::escape(segment)); | ||
| } | ||
|
|
||
| Regex::new(®ex_str) |
There was a problem hiding this comment.
For wildcard patterns that do not end in *, the generated regex is only start-anchored, so a rule such as *-4-5 also matches claude-haiku-4-55 because is_match can stop at the -4-5 prefix. Since the first matching route selects a single provider, suffix-style rules can route newer/different model IDs to the wrong provider; append $ when the pattern itself does not end with *.
Useful? React with 👍 / 👎.
| self.overlay = Overlay::ModelRouteProviderPicker { | ||
| pattern: raw, | ||
|
|
||
| selected: 0, |
There was a problem hiding this comment.
When editing an existing route, the provider picker always starts at index 0, so pressing Enter to keep the provider silently changes the route to the first provider whenever the current provider is not first in data.providers.rows. Seed selected from the existing route's provider_id so editing pattern/priority does not rewrite the provider unintentionally.
Useful? React with 👍 / 👎.
…Day#10/SaladDay#11 Dashboard multi-provider wave color: - observe_proxy_provider_activity no longer silently returns on the first tick, so provider samples align with main input/output samples from the start; resync provider samples after a proxy restart resets the main counter, fixing the wave degrading to a single accent color Codex review (PR SaladDay#277): - proxy: anchor non-trailing-* model route patterns at the end ($) so a suffix rule like "*-4-5" no longer matches "claude-haiku-4-55"; use "*sonnet*" to match a substring anywhere - tui: preselect the current provider when editing a model route so Enter does not silently move the route to the first provider
The multi-provider color stack was distributed across the full [0, stack_height) range by per-column token share, but the dot-matrix wave only renders the bottom `filled` rows (height scaled by window-max output). Minor providers' colors landed on blank rows and were invisible, so the legend color (e.g. DeepSeek blue) did not match the wave (all dominant purple). - compute_column_color_stacks takes column_filled_rows and fills only the rendered [stack_height-filled, stack_height) range: dominant at the bottom, minor on the top character row - split upper/lower color stacks so output and input shapes align independently - expose recent_samples/scale_samples (pub(super)) so the color logic reuses the same wave-scaling baseline as proxy_wave_lines - add color_stacks_only_fill_rendered_rows regression test
1 participant
Summary
实现基于模型名称的路由功能：通过通配符模式（如
*sonnet*、claude-*）将不同的模型请求路由到指定的 provider。核心功能
数据库 (Phase 1)
model_routes表：存储模型模式到 provider 的映射，支持优先级排序路由引擎 (Phase 2)
ModelRouter：通配符*转正则匹配，大小写不敏感，按优先级选择CLI 命令 (Phase 3)
cc-switch proxy model-route list|add|remove|toggle|updateTUI 管理界面 (Phase 4)
Bug 修复（Codex review）
is_model_routed标记）^锚定，防止前缀越界匹配（如claude-*匹配xclaude-opus）CI 状态
架构对齐