feat: Serper/cache/concurrency + token reporting, stream-terminator, and Sonnet/model-map fixes#4
Open
andyzengmath wants to merge 10 commits into
Open
Conversation
Add PowerShell and CMD examples for running the proxy on Windows, since launch.sh is bash-only. Co-authored-by: Copilot <[email protected]>
DuckDuckGo Lite has become unreliable due to aggressive CAPTCHA/bot detection on automated requests. This causes web search to silently fail with empty results. Replace with Exa and Parallel MCP endpoints (same approach used by OpenCode) as the primary search backend: - Uses JSON-RPC 2.0 MCP protocol (tools/call method) - Traffic split 50/50 between Exa and Parallel by default - Automatic cross-fallback (if one fails, tries the other) - Works without API keys out of the box - 25-second timeout per request Provider priority chain: 1. Brave Search API (if BRAVE_API_KEY set) 2. Exa/Parallel MCP (primary, free, no key needed) 3. DuckDuckGo Lite (fallback, may hit CAPTCHAs) 4. DuckDuckGo Instant Answer (last resort) New env vars (all optional): - WEBSEARCH_PROVIDER: force "exa" or "parallel" - EXA_API_KEY: optional key for Exa - PARALLEL_API_KEY: optional key for Parallel
- Add Serper.dev (Google SERP API) as fallback search provider via SERPER_API_KEY env var - Reorder provider chain: Exa/Parallel MCP (free) → Brave → Serper → DDG Lite → DDG Instant - Add 5-min search result cache to deduplicate identical queries across parallel agents - Add concurrency semaphore (max 2 concurrent searches) to prevent DDG rate-limit storms - Route all search calls through throttled wrapper (executeWebSearchThrottled) Fixes cascading 'DDG Lite error: fetch failed' / 'All search providers failed' errors that occur when multi-agent workflows (e.g. Workflow tool) fire many parallel web searches. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
- Reorder provider list to match new priority (MCP first, paid APIs as fallback) - Add Serper.dev to provider list and config table - Document rate-limit protection (cache + concurrency semaphore) Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
- Report input_tokens in streaming message_start/message_delta so Claude Code tracks context window usage (fixes HUD stuck at 0%) - Reorder usage extraction before sendStartIfNeeded() - Use ?? instead of || for token counts (0 is valid) - Add claude-opus-4.8 model mapping (was falling back to 4.6)
andyzengmath
changed the title
The caller loop's 'if (isDone) break' only breaks the inner for-loop, so after a finish_reason chunk emits the closing sequence, the outer while-loop continues and re-emits it on the [DONE] sentinel and again on stream end. Add a sentStop idempotency guard so the closing message_delta + message_stop are emitted exactly once regardless of how many terminal signals arrive.
Restore the concise per-request diagnostic line (model → mapped model | stream/sync | N messages) so model routing is visible at a glance.
Root cause: Copilot gates Claude Sonnet behind the Editor-Version and Copilot-Integration-Id headers that VS Code sends. Without them, every claude-sonnet-* request returned model_not_supported. - Add Editor-Version + Copilot-Integration-Id headers to both Copilot request paths (streaming + web-search collect) so gated models work - Rewrite MODEL_MAP against the live Copilot /models list: - Sonnet 4.5/4/3.5 -> claude-sonnet-4.6 (real supported ID; 4.5 does not exist on Copilot) - add claude-sonnet-5 passthrough - Opus 4.5/4.1/legacy -> claude-opus-4.6 (nearest supported) - add Opus 4.7 passthrough - Update advertised /models list to match reality Verified: all opus/sonnet/haiku families now return 200 through the proxy.
andyzengmath
changed the title
Address code-review findings on the model/token/header changes:
- Sonnet fallback: match the major version after the 'sonnet' token
(sonnet-5 -> 5) instead of a loose includes('5'), which previously
upgraded unmapped Sonnet-4.5 / dated builds to Sonnet-5.
- Search cache: skip caching empty results (a transient all-providers-fail
no longer suppresses a query for the full TTL), and bound the cache with
expired-entry eviction + a 500-entry cap to stop unbounded growth.
- Editor headers: make Editor-Version and Copilot-Integration-Id
env-overridable (COPILOT_EDITOR_VERSION / COPILOT_INTEGRATION_ID) so a
future Copilot version-gate can be fixed without a code change.
…timeout Four proxy-experience improvements from a full review: - stream_options: include_usage on streaming requests so Copilot reliably returns token usage in the final chunk (input_tokens was previously best-effort, depending on whether usage happened to arrive) - /count_tokens now estimates from actual message/system/tool text (~4 chars/token) instead of raw JSON byte length, so Claude Code's auto-compaction fires at the right time - Extended thinking: drop thinking / redacted_thinking content blocks instead of JSON-stringifying them into the conversation; the thinking request param is not forwarded (Copilot handles reasoning internally) - Request timeout: 120s AbortController (COPILOT_REQUEST_TIMEOUT_MS) on both Copilot fetch paths, and a mid-stream error handler that emits a proper terminator so a dropped connection no longer hangs or truncates silently
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Builds on the MCP web search work (#3) and adds search improvements plus several proxy correctness fixes.
Web search
Accurate token reporting (fixes context HUD stuck at 0%)
input_tokensin streamingmessage_startandmessage_deltaso Claude Code can track context-window usagesendStartIfNeeded()so tokens land inmessage_startwhen the first chunk carries them??instead of||for token counts (0 is valid;||wrongly treats it as falsy)No duplicate stream terminators
if (isDone) breakonly broke the innerfor, so the closingmessage_delta+message_stopwere re-emitted on[DONE]and again at stream end (3×). Added asentStopidempotency guard.Unlock Sonnet + correct model map to real Copilot IDs
Editor-VersionandCopilot-Integration-Idheaders VS Code sends. Without them everyclaude-sonnet-*request returnedmodel_not_supported.MODEL_MAPagainst the live Copilot/modelslist:4.5/4/3.5→claude-sonnet-4.6(4.5doesn't exist on Copilot), addclaude-sonnet-5passthrough4.5/4.1/legacy →claude-opus-4.6, add4.7/4.8passthroughclaude-haiku-4.5/modelslist to match realityTesting
Verified live through the proxy: all opus/sonnet/haiku families return 200 (Sonnet, opus-4.5, opus-4.1 all errored before), streaming emits exactly one terminator, and
message_deltareports realinput_tokens.Notes
main, in case you prefer to review them separately from the search features.