feat: per-host request gate with adaptive cooldown for concurrent fetches#10
Merged
Conversation
…ent fetches Concurrent same-host fetches — most visibly the MCP server firing several `fetch_feed` tool calls at once, each building a fresh HttpClient that shared no state — tripped provider per-domain rate limits (HTTP 429). Nothing local to one call could coordinate the burst. Add a process-wide per-host gate (src/ratelimit.rs) owned by a reused HttpClient that the MCP server builds once and shares across tool calls (also restoring connection reuse). Per authority (host:port): a concurrency cap (default 1) plus an adaptive cooldown — honoring `Retry-After` when sent, otherwise a bounded escalating backoff — and sticky spacing once a host is warm. The server absorbs waits up to a bound, then fails fast with a structured RATE_LIMITED error the caller can pace on. Paired changes: descriptive User-Agent, HTTP-date `Retry-After` parsing. Wiring is additive (fetch_feeds_with(&HttpClient)); SCHEMA_VERSION stays "1" (RATE_LIMITED is an additive error code, details are free-form). See ADR-0016.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Concurrent fetches to the same host trip provider per-domain rate limits (HTTP 429). The worst case is the MCP server:
fetch_feedtakes a single URL and built a freshHttpClientper call, sharing only the cache — so several concurrent tool calls burst one host with no coordination. A single in-flight retry (ADR-0015) can't help across concurrent requests. An empirical probe also showed some providers send noRetry-Afterand throttle even serial requests, so serialization alone isn't enough.Change
A process-wide per-host request gate (
src/ratelimit.rs) owned by a reusedHttpClientthatRssServerbuilds once and shares across tool calls (also restoring keep-alive). Keyed on authority (host:port):Retry-After(delta-seconds and HTTP-date) when sent; otherwise a bounded escalating backoff (2s·2^(n-1), capped 60s) that resets on success.RATE_LIMITEDerror carryingretry_afterso an agent can pace.Paired: descriptive
User-Agent; HTTP-dateRetry-Afterparsing. All tunables are env-overridable (RSS_HOST_CONCURRENCY,RSS_MAX_COOLDOWN_SECS,RSS_MAX_GATE_WAIT_SECS).Contract / invariants
SCHEMA_VERSIONstays"1"—RATE_LIMITEDis an additive error code; details are free-form.core(wiring is additive:fetch_feeds_with(&HttpClient)); no pacing logic inmcp.rs.RETRY_MAX_DELAY5s) is kept distinct from the sibling-facing cooldown cap.See ADR-0016 for rationale and alternatives.
Testing
cargo fmt --check,cargo clippy --all-targets -D warnings, andcargo testall pass (87 lib + integration). New tests cover gate serialization, per-host isolation, cooldown escalation/cap, sticky spacing, fail-fast ceiling, cancellation,Retry-Afterparsing (both forms), and theRATE_LIMITEDcontract. Reviewed via a multi-dimension adversarial pass; a live fetch confirms the happy path adds no latency.Follow-ups (deliberately out of scope)
fetch_feed(urls: [..]) so the server paces a batch internally — larger contract surface; possible future ADR.