MCP: trusted client keys with a higher rate limit - #198
Open
JakeSCahill wants to merge 2 commits into
Open
Conversation
The CS support bot shares the public MCP rate limit (60 req / 15 min per key) and gets throttled enough that it built a local docs-index fallback. Add MCP_TRUSTED_CLIENT_KEYS: secret keys that get their own bucket and a higher limit (MCP_TRUSTED_CLIENT_LIMIT, default 600 per 15 min). Also closes an existing limiter bypass: x-client-key used to key the bucket for ANY caller, so rotating the header minted fresh buckets. Untrusted requests now always key by client IP (with the existing proxy-header fallbacks); only keys in MCP_TRUSTED_CLIENT_KEYS get key-based bucketing.
✅ Deploy Preview for redpanda-documentation ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
With one KAPA_API_KEY, the support bot and all public MCP traffic drain the same upstream Kapa budget, so our own trusted tier can't stop public 429s from starving the bot. Add optional KAPA_API_KEY_TRUSTED: requests presenting a trusted x-client-key use it upstream, giving the bot its own Kapa budget end to end. The upstream client becomes a small per-key pool (client + transport per key, reset scoped per key; the process-guard reset clears all). Tool handlers can't see HTTP headers (the Netlify adapter doesn't populate the MCP SDK's requestInfo — verified empty), so the limiter middleware records the trust decision in an AsyncLocalStorage the handler reads back. Unset KAPA_API_KEY_TRUSTED = single-key behavior, unchanged. Tested locally by running the handler under node:http with an invalid default key and a valid trusted key: untrusted and wrong-key requests fail upstream, the trusted client key gets a real answer.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The CS support bot (redleader-next) calls our docs MCP for every customer question and shares the public rate limit — 60 requests per 15 minutes per key. It gets throttled enough that it grew a local docs-index fallback, which is a lower-fidelity answer source for customers.
Two commits:
1. Trusted client keys with a higher local limit. Keys listed in
MCP_TRUSTED_CLIENT_KEYS(comma-separated Netlify env var) presented viax-client-keyget their own bucket with a higher limit (MCP_TRUSTED_CLIENT_LIMIT, default 600 per 15 min). Also fixes an existing limiter bypass:x-client-keyused to key the bucket for any caller, so rotating the header minted a fresh bucket per request. Untrusted requests now always key by client IP; only trusted keys get key-based bucketing.2. Dedicated upstream Kapa key for trusted clients. Our local tier can't stop public traffic from draining the single upstream
KAPA_API_KEYbudget, so trusted requests can optionally useKAPA_API_KEY_TRUSTED— a second Kapa key with its own upstream budget — for end-to-end isolation. The upstream client becomes a per-key connection pool. Tool handlers never see HTTP headers (the Netlify adapter leaves the MCP SDK'srequestInfoempty — verified), so the limiter middleware records the trust decision in anAsyncLocalStoragethat the tool handler reads back. WithKAPA_API_KEY_TRUSTEDunset, behavior is identical to a single key.Tested:
npx vitest run limiter-key.test.ts(5 passing), plus a local end-to-end run of the real handler undernode:httpwith an invalid default key and a valid trusted key — untrusted and wrong-key requests fail upstream, the trusted client key returns a real docs answer.Rollout: generate a client key (
openssl rand -hex 24), setMCP_TRUSTED_CLIENT_KEYSin Netlify and share the key with the redleader-next config; ask Kapa for a second API key ([email protected]) and set it asKAPA_API_KEY_TRUSTED. Each piece works independently. Complements the retrieval filter in #197.