fix(cf-ai-gateway): route provider options through openaiCompatible key (#24432)#25573
Open
NathanDrake2406 wants to merge 5 commits intoanomalyco:devfrom
Open
fix(cf-ai-gateway): route provider options through openaiCompatible key (#24432)#25573NathanDrake2406 wants to merge 5 commits intoanomalyco:devfrom
NathanDrake2406 wants to merge 5 commits intoanomalyco:devfrom
Conversation
github-actions
Bot
added
the
needs:compliance
github-actions
Bot
removed
the
needs:compliance
Contributor
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
…etection
The OpenAI variants block held two undocumented date literals ("2025-11-13",
"2025-12-04") and a matcher that missed dotted gpt-5.x ids. The dates gate
which reasoning_effort tiers a model exposes, mirroring OpenAI's API rollout
schedule, but nothing in the file said so.
Extract OPENAI_NONE_EFFORT_RELEASE_DATE / OPENAI_XHIGH_EFFORT_RELEASE_DATE
with a comment naming what the dates mean, and lift the effort selection into
openaiReasoningEfforts(id, releaseDate). Replace the
"id.includes('gpt-5-') || id === 'gpt-5'" check with an anchored regex that
matches gpt-5, gpt-5-nano, gpt-5.4, openai/gpt-5.5, and rejects gpt-50 / gpt-5o.
The regex change is a real behaviour fix: gpt-5.x models now correctly expose
the "minimal" reasoning_effort tier, which OpenAI's API has accepted on the
gpt-5 family since launch. Previously variant=minimal was a no-op for any
gpt-5.x model.
Adds two regression tests pinning both halves: the dotted id now gets minimal,
and the gpt-50 lookalike still does not.
Variant input (variant: xhigh) and provider options
(provider.cloudflare-ai-gateway.models.<id>.options.reasoningEffort) on
cf-ai-gateway models routed through ai-gateway-provider were silently dropped.
Outgoing requests to gateway.ai.cloudflare.com used the OpenAI-compatible
endpoint without the reasoning_effort field set, so OpenAI upstreams ran at
their default effort regardless of user config.
sdkKey() had no case for "ai-gateway-provider" and fell back to the providerID
"cloudflare-ai-gateway". providerOptions() therefore wrote the payload under
that key, but ai-gateway-provider/unified wraps createOpenAICompatible({ name:
"Unified" }), and @ai-sdk/openai-compatible only reads compatibleOptions from
"openai-compatible" / "openaiCompatible" / "Unified" / "unified". The wrong
key was never read, so reasoningEffort never reached the request body.
variants() likewise had no "ai-gateway-provider" case, so workflow variant
inputs produced an empty options object.
Add the sdkKey case returning "openaiCompatible" (the camelCase form avoids
the SDK's deprecation warning emitted on the kebab form). Add a variants case
that dispatches on the model.api.id upstream prefix and reuses
openaiReasoningEfforts() for openai/* models, falling back to
WIDELY_SUPPORTED_EFFORTS for other upstreams since the Cloudflare /v1/compat
endpoint translates reasoning_effort to provider-native controls.
Adds an end-to-end test that wires the actual ai-gateway-provider +
@ai-sdk/openai-compatible chain through a stubbed fetch and asserts
reasoning_effort lands in the body Cloudflare AI Gateway forwards upstream.
The test also pins the legacy buggy key so a future refactor that resurrects
providerID-keyed providerOptions fails before it reaches users.
Fixes anomalyco#24432.
NathanDrake2406
force-pushed
the
fix/cf-ai-gateway-provider-options
branch
from
ad683bd to
a980260
Compare
Collaborator
|
/review |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
|
||
| // Computes the reasoning_effort tiers an OpenAI (or OpenAI-compatible upstream | ||
| // routed through it, e.g. cf-ai-gateway) model exposes. Returns null for models | ||
| // with no tunable effort knob (gpt-5-pro). Effort order: weakest → strongest. |
Contributor
There was a problem hiding this comment.
Suggestion: same ASCII style note here: this new comment introduces a Unicode arrow. Consider weakest to strongest or weakest -> strongest.
| import { createUnified } from "ai-gateway-provider/providers/unified" | ||
| import { ProviderTransform } from "@/provider/transform" | ||
|
|
||
| type Captured = { url: string; outerBody: any } |
Contributor
There was a problem hiding this comment.
Suggestion: this new test file introduces several anys (the captured body, fetch parameters, mock model, and providerOptions). The style guide asks us to avoid any when a narrower type is practical; a small shape for the captured gateway body plus Parameters<typeof fetch> / unknown for parsed JSON would keep the regression test type-safe without much extra code.
| // chain that provider.ts:811 builds at runtime, with only the network boundary | ||
| // stubbed. Asserts that `reasoning_effort` (and other provider options the | ||
| // transform emits) actually land in the body Cloudflare AI Gateway forwards | ||
| // upstream — which is the only place the bug was observable. |
Contributor
There was a problem hiding this comment.
Suggestion: the style guide says to default to ASCII when editing or creating files. This new file introduces an em dash here and Unicode arrows below; consider ASCII - / -> unless the Unicode punctuation is intentional.
…fforts comment A reviewer comment on PR anomalyco#25573 flagged that the new comment introduced a unicode arrow inconsistent with the repo's ASCII-default style for code comments. The arrow shows up as a glyph that does not survive every editor or terminal cleanly and breaks grep on plain ASCII. Reword "weakest -> strongest" using ASCII so the comment matches the surrounding style guidelines without changing its meaning.
…comments Reviewer feedback on PR anomalyco#25573 flagged two issues in the new regression test: the file used em dashes and unicode arrows in comments against the repo's ASCII-default style, and several `any` annotations (captured body, fetch parameters, mock model, providerOptions) bypassed the type system in a file whose entire purpose is catching upstream contract drift. The wide `any` annotations meant a future change to `Provider.Model` or to the AI SDK's `providerOptions` shape would silently typecheck instead of surfacing here, which defeats the regression test. The comments used unicode purely cosmetically. Replace `any` with the actual types the runtime uses: `Provider.Model` for the mock factory (constructed via `ModelID.make` / `ProviderID.make` to satisfy the branded ID schema), `Record<string, Record<string, JSONValue>>` for providerOptions to match what `generateText` accepts, `Parameters<typeof fetch>` for the stub's signature, and a typed `isRecord` guard to narrow parsed JSON without an `any` cast. Preserve Bun's `preconnect` method on the stub via `Object.assign` so the assignment satisfies `typeof fetch` honestly. Convert the unicode arrows and em dashes in the file's comments to ASCII. All 283 provider tests still pass and `bunx tsc --noEmit` is clean.
Author
Has anyone ever told you that you look like a mix of Arnold and Andre the Giant? |
Collaborator
|
lol! sometimes the former but never andre hahahaaa |
Collaborator
|
We will merge this in today, thx for fix |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes #24432
Type of change
What does this PR do?
reasoningEffortand workflowvariantinputs were silently dropped for cf-ai-gateway models routed throughai-gateway-provider. The outgoing request togateway.ai.cloudflare.comhad noreasoning_effortfield regardless of config.Root cause in
packages/opencode/src/provider/transform.ts:sdkKey()had no"ai-gateway-provider"case, so it fell back tomodel.providerID = "cloudflare-ai-gateway".ai-gateway-provider/unifiediscreateOpenAICompatible({ name: "Unified" }), and@ai-sdk/openai-compatibleonly reads provider options fromopenai-compatible/openaiCompatible/Unified/unified. Wrong key, never read.variants()had no case either, sovariant: xhighreturned{}.Fix: add the
sdkKeycase returning"openaiCompatible"(kebab form emits a runtime deprecation warning), and avariants()case dispatching onmodel.api.idupstream prefix.First commit is a refactor done so the new case could reuse logic: extracts
openaiReasoningEfforts()from the@ai-sdk/openaicase, names the unexplained2025-11-13/2025-12-04date literals, and replaces thegpt-5-substring matcher with an anchored regex so dotted ids likegpt-5.4are caught.Anthropic / Google on cf-ai-gateway resolve through bundled SDKs per models.dev and were unaffected.
How did you verify your code works?
test/provider/cf-ai-gateway-e2e.test.tsruns the realai-gateway-provider+@ai-sdk/openai-compatiblechain through a stubbed fetch and assertsreasoning_effortlands in the upstream body. Plus 7 unit tests for the variants / providerOptions / matcher changes.bunx tsc --noEmitclean. Manually repro'd: outgoing body went from noreasoning_efforttoreasoning_effort: "xhigh".Screenshots / recordings
N/A.
Checklist