Skip to content

chore(engine): subtree pull opencode used_fallback (0e98eac) #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
TimothyVang merged 2 commits into from
Jul 12, 2026
Merged

chore(engine): subtree pull opencode used_fallback (0e98eac) #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6d69514
Squashed 'engine/' changes from 3d9781b..0e98eac
Jul 11, 2026
4ac3602
chore(engine): subtree pull opencode m28 used_fallback (0e98eac)
Jul 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions engine/packages/opencode/src/session/llm.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -246,9 +246,11 @@ const live: Layer.Layer<
"llm.runtime": "native",
"llm.provider": input.model.providerID,
"llm.model": input.model.id,
"llm.used_fallback": native.used_fallback,
})
return {
type: "native" as const,
used_fallback: native.used_fallback,
stream: native.stream,
}
}
Expand All @@ -257,6 +259,7 @@ const live: Layer.Layer<
"llm.provider": input.model.providerID,
"llm.model": input.model.id,
"llm.native_unsupported_reason": native.reason,
"llm.used_fallback": native.used_fallback,
})
yield* Effect.logInfo("native runtime unavailable; falling back to ai-sdk", {
providerID: input.model.providerID,
Expand All @@ -266,18 +269,23 @@ const live: Layer.Layer<
agent: input.agent.name,
mode: input.agent.mode,
reason: native.reason,
used_fallback: native.used_fallback,
})
}

yield* Effect.logInfo("llm runtime selected", {
"llm.runtime": "ai-sdk",
"llm.provider": input.model.providerID,
"llm.model": input.model.id,
"llm.used_fallback": true,

@chatgpt-codex-connector chatgpt-codex-connector Bot Jul 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Report fallback only after native is attempted

When OPENCODE_EXPERIMENTAL_NATIVE_LLM is unset or false, control skips the native gate and reaches this unconditional AI-SDK branch. The session LLM boundary makes AI SDK the default and native an opt-in path, so emitting llm.used_fallback=true here marks every default request as a fallback even though no fallback occurred; any harness or metrics reading this field will misclassify normal runs. Only set this value when the native gate actually rejected a native attempt, or expose a separate runtime/default indicator.

Useful? React with 👍 / 👎.

})
// Default runtime path: AI SDK owns provider execution and tool dispatch;
// LLMAISDK.toLLMEvents below normalizes fullStream parts for the processor.
// used_fallback is true whenever this path runs (native gate rejected, or
// native not opted in). Downstream reads this field rather than inventing it.
return {
type: "ai-sdk" as const,
used_fallback: true as const,
result: streamText({
onError(error) {
bridge.fork(
Expand Down
30 changes: 20 additions & 10 deletions engine/packages/opencode/src/session/llm/native-runtime.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,15 @@ import type { LLMClientShape } from "@opencode-ai/llm/route"
import { LLMNative } from "./native-request"

export type RuntimeStatus =
| { readonly type: "supported"; readonly apiKey: string; readonly baseURL?: string }
| { readonly type: "unsupported"; readonly reason: string }
| { readonly type: "supported"; readonly apiKey: string; readonly baseURL?: string; readonly used_fallback: false }
| { readonly type: "unsupported"; readonly reason: string; readonly used_fallback: true }
export type StreamResult =
| { readonly type: "supported"; readonly stream: Stream.Stream<LLMEvent, unknown> }
| { readonly type: "unsupported"; readonly reason: string }
| {
readonly type: "supported"
readonly stream: Stream.Stream<LLMEvent, unknown>
readonly used_fallback: false
}
| { readonly type: "unsupported"; readonly reason: string; readonly used_fallback: true }

type StreamInput = {
readonly model: Provider.Model
Expand Down Expand Up @@ -68,36 +72,42 @@ function statusWithFetch(
): RuntimeStatus {
const npm = input.model.api.npm
if (!NATIVE_NPM.has(npm))
return { type: "unsupported", reason: `provider package is not natively supported: ${npm}` }
return unsupported(`provider package is not natively supported: ${npm}`)

// OAuth policy:
// - OpenAI OAuth + codex plugin `fetch` override can stay on the native path.
// - xAI OAuth intentionally falls back to AI SDK: the plugin fetch owns token
// refresh and bearer injection as the AI-SDK contract, not native Auth.
// - All other OAuth without an OpenAI fetch override falls back fail-closed.
if (input.auth?.type === "oauth") {
if (input.provider.id === "xai")
return { type: "unsupported", reason: "xAI OAuth uses AI SDK plugin fetch override" }
if (input.provider.id === "xai") return unsupported("xAI OAuth uses AI SDK plugin fetch override")
if (!(input.provider.id === "openai" && fetch))
return { type: "unsupported", reason: "OAuth auth requires a provider fetch override" }
return unsupported("OAuth auth requires a provider fetch override")
}

const apiKey = typeof input.provider.options.apiKey === "string" ? input.provider.options.apiKey : input.provider.key
if (!apiKey) return { type: "unsupported", reason: "API key is not configured" }
if (!apiKey) return unsupported("API key is not configured")

const baseURL =
typeof input.provider.options.baseURL === "string"
? input.provider.options.baseURL
: input.model.api.url || undefined
if (REQUIRES_BASE_URL.has(npm) && !baseURL) return { type: "unsupported", reason: "base URL is not configured" }
if (REQUIRES_BASE_URL.has(npm) && !baseURL) return unsupported("base URL is not configured")

return {
type: "supported",
apiKey,
baseURL,
used_fallback: false,
}
}

function unsupported(reason: string): RuntimeStatus {
// used_fallback is the runtime source of truth for AI SDK fallback: true only
// when this gate rejects native so the session LLM layer takes the AI SDK path.
return { type: "unsupported", reason, used_fallback: true }
}

export function stream(input: StreamInput): StreamResult {
const fetch = providerFetch(input)
const current = statusWithFetch(input, fetch)
Expand Down
94 changes: 88 additions & 6 deletions engine/packages/opencode/test/session/llm-native.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -567,7 +567,7 @@ describe("session.llm-native.request", () => {
provider: { ...providerInfo, options: {} },
auth: undefined,
}),
).toEqual({ type: "unsupported", reason: "API key is not configured" })
).toEqual({ type: "unsupported", reason: "API key is not configured", used_fallback: true })

expect(
LLMNativeRuntime.status({
Expand All @@ -580,15 +580,15 @@ describe("session.llm-native.request", () => {
provider: catalogProvider({ id: "ollama", apiKey: "ollama" }),
auth: undefined,
}),
).toEqual({ type: "unsupported", reason: "base URL is not configured" })
).toEqual({ type: "unsupported", reason: "base URL is not configured", used_fallback: true })

expect(
LLMNativeRuntime.status({
model: catalogModel({ providerID: "azure", npm: "@ai-sdk/azure", url: "" }),
provider: catalogProvider({ id: "azure", apiKey: "test-azure-key" }),
auth: undefined,
}),
).toEqual({ type: "unsupported", reason: "base URL is not configured" })
).toEqual({ type: "unsupported", reason: "base URL is not configured", used_fallback: true })

expect(
LLMNativeRuntime.status({
Expand All @@ -599,14 +599,92 @@ describe("session.llm-native.request", () => {
).toMatchObject({ type: "unsupported" })
})

test("run result records used_fallback false for native and true for AI SDK fallback", () => {
// Source of truth for caseforge/downstream: native gate emits used_fallback so
// callers never invent the value. Native path → false; AI SDK fallback → true.
const native = LLMNativeRuntime.status({
model: baseModel,
provider: providerInfo,
auth: undefined,
})
expect(native).toMatchObject({ type: "supported", used_fallback: false })
expect(typeof (native as { used_fallback?: unknown }).used_fallback).toBe("boolean")

const missingKey = LLMNativeRuntime.status({
model: baseModel,
provider: { ...providerInfo, options: {} },
auth: undefined,
})
expect(missingKey).toEqual({
type: "unsupported",
reason: "API key is not configured",
used_fallback: true,
})

const missingBaseURL = LLMNativeRuntime.status({
model: catalogModel({
providerID: "ollama",
npm: "@ai-sdk/openai-compatible",
url: "",
id: "llama3.2",
}),
provider: catalogProvider({ id: "ollama", apiKey: "ollama" }),
auth: undefined,
})
expect(missingBaseURL).toMatchObject({ type: "unsupported", used_fallback: true })

const unknownPackage = LLMNativeRuntime.status({
model: catalogModel({ providerID: "custom", npm: "unknown-provider" }),
provider: catalogProvider({ id: "custom", apiKey: "key" }),
auth: undefined,
})
expect(unknownPackage).toMatchObject({ type: "unsupported", used_fallback: true })

// stream() must carry the same field on both supported and unsupported results.
const client = {
stream: () => Stream.empty,
} as unknown as LLMClientShape
const supportedStream = LLMNativeRuntime.stream({
model: baseModel,
provider: providerInfo,
auth: undefined,
llmClient: client,
messages: [],
tools: {},
headers: {},
abort: new AbortController().signal,
})
expect(supportedStream).toMatchObject({ type: "supported", used_fallback: false })

const fallbackStream = LLMNativeRuntime.stream({
model: baseModel,
provider: { ...providerInfo, options: {} },
auth: undefined,
llmClient: client,
messages: [],
tools: {},
headers: {},
abort: new AbortController().signal,
})
expect(fallbackStream).toMatchObject({
type: "unsupported",
reason: "API key is not configured",
used_fallback: true,
})
})

test("falls back for OAuth and custom-fetch providers that are AI-SDK contracts", () => {
expect(
LLMNativeRuntime.status({
model: baseModel,
provider: providerInfo,
auth: { type: "oauth", refresh: "refresh", access: "access", expires: 1 },
}),
).toEqual({ type: "unsupported", reason: "OAuth auth requires a provider fetch override" })
).toEqual({
type: "unsupported",
reason: "OAuth auth requires a provider fetch override",
used_fallback: true,
})

// OpenAI OAuth with the codex plugin fetch override can stay on the native path.
const dummyFetch = Object.assign(async () => new Response(), {
Expand All @@ -619,7 +697,7 @@ describe("session.llm-native.request", () => {
provider: { ...providerInfo, options: { apiKey: OAUTH_DUMMY_KEY, fetch: dummyFetch } },
auth: { type: "oauth", refresh: "refresh", access: "access", expires: 1 },
}),
).toMatchObject({ type: "supported", apiKey: OAUTH_DUMMY_KEY })
).toMatchObject({ type: "supported", apiKey: OAUTH_DUMMY_KEY, used_fallback: false })

// xAI OAuth intentionally stays on AI SDK: plugin fetch owns refresh + bearer injection.
expect(
Expand All @@ -632,7 +710,11 @@ describe("session.llm-native.request", () => {
}),
auth: { type: "oauth", refresh: "refresh", access: "access", expires: 1 },
}),
).toEqual({ type: "unsupported", reason: "xAI OAuth uses AI SDK plugin fetch override" })
).toEqual({
type: "unsupported",
reason: "xAI OAuth uses AI SDK plugin fetch override",
used_fallback: true,
})
})

test("enables native runtime for Anthropic API-key models", () => {
Expand Down
Loading