Add native LLM core foundation#24712
Open
kitlangton wants to merge 99 commits intodevfrom
Open
Conversation
This was referenced Apr 29, 2026
- Structurally match recorded requests by canonical JSON so non-deterministic field ordering doesn't break replay. - Pluggable header allow-list and body redaction hook on the record/replay layer, so adapters with non-default auth (Anthropic, Bedrock) can plug in without touching this file. - Move the cassette-name dedupe set inside recordedTests() so two describe files using different prefixes can run in parallel. - Replace inline SSE template literals and per-file HTTP layers with shared test/lib helpers (sseEvents, fixedResponse, dynamicResponse, truncatedStream). - Tighten recorded-test assertions to exact text and usage so adapter parser regressions surface immediately instead of passing fuzzy length>0 checks. - Add cancellation and mid-stream transport-error tests for the OpenAI Chat adapter. - Add cross-phase patch tests that verify each phase sees an updated PatchContext and that same-order patches sort deterministically by id.
- shared sse helper now expects Effectful decodeChunk and process callbacks, so adapter parsers can be Effect.gen and yield typed ProviderChunkError instead of throwing across the sync mapAccum boundary. - parseJson returns Effect<unknown, ProviderChunkError> via Effect.try, matching the package style guide on yieldable errors. - OpenAI Chat finalizes accumulated tool inputs eagerly when finish_reason arrives, surfacing JSON parse failures at the boundary instead of at halt. onHalt stays sync and just emits from state. - generate's runFold reducer now mutates the accumulator instead of reallocating the events array on every chunk, dropping O(n^2) growth on long streams.
Gemini rejects integer enums, dangling required fields, untyped arrays, and object keywords on scalar schemas. The sanitizer was previously a divergent copy in OpenCode; this lands it in the package as a tool-schema patch with deterministic tests and selects it for Gemini-protocol or Gemini-named models. Also tightens the Gemini test suite: covers tool-choice none, drops the tool-input-delta assertion that Gemini does not actually emit, and confirms total usage stays undefined when only thoughtsTokenCount arrives.
…integration Updates the AGENTS.md TODO list: - mark Responses, Anthropic, and Gemini adapter coverage as done - mark the Gemini schema sanitizer port as done - add concrete next-step items for OpenCode integration: ModelRef bridge, request bridge, provider-quirk patches, request/stream parity tests, and a flagged rollout against existing session/llm.test.ts cases - add OpenAI-compatible Chat, Bedrock Converse, and Vertex routing as outstanding adapter/dispatch decisions
Every adapter's parse already produces LLMEvents (via the process callback in the shared sse helper), and every raise was Stream.make(event). The Chunk type parameter, the raise field, the RaiseState interface, and the Stream.flatMap raise step in client.stream were all pure overhead. - Adapter contract shrinks from <Draft, Target, Chunk> to <Draft, Target>. - All four adapters drop their raise: (event) => Stream.make(event) line. - client.stream skips the no-op flatMap. - AGENTS.md adapter section reflects the simpler contract.
Per the package style guide, sync if/return functions that need to fail should yield the error directly via Effect.gen rather than ladder Effect.fail / Effect.succeed across every branch. Touches all four adapters' tool-choice lowering. The naming-required validation now reads as 'guard, then return' rather than embedded in a chain of monadic returns. Behavior unchanged.
Locks down the error contract before OpenCode integration: - mid-stream provider errors (Anthropic 'event: error', OpenAI Responses 'type: error') surface as 'provider-error' LLMEvents - HTTP 4xx responses fail with ProviderRequestError before stream parsing begins (the executor contract) Anthropic already had both. Adds: - OpenAI Responses: provider-error fixture, code-fallback fixture, HTTP 400 - OpenAI Chat: HTTP 400 sad path - AGENTS.md TODO refreshed; live recordings of provider errors still pending
Schema-first, Effect-first tool loop:
- 'tool({ description, parameters, success, execute })' constructs a fully
typed Tool. parameters and success are Effect Schemas; execute is typed
against them and returns Effect<Success, ToolFailure>. Handler dependencies
are closed over at construction time so the runtime never sees per-tool
services.
- 'ToolRuntime.run(client, { request, tools, maxSteps?, stopWhen? })' streams
the model, decodes tool-call inputs against parameters, dispatches to the
matching handler, encodes results against success, emits tool-result events,
appends assistant + tool messages, and re-streams. Stops on non-tool-calls
finish, maxSteps, or stopWhen.
- Three recoverable error paths emit tool-error events so the model can
self-correct: unknown tool name, input fails parameters Schema, handler
returns ToolFailure. Defects fail the stream.
- 'ToolFailure' added to the schema and exported as the single forced error
channel for handlers.
- Tool definitions on the LLMRequest are derived via toJsonSchemaDocument so
consumers don't write JSON Schema by hand.
8 deterministic fixture tests cover the loop, errors, maxSteps, stopWhen, and
parallel tool calls in one step.
Introduces the four orthogonal axes that an LLM adapter is composed of: - Protocol — semantic API contract (lowering, validation, encoding, parsing). Examples: OpenAI Chat, Anthropic Messages, Bedrock Converse. - Endpoint — URL construction (baseURL + path + query params). - Auth — per-request transport authentication. Defaults to passthrough for adapters whose auth header is baked into model.headers. - Framing — byte stream to frames (SSE today; AWS event stream next). Adds Adapter.fromProtocol(...) which composes these into the existing AdapterDefinition shape so LLMClient.make(...) and the runtime registry do not change. Existing adapters keep working through Adapter.define until they migrate one at a time.
Extracts OpenAIChat.protocol so that: - openai-chat is now a four-line Adapter.fromProtocol composition over the protocol, the OpenAI base URL, default passthrough auth, and SSE framing. - openai-compatible-chat reuses OpenAIChat.protocol verbatim. The whole adapter is one Adapter.fromProtocol call that pins protocolId to openai-compatible-chat and requires a caller-supplied baseURL. Bug fixes in OpenAIChat.protocol now propagate to DeepSeek, TogetherAI, Cerebras, Baseten, Fireworks, DeepInfra, and any future OpenAI-compatible deployment without touching their files. Recorded replay byte-identical.
Extracts a Protocol implementation per provider and wires the adapter through Adapter.fromProtocol with explicit Endpoint, Auth, and Framing: - OpenAI Responses — Endpoint.baseURL with /responses path. - Anthropic Messages — adds anthropic-version header via the headers slot. - Gemini — endpoint embeds the model id and pins ?alt=sse at the URL level. - Bedrock Converse — keeps SigV4-or-Bearer auth as a typed Auth function; AWS event-stream framing is a typed Framing value alongside the protocol; Endpoint.baseURL gains a function-typed default so the URL host can carry the per-request region. Recorded replay byte-identical across all six adapters; full provider suite 83 pass, full llm suite 122 pass, opencode typecheck clean.
Updates the AGENTS.md adapter section to describe the four orthogonal axes that make up an adapter today (Protocol + Endpoint + Auth + Framing) and the canonical Adapter.fromProtocol composition. Adds a folder layout overview so the dependency direction (provider/* imports protocol/auth/ endpoint/framing, never the other way) is visible.
After migration to Adapter.fromProtocol, the sse() convenience wrapper and withQuery() URL builder are no longer called anywhere — Framing.sse and Endpoint.baseURL handle their responsibilities directly. Also inlines two exported-but-unused test constants (helloPrompt, weatherPrompt) per style guide.
…nstants Per style guide, single-use values should be inlined. Each adapter had a module-private constant used exactly once in its Adapter.fromProtocol call. Inlining removes 5 named constants (4 DEFAULT_BASE_URL + 1 defaultBaseURL + ANTHROPIC_VERSION) without loss of clarity — the string literal appears at the point of use.
Removes the provider field from the five migrated Adapter.fromProtocol calls. Setting provider scopes the adapter in the registry so requests must use the same provider id, which broke session/llm-native tests that build models with provider 'amazon-bedrock' against the bedrock-converse adapter. Adapters should stay protocol-only by default and only set provider when the deployment is genuinely scoped (e.g. an Azure-only adapter that does not work for native OpenAI). Restoring the original protocol-only registration.
Both helpers had the same shape: read `request.model.apiKey`, no-op if absent, otherwise merge a one-key header object. Lift that into a tiny `fromApiKey(from)` helper and define both in terms of it. The public surface (`Auth.bearer`, `Auth.apiKeyHeader`) is unchanged.
Add an optional `apiKey` field to `ModelRef` so authentication is no
longer baked into `model.headers` at construction time. Each provider
adapter now passes an `Auth` to `Adapter.fromProtocol` that reads
`request.model.apiKey` per request:
- OpenAI Chat / Responses / OpenAI-compatible Chat: `Auth.bearer`
- Anthropic Messages: `Auth.apiKeyHeader("x-api-key")`
- Gemini: `Auth.apiKeyHeader("x-goog-api-key")`
- Bedrock Converse: custom auth that uses `apiKey` for Bearer auth
and falls back to SigV4 with AWS credentials
The `model()` constructors no longer fold the API key into
`model.headers`. The OpenCode bridge sets `apiKey` directly instead of
building auth headers via the now-deleted `authHeader` helper. Test
assertions move from `headers: { authorization: "Bearer ..." }` to
`apiKey: "..."`.
After the apiKey migration, every adapter explicitly specified `auth`, and three of them (OpenAI Chat, OpenAI Responses, OpenAI-compatible Chat) all wrote `auth: Auth.bearer`. `Auth.bearer` is a no-op when `model.apiKey` is unset, so making it the default is strictly safer than the previous `Auth.passthrough` default — bearer-style adapters drop their explicit `auth` line, and adapters that need a different scheme opt out via `Auth.apiKeyHeader(...)` (Anthropic, Gemini) or a custom `Auth` (Bedrock SigV4 + Bearer). Update doc comments on `fromProtocol.auth`, `Auth` type, and `packages/llm/AGENTS.md` to reflect the new default.
The two paths are independent: `model.apiKey` produces a synchronous Bearer auth, while AWS credentials need an effectful sigv4 sign. Hoist the bearer path out of `Effect.gen` and reuse `Auth.bearer` directly, keeping the SigV4 path as a focused `Effect.gen` that owns the credential lookup, signing, and header merge. Inlines the now single-use `headersForSigning` and `signed` setup.
Promotes queryParams to a first-class ModelRef field used by Endpoint.baseURL, so deployment-level URL query params (Azure api-version, OpenAI-compatible provider knobs) live in a typed home instead of an opaque `native` bag. Also removes write-only dead fields from `native`: - openaiCompatibleProvider (set by family helper, never read) - opencodeProviderID, opencodeModelID (set by opencode bridge + native session builder, never read) - npm (set by opencode bridge, never read) After this commit `model.native` only carries genuinely provider-specific opaque options that no other adapter cares about (Bedrock's aws_credentials + aws_region for SigV4). Drops the now-dead ProviderShared.queryParams helper. Updates AGENTS.md doc on native is implicit through the new schema JSDoc.
The commit that promoted queryParams to a typed ModelRef field updated the implementation but left two JSDoc/doc references pointing at the old model.native.queryParams path.
…ModelInput queryParams is now inherited from ModelInput (via ModelRef) after the typed-field promotion. The explicit re-declaration was dead weight.
url.toString() was called twice on the same URL object — once for auth and once for jsonPost. Convert to string immediately and reuse.
The optional 'provider' field on Adapter / AdapterInput / FromProtocolInput
existed as a registry filter: requests with a different model.provider could
not find adapters that set it. After the four-axis migration no adapter
needed it (and an earlier pass removed it from the five migrated providers
because setting it broke session/llm-native tests).
Drop the field entirely and collapse the registry to a single-tier protocol
lookup. If a future deployment genuinely needs to be scoped (e.g. an
Azure-only OpenAI Responses adapter), reintroduce as 'scopedTo' with an
explicit name. Solve when needed, not before.
Also drops the test that exercised the now-removed two-tier lookup
('prefers provider-specific adapters over protocol fallbacks').
…compose
Two cleanups to make the adapter constructor surface honest about what is
canonical and what is an escape hatch:
- Adapter.compose existed to override pieces of an existing adapter, used
by OpenAI-compatible Chat before the four-axis migration. After the
migration nothing references it; OpenAI-compatible Chat composes via
fromProtocol({ protocol: OpenAIChat.protocol, ... }) instead. Delete
the function and its ComposeInput type.
- Adapter.define is the lower-level escape hatch for adapters whose
behavior genuinely cannot fit the Protocol/Endpoint/Auth/Framing model.
Its name implied it was the canonical entry point. Renamed to
Adapter.unsafe so the four-axis Adapter.fromProtocol(...) reads as the
obvious primary path and the escape hatch carries its escape semantics
in its name.
Updated test fixtures in adapter.test.ts and the AGENTS.md guidance.
After the auth-axis migration, the OpenCode bridge consults this enum solely to decide whether to read `provider.key` and stamp it on `model.apiKey`. The bearer / anthropic-api-key / google-api-key distinctions used to control which header the bridge wrote; that is now the adapter's Auth axis's job. Three of four variants were write-only after the migration. Collapse to: - 'key' — provider needs an API key - 'none' — provider does not (e.g. local) Updated all six provider resolvers and the resolver test fixtures.
Schema.toTaggedUnion('type') already provides LLMEvent.guards but uses
kebab-case bracket access (LLMEvent.guards['tool-call']). Adds an LLMEvent.is
namespace with camelCase aliases that delegate to the same guards, so
consumers can write events.filter(LLMEvent.is.toolCall) instead of
events.filter(LLMEvent.guards['tool-call']).
Migrated all callsites in src/llm.ts and the two test files for consistency.
LLMEvent.guards / .match / .cases / .isAnyOf remain available for callers
who want the Effect-canonical API.
LLMClient.prepare(request) returned a PreparedRequest with target: unknown. Callers building debug UIs / request previews / plan rendering had to cast target to the adapter's native shape at every read. Adds PreparedRequestOf<Target> in schema and a generic Target = unknown parameter on LLMClient.prepare so callers can opt in to a typed view: const prepared = yield* client.prepare<OpenAIChatTarget>(request) prepared.target.model // typed prepared.target.messages // typed The runtime payload is unchanged — the adapter still emits target: unknown and the consumer asserts the shape they expect from the configured adapter. The cast lives at the public boundary in adapter.ts; everything else stays honest about runtime types. Existing callers without the type argument still get target: unknown and nothing breaks. Test in openai-chat.test.ts proves the narrowing at the type level.
…is.* in AGENTS.md
kitlangton
force-pushed
the
llm-core-patch-api
branch
from
1d502b6 to
e9d84c6
Compare
kitlangton
commented
May 1, 2026
| @@ -396,6 +426,7 @@ | |||
| "@octokit/graphql": "9.0.2", | |||
| "@octokit/rest": "catalog:", | |||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
packages/llm, a native Effect-based LLM core with typed request/event schemas, provider adapters, patches, tool runtime, and recorded provider tests.OPENCODE_EXPERIMENTAL_LLM_NATIVE, while keeping the existing stream path as the default fallback.@opencode-ai/http-recorder.Safety
origin/dev.Testing
cd packages/opencode && bun run test test/provider/llm-bridge.test.ts test/session/llm-native.test.ts test/session/llm-native-events.test.ts test/session/llm-native-stream.test.tscd packages/opencode && bun typecheckcd packages/http-recorder && bun run testcd packages/http-recorder && bun typecheckcd packages/llm && bun run test test/provider/openai-compatible-chat.recorded.test.ts test/provider/anthropic-messages.recorded.test.ts test/provider/gemini.recorded.test.tscd packages/llm && bun typecheckbun turbo typecheck