diff --git a/AFFINITY.md b/AFFINITY.md new file mode 100644 index 000000000..eef04e2fb --- /dev/null +++ b/AFFINITY.md @@ -0,0 +1,146 @@ +# Client-carried affinity + +Floway can resolve one public model name or alias to several upstream/model +bindings. Client-carried affinity records which binding produced opaque +assistant state so a later turn can prefer that binding, or require it when +the state is not portable. + +Affinity is a gateway source membrane, not a protocol or a translation layer. +Ingress removes Floway metadata before interceptors, translators, and providers +run. Egress adds it only after events return to the client-facing protocol. + +## Envelope and framing + +Each API key has a hidden 256-bit `serverSecret` for gateway-private per-key +data. Normal key CRUD never exposes it; admin export/import preserves it. +Affinity derives an AES-256-GCM key with HKDF and encrypts: + +```ts +{ + version: 1, + origin?: 'raw' | 'base64' | 'base64url', + affinity: { + upstreamId: string, + modelId: string, + rules?: AliasRules, + upstreamItemId?: string, + syntheticItem?: true, + boundItem?: { + type: string, + upstreamItemId?: string, + contentHash: string, + }, + }, +} +``` + +Routing strength is not serialized. Ingress derives `prefer` or `force` from +the carrier's current protocol location. + +The carrier has no delimiter or magic prefix: + +```text +original bytes || IV[12] || ciphertext+tag || encryptedLength:u16be +``` + +The original bytes and a length-delimited protocol/slot domain are additional +authenticated data. Canonical Base64 and unpadded canonical Base64URL values +are decoded before the encrypted trailer is appended, so existing data is not +Base64-encoded twice. Other strings are stored as UTF-16 code units, preserving +even lone surrogates exactly. A synthetic carrier has no `origin` and no +original bytes. + +Authentication failure, an invalid frame, an unknown envelope, or another +key's carrier is foreign. Foreign values pass through byte-for-byte and add no +routing evidence, which allows nested Floway deployments to unwrap their own +layers independently. + +## Ingress and routing + +Ingress authenticates all recognized opaque fields and builds a request-local +candidate payload factory. Each attempt receives a fresh source payload: + +- ordinary/discardable blobs restore only for exact upstream, model, and + optional alias rules; +- force-state blobs restore for any candidate with the required upstream and + model, regardless of alias rules; +- incompatible owned blobs and authenticated synthetic elements are removed; +- foreign blobs remain unchanged. + +Every owned carrier also adds an exact-rules preference. The latest preferred +target that remains available moves first. Responses compaction and +program/program-output state additionally force upstream and model. Force +never narrows alias rules; an exact preferred rule variant still wins when it +is available. + +Responses expands `previous_response_id` and hydrates complete stored items +before affinity ingress. Persistence contains no separate routing record. + +## Egress layers + +Egress performs two independent operations: + +1. wrap every natural opaque/signature value with restoration metadata; +2. ensure the first logical assistant element has a carrier when no natural + value provides one. + +Chat Completions, Messages, and Responses do not buffer visible deltas for +affinity. Gemini deliberately delays one complete upstream event; the window +never grows beyond that event. + +### Chat Completions + +One choice is one logical element. `reasoning_opaque` is last-write-wins per +choice. Visible deltas pass through; Floway emits one wrapped natural or +synthetic opaque snapshot immediately before `finish_reason`, or immediately +before `[DONE]` when the upstream omits a finish reason. + +### Messages + +`signature_delta` is last-write-wins. Thinking text passes through and the +wrapped signature is emitted immediately before `content_block_stop`. +`redacted_thinking.data` is wrapped at block start. If the first block cannot +carry a blob, Floway inserts a `redacted_thinking` prefix at index zero and +shifts every original block index by one. + +### Gemini + +Gemini buffers at most one upstream event. Signature snapshots for each +same-event logical element are reduced to the latest value and placed on that +element's first content-bearing Part. Across events, a late signature can move +back only onto the immediately preceding buffered chunk. Empty text and +`thought` metadata alone do not make a Part content-bearing. Immediate +signature-only prefixes/trailers are moved onto adjacent content when the +one-event window can determine ownership. + +This adds one upstream event of latency. It deliberately favors direct Google +GenAI Chat compatibility and cannot repair a first-wins client when a natural +function signature arrives more than one continuation after the first chunk. +The evidence and exact client tradeoffs are recorded beside the +[Gemini affinity egress state machine](./packages/gateway/src/data-plane/chat/gemini/affinity/egress.ts). + +### Responses + +Natural carriers include top-level `encrypted_content`, program `fingerprint`, +and `agent_message.content[].encrypted_content`. A carrier-capable first item +without a natural value receives one at `output_item.done`; a program without +a natural fingerprint receives a synthetic fingerprint in the same slot. + +Other first items and `program_output` items receive an adjacent reasoning +prefix. Its `output_item.added` event is emitted immediately +without buffering the bound item's visible stream. Its encrypted +`output_item.done` is emitted immediately before the bound item's done event, +using the final upstream ID and a canonical SHA-256 content hash that excludes +the replaceable item ID. The hash also applies Codex's output-to-history +projection: message/function-call `status` and empty output-text `annotations` +or `logprobs` are omitted. Terminal output contains the same completed prefix. +Output indexes and sequence numbers include each inserted lifecycle event. + +On replay, the authenticated type and content hash must match the adjacent +item. A mismatch is a client input error. Preferred bindings restore IDs only +for exact optional rules; force-item bindings restore across rule variants on +the same upstream/model. Stored server-private payloads are re-keyed to the +restored wire ID before Responses interceptors run. + +Failed/error streams do not invent a missing final carrier. A prefix already +opened before an eventual failure cannot be retracted. diff --git a/AGENTS.md b/AGENTS.md index 655aa2bfa..f6c7c320d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -188,6 +188,15 @@ use declared subpath exports. Tests are co-located as `*_test.ts`; each package has its own `vitest.config.ts`, and the root config aggregates them through `test.projects`. +Client-carried affinity is a source-protocol membrane. Shared codec, routing, +and request context live under `data-plane/chat/shared/affinity`; each source +protocol owns its `affinity/ingress.ts` and `affinity/egress.ts`. Wire behavior +lives in `AFFINITY.md`, and candidate ordering lives in `RESOLUTION.md`. + +Native Responses persistence is independent from affinity. It stores complete +API-key-scoped items and snapshots for 30 days; HTTP `store: false` writes no +state, while WebSocket `store: false` is session-local. + Everything else — provider interfaces, request execution flow, interceptor shapes, translation pair layout, control-plane route surface, flag resolution, pricing — lives in the code and its comments. Read the relevant diff --git a/README.md b/README.md index e54d4829b..21b50afd5 100644 --- a/README.md +++ b/README.md @@ -137,9 +137,8 @@ Open the deployed URL (or `http://localhost:8788` for Node), log in with official Codex account login remains available for account-backed services. Import/export of upstreams, keys, and search config is in Settings. The -payload format is tied to the running deployment, so import only accepts a -file produced by a deployment at the same version — re-export from the -current deployment before importing. +current payload format is version 11 and is tied to the running deployment, so +import only accepts that exact version. Re-export before moving a deployment. ## Server Tools @@ -164,15 +163,21 @@ then both surfaces use that provider's alpha-search endpoint, while Messages search continues using the general provider. Passthrough failures are returned without falling back to another search backend. +## Client-carried Affinity + +Chat-shaped APIs carry encrypted per-key routing affinity inside their native +opaque reasoning/signature fields. Requests using this feature must continue +through the same Floway deployment and API key. See [AFFINITY.md](./AFFINITY.md) +for protocol placement and compatibility details. + ## Stateful Responses `/v1/responses` stores replayable Responses input and output items for API-key scoped HTTP requests. Clients can send `previous_response_id` to continue from a stored snapshot, or resend full input history; repeated full-history input is -deduplicated by content hash instead of stored again. HTTP `store: false` does -not create durable snapshots or input payload rows, but it keeps output item -metadata for routing; if a later `store: true` request echoes that item with a -full payload, the metadata row is filled in place. +deduplicated by content hash instead of stored again. Complete items and +snapshots expire 30 days after their latest snapshot reference. HTTP +`store: false` writes no state; affinity is carried independently by the client. The same endpoint accepts `GET` WebSocket upgrades for streaming Responses events. WebSocket `store: false` keeps replay state only inside the open diff --git a/RESOLUTION.md b/RESOLUTION.md index fc87164dd..a8ce5bfb4 100644 --- a/RESOLUTION.md +++ b/RESOLUTION.md @@ -394,15 +394,13 @@ Candidates are ordered before they reach dispatch: - Within a single upstream, the unprefixed branch precedes the prefixed one when both apply. -For Responses-shape inbound, the affinity walk -(`classifyResponsesItemAffinity`) adjusts the ordering by stored-item -affinity before dispatch sees it. It resolves stored ids from metadata, -rejects an `item_reference` whose durable payload is unavailable, and uses -the referenced row's actual item type to determine upstream affinity. The -candidate rewrite then bulk-loads the payloads it needs and replaces every -`item_reference` with the stored item before the upstream request is built. -The affinity walk never invents new candidates; it only narrows or re-orders -the list the resolver produced. +Chat-shaped ingress authenticates client-carried affinity before dispatch. +Ordinary carriers prefer the latest available candidate with the same upstream, +model, and optional alias rules. Non-portable state forces the recorded +upstream and model but never forces alias rules; an exact-rule preferred variant +still wins when available. Affinity only reorders or narrows candidates already +produced by resolution. Protocol placement and restoration are documented in +[AFFINITY.md](./AFFINITY.md). Serve dispatches the first candidate of the ordered list exactly once. The attempt's non-throwing result — an SSE-stream event handoff (chat) or @@ -428,3 +426,5 @@ internal-debug failure — is the request's final answer; an upstream retain both candidate paths instead of deduping. The unprefixed candidate precedes the prefix-stripped one in the ordered list, so it is the one dispatched. +- A missing preferred affinity target falls back normally. A missing forced + upstream/model target is an explicit error. diff --git a/TRANSLATION.md b/TRANSLATION.md index 283b66a69..3185129de 100644 --- a/TRANSLATION.md +++ b/TRANSLATION.md @@ -758,7 +758,9 @@ historical custom tool call items unchanged. order when later visible output arrives before earlier reasoning/tool output is complete. - Chat -> Messages stream translation keeps opaque-only reasoning in source - order and flushes pending final usage before `message_stop`. + order and flushes pending final usage before `message_stop`. Chat + `reasoning_opaque` and Messages `signature_delta` values are replacement + snapshots, not string fragments to concatenate. - Tool/function argument streams guard against infinite whitespace in generated arguments and emit an error rather than continuing a degenerate stream. @@ -773,6 +775,9 @@ historical custom tool call items unchanged. - Messages <-> Chat may still carry Anthropic opaque thinking through Chat `reasoning_opaque`, because that is a Messages/Chat compatibility surface and not a Responses encrypted-reasoning bridge. +- Floway affinity and native Responses persistence remain outside pure + translators; their source-boundary behavior is documented in + [AFFINITY.md](./AFFINITY.md). ## Standard OpenAI Field Policy diff --git a/apps/platform-node/src/migrate_test.ts b/apps/platform-node/src/migrate_test.ts index b3ef99c57..5e61a6ae8 100644 --- a/apps/platform-node/src/migrate_test.ts +++ b/apps/platform-node/src/migrate_test.ts @@ -32,6 +32,7 @@ test('applies all real migration files against a fresh sqlite', () => withTemp(a const colNames = apiKeyCols.results.map(r => r.name).toSorted(); assertEquals(colNames.includes('id'), true); assertEquals(colNames.includes('key'), true); + assertEquals(colNames.includes('server_secret'), true); // Every migration was recorded. const recorded = await db.prepare('SELECT COUNT(*) AS n FROM _migrations').first<{ n: number }>(); diff --git a/apps/web/src/components/settings/ExportSection.vue b/apps/web/src/components/settings/ExportSection.vue index 748110a48..b2d8a1583 100644 --- a/apps/web/src/components/settings/ExportSection.vue +++ b/apps/web/src/components/settings/ExportSection.vue @@ -47,7 +47,7 @@ const exportData = async () => {