Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
145 commits
Select commit Hold shift + click to select a range
da5d9b3
feat(gateway): add encrypted affinity blob codec
Menci Jul 15, 2026
eaba958
feat(gateway): identify exact affinity targets
Menci Jul 15, 2026
8d74536
feat(gateway): extract client-carried affinity at ingress
Menci Jul 15, 2026
b2ec186
feat(gateway): route chat sources by client affinity
Menci Jul 15, 2026
8f1cf61
fix(protocols): treat opaque reasoning as snapshots
Menci Jul 15, 2026
b648cb4
feat: add protocol affinity egress transforms
Menci Jul 15, 2026
2a9708c
feat(auth): persist per-key affinity secrets
Menci Jul 15, 2026
cdf4c76
test: await delayed affinity wrapping
Menci Jul 15, 2026
a5b7f51
style(auth): order affinity test imports
Menci Jul 15, 2026
9ea3670
feat(gateway): emit affinity on chat source streams
Menci Jul 15, 2026
91676d7
fix: defer mixed Gemini candidate finishes
Menci Jul 15, 2026
779be13
feat(responses): route hydrated state by client affinity
Menci Jul 15, 2026
24e3173
feat(responses): persist affinity-wrapped source output
Menci Jul 15, 2026
d76f3ad
fix(gateway): satisfy affinity target contracts
Menci Jul 15, 2026
d966656
refactor(responses): store only complete thirty-day state
Menci Jul 15, 2026
e0d0914
test(responses): replace metadata-affinity coverage
Menci Jul 15, 2026
ef52d6e
refactor(gateway): isolate translated Responses state
Menci Jul 15, 2026
6b17c3f
fix(responses): apply source output policy on WebSocket
Menci Jul 15, 2026
5e9034d
test(responses): await WebSocket source finalization
Menci Jul 15, 2026
d7b691b
fix(gemini): restore deferred signature placement
Menci Jul 15, 2026
3245080
test(provider): include candidate revision
Menci Jul 15, 2026
700bfd5
fix(gateway): narrow affinity metadata types
Menci Jul 15, 2026
0937f18
test(gateway): seed responder affinity context
Menci Jul 15, 2026
d06dae5
refactor(responses): require current payload descriptors
Menci Jul 15, 2026
d02c6f6
fix(responses): decode compressed payload bytes
Menci Jul 15, 2026
7fee007
docs: describe client-carried affinity architecture
Menci Jul 15, 2026
6a07a96
test(affinity): exercise a complete client round trip
Menci Jul 15, 2026
8ce9cec
style(gateway): normalize affinity imports
Menci Jul 15, 2026
66b8142
fix(affinity): finalize carriers from authoritative terminals
Menci Jul 15, 2026
33062df
refactor(affinity): tighten final-state invariants
Menci Jul 15, 2026
b7a533a
fix(affinity): satisfy tightened contracts
Menci Jul 15, 2026
631b4ef
refactor(affinity): separate state and enforce stream truth
Menci Jul 15, 2026
ecec352
docs: bind affinity carriers to their wire domains
Menci Jul 15, 2026
8e64c82
style(gateway): order cleanup imports
Menci Jul 15, 2026
027918e
fix(gemini): accept natural completion after all finish events
Menci Jul 15, 2026
67ee3e6
fix(affinity): preserve existing response state
Menci Jul 15, 2026
b56a9d5
fix(affinity): validate server secrets synchronously
Menci Jul 15, 2026
c869593
test(responses): use declared migration database API
Menci Jul 15, 2026
b62665f
test(sql): declare parameterized sql.js calls
Menci Jul 15, 2026
cc6929b
refactor(gateway): place affinity under shared chat concerns
Menci Jul 15, 2026
b32e61a
fix(gateway): resolve shared affinity imports
Menci Jul 15, 2026
75b0991
refactor(affinity): colocate protocol membranes
Menci Jul 15, 2026
694ac63
refactor(affinity): derive routing strength at ingress
Menci Jul 15, 2026
4eb9b43
feat(messages): anchor affinity in the first content block
Menci Jul 15, 2026
a21791a
feat(responses): anchor affinity in the first output item
Menci Jul 15, 2026
72357ca
fix(responses): derive synthetic prefix lazily
Menci Jul 15, 2026
4a23908
feat(gemini): use one-event affinity lookahead
Menci Jul 15, 2026
74a113b
test(gemini): await lookahead finalization
Menci Jul 15, 2026
5071304
refactor(gemini): retire deferred signature placement
Menci Jul 15, 2026
675bddd
docs: define structural affinity anchors
Menci Jul 15, 2026
27d8df1
style(affinity): normalize final imports and fixtures
Menci Jul 15, 2026
c707fa7
refactor(affinity): tighten stream invariants
Menci Jul 15, 2026
97eee83
refactor(affinity): apply final cleanup findings
Menci Jul 15, 2026
f66667c
fix(affinity): encode cleanup invariants in types
Menci Jul 15, 2026
71d3579
refactor(responses): collapse dead state plumbing
Menci Jul 15, 2026
1f9fb9f
fix(gemini): slide lookahead to the element boundary
Menci Jul 16, 2026
42d2c5e
style(gemini): format sliding lookahead fixtures
Menci Jul 16, 2026
c536a65
refactor(affinity): inline carrier domains
Menci Jul 16, 2026
1ddbb25
docs: keep entry documents concise
Menci Jul 16, 2026
9c1b780
docs(responses): record legacy affinity loss
Menci Jul 16, 2026
d5e43ea
docs: keep migration policy out of translation
Menci Jul 16, 2026
2c7d042
refactor(affinity): consolidate shared core
Menci Jul 16, 2026
5d02cec
fix(affinity): preserve stream extras and exact raw strings
Menci Jul 16, 2026
c4dd036
fix(responses): carry force-item provenance
Menci Jul 16, 2026
3d5d28b
fix(gemini): normalize signature snapshots per element
Menci Jul 16, 2026
e743881
test(responses): cover natural program fingerprint wrapping
Menci Jul 16, 2026
630b454
docs: distinguish force from exact rule preference
Menci Jul 16, 2026
9422510
test(responses): verify stored force-item provenance round trip
Menci Jul 16, 2026
7a36672
style(affinity): normalize consolidated imports
Menci Jul 16, 2026
e28aaed
fix(affinity): restore consolidated runtime imports
Menci Jul 16, 2026
b6bac5e
test(responses): model client output replay explicitly
Menci Jul 16, 2026
3edda5a
docs: record affinity compatibility boundaries
Menci Jul 16, 2026
26842a0
style(affinity): mark ingress codec imports as type-only
Menci Jul 16, 2026
6f82337
fix(gemini): normalize every affinity signature group
Menci Jul 16, 2026
826cec5
fix(responses): authenticate bound affinity items
Menci Jul 16, 2026
e69bce1
fix(responses): narrow validated item bindings
Menci Jul 16, 2026
57821c1
fix(gemini): preserve signature ownership across lookahead
Menci Jul 16, 2026
117c8dc
test(gemini): cover empty metadata carriers
Menci Jul 16, 2026
6862311
fix(responses): close bound carriers with final items
Menci Jul 16, 2026
3de8fbb
fix(responses): align affinity restoration with item semantics
Menci Jul 16, 2026
a535182
fix(responses): retain owned binding refinements
Menci Jul 16, 2026
e28afd1
test(responses): replay private state after affinity ID restore
Menci Jul 16, 2026
f9babdf
test(responses): render invalid affinity bindings as input errors
Menci Jul 16, 2026
411d2e9
test(responses): round-trip bindings from final item snapshots
Menci Jul 16, 2026
434b2d3
docs: separate affinity from translation
Menci Jul 16, 2026
1054ba7
docs: keep translation scoped to protocol mapping
Menci Jul 16, 2026
a89837b
refactor(affinity): remove affinity-specific secret residue
Menci Jul 16, 2026
e9ea963
refactor(responses): close carrier lifecycle cleanup findings
Menci Jul 16, 2026
b99a41a
fix(translate): keep available signatures with thinking
Menci Jul 16, 2026
123d544
refactor(gemini): clarify bounded signature placement
Menci Jul 16, 2026
3908eb5
fix(responses): normalize nullable store controls
Menci Jul 16, 2026
38ad2ef
fix(responses): synthesize missing program fingerprints
Menci Jul 16, 2026
b757702
fix(affinity): preserve extension fields across split frames
Menci Jul 16, 2026
4abb417
fix(messages): narrow retained signature events
Menci Jul 16, 2026
22f50cf
fix(gemini): suppress emptied signature events
Menci Jul 16, 2026
f4fc99b
fix(translate): retain first readable reasoning independently
Menci Jul 16, 2026
e9cbaba
fix(responses): synthesize carriers only at successful close
Menci Jul 16, 2026
512fa31
fix(responses): retain gateway IDs without persistence
Menci Jul 16, 2026
ab486b4
fix(responses): delete payload references before files
Menci Jul 16, 2026
eec7af4
fix(responses): record WebSocket input errors consistently
Menci Jul 16, 2026
0c0df22
refactor(gemini): merge suppressed candidate metadata canonically
Menci Jul 16, 2026
13c2665
refactor(translate): make reasoning closure explicit
Menci Jul 16, 2026
7ae0c77
refactor(affinity): centralize private target and secret helpers
Menci Jul 16, 2026
37f3a59
refactor(protocols): expose malformed Chat streams
Menci Jul 16, 2026
cba576b
refactor(responses): name the client output membrane
Menci Jul 16, 2026
f1b1a39
refactor(responses): name shared state lifetime
Menci Jul 16, 2026
b58350f
fix(gemini): preserve native empty stream structures
Menci Jul 16, 2026
7cf7079
fix(responses): keep lifecycle identity stable
Menci Jul 16, 2026
7cfc197
fix(gateway): finalize model-enumeration failures
Menci Jul 16, 2026
a58d16b
refactor(responses): separate client identity from persistence
Menci Jul 16, 2026
6ad923f
fix(translate): preserve opaque reasoning source order
Menci Jul 16, 2026
e1994af
fix(messages): preserve valid block and message boundaries
Menci Jul 16, 2026
3f30242
fix(responses): validate complete lifecycle state
Menci Jul 16, 2026
3e68ae4
test(responses): distinguish foreign and blob-less force state
Menci Jul 16, 2026
b10a7fc
fix(translate): keep interleaved tool arguments on their block
Menci Jul 16, 2026
f393fb1
refactor(responses): unify client lifecycle finalization
Menci Jul 16, 2026
8054395
docs: align stream placement boundaries
Menci Jul 16, 2026
cefd349
fix(api-keys): expose non-key constraint failures
Menci Jul 16, 2026
ad10ae2
fix(responses): discover and validate replay state before dispatch
Menci Jul 16, 2026
90c1628
test(responses): type context compaction fixtures explicitly
Menci Jul 16, 2026
9659e57
fix(translate): serialize parallel tool drafts
Menci Jul 16, 2026
06608c2
test(translate): align buffered tool event snapshots
Menci Jul 16, 2026
3357efb
fix(translate): retain repeated tool draft identity
Menci Jul 16, 2026
f318100
fix(translate): accumulate split parallel tool identities
Menci Jul 16, 2026
d1b038d
fix(responses): bind and restore id-less items
Menci Jul 16, 2026
4888832
fix(responses): retain only live snapshot state
Menci Jul 16, 2026
362aa4b
test(responses): expect snapshot lifetime refresh
Menci Jul 16, 2026
8c43625
fix(responses): refresh all reachable snapshot state
Menci Jul 16, 2026
f2cb6d5
test(responses): fill large payload randomness in bounded chunks
Menci Jul 16, 2026
4c13cbf
refactor(affinity): name exact-target framing explicitly
Menci Jul 16, 2026
c368d7f
fix(translate): retain deferred segment order
Menci Jul 16, 2026
2eda60b
fix(translate): skip empty deferred flushes
Menci Jul 16, 2026
eaadb22
fix(responses): refresh referenced snapshot lifetime
Menci Jul 16, 2026
426080e
docs: clarify deferred stream ordering
Menci Jul 16, 2026
b556588
fix(responses): refresh snapshot graphs atomically
Menci Jul 16, 2026
e53984c
fix(translate): preserve split and parallel tool boundaries
Menci Jul 16, 2026
6cb241d
fix(responses): canonicalize compaction carrier domains
Menci Jul 16, 2026
96c920c
fix(responses): normalize client-carried item projections
Menci Jul 16, 2026
acca73f
refactor(streams): clarify bounded ordering and refresh sequencing
Menci Jul 16, 2026
fc41a5c
docs: describe canonical Responses binding hashes
Menci Jul 16, 2026
64d31e8
docs(gemini): record id-less call boundary decision
Menci Jul 16, 2026
4fbd814
fix(migration): preserve safe Responses graph lifetime
Menci Jul 16, 2026
47e86e2
test(migration): order rows by preserved lifetime
Menci Jul 16, 2026
1d60c5d
fix(translate): retain latest streamed reasoning state
Menci Jul 16, 2026
7593389
refactor(responses): canonicalize stable item metadata
Menci Jul 16, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
146 changes: 146 additions & 0 deletions AFFINITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# Client-carried affinity

Floway can resolve one public model name or alias to several upstream/model
bindings. Client-carried affinity records which binding produced opaque
assistant state so a later turn can prefer that binding, or require it when
the state is not portable.

Affinity is a gateway source membrane, not a protocol or a translation layer.
Ingress removes Floway metadata before interceptors, translators, and providers
run. Egress adds it only after events return to the client-facing protocol.

## Envelope and framing

Each API key has a hidden 256-bit `serverSecret` for gateway-private per-key
data. Normal key CRUD never exposes it; admin export/import preserves it.
Affinity derives an AES-256-GCM key with HKDF and encrypts:

```ts
{
version: 1,
origin?: 'raw' | 'base64' | 'base64url',
affinity: {
upstreamId: string,
modelId: string,
rules?: AliasRules,
upstreamItemId?: string,
syntheticItem?: true,
boundItem?: {
type: string,
upstreamItemId?: string,
contentHash: string,
},
},
}
```

Routing strength is not serialized. Ingress derives `prefer` or `force` from
the carrier's current protocol location.

The carrier has no delimiter or magic prefix:

```text
original bytes || IV[12] || ciphertext+tag || encryptedLength:u16be
```

The original bytes and a length-delimited protocol/slot domain are additional
authenticated data. Canonical Base64 and unpadded canonical Base64URL values
are decoded before the encrypted trailer is appended, so existing data is not
Base64-encoded twice. Other strings are stored as UTF-16 code units, preserving
even lone surrogates exactly. A synthetic carrier has no `origin` and no
original bytes.

Authentication failure, an invalid frame, an unknown envelope, or another
key's carrier is foreign. Foreign values pass through byte-for-byte and add no
routing evidence, which allows nested Floway deployments to unwrap their own
layers independently.

## Ingress and routing

Ingress authenticates all recognized opaque fields and builds a request-local
candidate payload factory. Each attempt receives a fresh source payload:

- ordinary/discardable blobs restore only for exact upstream, model, and
optional alias rules;
- force-state blobs restore for any candidate with the required upstream and
model, regardless of alias rules;
- incompatible owned blobs and authenticated synthetic elements are removed;
- foreign blobs remain unchanged.

Every owned carrier also adds an exact-rules preference. The latest preferred
target that remains available moves first. Responses compaction and
program/program-output state additionally force upstream and model. Force
never narrows alias rules; an exact preferred rule variant still wins when it
is available.

Responses expands `previous_response_id` and hydrates complete stored items
before affinity ingress. Persistence contains no separate routing record.

## Egress layers

Egress performs two independent operations:

1. wrap every natural opaque/signature value with restoration metadata;
2. ensure the first logical assistant element has a carrier when no natural
value provides one.

Chat Completions, Messages, and Responses do not buffer visible deltas for
affinity. Gemini deliberately delays one complete upstream event; the window
never grows beyond that event.

### Chat Completions

One choice is one logical element. `reasoning_opaque` is last-write-wins per
choice. Visible deltas pass through; Floway emits one wrapped natural or
synthetic opaque snapshot immediately before `finish_reason`, or immediately
before `[DONE]` when the upstream omits a finish reason.

### Messages

`signature_delta` is last-write-wins. Thinking text passes through and the
wrapped signature is emitted immediately before `content_block_stop`.
`redacted_thinking.data` is wrapped at block start. If the first block cannot
carry a blob, Floway inserts a `redacted_thinking` prefix at index zero and
shifts every original block index by one.

### Gemini

Gemini buffers at most one upstream event. Signature snapshots for each
same-event logical element are reduced to the latest value and placed on that
element's first content-bearing Part. Across events, a late signature can move
back only onto the immediately preceding buffered chunk. Empty text and
`thought` metadata alone do not make a Part content-bearing. Immediate
signature-only prefixes/trailers are moved onto adjacent content when the
one-event window can determine ownership.

This adds one upstream event of latency. It deliberately favors direct Google
GenAI Chat compatibility and cannot repair a first-wins client when a natural
function signature arrives more than one continuation after the first chunk.
The evidence and exact client tradeoffs are recorded beside the
[Gemini affinity egress state machine](./packages/gateway/src/data-plane/chat/gemini/affinity/egress.ts).

### Responses

Natural carriers include top-level `encrypted_content`, program `fingerprint`,
and `agent_message.content[].encrypted_content`. A carrier-capable first item
without a natural value receives one at `output_item.done`; a program without
a natural fingerprint receives a synthetic fingerprint in the same slot.

Other first items and `program_output` items receive an adjacent reasoning
prefix. Its `output_item.added` event is emitted immediately
without buffering the bound item's visible stream. Its encrypted
`output_item.done` is emitted immediately before the bound item's done event,
using the final upstream ID and a canonical SHA-256 content hash that excludes
the replaceable item ID. The hash also applies Codex's output-to-history
projection: message/function-call `status` and empty output-text `annotations`
or `logprobs` are omitted. Terminal output contains the same completed prefix.
Output indexes and sequence numbers include each inserted lifecycle event.

On replay, the authenticated type and content hash must match the adjacent
item. A mismatch is a client input error. Preferred bindings restore IDs only
for exact optional rules; force-item bindings restore across rule variants on
the same upstream/model. Stored server-private payloads are re-keyed to the
restored wire ID before Responses interceptors run.

Failed/error streams do not invent a missing final carrier. A prefix already
opened before an eventual failure cannot be retracted.
Loading