Add x-session-affinity header for Fireworks AI prompt caching#23451
Open
richardanaya wants to merge 2 commits intoanomalyco:devfrom
Open
Add x-session-affinity header for Fireworks AI prompt caching#23451richardanaya wants to merge 2 commits intoanomalyco:devfrom
richardanaya wants to merge 2 commits intoanomalyco:devfrom
Conversation
Contributor
|
Hey! Your PR title Please update it to start with one of:
Where See CONTRIBUTING.md for details. |
github-actions
Bot
added
the
needs:compliance
richardanaya
force-pushed
the
dev
branch
3 times, most recently
from
465d9c2 to
7d77c3c
Compare
Fireworks AI uses per-replica in-memory caching that requires session affinity via the x-session-affinity HTTP header. Without this header, requests are routed to random replicas, causing cache misses even when the same session is reused. This fix adds the x-session-affinity header for Fireworks AI requests when a valid sessionId is provided in options. See: https://docs.fireworks.ai/guides/prompt-caching
github-actions
Bot
removed
the
needs:compliance
Contributor
|
Thanks for updating your PR! It now meets our contributing guidelines. 👍 |
Collaborator
|
Opencode already sends this header by default for all providers |
Collaborator
|
opencode/packages/opencode/src/session/llm.ts Line 378 in 6e01786 |
Collaborator
|
I just ran through http proxy, it is being sent through as expected |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fireworks AI uses per-replica in-memory caching that requires session affinity via the x-session-affinity HTTP header. Without this header, requests are routed to random replicas, causing cache misses even when the same session is reused.
This fix adds the x-session-affinity header for all Fireworks AI requests, using either a provided sessionId or a generated session identifier.
See: https://docs.fireworks.ai/guides/prompt-caching
Issue for this PR
Closes #23450
Type of change
What does this PR do?
Adds cacheing hints as per their documentation
If you paste a large clearly AI generated description here your PR may be IGNORED or CLOSED!
How did you verify your code works?
Tested that opencode could still run with
bun devagainst fireworks.Screenshots / recordings
N/A
Checklist
If you do not follow this template your PR will be automatically rejected.