feat: streaming support for coding agents (Claude Code & Codex) by Davidnet · Pull Request #586 · dataiku/kiji-proxy

Davidnet · 2026-07-02T21:42:13Z

Summary

Add SSE streaming support to the proxy so coding agents (Claude Code, Codex) work through it
New streaming pipeline (proxy/streaming.go) with provider-specific codecs for Anthropic (codec_anthropic.go) and OpenAI (codec_openai.go)
Record dashboard metrics for streamed responses
Docs: new docs/09-coding-agents.md guide and README updates

Main work was provided in #532

…g tests The rebase onto main brought in the dashboard metrics added in #547, which are recorded after the response is written. The SSE streaming path returns early, so streamed requests (Claude Code, Codex) were invisible to the dashboard; record them there too, with the request-masking time as the proxy-overhead latency (response restoration is interleaved with the upstream stream). Also adds the test coverage the streaming feature landed without: the SSE engine end-to-end over a wire (chunked framing, header rewrite, PII restore), both provider codecs (placeholder split across deltas, tool-call argument restore, per-channel carry buffers, passthrough events), stream sniffing, Codex host routing, and chatgpt.com interception gating.

shepherd.js is a frontend dependency already declared in src/frontend/package.json; the root entry was added accidentally. Regenerating the lockfile also syncs the recorded src/frontend version with main (1.4.0 -> 1.4.1).

github-actions · 2026-07-02T21:42:26Z

Warning

This PR touches 3+ distinct areas of the codebase.

Consider splitting into smaller, focused PRs — each covering a single semantic type.
This makes reviews easier and keeps the git history clean.

Categories found:

docs:

README.md
docs/09-coding-agents.md
docs/README.md

code:

src/backend/config/config.go
src/backend/processor/response.go
src/backend/providers/anthropic.go
src/backend/providers/openai.go
src/backend/providers/provider.go
src/backend/proxy/codec_anthropic.go
src/backend/proxy/codec_openai.go
src/backend/proxy/cors.go
src/backend/proxy/handler.go
src/backend/proxy/passthrough.go
src/backend/proxy/router.go
src/backend/proxy/streaming.go
src/backend/proxy/transparent.go
src/backend/proxy/transparent_factory.go

chore:

package-lock.json

test:

src/backend/config/config_test.go
src/backend/processor/response_test.go
src/backend/providers/providers_test.go
src/backend/proxy/passthrough_test.go
src/backend/proxy/router_test.go
src/backend/proxy/streaming_test.go

hanneshapke · 2026-07-03T01:14:10Z

@Davidnet Is this PR in addition to #532 ?

Davidnet · 2026-07-03T02:49:49Z

@Davidnet Is this PR in addition to #532 ?

Yes it was a rebase, and a modifications that I brought from my tests

Codex CLI startup and completions both broke when routed through the MITM proxy. Five fixes, verified end-to-end against a live Codex run: - Gate chatgpt.com interception by path: only /backend-api/codex/responses is masked; the streaming MCP transport, model list, and telemetry pass through verbatim via a new in-tunnel passthrough that streams incrementally and preserves framing across keep-alive requests. - Pin relayed responses to HTTP/1.1: upstream legs negotiate HTTP/2, and relaying resp.Proto verbatim produced "HTTP/2.0" status lines that strict clients (Codex's hyper) reject as malformed. - Create the MITM loop's bufio.Reader once per connection so buffered bytes of a pipelined next request are not dropped between iterations. - Sniff SSE when Content-Type is absent: the Codex backend streams SSE with no Content-Type header, which sent responses down the buffered path and skipped PII restoration. - Restore PII in nested .done/.completed Responses-API payloads (content_part.done, output_item.done, response.completed) — Codex renders its final message from these, not from the flat deltas. - Reject WebSocket upgrades on masked endpoints (fail closed): PII in switched protocols cannot be masked. Codex is steered to SSE via a provider config with supports_websockets=false, documented in docs/09-coding-agents.md.

The response-restoration logic replaced each masked (dummy) value with its original via a sequence of strings.ReplaceAll calls. When the fake-value generator produced a dummy that coincides with a real original from another mapping (e.g. "Priya"->"Nicole" alongside "Claude"->"Priya"), the passes chained: restoring "Nicole"->"Priya" and then re-replacing that "Priya"-> "Claude", so the client received the wrong PII ("Hi Claude" where the model wrote "Hi Nicole" for Priya). The bug was also map-order dependent. Replace both restore paths (streaming codecs and the buffered ResponseProcessor) with a shared single-pass strings.Replacer built by processor.BuildRestorer, keys ordered longest-first. strings.Replacer scans the input once and never re-substitutes its own output, so restored text is never re-replaced. The streaming restoreCore builds the replacer once per stream instead of iterating the map per delta. Adds regression tests for the exact collision and longest-match precedence.

Davidnet · 2026-07-03T19:26:07Z

Codex sample:

codex exec \
      --skip-git-repo-check \
      -c 'model_provider="kiji"' \
      -c 'model_providers.kiji={ name = "Kiji", base_url = "https://chatgpt.com/backend-api/codex", wire_api = "responses", requires_openai_auth = true, supports_websockets = false }' \
"Draft a short reply to David Nettleton at [email protected] confirming the meeting."
OpenAI Codex v0.142.5
--------
workdir: /Users/davidnet/sample
model: gpt-5.4
provider: kiji
approval: never
sandbox: read-only
reasoning effort: medium
reasoning summaries: none
session id: 019f295d-39d1-7971-a168-0b6dd53b08d6
--------
user
Draft a short reply to David Nettleton at [email protected] confirming the meeting.
codex
Subject: Re: Meeting Confirmation

Hi David,

Confirming the meeting works for me. Looking forward to it.

Best,
Jordan
tokens used
8,434

Claude code

set -x HTTP_PROXY http://127.0.0.1:8081
set -x HTTPS_PROXY http://127.0.0.1:8081
 set -x NODE_EXTRA_CA_CERTS "$HOME/Library/Application Support/Kiji Privacy Proxy/certs/ca.crt"
$ claude -p --model haiku "Draft a two-sentence email to Priya Raghavan ([email protected]) confirming Tuesday's sync. Sign it from me."
Here's a draft email:

---

**To:** [email protected]
**Subject:** Confirming Tuesday's Sync

Hi Priya,

I wanted to confirm our sync scheduled for Tuesday. Please let me know if you need to reschedule or if there's anything you'd like to go over beforehand.

Best regards,
Yuki Smith

hanneshapke · 2026-07-03T19:33:54Z

@Davidnet Great work, but a huge PR. Wanna review it together on Monday?

Davidnet · 2026-07-03T19:35:06Z

@Davidnet Great work, but a huge PR. Wanna review it together on Monday?

Yes, It was also a quite a bit of new things, and for sure

Davidnet · 2026-07-06T15:10:20Z

+{
+  "env": {
+    "HTTP_PROXY": "http://127.0.0.1:8081",
+    "HTTPS_PROXY": "http://127.0.0.1:8081",


HTTPS consideration

Davidnet · 2026-07-06T15:14:59Z

+| Variable | Value | Why |
+|----------|-------|-----|
+| `HTTP_PROXY` | `http://127.0.0.1:8081` | Routes Codex's HTTP traffic through the proxy |
+| `HTTPS_PROXY` | `http://127.0.0.1:8081` | Routes Codex's HTTPS traffic (the completions call) through the proxy |


HTTPS discussion (ibid.)

Davidnet · 2026-07-06T15:29:02Z

+	// until the first bytes arrive, which for a stream is the first event.
+	peek, _ := br.Peek(len("event:"))
+	s := string(peek)
+	for _, prefix := range []string{"event:", "data:", "id:", "retry:", ":"} {


Make this defensive

Maybe asking chatgpt every few days, and we can see if eventually changes

Pin six defects surfaced in review of the streaming SSE path. Each test asserts the correct behavior and currently fails: - carry buffer re-restores already-restored text (chained substitution) in both the OpenAI and Anthropic codecs - OpenAI codec fails open for /v1/chat/completions (chat.completion.chunk) - splitSafe cuts multi-byte UTF-8 runes -> json.Marshal inserts U+FFFD - restored PII spliced unescaped into partial_json tool-arg fragments - codec carry tail dropped when a stream ends without .done/content_block_stop

Fixes the six failing regression tests from 0e9b02d: - Carry buffers now hold RAW (pre-restore) text and restoration runs as a single left-to-right pass (streamRestore) that emits originals exactly once, so a restored original that coincides with another mapping's dummy can no longer be re-substituted across delta boundaries. - Text is only split on rune boundaries; an incomplete trailing UTF-8 rune is held back instead of being emitted truncated (no more U+FFFD). - Only a suffix that is a proper prefix of a dummy is held back, instead of a fixed longest-dummy-1 tail, which also reduces streaming latency. - Tool-call argument fragments (input_json_delta, function_call_arguments, chat tool_calls) restore with JSON-escaped originals so the arguments the client reassembles stay valid JSON; .done arguments fields likewise. - The OpenAI codec now handles /v1/chat/completions chunks (chat.completion.chunk: choices[].delta.content and tool_calls), which previously passed through unrestored. - Codecs expose flushTail(); streamSSEResponse flushes held carry tails at EOF so streams ending without .done/content_block_stop are not truncated. - Tail flushes after *.completed reuse the channel's recorded delta type instead of deriving a bogus response.completed.delta.

golangci-lint (goconst) flagged the literal "delta" and "content" map keys repeated across the SSE codecs.

hanneshapke and others added 7 commits July 2, 2026 16:01

fix: streaming support to work with Claude Code

df885c5

codex streaming

b7caf5a

documentation

8426f28

linter fixes

354cf5d

updated package-lock.json

427df29

chore: drop stray shepherd.js root dependency

6235d4a

shepherd.js is a frontend dependency already declared in src/frontend/package.json; the root entry was added accidentally. Regenerating the lockfile also syncs the recorded src/frontend version with main (1.4.0 -> 1.4.1).

Davidnet added 2 commits July 3, 2026 14:44

Davidnet requested a review from hanneshapke July 3, 2026 19:26

Davidnet marked this pull request as ready for review July 3, 2026 19:36

Davidnet commented Jul 6, 2026

View reviewed changes

hanneshapke approved these changes Jul 6, 2026

View reviewed changes

Davidnet added 4 commits July 6, 2026 13:59

docs: explain why the proxy env vars use http:// not https://

8a934cd

fix: extract repeated delta/content JSON keys into constants

479c7a7

golangci-lint (goconst) flagged the literal "delta" and "content" map keys repeated across the SSE codecs.

Davidnet merged commit 94fbb16 into main Jul 6, 2026
7 of 8 checks passed

github-actions Bot locked and limited conversation to collaborators Jul 6, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: streaming support for coding agents (Claude Code & Codex)#586

feat: streaming support for coding agents (Claude Code & Codex)#586
Davidnet merged 13 commits into
mainfrom
feat/streaming-feature

Davidnet commented Jul 2, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jul 2, 2026 •

edited

Loading

Uh oh!

hanneshapke commented Jul 3, 2026

Uh oh!

Davidnet commented Jul 3, 2026

Uh oh!

Davidnet commented Jul 3, 2026

Uh oh!

hanneshapke commented Jul 3, 2026

Uh oh!

Davidnet commented Jul 3, 2026

Uh oh!

Davidnet Jul 6, 2026

Uh oh!

Davidnet Jul 6, 2026

Uh oh!

Davidnet Jul 6, 2026

Uh oh!

Davidnet Jul 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

Davidnet commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

github-actions Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hanneshapke commented Jul 3, 2026

Uh oh!

Davidnet commented Jul 3, 2026

Uh oh!

Davidnet commented Jul 3, 2026

Codex sample:

Claude code

Uh oh!

hanneshapke commented Jul 3, 2026

Uh oh!

Davidnet commented Jul 3, 2026

Uh oh!

Davidnet Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

Davidnet Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

Davidnet Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

Davidnet Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Davidnet commented Jul 2, 2026 •

edited

Loading

github-actions Bot commented Jul 2, 2026 •

edited

Loading