feature request: support for JSON streaming (application/x-ndjson)

[salisbury-espinosa]

Description

Unlike a standard application/json response that returns a single JSON object or a complete array, an application/x-ndjson response streams individual JSON objects separated by newline characters ( \n ).
This allows clients to process data incrementally without waiting for the entire payload to download, which is common in logs, real-time events, or LLM (Large Language Model) streaming responses.

[pullfrog]

Posted a detailed implementation plan for application/x-ndjson streaming support on #3755. The plan covers 8 milestones across the IR layer, parsers, SDK plugin, a new ndjson streaming client (modeled after the existing SSE implementation), all 7 client implementations, runtime getParseAs() fixes, tests, and docs.

Task list (3/3 completed)

• Analyze issue requirements and gather codebase context
• Understand current content-type/response handling in the codebase
• Produce structured plan and post it

  ^{｜ Implement plan ➔ ｜ View workflow run ｜ Triggered by Pullfrog ｜ 𝕏}

[pullfrog]

Plan: application/x-ndjson streaming support

Context

application/x-ndjson (Newline Delimited JSON) is a streaming format where each line is a self-contained JSON object separated by \n. It's commonly used for log streaming, real-time events, and LLM responses. The codebase already has a full implementation of text/event-stream (SSE) that provides an excellent architectural blueprint — ndjson streaming is structurally simpler (no event:/id:/retry: fields, just one JSON object per line).

Current behavior: application/x-ndjson falls through the IR media type classification (returns undefined), and at runtime getParseAs() matches it as 'blob' because it starts with application/ — which is incorrect.

Milestones

1. IR layer — recognize ndjson as a media type

Files:

• packages/shared/src/ir/mediaType.ts — Add 'ndjson' to the IRMediaType union. Add a regex ndjsonMimeRegExp matching application/x-ndjson and application/ndjson (both are used in the wild). Insert the check in mediaTypeToIrMediaType() before the catch-all application/* branches.
• packages/shared/src/ir/__tests__/mediaType.test.ts — Add test cases for application/x-ndjson, application/ndjson, and application/x-ndjson; charset=utf-8.

Design note: An alternative is to reuse the existing 'json' IR media type. This is not recommended because ndjson requires fundamentally different parsing (line-by-line streaming vs. single JSON.parse), and the IR type drives downstream code generation and runtime behavior.

2. Parsers — propagate ndjson media type through the IR

Files:

• packages/shared/src/openApi/3.1.x/parser/mediaType.ts
• packages/shared/src/openApi/3.0.x/parser/mediaType.ts
• packages/shared/src/openApi/2.0.x/parser/mediaType.ts

These already call mediaTypeToIrMediaType() and propagate its result. No changes needed unless we want to add special Content handling for ndjson. The raw mediaType string (e.g. "application/x-ndjson") is already preserved on IRResponseObject.mediaType.

Verify: Confirm that the preference logic (contents.find((content) => content.type === 'json') || contents[0]) in the operation parsers handles the case where both application/json and application/x-ndjson are present on the same response. JSON should still win by default. If only ndjson is present, it should be selected as contents[0].

3. SDK plugin — detect ndjson and route to streaming client

Files:

• packages/openapi-ts/src/plugins/shared/utils/operation.ts — Add hasOperationNdjson() following the same pattern as hasOperationSse(), checking response.mediaType against application/x-ndjson and application/ndjson.
• packages/openapi-ts/src/plugins/@hey-api/sdk/shared/operation.ts:
  • In operationOptionsType(): handle ndjson like SSE (use streaming options type).
  • In operationStatements(): detect ndjson alongside SSE. Route ndjson operations to client.ndjson.{method}() (parallel to client.sse.{method}()).
  • In getResponseType() (Axios): map ndjson to 'stream'.

Design decision: Whether ndjson uses the same ServerSentEventsOptions/ServerSentEventsResult types or gets its own NdjsonStreamingOptions/NdjsonStreamingResult types. Recommendation: new types — ndjson doesn't have SSE-specific fields (sseDefaultRetryDelay, sseMaxRetryAttempts, onSseEvent, onSseError, etc.). A dedicated type keeps the API clean and avoids exposing irrelevant options.

4. Client core — implement ndjson streaming client

Files:

• packages/openapi-ts/src/plugins/@hey-api/client-core/bundle/ndjsonStreaming.ts (new file) — Implement createNdjsonClient() following the pattern of createSseClient() in serverSentEvents.ts, but simpler:
  • Read response.body through TextDecoderStream + getReader()
  • Buffer incoming text, split on \n
  • JSON.parse() each non-empty line
  • yield each parsed object through an AsyncGenerator
  • Support responseValidator and responseTransformer per-item
  • Support signal for cancellation
  • No retry logic (ndjson is not a reconnecting protocol like SSE)
  • Callbacks: onNdjsonData (per-object) and onNdjsonError (parse/network error)

Types to export:

type NdjsonStreamingOptions<TData = unknown> = Omit<RequestInit, 'method'> &
  Pick<Config, 'method' | 'responseTransformer' | 'responseValidator'> & {
    fetch?: typeof fetch;
    onNdjsonData?: (data: TData) => void;
    onNdjsonError?: (error: unknown) => void;
    onRequest?: (url: string, init: RequestInit) => Promise<Request>;
    serializedBody?: RequestInit['body'];
    url: string;
  };

type NdjsonStreamingResult<TData = unknown> = {
  stream: AsyncGenerator<TData, void, unknown>;
};

5. Client implementations — expose ndjson methods

Files (each client):

• packages/openapi-ts/src/plugins/@hey-api/client-fetch/bundle/client.ts
• packages/openapi-ts/src/plugins/@hey-api/client-next/bundle/client.ts
• packages/openapi-ts/src/plugins/@hey-api/client-ky/bundle/client.ts
• packages/openapi-ts/src/plugins/@hey-api/client-ofetch/bundle/client.ts
• packages/openapi-ts/src/plugins/@hey-api/client-axios/bundle/client.ts
• packages/openapi-ts/src/plugins/@hey-api/client-angular/bundle/client.ts
• packages/openapi-ts/src/plugins/@hey-api/client-nuxt/bundle/client.ts

For each client:

1. Add makeNdjsonFn() (parallel to makeSseFn()) that calls createNdjsonClient().
2. Expose ndjson: { get, post, ... } on the client object (parallel to sse: { ... }).

Types files (bundle/types.ts for each client):

• Import and use NdjsonStreamingOptions and NdjsonStreamingResult from client-core.
• Add NdjsonMethod type (parallel to SseMethod) and include ndjson property on the Client interface.

6. Runtime getParseAs() — fix the misclassification

Files:

• packages/openapi-ts/src/plugins/@hey-api/client-fetch/bundle/utils.ts
• packages/openapi-ts/src/plugins/@hey-api/client-next/bundle/utils.ts
• packages/openapi-ts/src/plugins/@hey-api/client-ky/bundle/utils.ts
• packages/openapi-ts/src/plugins/@hey-api/client-ofetch/bundle/utils.ts
• packages/custom-client/src/utils.ts

Add an ndjson check before the generic application/ → blob branch:

if (cleanContent === 'application/x-ndjson' || cleanContent === 'application/ndjson') {
  return 'stream';
}

This ensures that if someone uses parseAs: 'auto' (or doesn't set parseAs), ndjson responses return the raw ReadableStream rather than being consumed as a blob. In practice, the SDK will route ndjson operations through client.ndjson.{method}() and won't hit getParseAs() at all — but this fix is still needed for users calling the client directly.

7. Tests

• Unit tests: packages/shared/src/ir/__tests__/mediaType.test.ts — ndjson media type recognition.
• Snapshot tests: Add an OpenAPI spec file specs/3.1.x/ndjson-streaming.yaml with an endpoint returning application/x-ndjson. Run snapshot generation to verify the generated SDK routes through client.ndjson.{method}() and uses proper types.
• E2E / integration consideration: An ndjson test server would be ideal but may be out of scope for the initial PR. The snapshot tests validate code generation correctness. Runtime behavior is covered by the unit-testable createNdjsonClient() function.

8. Documentation

• Update docs to mention ndjson streaming alongside SSE (if there's a streaming/SSE page).
• Add a JSDoc example on NdjsonStreamingOptions.

Open questions

1. Retry semantics: SSE has built-in retry with Last-Event-ID. Ndjson has no such protocol-level reconnection. Should createNdjsonClient() support optional retry with user-provided resume logic (e.g., query parameter offset)? Recommendation: no retry in v1 — keep it simple.
2. Request body ndjson: Some APIs accept application/x-ndjson request bodies (e.g., Elasticsearch _bulk). The body serializer in the SDK would need a case for 'ndjson' that serializes an array of objects into newline-delimited JSON. This could be a follow-up.
3. parseAs: 'ndjson': Should we add a dedicated parseAs value for ndjson (returning the AsyncGenerator directly), or is 'stream' sufficient for the manual use case? Recommendation: 'stream' is sufficient — the client.ndjson.* methods handle the structured case.

Estimated scope

~15–20 files modified/created, touching the shared IR layer, all 7 client implementations, the SDK plugin, and test infrastructure. The SSE implementation provides a proven pattern to follow, significantly reducing design risk.

  ^{｜ Implement plan ➔ ｜ View workflow run ｜ Triggered by Pullfrog ｜ 𝕏}

[mrlubos]

@salisbury-espinosa I'll get around to this, but other things are more important right now. What are you doing in the meantime to work around it?

[salisbury-espinosa]

What are you doing in the meantime to work around it?

nothing yet (
I would like to do this through SDK methods (which generated with hey-api/openapi-ts) to ensure type-safety...