Context
The non-trivial "read one JSONL line with a byte cap, skip oversize lines and keep scanning" logic is implemented three times, and the per-line byte caps silently disagree.
Current state (verified locations)
internal/codex/codex.go:424 scanJSONLLines + :440 readJSONLLine ([]byte variant), cap jsonlLineCap = 64 MiB at codex.go:25internal/grok/grok.go:454 scanJSONLLines + :470 readJSONLLine — byte-for-byte identical to the codex copy, but cap jsonlLineCap = 16 MiB at grok.go:24internal/preview/preview.go:447 readJSONLLine (string variant), cap previewLineCap = 16 MiB at preview.go:329 (with a good comment explaining why bufio.Scanner was abandoned)
Task
- Create a new package
internal/jsonl with:
func ReadLine(r *bufio.Reader, max int) ([]byte, error) — copy the codex implementation.func ScanLines(r io.Reader, max int, visit func([]byte)) error — copy the codex implementation, taking max as a parameter instead of the package constant.- Move the preview comment about bufio.Scanner/ErrBufferFull onto
ReadLine so the rationale lives with the shared code.
- Migrate codex and grok to
jsonl.ScanLines(r, jsonlLineCap, visit); keep each package's existing cap constant and value so behavior is unchanged. Delete the local copies.
- Migrate
internal/preview to jsonl.ReadLine, converting with string(line) at the call site (loadMessages and anywhere else readJSONLLine is called). Delete the string variant.
- Move the line-reading unit tests (search each package's
_test.go for oversize-line / buffer-full cases) into internal/jsonl and delete duplicates. Add cases: empty file, line exactly at cap, line over cap followed by a normal line, missing trailing newline.
- Do NOT unify the cap values in this issue — leave 64 MiB for codex and 16 MiB for grok/preview, but add a one-line comment at each constant noting the divergence is inherited (a follow-up can reconcile them).
Acceptance criteria
- Only one implementation of the read/scan logic exists (
internal/jsonl).
go test ./..., go vet ./..., golangci-lint run, gofmt -l . pass/clean.go test -bench=. ./internal/codex ./internal/grok ./internal/preview shows no regression (±5%).
Context
The non-trivial "read one JSONL line with a byte cap, skip oversize lines and keep scanning" logic is implemented three times, and the per-line byte caps silently disagree.
Current state (verified locations)
internal/codex/codex.go:424scanJSONLLines+:440readJSONLLine([]bytevariant), capjsonlLineCap = 64 MiBatcodex.go:25internal/grok/grok.go:454scanJSONLLines+:470readJSONLLine— byte-for-byte identical to the codex copy, but capjsonlLineCap = 16 MiBatgrok.go:24internal/preview/preview.go:447readJSONLLine(stringvariant), cappreviewLineCap = 16 MiBatpreview.go:329(with a good comment explaining why bufio.Scanner was abandoned)Task
internal/jsonlwith:func ReadLine(r *bufio.Reader, max int) ([]byte, error)— copy the codex implementation.func ScanLines(r io.Reader, max int, visit func([]byte)) error— copy the codex implementation, takingmaxas a parameter instead of the package constant.ReadLineso the rationale lives with the shared code.jsonl.ScanLines(r, jsonlLineCap, visit); keep each package's existing cap constant and value so behavior is unchanged. Delete the local copies.internal/previewtojsonl.ReadLine, converting withstring(line)at the call site (loadMessagesand anywhere elsereadJSONLLineis called). Delete the string variant._test.gofor oversize-line / buffer-full cases) intointernal/jsonland delete duplicates. Add cases: empty file, line exactly at cap, line over cap followed by a normal line, missing trailing newline.Acceptance criteria
internal/jsonl).go test ./...,go vet ./...,golangci-lint run,gofmt -l .pass/clean.go test -bench=. ./internal/codex ./internal/grok ./internal/previewshows no regression (±5%).