feat(stream): add pull-based pipeline runtime [STREAM-1668] - #559

Closed

tryangul wants to merge 15 commits into

mainfrom

rbroughan/pull-based-arroyo

Commits on Jul 31, 2026

Full stage based implementation of pull streaming
tryangul
committed
c06dc04

Commits on Aug 6, 2026

Handle N:1 and N:0 use cases. Refactor StageResult, handlers and organize for clarity.
tryangul
committed
b13a354
Support concurrent stage application.
tryangul
committed
bca22ab
Actually commit offsets.
tryangul
committed
8ec1d2e

Commits on Aug 7, 2026

Add graceful termination aka cancellation support.
tryangul
committed
ce7c45f
Comment tweaks.
tryangul
committed
4711018
handle fatal kafka errors, preserve dlq headers, init coarsetime updater, move offset tracker.
tryangul
committed
cfe9d89
Basic rebalance handling.
tryangul
committed
a9cd406
Factor out PipelineRunner. Update examples.
tryangul
committed
f51cd71
Formatting and comment cleanup.
tryangul
committed
58eac3f
Fix stage name lifetimes.
tryangul
committed
e0c2e45
Use singleton updater to prevent leak.
tryangul
committed
546890a
Synchronize drain on rebalance.
tryangul
committed
7db0dbb
Drain regardless of outcome
tryangul
committed
ef7a538
Revert drain on rebalance due to potential threading deadlock.
tryangul
committed
e625c3a