feat(api): consume blocks via gRPC StreamBlocks (flagged, default off)#941
Merged
Conversation
Bumps go-openaudio to aee578e (#342) and wires the core-indexer to source blocks from CoreService.StreamBlocks instead of polling GetBlocks, gated by a new coreBlockStreamEnabled env (default off). When enabled, builds a gRPC (connect.WithGRPC) Core client from audiusdUrl and passes it via SetBlockStreamClient + Config.BlockStreamEnabled. The SDK's unary Core client is retained for status checks and the polling fallback, and the ETL auto-falls-back to polling if the endpoint doesn't support the stream — so this is safe to flip on per environment. Co-Authored-By: Claude Opus 4.7 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bumps
go-openaudiotoaee578e(OpenAudio/go-openaudio#342) and wires the core-indexer to source blocks from theCoreService.StreamBlocksgRPC stream instead of pollingGetBlocks, behind a newcoreBlockStreamEnabledenv — default off, so prod behavior is unchanged until flipped.What #342 gives us
The ETL can now consume blocks over a gRPC server stream: the node replays history from the resume height, then live-tails, gap-filling any heights its in-process pubsub drops. The ETL reconnects from its cursor on error and auto-falls-back to polling if the endpoint doesn't support the stream.
Changes
go.mod/go.sum:github.com/OpenAudio/go-openaudio(+/pkg/etl) →v1.3.1-0.20260609211151-aee578ec923bconfig/config.go: newCoreBlockStreamEnabled(coreBlockStreamEnabledenv, default off)indexer/indexer.go: when enabled, setConfig.BlockStreamEnabledand pass a gRPC (connect.WithGRPC) Core client viaSetBlockStreamClient. The SDK's unary Core client is retained for status checks + the polling fallback.Rollout
coreBlockStreamEnabled=trueon one indexer and validate: it tails live, resumes gap-free on restart, and (induced) stream errors fall back to polling cleanly.Validated locally with the harness in OpenAudio/go-openaudio#344 (devnet,
--stream, catch-up + restart-resume).Test plan
go mod tidy(only go.mod/go.sum),go build ./...,go vet ./indexer ./configcleancoreBlockStreamEnabled=trueon one indexer; confirm live-tail + gap-free restart + fallback🤖 Generated with Claude Code