add: throughput benchmark + docs for initial-load scenarios#23
Merged
Conversation
Introduces TestInsertThroughput and TestInsertThroughputByShape in test/integration/throughput_test.go to measure how fast Streambed drains a WAL backlog into Iceberg. Throughput is measured by tapping the writer's "flush completed" log line via a custom slog.Handler, which gives microsecond-accurate timing with no polling overhead. A comment at the log site in internal/iceberg/writer.go flags the log-format dependency so future changes to the log keys prompt a corresponding update to the benchmark. Also ignores .claude/ from the working tree. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
docs/benchmarks/README.md covers methodology (flush-log tapping rather
than Iceberg polling), reproduction steps, hardware disclosure, and
explicit caveats about localhost MinIO, Docker Desktop on macOS,
proto_version=1, initial-load vs steady-state, and single-run precision.
docs/benchmarks/initial-load.md captures results from a 1M-row run
across four transaction shapes (bulk / batch / oltp / single) on the
optimized flush config. Includes the finding that one-giant-transaction
("bulk") is ~17% slower than 1,000-row transactions at this scale, and
the expected 2.3x slowdown for pathological 1-row-per-commit workloads.
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
test/integration/throughput_test.gothat measures initial-load INSERT throughput (rows/sec landed in Iceberg) across multiple flush configs and transaction shapes.flush completedlog via a customslog.Handler— microsecond-accurate, zero MinIO contention, no polling lag. A comment at the log site ininternal/iceberg/writer.godocuments the dependency.docs/benchmarks/with methodology, reproduction steps, hardware disclosure, explicit caveats, and a first results file (initial-load.md).Headline findings (1M rows, localhost MinIO, Mac M3 Max, Docker Desktop)
Notable: one giant transaction (
bulk) is ~17% slower than 1,000-row transactions, stable across repeat runs. Small-but-not-tiny transactions (10–1,000 rows) are optimal.Caveats baked into the docs
proto_version '1'— no streaming of in-progress transactions. Noted as a known limitation.Test plan
docker compose -f test/integration/docker-compose.yml up -d --waitgo test -tags integration -v -timeout 1200s -run TestInsertThroughput ./test/integration/...passesgo test -tags integration -v -timeout 1200s -run TestInsertThroughputByShape ./test/integration/...passesdocs/benchmarks/README.mdrenders correctly on GitHubdocs/benchmarks/initial-load.mdrenders correctly on GitHub🤖 Generated with Claude Code