storage: resume bucket GC iterators after commits#662
Open
andrejbranch wants to merge 1 commit into
Open
Conversation
Bucket garbage collection commits after each full chunk and restarts the sharding-index iterator from the bucket prefix. On Vinyl, every restart can walk through the tombstones left by all preceding chunks before it reaches the next live tuple. The repeated prefix scan makes cleanup of a large bucket disproportionately expensive when the chunk is small. On Tarantool 2.11 and newer, retain the last tuple committed by a full chunk and pass it as the next TREE iterator's `after` position. Tarantool supports pagination from a tuple which is no longer present, so the next transaction resumes after the deleted tuple. Preserve the existing path for older Tarantool versions and non-TREE indexes. Add coverage for the resumable cursor, the pagination-disabled fallback, and the non-TREE fallback. NO_DOC=bugfix
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resume bucket GC iteration across transaction batches
Summary
This change avoids restarting bucket garbage-collection scans from the bucket prefix after every transaction. On Tarantool 2.11 and later, GC over a TREE bucket index now retains the last tuple committed by a full batch and passes it as the next iterator's
afterposition. Older Tarantool versions and unsupported index types retain the existing prefix-restarting path.Closes #661
Motivation
Bucket GC commits after
BUCKET_CHUNK_SIZEdeletes and then opens another iterator at{bucket_id}. For large Vinyl buckets, every new secondary-index iterator can revisit the tombstones and LSM history left by preceding batches before it reaches the next live tuple. Smaller transaction batches create more restarts and can therefore cause disproportionate cleanup CPU and wall time.Tarantool pagination supports resuming after a tuple that is no longer present in the index. Retaining the last committed tuple allows the next transaction to continue at the correct secondary-index position without changing delete, WAL, replication, transaction, trigger, or bucket-state semantics.
Changes
index_paginationcore feature flag for Tarantool 2.11 and later.afterpositions, the pagination-disabled fallback, and the non-TREE fallback.Compatibility
There is no configuration or public API change. Tarantool versions before 2.11 never receive the
afteriterator option and retain the current implementation. The optimization is also disabled for non-TREE sharding indexes.A garbage bucket is not writable through VShard while cleanup runs. Cursor resumption relies on the same lifecycle invariant as bucket sending: no tuples are inserted behind an iterator that has already passed their position.
Performance
A representative local benchmark used a 50,000-tuple Vinyl bucket with three indexes, synchronous replication, and a garbage-collection transaction size of 100. Values are medians across three fresh-cluster runs.
The optimization reduced cleanup wall time by approximately 88% and cleanup CPU by approximately 91%. It did not change the number of deletes, WAL rows, transactions, index updates, or synchronous commits.
Testing