Skip to content

storage: resume bucket GC iterators after commits#662

Open
andrejbranch wants to merge 1 commit into
tarantool:masterfrom
andrejbranch:bucket-gc-resumable-cursor
Open

storage: resume bucket GC iterators after commits#662
andrejbranch wants to merge 1 commit into
tarantool:masterfrom
andrejbranch:bucket-gc-resumable-cursor

Conversation

@andrejbranch

Copy link
Copy Markdown

Resume bucket GC iteration across transaction batches

Summary

This change avoids restarting bucket garbage-collection scans from the bucket prefix after every transaction. On Tarantool 2.11 and later, GC over a TREE bucket index now retains the last tuple committed by a full batch and passes it as the next iterator's after position. Older Tarantool versions and unsupported index types retain the existing prefix-restarting path.

Closes #661

Motivation

Bucket GC commits after BUCKET_CHUNK_SIZE deletes and then opens another iterator at {bucket_id}. For large Vinyl buckets, every new secondary-index iterator can revisit the tombstones and LSM history left by preceding batches before it reaches the next live tuple. Smaller transaction batches create more restarts and can therefore cause disproportionate cleanup CPU and wall time.

Tarantool pagination supports resuming after a tuple that is no longer present in the index. Retaining the last committed tuple allows the next transaction to continue at the correct secondary-index position without changing delete, WAL, replication, transaction, trigger, or bucket-state semantics.

Changes

  • Add an index_pagination core feature flag for Tarantool 2.11 and later.
  • Resume bucket GC from the last tuple committed by the preceding full batch when the sharding index is a TREE index and pagination is supported.
  • Preserve the current prefix-restarting behavior on Tarantool 1.10 and unsupported index types.
  • Keep the cursor local to one bucket and one space.
  • Continue checking the bucket generation and state after every full-batch commit.
  • Add tests that verify the initial prefix scan, subsequent after positions, the pagination-disabled fallback, and the non-TREE fallback.

Compatibility

There is no configuration or public API change. Tarantool versions before 2.11 never receive the after iterator option and retain the current implementation. The optimization is also disabled for non-TREE sharding indexes.

A garbage bucket is not writable through VShard while cleanup runs. Cursor resumption relies on the same lifecycle invariant as bucket sending: no tuples are inserted behind an iterator that has already passed their position.

Performance

A representative local benchmark used a 50,000-tuple Vinyl bucket with three indexes, synchronous replication, and a garbage-collection transaction size of 100. Values are medians across three fresh-cluster runs.

Iteration behavior Cleanup wall time Cleanup CPU time
Restart from bucket prefix 4.906 s 4.757 s
Resume after last tuple 0.602 s 0.414 s

The optimization reduced cleanup wall time by approximately 88% and cleanup CPU by approximately 91%. It did not change the number of deletes, WAL rows, transactions, index updates, or synchronous commits.

Testing

  • New resumable-cursor test passes for MemTX and Vinyl, with and without MVCC.
  • The test verifies the pagination-enabled cursor positions and forces the pagination-disabled fallback.
  • The test verifies that a HASH sharding index retains the non-cursor path.
  • Complete garbage-collector suite: 46 passed, 2 skipped.
  • Luacheck 0.26.1: 0 warnings and 0 errors across 61 files.

Bucket garbage collection commits after each full chunk and restarts the
sharding-index iterator from the bucket prefix. On Vinyl, every restart
can walk through the tombstones left by all preceding chunks before it
reaches the next live tuple. The repeated prefix scan makes cleanup of a
large bucket disproportionately expensive when the chunk is small.

On Tarantool 2.11 and newer, retain the last tuple committed by a full
chunk and pass it as the next TREE iterator's `after` position. Tarantool
supports pagination from a tuple which is no longer present, so the next
transaction resumes after the deleted tuple. Preserve the existing path
for older Tarantool versions and non-TREE indexes.

Add coverage for the resumable cursor, the pagination-disabled fallback,
and the non-TREE fallback.

NO_DOC=bugfix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Resume bucket garbage collection after the preceding batch

1 participant