fix(swa): port tombstone-aware eviction + dec_swa_lock_only from upsream #23882#49
fix(swa): port tombstone-aware eviction + dec_swa_lock_only from upsream #23882#49yyj6666667 wants to merge 1 commit into
Conversation
…ream sgl-project#23882 SWA-only eviction crashed on leaf nodes with full_lock_ref > 0, swa_lock_ref == 0 (beyond sliding window). Adds tombstone branch to free SWA without asserting, plus dec_swa_lock_only for early SWA lock release during decode. Unifies inc/dec_lock_ref return types across all cache implementations. Co-Authored-By: Claude Opus 4.6 <[email protected]>
There was a problem hiding this comment.
Code Review
This pull request optimizes Sliding Window Attention (SWA) cache management by introducing a configurable eviction interval and a mechanism for early release of leaf locks after the window has passed. It also refactors the cache locking API across multiple implementations to use structured result and parameter objects. Feedback identifies an inconsistency in the SWARadixCache._delete_leaf method, where len(node.key) is used instead of len(node.value) to update the evictable size, which could lead to accounting errors if the lengths differ.
| if not node.swa_tombstone: | ||
| self.swa_evictable_size_ -= len(node.key) |
There was a problem hiding this comment.
Inconsistency found: _delete_leaf uses len(node.key) to update swa_evictable_size_, while other methods like _insert_helper and evict use len(node.value). For consistency and to avoid potential issues if node.key and node.value differ in length, consider using len(node.value) if node.value is not None.
SWA-only eviction crashed on leaf nodes with full_lock_ref > 0, swa_lock_ref == 0 (beyond sliding window). Adds tombstone branch to free SWA without asserting, plus dec_swa_lock_only for early SWA lock release during decode. Unifies inc/dec_lock_ref return types across all cache implementations.