fix(runtime): retry allocation after reclaiming cached pages#1438
Closed
ppodolsky wants to merge 1 commit into
Closed
fix(runtime): retry allocation after reclaiming cached pages#1438ppodolsky wants to merge 1 commit into
ppodolsky wants to merge 1 commit into
Conversation
3 tasks
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a new page allocation reports
BufferTooBig, reclaim every unused cached page and retry the allocation once before returning the error.Autotuning can leave one-off pages cached across several size buckets. A later allocation can fail at the device boundary even though those pages have no live slices. The normal reserve path previously surfaced that failure without trying the memory manager's existing explicit cleanup.
The retry also checks the selected pool again after cleanup, since reclamation can make a suitable page immediately reusable without another device allocation.
Validation
allocation_oom_reclaims_cached_pages_and_retries, using a capacity-limited storage implementation to reproduce the cached-page failure deterministicallycargo fmt --all -- --checkcargo test -p cubecl-runtime allocation_oom_reclaims_cached_pages_and_retriesCompatibility
No public API or successful-allocation behavior changes. Cleanup and retry happen only after the underlying storage returns
BufferTooBig.Downstream validation
This fixes the allocation failure encountered by Hermes during repeated CUDA autotuning/training runs. No CubeK or Burn source change is required for this runtime-only fix; both consume it through their CubeCL revision.