Skip to content

Restore KVzip forward after context errors 🤖🤖🤖#244

Open
fallintoplace wants to merge 1 commit into
NVIDIA:mainfrom
fallintoplace:fix/kvzip-forward-restore
Open

Restore KVzip forward after context errors 🤖🤖🤖#244
fallintoplace wants to merge 1 commit into
NVIDIA:mainfrom
fallintoplace:fix/kvzip-forward-restore

Conversation

@fallintoplace

Copy link
Copy Markdown

Summary

  • Restore model.model.forward from an inner finally around the KVzip context yield.
  • Preserve the normal path where the original forward is restored before KVzip's post-yield compression/scoring phase.
  • Add regression coverage for both exceptional context exits and normal compression setup.

Root cause

KVzipPress.__call__ monkey-patches model.model.forward before yielding to the caller, but the original method was restored only after a normal return from the with block. If the caller raised inside the context, the outer cleanup removed hooks and reset internal state without restoring forward, leaving the model patched after exit.

Validation

  • uv run pytest tests/presses/test_kvzip_press.py
  • uv run isort kvpress/presses/kvzip_press.py tests/presses/test_kvzip_press.py
  • uv run black kvpress/presses/kvzip_press.py tests/presses/test_kvzip_press.py
  • uv run flake8 kvpress/presses/kvzip_press.py tests/presses/test_kvzip_press.py
  • uv run mypy kvpress/presses/kvzip_press.py --check-untyped-defs
  • make style

@copy-pr-bot

copy-pr-bot Bot commented Jun 16, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant