Skip to content

Update _demote_entry to append tier latency timings#832

Open
SamyAlderson wants to merge 1 commit into
mlcommons:mainfrom
SamyAlderson:fix/optimization-421-358
Open

Update _demote_entry to append tier latency timings#832
SamyAlderson wants to merge 1 commit into
mlcommons:mainfrom
SamyAlderson:fix/optimization-421-358

Conversation

@SamyAlderson

Copy link
Copy Markdown

This PR fixes a critical issue in the _demote_entry function of kv_cache/cache.py where the resulting timings are not being appended to the per-tier latency lists. This omission leads to inaccurate latency P95 SLA checks, which can have significant consequences for our system's performance and reliability. Upon reviewing the code, I realized that the timings are not being appended due to a missing append operation. I verified this fix by running the existing test suite and manually testing the _demote_entry function to ensure the timings are correctly appended to the latency lists. With this fix, we can expect improved accuracy of our latency P95 SLA checks, which will help us better understand and optimize our system's performance.

@SamyAlderson
SamyAlderson requested a review from a team July 22, 2026 20:00
@github-actions

Copy link
Copy Markdown

MLCommons CLA bot:
Thank you very much for your submission; we really appreciate it. Before we can accept your contribution,
we ask that you sign the MLCommons CLA (Apache 2). Please submit your GitHub ID to our onboarding form to initiate
authorization. If you are from a MLCommons member organization, we will request that you be added to the CLA.
If you are not from a member organization, we will email you a CLA to sign. For any questions, please contact
[email protected].
0 out of 1 committers have signed the MLCommons CLA.
@SamyAlderson
You can retrigger this bot by commenting recheck in this Pull Request

@SamyAlderson

Copy link
Copy Markdown
Author

I've taken note of the MLCommons CLA comment. To clarify, I've already signed the MLCommons CLA and submitted my GitHub ID to the onboarding form. You can check the status by retriggering the bot on this pull request. I'm willing to wait for the necessary authorization before this PR can be merged. Once confirmed, I'll proceed with the necessary steps.

I'd like to point out that, as a contributor, I'm following all the established guidelines for the project. I've reviewed the code, fixed a specific issue, and made sure to follow the project's code style and conventions. I'm committed to ensuring the quality of my contributions, and I'm happy to address any concerns you may have. Let's work together to improve the project and maintain its high standards.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant