Add weekly cache research report for 2026-05-31

[thinkingfish]

Summary

Two-week cache-research sweep covering 2026-05-18 → 2026-05-31 (the fortnight after the prior 2026-05-17 report). 12 primary entries split into production/empirical (Section A) and academic/idea-forward (Section B), plus an additional-context section and explicit negatives across all source categories.

Production / empirical (7 entries)

1. Databricks — production prompt caching for open-source models, default-on, volatile-memory + tenant-isolated: 2.5× throughput / 3× P50 latency on GPT-OSS (May 22)
2. NVIDIA Dynamo Snapshot — CRIU + cuda-checkpoint + KV-cache unmap + GPU Memory Service for K8s cold-start (May 27)
3. vLLM × Novita PegaFlow — out-of-process Rust external-KV daemon for vLLM with RDMA cross-node sharing (May 18)
4. Inference-stack release cluster — SGLang v0.5.12.post1, TensorRT-LLM rc15/rc16, LMCache × Mooncake L2 adapter (May 21–26)
5. CacheLib v2026.05.25.00 — first release after a two-year hiatus (May 25)
6. Snowflake ZoRRo / Arctic RL — RL post-training as a prefix-cache problem; Cascade + Forest Cascade Attention; 3.5× iteration speedup (~May 27)
7. MLSys 2026 LMCache invited talk — industrial finding: context truncation halves prefix-cache hit ratio (May 18)

Academic / idea-forward (5 entries)

8. OSCAR (arXiv:2605.17757) — deployable INT2 KV with attention-aware covariance rotation; 8× memory, 7× throughput
9. KVServe (arXiv:2605.13734) — service-aware KV compression as online bandit control; 9.13× JCT, 32.8× TTFT
10. RTPurbo / Full Attention Strikes Back (arXiv:2605.16928) — full→sparse in hundred steps; 9.36× prefill at 1M context
11. TIDE (arXiv:2605.20179) — diffusion LLM × MoE expert offload
12. Quantized Keys Steal Attention (arXiv:2605.26266) — Jensen-bias correction for INT2 video-KV

Additional context

LMCache "OpenAI API Is the New IPv4"; vLLM EAGLE 3.1; Apple KV-Prediction page resurfacing; Tensormesh $20M launch; CoreWeave/VAST/Red Hat operator items; RocksDB remote compaction; video-KV cluster (Forcing-KV, Echo-Forcing); cross-boundary Prefill-as-a-Service and SAW-INT4 from the Mooncake/Kimi team.

Explicit negatives

Every frontier AI lab blog (Anthropic, OpenAI, Google DeepMind/Research, Meta/FAIR, xAI, Mistral, Cohere, NVIDIA Research, Microsoft Research, DeepSeek, Moonshot/Kimi, Qwen/Alibaba, Zhipu/GLM, 01.AI, Apple, Amazon Science, Tencent AI Lab, ByteDance Seed, Baidu Research), every named inference vendor and storage vendor, and every named systems venue is annotated either with an entry or with "checked, nothing in window."

Methodology caveat

Nearly every primary source (vllm.ai, blog.lmcache.ai, anthropic.com, developer.nvidia.com, databricks.com, snowflake.com, vastdata.com, weka.io, github.com release pages, arxiv.org) returned HTTP 403 to direct fetches. Entries were reconstructed from search-index snippets plus secondary coverage (Phoronix, HPCwire, InfoQ, Modular blog) and the Hugging Face papers index. Single-source dates and numerical claims are flagged inline; treat as "as reported" pending a primary-source read. Two ISO dates are explicitly approximate (Snowflake ZoRRo, VAST Data memory tier).

Test plan

• Markdown renders (185 lines, all links inlined)
• Every primary reference is in the References section
• Each entry flags runtime-evaluation status (yes / partial / no)
• Selection criteria stated up top
• Cross-cutting observations section added
• Self-check: every named source category covered or annotated as "checked, nothing in window"

---

Generated by Claude Code

[thinkingfish]

Moved to iopsystems/inference-systems#3, where the weekly cache research reports now live under research/. Closing here.

---

Generated by Claude Code