Skip to content

digest: 2026-05-21 - #18

Merged
JustinPerea merged 1 commit into
mainfrom
claude/digest-2026-05-21
May 21, 2026
Merged

digest: 2026-05-21#18
JustinPerea merged 1 commit into
mainfrom
claude/digest-2026-05-21

Conversation

@JustinPerea

Copy link
Copy Markdown
Owner

Summary

9 items, 1 conflict flagged (AI-as-reviewer vs AI-as-author asymmetry).

Focus areas covered

  • Agent memory (1): Mem-π — generative memory with decision-content decoupled RL; >30% web nav improvement over retrieval baselines. 13th orthogonal memory-mechanism axis on the ledger.
  • Context engineering / Local & open models (3): OScaR (HKU, INT2 near-lossless KV, 5.3× memory reduction, code released), OCTOPUS (Stability AI, octahedral rotation codec, multimodal), Mix-Quant (NUS, prefill-only FP4, 3× prefill speedup).
  • Agent orchestration / Tools & connectors (2): IBM Temporal Semantic Cache for MCP plan-execute (1.67× workflow speedup, 30.6× median cache-hit speedup, fixes time-sensitive cache failure mode); DeepMind Co-Scientist (Nature publication; specialized hypothesis-generation / critique / ranking agents).
  • Coding agents / Evaluation (2): SpecBench (UCL; reward hacking in long-horizon coding agents; 28-pp gap per 10× code size; 2,900-line memorizing hash-table "compiler" failure mode), AI Reviewers Nature paper (GPT-5.2 60% > 48.2% top human, 16 recurring AI-only weaknesses).
  • Reasoning / inference-time compute (1): RELEX — RLVR weight trajectories are rank-1; 15% steps for full performance, 10–20× extrapolation.

Conflicts flagged

  • AI-as-reviewer (item digest: 2026-05-12 #9) vs. AI-as-author (last week's [arxiv:2605.19156] ResearchArena) — asymmetry, not contradiction. GPT-5.2 reviewer beats top human reviewer (60.0% vs 48.2%, p = 0.009) on Nature-family papers; same period, 0/117 generated papers under artifact-aware review meet top-tier venue standards. Both can be true: AI is competitive as reviewer, still failing as author. Operational signal: review/critique products are closer to production-ready than full-artifact-generation products. Open empirical question: does the AI-reviewer advantage hold when AI is reviewing AI-authored papers?

Cross-week sharpenings (not conflicts)

  • KV-cache codec convergence — OScaR (digest: 2026-05-06 #2, per-channel canalized rotation) and OCTOPUS (digest: 2026-05-07 #3, octahedral triplet quantization) land same week with different geometries. Track for third entrant that benchmarks both head-to-head.
  • Three independent low-rank linearity findings in the agentic LLM stack — [arxiv:2605.07990] (tool calling linearly readable in logits), [arxiv:2605.14038] (tool-necessity orthogonal probes), RELEX (digest: 2026-05-10 #7, rank-1 weight updates). Three different surfaces converging on dominant linear structure.
  • Agent-eval-credibility cluster now at 10+ papers in 4 weeks with SpecBench (digest: 2026-05-09 #6) adding the 28-pp/10×-code-size visible↔held-out gap.
  • Multi-agent role separation pattern now production-blessed at Nature-publication tier — DeepMind Co-Scientist (digest: 2026-05-11 #8) joins PREPING / SkillsVote in a three-way convergence on the role-separation + explicit-gating multi-agent pattern.

Local-model corner

  • OScaR (try-now, code released) — Qwen3-30B-A3B at Q4_K_M + INT2 KV could fit 1M-context within 64 GB unified memory. Constraint flips from "how much context fits" to "decode speed at 1M context".
  • Mix-Quant — argues prefill-asymmetric quant (FP4 prefill / BF16 decode) is the production-realistic agentic-inference shape. NVIDIA-specific today, principle portable to MLX.
  • Compositional argument — OScaR (KV) + ZEDA (active experts) + Mix-Quant (prefill) are three orthogonal optimizations on the same memory budget. Open question whether they compose multiplicatively.

Sources noted as quiet

  • anthropic.com/research: no new posts since May 14
  • microsoft.com/research/blog: no new posts since May 15
  • meta.ai/research: no new posts since April 8
  • mistral.ai/news: no new posts since April 29
  • openai.com/news: HTTP 403, persistent for 4+ weeks
  • allenai.org/blog: only Earth-science cross-post (OlmoEarth v1.1)

Sources flagged for next-run follow-up

  • DeepMind blog dates ambiguous for several May 2026 posts (Gemini 3.5 with action, Gemini Omni, Antigravity 2.0) — appear to be Google I/O 2026 cluster, dates need verification
  • NVIDIA Developer Blog 4-post agent series (May 19–20) — NVIDIA-Verified Agent Skills with MCP signing / OWASP LLM / MITRE ATLAS pattern is the first major-vendor productization of MCP skill governance; devblog-tier but worth tracking if Anthropic / Google adopt parallel framework
  • Google Research: Empirical Research Assistance (ERA, May 19) — product-tier framing, needs benchmark verification

Test plan

  • inbox/2026-05-21.md exists with 9 verified items, all primary sources opened
  • _meta/processed.json updated with 9 new IDs (validated as valid JSON)
  • _meta/claims-ledger.md appended with 9 new ledger entries
  • AI-Reviewer vs ResearchArena conflict cross-linked in both ledger entries
  • All 7 picked arXiv abstracts + DeepMind Co-Scientist blog post opened and verified directly
  • OScaR code-released claim verified at github.com/ZunhaiSu/OScaR-KV-Quant

https://claude.ai/code/session_01D6RZUnnUE2dMv6N2sZySDg


Generated by Claude Code

9 verified items covering agent memory (Mem-π generative memory), KV
cache quantization (OScaR INT2 near-lossless + code released, OCTOPUS
multimodal rotation codec), agentic inference (Mix-Quant prefill-only
quant, IBM Temporal Semantic Cache for MCP plan-execute), coding agent
reward hacking (SpecBench), RLVR training efficiency (RELEX rank-1
trajectories), multi-agent research (DeepMind Co-Scientist in Nature),
and AI-as-reviewer evaluation (Nature-family 60% > 48.2% top human).

1 conflict flagged: AI-as-reviewer (60% > top human) vs ResearchArena
AI-as-author (0/117 papers meet venue standards) — asymmetry is the
finding. KV codec same-week duplication (OScaR vs OCTOPUS) tracked as
open empirical question.

https://claude.ai/code/session_01D6RZUnnUE2dMv6N2sZySDg
@JustinPerea
JustinPerea merged commit 9fbf41b into main May 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants