digest: 2026-05-24

_meta/claims-ledger.md

@@ -117,3 +117,13 @@
 - **2026-05-23** [arxiv:2605.22106] CLAIM: ArborKV — structure-aware KV cache for tree-of-thoughts reasoning: tree-aware allocation policy + lightweight value estimator drive token-extractive eviction with lazy rehydration for branch back-tracking; **~4× peak KV memory reduction with near-full-retention accuracy**, enabling larger search configurations at fixed hardware | source: https://arxiv.org/abs/2605.22106 | status: open ; KV codec specifically for tree-shaped inference-time-compute (OpenDeepThink Bradley-Terry / MAP / InsightReplay); composes orthogonally with [arxiv:2605.19660] OScaR (per-token codec) and [arxiv:2605.18643] ZEDA (active-expert reduction) on the M5 Max long-context-inference budget
 - **2026-05-23** [arxiv:2605.22535] CLAIM: TerminalWorld (UCL/Meta/Tsinghua — Peter O'Hearn, Earl T. Barr, Mark Harman, Federica Sarro) — reverse-engineers benchmark from **80,870 real terminal recordings** → 1,530 validated tasks across 18 categories / 1,280 unique commands, 200-task Verified subset; **max pass rate 62.5%** across 8 frontier models × 6 agents; **Pearson r = 0.20 vs. Terminal-Bench** (the two benchmarks measure different things) | source: https://arxiv.org/abs/2605.22535 | status: open ; second "real recordings, not synthetic sandboxes" benchmark in 2 weeks after [arxiv:2605.10912] WildClawBench (62.2% Claude Opus 4.7 ceiling); r=0.20 means existing Terminal-Bench-based model rankings need re-validation
 - **2026-05-23** [hf-blog-2026-05-23-nvidia-nemotron-labs-diffusion] CLAIM: NVIDIA Nemotron-Labs Diffusion — open diffusion language models at 3B / 8B / 14B (text) + 8B VLM, base + instruction-tuned variants; generates 32-token blocks via parallel denoising; three modes in one model (AR / diffusion / self-speculation = diffusion drafts + AR verification); **8B +1.2% accuracy vs. Qwen3-8B; diffusion mode 2.6× faster than AR; self-speculation quadratic 6.4× faster (~865 tok/s on B200, ~4× AR baseline)**; fully open weights + Megatron Bridge training code + SGLang inference; 1.3T pretraining + 45B fine-tune tokens; commercially-friendly NVIDIA Nemotron Open Model License (text) | source: https://huggingface.co/blog/nvidia/nemotron-labs-diffusion | status: open ; first credible open diffusion LM at production scale; self-speculation pattern structurally aligned with [arxiv:2605.15077] AsyncFC and [arxiv:2605.22154] IdleSpec — three same-week orthogonal latency optimizations (one in decoder, two in harness)
+- **2026-05-24** [deepmind-blog-2026-05-19-gemini-3-5-flash] CLAIM: Gemini 3.5 Flash — Google DeepMind frontier-fast model, claimed to **outperform Gemini 3.1 Pro on challenging coding and agentic benchmarks**; **76.2% Terminal-Bench 2.1**, **84.2% multimodal understanding**, **~4× faster than other frontier models** | source: https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-5/ | status: open ; vendor-self-reported; direct `compare` against [arxiv:2605.22535] TerminalWorld (62.5% frontier ceiling on real-recording-derived tasks; Pearson r = 0.20 vs. Terminal-Bench) — open empirical question whether the 76.2% headline transfers to TerminalWorld
+- **2026-05-24** [arxiv:2605.22794] CLAIM: MOSS — autonomous-agent self-evolution via **source-level code rewriting** (vs. text-layer state changes — prompts, configs, skills); deterministic multi-stage pipeline with code modifications delegated to external coding agents, replay testing against production failure batches, user-consent-gated deployment, post-deploy health monitoring; **OpenClaw four-task mean grader score 0.25 → 0.61 in a single cycle without human intervention**; code on GitHub | source: https://arxiv.org/abs/2605.22794 | status: open ; 6th layer in the self-evolution stack above [arxiv:2605.06614] SkillOS, [arxiv:2605.18401] SkillsVote, [arxiv:2605.13941] EvolveMem, [arxiv:2605.22148] Ratchet, [arxiv:2605.09998] Continual Harness — attacks structural failures the others cannot reach
+- **2026-05-24** [arxiv:2605.22733] CLAIM: HarnessAPI — single typed Python + Pydantic source of truth auto-derives both streaming HTTP/SSE endpoints and MCP tool registrations with dynamic Pydantic type propagation; dual-mode content negotiation (SSE + JSON identical); **74% reduction in framework-facing boilerplate vs. manually maintained FastAPI + FastMCP dual-stack** across 6 representative skills; Apache 2.0, `pip install harnessapi` | source: https://arxiv.org/abs/2605.22733 | status: open ; production-side companion to [arxiv:2605.10787] ComplexMCP MCP-at-scale benchmark; immediately try-now-able for MCP companion stacks
+- **2026-05-24** [arxiv:2605.22791] CLAIM: Gated DeltaNet-2 (NVIDIA Labs — Hatamizadeh, Yejin Choi, Jan Kautz) — linear attention with **channel-wise decoupled erase gate b_t + write gate w_t** (vs. single scalar gate in Mamba-2/Gated DeltaNet); **1.3B parameters, 100B FineWeb-Edu tokens**; strongest overall results among Mamba-2, Gated DeltaNet, KDA, Mamba-3 on language modeling, commonsense reasoning, retrieval; specific wins on **long-context RULER NIAH multi-key retrieval**; code released as NVlabs/GatedDeltaNet-2 | source: https://arxiv.org/abs/2605.22791 | status: open ; sub-quadratic-attention path orthogonal to KV-cache compression line ([arxiv:2605.22106] ArborKV, [arxiv:2605.09649] learnable KV eviction, [arxiv:2605.19660] OScaR) — relevant local-fit precursor for any future >7B linear-attention agent backbone
+- **2026-05-24** [arxiv:2605.22786] CLAIM: LCGuard (RPI / IBM Research) — first paper to define **KV-cache leakage as a security boundary in multi-agent systems**; safety metric is "cache unsafe iff adversarial decoder can recover agent-specific sensitive inputs from it"; adversarial training framework learns representation-level transformations on shared caches; consistent reductions in reconstruction-based leakage and attack success while maintaining task performance vs. standard KV-sharing baselines | source: https://arxiv.org/abs/2605.22786 | status: open ; fourth angle on agent-safety surface alongside [arxiv:2605.04785] AgentTrust (call-level), [arxiv:2605.06455] PrefixGuard (trajectory-level), [arxiv:2605.14271] HarnessAudit-Bench (resource-access seam) — LCGuard attacks the *latent communication channel* others don't touch
+- **2026-05-24** [arxiv:2605.22643] CLAIM: Boiling the Frog — stateful multi-turn benchmark for agentic safety with persistent workspace state; incremental benign→risky escalation across turns; three-level operational risk taxonomy aligned with EU AI Act; **9 frontier models tested**; **44.4% aggregate attack success rate**; range **Claude Haiku 4.5 lowest 20.5% → Gemini 3.1 Flash Lite 92.9%** (4.5× spread); **Seed 2.0 Lite >80%**; **loss-of-control scenarios average 93.3% across all models** | source: https://arxiv.org/abs/2605.22643 | status: open ; temporal-attack dimension on the agent-eval-credibility thread joining [arxiv:2605.10448] Evidence-Supported Bounds / [arxiv:2605.12131] Rollout Cards / [arxiv:2605.11946] Counterfactual Trace Auditing / [arxiv:2605.12673] BenchJack / [arxiv:2605.14271] HarnessAudit-Bench
+- **2026-05-24** [arxiv:2605.22672] CLAIM: "Is Capability a Liability?" (Forecasting Research Institute — Merrill, Lee, Ezra Karger) — inverse scaling: **more capable LLMs produce worse distributional forecasts** on tasks involving superlinear growth and regime-change risk; failure concentrates in upper-tail (aggressive upward extrapolation, lower tail unchanged); tested on ForecastBench-Sim contamination-free + real-world COVID-19, measles, housing, hyperinflation; within-family Llama-3.1 analysis isolates both scale and post-training as contributors; **single-threshold scoring at conventional cutoffs reverses sign of capability-accuracy relationship vs. tail-inclusive scoring on the same outputs** | source: https://arxiv.org/abs/2605.22672 | status: open ; extends scoring-rule credibility thread from [arxiv:2605.04624] AuditRepairBench, [arxiv:2605.12131] Rollout Cards (agent-bench score swings under reporting rules) to forecasting benchmarks
+- **2026-05-24** [arxiv:2605.22511] CLAIM: Search-E1 — minimal recipe (**vanilla GRPO interleaved with offline self-distillation, no external supervision or auxiliary modules**) for search-augmented reasoning; after each GRPO iteration policy rolls out on training questions; token-level forward KL aligns inference-time distribution to model's privileged-context distribution that reveals more efficient sibling trajectories; **Qwen2.5-3B reaches 0.440 avg EM across 7 QA benchmarks**, claimed to surpass all open-source baselines at the same scale | source: https://arxiv.org/abs/2605.22511 | status: open ; minimal-pipeline counterpoint to [arxiv:2605.06326] TIR (4-stage SFT+RLVR), [arxiv:2605.06642] StraTA, [arxiv:2605.06078] BEACON, [arxiv:2605.15155] SDAR on agentic-RL training recipes
+- **2026-05-24** [arxiv:2605.22505] CLAIM: Shor (Yonsei/KAIST cluster — Kai Tzu-iunn Ong et al.) — **priority-ranking evaluation of harness optimizers**: optimizers rank which components most improve agent performance, evaluated step-level without full rollouts; **182 human-verified optimization scenarios across domains, designs, time stages**; priority-ranking performance **correlates with actual multi-step harness-optimization gains**, i.e. cheap step-level metric is valid proxy for expensive end-to-end measurement; open data + code | source: https://arxiv.org/abs/2605.22505 | status: open ; seventh paper in the agent-eval credibility thread; attacks evaluator *cost* where [arxiv:2605.10448] Evidence-Supported Bounds, [arxiv:2605.12131] Rollout Cards, [arxiv:2605.04624] AuditRepairBench, [arxiv:2605.12673] BenchJack, [arxiv:2605.11946] Counterfactual Trace Auditing, [arxiv:2605.14271] HarnessAudit-Bench attack evaluator *fidelity*
+- **2026-05-24** [arxiv:2605.22817] CLAIM: VPO (Vector Policy Optimization, MIT cluster — Bahlous-Boldi, Puri, Shenfeld, Damani, Risi, **Omar Khattab**, Hong, **Pulkit Agrawal**) — drop-in replacement for GRPO advantage estimator that trains policies to anticipate **vector-valued rewards** (e.g., per-test-case correctness in code generation) instead of scalar; preserves response-distribution diversity required for inference-time search; **matches or beats strongest scalar RL baselines on test-time search across 4 tasks**; **performance gap widens as search budget increases**; for evolutionary search (AlphaEvolve-style), **VPO models solve problems GRPO models cannot solve at all** | source: https://arxiv.org/abs/2605.22817 | status: open ; third axis with [arxiv:2605.08083] AutoTTS (controller synthesis), [arxiv:2605.15177] OpenDeepThink (Bradley-Terry aggregation of parallel reasoning) on "test-time search needs to be co-designed with training"

_meta/processed.json

@@ -1,4 +1,14 @@
 [
+  { "id": "deepmind-blog-2026-05-19-gemini-3-5-flash", "url": "https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-5/", "title": "Gemini 3.5: frontier intelligence with action", "date_covered": "2026-05-24" },
+  { "id": "arxiv:2605.22794", "url": "https://arxiv.org/abs/2605.22794", "title": "MOSS: Self-Evolution through Source-Level Rewriting in Autonomous Agent Systems", "date_covered": "2026-05-24" },
+  { "id": "arxiv:2605.22733", "url": "https://arxiv.org/abs/2605.22733", "title": "HarnessAPI: A Skill-First Framework for Unified Streaming APIs and MCP Tools", "date_covered": "2026-05-24" },
+  { "id": "arxiv:2605.22791", "url": "https://arxiv.org/abs/2605.22791", "title": "Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention", "date_covered": "2026-05-24" },
+  { "id": "arxiv:2605.22786", "url": "https://arxiv.org/abs/2605.22786", "title": "LCGuard: Latent Communication Guard for Safe KV Sharing in Multi-Agent Systems", "date_covered": "2026-05-24" },
+  { "id": "arxiv:2605.22643", "url": "https://arxiv.org/abs/2605.22643", "title": "Boiling the Frog: A Multi-Turn Benchmark for Agentic Safety", "date_covered": "2026-05-24" },
+  { "id": "arxiv:2605.22672", "url": "https://arxiv.org/abs/2605.22672", "title": "Is Capability a Liability? More Capable Language Models Make Worse Forecasts When It Matters Most", "date_covered": "2026-05-24" },
+  { "id": "arxiv:2605.22511", "url": "https://arxiv.org/abs/2605.22511", "title": "Search-E1: Self-Distillation Drives Self-Evolution in Search-Augmented Reasoning", "date_covered": "2026-05-24" },
+  { "id": "arxiv:2605.22505", "url": "https://arxiv.org/abs/2605.22505", "title": "Towards Direct Evaluation of Harness Optimizers via Priority Ranking (Shor)", "date_covered": "2026-05-24" },
+  { "id": "arxiv:2605.22817", "url": "https://arxiv.org/abs/2605.22817", "title": "Vector Policy Optimization: Training for Diversity Improves Test-Time Search", "date_covered": "2026-05-24" },
   { "id": "microsoft-research-2026-05-21-magenticlite-fara15", "url": "https://www.microsoft.com/en-us/research/blog/magenticlite-magenticbrain-fara1-5-an-agentic-experience-optimized-for-small-models/", "title": "MagenticLite, MagenticBrain, Fara1.5: An agentic experience optimized for small models", "date_covered": "2026-05-23" },
   { "id": "anthropic-research-2026-05-22-glasswing-initial-update", "url": "https://www.anthropic.com/research/glasswing-initial-update", "title": "Project Glasswing: An initial update", "date_covered": "2026-05-23" },
   { "id": "arxiv:2605.21850", "url": "https://arxiv.org/abs/2605.21850", "title": "ACC: Compiling Agent Trajectories for Long-Context Training", "date_covered": "2026-05-23" },