JustinPerea · JustinPerea · May 15, 2026
diff --git a/_meta/claims-ledger.md b/_meta/claims-ledger.md
@@ -73,3 +73,11 @@
 - **2026-05-14** [arxiv:2605.12925] CLAIM: AgentLens — analysis of 2,614 trajectories from 8 models on 60 SWE-bench Verified tasks finds **10.7% of passing solutions are "Lucky Passes"** (regression cycles, blind retries, missing verification, temporally disordered exploration); per-model Lucky rate ranges **0.5% → 23.2%**; releases 1,815 trajectories labeled Lucky/Solid/Ideal; **up to 5-position model rank shifts** when ranked by process quality instead of pass rate | source: https://arxiv.org/abs/2605.12925 | status: open ; SWE-bench-specific instantiation of [arxiv:2605.11946] Counterfactual Trace Auditing's "aggregate gains hide mixed-sign behavior changes" finding
 - **2026-05-14** [arxiv:2605.12975] CLAIM: PyRAG reformulates multi-hop RAG as program synthesis and execution — Python programs call retrieval as a tool, get deterministic feedback, self-repair against compiler/runtime errors; consistent gains over strong baselines on PopQA, HotpotQA, 2WikiMultihopQA, MuSiQue, Bamboogle under both training-free and RL-trained settings | source: https://arxiv.org/abs/2605.12975 | status: open ; multi-hop-specific complement to [arxiv:2605.05242] Direct Corpus Interaction (which replaced general retrieval with code-over-corpus); training-free result the operationally useful one for existing MCP-based stacks
 - **2026-05-14** [arxiv:2605.13301] CLAIM: SU-01 (30B-A3B reasoning backbone, ~340K curriculum-SFT trajectories + ~200 RL steps + test-time scaling, reasoning trajectories beyond 100K tokens) claims gold-medal-level performance on IMO 2025 / USAMO 2026 / IPhO 2024 / IPhO 2025; open recipe | source: https://arxiv.org/abs/2605.13301 | status: open ; same parameter class as Qwen3.6-35B-A3B → local-fit math reasoning subagent on M5 Max once weights release; recipe overlaps with [arxiv:2605.06326] Teaching Thinking Models to Reason with Tools (TIR)
+- **2026-05-15** [arxiv:2605.10912] CLAIM: WildClawBench — 60 human-authored bilingual multimodal tasks (~8 min / 20+ tool calls each) inside reproducible Docker with real CLI agent harnesses (OpenClaw, Claude Code, Codex, Hermes Agent); 19 frontier models evaluated; **Opus 4.7 best at 62.2% under OpenClaw, all others <60%**; **harness switching alone shifts a single model by up to 18 pp** | source: https://arxiv.org/abs/2605.10912 | status: open ; **resolves cross-benchmark Opus 4.7 spread** (Vals AI Finance 64.37% / AstaBench 58.0% / WildClawBench 62.2%) as a harness-variance phenomenon; sharpens [arxiv:2605.04624] AuditRepairBench by identifying harness as a leak channel above evaluator reconfiguration; joins [arxiv:2605.12673] BenchJack, [arxiv:2605.10448] Evidence-Supported Bounds, [arxiv:2605.12131] Rollout Cards, [arxiv:2605.12925] AgentLens, [arxiv:2605.11946] Counterfactual Trace Auditing as 6th paper in the agent-eval credibility thread
+- **2026-05-15** [arxiv:2605.15040] CLAIM: Microsoft Research Orchard — open-source agent training framework (not just orchestration) with three competitive recipes: **Orchard-SWE 67.5% SWE-bench Verified**, **Orchard-GUI 4B VLM 74.1% WebVoyager**, **Orchard-Claw 73.9% pass@3**; core innovation is Orchard Env sandbox-lifecycle primitive | source: https://arxiv.org/abs/2605.15040 | status: open ; runtime-substrate convergence with [arxiv:2605.10913] Shepherd typed event trace and [arxiv:2605.06614] SkillOS Markdown SkillRepo — three independent open frameworks pointing at "typed environment substrate, not orchestration glue, is the missing layer"; 4B GUI agent at competitive WebVoyager is local-fit for M5 Max
+- **2026-05-15** [arxiv:2605.13880] CLAIM: PREPING (KAIST AI) — task-free agent memory via Proposer/Solver/Validator separation, validator-gated entry into memory; reaches strong-playbook-baseline performance at **2.99× lower deploy cost on AppWorld, 2.23× on BFCL v3**; success requires explicit proposer control over feasibility/redundancy/coverage plus selective memory updates | source: https://arxiv.org/abs/2605.13880 | status: open ; **positive recipe answer to [arxiv:2605.12978] consolidation-decay** — structured/role-separated/validator-gated consolidation builds memory while background/same-model/default-on consolidation breaks it; same separation pattern as [arxiv:2605.06614] SkillOS curator/executor decomposition
+- **2026-05-15** [arxiv:2605.14678] CLAIM: π-Bench — 100 multi-turn proactive personal-assistant tasks across 5 personas; finds task completion and proactivity are statistically distinct capabilities; prior interactions improve hidden-intent resolution in subsequent tasks | source: https://arxiv.org/abs/2605.14678 | status: open ; capability-side analogue of [microsoft-research-2026-05-11-socialreasoning-bench] "completion ≠ value" finding from a different operational angle; open empirical question whether proactivity/optimality correlate at the model level across both benchmarks
+- **2026-05-15** [arxiv:2605.15177] CLAIM: OpenDeepThink — Bradley-Terry pairwise aggregation across parallel reasoning candidates replaces noisy point-wise LLM-as-judge selection; **Gemini 3.1 Pro Codeforces Elo +405 over 8 LLM-call rounds (~27 min)**; transfers across model strengths without retuning; gains concentrate in objectively verifiable domains and reverse in subjective ones; releases CF-73 dataset (73 expert-rated Codeforces problems, 99% agreement vs. official verdicts) | source: https://arxiv.org/abs/2605.15177 | status: open ; orthogonal to [arxiv:2605.10805] RACER (judge routing as DRO) — OpenDeepThink replaces judgment primitive; RACER routes between judge types; both attack judge-quality bottleneck in TTS
+- **2026-05-15** [arxiv:2605.15155] CLAIM: SDAR (gated self-distillation atop GRPO; sigmoid gate on detached token-level signals amplifies teacher-endorsed tokens, moderates rejection signals) tested on Qwen2.5/3 reports **+9.4% ALFWorld, +7.0% Search-QA, +10.2% WebShop-Acc** over GRPO baselines | source: https://arxiv.org/abs/2605.15155 | status: open ; **4th orthogonal mechanism hitting the ALFWorld 90+ plateau** alongside [arxiv:2605.06078] BEACON (92.9%), [arxiv:2605.06642] StraTA (93.1%), [arxiv:2605.05413] Constant-Context Skill Learning (89.6% unseen) — benchmark now appears method-saturated; gated self-distillation likely to land in trl/verifiers libraries
+- **2026-05-15** [arxiv:2605.15041] CLAIM: CAST treats historical execution trajectories as structured cases, extracts complexity + failure profiles, feeds them into fine-grained reward design and adaptive reasoning during RL; on BFCLv2 + ToolBench reports **+5.85 pp execution accuracy with 26% shorter reasoning length** | source: https://arxiv.org/abs/2605.15041 | status: open ; positive-policy answer to [arxiv:2605.00136] tool-use tax; orthogonal mechanism to [arxiv:2605.00737] hidden-state need estimators (CAST learns from completed traces; hidden-state estimators learn from in-call activations) and [arxiv:2605.07990] linearly-readable tool calls (mechanistic counterpart)
+- **2026-05-15** [hf-blog-2026-05-14-continuous-async] CLAIM: HF Transformers `ContinuousBatchingAsyncIOs` decouples CPU batch prep from GPU compute via 3 CUDA streams (H2D / compute / D2H) + CUDA events + double-buffering + carry-over masks; identifies 24% of total runtime as GPU-idle waiting for CPU prep; achieves **300.6s → 234.5s on 8K tokens, batch 32, 8B model — 22% generation-time speedup; GPU utilization 76.0% → 99.4%**; zero new kernels, no model changes | source: https://huggingface.co/blog/continuous_async | status: open ; portable pattern for llama.cpp and mlx-lm where same CPU-prep / GPU-compute idle-window problem exists on unified-memory architectures; sanity-check ceiling on serving-cost spreadsheets
diff --git a/_meta/processed.json b/_meta/processed.json
@@ -1,4 +1,12 @@
 [
+  { "id": "arxiv:2605.10912", "url": "https://arxiv.org/abs/2605.10912", "title": "WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation", "date_covered": "2026-05-15" },
+  { "id": "arxiv:2605.15040", "url": "https://arxiv.org/abs/2605.15040", "title": "Orchard: An Open-Source Agentic Modeling Framework", "date_covered": "2026-05-15" },
+  { "id": "arxiv:2605.13880", "url": "https://arxiv.org/abs/2605.13880", "title": "PREPING: Building Agent Memory without Tasks", "date_covered": "2026-05-15" },
+  { "id": "arxiv:2605.14678", "url": "https://arxiv.org/abs/2605.14678", "title": "π-Bench: Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflows", "date_covered": "2026-05-15" },
+  { "id": "arxiv:2605.15177", "url": "https://arxiv.org/abs/2605.15177", "title": "OpenDeepThink: Parallel Reasoning via Bradley-Terry Aggregation", "date_covered": "2026-05-15" },
+  { "id": "arxiv:2605.15155", "url": "https://arxiv.org/abs/2605.15155", "title": "Self-Distilled Agentic Reinforcement Learning (SDAR)", "date_covered": "2026-05-15" },
+  { "id": "arxiv:2605.15041", "url": "https://arxiv.org/abs/2605.15041", "title": "CAST: Case-Based Calibration of Adaptive Reasoning and Execution for LLM Tool Use", "date_covered": "2026-05-15" },
+  { "id": "hf-blog-2026-05-14-continuous-async", "url": "https://huggingface.co/blog/continuous_async", "title": "Unlocking asynchronicity in continuous batching", "date_covered": "2026-05-15" },
   { "id": "arxiv:2605.12978", "url": "https://arxiv.org/abs/2605.12978", "title": "Useful Memories Become Faulty When Continuously Updated by LLMs", "date_covered": "2026-05-14" },
   { "id": "arxiv:2605.12922", "url": "https://arxiv.org/abs/2605.12922", "title": "When Attention Closes: How LLMs Lose the Thread in Multi-Turn Interaction", "date_covered": "2026-05-14" },
   { "id": "arxiv:2605.13050", "url": "https://arxiv.org/abs/2605.13050", "title": "Context Training with Active Information Seeking", "date_covered": "2026-05-14" },