From 139f91faf7fd1312449fcc41b2b0283ca2edc9b4 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 22 May 2026 10:16:16 +0000 Subject: [PATCH] digest: 2026-05-22 8 items covering Gemini 3.5 Flash release, Microsoft MagenticLite/Fara1.5, agent-trajectory long-context training (ACC), RL-orchestrator beating GPT-5 (Maestro), NVIDIA Gated DeltaNet-2, skill-hygiene for self-evolving agents (Ratchet), decentralized multi-agent memory (DecentMem), and workflow compilation into weights. 2 conflicts flagged (DecentMem vs. Memory Curse; Gemini 3.5 MCP Atlas first-mover anchor). --- _meta/claims-ledger.md | 9 +++ _meta/processed.json | 9 +++ inbox/2026-05-22.md | 132 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 150 insertions(+) create mode 100644 inbox/2026-05-22.md diff --git a/_meta/claims-ledger.md b/_meta/claims-ledger.md index caba97d..fd2e3de 100644 --- a/_meta/claims-ledger.md +++ b/_meta/claims-ledger.md @@ -107,3 +107,12 @@ - **2026-05-21** [arxiv:2605.21468] CLAIM: RELEX (Yu Meng UVA cluster) — RLVR weight-update trajectories are **extremely low-rank**; rank-1 approximation captures majority of downstream gains; estimating rank-1 subspace from short observation window enables linear-regression extrapolation; matches full RLVR performance with **as few as 15% of training steps**, extrapolation **10–20× beyond observed prefix**; mechanism attributed to denoising effect from subspace projection | source: https://arxiv.org/abs/2605.21468 | status: open ; third independent low-rank linearity finding in the agentic LLM stack alongside [arxiv:2605.07990] (tool calling linearly readable in logits) and [arxiv:2605.14038] (tool-necessity orthogonal probe directions) — three different surfaces converging on dominant linear structure - **2026-05-21** [deepmind-blog-2026-05-19-co-scientist] CLAIM: DeepMind Co-Scientist (Nature publication, May 19) — multi-agent Gemini system with **specialized agents for hypothesis generation, critical evaluation, and ranking** + AlphaGo-style tournament evolution; integrates ChEMBL / UniProt / web search / AlphaFold as external tools; demonstrated across liver fibrosis treatments, ALS research pathways, cellular aging reversal with named external scientist collaborators | source: https://deepmind.google/blog/co-scientist-a-multi-agent-ai-partner-to-accelerate-research/ | status: open ; production-blessed at Nature-publication tier the same role-separation pattern as [arxiv:2605.13880] PREPING (Proposer/Solver/Validator) and [arxiv:2605.18401] SkillsVote (Collection/Search/Post-Execution gating); concrete instantiation of [arxiv:2605.15177] OpenDeepThink Bradley-Terry pairwise aggregation applied to research hypotheses - **2026-05-21** [arxiv:2605.20668] CLAIM: 45 domain scientists rate 2,960 criticisms from reviews of 82 Nature-family papers; **GPT-5.2-powered reviewing agent scores above each paper's top-rated human reviewer (60.0% vs. 48.2%, p = 0.009)**; all AI reviewers exceed lowest-rated human; AI systems exhibit **16 recurring weaknesses humans do not share**; framed as complements not substitutes | source: https://arxiv.org/abs/2605.20668 | status: open ; conflict-pair with [arxiv:2605.19156] ResearchArena (AI fails as author of full papers under artifact-aware review) — both can be true: AI is competitive **as reviewer** while still failing **as author** of artifact-aware-scored work; the asymmetry is the operational signal for product strategy (review/critique surfaces are closer to production-ready than full-artifact-generation surfaces) +- **2026-05-22** [google-blog-2026-05-19-gemini-3-5] CLAIM: Gemini 3.5 Flash (DeepMind / Google AI, May 19 release) reports **Terminal-Bench 2.1 76.2%, GDPval-AA 1656 Elo, MCP Atlas 83.6%, CharXiv Reasoning 84.2%**; **~4× faster output tokens/sec and "less than half the cost" of other frontier models**; outperforms Gemini 3.1 Pro on coding/agentic benchmarks; Pro variant in internal testing | source: https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-5/ | status: open ; vendor-self-reported; **MCP Atlas 83.6% is the first first-party MCP-specific eval number published in a frontier-model release post anywhere**, setting a comparative anchor that Anthropic / OpenAI have not yet provided +- **2026-05-22** [microsoft-research-2026-05-21-magenticlite-fara15] CLAIM: Microsoft Research released **MagenticLite (agentic application on Magentic-UI) + MagenticBrain (14B Qwen 3 14B fine-tune orchestrator) + Fara1.5 (4B / 9B / 27B computer-use family)** with all three components downloadable (GitHub + Microsoft Foundry); **Fara1.5-9B is SOTA among small models on OnlineMind2Web (300 web tasks), nearly doubling Fara-7B; 27B reaches >90%**; harness has FastAPI orchestration with context compaction + critical-point detection, sandboxed by Quicksand (QEMU) | source: https://www.microsoft.com/en-us/research/blog/magenticlite-magenticbrain-fara1-5-an-agentic-experience-optimized-for-small-models/ | status: open ; largest small-model agent-system release this cycle; pairs with [hf-blog-2026-04-29-ibm-granite-4-1] Granite 4.1-8B and [hf-blog-2026-05-08-allenai-emo] EMO as third lab arguing tight harness ↔ small-model codesign > generic frontier API for production agents +- **2026-05-22** [arxiv:2605.21850] CLAIM: ACC (Agent Context Compilation) converts existing agent trajectories from search / SWE / DB-query agents into long-context QA pairs by combining the original question with multi-turn tool responses + environment observations; **Qwen3-30B-A3B + ACC reaches MRCR 68.3 (+18.1) and GraphWalks 77.5 (+7.6), comparable to Qwen3-235B-A22B**, while preserving GPQA / MMLU-Pro / AIME / IFEval; mechanism shows task-adaptive attention restructuring + expert specialization; combinable with any long-context extension method | source: https://arxiv.org/abs/2605.21850 | status: open ; first paper to use agent-harness rollouts as long-context training data; closes the harness ↔ base-model loop; supports the **30B-A3B-as-local-fit-sweet-spot** thesis alongside [arxiv:2605.05191] Context-ReAct and [arxiv:2605.04050] LCM +- **2026-05-22** [arxiv:2605.22177] CLAIM: Maestro (Jianhua Tao cluster) RL-trains a **4B lightweight orchestrator** to compose ensembles of frozen expert models + a two-tier skill library; reports **70.1% average across 10 multimodal benchmarks vs. GPT-5 69.3% and Gemini-2.5-Pro 68.7%**; **59.5% on 4 challenging benchmarks when augmented with out-of-domain experts**; code released | source: https://arxiv.org/abs/2605.22177 | status: open ; second result in 30 days (alongside [arxiv:2605.05007] Uno-Orchestra) arguing **structured / learned orchestration over frozen experts > monolithic frontier model** as the cost-quality frontier; converges with [arxiv:2605.06614] SkillOS, [arxiv:2605.05413] Constant-Context Skill Learning, [arxiv:2605.18401] SkillsVote on explicit-skill-library pattern (4 independent confirmations in 4 weeks); open question whether Maestro + Uno-Orchestra compose +- **2026-05-22** [arxiv:2605.22791] CLAIM: Gated DeltaNet-2 (NVIDIA — Hatamizadeh / Choi / Kautz) decouples linear-attention memory management into **channel-wise erase gate b_t + channel-wise write gate w_t** (vs. prior single scalar gate); 1.3B params on 100B FineWeb-Edu tokens outperforms Mamba-2 / Gated DeltaNet / KDA / Mamba-3 across recurrent-only and hybrid settings, with named improvement on **RULER multi-key needle-in-a-haystack**; code released under NVIDIA Source Code License-NC; **no pre-trained weights yet** | source: https://arxiv.org/abs/2605.22791 | status: open ; conceptual parallel to [arxiv:2605.21463] Mem-π decoupling **when-to-write from what-to-write** — same insight at two scales (architecture-level recurrent state vs. agent-level explicit memory) in the same week +- **2026-05-22** [arxiv:2605.22148] CLAIM: Ratchet — single-agent skill-lifecycle loop in which a **frozen Claude Opus 4.7** writes / retrieves / curates / retires natural-language skills via 4 hygiene mechanisms (outcome-driven retirement, bounded active-cap, meta-skill authoring, pattern canonicalisation); **MBPP+ hard-100 pass@1 lifts from 0.258 baseline → 0.584 rolling mean (peak 0.658) over 100 rounds; SWE-bench Verified +0.22 peak over 20 rounds**; ablations isolate **retirement + meta-skill authoring as load-bearing components** | source: https://arxiv.org/abs/2605.22148 | status: open ; minimal-end of the skill-curation spectrum joining [arxiv:2605.06614] SkillOS, [arxiv:2605.13941] EvolveMem, [arxiv:2605.18401] SkillsVote, [arxiv:2605.21463] Mem-π — 5 independent papers in 5 weeks on skill-libraries + hygiene > raw memory accumulation; open question which primitives compose +- **2026-05-22** [arxiv:2605.22721] CLAIM: DecentMem (Hao / Long / Zhao) — multi-agent systems with **decentralized dual-pool memory** (exploitation pool of consolidated past trajectories + exploration pool of LLM-generated candidates), reweighted via LLM-as-judge feedback; theoretical O(log T) regret matching stochastic bandit lower bounds; **+23.8% accuracy over centralized memory, +52.5% over no-memory, 49% token reduction** across AutoGen / DyLAN / AgentNet × 5 benchmarks × Qwen3-4B/8B/14B + Gemma4-E2B/E4B | source: https://arxiv.org/abs/2605.22721 | status: open ; **directional contradiction with [arxiv:2605.08060] Memory Curse** (more accessible history degrades cooperation in 18/28 settings) — both can be true if the mechanism is task-evidence (DecentMem) vs. cooperation (Memory Curse); ledger now holds **conditional claim**: "more memory helps under decentralized dual-pool on task-evidence tasks; harms under flat history on cooperation"; empirical resolution requires running DecentMem under the Memory Curse 4-game protocol +- **2026-05-22** [arxiv:2605.22502] CLAIM: "Subterranean agents" — compiling agentic workflows into LLM weights (rather than running through external orchestrators) tested on **3 real workflows (travel booking 14 nodes, Zoom support 14 nodes, insurance claims 55 nodes with 6 decision hubs)** reports **near-frontier quality at two orders of magnitude less cost** vs. equivalent external-orchestrator implementation; abstract notes existing agent-orchestration framework ecosystem totals >290,000 GitHub stars | source: https://arxiv.org/abs/2605.22502 | status: open ; third paper this cycle (alongside [arxiv:2605.05007] Uno-Orchestra and [arxiv:2605.22177] Maestro) arguing **the orchestration cost line is collapsing under learned / compiled approaches**; no code release per abstract; abstract does not decompose "near-frontier quality" — needs evaluation on **held-out workflow variants** per [arxiv:2605.21384] SpecBench's visible↔held-out gap framework before treating the 100× as portable +- **2026-05-22** [google-research-2026-05-19-era] CLAIM: ERA (Google Research) — Gemini-based AI tool for expert-level scientific coding; uses **tree-search to optimize output code** by searching literature, writing/refining code, combining techniques, evaluating results; reports "expert-level performance" across genomics / public health / satellite imagery / neuroscience prediction / time-series / mathematics; **open-source at google-research/era** | source: https://research.google/blog/empirical-research-assistance-era-from-nature-publication-to-catalyzing-computational-discovery/ | status: open ; complements [deepmind-blog-2026-05-19-co-scientist] Co-Scientist (hypothesis generation vs. ERA's code optimization) — same lab, same week, two complementary research-agent surfaces; covers a different stage of the scientific-method workflow diff --git a/_meta/processed.json b/_meta/processed.json index 4066550..7bd709d 100644 --- a/_meta/processed.json +++ b/_meta/processed.json @@ -1,4 +1,13 @@ [ + { "id": "google-blog-2026-05-19-gemini-3-5", "url": "https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-5/", "title": "Gemini 3.5: Frontier intelligence with action (Flash release)", "date_covered": "2026-05-22" }, + { "id": "microsoft-research-2026-05-21-magenticlite-fara15", "url": "https://www.microsoft.com/en-us/research/blog/magenticlite-magenticbrain-fara1-5-an-agentic-experience-optimized-for-small-models/", "title": "MagenticLite, MagenticBrain, Fara1.5: An agentic experience optimized for small models", "date_covered": "2026-05-22" }, + { "id": "arxiv:2605.21850", "url": "https://arxiv.org/abs/2605.21850", "title": "ACC: Compiling Agent Trajectories for Long-Context Training", "date_covered": "2026-05-22" }, + { "id": "arxiv:2605.22177", "url": "https://arxiv.org/abs/2605.22177", "title": "Maestro: Reinforcement Learning to Orchestrate Hierarchical Model-Skill Ensembles", "date_covered": "2026-05-22" }, + { "id": "arxiv:2605.22791", "url": "https://arxiv.org/abs/2605.22791", "title": "Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention", "date_covered": "2026-05-22" }, + { "id": "arxiv:2605.22148", "url": "https://arxiv.org/abs/2605.22148", "title": "Ratchet: A Minimal Hygiene Recipe for Self-Evolving LLM Agents", "date_covered": "2026-05-22" }, + { "id": "arxiv:2605.22721", "url": "https://arxiv.org/abs/2605.22721", "title": "DecentMem: Self-Evolving Multi-Agent Systems via Decentralized Memory", "date_covered": "2026-05-22" }, + { "id": "arxiv:2605.22502", "url": "https://arxiv.org/abs/2605.22502", "title": "Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Cost", "date_covered": "2026-05-22" }, + { "id": "google-research-2026-05-19-era", "url": "https://research.google/blog/empirical-research-assistance-era-from-nature-publication-to-catalyzing-computational-discovery/", "title": "Empirical Research Assistance (ERA): From Nature publication to catalyzing Computational Discovery", "date_covered": "2026-05-22" }, { "id": "arxiv:2605.21463", "url": "https://arxiv.org/abs/2605.21463", "title": "Mem-π: Adaptive Memory through Learning When and What to Generate", "date_covered": "2026-05-21" }, { "id": "arxiv:2605.19660", "url": "https://arxiv.org/abs/2605.19660", "title": "OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond", "date_covered": "2026-05-21" }, { "id": "arxiv:2605.21226", "url": "https://arxiv.org/abs/2605.21226", "title": "OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under Optimal Squared Error Quantization", "date_covered": "2026-05-21" }, diff --git a/inbox/2026-05-22.md b/inbox/2026-05-22.md new file mode 100644 index 0000000..5b166f0 --- /dev/null +++ b/inbox/2026-05-22.md @@ -0,0 +1,132 @@ +--- +date: 2026-05-22 +type: research-digest +tags: [ai-research, daily-digest] +sources_checked: [arxiv-cs.AI, arxiv-cs.CL, arxiv-cs.MA, arxiv-cs.LG, hf-daily-papers, hf-blog, anthropic-news, anthropic-research, deepmind-blog, google-research-blog, microsoft-research-blog, openai-news, ai-meta-research, allenai-blog] +items_count: 8 +conflicts_flagged: 2 +--- + +# AI Research Digest — Friday, May 22, 2026 + +## TL;DR +- **Two production-grade small-model agent stacks shipped with weights this week.** Microsoft Research released **MagenticLite + MagenticBrain (14B Qwen3-FT orchestrator) + Fara1.5 (4B / 9B / 27B computer-use family)** on May 21 — the 9B variant "nearly doubles" Fara-7B on OnlineMind2Web and the 27B reaches >90%. Direct local-fit for the M5 Max — the entire agentic loop (orchestrator + computer-use) runs in <40 GB and the codesign (app + harness + models) is the operational pattern Justin's MCP companion stack should mirror. **All three components released** (MagenticLite GitHub, MagenticBrain + Fara1.5 on Microsoft Foundry). +- **Gemini 3.5 Flash is the new frontier-agent SOTA at half the cost.** May 19 release (flagged for follow-up in yesterday's digest, picked up today). Reported **Terminal-Bench 2.1 76.2%, GDPval-AA 1656 Elo, MCP Atlas 83.6%, CharXiv 84.2%**, with rollout across Google Antigravity, Gemini API, AI Studio, Android Studio, Gemini Enterprise. Pro variant still in internal testing. **"MCP Atlas 83.6%" is the first frontier-model MCP-specific eval surface that's published in a release post** — sets a comparative floor for any Anthropic / OpenAI MCP-tool benchmarking. +- **Long-context training gets a new free axis: compile agent trajectories.** Qisheng Su et al.'s **ACC (Agent Context Compilation)** converts existing agent rollouts (search / SWE / DB queries) into long-context QA pairs — no curation cost, scales with agent traffic. **Qwen3-30B-A3B with ACC matches Qwen3-235B-A22B on MRCR (68.3, +18.1) and GraphWalks (77.5, +7.6)** while preserving GPQA / MMLU-Pro / AIME / IFEval. The "agent rollouts are also long-context training data" angle is the first time the agent-harness flywheel feeds back into base-model long-context capability without separate curation. +- **A 4B RL orchestrator beat GPT-5 and Gemini-2.5-Pro on 10 multimodal benchmarks.** **Maestro (Jinyang Wu et al.)** treats heterogeneous multimodal tasks as sequential decisions over a frozen-expert + skill-library registry — **4B orchestrator reports 70.1% average vs. GPT-5 69.3% and Gemini-2.5-Pro 68.7%, with 59.5% on 4 hard benchmarks when augmented with out-of-domain experts**. Code released. This is the second result in 30 days (cf. [arxiv:2605.05007] Uno-Orchestra +16pp at ~10× lower cost) confirming **structured / learned orchestration over frozen experts > monolithic frontier model** as the cost-quality frontier for agent products. +- **NVIDIA released code (NC license) for the next linear-attention architecture: Gated DeltaNet-2.** Decouples the single scalar delta-rule gate into **channel-wise erase + channel-wise write gates**. Outperforms Mamba-2, Gated DeltaNet, KDA, Mamba-3 at 1.3B / 100B-token scale, with named improvements on RULER multi-key retrieval. The "linear attention is finally competitive on long-context needle-style tasks" thesis gets another datapoint — particularly relevant for any future MLX port targeting M5 Max long-context inference. +- **Conflict-of-the-week: "more memory" splits two ways.** **DecentMem (Hao et al.)** reports a **decentralized dual-pool memory beats centralized memory by +23.8% and no-memory by +52.5%** across AutoGen / DyLAN / AgentNet on Qwen3 4/8/14B + Gemma4 E2B/E4B. **Counter-pull**: this directly contradicts the [arxiv:2605.08060] "Memory Curse" finding from May 11 that **expanded accessible history degrades cooperation in 18/28 settings**. Both can be true if the failure mode is *coordination/cooperation* memory vs. *task-evidence* memory — but the ledger now has explicit competing claims on whether centralized vs. decentralized + dual-pool structure inherits the Memory Curse. + +## Items + +### 1. Gemini 3.5 Flash: Frontier Intelligence with Action (Google / DeepMind) +- **Source:** blog.google (May 19, 2026; surfaced through DeepMind discover/blog index) +- **Link:** https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-5/ +- **Authors / org:** Google DeepMind / Google AI +- **Focus area:** Frontier models / Coding agents / Tools & connectors +- **What it is:** Gemini 3.5 release. **Flash variant available immediately**, Pro in internal testing with rollout planned for next month. Reported headline benchmarks for 3.5 Flash: **Terminal-Bench 2.1 = 76.2%**, **GDPval-AA = 1656 Elo**, **MCP Atlas = 83.6%**, **CharXiv Reasoning = 84.2%**. "Outperforms Gemini 3.1 Pro on challenging coding and agentic benchmarks" and lands top-right on the Artificial Analysis intelligence-vs-speed index. **~4× faster output tok/s than other frontier models, "less than half the cost"** per the release post. Deployed across Gemini app / AI Mode / Antigravity platform / Gemini API / Android Studio / Enterprise Agent Platform. +- **Why it matters for Justin's agents:** Three direct pulls. (a) **MCP Atlas 83.6% is the first first-party MCP-eval number in a frontier-model release post** — anywhere. Anthropic's and OpenAI's MCP-tool releases have not published an MCP-specific eval surface; Google did. This sets a published-floor for Justin's MCP companion stack: if a custom MCP server + custom routing scaffold can't beat the bare-model Gemini-3.5-Flash on MCP Atlas, the scaffold is value-negative. (b) **Terminal-Bench 2.1 76.2% + GDPval-AA 1656 Elo at 4× speed and ~½ cost** is the **price-performance redraw** for any coding-agent product that buys frontier-model API tokens — including the Claude-Code-style harness Justin already operates. (c) **The Flash-before-Pro release order signals the agentic / coding tier is the priority deliverable**; Pro will follow with the long-horizon / reasoning advantage. Operationally this means **Flash is the right baseline to A/B against Justin's federal-data agent stack today**, not a deferred experiment. +- **Action signal:** `try-now` `compare` — wire Gemini 3.5 Flash into Justin's existing MCP-tool routing harness behind Switchcraft-style ([arxiv:2605.07112]) gating; measure Terminal-Bench / MCP-task surfaces against current Opus 4.7 / Sonnet 4.6 cost-per-task. Flag the MCP Atlas number as the new third-party comparative anchor in the claims ledger. + +### 2. MagenticLite + MagenticBrain + Fara1.5: Small-Model Agentic Experience (Microsoft Research) +- **Source:** Microsoft Research Blog (May 21, 2026) +- **Link:** https://www.microsoft.com/en-us/research/blog/magenticlite-magenticbrain-fara1-5-an-agentic-experience-optimized-for-small-models/ +- **Authors / org:** Microsoft Research (Magentic-One / Magentic-UI lineage) +- **Focus area:** Local & open models / Tools & connectors / Agent orchestration +- **What it is:** A three-layer agentic stack codesigned for small models. **MagenticLite** = experimental agentic application on top of Magentic-UI that operates across browser + local file system with approval gates and a QEMU-based sandbox (Quicksand). **MagenticBrain** = a **14B-parameter orchestrator fine-tuned from Qwen 3 14B** that plans / codes / delegates. **Fara1.5** = a computer-use model family in **4B / 9B / 27B sizes** for browser automation. Reported: **Fara1.5-9B nearly doubles Fara-7B on OnlineMind2Web (300 web tasks), SOTA among small models**; **27B reaches >90% on the same benchmark**. The harness is FastAPI with explicit context compaction + critical-point detection. **All three components released**: MagenticLite on GitHub, MagenticBrain + Fara1.5 on Microsoft Foundry. +- **Why it matters for Justin's agents:** This is the single largest **`local-fit`** release this week and arguably this month. Three pulls. (a) **MagenticBrain (14B Qwen3-FT) + Fara1.5-9B fits in <40 GB on M5 Max** with room for KV cache and tool-output context — the **entire two-model agentic loop is locally hostable**, no API dependency. The Magentic-One architecture (orchestrator + specialized worker models) is now downloadable. (b) The **codesign principle** ("we trained MagenticBrain with awareness of MagenticLite's harness behavior") is the operational pattern Justin's MCP companion stack should mirror: don't bolt a generic small model onto a generic harness — the gains in the SOTA-among-small-models number come from the **harness ↔ model coevolution**, not either layer alone. (c) **Quicksand sandbox + approval gates + critical-point detection** is the right safety-and-trust scaffold for any companion app that touches a user's local filesystem; the design is now public reference for what production-grade computer-use looks like at the small-model tier. Pairs with [hf-blog-2026-04-29-ibm-granite-4-1] Granite 4.1 and [hf-blog-2026-05-08-allenai-emo] EMO as the third major small-model-system release of the cycle — three independent labs now arguing the **<20B-active model + tight harness > generic frontier-model API call** trade exists for production agents. +- **Action signal:** `try-now` `local-fit` — pull MagenticLite from GitHub, run MagenticBrain (14B) + Fara1.5-9B locally on M5 Max, A/B against current Opus-4.7-via-API browsing/file workflow on a federal-data fetching task. The headline question to verify: does the harness ↔ small-model codesign actually beat frontier-API at Justin's task latency / token-cost envelope? + +### 3. ACC: Compiling Agent Trajectories for Long-Context Training +- **Source:** arXiv (cs.CL / cs.LG, submitted May 21, 2026) +- **Link:** https://arxiv.org/abs/2605.21850 +- **Authors / org:** Qisheng Su, Zhen Fang, Shiting Huang, Yu Zeng, Yiming Zhao, Kou Shi, Ziao Zhang, Lin Chen, Zehui Chen, Lijun Wu, Feng Zhao (affiliation not disclosed in abstract; author cluster looks USTC / cross-institution) +- **Focus area:** Context engineering / Training methodology +- **What it is:** Names a supervision blind spot — **standard agent SFT masks tool responses and only trains turn-level tool selection**, leaving the scattered evidence across tool calls unused as training signal. **ACC (Agent Context Compilation)** converts existing agent trajectories from search, SWE, and DB-query agents into **long-context QA pairs** by combining the original question with all multi-turn tool responses + environment observations, and training the model to answer directly (no tool use) over the assembled context. Reports: **Qwen3-30B-A3B with ACC reaches MRCR 68.3 (+18.1) and GraphWalks 77.5 (+7.6) — comparable to Qwen3-235B-A22B**, while preserving GPQA / MMLU-Pro / AIME / IFEval. Mechanism analysis: ACC-trained model exhibits **task-adaptive attention restructuring + expert specialization**. Combinable with any existing long-context extension or training method. +- **Why it matters for Justin's agents:** Two operational implications and one mechanistic one. (a) **The agent-harness rollout flywheel is now a training resource for long-context capability** — every MCP companion-app rollout Justin generates is convertible into ACC-style long-context QA data with no annotation cost. The harness ↔ base-model loop closes. (b) The **30B-A3B-matches-235B-A22B-on-long-context-QA** result is the **clean operational case** for staying on the 30B-A3B tier locally: if MRCR and GraphWalks (cross-turn coreference, graph traversal) are saturated at 30B with ACC, the marginal gain from running 235B disappears for these workload classes. (c) The **attention restructuring + expert specialization** observation says **MoE experts can learn long-context dependencies that dense models in the same family don't** when trained on the right signal — relevant to all Qwen3-30B-A3B-class production decisions. Pairs with [arxiv:2605.05191] Context-ReAct (5 atomic operators) and [arxiv:2605.04050] LCM (recursive context compression) as the third mechanistically-distinct attack on the "context is the bottleneck for 30B-class agents" problem; all three converge on the 30B-A3B sweet spot. +- **Action signal:** `try-now` `local-fit` — when ACC code or training recipe drops, the technique is portable: any Justin-produced agent trajectory becomes long-context QA training data for a downstream Qwen3-30B-A3B LoRA. The flywheel is the experimental design. + +### 4. Maestro: RL Orchestration of Hierarchical Model-Skill Ensembles +- **Source:** arXiv (cs.AI / cs.MM, submitted May 21, 2026) +- **Link:** https://arxiv.org/abs/2605.22177 +- **Authors / org:** Jinyang Wu, Guocheng Zhai, Ruihan Jin, Yuhao Shen, Zhengxi Lu, Fan Zhang, Haoran Luo, Zheng Lian, Zhengqi Wen, Jianhua Tao (Jianhua Tao is a recognized multimodal / affective-computing PI, likely THU / NLPR) +- **Focus area:** Agent orchestration / Frontier models +- **What it is:** Reframes heterogeneous multimodal tasks as **sequential decision-making over a hierarchical model-skill registry**. A **lightweight 4B policy** is RL-trained to dynamically compose ensembles of **frozen expert models** plus a **two-tier skill library**, deciding when to invoke external experts and which model-skill pairs to select. Reports: **70.1% avg across 10 multimodal benchmarks, beating GPT-5 (69.3%) and Gemini-2.5-Pro (68.7%)**; **59.5% avg on 4 challenging benchmarks when augmented with out-of-domain experts**. Maintains low latency. **Code released**. +- **Why it matters for Justin's agents:** This is the **strongest "small-RL-policy + frozen-expert ensemble" result** so far — the orchestrator is 4B (cheap to train and serve), the experts are frozen (no fine-tune cost), and the combination beats two of the strongest closed-source frontier models on the benchmark suite. Three operational pulls. (a) **The dispatch / routing problem in Justin's MCP companion stack** (which model handles which sub-task) is reframed: the *router* is the part to train (small, RL); the *workers* are frozen frontier models pulled by API or small models pulled locally. The 4B-orchestrator scale is **trainable on rented compute, deployable on M5 Max**. (b) The **"two-tier skill library"** echoes [arxiv:2605.06614] SkillOS, [arxiv:2605.05413] Constant-Context Skill Learning, and [arxiv:2605.18401] SkillsVote — **four independent confirmations in 4 weeks that explicit skill-libraries beat free-form prompting** for agent orchestration. (c) Pairs as **second confirmation** of [arxiv:2605.05007] Uno-Orchestra (+16pp over strongest workflow at ~10× lower cost) — both papers say **learned structured orchestration is the new SOTA frontier, not "bigger model"**. Open question: do Maestro + Uno-Orchestra's mechanisms compose, or do they collide on the same axis? +- **Action signal:** `try-now` `compare` — clone Maestro repo; replace the multimodal benchmarks with Justin's federal-data tool-routing scenarios; ask whether a 4B-orchestrator-over-frozen-frontier-experts pattern survives the workload swap. The composability-with-Uno-Orchestra question is worth a dedicated experiment. + +### 5. Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention (NVIDIA) +- **Source:** arXiv (cs.LG / cs.CL, submitted May 21, 2026) +- **Link:** https://arxiv.org/abs/2605.22791 +- **Authors / org:** Ali Hatamizadeh, Yejin Choi, Jan Kautz — **NVIDIA** (Hatamizadeh + Kautz on the architecture side; Yejin Choi co-affiliated) +- **Focus area:** Context engineering / Local & open models +- **What it is:** Builds on the linear-attention / delta-rule line (Gated DeltaNet, KDA, Mamba-2/3). Identifies that a **single scalar gate conflates two distinct memory operations** — erasing what's stored and writing what's new. Replaces it with **channel-wise erase gate b_t + channel-wise write gate w_t**, with fast-weight WY chunkwise algorithm and gate-aware backward fused in Triton. Reports: **1.3B-parameter models, 100B FineWeb-Edu tokens, outperforms Mamba-2 / Gated DeltaNet / KDA / Mamba-3** across recurrent-only and hybrid settings, with **explicit improvement on RULER multi-key needle-in-a-haystack**. **Code released** at github.com/NVlabs/GatedDeltaNet-2 under NVIDIA Source Code License-NC (training code; **no pre-trained weights yet**). +- **Why it matters for Justin's agents:** Linear-attention architectures matter for local-fit because they decouple memory cost from sequence length. Three pulls. (a) **The RULER multi-key improvement is the test that's been hardest for linear attention** — Mamba-class architectures historically lose to softmax-attention transformers on multi-key recall. Gated DeltaNet-2 narrowing or closing this gap is the precondition for *any* linear-attention model to be the default architecture for long-context agentic workloads. (b) The **erase/write decoupling** is mechanistically novel — it argues memory management in a recurrent network needs at least two orthogonal controls (forget rate + integration rate), and that conflating them was the failure mode in prior gated variants. **Direct conceptual parallel** to Mem-π [arxiv:2605.21463] decoupling "when to write" from "what to write" at the agent-memory layer — same week, same insight at two different scales (architecture-level recurrent state vs. agent-level explicit memory). (c) NC license means **research-only** for now; Justin can study the algorithm but can't ship a product on it. The MLX port is feasible at the architecture level if a permissively-licensed reimplementation appears. +- **Action signal:** `track` `local-fit` — wait for pre-trained weights or a permissively-licensed reimplementation. The conceptual takeaway (erase ≠ write, decouple them) is portable to any memory-management module Justin builds — flag in claims ledger and revisit when an MLX-compatible Gated DeltaNet-2 lands. + +### 6. Ratchet: Minimal Hygiene Recipe for Self-Evolving LLM Agents +- **Source:** arXiv (cs.AI / cs.SE, submitted May 21, 2026; CC BY-NC-SA 4.0) +- **Link:** https://arxiv.org/abs/2605.22148 +- **Authors / org:** Xing Zhang, Yanwei Cui, Guanghui Wang, Ziyuan Li, Wei Qiu, Bing Zhu, Peiyang He (affiliation not disclosed) +- **Focus area:** Agent memory / Coding agents +- **What it is:** **Single-agent skill-lifecycle loop** where a frozen LLM writes, retrieves, curates, and retires its own natural-language skills. Four hygiene mechanisms: **outcome-driven retirement, bounded active-cap, meta-skill authoring guidance, pattern canonicalisation**. Reported (with **Claude Opus 4.7** as the frozen LLM): **MBPP+ hard-100 pass@1 from 0.258 → 0.584 rolling mean (peak 0.658) over 100 rounds**; **SWE-bench Verified +0.22 peak lift over 20 rounds**. Eight ablations isolate retirement and meta-skill authoring as the **load-bearing components**. +- **Why it matters for Justin's agents:** The clean operational claim is **"a frozen Opus 4.7 with a Ratchet skill-curation loop ~doubles MBPP+ hard-100 pass@1"** — without any fine-tuning, weight changes, or new model. Three pulls. (a) The **load-bearing-mechanism ablation** (retirement + meta-skill authoring carry the lift; bounded cap + canonicalization secondary) is the right *minimal* prescription — most self-evolving agent stacks over-engineer the curation layer; Ratchet says **the two operations that matter are killing bad skills and writing meta-skills**. (b) **Opus 4.7 is Justin's working model** (per Anthropic finance-agents release ledger entry) — this is a directly applicable recipe, not a research demo on a different frontier. The SWE-bench Verified +0.22 lift is in the same regime as the gains Anthropic itself ships with each Claude generation. (c) Pairs with [arxiv:2605.06614] SkillOS (frozen executor + trainable curator, Markdown SkillRepo), [arxiv:2605.13941] EvolveMem (AutoResearch-based self-evolving memory), and [arxiv:2605.18401] SkillsVote (lifecycle governance with collection / recommendation / evolution) — Ratchet is the **minimal** end of the skill-curation spectrum, the others scale up. Five independent papers in 5 weeks now argue skill-libraries + hygiene > raw memory accumulation. The **convergence is strong**; the open question is **which curation primitives compose** and which are redundant. +- **Action signal:** `try-now` — Ratchet has no public code per the abstract, but the four hygiene mechanisms (outcome-driven retirement, bounded cap, meta-skill authoring, canonicalization) are portable today. Add a retirement primitive to any Justin-built Claude Opus 4.7 + MCP skill-library workflow; measure delta on a held-out coding subtask suite. The published number to beat: **~doubling pass@1 over 100 rounds**. + +### 7. DecentMem: Self-Evolving Multi-Agent Systems via Decentralized Memory +- **Source:** arXiv (cs.MA / cs.CL, submitted May 21, 2026) +- **Link:** https://arxiv.org/abs/2605.22721 +- **Authors / org:** Guangya Hao, Yunbo Long, Zhuokai Zhao (affiliation not disclosed) +- **Focus area:** Agent memory / Agent orchestration +- **What it is:** Each agent in a multi-agent system maintains a **dual-pool memory** — an **exploitation pool** of consolidated past trajectories + an **exploration pool** of LLM-generated candidates for unseen contexts — reweighted via LLM-as-judge feedback. **Theoretical claim:** O(log T) cumulative regret matching stochastic bandit lower bounds. **Empirical:** **+23.8% accuracy over centralized-memory baselines, +52.5% over no-memory, 49% token reduction** across three MAS frameworks (AutoGen, DyLAN, AgentNet) × five benchmarks (math, code, QA, embodied) × backbones (Qwen3 4B/8B/14B and Gemma4 E2B/E4B). +- **Why it matters for Justin's agents:** Two operational pulls and one open conflict. (a) **The decentralized + dual-pool structure is the production-realistic pattern for any multi-agent MCP stack**: each subagent owns its own memory, exploitation vs. exploration is explicit, no global shared store. The 49% token reduction is the cost number that makes this **operationally cheaper than centralized memory**, not just better. (b) The backbone list (Qwen3 4/8/14B + Gemma4 E2B/E4B) is **exactly the local-fit set on M5 Max** — every model in the eval is runnable in Justin's hardware envelope. This is not a frontier-API result that doesn't port; it's a local-multi-agent result that does. (c) **Open conflict with [arxiv:2605.08060] Memory Curse** (May 11): the Memory Curse paper reported expanded accessible history *degrades* cooperation in 18 of 28 model-game settings, with mechanism = longer recall erodes forward-looking cooperative intent. DecentMem reports the opposite directional effect when memory is *decentralized* and *dual-pool*. Both can be true if the failure mode is cooperation-specific (Memory Curse) vs. task-evidence (DecentMem) — the actionable question is whether the dual-pool / decentralization structure inherits the curse on cooperative subtasks. Cross-reference both in claims ledger. +- **Action signal:** `try-now` `compare` `local-fit` — DecentMem with Gemma4-E4B or Qwen3-14B backbones is directly testable on M5 Max under AutoGen or AgentNet harnesses. **The first experiment to run is the cross-replication of Memory Curse**: does the DecentMem dual-pool design degrade cooperation under the same 4-game protocol, or does it escape the curse? + +### 8. Compiling Agentic Workflows into LLM Weights +- **Source:** arXiv (cs.AI, submitted May 21, 2026) +- **Link:** https://arxiv.org/abs/2605.22502 +- **Authors / org:** Simon Dennis, Rivaan Patil, Kevin Shabahang, Hao Guo (affiliation not disclosed) +- **Focus area:** Agent orchestration / Local & open models +- **What it is:** Argues that **"subterranean agents"** — fine-tuned models with procedural logic compiled into their weights — should compete with the ~290,000+ GitHub-star ecosystem of external orchestrators (LangGraph, AutoGen, CrewAI, etc.). Tests across three real workflows: **travel booking (14 nodes), Zoom support (14 nodes with product-specific knowledge), insurance claims (55 nodes with 6 decision hubs)**. Reports **"near-frontier" quality at two orders of magnitude less cost** vs. the equivalent external-orchestrator implementation. +- **Why it matters for Justin's agents:** This is the **orthogonal axis** to the orchestration zoo Justin uses today (MCP + multiple framework adapters + tool routing). Three pulls. (a) If the **workflow logic is stable enough to be worth fine-tuning into weights** (federal-data ETL pipelines are exactly this — high-volume, mostly deterministic, occasional branching), then **compiling the workflow into a small-model weight set is a 100×-cost-reduction lever** vs. paying frontier-model tokens per orchestration step. (b) The 14-node / 14-node / 55-node workflow complexities are **typical of Justin's actual MCP-server-fronted business logic** — not contrived. This is a real comparison point. (c) **The headline cost number (100×) is large enough that it's worth verifying independently** — the abstract gives the *direction* but not the breakdown (what counts as "quality," which workflows survive compilation). Pairs with [arxiv:2605.05007] Uno-Orchestra (~10× cost reduction at +16pp accuracy via learned orchestration) and [arxiv:2605.22177] Maestro (4B orchestrator beats GPT-5) — **three convergent papers this cycle arguing the orchestration cost line is collapsing under learned/compiled approaches**. Open question: where's the failure mode? Compiled workflows can't introspect; if the workflow changes, you re-train. The trade-off vs. external orchestrators is rigidity ↔ cost. +- **Action signal:** `track` — no code release per abstract; wait for either a code drop or a third-party reproduction of the 100× cost claim. The **portable pattern today**: for any Justin-built MCP workflow with stable structure, prototype a small-model fine-tune that absorbs the workflow logic vs. the equivalent LangGraph / framework orchestration; measure cost-per-task and quality on held-out cases. If the compiled version is within 5 pp of the orchestrated version, the cost lever is real for that workflow class. + +## Local-model corner +Two items today qualify as `local-fit`: + +**MagenticBrain (14B Qwen3-FT) + Fara1.5-9B (Microsoft Research)** +- **Model:** MagenticBrain (14B, Qwen 3 14B fine-tune) + Fara1.5-9B (computer-use specialist) +- **Memory fit:** ~28 GB + ~18 GB at BF16 = ~46 GB; comfortably in 64 GB unified memory with 18+ GB headroom for KV cache, tool-output context, and the MagenticLite harness. At Q4_K_M (when GGUF conversions appear), both models drop to ~7 GB + ~5 GB = ~12 GB combined, leaving ~50 GB for very long context. +- **Realistic tasks:** Browser automation + local-filesystem operations through approval-gated workflow; Fara1.5-9B is reported SOTA among small models on OnlineMind2Web. Not yet validated outside the Microsoft Research demo workflows — verify before betting product-line architecture on it. +- **How to run:** MagenticLite is the FastAPI harness (clone from GitHub); MagenticBrain + Fara1.5 weights pulled from Microsoft Foundry. **Apple Silicon support is not first-party** — initial path is via llama.cpp / MLX after community GGUF/MLX conversions; until then, native PyTorch on M5 Max via MPS backend. +- **Speed estimate:** No M5 Max-specific benchmarks published; for reference, Qwen 3 14B on M5 Max runs ~22–28 tok/s at Q4_K_M in llama.cpp builds optimized for Metal. Two-model orchestrator + worker pipeline will be bounded by the slower of the two passes. + +**ACC-trained Qwen3-30B-A3B (paper-claimed, weights pending)** +- **Model:** Qwen3-30B-A3B + ACC fine-tune (paper claims match Qwen3-235B-A22B on MRCR / GraphWalks) +- **Memory fit:** Base Qwen3-30B-A3B at Q4_K_M fits ~17 GB; with the ACC training applied as a LoRA delta, total stays well under 24 GB even with extended context. KV-cache headroom for 128K context is ~12–16 GB depending on quantization → comfortably fits. +- **Realistic tasks:** Long-context QA over assembled tool-output context; specifically the cross-turn coreference + graph-traversal workload class. Not validated for free-form reasoning or general chat — the ACC training target is narrow. +- **How to run:** When training recipe drops, recommended path is to LoRA-fine-tune existing Qwen3-30B-A3B on user-generated agent trajectories (Justin's own MCP harness logs are eligible training data) via MLX-LM's LoRA pipeline. Inference on llama.cpp Metal. +- **Speed estimate:** Qwen3-30B-A3B at Q4_K_M on M5 Max runs ~40–55 tok/s in llama.cpp Metal as of last benchmark cycle; ACC delta is structural to the training, no inference-time overhead. + +**Gated DeltaNet-2** also has theoretical M5 Max relevance (linear attention → flat memory cost in sequence length), but **NC license + no pre-trained weights** make it a `track` item, not `try-now`. + +## Conflicts & open questions +Cross-referenced against `_meta/claims-ledger.md`: + +- **DecentMem (+23.8% over centralized memory) vs. Memory Curse [arxiv:2605.08060] (more memory degrades cooperation in 18/28 settings).** Both can be true, but the directions on whether *more memory* helps or hurts agents are now explicitly opposed in the ledger. The mechanistic difference appears to be **(a) what task — cooperation/coordination vs. task-evidence retrieval; (b) what structure — decentralized + dual-pool vs. flat shared store**. The cleanest empirical resolution is to run the DecentMem dual-pool architecture under the Memory Curse's 4-game cooperation protocol and report whether the curse persists. Until then, the ledger should reflect **conditional claims** ("more memory helps under decentralized dual-pool on task-evidence tasks; harms under flat history on cooperation tasks") rather than a unified rule. +- **Gemini 3.5 Flash MCP Atlas = 83.6% vs. unpublished frontier MCP-eval numbers.** Anthropic and OpenAI have not published comparable MCP-specific eval numbers in their model release posts. Until reproduction or comparable disclosure lands, the 83.6% sits in the ledger as a **first-mover number with no comparative anchor**. Watch for Anthropic / OpenAI to publish MCP-Atlas-comparable numbers in their next release cycles; until then, treat the 83.6% as Google-self-reported and not yet third-party-validated. +- **Open question: Maestro (4B RL orchestrator beats GPT-5) + Uno-Orchestra (learned orchestration over fixed prompts, +16pp at 10× lower cost) — do these compose?** Both papers argue for **learned structured orchestration over frozen experts**, but their training signal differs (Maestro = RL over expert composition; Uno-Orchestra = RL over decomposition depth + worker model + budget). Composability would be a 4B orchestrator that also chooses decomposition depth — open empirical question whether the gains are additive or one dominates. +- **Open question: Compiling Agentic Workflows into LLM Weights (100× cost) is too clean to be the headline without a breakdown.** The abstract doesn't decompose what counts as "near-frontier quality" — visible-test pass rate, held-out generalization, brittleness under workflow drift? Cross-reference with [arxiv:2605.21384] SpecBench's visible↔held-out gap finding: any compiled-workflow approach should be evaluated on held-out variants, not just the workflow snapshot it was compiled against. Until that evaluation lands, treat the 100× number as **upper-bound under best-case workflow stability**. + +## Sources checked +- **arxiv-cs.AI / cs.CL / cs.MA / cs.LG:** 4 sources, 65+ recent submissions reviewed, 7 picked (ACC, Maestro, Gated DeltaNet-2, Ratchet, DecentMem, Compiling Workflows, plus background context from DeferMem, Memory-R2, Trace2Skill, HarnessAPI, IdleSpec, SpecHop, MOSS, Search-E1, FlyRoute, LCGuard). +- **huggingface.co/papers (Daily Papers, May 22):** WorldKV, ACC, Maestro, Spreadsheet-RL, KVServe, TransitLM among trending — surfaced ACC + Maestro to top of curation. +- **huggingface.co/blog:** no new posts in May 20–22 window (most recent: OlmoEarth v1.1 + Ettin Reranker Family on May 19; both already considered). +- **anthropic.com/news + anthropic.com/research:** no new posts since May 19 ("Widening the conversation on frontier AI" + KPMG announcement); both flagged but not material to today's focus areas. +- **deepmind.google/discover/blog:** **Gemini 3.5 (May 19) picked** as item #1 — flagged for follow-up by yesterday's curator, picked up today. Other items (Gemini Omni, Antigravity 2.0, Gemini for Science) considered: Gemini Omni has no benchmarks published (skipped per "no benchmarks → `track`, not feature"), Antigravity 2.0 page returns empty content (could not access — retry next run), Gemini for Science covers I/O 2026 product surface without research detail. +- **research.google/blog:** **ERA (Empirical Research Assistance, May 19)** considered — Gemini-based tree-search scientific coding agent, open-source at `google-research/era`. Substantive but **near-duplicate of the Co-Scientist coverage** from yesterday's digest (same lab, same week, complementary surface — Co-Scientist for hypotheses, ERA for code). **Dropped** from item list to preserve quality-over-quantity rule; tracked in processed.json for reference. +- **microsoft.com/en-us/research/blog:** **MagenticLite / MagenticBrain / Fara1.5 (May 21) picked** as item #2 — largest local-model release this week. +- **openai.com/research + openai.com/news:** 403 on both index pages this run (could not access — retry next run). Best-effort fallback via cross-referenced aggregators surfaced no major OpenAI release in the May 20–22 window. +- **ai.meta.com/research:** no new posts since April 8. +- **allenai.org/blog:** OlmoEarth v1.1 (May 19) — remote-sensing model family, 3× compute reduction; out-of-scope for agent focus areas. +- **Hacker News / r/LocalLLaMA:** scanned for "agent", "MCP", "context", "local model" mentions — surfaced no items that weren't already covered by primary sources above.