DeepScholar

An agentic research loop for improving vision-language-action and robot policy models.

English  ·  简体中文  ·  Start From A Baseline  ·  Research Loop  ·  Quick Start  ·  Configuration  ·  Artifacts

---

DeepScholar turns model-improvement work into a reproducible research loop. Give it a baseline policy, an evaluation protocol, and an execution budget. It proposes candidate ideas, debates them, generates isolated configs, runs proxy and final evaluations, records failures, and writes a technical report for the next round.

The goal is not to run one more experiment. The goal is to make the improvement process itself repeatable: every hypothesis, patch, log, checkpoint, metric, and lesson is preserved.

0 Manual intervention during the loop

>24h Long-running autonomous studies

+2.0 pp Observed gains on selected LIBERO runs

Start From A Baseline Policy

Starting from a known checkpoint is often faster than manually designing the next experiment.

DeepScholar can take a baseline VLA or robot policy study and turn it into a structured self-improvement run:

1. Define the research target: model family, dataset, task suite, metric, and eval protocol.
2. Let the loop generate candidates: conservative hyperparameter changes plus more experimental module ideas.
3. Run proxy training and evaluation: cheap gates reject weak candidates before spending full budget.
4. Promote promising patches: final training and evaluation confirm whether a change beats the calibrated baseline.
5. Write the memory: successful recipes and failure cards seed the next iteration.

"Improve this SmolVLA LIBERO baseline. Keep the task suite, metric, horizon, and camera protocol fixed unless a candidate explicitly targets evaluation."

What you get back is not an untracked pile of shell logs. You get:

• Candidate configs with isolated YAML patches.
• Aligned evaluations that keep task, metric, horizon, cameras, and protocol fields consistent.
• Experiment records linking patches to logs, checkpoints, eval videos, metrics, and reports.
• Long-term memory so later rounds reuse what worked and avoid repeated failures.

What DeepScholar Handles

• End-to-end research loop: idea generation, debate, planning, config generation, execution, evaluation, memory, and reporting.
• Agent and non-agent modes: use real Cursor SDK agents when available, or fall back to deterministic local heuristics.
• Safe candidate generation: supports conservative tuning and frontier module or architecture experiments.
• GPU-aware execution: assigns candidate experiments to idle GPUs and queues the rest automatically.
• Reproducible artifacts: stores agent traces, train/eval logs, eval videos, checkpoints, registries, manifests, and reports.

Current Results

---
config:
  themeVariables:
    xyChart:
      plotColorPalette: "#2563eb, #dc2626"
---
xychart-beta
  title "DeepScholar - Base Model vs Self-Improve"
  x-axis ["SmolVLA Base", "SmolVLA Improve", "Pi0.5 Base", "Pi0.5 Improve"]
  y-axis "Success Rate" 0.50 --> 1.00
  bar "Base model" [0.64, 0, 0.92, 0]
  bar "Self-improve" [0, 0.66, 0, 0.94]

Loading

• SmolVLA: starting from the lerobot/smolvla_libero base model, improves the LIBERO-10 subset success rate from 64.0% to 66.0% (+2.0 pp).
• Pi0.5: starting from the lerobot/pi05_libero_finetuned base model, improves the LIBERO-10 subset success rate from 92.0% to 94.0% (+2.0 pp).

Legend: blue = calibrated baseline; red = DeepScholar best result.

Research Loop

DeepScholar separates scientific reasoning, candidate generation, execution, evaluation, and long-term memory so each experiment remains reproducible.

%%{init: {"theme": "base", "themeVariables": {"fontFamily": "Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, Segoe UI, sans-serif", "primaryColor": "#f8fafc", "primaryTextColor": "#0f172a", "primaryBorderColor": "#cbd5e1", "lineColor": "#64748b", "tertiaryColor": "#f1f5f9"}}}%%
flowchart TB
  Goal["<b>Research Goal</b><br/>baseline model / dataset / eval protocol / target metric"]:::hero

  subgraph Loop["DeepScholar Self-Improvement Loop"]
    direction LR

    subgraph Think["1. Reason"]
      direction TB
      R["Researcher<br/><small>papers, logs, prior failures</small>"]:::agent
      I["Idea Generator<br/><small>safe hparams + frontier modules</small>"]:::agent
      D["Debate Panel<br/><small>reviewer, mentor, engineer, skeptic</small>"]:::agent
      P["Execution Planner<br/><small>budget, gates, rollback</small>"]:::agent
    end

    subgraph Build["2. Build"]
      direction TB
      X["Candidate Builder<br/><small>isolated YAML patches</small>"]:::build
      Y["Candidate Configs<br/><small>workspace/configs/*.yaml</small>"]:::build
      S["Sandbox Apply<br/><small>whitelist + rollback boundary</small>"]:::build
    end

    subgraph Run["3. Execute"]
      direction TB
      T["Proxy Train<br/><small>LeRobot / custom command</small>"]:::run
      V["Proxy Eval<br/><small>aligned metric parser</small>"]:::run
      F["Final Eval<br/><small>more episodes / seeds</small>"]:::run
    end

    subgraph Decide["4. Decide"]
      direction TB
      B0["Baseline Eval<br/><small>calibrated success rate</small>"]:::eval
      G1{"Stage-A<br/>Proxy Gate"}:::gate
      G2{"Stage-B<br/>Final Gate"}:::gate
    end
  end

  subgraph Memory["Memory and Reporting"]
    direction LR
    Mem["Long-Term Memory<br/><small>recipes + failure cards</small>"]:::memory
    Reg["Experiment Registry<br/><small>experiments.jsonl</small>"]:::memory
    Man["Session Manifest<br/><small>logs, evals, checkpoints</small>"]:::memory
    Rep["Technical Report<br/><small>round summary + best result</small>"]:::report
  end

  Goal --> R
  Goal --> B0
  R --> I --> D --> P --> X --> Y --> S --> T --> V --> G1
  B0 --> G1
  G1 -- pass --> F --> G2
  G1 -- fail --> Mem
  G2 -- accept --> Reg --> Rep
  G2 -- reject --> Mem
  T -. trace .-> Man
  V -. trace .-> Man
  F -. trace .-> Man
  Reg -. index .-> Man
  Mem -. reuse lessons .-> R
  Rep -. next prompt .-> R

  classDef hero fill:#020617,stroke:#020617,color:#ffffff,stroke-width:2px;
  classDef agent fill:#eef2ff,stroke:#6366f1,color:#312e81,stroke-width:1.5px;
  classDef build fill:#faf5ff,stroke:#a855f7,color:#581c87,stroke-width:1.5px;
  classDef run fill:#ecfeff,stroke:#06b6d4,color:#164e63,stroke-width:1.5px;
  classDef eval fill:#f0fdf4,stroke:#22c55e,color:#14532d,stroke-width:1.5px;
  classDef gate fill:#fff7ed,stroke:#f97316,color:#7c2d12,stroke-width:2px;
  classDef memory fill:#fdf2f8,stroke:#ec4899,color:#831843,stroke-width:1.5px;
  classDef report fill:#fffbeb,stroke:#f59e0b,color:#78350f,stroke-width:1.5px;

Loading

Repository Layout

src/research_loop/
  agent_mode/                 # Cursor SDK agent mode
    prompts/*.md              # Role definitions: researcher, idea, debate, planner, executor, reflector
    roles.py                  # Load Markdown prompts and call agents
  non_agent_mode/             # Local fallback mode
    roles.py                  # Deterministic heuristic roles
  loop.py                     # Main research loop
  runner.py                   # LeRobot train/eval command runner
  evaluation.py               # Stage-A / Stage-B gates
  registry.py                 # Experiment registry and memory
  report.py                   # Final report writer
src/sitecustomize.py           # Project-local LeRobot compatibility patches loaded by runner subprocesses

workspace/
  configs/                    # Generated candidate patches
  artifacts/
    <session_id>/             # Same name as artifacts/agent_runs/<session_id>
      runs/                   # train/eval logs
      eval/                   # eval_info + videos
      checkpoints/            # trained checkpoints

artifacts/
  agent_runs/                 # Agent call records grouped by session
  reports/                    # Markdown reports
  registry/                   # experiments.jsonl
  memory/                     # failure and recipe memory

Quick Start

1. Install

Create a Python environment and install DeepScholar in editable mode:

cd /path/to/DeepScholar
python -m venv .venv
source .venv/bin/activate
pip install -e .

For the SmolVLA/LIBERO example, install LeRobot and verify the commands are available:

lerobot-train --help
lerobot-eval --help

2. Configure

Use one of the example configs:

configs/smolvla-self-improve.json
configs/pi0fast-self-improve.json
configs/pi05-self-improve.json

You can extend DeepScholar to other model families by copying one of these configs and updating the baseline model, dataset, train/eval commands, and eval protocol fields.

3. Run

Start the full loop without agents:

source .venv/bin/activate
vla-loop --config configs/smolvla-self-improve.json

To run with Cursor SDK agents, provide an API key:

export CURSOR_API_KEY="your_cursor_api_key"
vla-loop --config configs/smolvla-self-improve.json

If CURSOR_API_KEY is missing, DeepScholar still runs in non-agent mode.

Configuration Guide

The config file controls the research target, baseline, execution budget, agents, and GPU scheduling. In most runs, these are the fields you will edit:

Key Parameters

• goal.description: research objective shown to agents and reports.
• goal.baseline_model.framework: execution backend, usually lerobot or command.
• goal.baseline_model.checkpoint / model_id: baseline checkpoint used as the starting point.
• goal.baseline_model.eval_protocol: task, metric, horizon, cameras, and other eval settings that must stay aligned across candidates.
• goal.baseline_success_rate: calibrated baseline used by improvement gates.
• goal.baseline_model.command_templates.train: train command template.
• goal.baseline_model.command_templates.eval: eval command template that emits parseable metrics.
• runner.proxy_train_steps: cheap Stage-A training budget for filtering candidates.
• runner.final_train_steps: larger Stage-B training budget for promising candidates.
• agent.enabled: whether to use Cursor SDK agents instead of local heuristic roles.
• agent.model: Cursor SDK model id passed to Agent.prompt().
• agent.enable_code_agent: whether agents may edit code in addition to generated YAML configs.
• gpu.ids: local GPU allowlist; [] means all GPUs visible to nvidia-smi.
• gpu.max_parallel_experiments: max concurrent candidate runs; 0 or "auto" uses idle GPUs automatically.

For non-LeRobot models, set goal.baseline_model.framework to command and provide train/eval shell commands that produce parseable metrics.

Set a readable session tag when comparing multiple runs:

export CURSOR_API_KEY="your_cursor_api_key"
export DEEPSCHOLAR_AGENT_RUN_TAG="smolvla_libero_trial_001"
vla-loop --config configs/smolvla-self-improve.json

The agent model is configured in JSON via agent.model; there is no CLI flag. Session metadata is recorded in:

artifacts/agent_runs/<session_id>/_session_meta.json

How Results Are Organized

Each run writes a manifest and a set of linked artifacts. Start with:

artifacts/agent_runs/<session_id>/manifest.json

manifest.json is the best entry point for reproducing a run. It links the agent session, baseline record, candidate patch IDs, train/eval logs, eval videos, checkpoints, registry rows, and final report.

Then inspect the supporting files:

artifacts/agent_runs/<session_id>/
artifacts/reports/*.md
artifacts/registry/experiments.jsonl
workspace/artifacts/<session_id>/runs/<patch_id>/proxy_train.log
workspace/artifacts/<session_id>/runs/<patch_id>/proxy_eval.log
workspace/artifacts/<session_id>/eval/<patch_id>_proxy/
workspace/artifacts/<session_id>/checkpoints/<patch_id>/

For LeRobot LIBERO logs, use aggregated metrics as the source of truth:

Overall Aggregated Metrics:
{'pc_success': 90.0, 'n_episodes': 30, ...}

Do not confuse running_success_rate in a progress bar with final experiment success.