DRACO — 25 Hard Agentic Research Tasks (public replication package)

The verification package for the DRACO top-25 benchmark results published by Hyperspace (July 2026): the tasks, the rubrics, the judge, and the per-task grade receipts behind every number on the published board — including Kimi-3 + Hyperspace verification system = 83.0, above Grok-4.5 (80.6), GPT-5.6 (79.4 best config), and Opus-4.8 (73.3) running solo.

Contamination notice: these tasks and rubrics are public as of 2026-07-21. Any run after this date may be contaminated by publication (models trained or tuned on the tasks). The published numbers were produced before release. If you rerun and get different numbers, both provider drift and contamination are candidate explanations.

What's here

tasks/            25 task files (id, domain, problem, per-criterion rubric) + TASKSET-TOP25.txt
grading/          grok_grade.py (the judge driver) + grade_config.sh (batch grader)
harness/          HARNESS.md — architecture of the Hyperspace harness that produced the runs
                  (described in full; the implementation ships in the Hyperspace CLI)
grades/           per-task grade JSONs for every published cell (the receipts) —
                  each file: per-criterion verdicts + `normalized` (0–100)

Scoring

Every published number is exactly:

mean(grade['normalized'] for grade in grades/<config>/*.json)

No weighting, no selection. Rubric dimensions: factual accuracy 35% · breadth & depth 30% · citation quality 20% · presentation 15%, applied per-criterion by the judge.

Judge: grok-4.3 (xAI) — deliberately a competitor's model — batched per-criterion mode, temperature 0. A Gemini-family grader served as a second judge in earlier development rounds; the published board is single-judge grok-4.3 for consistency.

Verify any published number from the receipts:

python3 - <<'PY'
import json, glob
g = [json.load(open(f))['normalized'] for f in glob.glob('grades/kimi_k3_system/*.json')]
print(round(sum(g)/len(g), 2), len(g))   # -> 83.03 25
PY

Reproducing / participating

1. Run the tasks — feed the 25 problems in tasks/ to any model or agentic harness of your choice (bring your own model/API access), or use the shipped Hyperspace CLI (hyperspace web-research "<question>" --depth deep). The harness architecture behind the published runs is documented in harness/HARNESS.md.
2. Grade with the same judge — one answer file per task, then:

   OR_GRADER_MODEL='x-ai/grok-4.3' bash grading/grade_config.sh <your_config_dir> 6

   (Judge access via your own OpenRouter or xAI key.)
3. Compare against the receipts in grades/.

Observed costs (July 2026): a solo 25-task run ≈ $5–15 of provider API; grading ≈ $3–5. Tasks take minutes to over an hour each; run concurrently.

Disclosure (read before comparing numbers)

• System vs solo: the kimi_k3_system cell is a two-pass pipeline (draft, then a verify-and-complete pass — see harness/HARNESS.md — roughly 2× the inference of a solo run). The frontier baselines are solo, one pass. That asymmetry is the published claim: the system, not the raw model, is what tops the board. Solo-to-solo on the same harness, Grok-4.5 leads Kimi-3 80.6 to 79.9.
• Harness versions: cells suffixed _v2 ran a reliability-hardened loop; un-suffixed solo cells ran an earlier, smaller-budget loop. The hardened loop is not a universal lift (it helped Grok +5.7 and Kimi, was neutral for Opus +0.9, and hurt GPT-5.6 −5.0). GPT-5.6's best config (79.4) is its earlier-loop run.
• Transport blanks: a 0.0 grade whose answer was empty (provider returned nothing) is an infrastructure failure, not a model failure. Published "valid-only" figures exclude these; genuine low-scoring answers are always kept.
• Claude Code cells (opus_cc_solo, kimi3_cc_solo) ran Anthropic's Claude Code CLI as the harness (Kimi-3 via Moonshot's Anthropic-compatible endpoint). site_fable5 and delphi_v2 are earlier-generation Hyperspace pipeline cells (same tasks, same judge).

The published board (grok-4.3 judge, top-25)

Cell	Score
Hyperspace Superintelligence (frontier+open system)	87.1
Fable-5 solo (site-era pipeline)	86.1
Kimi-3 · Hyperspace verification system (open)	83.0
Hyperspace open-models panel	81.0
Grok-4.5 solo	80.6
GPT-5.6 solo (best config)	79.4
Opus-4.8 solo	73.3
Opus-4.8 · Claude Code	69.3
Kimi-3 · Claude Code	69.0
Hypermetal (local panel, on-device)	53.8
GLM-5.2 solo	51.3
Bonsai-27B (on-device, $0)	49.1
Inkling solo	45.8

License: tasks, rubrics, and grading code released for research and replication. Attribution appreciated: DRACO benchmark, Hyperspace (2026).