The run that turns a 1-model board into a real benchmark. Runs only after QA integrity, clean-replay grading, and the preset/manifest slices land. - [ ] Harness-parity contract: one agentic protocol for every model; per-model deviations documented (otherwise the leaderboard ranks scaffolds, not models) - [ ] Result-denominator policy: classify timeouts / harness crashes / no-patch / infra / auth failures before publishing - [ ] Unpublished shakedown run: 2–3 models to validate replay + provider plumbing - [ ] Published run: ≥5 materially different models (Ollama Cloud kimi / minimax / deepseek / qwen / glm, gpt-5.5, …) × 10 tasks × ≥3 trials, graded via clean replay - [ ] Publish with Wilson CIs + explicit n, raw trial rows, cost/latency axis, negative-case results, and known-telemetry-gap notes
The run that turns a 1-model board into a real benchmark. Runs only after QA integrity, clean-replay grading, and the preset/manifest slices land.