You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
MVP-completion slice: stop selecting official tasks by hand and make every published run self-describing.
--preset flag in the headless CLI selecting active admitted tasks by metadata (single flag over task.yaml metadata — no preset registry; there is one real preset today)
Release report + leaderboard export name the preset, exact task ids, task versions, and a corpus manifest digest (pin published numbers to a frozen corpus snapshot, not a drifting HEAD)
Export raw trial rows and first-class f2p/p2p fields (baselineHiddenFailed / referencePublicPassed / referenceHiddenPassed) — fields only; no dashboards until the corpus is bigger
Record per run: harness version, prompt template, tool permissions, timeouts, retry policy, model/provider config
MVP-completion slice: stop selecting official tasks by hand and make every published run self-describing.
--presetflag in the headless CLI selecting active admitted tasks by metadata (single flag overtask.yamlmetadata — no preset registry; there is one real preset today)baselineHiddenFailed/referencePublicPassed/referenceHiddenPassed) — fields only; no dashboards until the corpus is bigger