Skip to content

Enrich trace v0 with blocker_codes #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
aid-ninja merged 2 commits into from
Jun 21, 2026
Merged

Enrich trace v0 with blocker_codes #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4cae6f4
trace: enrich v0 with blocker_codes
aid-ninja Jun 21, 2026
6cf4328
trace: clarify blocker_codes is the observed-during-run set (incl. cl…
aid-ninja Jun 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
- **Trace artifact v0 (emit-only).** Every `bean-run` now writes a stable post-run record to
`.bean/runs/<run_id>.json`: `schema_version`, `run_id`, `goal`, `started_at`/`ended_at`,
`status`, `certificate`, `rounds`, `pivot_count`, `blockers_opened`/`blockers_closed`,
`verifier_verdicts`, `residuals`, `artifacts_changed`, `metadata`. One file per run (not a
`blocker_codes`, `verifier_verdicts`, `residuals`, `artifacts_changed`, `metadata`. One file per run (not a
rolling file), so future tooling can analyze a corpus of runs without scraping transcripts.
The top-level shape is **fixed** (`additionalProperties: false`); additive fields go under the
`metadata` hatch. Emission **fails closed**: if the trace can't be written, `bean-run` exits
Expand Down
10 changes: 10 additions & 0 deletions rs/src/bin/bean-run.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -433,6 +433,15 @@ fn main() {
.iter()
.filter(|k| !final_blockers.contains(*k))
.count();
// the distinct blocker CODES (E_*) seen across the run — sorted+unique via BTreeSet — so the
// trace analyzer can rank which gate conditions fire most. Derived from the code:claim keys.
let blocker_codes: Vec<String> = blockers_seen
.iter()
.filter_map(|k| k.split_once(':').map(|(c, _)| c.to_string()))
.filter(|c| !c.is_empty())
.collect::<BTreeSet<String>>()
.into_iter()
.collect();
let pivot_count = trace
.iter()
.filter(|t| t.get("pivot").and_then(|v| v.as_bool()).unwrap_or(false))
Expand Down Expand Up @@ -479,6 +488,7 @@ fn main() {
"pivot_count": pivot_count,
"blockers_opened": blockers_seen.len(),
"blockers_closed": blockers_closed,
"blocker_codes": blocker_codes,
"verifier_verdicts": verifier_verdicts,
"residuals": residuals,
"artifacts_changed": Vec::<String>::new(),
Expand Down
7 changes: 7 additions & 0 deletions schemas/trace.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
"pivot_count",
"blockers_opened",
"blockers_closed",
"blocker_codes",
"verifier_verdicts",
"residuals",
"artifacts_changed"
Expand Down Expand Up @@ -68,6 +69,12 @@
"minimum": 0,
"description": "Of those, the count not present in the final signal."
},
"blocker_codes": {
"type": "array",
"items": { "type": "string" },
"uniqueItems": true,
"description": "The set of distinct blocker codes (E_*) OBSERVED DURING the run, including ones whose blockers were later closed — sorted, de-duplicated. Not a count, not final-state-only. Lets analyzers rank which gate conditions recur."
},
"verifier_verdicts": {
"type": "array",
"description": "The scrubbed verdicts bean-verify deposited under .bean/verdicts/, embedded verbatim.",
Expand Down
33 changes: 17 additions & 16 deletions skills/bean/references/trace.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,22 +16,23 @@ start there.

Schema: [`schemas/trace.schema.json`](../../../schemas/trace.schema.json) (`schema_version: "trace/v0"`).

| Field | Meaning |
| ------------------------- | ---------------------------------------------------------------------------------- |
| `schema_version` | `"trace/v0"` — bump on any breaking shape change. |
| `run_id` | Unique per run (`run-<nanos-since-epoch>`); also the filename stem. |
| `goal` | The run goal from `run.json`. |
| `started_at` / `ended_at` | Unix epoch milliseconds. |
| `status` | Final outcome: `ready` / `converged-with-residuals` / `budget-exceeded` / `stuck`. |
| `certificate` | Final convergence certificate from `bean-check`. |
| `rounds` | Driver rounds executed. |
| `pivot_count` | No-progress rounds turned into pivots. |
| `blockers_opened` | Distinct open fronts (`code:claim`) seen across the run. |
| `blockers_closed` | Of those, how many are absent from the final signal. |
| `verifier_verdicts` | The scrubbed verdicts from `.bean/verdicts/`, embedded verbatim. |
| `residuals` | Claims tagged `residual`, each `{id, reason}` (reason = the claim content). |
| `artifacts_changed` | Files the run changed, when available. v0 does not track this yet → `[]`. |
| `metadata` | Extension hatch (`{}` by default). Additive/experimental fields go here. |
| Field | Meaning |
| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `schema_version` | `"trace/v0"` — bump on any breaking shape change. |
| `run_id` | Unique per run (`run-<nanos-since-epoch>`); also the filename stem. |
| `goal` | The run goal from `run.json`. |
| `started_at` / `ended_at` | Unix epoch milliseconds. |
| `status` | Final outcome: `ready` / `converged-with-residuals` / `budget-exceeded` / `stuck`. |
| `certificate` | Final convergence certificate from `bean-check`. |
| `rounds` | Driver rounds executed. |
| `pivot_count` | No-progress rounds turned into pivots. |
| `blockers_opened` | Distinct open fronts (`code:claim`) seen across the run. |
| `blockers_closed` | Of those, how many are absent from the final signal. |
| `blocker_codes` | The **set of distinct blocker codes (`E_*`) observed during the run, including ones whose blockers were later closed** — sorted, unique. Not a count, not final-state-only; lets analyzers rank which gate conditions recur. |
| `verifier_verdicts` | The scrubbed verdicts from `.bean/verdicts/`, embedded verbatim. |
| `residuals` | Claims tagged `residual`, each `{id, reason}` (reason = the claim content). |
| `artifacts_changed` | Files the run changed, when available. v0 does not track this yet → `[]`. |
| `metadata` | Extension hatch (`{}` by default). Additive/experimental fields go here. |

## Stability rules

Expand Down
13 changes: 12 additions & 1 deletion test/conformance.mjs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -335,6 +335,7 @@ console.log(`${dpass}/3 driver smoke checks pass`);
"pivot_count",
"blockers_opened",
"blockers_closed",
"blocker_codes",
"verifier_verdicts",
"residuals",
"artifacts_changed",
Expand All @@ -346,6 +347,15 @@ console.log(`${dpass}/3 driver smoke checks pass`);
const t = JSON.parse(fs.readFileSync(path.join(runsDir, files[0]), "utf8"));
const missing = ALLOWED.filter((k) => !(k in t));
const unknown = Object.keys(t).filter((k) => !ALLOWED.includes(k));
const bc = t.blocker_codes;
// blocker_codes must be a sorted, unique string[] and (for this fixture, where round 1
// has an open risk) contain E_OPEN_RISK.
const bcOk =
Array.isArray(bc) &&
bc.every((x) => typeof x === "string") &&
JSON.stringify(bc) === JSON.stringify([...bc].sort()) &&
new Set(bc).size === bc.length &&
bc.includes("E_OPEN_RISK");
const shapeOk =
t.schema_version === "trace/v0" &&
missing.length === 0 &&
Expand All @@ -355,12 +365,13 @@ console.log(`${dpass}/3 driver smoke checks pass`);
Array.isArray(t.verifier_verdicts) &&
Array.isArray(t.residuals) &&
Array.isArray(t.artifacts_changed) &&
bcOk &&
typeof t.metadata === "object" &&
!Array.isArray(t.metadata);
ok = shapeOk;
why = shapeOk
? ""
: `schema_version=${t.schema_version} missing=[${missing}] unknown=[${unknown}] status=${t.status} vs ${report.outcome}`;
: `schema_version=${t.schema_version} missing=[${missing}] unknown=[${unknown}] blocker_codes=${JSON.stringify(bc)} status=${t.status} vs ${report.outcome}`;
}
if (ok) {
console.log(" ok trace artifact v0 written with stable shape");
Expand Down
Loading