Measure Codectx impact across 20 real repository changes

[greygoody]

Intent

Test whether Codectx context packets plus the explicit workcell method improve accepted engineering outcomes compared with a simpler branch/PR workflow.

Authority

• This repository owns Codectx behavior, local task evidence, candidate identities, and integration proof.
• greygoody/agentic-workflows owns the reusable study procedure and event envelope.
• greygoody/orb-eval owns event validation and derived projections.
• Do not copy raw runs into agentic-workflows, context, or a temporary workspace.

Study

Commit the study assignment before outcomes are known:

• 10 control changes: ordinary issue → branch → agent → tests → review → merge.
• 10 treatment changes: bounded workcell → Codectx packet → declared proof → review → exact-candidate acceptance → closure.
• Alternate assignments and compare within task class/risk where possible.
• Keep model, reviewer, tools, and CI environment stable enough for interpretation.

Required events

Emit agentic_impact_event v1 records for:

• work start;
• candidate submission with exact commit;
• proof result and failure classification;
• proof ready;
• review decision;
• human authority, clarification, repair, or takeover minutes;
• agent usage and cost;
• closure/reopen;
• escaped defects linked to repair workcells.

Primary outcomes

• escaped defects and severity per accepted change;
• human repair minutes per accepted change;
• cost per defect-free accepted change.

Also report cycle time, closure latency, CI retries, review rounds, reopen count, and first-candidate acceptance.

Minimal implementation

• Add one project-local JSONL event path ignored or retained according to existing Codectx runtime policy.
• Add a narrow command or just recipe that emits candidate/proof/closure events without becoming a workflow engine.
• Validate and summarize through orb-eval impact-validate and orb-eval impact-summarize.
• Do not add a database, daemon, or new repository.

Closure

• 20 assignments fixed before execution.
• 20 changes completed or explicitly excluded with reason.
• Event streams validate.
• Variant summary regenerated from retained events.
• At least one external developer executes the treatment without live interpretation.
• Decision records which Codectx/workcell steps to retain, simplify, or delete.