Dogfood operator rolling proof log

[bmdhodl]

This issue is the rolling artifact sink for the recurring AgentGuard dogfood operator.

Contract going forward:

• Each run should leave a proof artifact under proof/dogfood/YYYY-MM-DD/ in the repo worktree.
• Each run should update this issue with the concrete commands run, guard behavior observed, artifact paths, and any blocker or follow-up created from the run.
• If a run produces a safe fix, the run should open or update a PR instead of only posting here.
• If the run finds a real blocker, it should open or update a dedicated issue and link it here.

This issue exists so the automation produces durable GitHub artifacts instead of status-only summaries.

[bmdhodl]

Dogfood proof run - 2026-05-21 (second run)

Proof PR: #491

In-repo artifacts:

• proof/dogfood/2026-05-21/summary.md
• proof/dogfood/2026-05-21/agentguard_demo_traces.jsonl
• proof/dogfood/2026-05-21/coding_agent_review_loop_traces.jsonl
• proof/dogfood/2026-05-21/agentguard-demo-report.txt
• proof/dogfood/2026-05-21/coding-agent-incident.md

Commands passed:

• agentguard doctor
• python -m agentguard.cli doctor
• python -m agentguard.cli demo
• python examples\coding_agent_review_loop.py
• python -m agentguard.cli report agentguard_demo_traces.jsonl
• python -m agentguard.cli incident coding_agent_review_loop_traces.jsonl
• python -m pytest sdk/tests/test_demo.py sdk/tests/test_doctor.py sdk/tests/test_cli_report.py sdk/tests/test_example_starters.py sdk/tests/test_mcp_registry_metadata.py -q -> 35 passed
• python scripts\sdk_release_guard.py --check-mcp-npm -> Release guard passed

Guard behavior observed from raw JSONL traces:

• demo trace emitted guard.budget_warning, guard.budget_exceeded, guard.loop_detected, and guard.retry_limit_exceeded
• review-loop trace emitted guard.budget_exceeded at $0.0510 > $0.0450 and guard.retry_limit_exceeded for apply_patch attempt 4

Repo-health notes:

• PR Add dogfood proof for 2026-05-21 #491 is green and blocked only on required review.
• Existing open PRs are also green/review-blocked; no unresolved review threads found.
• Roadmap remains 13 days old and architecture remains 3 weeks old; [ops-cadence] Docs need refresh #473 continues to track ops freshness.
• Release/package alignment remains good: GitHub/PyPI/local SDK 1.2.10, npm/local MCP 0.2.2.

[bmdhodl]

Dogfood proof run - 2026-05-21 run 3

Proof PR: #493

In-repo artifacts:

• proof/dogfood/2026-05-21/summary-run3.md
• proof/dogfood/2026-05-21/agentguard_doctor_trace-run3.jsonl
• proof/dogfood/2026-05-21/agentguard_demo_traces-run3.jsonl
• proof/dogfood/2026-05-21/coding_agent_review_loop_traces-run3.jsonl
• proof/dogfood/2026-05-21/agentguard-demo-report-run3.txt
• proof/dogfood/2026-05-21/coding-agent-incident-run3.md

Commands passed:

• agentguard doctor
• python -m agentguard.cli doctor
• python -m agentguard.cli demo
• python examples\coding_agent_review_loop.py
• python -m agentguard.cli report agentguard_demo_traces.jsonl
• python -m agentguard.cli incident coding_agent_review_loop_traces.jsonl
• python -m pytest sdk/tests/test_demo.py sdk/tests/test_doctor.py sdk/tests/test_cli_report.py sdk/tests/test_example_starters.py sdk/tests/test_mcp_registry_metadata.py -q -> 35 passed
• python scripts\sdk_release_guard.py --check-mcp-npm -> Release guard passed

Guard behavior observed from raw JSONL traces:

• demo trace emitted guard.budget_warning, guard.budget_exceeded, guard.loop_detected, and guard.retry_limit_exceeded
• review-loop trace emitted guard.budget_exceeded at $0.0510 > $0.0450 on attempt 3 and guard.retry_limit_exceeded for apply_patch attempt 4

Repo-health notes:

• PR Add dogfood proof for 2026-05-21 run 3 #493 CI and CodeQL are green; blocked only on required review.
• New PR Fix 3 P0 bugs: async tracer crash, sync/async divergence, MCP TOCTOU race #492 is green but merge-conflicted (DIRTY) and needs conflict repair/rebase before it can move.
• All checked open PR review threads are resolved; no active unresolved threads found.
• Roadmap remains 13 days old and architecture remains 3 weeks old; [ops-cadence] Docs need refresh #473 continues to track ops freshness.
• Release/package alignment remains good: GitHub/PyPI/local SDK 1.2.10, npm/local MCP 0.2.2.

[bmdhodl]

Dogfood operator run 4 proof for 2026-05-21 is published in PR #494: #494

Repo artifact path: proof/dogfood/2026-05-21/summary-run4.md

Observed guard behavior from raw traces:

• agentguard_demo_traces-run4.jsonl: guard.budget_warning=1, guard.budget_exceeded=1, guard.loop_detected=1, guard.retry_limit_exceeded=1
• coding_agent_review_loop_traces-run4.jsonl: guard.budget_exceeded=1, guard.retry_limit_exceeded=1
• agentguard_doctor_trace-run4.jsonl: doctor.verify=2

Validation:

• python -m pytest sdk/tests/test_demo.py sdk/tests/test_doctor.py sdk/tests/test_cli_report.py sdk/tests/test_example_starters.py sdk/tests/test_mcp_registry_metadata.py -q -> 35 passed in 1.57s
• python scripts\sdk_release_guard.py --check-mcp-npm -> Release guard passed.

Repo-health notes: roadmap is 13 days old and architecture is 3 weeks old; official MCP registry still reports 0.2.1; Glama API returned 200 but still no indexed tools in this run.

[bmdhodl]

Dogfood proof - 2026-05-22 run1

PR: #495
Commit: aa48b12

Repo artifacts:

• proof/dogfood/2026-05-22/summary-run1.md
• proof/dogfood/2026-05-22/agentguard_doctor_trace-run1.jsonl
• proof/dogfood/2026-05-22/agentguard_demo_traces-run1.jsonl
• proof/dogfood/2026-05-22/coding_agent_review_loop_traces-run1.jsonl
• proof/dogfood/2026-05-22/agentguard_demo_report-run1.md
• proof/dogfood/2026-05-22/coding_agent_review_loop_incident-run1.md

Proof commands passed:

• agentguard doctor
• python -m agentguard.cli doctor
• python -m agentguard.cli demo
• python examples/coding_agent_review_loop.py
• python -m agentguard.cli report agentguard_demo_traces.jsonl
• python -m agentguard.cli incident coding_agent_review_loop_traces.jsonl

Concrete guard behavior observed:

• Demo trace emitted guard.budget_warning=1, guard.budget_exceeded=1, guard.loop_detected=1, and guard.retry_limit_exceeded=1.
• Review-loop trace emitted guard.budget_exceeded=1 at $0.0510 > $0.0450 and guard.retry_limit_exceeded=1 for apply_patch attempt 4.

Validation:

• Focused pytest slice: 35 passed in 1.48s.
• python scripts/sdk_release_guard.py --check-mcp-npm: Release guard passed.
• git diff --check: clean.

Repo health note:

• Open PRs remain review-required; Fix 3 P0 bugs: async tracer crash, sync/async divergence, MCP TOCTOU race #492 is still dirty/conflicted.
• Staleness warning remains: roadmap 13 days old, architecture 3 weeks old, tracked by [ops-cadence] Docs need refresh #473.

[bmdhodl]

Follow-up for 2026-05-22 run1: PR #495 was amended to commit 3fa37c2 after automated review. The generated Markdown proof files are now UTF-8 without BOM, and summary-run1.md clarifies that report/incident commands used fresh root trace files before copying to -run1 artifacts.

Post-review state: CI and CodeQL are green, all three Copilot review threads are resolved, and the PR remains blocked only on required human review.

[bmdhodl]

Dogfood proof - 2026-05-22 run 2

PR: #496
Branch: codex/dogfood-proof-2026-05-22-run2
Commit: f9a5630

Artifacts:

• proof/dogfood/2026-05-22/summary-run2.md
• proof/dogfood/2026-05-22/agentguard_demo_traces-run2.jsonl
• proof/dogfood/2026-05-22/coding_agent_review_loop_traces-run2.jsonl
• proof/dogfood/2026-05-22/agentguard_doctor_trace-run2.jsonl
• proof/dogfood/2026-05-22/demo-report-run2.md
• proof/dogfood/2026-05-22/coding-agent-incident-run2.md
• proof/dogfood/2026-05-22/guard-events-run2.json

Commands passed:

• agentguard doctor
• python -m agentguard.cli doctor
• python -m agentguard.cli demo
• python examples\coding_agent_review_loop.py
• python -m agentguard.cli report agentguard_demo_traces.jsonl
• python -m agentguard.cli incident coding_agent_review_loop_traces.jsonl
• python -m pytest sdk/tests/test_demo.py sdk/tests/test_doctor.py sdk/tests/test_cli_report.py sdk/tests/test_example_starters.py sdk/tests/test_mcp_registry_metadata.py -q -> 35 passed in 1.60s
• python scripts\sdk_release_guard.py --check-mcp-npm -> Release guard passed
• git diff --check -> clean

Concrete guard behavior:

• demo trace emitted guard.budget_warning, guard.budget_exceeded, guard.loop_detected, and guard.retry_limit_exceeded
• review-loop trace emitted guard.budget_exceeded at $0.0510 > $0.0450 and guard.retry_limit_exceeded for apply_patch attempt 4

[bmdhodl]

Follow-up for 2026-05-22 run 2: PR #496 was amended to commit 0755143 after automated review caught that guard-events-run2.json was not valid JSON. The artifact now has one top-level JSON object with a traces array. Reran focused pytest (35 passed in 1.38s), release guard, and git diff --check; CI and CodeQL are green and both review threads are resolved.

[bmdhodl]

Dogfood proof - 2026-05-22 run 3

Artifacts:

• PR: Add dogfood proof for 2026-05-22 run 3 #497
• Repo path: proof/dogfood/2026-05-22/run3/summary.md
• Raw traces: agentguard_doctor_trace.jsonl, agentguard_demo_traces.jsonl, coding_agent_review_loop_traces.jsonl

Proof commands passed:

• agentguard doctor
• python -m agentguard.cli doctor
• python -m agentguard.cli demo
• python examples\coding_agent_review_loop.py
• python -m agentguard.cli report agentguard_demo_traces.jsonl
• python -m agentguard.cli incident coding_agent_review_loop_traces.jsonl
• python -m pytest sdk/tests/test_demo.py sdk/tests/test_doctor.py sdk/tests/test_cli_report.py sdk/tests/test_example_starters.py sdk/tests/test_mcp_registry_metadata.py -q -> 35 passed
• python scripts/sdk_release_guard.py --check-mcp-npm -> Release guard passed

Concrete guard behavior observed:

• Demo trace: guard.budget_warning, guard.budget_exceeded, guard.loop_detected, guard.retry_limit_exceeded
• Review-loop trace: guard.budget_exceeded at $0.0510 > $0.0450, plus guard.retry_limit_exceeded on apply_patch attempt 4

Repo health notes:

• Roadmap is 13 days old; architecture is 3 weeks old. Keep [ops-cadence] Docs need refresh #473 as the active freshness tracker.
• GitHub release, PyPI, local SDK, npm MCP, and local MCP metadata are aligned at v1.2.10 / 0.2.2.
• MCP Registry still reports 0.2.1; Glama API still returns tools: [].

[bmdhodl]

Dogfood proof - 2026-05-22 run4

PR: #498
Artifacts: proof/dogfood/2026-05-22/run4/

Commands run:

• agentguard doctor
• python -m agentguard.cli doctor
• python -m agentguard.cli demo
• python examples\coding_agent_review_loop.py
• python -m agentguard.cli report agentguard_demo_traces.jsonl
• python -m agentguard.cli incident coding_agent_review_loop_traces.jsonl

Concrete guard behavior observed:

• agentguard_demo_traces-run4.jsonl: guard.budget_warning=1, guard.budget_exceeded=1, guard.loop_detected=1, guard.retry_limit_exceeded=1.
• coding_agent_review_loop_traces-run4.jsonl: guard.budget_exceeded=1 at $0.0510 > $0.0450, plus guard.retry_limit_exceeded=1 on apply_patch attempt 4.

Validation:

• Focused pytest slice: 35 passed in 1.84s.
• python scripts\sdk_release_guard.py --check-mcp-npm: Release guard passed.
• git diff --check: clean.

Repo health note: roadmap and architecture remain stale by repo policy; issue #473 continues to track docs freshness. MCP Registry still returned 404 for the direct server endpoint from this environment, while Glama still returns an empty tools array.

[bmdhodl]

Dogfood operator proof for 2026-05-22 run5 is published.

Artifacts:

• PR: Add dogfood proof for 2026-05-22 run 5 #499
• Repo path: proof/dogfood/2026-05-22/run5/
• Summary: proof/dogfood/2026-05-22/run5/summary-run5.md
• Parsed guard events: proof/dogfood/2026-05-22/run5/guard-events-run5.json
• Raw traces: agentguard_demo_traces-run5.jsonl, coding_agent_review_loop_traces-run5.jsonl, agentguard_doctor_trace-run5.jsonl

Proof commands passed:

• agentguard doctor
• python -m agentguard.cli doctor
• python -m agentguard.cli demo
• python examples\coding_agent_review_loop.py
• python -m agentguard.cli report agentguard_demo_traces.jsonl
• python -m agentguard.cli incident coding_agent_review_loop_traces.jsonl

Concrete guard behavior observed:

• Demo trace emitted guard.budget_warning, guard.budget_exceeded, guard.loop_detected, and guard.retry_limit_exceeded.
• Coding-agent review-loop trace emitted guard.budget_exceeded at $0.0510 > $0.0450 and guard.retry_limit_exceeded on apply_patch attempt 4.

Validation:

• Focused pytest slice: 35 passed.
• python scripts/sdk_release_guard.py --check-mcp-npm: Release guard passed.
• git diff --check: clean.
• UTF-8 BOM check: none found in run5 files.

Current blockers still tracked:

• [ops-cadence] Docs need refresh #473 docs freshness: roadmap 13 days ago, architecture 3 weeks ago.
• MCP Registry still reports 0.2.1; npm/local MCP are 0.2.2.
• Glama API still returns tools: [].

[bmdhodl]

Follow-up on run5 PR review: PR #499 was amended after automated review surfaced a real double-counting issue in the coding-agent review-loop proof.

Fix included in #499:

• examples/coding_agent_review_loop.py now records cumulative guard spend as cost_used_usd instead of additive cost_usd.
• Run5 traces/reports were regenerated.
• review-loop-incident-run5.md now reports estimated cost $0.0510.
• Regression coverage was added in sdk/tests/test_example_starters.py.

Post-fix status:

• CI and CodeQL are green.
• All four automated review threads are resolved.
• PR remains blocked only on required human review.

[bmdhodl]

Dogfood proof - 2026-05-22 run6

PR: #500
Branch: codex/dogfood-proof-2026-05-22-run6
Commit: 65df458

Artifacts:

• proof/dogfood/2026-05-22/run6/summary-run6.md
• proof/dogfood/2026-05-22/run6/guard-events-run6.json
• proof/dogfood/2026-05-22/run6/agentguard_doctor_trace-run6.jsonl
• proof/dogfood/2026-05-22/run6/agentguard_demo_traces-run6.jsonl
• proof/dogfood/2026-05-22/run6/coding_agent_review_loop_traces-run6.jsonl
• proof/dogfood/2026-05-22/run6/demo-report-run6.md
• proof/dogfood/2026-05-22/run6/review-loop-incident-run6.md

Proof commands passed:

• agentguard doctor
• python -m agentguard.cli doctor
• python -m agentguard.cli demo
• python examples\coding_agent_review_loop.py
• python -m agentguard.cli report agentguard_demo_traces.jsonl
• python -m agentguard.cli incident coding_agent_review_loop_traces.jsonl

Concrete guard behavior observed:

• Demo trace emitted guard.budget_warning, guard.budget_exceeded, guard.loop_detected, and guard.retry_limit_exceeded.
• Coding-agent review loop emitted guard.budget_exceeded at $0.0510 > $0.0450 and guard.retry_limit_exceeded on apply_patch attempt 4.

Validation:

• Focused pytest slice: 35 passed.
• python scripts\sdk_release_guard.py --check-mcp-npm: Release guard passed.
• git diff --check: clean.
• UTF-8 BOM check: none found in run6 files.

Repo-health notes:

• The previously installed agentguard entrypoint was stale from another editable worktree; python -m pip install -e ./sdk repaired this checkout before proof.
• Roadmap remains 13 days old and architecture remains 3 weeks old; [ops-cadence] Docs need refresh #473 continues to track ops freshness.
• Release/package alignment is still good: GitHub/PyPI/local SDK 1.2.10, npm/local MCP 0.2.2.
• Glama API still returns tools: []; direct MCP Registry server endpoint returned 404 from this environment.

[bmdhodl]

Follow-up on run6 PR review: PR #500 was amended to commit d46bbf4 after automated review.

Fixes included:

• documented commands now use portable / path separators
• guard-events-run6.json now points at committed *-run6.jsonl trace filenames
• summary records the health-check commands that produced Glama/MCP Registry/package findings
• incident report now includes an operator note explaining the current main report estimate double-counts parent span aggregate plus child cost events while the raw guard event proves the $0.0510 > $0.0450 stop

Post-fix status:

• CI and CodeQL are green
• all four review threads are resolved
• PR remains blocked only on required human review

[bmdhodl]

Dogfood run7 proof update: PR #501 adds the latest trace-backed proof bundle.

Artifacts:

• proof/dogfood/2026-05-22/run7/summary-run7.md
• proof/dogfood/2026-05-22/run7/agentguard-demo-traces-run7.jsonl
• proof/dogfood/2026-05-22/run7/coding-agent-review-loop-traces-run7.jsonl
• proof/dogfood/2026-05-22/run7/guard-events-run7.json
• report/incident outputs in the same run directory

Observed enforcement:

• demo emitted guard.budget_warning, guard.budget_exceeded, guard.loop_detected, and guard.retry_limit_exceeded
• review-loop emitted guard.budget_exceeded at $0.0510 > $0.0450 and guard.retry_limit_exceeded for apply_patch attempt 4

Validation:

• focused pytest slice: 35 passed
• python scripts/sdk_release_guard.py --check-mcp-npm: passed
• git diff --check: clean
• run7 text/Markdown/JSON BOM check: clean

PR: #501