Skip to content

feat: better-harness weekly self-review + Worker overview dashboard page - #12

Merged
nattiini45 merged 5 commits into
mainfrom
fix/better-harness-remediation
Jul 30, 2026
Merged

feat: better-harness weekly self-review + Worker overview dashboard page#12
nattiini45 merged 5 commits into
mainfrom
fix/better-harness-remediation

Conversation

@nattiini45

Copy link
Copy Markdown
Owner

Summary

This PR brings two related features to the fork:

1. Better Harness weekly self-review (96cd091)

  • Bake the pinned QoderAI/better-harness CLI into the openclaw-base, hermes, and qwenpaw images (Node 22 via NodeSource for the Python images; npm install --omit=dev).
  • Ship a builtin better-harness skill in every agent template (Manager, OpenClaw/CoPaw/Hermes/OpenHuman/QwenPaw workers, Team Leader).
  • Each agent runs a deterministic, read-only better-harness harness source evidence collection at most once per 7 days (cadence guard, state persisted to MinIO), authors findings, and reports them to the Manager — never the human admin.
  • Weekly trigger is native per runtime: QwenPaw/Hermes worker daemons run a supervised run_better_harness_weekly_loop; OpenClaw/CoPaw/Manager/Team Leader invoke run-weekly.sh from heartbeat/session-wake.
  • Manager-mediated harness-integration skill aggregates all agents findings and presents one fleet-wide plan for approval.
  • AGENTTEAMS_BETTER_HARNESS_ENABLED env gate (default on) for a fleet-wide kill switch.

2. Worker overview dashboard page (5c67d1a)

  • New Workers tab in the dashboard web UI listing every worker; clicking a worker opens a drill-down detail with current health (health-probe strip), an event/failure timeline, and the worker'"'"'s better-harness weekly findings (from MinIO shared/harness-reports/<name>/).
  • New controller event-history store: WorkerStatus.RecentEvents bounded ring buffer (newest first, capped at 50) recorded by the health monitor (health transitions, zombie failures) and lifecycle handlers (wake/sleep/ensure-ready).
  • New endpoint GET /api/v1/workers/{name}/events; CRD schema updated and synced to Helm.

Verification

  • Controller: go build, go vet, and full tests pass (api, server, controller), including new event tests.
  • Dashboard proxy: 69/69 tests pass, including new allowlist assertions for the events/status routes.
  • Dashboard web: vite build succeeds; Workers tab verified in a browser.

…-mediated fleet integration

Bake the pinned QoderAI/better-harness CLI into the openclaw-base, hermes, and
qwenpaw images (Node 22 via NodeSource for the Python images; npm install
--omit=dev since the project is npm-managed) and ship a builtin better-harness
skill in every agent template (Manager, OpenClaw/CoPaw/Hermes/OpenHuman/QwenPaw
workers, Team Leader).

Each agent runs a deterministic, read-only `better-harness harness source`
evidence collection at most once per 7 days (run-weekly.sh cadence guard,
state persisted to MinIO so it survives restarts), authors findings, and
reports them to the Manager over Matrix/shared storage — never the human user.
The weekly trigger is native to each runtime: QwenPaw and Hermes worker daemons
run a supervised run_better_harness_weekly_loop background task; OpenClaw/CoPaw/
Manager/Team Leader invoke run-weekly.sh from their heartbeat/session-wake.

The Manager aggregates all agents' reports via the new harness-integration skill
(aggregate-reports.sh) into one consolidated summary, presents a single
fleet-wide integration plan to the admin for approval, and applies approved
changes to the manager-owned builtin templates / controller-owned config so they
propagate to every agent via upgrade-builtins.sh / pushBuiltinSkills. Agents
never self-apply hooks/loops/skill edits.

Add an AGENTTEAMS_BETTER_HARNESS_ENABLED env gate (default on) propagated to
every worker/manager container to disable the weekly review fleet-wide.

Tests: pushBuiltinSkills mirrors better-harness for all runtimes (Go); worker
weekly-loop unit tests (qwenpaw + hermes); Hermes skill-sync install test;
QwenPaw cadence-state persistence test. Live full-stack integration tests are
deferred (require an LLM API key and full deployment).
…ry timeline

Add a new Workers tab to the dashboard web UI listing every worker; clicking a
worker opens a drill-down detail with current health (reusing the health-probe
strip), an event/failure timeline, and the worker'"'"'s better-harness weekly
findings (read from the existing MinIO shared/harness-reports/<name>/ space).

Back it with a new controller event-history store: WorkerStatus.RecentEvents is
a bounded ring buffer (newest first, capped at 50) recorded by the health
monitor (health transitions, zombie failures) and the lifecycle handlers
(wake/sleep/ensure-ready), exposed via GET /api/v1/workers/{name}/events.
CRD schema updated and synced to Helm.

Tests: controller unit tests for event capping/ordering, lifecycle recording,
and the events endpoint; proxy allowlist tests for the events/status routes.
Web build verified; Workers tab verified in a browser.
…mediation

# Conflicts:
#	dashboard/web/src/style.css
…refix

The AI provider management feature (merged from main) generated route names
with the retired hiclaw-<name>-route prefix, breaking the AgentTeams
rename-contract check (helm-lint). Rename to agentteams-<name>-route across
the gateway client, Higress driver, gateway handler, and their tests.
*out = new(string)
**out = **in
}
if in.ServiceEnabled != nil {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CRITICAL: BackendRuntime deepcopy removed while field is still *string

WorkerSpec.BackendRuntime is declared as *string (pointer type) at types.go:239, but its deep-copy logic has been removed in this hunk. Without this block, DeepCopyInto will shallow-copy the pointer — both the original and the copy share the same underlying *string. In the Kubernetes controller-runtime, status updates and reconciliation copies could alias the same underlying pointer, causing data races or unintended mutation of the original Worker spec.

Restore the if in.BackendRuntime != nil { ... } block, or re-run controller-gen object paths=./... to regenerate correctly.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

@nattiini45 nattiini45 Jul 30, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified as a false positive, with evidence:

  1. The base branch's zz_generated.deepcopy.go contained a duplicate if in.BackendRuntime != nil { ... } block inside WorkerSpec.DeepCopyInto (two occurrences - one after DeployMode, one after Env). This hunk removed the duplicate; a valid block remains (now after the Env block, matching controller-gen's canonical field order per types.go).

  2. Re-running controller-gen object paths=./api/v1beta1 on this branch produces zero diff - the committed file exactly matches machine-generated output, with exactly one BackendRuntime deepcopy block. No pointer aliasing is possible.

  3. go build, go vet, and go test pass for the api, server, controller, service, gateway, and config packages.

The other three findings (unpinned BETTER_HARNESS_REF, missing jq guard, missing keyboard handler) were confirmed and fixed in 658f8cd.

Comment thread openclaw-base/Dockerfile Outdated
# better-harness skill for weekly agent self-review. Node 22 (installed above
# via NodeSource) satisfies the CLI's >=22.20 engine constraint. The project is
# npm-managed (packageManager: npm), so install with npm, not pnpm.
ARG BETTER_HARNESS_REF=main

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: BETTER_HARNESS_REF=main makes builds non-reproducible

The default main branch is a moving target — git clone --depth 1 --branch main fetches whatever HEAD is at build time. If the upstream QoderAI/better-harness repo introduces a breaking change or force-pushes, all image builds will silently change behavior or break.

Pin to a specific commit SHA or tag (as done for HERMES_GIT_REF=v2026.4.16 in the hermes Dockerfile) to ensure reproducible builds. The same unpinned default appears in hermes/Dockerfile:87 and qwenpaw/Dockerfile:109.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

# Collect every findings file (any agent, any date).
mapfile -t FILES < <(find "${LOCAL_DIR}" -mindepth 2 -maxdepth 2 -name '*.json' 2>/dev/null | sort)
if [ "${#FILES[@]}" -eq 0 ]; then
jq -cn '{agents:0, reports:0, findings:[]}'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: jq used without verifying availability

mc is guarded with command -v mc on line 30, but jq is called directly here and at lines 49, 55, and 61. If the container image lacks jq, the script fails with an unhelpful "command not found" error. Add an early guard after the PREFIX validation:

if ! command -v jq >/dev/null 2>&1; then
    log "ERROR: jq is required but not found on PATH"
    exit 1
fi

Reply with @kilocode-bot fix it to have Kilo Code address this issue.

Comment thread dashboard/web/src/panels/workers.js Outdated
}
el.innerHTML = workers.map((w) => workerRow(w)).join('');
el.querySelectorAll('[data-worker-name]').forEach((card) => {
card.addEventListener('click', () => onOpen(card.dataset.workerName));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SUGGESTION: Add keyboard handler for Enter/Space on worker cards

The card has role="button" and tabindex="0" (line 106), making it keyboard-focusable, but only a click listener is registered here. Keyboard users pressing Enter or Space won't trigger the detail view. Add a keydown handler alongside the click listener.


Reply with @kilocode-bot fix it to have Kilo Code address this issue.

@kilo-code-bot

kilo-code-bot Bot commented Jul 30, 2026

Copy link
Copy Markdown

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Incremental Review (658f8cd)

All 4 issues from the previous review are resolved:

  • CRITICAL (BackendRuntime deepcopy): False positive — the main branch had a duplicate deepcopy block; the PR correctly removed one copy while keeping the properly-positioned copy at zz_generated.deepcopy.go:1359.
  • WARNING (unpinned BETTER_HARNESS_REF): Fixed — all 3 Dockerfiles pin to commit SHA 4b871d9b.
  • WARNING (missing jq guard): Fixed — aggregate-reports.sh now checks command -v jq early.
  • SUGGESTION (keyboard access): Fixed — Enter/Space keydown handler added to worker cards.

No new issues found in the incremental diff.

Files Reviewed (6 files)
  • changelog/current.md
  • dashboard/web/src/panels/workers.js
  • hermes/Dockerfile
  • manager/agent/skills/harness-integration/scripts/aggregate-reports.sh
  • openclaw-base/Dockerfile
  • qwenpaw/Dockerfile
Previous Review Summary (commit 7d26a2d)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 7d26a2d)

Status: 4 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 1
WARNING 2
SUGGESTION 1
Issue Details (click to expand)

CRITICAL

File Line Issue
agentteams-controller/api/v1beta1/zz_generated.deepcopy.go 1347 BackendRuntime deepcopy removed while field is still *string — causes pointer aliasing

WARNING

File Line Issue
openclaw-base/Dockerfile 131 BETTER_HARNESS_REF=main unpinned — non-reproducible builds (also hermes/Dockerfile:87, qwenpaw/Dockerfile:109)
manager/agent/skills/harness-integration/scripts/aggregate-reports.sh 37 jq used without availability check — ungraceful failure

SUGGESTION

File Line Issue
dashboard/web/src/panels/workers.js 97 Worker cards keyboard-inaccessible — no Enter/Space handler
Files Reviewed (63 files)
  • agentteams-controller/api/v1beta1/types.go - 1 issue (deepcopy target)
  • agentteams-controller/api/v1beta1/worker_events.go
  • agentteams-controller/api/v1beta1/zz_generated.deepcopy.go - 1 CRITICAL issue
  • agentteams-controller/config/crd/workers.agentteams.io.yaml
  • agentteams-controller/internal/config/config.go
  • agentteams-controller/internal/controller/health_monitor_controller.go
  • agentteams-controller/internal/gateway/client.go
  • agentteams-controller/internal/gateway/client_test.go
  • agentteams-controller/internal/gateway/higress.go
  • agentteams-controller/internal/gateway/types.go
  • agentteams-controller/internal/server/gateway_handler.go
  • agentteams-controller/internal/server/gateway_handler_test.go
  • agentteams-controller/internal/server/http.go
  • agentteams-controller/internal/server/lifecycle_handler.go
  • agentteams-controller/internal/server/resource_handler.go
  • agentteams-controller/internal/server/types.go
  • agentteams-controller/internal/server/worker_events_test.go
  • agentteams-controller/internal/service/deployer_better_harness_test.go
  • agentteams-controller/internal/service/worker_env.go
  • changelog/current.md
  • dashboard/server/test/allowlist.test.js
  • dashboard/web/index.html
  • dashboard/web/src/api.js
  • dashboard/web/src/main.js
  • dashboard/web/src/panels/workers.js - 1 SUGGESTION issue
  • dashboard/web/src/style.css
  • helm/agentteams/crds/workers.agentteams.io.yaml
  • hermes/Dockerfile
  • hermes/src/hermes_worker/better_harness.py
  • hermes/src/hermes_worker/worker.py
  • hermes/tests/test_better_harness.py
  • hermes/tests/test_worker_sync.py
  • manager/agent/HEARTBEAT.md
  • manager/agent/copaw-manager-agent/HEARTBEAT.md
  • manager/agent/copaw-worker-agent/AGENTS.md
  • manager/agent/copaw-worker-agent/skills/better-harness/SKILL.md
  • manager/agent/copaw-worker-agent/skills/better-harness/scripts/run-weekly.sh
  • manager/agent/hermes-worker-agent/AGENTS.md
  • manager/agent/hermes-worker-agent/skills/better-harness/SKILL.md
  • manager/agent/hermes-worker-agent/skills/better-harness/scripts/run-weekly.sh
  • manager/agent/openhuman-worker-agent/AGENTS.md
  • manager/agent/openhuman-worker-agent/skills/better-harness/SKILL.md
  • manager/agent/openhuman-worker-agent/skills/better-harness/scripts/run-weekly.sh
  • manager/agent/qwenpaw-worker-agent/AGENTS.md
  • manager/agent/qwenpaw-worker-agent/skills/better-harness/SKILL.md
  • manager/agent/qwenpaw-worker-agent/skills/better-harness/scripts/run-weekly.sh
  • manager/agent/skills/better-harness/SKILL.md
  • manager/agent/skills/better-harness/scripts/run-weekly.sh
  • manager/agent/skills/harness-integration/SKILL.md
  • manager/agent/skills/harness-integration/scripts/aggregate-reports.sh - 1 WARNING issue
  • manager/agent/skills/worker-management/scripts/push-worker-skills.sh
  • manager/agent/team-leader-agent/HEARTBEAT.md
  • manager/agent/team-leader-agent/skills/better-harness/SKILL.md
  • manager/agent/team-leader-agent/skills/better-harness/scripts/run-weekly.sh
  • manager/agent/worker-agent/AGENTS.md
  • manager/agent/worker-agent/skills/better-harness/SKILL.md
  • manager/agent/worker-agent/skills/better-harness/scripts/run-weekly.sh
  • openclaw-base/Dockerfile - 1 WARNING issue
  • qwenpaw/Dockerfile
  • qwenpaw/src/qwenpaw_worker/better_harness.py
  • qwenpaw/src/qwenpaw_worker/worker.py
  • qwenpaw/tests/test_better_harness.py
  • qwenpaw/tests/test_sync.py

Fix these issues in Kilo Cloud


Reviewed by mimo-v2.5-pro · Input: 103K · Output: 18.1K · Cached: 785.2K

- Pin BETTER_HARNESS_REF to commit SHA 4b871d9b in openclaw-base, hermes, and qwenpaw Dockerfiles (init/fetch/checkout pattern; branch/tag overrides still work) for reproducible builds
- Guard jq availability in harness-integration aggregate-reports.sh, failing fast with a clear error
- Activate dashboard worker cards with Enter/Space in addition to click (role=button cards were keyboard-focusable but not operable)

The CRITICAL deepcopy finding was verified as a false positive: controller-gen regeneration produces zero diff; the PR hunk removed a duplicate BackendRuntime block present in the base file.
@nattiini45
nattiini45 merged commit 2bd7d2b into main Jul 30, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant