Meta-harness optimization loop wired onto Islo sandboxes. POC: 0/5→5/5 in four proposer steps. Built on islo.dev.

Project page for Meta-harness on Islo (POC). https://zozo123.github.io/meta-harness-on-islo-page/

AI-operated company. Building agent-friend: universal tool adapter for AI agents. @tool → OpenAI, Claude, Gemini, MCP. Live 24/7 on Twitch.

A reasoning benchmark runner for comparing LLMs as OpenClaw agents use them. 52 prompts, 3 eval sets, 11 traps, LLM-as-judge, tier-based leaderboard.

Scenario Testing for AI Agents

Live, open-source benchmark for comparing AI coding agents on real GitHub issues

Online evals on live agent traces. Open-source, self-hostable, OpenTelemetry-native eval harness with regression detection.

A curated list of benchmarks, harnesses, leaderboards, and tools for evaluating AI coding agents.

Local-first synthetic finance anomaly trainer for agent evals.

Transcript-first evaluation tool for comparing coding-agent sessions across Codex, Claude Code, and Pi.

Agent and LLM evaluation harness — golden datasets, multi-scorer execution, regression detection across model versions, cost-quality leaderboards, and CI gates for model promotion.

Regression testing for AI agents — snapshot tool-calls, diff in CI, fail on regressions. A GitHub Action. By Viprasol Tech.

开源通用 AI Agent 真实任务评测 · 同 Prompt、客观开奖、评分细则全公开 | Open-source evaluation of general-purpose AI Agents on real-world tasks with verifiable outcomes — by PingWest / 硅星人

agent-convergence-scorer is a CLI and Python library that scores how lexically similar N agent or LLM outputs are: exact-match rate, Jaccard token overlap, divergence point, and a composite convergence score over any list of runs. An eval primitive for measuring reproducibility and fan-out collapse. Lexical, not semantic. Zero deps.

Trace-first evaluation harness for deciding whether AI agents deserve more tokens, permissions, and trust

Lightweight CI-native regression and behavior-aware evaluation toolkit for black-box agent workflows.

Binary-criteria evaluation harness for Claude skills with planned extension to plugins, agents, and MCP servers. Score every change yes/no across 7 layers — package integrity, trigger quality, functional quality, regression protection, baseline value, model variance, rollout safety. Never gradients.

Vendor-neutral research umbrella for measuring AI plugin, agent, and MCP server quality across CLI runtimes (Claude Code, Gemini CLI, Copilot CLI, Codex CLI).