Keep every agent in fine fettle. While you sleep, NightFettle uses Codex to health-check every AI agent your team runs, diagnoses and fixes what's degrading, benchmark-gates every change, and hands you a morning report.
Built for OpenAI Build Week 2026 · Track: Work & Productivity · Powered by Codex + GPT-5.6
🌐 Live demo: fettle-sage.vercel.app
Every team now ships AI agents — support bots, internal copilots, workflow automations. But agents rot silently:
- A tool's API changes and the agent's calls start failing.
- Prompts get appended to endlessly until token cost doubles.
- Success rate drifts down, latency drifts up — and nobody notices until it breaks in production or the bill explodes.
Traditional software has CI/CD, Dependabot, health checks, and alerts to catch this. AI agents have none of it. Teams don't need another, smarter agent — they need an ops layer that keeps the agents they already run healthy.
NightFettle is the AgentOps night shift. When your agents go idle, it takes over:
- Health check — reads each agent's traces and computes tool-success rate, prompt size, latency, cost, and error rate; flags what's wrong.
- Diagnose & fix with Codex — hands the health report and a throwaway copy of the agent source to Codex (GPT-5.6), which locates the problem and produces a concrete diff.
- Benchmark gate — nothing ships blindly: machine-checkable acceptance criteria run against the working copy. The fix is copied back only after the gate passes and the operator supplies
--apply; otherwise the original stays untouched. - Morning report — serves a dashboard of the night's repairs, trace-replay results, projected savings, and changes awaiting sign-off.
You wake up and last night's work is already done.
OpenAI's own Frontier platform proved that self-improving agents (evaluate → optimize loops) are the future — but it's enterprise-grade and closed inside one platform. NightFettle brings that capability to every developer and small team: open-source and framework-agnostic. Any agent written with the Agents SDK, LangGraph, or your own stack can plug in.
| Tool | What it does | How NightFettle differs |
|---|---|---|
| Letta (sleep-time compute) | Rewrites an agent's memory while idle | NightFettle fixes the code and prompts — it edits engineering artifacts |
| AgentOps / Langfuse / Arize | Observe: monitor, dashboards, alerts | NightFettle doesn't just tell you it broke — Codex fixes it |
| OpenAI Frontier | Enterprise, self-optimizing inside the platform | NightFettle is open, framework-agnostic, for developers' own agents |
In one line: others monitor or tweak config — NightFettle lets Codex change the code and gates every release on a benchmark.
git clone https://github.com/Vector897/Fettle.git
cd Fettle/cli
npm install
npx tsx src/index.ts check
npx tsx src/index.ts fix support-botThe repair command edits a throwaway working copy, prints the Codex diff and Session ID, and runs the benchmark gate. Add --apply only when you want a passing change copied back to the demo agent. On Windows systems where local security software blocks the Codex sandbox, --bypass-sandbox is available; NightFettle still scopes the run to its temporary copy.
Run the dashboard separately:
cd ../web
npm install
npm run dev # open http://localhost:3000- TypeScript CLI — health metrics, trace fixtures, working-copy isolation, diffs, and benchmark gates
- Codex CLI running GPT-5.6 — the repair engine that reads and edits agent code
- Next.js 16 + React 19 + Tailwind CSS 4 — the static morning-report dashboard
- Deployment target: Vercel
Codex is both the product's engine and part of the way we built and hardened NightFettle. We ran the same repair loop that users run against four deliberately unhealthy agents. Those sessions gave us real diffs to validate working-copy isolation, exposed weak benchmark criteria and transient-file noise, and proved that a failed repair is discarded rather than shipped.
| Build / validation milestone | What Codex did | Codex Session ID |
|---|---|---|
| Stale tool-schema repair | Updated the LangChain tool schema and request payloads from invoice_id to invoiceId. The gate passed; fixture trace replay moved from 10/14 to 14/14 successful calls. |
019f66f1-235e-7812-8a55-4aea689283b9 |
| Prompt coaching | Replaced contradictory legacy instructions with a concise research brief and added a TTL source cache. Five hardened checks passed. | 019f6709-6c79-7562-a668-ddf2321b546f |
| Benchmark rejection / rollback | Isolated untrusted database content and removed the connection string from error logs, but left a hard-coded password in source. NightFettle rejected the repair and discarded the working copy. | 019f6703-0ad5-76f1-b5eb-ac20cd175d64 |
| Security-patch retry | Moved the database URL to ANALYTICS_DB_URL, serialized untrusted context as data, and sanitized error logging. The security gate then passed. |
019f6708-7719-79a3-8781-aeb62b3ad99e |
| Retry-storm repair | Added a three-retry limit and capped exponential backoff, replacing the unbounded zero-delay loop. | 019f670b-2626-73b1-b8fa-af79ac7edb05 |
All runs above were executed through NightFettle against throwaway working copies. The seeded unhealthy agents were left unchanged for a reproducible demo. Percentages labelled as trace replay are results over the checked-in fixture traces, not claims about live production traffic.
- 📺 Demo video (<3 min): TODO YouTube link
- 🖼️ Screenshots: TODO
Import this repository into Vercel with these project settings:
- Framework Preset: Next.js
- Root Directory:
web - Include source files outside of the Root Directory: enabled (the static dashboard reads
demo/during the build) - Build / Install / Output settings: use the detected Next.js defaults
The checked-in .vercelignore keeps local governance notes and environment files out of deployment uploads.
MIT — see LICENSE.