Skip to content

Latest commit

 

History

History
453 lines (273 loc) · 20.2 KB

File metadata and controls

453 lines (273 loc) · 20.2 KB

Build New Agents — Rules

These rules are the operational checklist for designing a new agent.

B1. Define the job to be done before choosing architecture.

Write down the desired outcome, environment, and success criteria first.

Why: architecture chosen before the actual job is understood usually turns into unnecessary complexity.

B2. Use the simplest interaction level that can solve the task.

Prefer script → prompted app → tool-calling agent → multi-step agent → MAS in that order.

Why: every jump up the ladder adds failure surface, cost, and debugging overhead.

B3. Write an explicit agent contract.

Specify mission, trigger conditions, inputs, outputs, tools, guardrails, stop conditions, and evaluation criteria.

Why: vague agents become vague implementations.

B4. Make the TAO loop explicit for non-trivial agents.

Design how the agent thinks, acts, observes, and decides whether to continue.

Why: hidden iteration often becomes hidden failure.

B5. Limit each reasoning step to one meaningful action when tool use matters.

Force the agent to observe the result of one tool call before deciding the next step.

Why: sequential observation catches errors that parallel guessing hides.

B6. Design every tool as a stable contract.

Use clear descriptions, typed arguments, structured outputs, consistent identifiers, and useful errors.

Why: most agent failures blamed on “the model” are really broken tool interfaces.

B7. Never make downstream tools guess identifier mappings.

If one tool returns a resource the next tool must mutate, return the exact ID or key the second tool expects.

Why: mismatched IDs create cascading hallucinations and loops.

B8. Give each agent only the tools it actually needs.

Specialists should have narrower tool sets than a coordinator, not wider ones.

Why: reducing tool surface improves accuracy and makes debugging easier.

B9. Require self-contained delegated tasks in multi-agent systems.

A specialist task must carry all the context needed to execute it correctly.

Why: specialists cannot rely on shared hidden context being preserved.

B10. Add hard limits for looping and failure recovery.

Specify max steps, max turns, timeout behavior, and what the agent should do after repeated failure.

Why: otherwise a broken tool or prompt mismatch can burn tokens indefinitely.

B11. Put approvals around irreversible or state-changing actions.

Rebooking, deletion, cancellation, writes, and external side effects need an explicit confirmation policy.

Why: prompts alone are not enough protection for high-impact actions.

B12. Design memory and retrieval only where the task needs them.

Choose direct lookup, exact match, or semantic retrieval intentionally instead of adding generic “memory.”

Why: unnecessary memory layers make agents slower and harder to reason about.

B13. Add evaluation before calling the agent “done.”

Test realistic tasks, final outputs, trajectories, and resulting environment state.

Why: text that sounds correct can still hide wrong actions or broken state transitions.

B14. Turn every discovered bug into a regression case.

When the agent fails because of a prompt, tool, or architecture issue, preserve that task in the eval suite.

Why: otherwise the same bug will reappear after the next refactor.

B15. Keep prompts lean and explain the why behind critical instructions.

Prefer understandable behavior constraints over giant lists of brittle commands.

Why: agents generalize better when they understand the reason for a rule.

Foundations + Tools & MCP — Rules

These rules are the actionable layer for this chapter. They are narrower than the final skill and focus only on foundations, tools, and MCP.

F1. State capability boundaries before answering capability-dependent requests.

If the answer depends on live data, a database, or an external system, say whether you actually have that access.

Why: capability limits must be explicit so the agent does not overclaim.

F2. Never invent facts that should come from a tool.

Booking details, file contents, live status, IDs, prices, and current system state must be retrieved, not guessed.

Why: honesty about unavailable information is a core behavior, not a style preference.

F3. Treat every tool as a contract with four parts.

When evaluating or defining a tool, check that it has a description, executable function, typed inputs, and typed output.

Why: this is the minimum structure needed to make tool calling reliable.

F4. Separate tool selection from tool execution.

The model decides which tool to call and with what arguments; the runtime performs the call and returns the result.

Why: confusing these roles leads to hallucinated tool results and broken reasoning.

F5. Prefer typed or structured tool outputs over free-form prose.

If later reasoning depends on the result, return something machine-readable whenever possible.

Why: structured output is easier for the agent to use safely.

F6. Make tool failures actionable.

Errors should help the agent decide whether to ask for missing input, retry safely, or choose another path.

Why: clear errors are a core tool-design principle.

F7. Design side-effecting tools to be idempotent when possible.

If retries can happen, repeated calls should not duplicate effects.

Why: retries after timeouts or restarts are a normal operational reality.

F8. Prefer generated tool schemas over duplicated manual metadata once the system grows.

Use wrappers or decorators that derive the interface from code instead of hand-copying signatures in multiple places.

Why: the wrap_tool pattern reduces drift between implementation and contract.

F9. Add behavioral policies even when tools exist.

Tool access increases capability, but it does not replace policy. Keep explicit rules like “never invent details” and “request confirmation before cancellation.”

Why: tool access does not replace policy constraints.

F10. Use MCP to solve integration scaling problems, not to look sophisticated.

Reach for MCP when tool reuse, API churn, shared ownership, or format differences become the real issue.

Why: start simple and add MCP only when complexity forces you.

F11. Keep Host, Client, and Server responsibilities distinct in MCP.

Do not collapse orchestration, transport, and capability exposure into one vague mental model.

Why: the architecture only helps when the boundaries stay meaningful.

F12. Map abstract needs to concrete runtime tools before acting.

Decide whether you need exact search, semantic search, direct file reads, MCP context, or shell inspection before you do anything.

Why: enrichment for the actual Oz runtime matters more than repeating abstract theory.

F13. Prefer an existing relevant MCP server over ad-hoc shell or web work.

If the needed context is already exposed through MCP, use that clean integration boundary first.

Why: this fits both reuse goals and the runtime’s tool preference.

Memory & Guardrails — Rules

These rules are the operational checklist for adding memory and safety to a new agent.

M1. Choose short-term, long-term, and context memory separately.

Do not collapse “memory” into one vague feature.

Why: each layer solves a different problem and has different cost and persistence trade-offs.

M2. Start with the smallest memory design that satisfies the task.

Use a sliding window or thread history before inventing a heavy memory architecture.

Why: unnecessary memory makes agents slower and harder to debug.

M3. Use retrieval for dynamic or document-backed knowledge before training the model.

Reach for RAG when the task depends on changing policies, manuals, or records.

Why: retrieval keeps the agent grounded and is cheaper to iterate on.

M4. Normalize inputs before retrieval or tool calls.

Resolve relative dates, user shorthand, and other ambiguous formats before acting.

Why: many downstream failures are really input-shape mismatches.

M5. Use query rewriting, decomposition, or HyDE when the user’s language does not match the corpus.

Adapt the search query to the document language instead of blaming retrieval quality too early.

Why: a weak query can make a good retrieval system look broken.

M6. Treat tool output as untrusted data until inspected.

Never assume retrieved text, file contents, or tool responses are safe instructions.

Why: indirect prompt injection often arrives through tool output, not user text.

M7. Add a relevance or scope gate when the agent should only handle a bounded domain.

Block clearly off-topic requests before they reach the core agent loop.

Why: this reduces unnecessary model work and narrows the attack surface.

M8. Mask PII before logging.

Passport numbers, emails, phone numbers, and similar data should be redacted in logs.

Why: logging raw sensitive data creates avoidable compliance and security risk.

M9. Inspect high-risk tool outputs before feeding them back into the model.

Add a guard layer after retrieval or tool calls when the source can contain adversarial content.

Why: otherwise the agent may obey poisoned text embedded in the tool response.

M10. Put hard limits on loops and tool usage.

Define max steps, max turns, or similar caps.

Why: without hard stops, a broken prompt or API contract can lead to runaway behavior.

M11. Require human approval for irreversible actions.

Pause before booking, cancelling, deleting, or making externally visible state changes.

Why: approval is an architectural safety boundary, not just a conversational courtesy.

M12. Validate critical action inputs with schemas before execution.

Use structured validation for things like IDs, emails, and required booking fields.

Why: schema validation catches malformed or fabricated arguments before side effects happen.

M13. Layer guardrails instead of betting on one perfect detector.

Combine simple deterministic checks with stronger semantic checks where needed.

Why: guardrail quality improves when each layer handles the part it is best at.

M14. Test memory and safety with failure cases, not only happy paths.

Include off-topic prompts, poisoned tool output, missing data, and restart scenarios.

Why: memory and guardrails only prove their value when the system is stressed.

Workflow, MAS & Multimodal — Rules

These rules are the operational checklist for designing the execution loop and decomposition shape of a new agent.

W1. Make the TAO loop explicit when the agent must use tools or iterate.

Define how the agent thinks, acts, observes, and decides whether to continue.

Why: hidden loops are harder to debug and easier to break.

W2. End every reasoning cycle with a continue-or-stop decision.

After each observation, decide whether the task is solved, needs another step, or should stop with a limitation.

Why: without an explicit stop decision, agents drift into repetition.

W3. Use ReAct when tools matter.

Prefer ReAct over plain chain-of-thought when the task depends on live tool results.

Why: observation after each action lets the agent correct the plan instead of committing to one long guess.

W4. Keep action generation separate from execution.

Use stop-and-parse or native function-calling boundaries so the runtime, not the model, performs the action.

Why: this blocks the model from inventing tool outcomes.

W5. Sandbox any code-agent execution.

If the model generates code as the action format, run it in an isolated environment with constrained permissions.

Why: code execution has a larger blast radius than typed tool calls.

W6. Prefer structured outputs for plans, handoffs, and specialist results.

Use schemas for subtasks, statuses, and result payloads instead of free-form prose.

Why: structured communication is easier to validate and easier for other agents to consume.

W7. Prefer a single agent while the task remains coherent.

Stay with one agent when the domain is tight and the tool surface is still manageable.

Why: coordination overhead is real and should be earned by better reliability.

W8. Move to MAS when decomposition makes prompts and tool surfaces simpler.

Split the system when domains, tools, or verification needs stop fitting one coherent agent.

Why: specialization is only worth it when it actually reduces confusion.

W9. Give each specialist a narrow role and a narrow toolset.

Do not hand every agent the coordinator’s full tool inventory.

Why: narrower scope improves accuracy and reduces accidental misuse.

W10. Make every handoff self-contained.

Delegated tasks must include the exact entities, dates, IDs, and constraints the specialist needs.

Why: specialists should not depend on hidden context surviving delegation.

W11. Add a critic when verification matters more than latency.

Use a critic for multi-part, risky, or high-accuracy tasks where incomplete answers are costly.

Why: a dedicated verifier catches omissions that the producer agent may miss.

W12. Trace the full workflow, not only the final answer.

Preserve enough structure to inspect thoughts, tool calls, specialist results, and critic feedback.

Why: architecture failures are usually diagnosed from the path, not from the last sentence.

W13. Benchmark competing architectures on the same scenarios.

Compare single-agent, MAS, and MAS-plus-critic designs against the same tasks.

Why: architecture decisions should be grounded in measured completeness, latency, and cost.

W14. Count coordination cost as part of correctness.

More agents mean more calls, more latency, and more chances for cascading error.

Why: the “best” architecture is the one whose benefits justify its coordination burden.

W15. Treat multimodality as additional specialist work, not as decoration.

Add vision or voice components only when the user’s job requires them and define their boundaries clearly.

Why: multiple modalities increase latency, tooling complexity, and handoff requirements.

Production Engineering — Rules

These rules are the operational checklist for making a new agent deployable and operable.

P1. Do not call an agent production-ready until its operating model is written down.

Specify deployment assumptions, failure handling, observability, and ownership alongside the prompt and tool design.

Why: a prototype that works once is not yet a production system.

P2. Treat eval as a release gate, not as optional polish.

Require representative evaluations before rollout and again before risky changes.

Why: production quality starts with evidence, not confidence.

P3. Put timeouts, retries, and stop conditions into the runtime contract.

Bound request duration, retry behavior, and total action count explicitly.

Why: production incidents often come from unbounded waiting or looping.

P4. Design a degraded mode before the first outage.

Define how the agent behaves when a model, tool, or dependency is unavailable.

Why: safe partial service is usually better than chaotic failure.

P5. Roll out risky changes gradually.

Prefer staged rollout, canaries, or limited exposure before full deployment.

Why: the cheapest failure to handle is the one that only reached a small slice of traffic.

P6. Keep permissions narrower in production than in prototypes.

Limit tool access, side effects, and data exposure to the smallest surface the job needs.

Why: scale multiplies the blast radius of a bad decision.

P7. Log and trace the full request trajectory.

Capture request IDs, tool calls, guardrail triggers, retries, failures, and outcome categories.

Why: agent failures are usually diagnosable only from the path, not from the final answer alone.

P8. Separate incident detection from incident diagnosis.

Define fast signals for alerting and richer traces for root-cause analysis.

Why: the system must notice a problem quickly before an operator has time to investigate deeply.

P9. Put cost and latency budgets into the design.

Set expected limits for model usage, subagent fan-out, retrieval volume, and end-to-end response time.

Why: an accurate agent that is too slow or too expensive may still fail operationally.

P10. Prefer shared platform components over bespoke production glue.

Use common tracing, auth, eval, and deployment layers when designing internal agents.

Why: duplicated infrastructure increases maintenance burden and inconsistency.

P11. Turn production incidents into eval cases.

Add recurring failures, edge cases, and operator complaints back into the regression suite.

Why: continuous improvement only works if operations feed the evaluation loop.

P12. Make rollback straightforward.

Design deployments so a bad prompt, model, or tool version can be reverted quickly.

Why: recovery speed is part of reliability.

Evaluation & Quality — Rules

These rules are the operational checklist for evaluating a new agent.

E1. Design the eval before calling the agent ready.

Define tasks, success conditions, and the main failure modes before rollout.

Why: an agent without an eval plan can only be judged by demos.

E2. Encode expected state in the task itself.

Use fields like expected_state_changes, validation_rule, and policies_to_check instead of prose notes.

Why: structured expectations are what make automated grading possible.

E3. Mark read-only tasks explicitly.

Represent read-only tasks with expected_state_changes = None and fail the task if state changes anyway.

Why: unintended mutation is a real agent failure.

E4. Separate single-turn and dialogue tasks.

Use a field like needs_dialogue and do not pretend a single-turn run evaluates confirmation or clarification behavior.

Why: dialogue logic is invisible without a user partner.

E5. Put max_turns and max_steps into every dialogue harness.

Hard-cap both user turns and total trajectory steps.

Why: bad tool contracts or bad prompts can otherwise loop indefinitely.

E6. Fix tool-contract bugs before blaming the model.

If one tool returns identifiers another tool cannot consume, repair the interface first.

Why: API mismatch can create fake “reasoning” failures.

E7. Use deterministic state graders for stateful tasks.

Compare environment state before and after the run and apply explicit comparison rules where needed.

Why: state is more reliable than response prose when checking mutations.

E8. Grade policy compliance separately from final state.

Check ordering, confirmation, and required lookup rules independently.

Why: an agent can reach the right final state while still violating process constraints.

E9. Use explicit rubrics, not vibe-based judge prompts.

Define good, so-so, and bad with concrete conditions.

Why: vague judge prompts drift toward leniency and inconsistent scoring.

E10. Judge one criterion per prompt when anchor effect matters.

Split usefulness, groundedness, and efficiency if a single prompt makes them bleed together.

Why: rating one criterion first can bias the others.

E11. Keep a human-labeled calibration set.

Store expert labels for a small but representative task set.

Why: judge tuning without ground truth becomes guesswork.

E12. Measure agreement, not only average score.

Track Cohen’s κ or equivalent plus disagreement counts.

Why: matching human labels matters more than sounding plausible.

E13. Watch for judge bias with a simple aggregate score.

Use a coarse quality score or label-distribution check to detect “everything is good” behavior.

Why: a judge can be systematically lenient even when its outputs look structured.

E14. Use hybrid grading.

Run deterministic gates first, then add semantic judging where code cannot decide cleanly.

Why: code is cheaper and more stable, while LLM judges are flexible but noisier.

E15. Prefer a hybrid environment for serious evals.

Use real code paths with isolated data or instances whenever possible.

Why: full mocks drift and full production is risky.

E16. Turn every discovered failure into a regression task.

When the agent misses a case in testing or production, add that case back into the suite.

Why: otherwise the same bug returns after the next prompt or tool change.

E17. Track quality, latency, and cost together.

Record runtime, call count, and estimated spend for each eval version.

Why: the highest-scoring judge or harness may be too slow or too expensive to keep.

E18. Use Pass@k and Pass^k for non-deterministic agents.

Run multiple trials when stability matters and report both “can succeed” and “always succeeds.”

Why: one lucky run is not reliability.