These rules are the operational checklist for designing a new agent.
Write down the desired outcome, environment, and success criteria first.
Why: architecture chosen before the actual job is understood usually turns into unnecessary complexity.
Prefer script → prompted app → tool-calling agent → multi-step agent → MAS in that order.
Why: every jump up the ladder adds failure surface, cost, and debugging overhead.
Specify mission, trigger conditions, inputs, outputs, tools, guardrails, stop conditions, and evaluation criteria.
Why: vague agents become vague implementations.
Design how the agent thinks, acts, observes, and decides whether to continue.
Why: hidden iteration often becomes hidden failure.
Force the agent to observe the result of one tool call before deciding the next step.
Why: sequential observation catches errors that parallel guessing hides.
Use clear descriptions, typed arguments, structured outputs, consistent identifiers, and useful errors.
Why: most agent failures blamed on “the model” are really broken tool interfaces.
If one tool returns a resource the next tool must mutate, return the exact ID or key the second tool expects.
Why: mismatched IDs create cascading hallucinations and loops.
Specialists should have narrower tool sets than a coordinator, not wider ones.
Why: reducing tool surface improves accuracy and makes debugging easier.
A specialist task must carry all the context needed to execute it correctly.
Why: specialists cannot rely on shared hidden context being preserved.
Specify max steps, max turns, timeout behavior, and what the agent should do after repeated failure.
Why: otherwise a broken tool or prompt mismatch can burn tokens indefinitely.
Rebooking, deletion, cancellation, writes, and external side effects need an explicit confirmation policy.
Why: prompts alone are not enough protection for high-impact actions.
Choose direct lookup, exact match, or semantic retrieval intentionally instead of adding generic “memory.”
Why: unnecessary memory layers make agents slower and harder to reason about.
Test realistic tasks, final outputs, trajectories, and resulting environment state.
Why: text that sounds correct can still hide wrong actions or broken state transitions.
When the agent fails because of a prompt, tool, or architecture issue, preserve that task in the eval suite.
Why: otherwise the same bug will reappear after the next refactor.
Prefer understandable behavior constraints over giant lists of brittle commands.
Why: agents generalize better when they understand the reason for a rule.
These rules are the actionable layer for this chapter. They are narrower than the final skill and focus only on foundations, tools, and MCP.
If the answer depends on live data, a database, or an external system, say whether you actually have that access.
Why: capability limits must be explicit so the agent does not overclaim.
Booking details, file contents, live status, IDs, prices, and current system state must be retrieved, not guessed.
Why: honesty about unavailable information is a core behavior, not a style preference.
When evaluating or defining a tool, check that it has a description, executable function, typed inputs, and typed output.
Why: this is the minimum structure needed to make tool calling reliable.
The model decides which tool to call and with what arguments; the runtime performs the call and returns the result.
Why: confusing these roles leads to hallucinated tool results and broken reasoning.
If later reasoning depends on the result, return something machine-readable whenever possible.
Why: structured output is easier for the agent to use safely.
Errors should help the agent decide whether to ask for missing input, retry safely, or choose another path.
Why: clear errors are a core tool-design principle.
If retries can happen, repeated calls should not duplicate effects.
Why: retries after timeouts or restarts are a normal operational reality.
Use wrappers or decorators that derive the interface from code instead of hand-copying signatures in multiple places.
Why: the wrap_tool pattern reduces drift between implementation and contract.
Tool access increases capability, but it does not replace policy. Keep explicit rules like “never invent details” and “request confirmation before cancellation.”
Why: tool access does not replace policy constraints.
Reach for MCP when tool reuse, API churn, shared ownership, or format differences become the real issue.
Why: start simple and add MCP only when complexity forces you.
Do not collapse orchestration, transport, and capability exposure into one vague mental model.
Why: the architecture only helps when the boundaries stay meaningful.
Decide whether you need exact search, semantic search, direct file reads, MCP context, or shell inspection before you do anything.
Why: enrichment for the actual Oz runtime matters more than repeating abstract theory.
If the needed context is already exposed through MCP, use that clean integration boundary first.
Why: this fits both reuse goals and the runtime’s tool preference.
These rules are the operational checklist for adding memory and safety to a new agent.
Do not collapse “memory” into one vague feature.
Why: each layer solves a different problem and has different cost and persistence trade-offs.
Use a sliding window or thread history before inventing a heavy memory architecture.
Why: unnecessary memory makes agents slower and harder to debug.
Reach for RAG when the task depends on changing policies, manuals, or records.
Why: retrieval keeps the agent grounded and is cheaper to iterate on.
Resolve relative dates, user shorthand, and other ambiguous formats before acting.
Why: many downstream failures are really input-shape mismatches.
Adapt the search query to the document language instead of blaming retrieval quality too early.
Why: a weak query can make a good retrieval system look broken.
Never assume retrieved text, file contents, or tool responses are safe instructions.
Why: indirect prompt injection often arrives through tool output, not user text.
Block clearly off-topic requests before they reach the core agent loop.
Why: this reduces unnecessary model work and narrows the attack surface.
Passport numbers, emails, phone numbers, and similar data should be redacted in logs.
Why: logging raw sensitive data creates avoidable compliance and security risk.
Add a guard layer after retrieval or tool calls when the source can contain adversarial content.
Why: otherwise the agent may obey poisoned text embedded in the tool response.
Define max steps, max turns, or similar caps.
Why: without hard stops, a broken prompt or API contract can lead to runaway behavior.
Pause before booking, cancelling, deleting, or making externally visible state changes.
Why: approval is an architectural safety boundary, not just a conversational courtesy.
Use structured validation for things like IDs, emails, and required booking fields.
Why: schema validation catches malformed or fabricated arguments before side effects happen.
Combine simple deterministic checks with stronger semantic checks where needed.
Why: guardrail quality improves when each layer handles the part it is best at.
Include off-topic prompts, poisoned tool output, missing data, and restart scenarios.
Why: memory and guardrails only prove their value when the system is stressed.
These rules are the operational checklist for designing the execution loop and decomposition shape of a new agent.
Define how the agent thinks, acts, observes, and decides whether to continue.
Why: hidden loops are harder to debug and easier to break.
After each observation, decide whether the task is solved, needs another step, or should stop with a limitation.
Why: without an explicit stop decision, agents drift into repetition.
Prefer ReAct over plain chain-of-thought when the task depends on live tool results.
Why: observation after each action lets the agent correct the plan instead of committing to one long guess.
Use stop-and-parse or native function-calling boundaries so the runtime, not the model, performs the action.
Why: this blocks the model from inventing tool outcomes.
If the model generates code as the action format, run it in an isolated environment with constrained permissions.
Why: code execution has a larger blast radius than typed tool calls.
Use schemas for subtasks, statuses, and result payloads instead of free-form prose.
Why: structured communication is easier to validate and easier for other agents to consume.
Stay with one agent when the domain is tight and the tool surface is still manageable.
Why: coordination overhead is real and should be earned by better reliability.
Split the system when domains, tools, or verification needs stop fitting one coherent agent.
Why: specialization is only worth it when it actually reduces confusion.
Do not hand every agent the coordinator’s full tool inventory.
Why: narrower scope improves accuracy and reduces accidental misuse.
Delegated tasks must include the exact entities, dates, IDs, and constraints the specialist needs.
Why: specialists should not depend on hidden context surviving delegation.
Use a critic for multi-part, risky, or high-accuracy tasks where incomplete answers are costly.
Why: a dedicated verifier catches omissions that the producer agent may miss.
Preserve enough structure to inspect thoughts, tool calls, specialist results, and critic feedback.
Why: architecture failures are usually diagnosed from the path, not from the last sentence.
Compare single-agent, MAS, and MAS-plus-critic designs against the same tasks.
Why: architecture decisions should be grounded in measured completeness, latency, and cost.
More agents mean more calls, more latency, and more chances for cascading error.
Why: the “best” architecture is the one whose benefits justify its coordination burden.
Add vision or voice components only when the user’s job requires them and define their boundaries clearly.
Why: multiple modalities increase latency, tooling complexity, and handoff requirements.
These rules are the operational checklist for making a new agent deployable and operable.
Specify deployment assumptions, failure handling, observability, and ownership alongside the prompt and tool design.
Why: a prototype that works once is not yet a production system.
Require representative evaluations before rollout and again before risky changes.
Why: production quality starts with evidence, not confidence.
Bound request duration, retry behavior, and total action count explicitly.
Why: production incidents often come from unbounded waiting or looping.
Define how the agent behaves when a model, tool, or dependency is unavailable.
Why: safe partial service is usually better than chaotic failure.
Prefer staged rollout, canaries, or limited exposure before full deployment.
Why: the cheapest failure to handle is the one that only reached a small slice of traffic.
Limit tool access, side effects, and data exposure to the smallest surface the job needs.
Why: scale multiplies the blast radius of a bad decision.
Capture request IDs, tool calls, guardrail triggers, retries, failures, and outcome categories.
Why: agent failures are usually diagnosable only from the path, not from the final answer alone.
Define fast signals for alerting and richer traces for root-cause analysis.
Why: the system must notice a problem quickly before an operator has time to investigate deeply.
Set expected limits for model usage, subagent fan-out, retrieval volume, and end-to-end response time.
Why: an accurate agent that is too slow or too expensive may still fail operationally.
Use common tracing, auth, eval, and deployment layers when designing internal agents.
Why: duplicated infrastructure increases maintenance burden and inconsistency.
Add recurring failures, edge cases, and operator complaints back into the regression suite.
Why: continuous improvement only works if operations feed the evaluation loop.
Design deployments so a bad prompt, model, or tool version can be reverted quickly.
Why: recovery speed is part of reliability.
These rules are the operational checklist for evaluating a new agent.
Define tasks, success conditions, and the main failure modes before rollout.
Why: an agent without an eval plan can only be judged by demos.
Use fields like expected_state_changes, validation_rule, and policies_to_check instead of prose notes.
Why: structured expectations are what make automated grading possible.
Represent read-only tasks with expected_state_changes = None and fail the task if state changes anyway.
Why: unintended mutation is a real agent failure.
Use a field like needs_dialogue and do not pretend a single-turn run evaluates confirmation or clarification behavior.
Why: dialogue logic is invisible without a user partner.
Hard-cap both user turns and total trajectory steps.
Why: bad tool contracts or bad prompts can otherwise loop indefinitely.
If one tool returns identifiers another tool cannot consume, repair the interface first.
Why: API mismatch can create fake “reasoning” failures.
Compare environment state before and after the run and apply explicit comparison rules where needed.
Why: state is more reliable than response prose when checking mutations.
Check ordering, confirmation, and required lookup rules independently.
Why: an agent can reach the right final state while still violating process constraints.
Define good, so-so, and bad with concrete conditions.
Why: vague judge prompts drift toward leniency and inconsistent scoring.
Split usefulness, groundedness, and efficiency if a single prompt makes them bleed together.
Why: rating one criterion first can bias the others.
Store expert labels for a small but representative task set.
Why: judge tuning without ground truth becomes guesswork.
Track Cohen’s κ or equivalent plus disagreement counts.
Why: matching human labels matters more than sounding plausible.
Use a coarse quality score or label-distribution check to detect “everything is good” behavior.
Why: a judge can be systematically lenient even when its outputs look structured.
Run deterministic gates first, then add semantic judging where code cannot decide cleanly.
Why: code is cheaper and more stable, while LLM judges are flexible but noisier.
Use real code paths with isolated data or instances whenever possible.
Why: full mocks drift and full production is risky.
When the agent misses a case in testing or production, add that case back into the suite.
Why: otherwise the same bug returns after the next prompt or tool change.
Record runtime, call count, and estimated spend for each eval version.
Why: the highest-scoring judge or harness may be too slow or too expensive to keep.
Run multiple trials when stability matters and report both “can succeed” and “always succeeds.”
Why: one lucky run is not reliability.