|
1 | | -AI agents are smart software services that combine generative AI models with contextual data and the ability to automate tasks based on user input and environmental factors that they perceive. |
| 1 | +AI agents are smart applications that use **language models** to understand what you need and then **take action** to help you. They can answer questions, make decisions, and complete tasks automatically. What makes agents special is that they **remember your conversation** and can **actually do things**, not just chat with you like a typical chatbot. |
2 | 2 |
|
3 | | -For example, an organization might build an AI agent to help employees manage expense claims. The agent might use a generative model combined with corporate expenses policy documentation to answer employee questions about what expenses can be claimed and what limits apply. Additionally, the agent could use a programmatic function to automatically submit expense claims for regularly repeated expenses (such as a monthly cellphone bill) or intelligently route expenses to the appropriate approver based on claim amounts. |
| 3 | +:::image type="content" source="../media/ai-agent-architecture.png" alt-text="Diagram of an agent's components consisting of a language model, instructions, and tools."::: |
| 4 | + |
| 5 | +## Identify the expense agent's capabilities |
| 6 | + |
| 7 | +Recall the expense management agent from the introduction—an AI agent that helps employees manage expense claims by answering policy questions and automating claim submissions. Let's examine the three essential capabilities that make this agent effective: |
| 8 | + |
| 9 | +- **Knowledge integration and reasoning**: Uses a generative model with corporate policy documentation to answer questions accurately. |
| 10 | + |
| 11 | +- **Task automation through functions**: Executes programmatic functions to submit expense claims automatically. |
| 12 | + |
| 13 | +- **Intelligent decision-making**: Routes expenses to appropriate approvers based on business rules and claim amounts. |
4 | 14 |
|
5 | 15 | An example of the expenses agent scenario is shown in the following diagram. |
6 | 16 |
|
7 | | - |
| 17 | +:::image type="content" source="../media/expenses-agent.png" alt-text="Diagram of an expenses agent answering questions and submitting claims."::: |
8 | 18 |
|
9 | 19 | The diagram shows the following process: |
10 | 20 |
|
11 | | -1. A user asks the expense agent a question about expenses that can be claimed. |
12 | | -1. The expenses agent accepts the question as a prompt. |
13 | | -1. The agent uses a knowledge store containing expenses policy information to ground the prompt. |
14 | | -1. The grounded prompt is submitted to the agent's language model to generate a response. |
15 | | -1. The agent generates an expense claim on behalf of the user and submits it to be processed and generate a check payment. |
| 21 | +1. A user asks the expense agent a **question about expenses** that can be claimed. |
| 22 | +1. The expenses agent accepts the question as a **prompt**. |
| 23 | +1. The agent uses a **knowledge store** containing expenses policy information to **ground the prompt**. |
| 24 | +1. The grounded prompt is submitted to the agent's **language model** to **generate a response**. |
| 25 | +1. The agent **generates an expense claim** on behalf of the user and submits it to be processed and generate a check payment. |
16 | 26 |
|
17 | | -In more complex scenarios, organizations can develop *multi-agent* solutions in which multiple agents coordinate work between them. For example, a travel booking agent could book flights and hotels for employees and automatically submit expense claims with appropriate receipts to the expenses agent, as shown in this diagram: |
| 27 | +## Explore the travel agent's capabilities |
18 | 28 |
|
19 | | - |
| 29 | +In the previous unit, you also learned about a travel booking agent that extends this scenario into a multi-agent solution. This agent books flights and hotels, then automatically coordinates with the expense agent to submit claims. Here's how the travel agent demonstrates multi-agent coordination: |
| 30 | + |
| 31 | +- **Service integration**: Books flights and hotels through external travel service APIs. |
| 32 | +- **Cross-agent communication**: Initiates expense claims through the expense agent with appropriate receipts. |
| 33 | +- **End-to-end automation**: Completes the entire travel booking and expense submission workflow without manual intervention. |
| 34 | + |
| 35 | +An example of the multi-agent scenario is shown in the following diagram: |
| 36 | + |
| 37 | +:::image type="content" source="../media/multi-agent.png" alt-text="Diagram of a travel booking agent working with an expenses agent."::: |
20 | 38 |
|
21 | 39 | The diagram shows the following process: |
22 | 40 |
|
23 | | -1. A user provides details of an upcoming trip to a travel booking agent. |
24 | | -1. The travel booking agent automates the booking of flight tickets and hotel reservations. |
25 | | -1. The travel booking agent initiates an expense claim for the travel costs through the expense agent. |
26 | | -1. The expense agent submits the expense claim for processing. |
| 41 | +1. A user provides **details of an upcoming trip** to a travel booking agent. |
| 42 | +1. The travel booking agent **automates the booking** of flight tickets and hotel reservations. |
| 43 | +1. The travel booking agent **initiates an expense claim** for the travel costs through the expense agent. |
| 44 | +1. The expense agent **submits the expense claim** for processing. |
27 | 45 |
|
28 | | -## Understand security risks of AI Agents |
| 46 | +## Understand security risks of AI agents |
29 | 47 |
|
30 | 48 | As AI agents become more autonomous and integrated into enterprise systems, they introduce new security considerations that go beyond traditional application threats. Because agents can access sensitive data, make decisions, and act independently, developers and organizations must design with security in mind from the start. |
31 | 49 |
|
32 | | -The table below summarizes key security risks to consider when developing or deploying AI agents: |
33 | | - |
34 | | -| **Risk Area** | **Description** | **Example / Impact** | |
35 | | -|----------------|-----------------|----------------------| |
36 | | -| **Data Leakage and Privacy Exposure** | AI agents often access sensitive business or user data to perform tasks. Without proper controls, they can unintentionally expose or share confidential information. | An agent summarizing internal files accidentally includes private data in a customer-facing chat. | |
37 | | -| **Prompt Injection and Manipulation Attacks** | Malicious users can craft inputs that override an agent’s intended behavior, tricking it into revealing data or performing unauthorized actions. | A user embeds hidden instructions in a message, causing the agent to leak system credentials. | |
38 | | -| **Unauthorized Access and Privilege Escalation** | Weak authentication or access controls can let agents—or bad actors controlling them—access data or systems they shouldn’t. | An AI agent connected to a CRM tool performs admin-level actions, like exporting or deleting records. | |
39 | | -| **Data Poisoning** | Attackers may corrupt training or contextual data, causing the agent to make biased, incorrect, or unsafe decisions. | A poisoned dataset causes a customer support agent to recommend fraudulent or harmful content. | |
40 | | -| **Supply Chain Vulnerabilities** | Agents often rely on external APIs, plugins, or model endpoints, which expand the attack surface. | A compromised third-party plugin injects malicious code into the agent’s workflow. | |
41 | | -| **Over-Reliance on Autonomous Actions** | Highly autonomous agents may execute unintended actions if not carefully constrained or validated. | An agent mistakenly sends payments or publishes unverified content. | |
42 | | -| **Inadequate Auditability and Logging** | Without detailed logging, it’s difficult to trace actions or detect malicious behavior early. | Security teams cannot identify data misuse due to missing or incomplete activity logs. | |
43 | | -| **Model Inversion and Output Leakage** | Attackers might exploit model outputs to infer sensitive data used during training or prompting. | Repeated queries extract private information that was part of a fine-tuning dataset. | |
44 | | - |
45 | | -### Mitigation Strategies |
46 | | -To reduce these risks, developers should adopt a **security-by-design** approach that includes: |
47 | | - |
48 | | -- Enforcing **role-based access controls (RBAC)** and **least privilege** permissions. |
49 | | -- Adding **prompt filtering and validation** layers to prevent injection attacks. |
50 | | -- Sandboxing or gating sensitive operations behind **human-in-the-loop approvals**. |
51 | | -- Maintaining **comprehensive logging and traceability** for all agent actions. |
52 | | -- Auditing **third-party dependencies** and integrations regularly. |
53 | | -- Continuously retraining and validating models to detect **data drift** or **poisoning attempts**. |
54 | | - |
55 | | -By embedding these practices early in development, organizations can deploy AI agents safely and confidently in real-world environments. |
| 50 | +| **What you might experience** | **Risk area** | **What's happening** | |
| 51 | +|-------------------------------|---------------|----------------------| |
| 52 | +| *"The agent just shared confidential salary data in a customer chat!"* | **Data leakage and privacy exposure** | The agent accessed sensitive information but lacked proper controls to prevent exposing it externally. | |
| 53 | +| *"Someone tricked the agent into revealing our database password."* | **Prompt injection and manipulation attacks** | A malicious user crafted an input that overrode the agent's intended behavior. | |
| 54 | +| *"Our support agent is now deleting customer records—but it shouldn't have that permission!"* | **Unauthorized access and privilege escalation** | Weak access controls allowed the agent to perform actions beyond its intended scope. | |
| 55 | +| *"The agent started recommending fraudulent products after we updated the training data."* | **Data poisoning** | Someone corrupted the agent's training or contextual data, causing unsafe outputs. | |
| 56 | +| *"A third-party plugin we integrated is now sending our data to an unknown server."* | **Supply chain vulnerabilities** | External dependencies introduced security vulnerabilities into the agent's workflow. | |
| 57 | +| *"The agent automatically processed a refund without verifying the request."* | **Over-reliance on autonomous actions** | The agent executed an action without proper validation or human oversight. | |
| 58 | +| *"We can't figure out who accessed what data or when."* | **Inadequate auditability and logging** | Missing or incomplete logs make it impossible to trace agent actions or detect misuse. | |
| 59 | +| *"Someone extracted customer information by repeatedly querying the agent."* | **Model inversion and output leakage** | The attacker exploited model outputs to infer sensitive data from training or prompting. | |
| 60 | + |
| 61 | +### Protect your agents with security best practices |
| 62 | + |
| 63 | +To reduce these risks, adopt a **security-by-design** approach from day one. Here's how to build safer AI agents: |
| 64 | + |
| 65 | +- **Control access tightly**: Enforce **role-based access controls (RBAC)** and **least privilege** permissions—agents should only access what they absolutely need. |
| 66 | + |
| 67 | +- **Validate all inputs**: Add **prompt filtering and validation** layers to catch and block injection attacks before they reach your agent. |
| 68 | + |
| 69 | +- **Add human oversight for critical actions**: Sandbox or gate sensitive operations behind **human-in-the-loop approvals**—don't let agents make high-stakes decisions alone. |
| 70 | + |
| 71 | +- **Track everything**: Maintain **comprehensive logging and traceability** for all agent actions—you need to know who did what, when, and why. |
| 72 | + |
| 73 | +- **Monitor your supply chain**: Audit **third-party dependencies** and integrations regularly—external plugins and APIs can be attack vectors. |
| 74 | + |
| 75 | +- **Keep your models healthy**: Continuously retrain and validate models to detect **data drift** or **poisoning attempts**—agent quality degrades over time without maintenance. |
| 76 | + |
| 77 | +When you embed these practices early in development, you can deploy AI agents safely and confidently in real-world environments. |
0 commit comments