You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[SRE Agent] Update agent features and capabilities documentation
Update documentation for agent playground, hooks, reasoning, sub-agents,
test tool playground, deep investigation, workflow automation, memory,
skills, and run modes.
Co-authored-by: Copilot <[email protected]>
description: Intercept and control agent behavior with custom scripts or LLM-based validation that runs before or after specific agent actions.
4
4
ms.topic: conceptual
5
5
ms.service: azure-sre-agent
6
-
ms.date: 03/09/2026
6
+
ms.date: 03/18/2026
7
7
author: craigshoemaker
8
8
ms.author: cshoe
9
9
ms.ai-usage: ai-assisted
@@ -13,7 +13,7 @@ ms.custom: hooks, agent hooks, stop hook, post tool use, validation, audit, poli
13
13
14
14
# Agent hooks in Azure SRE Agent
15
15
16
-
Hooks are custom checkpoints that intercept and control agent behavior at key moments. Use hooks to enforce quality gates on agent responses, audit and control tool usage, block dangerous operations with policy enforcement, and prevent early task completion by validating agent output.
16
+
Hooks are custom checkpoints that intercept and control agent behavior at key moments. Use hooks to enforce quality gates on agent responses, audit and control tool usage, block dangerous operations by enforcing policies, and prevent early task completion by validating agent output.
|**PostToolUse**| A tool finishes executing successfully | Audit usage, block results, inject extra context |
48
+
49
+
### Two levels of hooks
50
+
51
+
Hooks operate at two levels:
52
+
53
+
| Level | Where to configure | Scope |
54
+
|-------|--------------------|-------|
55
+
|**Agent level**|**Builder → Hooks** in the portal | Applies to the entire agent including all threads and all custom agents |
56
+
|**Custom agent level**|**Agent Canvas → Custom agent → Manage Hooks**, or via the REST API v2 | Applies only when that specific custom agent runs |
57
+
58
+
Both levels can coexist. If an agent-level hook and a custom-agent-level hook both match the same event, **both run**. The agent-level hooks fire first.
48
59
49
60
### Execution types
50
61
@@ -55,11 +66,11 @@ You can implement hooks by using either an LLM or a shell script:
55
66
|**Prompt**| An LLM evaluates your prompt and returns a JSON decision | Nuanced validation ("Is this response complete?") |
56
67
|**Command**| A bash or Python script runs in a sandboxed environment | Deterministic checks, policy enforcement, auditing |
57
68
58
-
**Prompt hooks** are powerful for subjective evaluation, such as checking if a response addresses all user concerns or verifying that an investigation was thorough enough. They use the `$ARGUMENTS` placeholder to receive the full hook context. If `$ARGUMENTS` isn't present in the prompt, the context is appended automatically. Prompt hooks also receive `ReadFile` and `GrepSearch` tools when a conversation transcript is available, which lets the LLM reason about the full conversation history.
69
+
**Prompt hooks** are powerful for subjective evaluation, such as checking if a response addresses all user concerns or verifying that an investigation was thorough enough. They use the `$ARGUMENTS` placeholder to receive the full hook context. If `$ARGUMENTS` isn't present in the prompt, the context is appended automatically. When a conversation transcript is available, prompt hooks also receive `ReadFile` and `GrepSearch` tools, which let the LLM reason about the full conversation history.
59
70
60
71
**Command hooks** are better for deterministic checks, such as validating that a response contains required markers, blocking dangerous commands, or logging tool usage to an external system.
61
72
62
-
## What makes this different
73
+
## What makes this approach different
63
74
64
75
The following table compares agent behavior with and without hooks.
65
76
@@ -83,21 +94,15 @@ Hooks don't replace run mode safety controls - they complement them. Run modes c
83
94
84
95
## Configure hooks
85
96
86
-
Hooks require the **v2 YAML format** (`api_version: azuresre.ai/v2`, `kind: ExtendedAgent`). Define hooks under `spec.hooks` in the agent configuration.
97
+
The easiest way to create hooks is through the portal UI:
87
98
88
-
> [!WARNING]
89
-
> The portal's Subagent builder YAML tab displays agent configuration in **v1 format only**. It doesn't show or support editing hooks. To configure hooks, use the **REST API v2** endpoint:
90
-
>
91
-
> ```
92
-
> PUT /api/v2/extendedAgent/agents/{agentName}
93
-
> Content-Type: application/json
94
-
> ```
95
-
>
96
-
> Hooks configured through the API are active even though they don't appear in the portal YAML view. You can verify hooks are working by testing the agent in the portal's **Test playground**.
97
-
>
98
-
> :::image type="content" source="media/agent-hooks/hooks-portal-v1-limitation.png" alt-text="Screenshot of the portal YAML tab showing v1 format without hooks." lightbox="media/agent-hooks/hooks-portal-v1-limitation.png":::
99
+
1.**Agent-level hooks:** Go to **Builder** → **Hooks** → select **Create hook**.
100
+
2.**Custom-agent-level hooks:** Go to **Agent Canvas** → select a custom agent → **Manage Hooks**.
101
+
102
+
> [!TIP]
103
+
> You can also configure hooks through **REST API v2** by using `PUT /api/v2/extendedAgent/agents/{agentName}`. The YAML format in the following section shows the full configuration schema. To learn more, see the [API tutorial](tutorial-agent-hooks.md).
99
104
>
100
-
> The portal YAML tab displays v1 format. Hooks aren't visible here but are active on the server.
105
+
> The **Agent Canvas YAML** tab displays v1 format and doesn't show hooks. Use the **Hooks** page under **Builder** to view and manage hooks.
101
106
102
107
The following example shows a complete hook configuration:
103
108
@@ -168,7 +173,7 @@ Command hooks can also use exit codes instead of JSON output:
168
173
|---|---|
169
174
|`0` with no output | Allow (no objection) |
170
175
|`0` with JSON | Parse JSON for decision |
171
-
|`2`| Always block — stderr becomes the reason |
176
+
|`2`| Always block. stderr becomes the reason |
172
177
| Other | Uses `failMode` setting (`allow` or `block`) |
173
178
174
179
> [!CAUTION]
@@ -252,7 +257,7 @@ The following limits apply to agent hooks.
252
257
253
258
## Example: Audit all tool usage
254
259
255
-
The following PostToolUse hook logs every tool call and injects an audit context message:
260
+
The following PostToolUse hook logs every tool call and adds an audit context message:
256
261
257
262
```yaml
258
263
hooks:
@@ -279,7 +284,7 @@ hooks:
279
284
print(json.dumps(output))
280
285
```
281
286
282
-
The `additionalContext` field is injected as a user message into the conversation, giving the agent visibility into the audit trail.
287
+
The `additionalContext` field is added as a user message into the conversation, giving the agent visibility into the audit trail.
283
288
284
289
## Example: Require a completion marker
285
290
@@ -308,25 +313,28 @@ hooks:
308
313
309
314
Follow these guidelines when you configure agent hooks:
310
315
311
-
1. **Always provide a reason when rejecting** — Rejections without reasons are treated as approvals.
1. **Be specific with matchers**:Overly broad PostToolUse matchers can cause performance problems.
320
+
1. **Test hooks thoroughly**:Hooks that always reject can cause loops (mitigated by `maxRejections`).
321
+
1. **Log to stderr**: Use stderr for debugging output. The system parses stdout as the hook result.
317
322
318
323
## Try it yourself
319
324
320
325
The following screenshot shows a Stop hook in action. The agent initially responds with just "4", but the hook rejects the response because the completion marker is missing. The agent then continues and adds the marker.
321
326
322
327
:::image type="content" source="media/agent-hooks/hooks-stop-hook-working.png" alt-text="Screenshot showing a Stop hook in action where the agent response is decorated with a completion marker after hook rejection." lightbox="media/agent-hooks/hooks-stop-hook-working.png":::
#customer intent: As an SRE, I want to test my agent configurations in an isolated playground so that I can iterate quickly without affecting production workflows.
12
12
---
13
13
14
14
# Agent playground in Azure SRE Agent
15
-
Test subagent behavior in real time before deploying changes. Edit instructions, tools, and handoffs with instant feedback in a split-screen layout. Evaluate agent quality with AI-powered scoring and quick fixes.
15
+
Test custom agent behavior in real time before deploying changes. Edit instructions, tools, and handoffs with instant feedback in a split-screen layout. Evaluate agent quality with AI-powered scoring and quick fixes.
16
16
17
17
## The problem
18
18
@@ -22,41 +22,41 @@ Without a dedicated testing environment, you deploy changes to see how they beha
22
22
23
23
## How the playground works
24
24
25
-
The playground is a dedicated view in the **Subagent builder** alongside Canvas and Table views. Select **Test playground** from the view toggle to enter a split-screen environment where you edit on the left and test on the right.
25
+
The playground is a dedicated view in the **Agent Canvas** alongside Canvas and Table views. Select **Test playground** from the view toggle to enter a split-screen environment where you edit and test.
26
26
27
27
:::image type="content" source="media/common/playground-agent-selected.png" alt-text="Screenshot of agent playground showing split-screen layout with form editor on left and chat test panel on right.":::
28
28
29
29
### Select what to test
30
30
31
-
Use the **Subagent/Tool** dropdown at the top to choose what to test.
31
+
Use the **Custom agent/Tool** dropdown at the top to choose what to test.
32
32
33
33
| Entity | What you can test |
34
34
|---|---|
35
-
|**Subagent**| Instructions, tools, handoffs, and memory in a live chat |
36
-
|**Main agent (meta_agent)**| Override the orchestrator prompt and test routing behavior |
35
+
|**Custom agent**| Instructions, tools, handoffs, and memory in a live chat |
36
+
|**Your agent**| Override the orchestrator prompt and test routing behavior |
37
37
|**System tool**| Execute built-in tools with custom parameters |
38
38
|**Kusto tool**| Run queries against your connected clusters |
39
39
40
-
:::image type="content" source="media/common/playground-entity-selector.png" alt-text="Screenshot of entity selector dropdown showing subagents and tools available for testing.":::
40
+
:::image type="content" source="media/common/playground-entity-selector.png" alt-text="Screenshot of entity selector dropdown showing custom agents and tools available for testing.":::
41
41
42
42
### Edit and test side by side
43
43
44
-
For subagents, the playground splits into two panels.
44
+
For custom agents, the playground splits into two panels.
45
45
46
-
**Left panel — Editor:**
46
+
**Editor:**
47
47
48
-
-**Form view** — Edit subagent name, instructions, handoff instructions, handoff subagents, tools, and knowledge base access.
49
-
-**YAML view** — Edit the full agent configuration as YAML.
48
+
-**Form view**: Edit custom agent name, instructions, handoff instructions, handoff custom agents, tools, and knowledge base access.
49
+
-**YAML view**: Edit the full agent configuration as YAML.
50
50
51
-
**Right panel — Testing:**
51
+
**Testing:**
52
52
53
-
-**Test tab** — Chat with your agent by using the current configuration.
54
-
-**Evaluation tab** — Run AI-powered quality analysis.
53
+
-**Test tab**: Chat with your agent by using the current configuration.
54
+
-**Evaluation tab**: Run AI-powered quality analysis.
55
55
56
56
> [!NOTE]
57
57
> When you modify the configuration, chat input is disabled until you select **Apply** to save your changes or **Discard** to revert. This behavior prevents testing stale configurations. Selecting **Apply** also starts a fresh chat thread so you can test the updated configuration from scratch.
58
58
59
-
## What makes this different
59
+
## What makes this approach different
60
60
61
61
Unlike testing in live conversations, the playground provides an isolated environment where changes don't affect production threads. The split-screen layout means you see the effect of instruction changes immediately without switching between views or waiting for deployments.
62
62
@@ -82,47 +82,50 @@ The evaluation returns the following scores:
82
82
|**Completeness**| Whether the prompt covers role, goal, and operational guidance |
83
83
|**Tool fit**| Whether the right tools are configured |
84
84
|**Prompt clarity**| How clear and actionable the instructions are |
85
+
|**Actionability**| Whether responses include concrete, executable next steps |
85
86
|**Safety**| Error handling, confirmation prompts, and safeguards |
86
87
87
88
### Quick fixes
88
89
89
-
When evaluation identifies improvements, select **Review and apply** to open the quick fixes dialog. Select the fixes you want, preview the YAML diff on the right, and then use the **Accept selected fixes** button. You can choose to continue editing or save immediately.
90
+
When evaluation identifies improvements, select **Review and apply** to open the quick fixes dialog. Select the fixes you want, preview the YAML diff, and then use the **Accept selected fixes** button. You can choose to continue editing or save immediately.
90
91
91
92
> [!TIP]
92
93
> Run evaluation after a few test conversations. The evaluation considers chat behavior alongside your configuration to provide more accurate scoring.
93
94
94
95
> [!NOTE]
95
-
> If you change the agent configuration after running an evaluation, the results are marked as **outdated** and you're prompted to re-evaluate. Similarly, new chat activity after an evaluation marks results as **stale**. Re-evaluate to get insights that reflect your latest testing.
96
+
> If you change the agent configuration after running an evaluation, the results are marked as **outdated** and you're prompted to reevaluate. Similarly, new chat activity after an evaluation marks results as **stale**. Reevaluate to get insights that reflect your latest testing.
96
97
97
98
## Test tools in isolation
98
99
99
100
You can test system tools and Kusto tools independently from the agent playground.
100
101
101
102
### System tools
102
103
103
-
Select a system tool from the **Subagent/Tool** dropdown to test built-in capabilities independently. Enter parameter values and select **Execute Tool** to see the raw JSON output.
104
+
Select a system tool from the **Custom agent/Tool** dropdown to test built-in capabilities independently. Enter parameter values and select **Execute Tool** to see the raw JSON output.
104
105
105
106
### Kusto tools
106
107
107
-
Select a Kusto tool to test your query against connected clusters. The test panel shows query results with row counts, columns, and execution time. Adjust your KQL on the left and rerun on the right.
108
+
Select a Kusto tool to test your query against connected clusters. The test panel shows query results with row counts, columns, and execution time. Adjust your KQL and rerun.
108
109
109
110
For step-by-step instructions, see [Test a tool in the playground](test-tool-playground.md).
110
111
111
112
## AI-assisted configuration
112
113
113
-
The playground includes two AI assistance features for refining subagent instructions:
114
+
The playground includes two AI assistance features for refining custom agent instructions:
114
115
115
116
-**Refine with AI**: Rewrites your instructions and handoff description in place. This feature directly replaces your current text with an AI-improved version, so review the changes before saving.
116
117
-**View AI suggestions**: Opens a read-only panel alongside the form showing AI recommendations: suggestions for improvement, warnings about potential problems, and improved versions of your instructions and handoff description. This feature doesn't modify your configuration. Use it as a reference while editing.
117
118
118
-
## Next step
119
+
## Get started
119
120
120
-
> [!div class="nextstepaction"]
121
-
> [Test a tool in the playground](./test-tool-playground.md)
121
+
| Resource | What you learn |
122
+
|---|---|
123
+
|[Test a tool in the playground](test-tool-playground.md)| Step-by-step walkthrough of the playground interface |
0 commit comments