You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: copilot/employee-self-service/usage-analytics.md
+25-11Lines changed: 25 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -23,12 +23,12 @@ Monitoring the usage analytics of the Employee Self-Service agent should be part
23
23
24
24
There are two approaches in consuming analytics:
25
25
26
-
1.Systematically monitoring agent usage, effectiveness, quality, and satisfaction, which fall within the responsibilities of the service owner and/or creator. [Learn more](/microsoft-copilot-studio/analytics-overview) about Copilot Studio analytics documentation.
27
-
1.Review agent usage, satisfaction scores, and other metrics to assess the agent's return on investment. [Learn more](/viva/insights/org-team-insights/copilot-dashboard) about Copilot.
26
+
1.Service owners or makers systematically monitoring agent usage, effectiveness, quality, and satisfaction. [Learn more](/microsoft-copilot-studio/analytics-overview) about Copilot Studio analytics documentation.
27
+
1.Organization leaders review agent usage, satisfaction scores, and other metrics to assess the agent's return on investment. [Learn more](/viva/insights/org-team-insights/copilot-dashboard) about Copilot.
28
28
29
29
## Measure what matters
30
30
31
-
Employee Self-Service atelemetry is designed to help organizations move beyond basic usage reporting and toward **operational clarity, trust, and continuous improvement**. While Employee Self-Service collects a single, consistent telemetry stream, **different stakeholders interpret that telemetry through different lenses**, depending on the decisions they're responsible for making.
31
+
Employee Self-Service telemetry is designed to help organizations move beyond basic usage reporting and toward **operational clarity, trust, and continuous improvement**. While Employee Self-Service collects a single, consistent telemetry stream, **different stakeholders interpret that telemetry through different lenses**, depending on the decisions they're responsible for making.
32
32
33
33
This article explains how to interpret Employee Self-Service telemetry for each stakeholder in your organization, using the same storytelling model that Microsoft's product and engineering teams use internally. This information allows customers and product teams to align on what "good" looks like and what actions to take next.
34
34
@@ -45,7 +45,7 @@ This model ensures telemetry is decision-oriented as well as descriptive. The sa
45
45
|**Intent**|What employees are trying to do. |
46
46
|**Behavior**|How the Employee Self-Service agent is used. |
47
47
|**Outcome**|Business and experience impact. |
48
-
|**Action**|What should change as a result.|
48
+
|**Action**|What should change as a result?|
49
49
50
50
## Stakeholder-based interpretation guide
51
51
@@ -79,11 +79,11 @@ Executives should focus on **outcome-level telemetry**, not raw interaction coun
79
79
- Sustained adoption beyond pilot phases (trust and repeat usage)
80
80
- Predictable scale without increasing support or incident load
[Copilot Studio - Analytics](/microsoft-copilot-studio/analytics-overview)
122
+
117
123
#### Interpreting Telemetry by verticals
118
124
This section is designed to help you operationalize the Employee Self-Service telemetry across your organization by anchoring analytics to real scenarios, clear stakeholder questions, and concrete actions.
119
125
@@ -125,7 +131,7 @@ This section is designed to help you operationalize the Employee Self-Service te
- Focus telemetry reviews on top ticket-deflecting scenarios
185
191
- Use diagnostics to distinguish config issues vs. product gaps
186
192
- Validate performance telemetry before scaling rollout
187
-
- Pair telemetry with readiness and ALM checks for production moves
193
+
- Pair telemetry with readiness and Application Lifecycle Management (ALM) checks for production moves
188
194
189
195
***Facilities & Workplace services***
190
196
**Typical Workplace scenarios**
@@ -220,7 +226,7 @@ This section is designed to help you operationalize the Employee Self-Service te
220
226
- Use telemetry to justify integration investments
221
227
- Track sentiment trends as a proxy for workplace trust
222
228
223
-
### IT Administrators & Makers ###
229
+
### 3. IT Administrators & Makers ###
224
230
**Primary question**
225
231
226
232
*Is the Employee Self-Service agent configured correctly, stable, and ready to scale?*
@@ -244,6 +250,12 @@ Administrators and Makers should interpret telemetry as **health and readiness i
244
250
- Validates environments before promoting to production
245
251
- Coordinate changes using ALM and readiness checks
246
252
253
+
**What to use When**
254
+
255
+
[Copilot Studio - Analytics](/microsoft-copilot-studio/analytics-overview)
256
+
257
+
[Capture telemetry with Application Insights](/microsoft-copilot-studio/advanced-bot-framework-composer-capture-telemetry)
258
+
247
259
## Employee Self-Service analytics and evaluations: A unified playbook to accelerate time-to-value (TTV)
248
260
249
261
### Why combine telemetry and evaluations?
@@ -263,3 +275,5 @@ The shared operating model:
263
275
**'Intent ➡️ Behavior ➡️ Outcome ➡️ Action'**
264
276
265
277
Evals plug into the same model by providing repeatable evidence about whether the agent can reliably deliver the intended outcomes before you expose the change to broad employee usage.
0 commit comments