You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Run KQL queries on the Microsoft Sentinel data lake using APIs
15
15
16
-
As security data lakes become the backbone of modern analytics platforms, organizations need flexible programmatic access that enables automation, scale, and seamless integration. While interactive tools and portals support data exploration in the Defender portal, many real-world workflows require API-based query execution.
16
+
Microsoft Sentinel data lake supports running Kusto Query Language (KQL) queries programmatically by using REST APIs. This enables security teams and automation systems to retrieve analytical results without using the Azure portal or interactive query editors.
17
+
This article explains when to use the API, required permissions, and how to submit a basic query request.
17
18
18
-
By running KQL (Kusto Query Language) queries on the Microsoft Sentinel data lake through APIs, you can embed analytics directly into automation workflows, background services, and intelligent agents without relying on manual query execution.
19
+
## When to use the KQL query API
19
20
20
-
This article explains how to run KQL queries against the Microsoft Sentinel data lake using REST APIs, including authentication requirements, sample code, and key considerations.
21
+
The KQL query API is designed for system-to-system access scenarios, including:
22
+
- Automation and orchestration workflows
23
+
- Background services and scheduled jobs
24
+
- Agent and security tools that require query results as input
25
+
- Integration with external systems or agents
21
26
22
-
## Why run KQL queries via API?
27
+
For interactive investigation and ad-hoc analysis, run KQL queries from the Defender portal instead.
23
28
24
-
Traditional query experiences, such as dashboards and query editors, are optimized for human interaction. APIs, on the other hand, are optimized for systems.
25
29
26
-
Running KQL through an API enables:
27
30
28
-
-**Automation-first analytics** - Embed analytics into automated workflows and playbooks.
29
-
-**Repeatable and scheduled insights** - Run queries on a schedule without manual intervention.
30
-
-**Integration with external systems and agents** - Connect analytics to external applications, AI agents, and custom tooling.
31
-
-**Consistent query execution at scale** - Execute queries consistently across multiple environments.
31
+
## Authentication and permissions
32
32
33
-
## Common scenarios
33
+
You can authenticate to the Sentinel data lake API by using either:
34
+
- A service principal
35
+
- A user access token
34
36
35
-
The following scenarios demonstrate where API-based KQL queries add the most value.
37
+
> [!NOTE]
38
+
> Using a service principal currently Entra ID roles and Microsoft Defender XDR unified RBAC roles aren't supported for querying the Sentinel data lake through this API.
36
39
37
-
### Automated monitoring and alerting
40
+
##Calling the API
38
41
39
-
SOC teams often want to continuously analyze data in their lake to detect anomalies, trends, or policy violations.
42
+
**API endpoint**
40
43
41
-
With API-based KQL execution, you can:
44
+
All KQL queries are submitted by using the following REST endpoint:
42
45
43
-
- Run queries as part of automated workflows and playbooks.
44
-
- Evaluate query results programmatically.
45
-
- Trigger downstream actions such as alerts, tickets, or notifications.
This approach turns KQL into a signal engine, not just an exploration tool.
48
+
**Request body format**
48
49
49
-
### Powering intelligent agents
50
+
A query request consists of:
50
51
51
-
AI agents require programmatic access to data lakes to retrieve timely, relevant context for decision making. Using KQL over an API allows agents to:
52
+
- The KQL query
53
+
- The target workspace, specified as workspaceName-workspaceId
52
54
53
-
- Dynamically query the data lake based on user intent or system context.
54
-
- Retrieve aggregated or filtered results on demand.
55
-
- Combine analytical results with reasoning and decision logic.
55
+
**Payload parameters**
56
56
57
-
In this model, KQL acts as the analytical retrieval layer, while the agent focuses on orchestration, reasoning, and action.
57
+
|Field|Description|
58
+
|-----|-----------|
59
+
|csl|The KQL query to execute|
60
+
|db|The Sentinel workspace name and workspace ID|
58
61
59
-
### Embedding analytics into business workflows
62
+
Sample payload:
60
63
61
-
Organizations often want analytics embedded directly into CI/CD and operational pipelines. Instead of exporting data or duplicating logic, you can:
62
-
63
-
- Run KQL queries inline via API.
64
-
- Use results as inputs to other systems.
65
-
- Keep analytics logic centralized and consistent.
66
-
67
-
This approach reduces drift between analytics code and application code.
68
-
69
-
## Prerequisites
70
-
71
-
To run KQL queries against the Microsoft Sentinel data lake using APIs, you need:
72
-
73
-
-**A Microsoft Sentinel workspace onboarded to the data lake.** For more information, see [Onboarding to Microsoft Sentinel data lake](sentinel-lake-onboarding.md).
74
-
-**A user token or a service principal.** For more information on creating a service principal, see [Create a Microsoft Entra application and service principal that can access resources](/entra/identity-platform/howto-create-service-principal-portal).
75
-
-**Appropriate permissions** to execute queries on the Microsoft Sentinel data lake. Azure RBAC roles such as **Log Analytics Reader** or **Log Analytics Contributor** on the workspace are required.
76
-
-**Familiarity with KQL** and API-based query execution patterns. For more information on KQL syntax, see [Kusto Query Language (KQL) overview](/azure/data-explorer/kusto/query/).
77
-
78
-
## High-level flow
79
-
80
-
When you run a KQL query via API, the following steps occur:
81
-
82
-
1. A client authenticates to the Microsoft Sentinel data lake platform.
83
-
1. The client submits a KQL query via the REST API.
84
-
1. The query executes against data stored in the data lake.
85
-
1. Results are returned in a structured, machine-readable format.
86
-
1. The client processes or acts on the results.
87
-
88
-
## Authentication
89
-
90
-
To authenticate to the Microsoft Sentinel data lake API, you need to acquire a bearer token from Microsoft Entra ID. Use the Microsoft Authentication Library (MSAL) to acquire either a v1.0 or v2.0 access token. For more information on acquiring access tokens, see [Acquire and cache tokens with Microsoft Authentication Library](/entra/identity-platform/msal-acquire-cache-tokens).
91
-
92
-
### Create a service principal
93
-
94
-
If you're using a service principal for authentication, you need to register an application in Microsoft Entra ID:
95
-
96
-
1. Register the application with Microsoft Entra ID and create a service principal. For more information, see [Create a Microsoft Entra application and service principal that can access resources](/entra/identity-platform/howto-create-service-principal-portal).
97
-
98
-
1. Save the following values from your app registration:
99
-
-**Tenant ID** - Your Microsoft Entra tenant ID.
100
-
-**Client ID** - The application (client) ID of your registered application.
101
-
-**Client secret** - The secret created for your application.
102
-
103
-
1. Assign the appropriate Azure RBAC role to the service principal to query the Microsoft Sentinel data lake. For more information, see [Assign Azure roles using the Azure portal](/azure/role-based-access-control/role-assignments-portal).
104
-
105
-
### Acquire an access token
106
-
107
-
Use MSAL to acquire an access token for the Sentinel data lake API. The scope for the data lake API is `4500ebfb-89b6-4b14-a480-7f749797bfcd/.default`.
108
-
109
-
For more information on acquiring tokens using MSAL, see [Microsoft Authentication Library (MSAL) overview](/entra/identity-platform/msal-overview).
110
-
111
-
## Run a KQL query using Python
112
-
113
-
The following Python example demonstrates how to authenticate using a service principal and execute a KQL query on the Sentinel data lake using the REST API.
Consider the following points when planning to execute KQL queries on the data lake via API.
259
-
260
-
### Service principal permissions
261
-
262
-
When using a service principal, Azure RBAC roles can be assigned at the Sentinel workspace level. Entra ID roles or XDR unified RBAC roles aren't supported for this scenario. Alternatively, use user tokens with Entra ID roles.
263
-
264
-
### Latency and cost considerations
265
-
266
-
The data lake is optimized for cost-effective storage, and queries might be slower than queries executed in the analytics tier. Design frequent or automated queries carefully, considering the query and concurrency limits.
267
-
268
-
### Result size limits
84
+
You can include additional execution options in the request payload, such as:
269
85
270
-
Queries are subject to limits on execution time and response size. Review the [Microsoft Sentinel data lake service limits](sentinel-lake-service-limits.md) when designing your workflows.
86
+
- Server timeout
87
+
- Query consistency
88
+
- Read-only enforcement
271
89
272
-
### Query limits
90
+
These options are useful when running queries in automated or high-scale environments.
91
+
92
+
## Service limits and considerations
273
93
274
-
For information on query execution limits, concurrency, and timeout values, see [Microsoft Sentinel data lake service limits](sentinel-lake-service-limits.md).
94
+
Query execution is subject to time and result size limits. For current limits, see: [Microsoft Sentinel data lake service limits](sentinel-lake-service-limits.md)
0 commit comments