Skip to content

Commit e82e7d9

Browse files
EdB-MSFTEdB-MSFT
committed
KQL API
1 parent 7435b02 commit e82e7d9

2 files changed

Lines changed: 287 additions & 0 deletions

File tree

‎articles/sentinel/TOC.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -651,6 +651,9 @@
651651
- name: Run KQL queries
652652
href: datalake/kql-queries.md
653653
displayName: data lake
654+
- name: KQL using the API
655+
href: datalake/kql-queries-api.md
656+
displayName: data lake
654657
- name: Sample KQL queries
655658
href: datalake/kql-sample-queries.md
656659
displayName: data lake KQL

‎articles/sentinel/datalake/kql-queries-api.md‎

Lines changed: 284 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,284 @@
1+
---
2+
title: Run KQL queries on Microsoft Sentinel data lake using APIs
3+
titleSuffix: Microsoft Security
4+
description: Learn how to run KQL queries against the Microsoft Sentinel data lake programmatically using REST APIs. Enable automation, intelligent agents, and scalable analytics.
5+
author: EdB-MSFT
6+
ms.service: microsoft-sentinel
7+
ms.subservice: sentinel-platform
8+
ms.topic: how-to
9+
ms.date: 03/17/2026
10+
ms.author: edbaynash
11+
ms.collection: ms-security
12+
---
13+
14+
# Run KQL queries on the Microsoft Sentinel data lake using APIs
15+
16+
As security data lakes become the backbone of modern analytics platforms, organizations need flexible programmatic access that enables automation, scale, and seamless integration. While interactive tools and portals support data exploration in the Defender portal, many real-world workflows require API-based query execution.
17+
18+
By running KQL (Kusto Query Language) queries on the Microsoft Sentinel data lake through APIs, you can embed analytics directly into automation workflows, background services, and intelligent agents without relying on manual query execution.
19+
20+
This article explains how to run KQL queries against the Microsoft Sentinel data lake using REST APIs, including authentication requirements, sample code, and key considerations.
21+
22+
## Why run KQL queries via API?
23+
24+
Traditional query experiences, such as dashboards and query editors, are optimized for human interaction. APIs, on the other hand, are optimized for systems.
25+
26+
Running KQL through an API enables:
27+
28+
- **Automation-first analytics** - Embed analytics into automated workflows and playbooks.
29+
- **Repeatable and scheduled insights** - Run queries on a schedule without manual intervention.
30+
- **Integration with external systems and agents** - Connect analytics to external applications, AI agents, and custom tooling.
31+
- **Consistent query execution at scale** - Execute queries consistently across multiple environments.
32+
33+
## Common scenarios
34+
35+
The following scenarios demonstrate where API-based KQL queries add the most value.
36+
37+
### Automated monitoring and alerting
38+
39+
SOC teams often want to continuously analyze data in their lake to detect anomalies, trends, or policy violations.
40+
41+
With API-based KQL execution, you can:
42+
43+
- Run queries as part of automated workflows and playbooks.
44+
- Evaluate query results programmatically.
45+
- Trigger downstream actions such as alerts, tickets, or notifications.
46+
47+
This approach turns KQL into a signal engine, not just an exploration tool.
48+
49+
### Powering intelligent agents
50+
51+
AI agents require programmatic access to data lakes to retrieve timely, relevant context for decision making. Using KQL over an API allows agents to:
52+
53+
- Dynamically query the data lake based on user intent or system context.
54+
- Retrieve aggregated or filtered results on demand.
55+
- Combine analytical results with reasoning and decision logic.
56+
57+
In this model, KQL acts as the analytical retrieval layer, while the agent focuses on orchestration, reasoning, and action.
58+
59+
### Embedding analytics into business workflows
60+
61+
Organizations often want analytics embedded directly into CI/CD and operational pipelines. Instead of exporting data or duplicating logic, you can:
62+
63+
- Run KQL queries inline via API.
64+
- Use results as inputs to other systems.
65+
- Keep analytics logic centralized and consistent.
66+
67+
This approach reduces drift between analytics code and application code.
68+
69+
## Prerequisites
70+
71+
To run KQL queries against the Microsoft Sentinel data lake using APIs, you need:
72+
73+
- **A Microsoft Sentinel workspace onboarded to the data lake.** For more information, see [Onboarding to Microsoft Sentinel data lake](sentinel-lake-onboarding.md).
74+
- **A user token or a service principal.** For more information on creating a service principal, see [Create a Microsoft Entra application and service principal that can access resources](/entra/identity-platform/howto-create-service-principal-portal).
75+
- **Appropriate permissions** to execute queries on the Microsoft Sentinel data lake. Azure RBAC roles such as **Log Analytics Reader** or **Log Analytics Contributor** on the workspace are required.
76+
- **Familiarity with KQL** and API-based query execution patterns. For more information on KQL syntax, see [Kusto Query Language (KQL) overview](/azure/data-explorer/kusto/query/).
77+
78+
## High-level flow
79+
80+
When you run a KQL query via API, the following steps occur:
81+
82+
1. A client authenticates to the Microsoft Sentinel data lake platform.
83+
1. The client submits a KQL query via the REST API.
84+
1. The query executes against data stored in the data lake.
85+
1. Results are returned in a structured, machine-readable format.
86+
1. The client processes or acts on the results.
87+
88+
## Authentication
89+
90+
To authenticate to the Microsoft Sentinel data lake API, you need to acquire a bearer token from Microsoft Entra ID. Use the Microsoft Authentication Library (MSAL) to acquire either a v1.0 or v2.0 access token. For more information on acquiring access tokens, see [Acquire and cache tokens with Microsoft Authentication Library](/entra/identity-platform/msal-acquire-cache-tokens).
91+
92+
### Create a service principal
93+
94+
If you're using a service principal for authentication, you need to register an application in Microsoft Entra ID:
95+
96+
1. Register the application with Microsoft Entra ID and create a service principal. For more information, see [Create a Microsoft Entra application and service principal that can access resources](/entra/identity-platform/howto-create-service-principal-portal).
97+
98+
1. Save the following values from your app registration:
99+
- **Tenant ID** - Your Microsoft Entra tenant ID.
100+
- **Client ID** - The application (client) ID of your registered application.
101+
- **Client secret** - The secret created for your application.
102+
103+
1. Assign the appropriate Azure RBAC role to the service principal to query the Microsoft Sentinel data lake. For more information, see [Assign Azure roles using the Azure portal](/azure/role-based-access-control/role-assignments-portal).
104+
105+
### Acquire an access token
106+
107+
Use MSAL to acquire an access token for the Sentinel data lake API. The scope for the data lake API is `4500ebfb-89b6-4b14-a480-7f749797bfcd/.default`.
108+
109+
For more information on acquiring tokens using MSAL, see [Microsoft Authentication Library (MSAL) overview](/entra/identity-platform/msal-overview).
110+
111+
## Run a KQL query using Python
112+
113+
The following Python example demonstrates how to authenticate using a service principal and execute a KQL query on the Sentinel data lake using the REST API.
114+
115+
```python
116+
import requests
117+
import msal
118+
119+
# Service principal / Entra app settings
120+
TENANT_ID = "<your-tenant-id>"
121+
CLIENT_ID = "<your-client-id>"
122+
CLIENT_SECRET = "<your-client-secret>"
123+
124+
# Token authority
125+
AUTHORITY = f"https://login.microsoftonline.com/{TENANT_ID}"
126+
127+
# API scope for Sentinel data lake
128+
SCOPE = ["4500ebfb-89b6-4b14-a480-7f749797bfcd/.default"]
129+
130+
# KQL query payload
131+
KQL_QUERY = {
132+
"csl": "SigninLogs | take 10",
133+
"db": "<workspace-name>-<workspace-id>",
134+
"properties": {
135+
"Options": {
136+
"servertimeout": "00:04:00",
137+
"queryconsistency": "strongconsistency",
138+
"query_language": "kql",
139+
"request_readonly": False,
140+
"request_readonly_hardline": False
141+
}
142+
}
143+
}
144+
145+
# Acquire token using client credentials
146+
app = msal.ConfidentialClientApplication(
147+
client_id=CLIENT_ID,
148+
authority=AUTHORITY,
149+
client_credential=CLIENT_SECRET
150+
)
151+
152+
result = app.acquire_token_for_client(scopes=SCOPE)
153+
154+
if "access_token" not in result:
155+
raise RuntimeError(
156+
f"Token acquisition failed: {result.get('error')} - {result.get('error_description')}"
157+
)
158+
159+
access_token = result["access_token"]
160+
161+
# Call the KQL API
162+
headers = {
163+
"Authorization": f"Bearer {access_token}",
164+
"Content-Type": "application/json"
165+
}
166+
167+
url = "https://api.securityplatform.microsoft.com/lake/kql/v2/rest/query"
168+
response = requests.post(url, headers=headers, json=KQL_QUERY)
169+
170+
if response.status_code == 200:
171+
print("Query Results:")
172+
print(response.json())
173+
else:
174+
print(f"Error {response.status_code}: {response.text}")
175+
```
176+
177+
Replace the following placeholders:
178+
179+
| Placeholder | Description |
180+
|---|---|
181+
| `<your-tenant-id>` | Your Microsoft Entra tenant ID. |
182+
| `<your-client-id>` | The application (client) ID from your app registration. |
183+
| `<your-client-secret>` | The client secret from your app registration. |
184+
| `<workspace-name>-<workspace-id>` | Your workspace name and ID in the format `workspace1-12345678-abcd-abcd-1234-1234567890ab`. |
185+
186+
## Run a KQL query using an HTTP client
187+
188+
You can call the API directly using any HTTP client. The following example shows the REST API request format.
189+
190+
### Request
191+
192+
```http
193+
POST https://api.securityplatform.microsoft.com/lake/kql/v2/rest/query
194+
Authorization: Bearer <access_token>
195+
Content-Type: application/json
196+
197+
{
198+
"db": "default",
199+
"csl": "EntraUsers | take 10"
200+
}
201+
```
202+
203+
### Request parameters
204+
205+
| Parameter | Value |
206+
|---|---|
207+
| **Request URI** | `https://api.securityplatform.microsoft.com/lake/kql/v2/rest/query` |
208+
| **Method** | POST |
209+
| **Authorization header** | `Bearer <access_token>` |
210+
| **Content-Type** | `application/json` |
211+
212+
### Request payload
213+
214+
| Property | Description |
215+
|---|---|
216+
| `db` | The database identifier. Use `default` for the default workspace, or specify the workspace in the format `<workspace-name>-<workspace-id>`. |
217+
| `csl` | The KQL query to execute. |
218+
| `properties` | Optional. Query execution properties such as timeout and consistency settings. |
219+
220+
### Payload examples
221+
222+
**Query using default workspace:**
223+
224+
```json
225+
{
226+
"db": "default",
227+
"csl": "EntraUsers | take 10"
228+
}
229+
```
230+
231+
**Query using a specific workspace:**
232+
233+
```json
234+
{
235+
"csl": "SigninLogs | take 10",
236+
"db": "workspace1-12345678-abcd-abcd-1234-1234567890ab"
237+
}
238+
```
239+
240+
**Query with optional properties:**
241+
242+
```json
243+
{
244+
"csl": "SigninLogs | take 10",
245+
"db": "workspace1-12345678-abcd-abcd-1234-1234567890ab",
246+
"properties": {
247+
"Options": {
248+
"servertimeout": "00:04:00",
249+
"queryconsistency": "strongconsistency",
250+
"query_language": "kql"
251+
}
252+
}
253+
}
254+
```
255+
256+
## Considerations and limitations
257+
258+
Consider the following points when planning to execute KQL queries on the data lake via API.
259+
260+
### Service principal permissions
261+
262+
When using a service principal, Azure RBAC roles can be assigned at the Sentinel workspace level. Entra ID roles or XDR unified RBAC roles aren't supported for this scenario. Alternatively, use user tokens with Entra ID roles.
263+
264+
### Latency and cost considerations
265+
266+
The data lake is optimized for cost-effective storage, and queries might be slower than queries executed in the analytics tier. Design frequent or automated queries carefully, considering the query and concurrency limits.
267+
268+
### Result size limits
269+
270+
Queries are subject to limits on execution time and response size. Review the [Microsoft Sentinel data lake service limits](sentinel-lake-service-limits.md) when designing your workflows.
271+
272+
### Query limits
273+
274+
For information on query execution limits, concurrency, and timeout values, see [Microsoft Sentinel data lake service limits](sentinel-lake-service-limits.md).
275+
276+
## Related content
277+
278+
- [Run KQL queries on the Microsoft Sentinel data lake](kql-queries.md)
279+
- [Microsoft Sentinel data lake overview](sentinel-lake-overview.md)
280+
- [Onboarding to Microsoft Sentinel data lake](sentinel-lake-onboarding.md)
281+
- [Microsoft Sentinel data lake service limits](sentinel-lake-service-limits.md)
282+
- [Create a Microsoft Entra application and service principal that can access resources](/entra/identity-platform/howto-create-service-principal-portal)
283+
- [Microsoft Authentication Library (MSAL) overview](/entra/identity-platform/msal-overview)
284+
- [Acquire and cache tokens with Microsoft Authentication Library](/entra/identity-platform/msal-acquire-cache-tokens)

0 commit comments

Comments
 (0)