You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/stream-analytics/azure-data-explorer-managed-identity.md
+87-30Lines changed: 87 additions & 30 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,11 +1,13 @@
1
1
---
2
-
title: Use managed identities to access Azure Data Explorer from an Azure Stream Analytics job
2
+
title: Use Managed Identity for Azure Data Explorer Output
3
3
description: This article describes how to use managed identities to authenticate your Azure Stream Analytics job to an Azure Data Explorer output.
4
-
author: AliciaLiMicrosoft
5
-
ms.author: ali
4
+
#customer intent: As a data engineer, I want to configure a managed identity for my Azure Stream Analytics job so that I can securely access Azure Data Explorer without storing access keys.
With managed identities, the Azure platform manages this runtime identity. You don't need to store and protect access keys in your application code or configuration, either for the identity itself, or for the resources you need to access. For more information on managed identities for Azure Stream Analytics, see [Managed identities for Azure Stream Analytics](stream-analytics-managed-identities-overview.md).
19
21
20
-
This article shows you how to enable system-assigned managed identity for an Azure Data Explorer output of a Stream Analytics job through the Azure portal.Before you can enable system-assigned managed identity, you must first have a Stream Analytics job and an Azure Data Explorer resource.
22
+
This article shows you how to enable system-assigned or user-assigned managed identity for an Azure Data Explorer output of a Stream Analytics job through the Azure portal.Before you can enable managed identity, you must first have a Stream Analytics job and an Azure Data Explorer resource.
21
23
22
-
> [!NOTE]
23
-
> Azure Data Explorer supports only managed identities. You will not be able to authenticate to your Azure Data Explorer clusters with connection string.
24
+
> [!IMPORTANT]
25
+
> - Azure Data Explorer supports **only managed identities** for authentication. You can't authenticate to your Azure Data Explorer clusters with connection strings or keys.
26
+
> - Permissions are granted at the **Azure Data Explorer database level**, not at the cluster IAM level.
24
27
25
28
## Create a managed identity
26
29
27
30
First, you create a managed identity for your Azure Stream Analytics job.
28
31
29
32
1. In the Azure portal, open your Azure Stream Analytics job.
30
33
31
-
2. Fromtheleftnavigationmenu, select**Managed Identity**located under*Configure*. Then, check the box next to **Use System-assigned Managed Identity** and select **Save**.
:::image type="content" source="media/event-hubs-managed-identity/system-assigned-managed-identity.png" alt-text="Screenshot of the Azure portal showing where to select managed identity in your stream analytics job.":::
36
+
1. Choose **Select identity** on the toolbar.
34
37
35
-
3. A service principal for the Stream Analytics job's identity is created in Azure Active Directory. The life cycle of the newly created identity is managed by Azure. When the Stream Analytics job is deleted, the associated identity (that is, the service principal) is automatically deleted by Azure.
38
+
1. In the **Select identity** pane, for **Identity to use with job**, select **System assigned**. Alternatively, you can enable **User-assigned Managed Identity** if you prefer a reusable identity across multiple jobs.
36
39
37
-
When you save the configuration, the Object ID (OID) of the service principal is listed as the Principal ID as shown below:
40
+
1. Select **Save**.
38
41
39
-
:::image type="content" source="media/event-hubs-managed-identity/principal-id.png" alt-text="Screenshot of the Azure portal showing how to select the Principal ID of your stream analytics job.":::
42
+
:::image type="content" source="media/event-hubs-managed-identity/system-assigned-managed-identity.png" alt-text="Screenshot of the Azure portal showing where to select managed identity in your stream analytics job." lightbox="media/event-hubs-managed-identity/system-assigned-managed-identity.png":::
40
43
41
-
The service principal has the same name as the Stream Analytics job. For example, if the name of your job is `MyASAJob`, the name of the service principal is also `MyASAJob`.
44
+
3. A service principal for the Stream Analytics job's identity is created in Microsoft Entra ID. The life cycle of the newly created identity is managed by Azure. When the Stream Analytics job is deleted, the associated identity (that is, the service principal) is automatically deleted by Azure.
45
+
46
+
### Choose between system-assigned and user-assigned identity
47
+
48
+
| Identity type | When to use |
49
+
|---|---|
50
+
|**System-assigned**| Simpler setup; lifecycle is tied to the Stream Analytics job |
51
+
|**User-assigned**| Reusable across multiple jobs; useful for centralized access control |
52
+
53
+
For more information on user-assigned managed identities, see [Use user-assigned managed identities for Azure Stream Analytics](stream-analytics-user-assigned-managed-identity-overview.md).
54
+
55
+
When you save the configuration, the Object ID (OID) of the service principal is listed as the Principal ID as shown below:
56
+
57
+
:::image type="content" source="media/event-hubs-managed-identity/principal-id.png" alt-text="Screenshot of the Azure portal showing how to select the Principal ID of your stream analytics job." lightbox="media/event-hubs-managed-identity/principal-id.png":::
58
+
59
+
The service principal has the same name as the Stream Analytics job. For example, if the name of your job is `MyASAJob`, the name of the service principal is also `MyASAJob`.
42
60
43
61
## Grant the Stream Analytics job permissions to access Azure Data Explorer
44
62
45
-
For the Stream Analytics job to access your Azure Data Explorer cluster using managed identity, the service principal you created must have special permissions to your Azure Data Explorer resource. In this step, you can assign a role to your stream analytics job's system-assigned managed identity. Azure provides the below Azure built-in roles for authorizing access to an Azure Data Explorer namespace. For Azure Stream Analytics you would need these roles:
63
+
For the Stream Analytics job to access your Azure Data Explorer database using managed identity, the service principal you created must have special permissions to your Azure Data Explorer **database**. In this step, you assign roles to your Stream Analytics job's managed identity at the database level.
| Ingestor | Can ingest data into all existing tables in the database, but can't query the data. |
50
-
| Monitor | Can execute `.show` commands in the context of the database and its child entities. |
65
+
Azure Data Explorer provides the following built-in roles for database access. For Azure Stream Analytics, you need **both** of these roles:
51
66
52
-
For more information about roles supported Azure Data Explorer, see [Role-based access control in Azure Data Explorer](/kusto/access-control/role-based-access-control?view=azure-data-explorer&preserve-view=true#roles-and-permissions).
67
+
| Role | Permissions |
68
+
|---|---|
69
+
|**Ingestor**| Can ingest data into all existing tables in the database, but can't query the data. |
70
+
|**Monitor**| Can execute `.show` commands in the context of the database and its child entities. |
53
71
54
-
1. Select **Access control (IAM)**.
72
+
For more information about roles supported by Azure Data Explorer, see [Role-based access control in Azure Data Explorer](/kusto/access-control/role-based-access-control?view=azure-data-explorer&preserve-view=true#roles-and-permissions).
55
73
56
-
2. Select **Add** > **Add role assignment** to open the **Add role assignment** page.
74
+
### Assign database permissions
57
75
58
-
3. Assign the following role. For detailed steps, see [Assign Azure roles using the Azure portal](/azure/role-based-access-control/role-assignments-portal).
76
+
1. In the Azure portal, open your **Azure Data Explorer cluster**.
59
77
60
-
| Setting | Value |
61
-
| --- | --- |
62
-
| Role | Ingestor and Monitor |
63
-
| Assign access to | User, group, or service principal |
64
-
| Members |\<Name of your Stream Analytics job> |
78
+
1. Select **Databases** from the left navigation menu, then select your target database.
65
79
66
-

80
+
1. Select **Permissions** from the left navigation menu.
81
+
82
+
1. Select **Add** and choose **Ingestor**.
83
+
84
+
:::image type="content" source="media/event-hubs-managed-identity/monitor-ingestor-roles.png" alt-text="Screenshot of the Azure portal showing how to add Ingestor and Monitor roles." lightbox="media/event-hubs-managed-identity/monitor-ingestor-roles.png":::
85
+
86
+
1. Search for and select your Stream Analytics job's managed identity (it has the same name as your Stream Analytics job).
87
+
88
+
1. Select **Select** to confirm.
89
+
90
+
1. Repeat steps 4-6 to add the **Monitor** role.
67
91
68
92
> [!NOTE]
69
93
> Due to global replication or caching latency, there may be a delay when permissions are revoked or granted. Changes should be reflected within 8 minutes.
@@ -75,10 +99,43 @@ Now that your managed identity is configured, you're ready to add the Azure Data
75
99
76
100
1. Go to your Stream Analytics job and navigate to the **Outputs** page under **Job Topology**.
77
101
78
-
1. Select **Add > Azure Data Explorer**. In the output properties window, search and select your Azure Data Explorer cluster or type in the URL of your cluster and select **Managed Identity: System assigned** from the *Authentication mode* drop-down menu.
102
+
1. Select **Add > Azure Data Explorer**.
103
+
104
+
:::image type="content" source="media/event-hubs-managed-identity/select-azure-data-explorer.png" alt-text="Screenshot of the Azure Stream Analytics job showing how to select Azure Data Explorer as an output." lightbox="media/event-hubs-managed-identity/select-azure-data-explorer.png":::
105
+
106
+
1. In the output properties window, search and select your Azure Data Explorer cluster or type in the URL of your cluster and select **Managed Identity: System assigned** from the *Authentication mode* drop-down menu.
107
+
108
+
1. Fill out the rest of the properties, including:
109
+
-**Database name**: The target database in your Azure Data Explorer cluster
110
+
-**Table name**: The target table where data will be ingested
111
+
112
+
:::image type="content" source="media/event-hubs-managed-identity/azure-data-explorer-output.png" alt-text="Screenshot of the Azure Stream Analytics job showing how to configure Azure Data Explorer output." lightbox="media/event-hubs-managed-identity/azure-data-explorer-output.png":::
113
+
114
+
1. Select **Save**.
115
+
116
+
## Ensure table schema compatibility
117
+
118
+
For ingestion to succeed, your Stream Analytics query output must match the Azure Data Explorer table schema:
119
+
120
+
-**Column names** must exactly match (case-sensitive)
121
+
-**Data types** must be compatible
122
+
-**Column order** should align with the table schema
123
+
124
+
Extra or mismatched columns cause ingestion failures. Stream Analytics sends data to Azure Data Explorer using CSV ingestion.
125
+
126
+
> [!TIP]
127
+
> Use the `.show table <TableName> schema as json` command in Azure Data Explorer to verify your table schema matches your Stream Analytics query output.
128
+
129
+
130
+
## Troubleshooting checklist
79
131
80
-
1. Fill out the rest of the properties and select **Save**.
132
+
If you experience issues, verify the following:
81
133
134
+
-:heavy_check_mark: Managed identity is enabled on the Stream Analytics job
135
+
-:heavy_check_mark: Identity is added as **Database Ingestor** and **Database Monitor** at the database level (not cluster IAM)
136
+
-:heavy_check_mark: Azure Data Explorer output is configured in the Stream Analytics job
137
+
-:heavy_check_mark: Table schema matches the Stream Analytics query output exactly
138
+
-:heavy_check_mark: Sufficient time has passed for permission propagation (up to 8 minutes)
0 commit comments