Skip to content

Commit 23787a7

Browse files
authored
Merge pull request #311449 from EdB-MSFT/azure-storage-blob-connectors
Azure storage blob connectors
2 parents 71e11ef + b9aa7c7 commit 23787a7

21 files changed

Lines changed: 624 additions & 1 deletion

articles/sentinel/TOC.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -721,6 +721,8 @@
721721
href: connect-services-windows-based.md
722722
- name: Azure Functions API connection
723723
href: connect-azure-functions-template.md
724+
- name: Azure Storage blob connectors
725+
href: setup-azure-storage-connector.md
724726
- name: CEF/Syslog
725727
items:
726728
- name: Overview
@@ -1034,6 +1036,14 @@
10341036
href: data-connector-connection-rules-reference.md
10351037
- name: GCP data connectors API reference
10361038
href: data-connection-rules-reference-gcp.md
1039+
- name: Azure Storage Blob data connectors
1040+
items:
1041+
- name: Azure Storage Blob data connector reference
1042+
href: data-connection-rules-reference-azure-storage.md
1043+
- name: Enable network security for Azure Storage Blob data connector
1044+
href: enable-storage-network-security.md
1045+
- name: Troubleshoot Azure Storage Blob connector issues
1046+
href: azure-storage-blob-connector-troubleshoot.md
10371047
- name: Sample API requests for creating Data Collection Rules (DCRs)
10381048
href: api-dcr-reference.md
10391049
- name: Microsoft Purview Information Protection reference
Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
---
2+
title: Troubleshoot Azure Storage Blob connector issues - Microsoft Sentinel
3+
description: Troubleshoot Azure Storage Blob connector issues in Microsoft Sentinel.
4+
author: EdB-MSFT
5+
ms.author: edbaynash
6+
ms.topic: troubleshooting
7+
ms.date: 02/19/2026
8+
ms.service: microsoft-sentinel
9+
10+
#Customer intent: As a security engineer, I want to troubleshoot Azure Storage Blob connector issues so that I can ensure seamless log ingestion into Microsoft Sentinel.
11+
12+
---
13+
14+
# Troubleshoot Azure Storage Blob connector issues
15+
16+
The Azure Storage Blob connector simplifies the process of ingesting data from Azure Storage Blobs to Microsoft Sentinel.
17+
18+
This article describes how to quickly identify the cause of issues occurring with the Azure Storage Blob connector so you can find the steps needed to resolve the issues.
19+
20+
Learn how to [connect Microsoft Sentinel to Azure Storage Blob to ingest data](setup-azure-storage-connector.md).
21+
22+
## Microsoft Sentinel doesn't receive data from the Azure Storage Blob connector
23+
24+
The logs for the Azure Storage Blob connector aren't visible in the Microsoft Sentinel workspace for more than 30 minutes after the connector was connected.
25+
26+
Before you search for a cause and solution, review these considerations:
27+
28+
- It can take around 20-30 minutes from the moment the connector is connected until data is ingested into the workspace.
29+
- The connector's connection status indicates that a collection rule exists; it doesn't indicate that data was ingested. If the status of the Azure Storage Blob connector is green, there's a collection rule for one of the data types, but still no data.
30+
31+
### Determine the cause of your problem
32+
33+
In this section, we cover these causes:
34+
35+
1. [The data isn't ingested to the Azure Storage Blob container.](#cause-1-the-data-isnt-ingested-to-the-azure-storage-blob-container)
36+
1. [The Azure Storage queue isn't receiving notifications for blob created events.](#cause-2-the-azure-storage-queue-isnt-receiving-notifications-for-blob-created-events)
37+
1. [The Azure Storage Blob connector permissions or networking policies aren't set properly.](#cause-3-the-azure-storage-blob-connector-permissions-or-networking-policies-arent-set-properly)
38+
1. [The queue message content or Azure Storage Blob data format is invalid.](#cause-4-the-queue-message-content-or-azure-storage-blob-data-format-is-invalid)
39+
1. [The DCR or DCE configuration is incorrect.](#cause-5-the-dcr-or-dce-configuration-is-incorrect)
40+
41+
> [!TIP]
42+
> If the health feature isn't enabled, [enable it](enable-monitoring.md).
43+
44+
### Cause 1: The data isn't ingested to the Azure Storage Blob container
45+
46+
The upstream system isn't successfully delivering data to the expected storage container.
47+
48+
#### Solution: Verify data ingress on the storage account
49+
50+
1. Verify that the storage account has hierarchical namespace enabled (Azure Data Lake Storage Gen2). The connector requires ADLS Gen2 storage accounts.
51+
1. In the Azure portal, navigate to the storage account and review the **Metrics** to confirm ingress activity.
52+
1. Check that the expected storage container is receiving blobs.
53+
1. If the container is empty, review the upstream producing system for issues delivering data to the account.
54+
55+
### Cause 2: The Azure Storage queue isn't receiving notifications for blob created events
56+
57+
Event Grid system topics facilitate blob created events from the source storage account to the target storage queue. If the system topic or subscription isn't configured correctly, events won't flow to the queue.
58+
59+
#### Solution: Verify Event Grid system topic and subscription
60+
61+
1. Check the resource group's deployments for any failures when creating the system topic resources. If a system topic already exists on the storage account from a different configuration, it might conflict with the connector's subscription.
62+
1. If the Event Grid system topic and subscription were successfully created, review the metrics of the Event Grid subscription. Consider [enabling diagnostic logs](/azure/event-grid/enable-diagnostic-logs-topic).
63+
1. Verify that the subscription source maps to the correct storage account and target queue.
64+
1. Review the filtering criteria specified under **Additional Features** to ensure it matches expected patterns.
65+
1. The Event Grid subscription's metrics should show events that match the blob created criteria. If the metrics indicate **Delivery Failed Events**, review the Event Grid's diagnostic logs and continue to the next section for policy-related issues.
66+
1. Check the storage queue metrics to confirm messages are arriving. In the Azure portal, navigate to the storage account, select **Queues**, and review the **Approximate Messages Count** for the connector queue. If the queue is empty but Event Grid shows successful deliveries, the connector might be consuming messages but failing to process the referenced blobs.
67+
68+
:::image type="content" source="./media/azure-storage-blob-connector-troubleshoot/event-grid-subscription-metrics.png" lightbox="./media/azure-storage-blob-connector-troubleshoot/event-grid-subscription-metrics.png" alt-text="Screenshot showing an Event Grid subscription metrics chart.":::
69+
70+
### Cause 3: The Azure Storage Blob connector permissions or networking policies aren't set properly
71+
72+
This issue is caused by incorrect role-based access policies applied to the storage account(s) hosting the blob container and/or the message queue, or by network security settings blocking traffic.
73+
74+
#### Solution: Verify permissions and networking
75+
76+
Review the [Set up your Azure Storage connector](setup-azure-storage-connector.md) documentation and ensure the following are in place:
77+
78+
**Service principal and RBAC checks:**
79+
80+
1. Verify that the application ID of the service principal matches the application IDs per Azure environment specified in the [Azure Storage Blob connectors API reference](data-connection-rules-reference-azure-storage.md#authentication-configuration).
81+
1. Verify that the service principal has the **Storage Blob Data Reader** role on the storage accounts hosting the blob container.
82+
1. Verify that the service principal has the **Storage Queue Data Contributor** role on the storage accounts hosting the message queues.
83+
84+
> [!TIP]
85+
> Clues to failures of this kind are exposed on connectivity check issues during installation and in the `SentinelHealth` table for active connectors.
86+
87+
**Network security checks:**
88+
89+
If the service principal and RBAC troubleshooting doesn't surface an issue, the network security settings on the storage account(s) might be causing the problem. The network security solution for this connector relies on [Network Security Perimeter](/azure/private-link/network-security-perimeter-concepts) based protection. Review the [Enable network security](enable-storage-network-security.md) documentation to ensure all steps were followed correctly, and check the following:
90+
91+
- If you aren't using NSP to protect the resource, check the storage account's **Networking** blade to ensure public network access is enabled.
92+
- Verify that the storage account isn't using selected network limits via IPv4 CIDR addresses. This approach doesn't work with the connector's IP traffic due to the documented [storage firewall limitations](/azure/storage/common/storage-network-security-limitations#restrictions-for-ip-network-rules) around IP ranges and region affinity of the caller and the account.
93+
- If NSP is being used to protect the account, enable the [perimeter's diagnostic logs](/azure/private-link/network-security-perimeter-diagnostic-logs) to troubleshoot. NSP rules only apply to resources in **Enforced** access mode. Alternatively, **Transition** mode doesn't apply the rules on the resource while continuing to collect telemetry on traffic patterns. Review the profile associated with the storage account(s):
94+
- Check that inbound rules for the producer are in place. Check for blob write failures on the producer.
95+
- Check that inbound rules for the connector are in place per the [Enable network security](enable-storage-network-security.md) documentation. Verify that the `Scuba` service tag IP ranges are included in the NSP inbound rules.
96+
- Check that inbound rules include a rule for the subscription of the storage account and Event Grid system topic.
97+
- Ensure the Event Grid system topic subscription is using **System Assigned** managed identity-based delivery.
98+
99+
:::image type="content" source="./media/azure-storage-blob-connector-troubleshoot/system-topic-managed-identity-type.png" lightbox="./media/azure-storage-blob-connector-troubleshoot/system-topic-managed-identity-type.png" alt-text="Screenshot of Network Security Perimeter rules showing inbound rules for the producer and connector, including Scuba service tag and subscription rule.":::
100+
101+
102+
### Cause 4: The queue message content or Azure Storage Blob data format is invalid
103+
104+
The blob data format or queue message structure doesn't match the expected configuration.
105+
106+
#### Solution: Verify data format and queue messages
107+
108+
1. Check the `SentinelHealth` table for clues about invalid messages by running the following query:
109+
110+
```kusto
111+
SentinelHealth
112+
| where TimeGenerated > ago(1d)
113+
| where SentinelResourceKind == "StorageAccountBlobContainer"
114+
| where Status != "Success"
115+
| distinct TimeGenerated, OperationName, SentinelResourceName, Status, Description
116+
```
117+
118+
1. If the health data references invalid data, verify that the format of the blobs uploaded to storage matches the serialization and compression model set in the connector's response configuration. The connector supports the following formats: **JSON**, **CSV**, **XML**, and **Parquet**. For more information, see the [response configuration](data-connection-rules-reference-azure-storage.md#response-configuration) in the API reference.
119+
1. Queue message format exceptions result from messages in the queue not aligning to the `EventGridSchema` for `BlobCreated` events. Check the Event Grid subscription's **Filters** section to confirm:
120+
- The filter is set to **Blob Created**.
121+
- The event schema is **EventGridSchema**.
122+
123+
:::image type="content" source="./media/azure-storage-blob-connector-troubleshoot/system-topic-filters.png" lightbox="./media/azure-storage-blob-connector-troubleshoot/system-topic-filters.png" alt-text="Screenshot of Event Grid system topic filters.":::
124+
125+
### Cause 5: The DCR or DCE configuration is incorrect
126+
127+
The Data Collection Rule (DCR) or Data Collection Endpoint (DCE) referenced by the connector is misconfigured, causing ingestion failures after the connector successfully reads the blob data.
128+
129+
#### Solution: Verify the DCR and DCE configuration
130+
131+
1. Verify that the DCE referenced in the connector's `dcrConfig.dataCollectionEndpoint` property is accessible and in the same region as the Log Analytics workspace.
132+
1. Verify that the DCR immutable ID in `dcrConfig.dataCollectionRuleImmutableId` is correct. You can find the immutable ID by viewing the DCR resource properties in the Azure portal or using the [DCR API](/rest/api/monitor/data-collection-rules/get).
133+
1. Verify that the `streamName` in the connector configuration matches the `streamDeclarations` defined in the DCR.
134+
1. If the DCR uses a KQL transformation query, test the query independently in the Log Analytics workspace to ensure it's valid.
135+
1. Verify that the destination table defined in the DCR exists in the Log Analytics workspace and has the expected schema.
136+
1. For more information on configuring the DCR, see the [DCR configuration](data-connection-rules-reference-azure-storage.md#dcr-configuration) section in the API reference.
137+
138+
## Next steps
139+
140+
In this article, you learned how to quickly identify causes and resolve common issues with the Azure Storage Blob connector.
141+
142+
We welcome feedback, suggestions, requests for features, bug reports or improvements and additions. Go to the [Microsoft Sentinel GitHub repository](https://github.com/Azure/Azure-Sentinel) to create an issue or fork and upload a contribution.
143+
144+
## Related content
145+
146+
- [Set up your Azure Storage connector to stream logs to Microsoft Sentinel](setup-azure-storage-connector.md)
147+
- [Azure Storage Blob data connector reference for the Codeless Connector Framework](data-connection-rules-reference-azure-storage.md)
148+
- [Enable network security on connector integrated storage resources](enable-storage-network-security.md)
149+
- [Create a codeless connector for Microsoft Sentinel](create-codeless-connector.md)

articles/sentinel/create-codeless-connector.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,10 +136,11 @@ To learn from an example, see the [Data connector definitions reference example
136136

137137
### Data connection rules
138138

139-
There are currently two kinds of data connection rules possible for defining your CCF data connector.
139+
There are currently three kinds of data connection rules possible for defining your CCF data connector.
140140

141141
- `RestApiPoller` kind allows you to customize paging, authorization and expected request/response payloads for your data source. For more information, see [RestApiPoller data connector connection rules reference](data-connector-connection-rules-reference.md).
142142
- `GCP` kind allows you to decrease your development time by automatically configuring paging and expected response payloads for your Google Cloud Platform (GCP) data source. For more information, see [GCP data connector connection rules reference](data-connection-rules-reference-gcp.md)
143+
- `StorageAccountBlobContainer` kind allows you to ingest from an Azure Storage Blob data source. For more information, see [Azure Storage Blob connectors API reference](data-connection-rules-reference-azure-storage.md).
143144

144145
Use an [API testing tool](#testing-apis) to call the data connector API to create the data connector which combines the connection rules and previous components. Verify the connector is now connected in the UI.
145146

0 commit comments

Comments
 (0)