You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Set up your Azure Storage connector to stream logs to Microsoft Sentinel
16
16
17
-
The Azure Storage Blob connector simplifies the process of collecting logs from Azure Storage. This connector enables ISVs and users to build scalable connectors on top of integrations with Azure Storage through the fully managed Codeless Connector Framework (CCF) solution.
17
+
The Azure Storage Blob connector simplifies collecting logs from Azure Storage. It lets ISVs and users build scalable connectors on top of Azure Storage integrations through the fully managed Codeless Connector Framework (CCF).
18
18
19
-
This article describes the underlying resources used to facilitate the connector and provides step-by-step instructions for creating your first Azure Storage connector.
19
+
This article summarizes the connector resources and provides steps to create and validate your first Azure Storage connector.
20
20
21
21
## Prerequisites
22
22
23
-
Before you begin, ensure you have the following:
23
+
Before you begin, ensure you have:
24
24
25
-
- An Azure Data Lake Storage Gen 2 account (supports blobs and queues) with a container created for the data.
26
-
- A Microsoft Sentinel workspace with the required permissions to create data connectors.
27
-
- Permissions to create and manage Event Grid system topics and subscriptions on the storage account.
25
+
- An Azure Storage account with hierarchical namespace enabled (Azure Data Lake Storage Gen2) and a container that holds the log files.
26
+
- A Microsoft Sentinel workspace with a Microsoft Sentinel Contributor or higher role to create data connectors.
27
+
- Owner or EventGrid Contributor role permissions on the storage account to create Event Grid system topics and subscriptions.
28
+
29
+
> [!NOTE]
30
+
> Make sure the **Microsoft.EventGrid** resource provider is registered in the subscription that contains the storage account.
28
31
29
32
## Connector resource overview
30
33
31
-
The Azure Storage Blob connector uses a queue-based blob-pointer model to subscribe to blobcreated events in your storage account. An Event Grid system topic subscription listens for blob creation activity and pushes events based on a configurable filtering criterion to the Azure Storage queue. This allows multiple connector instances to ingest from the same Azure Storage container while scoping the files based on separate folder directories and file patterns. This filtering can be controlled through the portal or the connector ARM template to scope the blob prefix and suffix patterns.
34
+
The Azure Storage Blob connector uses a queue-based blob-pointer model to subscribe to blob-created events in your storage account. An Event Grid system topic subscription listens for blob creation activity and pushes events, based on configurable filtering criteria, to an Azure Storage queue. Multiple connector instances can ingest from the same container while scoping files by folder and file pattern. You can control filtering through the portal or the connector ARM template by setting blob prefix and suffix patterns.
32
35
33
36
:::image type="content" source="./media/setup-azure-storage-connector/overview-diagram.png" lightbox="./media/setup-azure-storage-connector/overview-diagram.png" alt-text="A diagram showing the Azure Storage Blob connector architecture, including blob created events, Event Grid, storage queue, and Microsoft Sentinel ingestion flow.":::
34
37
35
38
The Microsoft Sentinel connector:
36
39
37
-
- Polls the Azure Storage queue frequently for messages, filtering to blobcreated events.
38
-
- Fetches the files from the Azure Storage Blob container based on the path specified in the queue message.
39
-
-On successful forwarding to the destination, deletes the queue message.
40
+
- Polls the Azure Storage queue for blob-created messages.
41
+
- Fetches files from the Azure Storage Blob container based on the path in the queue message.
42
+
-Deletes the queue message after successful forwarding.
40
43
41
-
To authenticate with the Azure Storage Blob container and queue, the connector uses a service principal accessible to the connector's application. For details on the application per Azure environment, see the [Azure Storage Blob connectors API reference](data-connection-rules-reference-azure-storage.md). We strongly recommend that you use the automation in the ARM template to validate that the service principal exists and apply necessary role-based access to the storage account.
44
+
The connector authenticates to the Storage Account by using a service principal accessible to the connector application. For the application IDs per cloud and the full template schema, see the [Azure Storage Blob connectors API reference](data-connection-rules-reference-azure-storage.md). Use the ARM template automation to verify that the service principal exists and to apply the required role assignments on the storage account.
42
45
43
46
## Create an Azure Storage Blob connector
44
47
45
-
To create your first Azure Storage Blob connector, follow these steps:
48
+
1. Review and adapt the example ARM template in the [Azure Storage Blob connectors API reference](data-connection-rules-reference-azure-storage.md#build-the-azure-storage-blob-ccf-data-connector). Set the container name, queue name (if not auto-created), blob prefix/suffix filters, and destination table mapping.
49
+
2. Deploy the template by following [Create a codeless connector for Microsoft Sentinel](create-codeless-connector.md#data-connection-rules). Ensure the deployment scope matches the storage account and Microsoft Sentinel workspace.
50
+
3. After deployment, confirm the connector instance is created in Microsoft Sentinel and that the Event Grid subscription status is **Healthy**.
51
+
52
+
## Validate the connector
46
53
47
-
1. Review and update the example template included in the [Azure Storage Blob connectors API reference](data-connection-rules-reference-azure-storage.md#build-the-azure-storage-blob-ccf-data-connector).
48
-
1. Follow the steps in [Create a codeless connector for Microsoft Sentinel](create-codeless-connector.md#data-connection-rules) to deploy the connector using the template.
54
+
- Upload a sample file that matches your prefix/suffix filter and confirm that queue messages are created and consumed.
55
+
- Verify ingestion in the target table in Microsoft Sentinel and check for errors in the connector health blade.
56
+
- If you use network restrictions, confirm that the connector-managed resources can reach the blob and queue endpoints.
49
57
50
58
## Troubleshooting
51
59
52
-
For information on troubleshooting Azure Storage Blob connector issues, see [Troubleshoot Azure Storage Blob connector issues](azure-storage-blob-connector-troubleshoot.md).
60
+
For troubleshooting steps, see [Troubleshoot Azure Storage Blob connector issues](azure-storage-blob-connector-troubleshoot.md).
0 commit comments