Skip to content

Commit a24b40f

Browse files
Carolyn135Carolyn135
authored
Merge pull request #314758 from EdB-MSFT/user/fuqingwang/AWSEKSDoc
User/fuqingwang/awseks doc
2 parents 3ed0b0a + 3be9679 commit a24b40f

3 files changed

Lines changed: 283 additions & 0 deletions

File tree

‎articles/sentinel/TOC.yml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -791,6 +791,8 @@
791791
href: connect-aws.md
792792
- name: AWS S3 WAF logs
793793
href: connect-aws-s3-waf.md
794+
- name: AWS EKS logs
795+
href: connect-aws-eks.md
794796
- name: Enable attack disruption actions on AWS
795797
href: aws-disruption.md
796798
- name: Troubleshoot AWS S3 connector issues

‎articles/sentinel/connect-aws-configure-environment.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,7 @@ Microsoft Sentinel provides these AWS connectors:
155155

156156
- [Amazon Web Services Web Application Firewall (WAF) connector](connect-aws-s3-waf.md): Ingests AWS WAF logs, collected in AWS S3 buckets, to Microsoft Sentinel.
157157
- [Amazon Web Services service log connector](connect-aws.md): Ingests AWS service logs, collected in AWS S3 buckets, to Microsoft Sentinel.
158+
- [Amazon Web Services Elastic Kubernetes Service (EKS) log connector](connect-aws-eks.md): Ingests AWS EKS audit logs, collected in AWS S3 buckets, to Microsoft Sentinel.
158159

159160
---
160161

‎articles/sentinel/connect-aws-eks.md‎

Lines changed: 280 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,280 @@
1+
---
2+
title: Connect Microsoft Sentinel to Amazon Web Services to ingest AWS EKS logs
3+
description: Use the Amazon Web Services (AWS) S3-based Elastic Kubernetes Service (EKS) connector to ingest AWS EKS audit logs, collected in AWS S3 buckets, to Microsoft Sentinel.
4+
author: EdB-MSFT
5+
ms.author: edbaynash
6+
ms.topic: how-to
7+
ms.date: 04/15/2026
8+
appliesto:
9+
- Microsoft Sentinel in the Microsoft Defender portal
10+
- Microsoft Sentinel in the Azure portal
11+
ms.collection: usx-security
12+
#Customer intent: As a security operator, I want to ingest Elastic Kubernetes Service (EKS) audit logs from my Amazon Web Services S3 bucket to my Microsoft Sentinel workspace, so that security analysts can monitor Kubernetes cluster activities and detect security threats.
13+
---
14+
15+
# Connect Microsoft Sentinel to Amazon Web Services to ingest AWS EKS logs
16+
17+
Use the Amazon Web Services (AWS) S3-based Elastic Kubernetes Service (EKS) connector to ingest AWS EKS audit logs, collected in AWS S3 buckets, to Microsoft Sentinel. AWS EKS audit logs are detailed records of API server requests, authentication decisions, and cluster activities within your Kubernetes clusters. These records contain information such as the time the request was received, the specifics of the request, the user making the request, and the action taken. This log analysis is essential for maintaining the security and compliance of containerized applications running on EKS clusters.
18+
19+
This connector features an *AWS CloudFormation*-based onboarding script to streamline the creation of the AWS resources used by the connector.
20+
21+
> [!IMPORTANT]
22+
> - The **Amazon Web Services S3 EKS** data connector is currently in preview. The [Azure Preview Supplemental Terms](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) include additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.
23+
>
24+
> - [!INCLUDE [unified-soc-preview-no-alert](includes/unified-soc-preview-without-alert.md)]
25+
26+
## Overview
27+
28+
The **Amazon Web Services S3 EKS** data connector serves the following use cases:
29+
30+
- **Kubernetes security monitoring and threat detection:** Analyze AWS EKS audit logs to help identify and respond to security threats such as unauthorized access, privilege escalation, and suspicious API calls within your Kubernetes clusters. By ingesting these logs into Microsoft Sentinel, you can use its advanced analytics and threat intelligence to detect and investigate malicious activities targeting your containerized workloads.
31+
32+
- **Compliance and auditing for containerized environments:** AWS EKS audit logs provide detailed records of all API server interactions, which are crucial for compliance reporting and auditing purposes in containerized environments. The connector ensures that these audit logs are available within Microsoft Sentinel for easy access and analysis, helping meet regulatory requirements for container security.
33+
34+
- **DevSecOps and cluster governance:** Monitor developer activities, resource access patterns, and configuration changes within your EKS clusters to ensure proper governance and security practices in your DevSecOps workflows.
35+
36+
This article explains how to configure the Amazon Web Services S3 EKS connector. The process of setting it up has two parts: the AWS side and the Microsoft Sentinel side. Each side's process produces information used by the other side. This two-way authentication creates secure communication.
37+
38+
## Prerequisites
39+
40+
- You must have write permission on the Microsoft Sentinel workspace.
41+
42+
- Install the Amazon Web Services solution from the **Content Hub** in Microsoft Sentinel. If you already installed an earlier version of the solution, update the solution in the content hub to ensure you have the latest version that includes this connector. For more information, see [Discover and manage Microsoft Sentinel out-of-the-box content](sentinel-solutions-deploy.md).
43+
44+
- You must have an existing AWS EKS cluster with audit logging enabled, or the ability to enable audit logging on your EKS cluster during the setup process.
45+
46+
- You must have appropriate AWS IAM permissions to:
47+
- Create IAM roles and policies
48+
- Create S3 buckets and configure bucket policies
49+
- Create SQS queues and configure queue policies
50+
- Create CloudFormation stacks
51+
- Configure EKS cluster logging settings
52+
- Create Kinesis Data Firehose delivery streams
53+
- Create Lambda functions
54+
55+
## Enable and configure the Amazon Web Services S3 EKS connector
56+
57+
To enable and configure the connector, complete the following tasks:
58+
59+
- **In your AWS environment:**
60+
61+
The **Amazon Web Services S3 EKS** connector page in Microsoft Sentinel provides downloadable AWS CloudFormation stack templates that automate the following AWS tasks:
62+
63+
- Configure your AWS EKS cluster to send audit logs to **CloudWatch Logs**.
64+
65+
- Create a **Kinesis Data Firehose delivery stream** to transform and deliver logs from CloudWatch to S3.
66+
67+
- Create an **S3 bucket** to store the processed audit logs.
68+
69+
- Create a **Simple Queue Service (SQS) queue** to provide notification when new log files are created in S3.
70+
71+
- Create a **web identity provider** to authenticate users to AWS through OpenID Connect (OIDC).
72+
73+
- Create an **assumed role** to grant permissions to users authenticated by the OIDC web identity provider to access your AWS resources.
74+
75+
- Attach the appropriate **IAM permissions policies** to grant the assumed role access to the appropriate resources (S3 bucket, SQS).
76+
77+
- Create a **Lambda function** to transform EKS audit logs into the format expected by Microsoft Sentinel.
78+
79+
- **In Microsoft Sentinel:**
80+
81+
- Configure the **Amazon Web Services S3 EKS Connector** in the Microsoft Sentinel portal by adding **log collectors** that poll the SQS queue and retrieve log data from the S3 bucket. [See the instructions below](#add-log-collectors).
82+
83+
## Set up the AWS environment
84+
85+
To simplify the onboarding process, the **Amazon Web Services S3 EKS** connector page in Microsoft Sentinel provides downloadable templates for use with the AWS CloudFormation service. The CloudFormation service uses these templates to automatically create resource stacks in AWS. These stacks include the resources described in this article, along with credentials, permissions, and policies.
86+
87+
> [!NOTE]
88+
> Use the automatic setup process. For special cases, see the [manual setup instructions](connect-aws-configure-environment.md#manual-setup).
89+
90+
### Prepare the template files
91+
92+
To run the script that sets up the AWS environment, use the following steps:
93+
94+
1. In the Azure portal, from the Microsoft Sentinel navigation menu, expand **Configuration**, and select **Data connectors**.
95+
96+
In the Defender portal, from the quick launch menu, expand **Microsoft Sentinel > Configuration** and select **Data connectors**.
97+
98+
1. Select **Amazon Web Services S3 EKS** from the list of data connectors.
99+
100+
If you don't see the connector, install the Amazon Web Services solution from the **Content hub** under **Content management** in Microsoft Sentinel, or update the solution to the latest version.
101+
102+
1. In the details pane for the connector, select **Open connector page**.
103+
104+
:::image type="content" source="media/connect-aws-s3-waf/find-aws-waf-connector.png" alt-text="Screenshot of data connectors gallery showing AWS S3 EKS connector.":::
105+
106+
1. In the **Configuration** section, under **1. AWS CloudFormation Deployment**, select the [AWS CloudFormation Stacks](https://aka.ms/awsCloudFormationLink#/stacks/create) link. This action opens the AWS console in a new browser tab.
107+
108+
1. Return to the tab of the portal where you have Microsoft Sentinel open. Select **Download** under *Template 1: OpenID Connect authentication deployment* to download the template that creates the OIDC web identity provider. The template is downloaded as a JSON file to your designated downloads folder.
109+
110+
> [!NOTE]
111+
> If you already have an OIDC web identity provider from a previous AWS connector setup, skip this step.
112+
113+
1. Select **Download** under *Template 2: AWS EKS resources deployment* to download the template that creates the other AWS resources. The template is downloaded as a JSON file to your designated downloads folder.
114+
115+
:::image type="content" source="media/connect-aws-s3-waf/configure-connector.png" alt-text="Screenshot of AWS S3 EKS connector configuration page.":::
116+
117+
### Create AWS CloudFormation stacks
118+
119+
Return to the AWS Console browser tab, which is open to the AWS CloudFormation page for creating a stack.
120+
121+
If you're not already signed in to AWS, sign in now. You're redirected to the AWS CloudFormation page.
122+
123+
#### Create the OIDC web identity provider
124+
125+
> [!IMPORTANT]
126+
> If you already have the OIDC web identity provider from a previous AWS connector setup, skip this step and proceed to [Create the remaining AWS resources](#create-the-remaining-aws-resources).<br>If you already have an OIDC Connect provider set up for Microsoft Defender for Cloud, add Microsoft Sentinel as an audience to your existing provider (Commercial: `api://1462b192-27f7-4cb9-8523-0f4ecb54b47e`, Government:`api://d4230588-5f84-4281-a9c7-2c15194b28f7`). Don't try to create a new OIDC provider for Microsoft Sentinel.
127+
128+
Follow the instructions on the AWS Console page for creating a new stack.
129+
130+
1. Specify a template and upload a template file.
131+
132+
1. Select **Choose file** and locate the *Template 1: OpenID connect authentication deployment.json* file you downloaded.
133+
134+
1. Choose a name for the stack.
135+
136+
1. Advance through the rest of the process and create the stack.
137+
138+
#### Create the remaining AWS resources
139+
140+
1. Return to the AWS CloudFormation stacks page and create a new stack.
141+
142+
1. Select **Choose file** and locate the *Template 2: AWS EKS resources deployment.json* file you downloaded.
143+
144+
1. Choose a name for the stack.
145+
146+
1. When prompted, enter the following parameters:
147+
148+
- **EKSClusterName**: Enter the name of your existing EKS cluster.
149+
- **Microsoft Sentinel Workspace ID**: To find your Workspace ID:
150+
- In the Azure portal, in the Microsoft Sentinel navigation menu, expand **Configuration** and select **Settings**. Select the **Workspace settings** tab, and find the Workspace ID on the Log Analytics workspace page.
151+
- In the Defender portal, in the quick launch menu, expand **System** and select **Settings**. Select **Microsoft Sentinel**, then select **Log Analytics settings** under **Settings for `[WORKSPACE_NAME]`**. Find the Workspace ID on the Log Analytics workspace page, which opens in a new browser tab.
152+
- **BucketName**: Enter a unique name for the S3 bucket where EKS audit logs are stored.
153+
- **SentinelSQSQueueName**: Enter a name for the SQS queue (default: MicrosoftSentinelEKSSqs).
154+
- **AwsRoleName**: Enter a name for the IAM role (must start with "OIDC_", default: OIDC_MicrosoftSentinelRoleEKS).
155+
156+
1. Advance through the rest of the process and create the stack.
157+
158+
1. After the stack creation is complete, go to the **Outputs** section of the CloudFormation stack and note the following values:
159+
- **SentinelRoleArn**: The ARN of the IAM role created for Microsoft Sentinel access.
160+
- **SentinelSQSQueueURL**: The URL of the SQS queue.
161+
- **Step1EnableEKSAuditLogging**: AWS CLI command to enable EKS audit logging.
162+
- **Step2CreateSubscriptionFilter**: AWS CLI command to create the CloudWatch Logs subscription filter.
163+
164+
### Enable EKS audit logging and configure log streaming
165+
166+
After creating the CloudFormation stacks, enable audit logging on your EKS cluster and configure log streaming:
167+
168+
1. If audit logging isn't already enabled on your EKS cluster, run the command provided in the **Step1EnableEKSAuditLogging** output from the CloudFormation stack.
169+
170+
1. Wait about five minutes for audit logs to start appearing in CloudWatch Logs.
171+
172+
1. Run the command provided in the **Step2CreateSubscriptionFilter** output to create a subscription filter that streams audit logs from CloudWatch to the Kinesis Data Firehose delivery stream.
173+
174+
1. The Lambda function automatically transforms the EKS audit logs into the format expected by Microsoft Sentinel and delivers them to S3, where they trigger SQS notifications for ingestion.
175+
176+
## Add log collectors
177+
178+
When you create the resource stacks and configure EKS audit logging, return to the browser tab open to the data connector page in Microsoft Sentinel, and begin the second part of the configuration process.
179+
180+
1. In the **Configuration** section, under **2. Connect new collectors**, select **Add new collector**.
181+
182+
:::image type="content" source="media/connect-aws-s3-waf/add-new-collector.png" alt-text="Screenshot of second part of AWS EKS connector configuration." lightbox="media/connect-aws-s3-waf/add-new-collector.png":::
183+
184+
1. Enter the role ARN of the IAM role that you created. Use the value from the **SentinelRoleArn** output of your CloudFormation stack (for example, `arn:aws:iam::{AWS_ACCOUNT_ID}:role/OIDC_MicrosoftSentinelRoleEKS`).
185+
186+
1. Enter the SQS queue URL that you created. Use the value from the **SentinelSQSQueueURL** output of your CloudFormation stack (for example, `https://sqs.{AWS_REGION}.amazonaws.com/{AWS_ACCOUNT_ID}/MicrosoftSentinelEKSSqs`).
187+
188+
1. Select **Connect** to add the collector. This action creates a data collection rule for the Azure Monitor Agent to retrieve the logs and ingest them into the dedicated *AWSEKSLogs_CL* table in your Log Analytics workspace.
189+
190+
:::image type="content" source="media/connect-aws-s3-waf/enter-collector-details.png" alt-text="Screenshot of adding new collector for EKS logs.":::
191+
192+
## Verify data ingestion
193+
194+
1. After setting up the connector, go to the **Logs** page (or the **Advanced hunting** page in the Defender portal) and run the following query. If you get any results, the connector is working properly.
195+
196+
```kusto
197+
AWSEKSLogs_CL
198+
| take 10
199+
```
200+
201+
1. You can also run more specific queries to explore your EKS audit data.
202+
203+
```kusto
204+
// View recent EKS audit events by verb (API action)
205+
AWSEKSLogs_CL
206+
| where TimeGenerated > ago(1h)
207+
| summarize count() by Verb
208+
| order by count_ desc
209+
```
210+
211+
```kusto
212+
// Monitor authentication decisions
213+
AWSEKSLogs_CL
214+
| where TimeGenerated > ago(24h)
215+
| where AuthDecision != ""
216+
| summarize count() by AuthDecision, User
217+
| order by count_ desc
218+
```
219+
220+
```kusto
221+
// Track failed requests (non-200 response codes)
222+
AWSEKSLogs_CL
223+
| where TimeGenerated > ago(24h)
224+
| where ResponseCode != 200
225+
| project TimeGenerated, User, Verb, ObjectRef, ResponseCode, SourceIPs
226+
| order by TimeGenerated desc
227+
```
228+
229+
## Schema reference
230+
231+
The EKS audit logs ingest into the **AWSEKSLogs_CL** table with the following schema:
232+
233+
| Column | Type | Description |
234+
|--------|------|-------------|
235+
| TimeGenerated | datetime | The time when the audit event was generated |
236+
| AwsAccountId | string | AWS account ID where the EKS cluster is located |
237+
| Region | string | AWS region where the EKS cluster is located |
238+
| ClusterName | string | Name of the EKS cluster |
239+
| Verb | string | The HTTP verb associated with the API request (GET, POST, PUT, DELETE, etc.) |
240+
| User | string | Information about the user making the request |
241+
| SourceIPs | dynamic | Array of source IP addresses from which the request originated |
242+
| UserAgent | string | User agent string of the client making the request |
243+
| ObjectRef | string | Reference to the Kubernetes object being accessed |
244+
| ResponseCode | int | HTTP response code for the API request |
245+
| Stage | string | Stage of the request processing (RequestReceived, ResponseStarted, ResponseComplete, Panic) |
246+
| AuthDecision | string | Authorization decision made by the API server |
247+
| RawEvent | dynamic | Complete raw audit event data for advanced analysis |
248+
249+
## Troubleshooting
250+
251+
### Common issues and solutions
252+
253+
- **No data appears in AWSEKSLogs_CL table:**
254+
- Verify that EKS audit logging is enabled on your cluster.
255+
- Check that the CloudWatch Logs subscription filter is correctly configured.
256+
- Ensure the Lambda function processes logs without errors. Check CloudWatch Logs for Lambda function logs.
257+
- Verify that S3 bucket notifications are configured correctly to trigger SQS messages.
258+
259+
- **CloudFormation stack creation fails:**
260+
- Ensure you have sufficient IAM permissions to create all required resources.
261+
- Check that the EKS cluster name you provided exists in your account.
262+
- Verify that the S3 bucket name is globally unique.
263+
264+
- **Authentication errors:**
265+
- Verify that the OIDC web identity provider is correctly configured.
266+
- Ensure the IAM role permissions are sufficient for accessing S3 and SQS resources.
267+
- Check that the workspace ID used in the CloudFormation template matches your Microsoft Sentinel workspace.
268+
269+
### Advanced monitoring
270+
271+
If you haven't already done so, implement data connector health monitoring so that you can know when connectors aren't receiving data or have other issues. For more information, see [Monitor the health of your data connectors](monitor-data-connector-health.md).
272+
273+
## Next steps
274+
275+
In this document, you learned how to connect AWS EKS audit logs to Microsoft Sentinel for comprehensive Kubernetes security monitoring. To learn more about Microsoft Sentinel, see the following articles:
276+
277+
- Learn how to [get visibility into your data, and potential threats](get-visibility.md).
278+
- Get started [detecting threats with Microsoft Sentinel](detect-threats-built-in.md).
279+
- [Use workbooks](monitor-your-data.md) to monitor your data.
280+
- Learn about [Microsoft Sentinel solutions for container security](sentinel-solutions-catalog.md).

0 commit comments

Comments
 (0)