Skip to content

Commit 10fde9a

Browse files
ttorblettorble
authored
Merge pull request #310925 from EdB-MSFT/what-to-log-in-lake
what to ingest into the lake
2 parents 5596feb + 22f1a3d commit 10fde9a

1 file changed

Lines changed: 74 additions & 0 deletions

File tree

‎articles/sentinel/datalake/sentinel-lake-connectors.md‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,80 @@ Tables created by using the Logs Ingestion API or Azure Monitor Agent (AMA) and
4949

5050
When you onboard to both Microsoft Defender and Microsoft Sentinel and then onboard to the data lake, you no longer see auxiliary log tables in Microsoft Defender’s Advanced hunting or in the Microsoft Sentinel Azure portal. The auxiliary table data is available in the data lake and you can query it by using KQL queries or Jupyter notebooks. Find KQL queries under **Microsoft Sentinel** > **Data lake exploration** in the Defender portal.
5151

52+
## Direct ingestion to the data lake tier
53+
54+
Depending on your organization's security needs, you might choose to ingest some log sources directly into the data
55+
lake. Directly ingesting logs to the data lake allows you to better manage costs by optimizing data retention and storage based on the value of the data for real-time detection versus long-term analysis.
56+
57+
Ingest high-volume logs that are less critical for real-time detection but valuable for deep analysis and forensics directly to the lake, and ingest only high-value logs to the analytics tier. Note that logs ingested to the analytics tier are also mirrored to the data lake.
58+
59+
Use the following table to prioritize which sources you should ingest directly to the data lake versus the analytics tier.
60+
61+
| Log source type | Typical log volume | Value for real-time threat detection and alerting | Value for threat hunting | Value for incident investigation and forensics | Ingest to data lake |
62+
|-------------------------------------------------|--------------------|-------------------------------------|----------------|-----------------------------------|-----------------------|
63+
| AAA (TACACS/Radius) | Medium | High | High | High | Yes |
64+
| Active Directory (on-premises) | High | High | High | High | No |
65+
| Application Logs | High | Medium | Medium | High | Yes |
66+
| AV Logs (Windows Events 5000s & 3rd party) | Medium | High | High | High | No |
67+
| Azure Activity | Medium | High | High | High | No |
68+
| Biometric Access System Logs | Low | Medium | Low | High | Yes |
69+
| Building Security System Logs | Low | Low | Low | Medium | Yes |
70+
| Call Center/VoIP Logs | Medium | Low | Low | Medium | Yes |
71+
| CASB | High | High | High | High | Yes |
72+
| Citrix/Horizon/ALBs | Medium | Medium | Medium | High | Yes |
73+
| Cloud IAM | Medium | High | High | High | No |
74+
| Cloud PaaS | High | High | High | High | Yes |
75+
| Cloud Security Controls | Medium | High | Medium | High | No |
76+
| Cloud Storage (S3, Blob, etc.) Logs | High | High | High | High | No |
77+
| CRM Audit Logs | Low-Medium | Low | Low | Medium | Yes |
78+
| Database Audit Tools | Medium | High | High | High | Yes |
79+
| DHCP Logs | Medium | Medium | Medium | High | Yes |
80+
| DLP Alerts | Low | High | High | High | Yes |
81+
| DNS Logs | High | High | High | High | Yes |
82+
| Endpoint Detection and Response (EDR) (Alerts) | Medium | High | High | High | No |
83+
| Endpoint Detection and Response (EDR) (Raw) | High | High | High | High | Yes |
84+
| Email Security (3rd party alerts) | Medium | High | Medium | High | No |
85+
| ERP Audit Logs | Low-Medium | Low | Low | Medium | Yes |
86+
| File Integrity | Low | Medium | Medium | High | Yes |
87+
| Firewall Threat/Malware/IPS/IDS | High | High | High | High | No |
88+
| Firewall Traffic Logs | High | High | High | High | Yes |
89+
| GitHub/GitLab/Code Repo Logs | Low-Medium | Medium | Medium | High | Yes |
90+
| Google Workspace Logs | Medium | Medium | Medium | High | Yes |
91+
| Identity (Entra ID, Okta, LDAP) | Medium | High | High | High | No |
92+
| IIS/Apache Logs | Medium | High | High | High | Yes |
93+
| IoT Device Logs | High | Medium | Medium | Medium | Yes |
94+
| Kubernetes/Container Logs (alerts, critical) | High | High | High | High | No |
95+
| Kubernetes/Container Logs (raw logs) | High | High | High | High | Yes |
96+
| LAN/WAN Router Switch | High | Medium | Medium | Medium | Yes |
97+
| Linux Server AuditD | Medium | High | High | High | No |
98+
| Mobile Device Management (Intune) | Medium | Medium | Medium | Medium | Yes |
99+
| Microsoft Office Logs (Teams, Office, SharePoint)| Medium | Medium | Medium | High | No |
100+
| Microsoft XDR Alerts (Defender: Office, Identity, Endpoint, CloudApp) | Medium | High | High | High | No |
101+
| Multifactor authentication (MFA) | Medium | High | Medium | High | No |
102+
| Netflow | High | Medium | High | Medium | Yes |
103+
| Network Detection (Corelight, Vectra, Darktrace)| High | High | High | High | No |
104+
| OT/ICS System Logs | Medium | High | High | High | Yes |
105+
| PAM (Privileged Access Management) | Low | High | High | High | No |
106+
| PIM (Privileged Identity Management) | Low | High | High | High | No |
107+
| POS System Logs | High | High | High | High | Yes |
108+
| Proxy Logging (URL filtering) | High | High | High | High | Yes |
109+
| Salesforce Audit Logs | Medium | Medium | Medium | High | Yes |
110+
| SD-WAN | Medium | Medium | Medium | Medium | Yes |
111+
| ServiceNow Audit Logs | Low | Low | Low | Medium | Yes |
112+
| SIEM/SOAR Platform Logs | Medium | High | High | High | No |
113+
| Slack/Teams Collaboration Logs | Medium | Low | Medium | Medium | Yes |
114+
| Sysmon (Endpoint, for EDR complement) | Medium | High | High | High | Yes |
115+
| Threat Intelligence Indicators | Low | High | High | High | No |
116+
| VDI Logs | Medium | Medium | Medium | High | Yes |
117+
| VPN | Medium | High | High | High | No |
118+
| Vulnerability Scanning | Low | Medium | Medium | Medium | Yes |
119+
| Web Application Firewall (WAF) Logs | Medium | High | High | High | Yes |
120+
| Windows Server Events | High | High | High | High | No |
121+
| XDR Source Logs (Defender: Office, Identity, Endpoint, CloudApp) | Medium | High | High | High | No |
122+
| Zoom Meeting Logs | Low-Medium | Low | Low | Medium | Yes |
123+
124+
125+
52126

53127
## Related articles
54128

0 commit comments

Comments
 (0)