Skip to content

Commit e25e8a3

Browse files
Stacyrch140Stacyrch140
authored
Merge pull request #304479 from halkazwini/nw-read
Read flow logs updates
2 parents ee017e3 + d968ad5 commit e25e8a3

1 file changed

Lines changed: 95 additions & 95 deletions

File tree

‎articles/network-watcher/flow-logs-read.md‎

Lines changed: 95 additions & 95 deletions
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,9 @@ author: halkazwini
66
ms.author: halkazwini
77
ms.service: azure-network-watcher
88
ms.topic: how-to
9-
ms.date: 09/26/2024
9+
ms.date: 08/20/2025
1010
ms.custom: devx-track-azurepowershell
1111

12-
#CustomerIntent: As an Azure administrator, I want to read my flow logs using a PowerShell script so I can see the latest data.
1312
# Customer intent: As an Azure administrator, I want to efficiently read and analyze my Network Watcher flow logs using PowerShell scripts, so that I can quickly access and interpret network data without downloading entire log files.
1413
---
1514

@@ -29,39 +28,41 @@ The concepts discussed in this article aren't limited to the PowerShell and are
2928

3029
- Flow logs in a region or more. For more information, see [Create network security group flow logs](nsg-flow-logs-manage.md#create-a-flow-log) or [Create virtual network flow logs](vnet-flow-logs-manage.md#create-a-flow-log).
3130

32-
- Necessary RBAC permissions for the subscriptions of flow logs and storage account. For more information, see [Network Watcher RBAC permissions](required-rbac-permissions.md).
31+
- Necessary Azure role-based access control (Azure RBAC) permissions for the subscriptions of flow logs and storage account. For more information, see [Network Watcher RBAC permissions](required-rbac-permissions.md).
3332

3433
## Retrieve the blocklist
3534

36-
# [**Network security group flow logs**](#tab/nsg)
35+
In this section, you set up the variables required to query the flow log blob and list the blocks within the [CloudBlockBlob](/dotnet/api/microsoft.azure.storage.blob.cloudblockblob) block blob.
36+
37+
# [**Virtual network flow logs**](#tab/vnet)
3738

38-
The following PowerShell script sets up the variables needed to query the network security group flow log blob and list the blocks within the [CloudBlockBlob](/dotnet/api/microsoft.azure.storage.blob.cloudblockblob) block blob. Update the script to contain valid values for your environment, specifically "yourSubscriptionId", "FLOWLOGSVALIDATIONWESTCENTRALUS", "V2VALIDATIONVM-NSG", "yourStorageAccountName", "ml-rg", "000D3AF87856", "11/11/2018 03:00". For example, yourSubscriptionId should be replaced with your subscription ID.
39+
Update the PowerShell script with valid values for your environment.
3940

4041
```powershell
41-
function Get-NSGFlowLogCloudBlockBlob {
42+
function Get-VNetFlowLogCloudBlockBlob {
4243
[CmdletBinding()]
4344
param (
4445
[string] [Parameter(Mandatory=$true)] $subscriptionId,
45-
[string] [Parameter(Mandatory=$true)] $NSGResourceGroupName,
46-
[string] [Parameter(Mandatory=$true)] $NSGName,
46+
[string] [Parameter(Mandatory=$true)] $region,
47+
[string] [Parameter(Mandatory=$true)] $VNetFlowLogName,
4748
[string] [Parameter(Mandatory=$true)] $storageAccountName,
4849
[string] [Parameter(Mandatory=$true)] $storageAccountResourceGroup,
4950
[string] [Parameter(Mandatory=$true)] $macAddress,
5051
[datetime] [Parameter(Mandatory=$true)] $logTime
5152
)
5253
5354
process {
54-
# Retrieve the primary storage account key to access the network security group logs
55+
# Retrieve the primary storage account key to access the virtual network flow logs
5556
$StorageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $storageAccountResourceGroup -Name $storageAccountName).Value[0]
5657
5758
# Setup a new storage context to be used to query the logs
58-
$ctx = New-AzStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
59+
$ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $StorageAccountKey
5960
60-
# Container name used by network security group flow logs
61-
$ContainerName = "insights-logs-networksecuritygroupflowevent"
61+
# Container name used by virtual network flow logs
62+
$ContainerName = "insights-logs-flowlogflowevent"
6263
63-
# Name of the blob that contains the network security group flow log
64-
$BlobName = "resourceId=/SUBSCRIPTIONS/${subscriptionId}/RESOURCEGROUPS/${NSGResourceGroupName}/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/${NSGName}/y=$($logTime.Year)/m=$(($logTime).ToString("MM"))/d=$(($logTime).ToString("dd"))/h=$(($logTime).ToString("HH"))/m=00/macAddress=$($macAddress)/PT1H.json"
64+
# Name of the blob that contains the virtual network flow log
65+
$BlobName = "flowLogResourceID=/$($subscriptionId.ToUpper())_NETWORKWATCHERRG/NETWORKWATCHER_$($region.ToUpper())_$($VNetFlowLogName.ToUpper())/y=$($logTime.Year)/m=$(($logTime).ToString("MM"))/d=$(($logTime).ToString("dd"))/h=$(($logTime).ToString("HH"))/m=00/macAddress=$($macAddress)/PT1H.json"
6566
6667
# Gets the storage blog
6768
$Blob = Get-AzStorageBlob -Context $ctx -Container $ContainerName -Blob $BlobName
@@ -74,7 +75,7 @@ function Get-NSGFlowLogCloudBlockBlob {
7475
}
7576
}
7677
77-
function Get-NSGFlowLogBlockList {
78+
function Get-VNetFlowLogBlockList {
7879
[CmdletBinding()]
7980
param (
8081
[Microsoft.Azure.Storage.Blob.CloudBlockBlob] [Parameter(Mandatory=$true)] $CloudBlockBlob
@@ -88,41 +89,40 @@ function Get-NSGFlowLogBlockList {
8889
}
8990
}
9091
92+
$CloudBlockBlob = Get-VNetFlowLogCloudBlockBlob -subscriptionId "yourSubscriptionId" -region "yourVNetFlowLogRegion" -VNetFlowLogName "yourVNetFlowLogName" -storageAccountName "yourStorageAccountName" -storageAccountResourceGroup "yourStorageAccountRG" -macAddress "0022485D8CF8" -logTime "07/09/2023 03:00"
9193
92-
$CloudBlockBlob = Get-NSGFlowLogCloudBlockBlob -subscriptionId "yourSubscriptionId" -NSGResourceGroupName "FLOWLOGSVALIDATIONWESTCENTRALUS" -NSGName "V2VALIDATIONVM-NSG" -storageAccountName "yourStorageAccountName" -storageAccountResourceGroup "ml-rg" -macAddress "000D3AF87856" -logTime "11/11/2018 03:00"
93-
94-
$blockList = Get-NSGFlowLogBlockList -CloudBlockBlob $CloudBlockBlob
94+
$blockList = Get-VNetFlowLogBlockList -CloudBlockBlob $CloudBlockBlob
9595
```
9696

97-
# [**Virtual network flow logs**](#tab/vnet)
97+
# [**Network security group flow logs**](#tab/nsg)
9898

99-
The following PowerShell script sets up the variables needed to query the virtual network flow log blob and list the blocks within the [CloudBlockBlob](/dotnet/api/microsoft.azure.storage.blob.cloudblockblob) block blob. Update the script to contain valid values for your environment.
99+
Update the PowerShell script with valid values for your environment, specifically `yourSubscriptionId`, `FLOWLOGSVALIDATIONWESTCENTRALUS`, `V2VALIDATIONVM-NSG`, `yourStorageAccountName`, `ml-rg`, `000D3AF87856`, `11/11/2018 03:00`. For example, `yourSubscriptionId` should be replaced with your Azure subscription ID.
100100

101101
```powershell
102-
function Get-VNetFlowLogCloudBlockBlob {
102+
function Get-NSGFlowLogCloudBlockBlob {
103103
[CmdletBinding()]
104104
param (
105105
[string] [Parameter(Mandatory=$true)] $subscriptionId,
106-
[string] [Parameter(Mandatory=$true)] $region,
107-
[string] [Parameter(Mandatory=$true)] $VNetFlowLogName,
106+
[string] [Parameter(Mandatory=$true)] $NSGResourceGroupName,
107+
[string] [Parameter(Mandatory=$true)] $NSGName,
108108
[string] [Parameter(Mandatory=$true)] $storageAccountName,
109109
[string] [Parameter(Mandatory=$true)] $storageAccountResourceGroup,
110110
[string] [Parameter(Mandatory=$true)] $macAddress,
111111
[datetime] [Parameter(Mandatory=$true)] $logTime
112112
)
113113
114114
process {
115-
# Retrieve the primary storage account key to access the virtual network flow logs
115+
# Retrieve the primary storage account key to access the network security group logs
116116
$StorageAccountKey = (Get-AzStorageAccountKey -ResourceGroupName $storageAccountResourceGroup -Name $storageAccountName).Value[0]
117117
118118
# Setup a new storage context to be used to query the logs
119-
$ctx = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $StorageAccountKey
119+
$ctx = New-AzStorageContext -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey
120120
121-
# Container name used by virtual network flow logs
122-
$ContainerName = "insights-logs-flowlogflowevent"
121+
# Container name used by network security group flow logs
122+
$ContainerName = "insights-logs-networksecuritygroupflowevent"
123123
124-
# Name of the blob that contains the virtual network flow log
125-
$BlobName = "flowLogResourceID=/$($subscriptionId.ToUpper())_NETWORKWATCHERRG/NETWORKWATCHER_$($region.ToUpper())_$($VNetFlowLogName.ToUpper())/y=$($logTime.Year)/m=$(($logTime).ToString("MM"))/d=$(($logTime).ToString("dd"))/h=$(($logTime).ToString("HH"))/m=00/macAddress=$($macAddress)/PT1H.json"
124+
# Name of the blob that contains the network security group flow log
125+
$BlobName = "resourceId=/SUBSCRIPTIONS/${subscriptionId}/RESOURCEGROUPS/${NSGResourceGroupName}/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/${NSGName}/y=$($logTime.Year)/m=$(($logTime).ToString("MM"))/d=$(($logTime).ToString("dd"))/h=$(($logTime).ToString("HH"))/m=00/macAddress=$($macAddress)/PT1H.json"
126126
127127
# Gets the storage blog
128128
$Blob = Get-AzStorageBlob -Context $ctx -Container $ContainerName -Blob $BlobName
@@ -135,7 +135,7 @@ function Get-VNetFlowLogCloudBlockBlob {
135135
}
136136
}
137137
138-
function Get-VNetFlowLogBlockList {
138+
function Get-NSGFlowLogBlockList {
139139
[CmdletBinding()]
140140
param (
141141
[Microsoft.Azure.Storage.Blob.CloudBlockBlob] [Parameter(Mandatory=$true)] $CloudBlockBlob
@@ -149,9 +149,10 @@ function Get-VNetFlowLogBlockList {
149149
}
150150
}
151151
152-
$CloudBlockBlob = Get-VNetFlowLogCloudBlockBlob -subscriptionId "yourSubscriptionId" -region "yourVNetFlowLogRegion" -VNetFlowLogName "yourVNetFlowLogName" -storageAccountName "yourStorageAccountName" -storageAccountResourceGroup "yourStorageAccountRG" -macAddress "0022485D8CF8" -logTime "07/09/2023 03:00"
153152
154-
$blockList = Get-VNetFlowLogBlockList -CloudBlockBlob $CloudBlockBlob
153+
$CloudBlockBlob = Get-NSGFlowLogCloudBlockBlob -subscriptionId "yourSubscriptionId" -NSGResourceGroupName "FLOWLOGSVALIDATIONWESTCENTRALUS" -NSGName "V2VALIDATIONVM-NSG" -storageAccountName "yourStorageAccountName" -storageAccountResourceGroup "ml-rg" -macAddress "000D3AF87856" -logTime "11/11/2018 03:00"
154+
155+
$blockList = Get-NSGFlowLogBlockList -CloudBlockBlob $CloudBlockBlob
155156
```
156157

157158
---
@@ -172,23 +173,25 @@ Mzk1YzQwM2U0ZWY1ZDRhOWFlMTNhYjQ3OGVhYmUzNjk= 2675 True
172173
ZjAyZTliYWE3OTI1YWZmYjFmMWI0MjJhNzMxZTI4MDM= 2 True
173174
```
174175

175-
176176
## Read the block blob
177177

178178
In this section, you read the `$blocklist` variable to retrieve the data. In the following example, we iterate through the blocklist to read the bytes from each block and store them in an array. Use the [DownloadRangeToByteArray](/dotnet/api/microsoft.azure.storage.blob.cloudblob.downloadrangetobytearray) method to retrieve the data.
179179

180-
# [**Network security group flow logs**](#tab/nsg)
180+
# [**Virtual network flow logs**](#tab/vnet)
181181

182182
```powershell
183-
function Get-NSGFlowLogReadBlock {
183+
function Get-VNetFlowLogReadBlock {
184184
[CmdletBinding()]
185185
param (
186186
[System.Array] [Parameter(Mandatory=$true)] $blockList,
187187
[Microsoft.Azure.Storage.Blob.CloudBlockBlob] [Parameter(Mandatory=$true)] $CloudBlockBlob
188188
189189
)
190+
$blocklistResult = $blockList.Result
191+
190192
# Set the size of the byte array to the largest block
191-
$maxvalue = ($blocklist | measure Length -Maximum).Maximum
193+
$maxvalue = ($blocklistResult | Measure-Object Length -Maximum).Maximum
194+
Write-Host "Max value is ${maxvalue}"
192195
193196
# Create an array to store values in
194197
$valuearray = @()
@@ -197,16 +200,16 @@ function Get-NSGFlowLogReadBlock {
197200
$index = 0
198201
199202
# Loop through each block in the block list
200-
for($i=0; $i -lt $blocklist.count; $i++)
203+
for($i=0; $i -lt $blocklistResult.count; $i++)
201204
{
202205
# Create a byte array object to story the bytes from the block
203206
$downloadArray = New-Object -TypeName byte[] -ArgumentList $maxvalue
204207
205208
# Download the data into the ByteArray, starting with the current index, for the number of bytes in the current block. Index is increased by 3 when reading to remove preceding comma.
206-
$CloudBlockBlob.DownloadRangeToByteArray($downloadArray,0,$index, $($blockList[$i].Length)) | Out-Null
209+
$CloudBlockBlob.DownloadRangeToByteArray($downloadArray,0,$index, $($blockListResult[$i].Length)) | Out-Null
207210
208211
# Increment the index by adding the current block length to the previous index
209-
$index = $index + $blockList[$i].Length
212+
$index = $index + $blockListResult[$i].Length
210213
211214
# Retrieve the string from the byte array
212215
@@ -218,24 +221,22 @@ function Get-NSGFlowLogReadBlock {
218221
#Return the Array
219222
$valuearray
220223
}
221-
$valuearray = Get-NSGFlowLogReadBlock -blockList $blockList -CloudBlockBlob $CloudBlockBlob
224+
225+
$valuearray = Get-VNetFlowLogReadBlock -blockList $blockList -CloudBlockBlob $CloudBlockBlob
222226
```
223227

224-
# [**Virtual network flow logs**](#tab/vnet)
228+
# [**Network security group flow logs**](#tab/nsg)
225229

226230
```powershell
227-
function Get-VNetFlowLogReadBlock {
231+
function Get-NSGFlowLogReadBlock {
228232
[CmdletBinding()]
229233
param (
230234
[System.Array] [Parameter(Mandatory=$true)] $blockList,
231235
[Microsoft.Azure.Storage.Blob.CloudBlockBlob] [Parameter(Mandatory=$true)] $CloudBlockBlob
232236
233237
)
234-
$blocklistResult = $blockList.Result
235-
236238
# Set the size of the byte array to the largest block
237-
$maxvalue = ($blocklistResult | Measure-Object Length -Maximum).Maximum
238-
Write-Host "Max value is ${maxvalue}"
239+
$maxvalue = ($blocklist | measure Length -Maximum).Maximum
239240
240241
# Create an array to store values in
241242
$valuearray = @()
@@ -244,16 +245,16 @@ function Get-VNetFlowLogReadBlock {
244245
$index = 0
245246
246247
# Loop through each block in the block list
247-
for($i=0; $i -lt $blocklistResult.count; $i++)
248+
for($i=0; $i -lt $blocklist.count; $i++)
248249
{
249250
# Create a byte array object to story the bytes from the block
250251
$downloadArray = New-Object -TypeName byte[] -ArgumentList $maxvalue
251252
252253
# Download the data into the ByteArray, starting with the current index, for the number of bytes in the current block. Index is increased by 3 when reading to remove preceding comma.
253-
$CloudBlockBlob.DownloadRangeToByteArray($downloadArray,0,$index, $($blockListResult[$i].Length)) | Out-Null
254+
$CloudBlockBlob.DownloadRangeToByteArray($downloadArray,0,$index, $($blockList[$i].Length)) | Out-Null
254255
255256
# Increment the index by adding the current block length to the previous index
256-
$index = $index + $blockListResult[$i].Length
257+
$index = $index + $blockList[$i].Length
257258
258259
# Retrieve the string from the byte array
259260
@@ -265,8 +266,7 @@ function Get-VNetFlowLogReadBlock {
265266
#Return the Array
266267
$valuearray
267268
}
268-
269-
$valuearray = Get-VNetFlowLogReadBlock -blockList $blockList -CloudBlockBlob $CloudBlockBlob
269+
$valuearray = Get-NSGFlowLogReadBlock -blockList $blockList -CloudBlockBlob $CloudBlockBlob
270270
```
271271

272272
---
@@ -275,6 +275,50 @@ The `$valuearray` array contains now the string value of each block. To verify t
275275

276276
The results of this value are shown in the following example:
277277

278+
# [**Virtual network flow logs**](#tab/vnet)
279+
280+
```json
281+
{
282+
"time": "2023-07-09T03:59:30.2837112Z",
283+
"flowLogVersion": 4,
284+
"flowLogGUID": "abcdef01-2345-6789-0abc-def012345678",
285+
"macAddress": "0022485D8CF8",
286+
"category": "FlowLogFlowEvent",
287+
"flowLogResourceID": "/SUBSCRIPTIONS/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e/RESOURCEGROUPS/NETWORKWATCHERRG/PROVIDERS/MICROSOFT.NETWORK/NETWORKWATCHERS/NETWORKWATCHER_EASTUS/FLOWLOGS/MYVNET-MYRESOURCEGROUP-FLOWLOG",
288+
"targetResourceID": "/subscriptions/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e/resourceGroups/myResourceGroup/providers/Microsoft.Network/virtualNetworks/myVNet",
289+
"operationName": "FlowLogFlowEvent",
290+
"flowRecords": {
291+
"flows": [
292+
{
293+
"aclID": "00000000-1234-abcd-ef00-c1c2c3c4c5c6",
294+
"flowGroups": [
295+
{
296+
"rule": "BlockHighRiskTCPPortsFromInternet",
297+
"flowTuples": [
298+
"1688875131557,45.119.212.87,192.168.0.4,53018,3389,6,I,D,NX,0,0,0,0"
299+
]
300+
},
301+
{
302+
"rule": "Internet",
303+
"flowTuples": [
304+
"1688875103311,35.203.210.145,192.168.0.4,56688,52113,6,I,D,NX,0,0,0,0",
305+
"1688875119073,162.216.150.87,192.168.0.4,50111,9920,6,I,D,NX,0,0,0,0",
306+
"1688875119910,205.210.31.253,192.168.0.4,54699,1801,6,I,D,NX,0,0,0,0",
307+
"1688875121510,35.203.210.49,192.168.0.4,49250,33013,6,I,D,NX,0,0,0,0",
308+
"1688875121684,162.216.149.206,192.168.0.4,49776,1290,6,I,D,NX,0,0,0,0",
309+
"1688875124012,91.148.190.134,192.168.0.4,57963,40544,6,I,D,NX,0,0,0,0",
310+
"1688875138568,35.203.211.204,192.168.0.4,51309,46956,6,I,D,NX,0,0,0,0",
311+
"1688875142490,205.210.31.18,192.168.0.4,54140,30303,6,I,D,NX,0,0,0,0",
312+
"1688875147864,194.26.135.247,192.168.0.4,53583,20232,6,I,D,NX,0,0,0,0"
313+
]
314+
}
315+
]
316+
}
317+
]
318+
}
319+
}
320+
```
321+
278322
# [**Network security group flow logs**](#tab/nsg)
279323

280324
```json
@@ -336,50 +380,6 @@ The results of this value are shown in the following example:
336380
}
337381
```
338382

339-
# [**Virtual network flow logs**](#tab/vnet)
340-
341-
```json
342-
{
343-
"time": "2023-07-09T03:59:30.2837112Z",
344-
"flowLogVersion": 4,
345-
"flowLogGUID": "abcdef01-2345-6789-0abc-def012345678",
346-
"macAddress": "0022485D8CF8",
347-
"category": "FlowLogFlowEvent",
348-
"flowLogResourceID": "/SUBSCRIPTIONS/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e/RESOURCEGROUPS/NETWORKWATCHERRG/PROVIDERS/MICROSOFT.NETWORK/NETWORKWATCHERS/NETWORKWATCHER_EASTUS/FLOWLOGS/MYVNET-MYRESOURCEGROUP-FLOWLOG",
349-
"targetResourceID": "/subscriptions/aaaa0a0a-bb1b-cc2c-dd3d-eeeeee4e4e4e/resourceGroups/myResourceGroup/providers/Microsoft.Network/virtualNetworks/myVNet",
350-
"operationName": "FlowLogFlowEvent",
351-
"flowRecords": {
352-
"flows": [
353-
{
354-
"aclID": "00000000-1234-abcd-ef00-c1c2c3c4c5c6",
355-
"flowGroups": [
356-
{
357-
"rule": "BlockHighRiskTCPPortsFromInternet",
358-
"flowTuples": [
359-
"1688875131557,45.119.212.87,192.168.0.4,53018,3389,6,I,D,NX,0,0,0,0"
360-
]
361-
},
362-
{
363-
"rule": "Internet",
364-
"flowTuples": [
365-
"1688875103311,35.203.210.145,192.168.0.4,56688,52113,6,I,D,NX,0,0,0,0",
366-
"1688875119073,162.216.150.87,192.168.0.4,50111,9920,6,I,D,NX,0,0,0,0",
367-
"1688875119910,205.210.31.253,192.168.0.4,54699,1801,6,I,D,NX,0,0,0,0",
368-
"1688875121510,35.203.210.49,192.168.0.4,49250,33013,6,I,D,NX,0,0,0,0",
369-
"1688875121684,162.216.149.206,192.168.0.4,49776,1290,6,I,D,NX,0,0,0,0",
370-
"1688875124012,91.148.190.134,192.168.0.4,57963,40544,6,I,D,NX,0,0,0,0",
371-
"1688875138568,35.203.211.204,192.168.0.4,51309,46956,6,I,D,NX,0,0,0,0",
372-
"1688875142490,205.210.31.18,192.168.0.4,54140,30303,6,I,D,NX,0,0,0,0",
373-
"1688875147864,194.26.135.247,192.168.0.4,53583,20232,6,I,D,NX,0,0,0,0"
374-
]
375-
}
376-
]
377-
}
378-
]
379-
}
380-
}
381-
```
382-
383383
---
384384

385385
## Related content

0 commit comments

Comments
 (0)