Skip to content

Commit 07239df

Browse files
prmerger-automator[bot]prmerger-automator[bot]
authored
Merge pull request #249295 from halkazwini/nw-ta-schema2
Add new fields and cancel the tabs for the first table
2 parents 79af29c + b3b5598 commit 07239df

File tree

1 file changed

+49
-40
lines changed

1 file changed

+49
-40
lines changed

‎articles/network-watcher/traffic-analytics-schema.md

Lines changed: 49 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@ author: halkazwini
66
ms.author: halkazwini
77
ms.service: network-watcher
88
ms.topic: concept-article
9-
ms.date: 08/16/2023
9+
ms.date: 08/24/2023
10+
#CustomerIntent: As a administrator, I want learn about traffic analytics schema so I can easily use the queries and understand their output.
1011
---
1112

1213
# Schema and data aggregation in Azure Network Watcher traffic analytics
@@ -92,9 +93,9 @@ https://{storageAccountName}@insights-logs-networksecuritygroupflowevent/resoure
9293

9394
Traffic analytics is built on top of Azure Monitor logs, so you can run custom queries on data decorated by traffic analytics and set alerts.
9495

95-
The following table lists the fields in the schema and what they signify.
96+
### NSG flow logs
9697

97-
# [**NSG flow logs**](#tab/nsg)
98+
The following table lists the fields in the schema and what they signify for NSG flow logs.
9899

99100
| Field | Format | Comments |
100101
| ----- | ------ | -------- |
@@ -115,7 +116,7 @@ The following table lists the fields in the schema and what they signify.
115116
| **L7Protocol_s** | Protocol Name | Derived from destination port. |
116117
| **FlowDirection_s** | - I = Inbound <br> - O = Outbound | Direction of the flow: in or out of network security group per flow log. |
117118
| **FlowStatus_s** | - A = Allowed <br> - D = Denied | Status of flow whether allowed or denied by the network security group per flow log. |
118-
| **NSGList_s** | \<SUBSCRIPTIONID>\/<RESOURCEGROUP_NAME>\/<NSG_NAME> | Network security group associated with the flow. |
119+
| **NSGList_s** | \<SUBSCRIPTIONID\>/\<RESOURCEGROUP_NAME\>/\<NSG_NAME\> | Network security group associated with the flow. |
119120
| **NSGRules_s** | \<Index value 0>\|\<NSG_RULENAME>\|\<Flow Direction>\|\<Flow Status>\|\<FlowCount ProcessedByRule> | Network security group rule that allowed or denied this flow. |
120121
| **NSGRule_s** | NSG_RULENAME | Network security group rule that allowed or denied this flow. |
121122
| **NSGRuleType_s** | - User Defined <br> - Default | The type of network security group rule used by the flow. |
@@ -126,23 +127,26 @@ The following table lists the fields in the schema and what they signify.
126127
| **Region_s** | Azure region of virtual network / network interface / virtual machine that the IP in the flow belongs to. | Applicable only for FlowType = S2S, P2S, AzurePublic, ExternalPublic, MaliciousFlow, and UnknownPrivate flow types (flow types where only one side is Azure). |
127128
| **Region1_s** | Azure Region | Azure region of virtual network / network interface / virtual machine that the source IP in the flow belongs to. |
128129
| **Region2_s** | Azure Region | Azure region of virtual network that the destination IP in the flow belongs to. |
129-
| **NIC_s** | \<resourcegroup_Name>\/\<NetworkInterfaceName> | NIC associated with the VM sending or receiving the traffic. |
130-
| **NIC1_s** | <resourcegroup_Name>/\<NetworkInterfaceName> | NIC associated with the source IP in the flow. |
131-
| **NIC2_s** | <resourcegroup_Name>/\<NetworkInterfaceName> | NIC associated with the destination IP in the flow. |
132-
| **VM_s** | <resourcegroup_Name>\/\<NetworkInterfaceName> | Virtual Machine associated with the Network interface NIC_s. |
133-
| **VM1_s** | <resourcegroup_Name>/\<VirtualMachineName> | Virtual Machine associated with the source IP in the flow. |
134-
| **VM2_s** | <resourcegroup_Name>/\<VirtualMachineName> | Virtual Machine associated with the destination IP in the flow. |
135-
| **Subnet_s** | <ResourceGroup_Name>/<VirtualNetwork_Name>/\<SubnetName> | Subnet associated with the NIC_s. |
136-
| **Subnet1_s** | <ResourceGroup_Name>/<VirtualNetwork_Name>/\<SubnetName> | Subnet associated with the Source IP in the flow. |
137-
| **Subnet2_s** | <ResourceGroup_Name>/<VirtualNetwork_Name>/\<SubnetName> | Subnet associated with the Destination IP in the flow. |
138-
| **ApplicationGateway1_s** | \<SubscriptionID>/\<ResourceGroupName>/\<ApplicationGatewayName> | Application gateway associated with the Source IP in the flow. |
139-
| **ApplicationGateway2_s** | \<SubscriptionID>/\<ResourceGroupName>/\<ApplicationGatewayName> | Application gateway associated with the Destination IP in the flow. |
140-
| **LoadBalancer1_s** | \<SubscriptionID>/\<ResourceGroupName>/\<LoadBalancerName> | Load balancer associated with the Source IP in the flow. |
141-
| **LoadBalancer2_s** | \<SubscriptionID>/\<ResourceGroupName>/\<LoadBalancerName> | Load balancer associated with the Destination IP in the flow. |
142-
| **LocalNetworkGateway1_s** | \<SubscriptionID>/\<ResourceGroupName>/\<LocalNetworkGatewayName> | Local network gateway associated with the Source IP in the flow. |
143-
| **LocalNetworkGateway2_s** | \<SubscriptionID>/\<ResourceGroupName>/\<LocalNetworkGatewayName> | Local network gateway associated with the Destination IP in the flow. |
144-
| **ConnectionType_s** | Possible values are VNetPeering, VpnGateway, and ExpressRoute | Connection Type. |
145-
| **ConnectionName_s** | \<SubscriptionID>/\<ResourceGroupName>/\<ConnectionName> | Connection Name. For flow type P2S, it is formatted as \<gateway name\>_\<VPN Client IP\>. |
130+
| **NIC_s** | \<resourcegroup_Name\>/\<NetworkInterfaceName\> | NIC associated with the VM sending or receiving the traffic. |
131+
| **NIC1_s** | \<resourcegroup_Name\>/\<NetworkInterfaceName\> | NIC associated with the source IP in the flow. |
132+
| **NIC2_s** | \<resourcegroup_Name\>/\<NetworkInterfaceName> | NIC associated with the destination IP in the flow. |
133+
| **VM_s** | \<resourcegroup_Name\>/\<NetworkInterfaceName\> | Virtual Machine associated with the Network interface NIC_s. |
134+
| **VM1_s** | \<resourcegroup_Name\>/\<VirtualMachineName\> | Virtual Machine associated with the source IP in the flow. |
135+
| **VM2_s** | \<resourcegroup_Name\>/\<VirtualMachineName\> | Virtual Machine associated with the destination IP in the flow. |
136+
| **Subnet_s** | \<ResourceGroup_Name\>/\<VirtualNetwork_Name\>/\<SubnetName\> | Subnet associated with the NIC_s. |
137+
| **Subnet1_s** | \<ResourceGroup_Name\>/\<VirtualNetwork_Name\>/\<SubnetName\> | Subnet associated with the Source IP in the flow. |
138+
| **Subnet2_s** | \<ResourceGroup_Name\>/<VirtualNetwork_Name\>/\<SubnetName\> | Subnet associated with the Destination IP in the flow. |
139+
| **ApplicationGateway1_s** | \<SubscriptionID\>/\<ResourceGroupName\>/\<ApplicationGatewayName\> | Application gateway associated with the Source IP in the flow. |
140+
| **ApplicationGateway2_s** | \<SubscriptionID\>/\<ResourceGroupName\>/\<ApplicationGatewayName\> | Application gateway associated with the Destination IP in the flow. |
141+
| **ExpressRouteCircuit1** | \<SubscriptionID\>/\<ResourceGroupName\>/\<ExpressRouteCircuitName\> | ExpressRoute circuit ID - when flow is sent from site via ExpressRoute. |
142+
| **ExpressRouteCircuit2** | \<SubscriptionID\>/\<ResourceGroupName\>/\<ExpressRouteCircuitName\> | ExpressRoute circuit ID - when flow is received from cloud by ExpressRoute. |
143+
| **ExpressRouteCircuitPeeringType** | - AzurePrivatePeering <br> - AzurePublicPeering <br> - MicrosoftPeering | ExpressRoute peering type involved in the flow. |
144+
| **LoadBalancer1_s** | \<SubscriptionID\>/\<ResourceGroupName\>/\<LoadBalancerName\> | Load balancer associated with the Source IP in the flow. |
145+
| **LoadBalancer2_s** | \<SubscriptionID\>/\<ResourceGroupName\>/\<LoadBalancerName\> | Load balancer associated with the Destination IP in the flow. |
146+
| **LocalNetworkGateway1_s** | \<SubscriptionID\>/\<ResourceGroupName\>/\<LocalNetworkGatewayName\> | Local network gateway associated with the Source IP in the flow. |
147+
| **LocalNetworkGateway2_s** | \<SubscriptionID\>/\<ResourceGroupName\>/\<LocalNetworkGatewayName\> | Local network gateway associated with the Destination IP in the flow. |
148+
| **ConnectionType_s** | - VNetPeering <br> - VpnGateway <br> - ExpressRoute | The onnection Type. |
149+
| **ConnectionName_s** | \<SubscriptionID\>/\<ResourceGroupName\>/\<ConnectionName\> | The connection Name. For flow type P2S, it is formatted as \<gateway name\>_\<VPN Client IP\>. |
146150
| **ConnectingVNets_s** | Space separated list of virtual network names | In case of hub and spoke topology, hub virtual networks are populated here. |
147151
| **Country_s** | Two letter country code (ISO 3166-1 alpha-2) | Populated for flow type ExternalPublic. All IP addresses in PublicIPs_s field share the same country code. |
148152
| **AzureRegion_s** | Azure region locations | Populated for flow type AzurePublic. All IP addresses in PublicIPs_s field share the Azure region. |
@@ -159,6 +163,7 @@ The following table lists the fields in the schema and what they signify.
159163
| **PublicIPs_s** | <PUBLIC_IP>\|\<FLOW_STARTED_COUNT>\|\<FLOW_ENDED_COUNT>\|\<OUTBOUND_PACKETS>\|\<INBOUND_PACKETS>\|\<OUTBOUND_BYTES>\|\<INBOUND_BYTES> | Entries separated by bars. |
160164
| **SrcPublicIPs_s** | <SOURCE_PUBLIC_IP>\|\<FLOW_STARTED_COUNT>\|\<FLOW_ENDED_COUNT>\|\<OUTBOUND_PACKETS>\|\<INBOUND_PACKETS>\|\<OUTBOUND_BYTES>\|\<INBOUND_BYTES> | Entries separated by bars. |
161165
| **DestPublicIPs_s** | <DESTINATION_PUBLIC_IP>\|\<FLOW_STARTED_COUNT>\|\<FLOW_ENDED_COUNT>\|\<OUTBOUND_PACKETS>\|\<INBOUND_PACKETS>\|\<OUTBOUND_BYTES>\|\<INBOUND_BYTES> | Entries separated by bars. |
166+
| **IsFlowCapturedAtUDRHop** | - True <br> - False | If the flow was captured at a UDR hop, the value is True. |
162167

163168
> [!IMPORTANT]
164169
> The traffic analytics schema was updated on August 22, 2019. The new schema provides source and destination IPs separately, removing the need to parse the `FlowDirection` field so that queries are simpler. The updated schema had the following changes:
@@ -167,7 +172,9 @@ The following table lists the fields in the schema and what they signify.
167172
> - Deprecated fields: `VMIP_s`, `Subscription_s`, `Region_s`, `NSGRules_s`, `Subnet_s`, `VM_s`, `NIC_s`, `PublicIPs_s`, `FlowCount_d`
168173
> - New fields: `SrcPublicIPs_s`, `DestPublicIPs_s`, `NSGRule_s`
169174
170-
# [**VNet flow logs (preview)**](#tab/vnet)
175+
### VNet flow logs (preview)
176+
177+
The following table lists the fields in the schema and what they signify for VNet flow logs.
171178

172179
| Field | Format | Comments |
173180
| ----- | ------ | -------- |
@@ -190,28 +197,31 @@ The following table lists the fields in the schema and what they signify.
190197
| **L7Protocol** | Protocol Name | Derived from destination port. |
191198
| **FlowDirection** | - **I** = Inbound <br> - **O** = Outbound | Direction of the flow: in or out of the network security group per flow log. |
192199
| **FlowStatus** | - **A** = Allowed <br> - **D** = Denied | Status of flow: allowed or denied by network security group per flow log. |
193-
| **NSGList** |\<SUBSCRIPTIONID>/<RESOURCEGROUP_NAME>/<NSG_NAME> | Network security group associated with the flow. |
200+
| **NSGList** | \<SUBSCRIPTIONID\>/\<RESOURCEGROUP_NAME\>/\<NSG_NAME\> | Network security group associated with the flow. |
194201
| **NSGRule** | NSG_RULENAME | Network security group rule that allowed or denied the flow. |
195202
| **NSGRuleType** | - User Defined <br> - Default | The type of network security group rule used by the flow. |
196203
| **MACAddress** | MAC Address | MAC address of the NIC at which the flow was captured. |
197204
| **SrcSubscription** | Subscription ID | Subscription ID of virtual network / network interface / virtual machine that the source IP in the flow belongs to. |
198205
| **DestSubscription** | Subscription ID | Subscription ID of virtual network / network interface / virtual machine that the destination IP in the flow belongs to. |
199206
| **SrcRegion** | Azure Region | Azure region of virtual network / network interface / virtual machine to which the source IP in the flow belongs to. |
200207
| **DestRegion** | Azure Region | Azure region of virtual network to which the destination IP in the flow belongs to. |
201-
| **SecNIC** | <resourcegroup_Name>/\<NetworkInterfaceName> | NIC associated with the source IP in the flow. |
202-
| **DestNIC** | <resourcegroup_Name>/\<NetworkInterfaceName> | NIC associated with the destination IP in the flow. |
203-
| **SrcVM** | <resourcegroup_Name>/\<VirtualMachineName> | Virtual machine associated with the source IP in the flow. |
204-
| **DestVM** | <resourcegroup_Name>/\<VirtualMachineName> | Virtual machine associated with the destination IP in the flow. |
205-
| **SrcSubnet** | <ResourceGroup_Name>/<VirtualNetwork_Name>/\<SubnetName> | Subnet associated with the source IP in the flow. |
206-
| **DestSubnet** | <ResourceGroup_Name>/<VirtualNetwork_Name>/\<SubnetName> | Subnet associated with the destination IP in the flow. |
207-
| **SrcApplicationGateway** | \<SubscriptionID>/\<ResourceGroupName>/\<ApplicationGatewayName> | Application gateway associated with the source IP in the flow. |
208-
| **DestApplicationGateway** | \<SubscriptionID>/\<ResourceGroupName>/\<ApplicationGatewayName> | Application gateway associated with the destination IP in the flow. |
209-
| **SrcLoadBalancer** | \<SubscriptionID>/\<ResourceGroupName>/\<LoadBalancerName> | Load balancer associated with the source IP in the flow. |
210-
| **DestLoadBalancer** | \<SubscriptionID>/\<ResourceGroupName>/\<LoadBalancerName> | Load balancer associated with the destination IP in the flow. |
211-
| **SrcLocalNetworkGateway** | \<SubscriptionID>/\<ResourceGroupName>/\<LocalNetworkGatewayName> | Local network gateway associated with the source IP in the flow. |
212-
| **DestLocalNetworkGateway** | \<SubscriptionID>/\<ResourceGroupName>/\<LocalNetworkGatewayName> | Local network gateway associated with the destination IP in the flow. |
213-
| **ConnectionType** | Possible values are VNetPeering, VpnGateway, and ExpressRoute | The connection type. |
214-
| **ConnectionName** | \<SubscriptionID>/\<ResourceGroupName>/\<ConnectionName> | The connection name. For flow type P2S, it's formatted as \<GatewayName>_\<VPNClientIP> |
208+
| **SecNIC** | \<resourcegroup_Name\>/\<NetworkInterfaceName\> | NIC associated with the source IP in the flow. |
209+
| **DestNIC** | \<resourcegroup_Name\>/\<NetworkInterfaceName\> | NIC associated with the destination IP in the flow. |
210+
| **SrcVM** | \<resourcegroup_Name\>/\<VirtualMachineName\> | Virtual machine associated with the source IP in the flow. |
211+
| **DestVM** | \<resourcegroup_Name\>/\<VirtualMachineName\> | Virtual machine associated with the destination IP in the flow. |
212+
| **SrcSubnet** | \<ResourceGroup_Name\>/\<VirtualNetwork_Name\>/\<SubnetName\> | Subnet associated with the source IP in the flow. |
213+
| **DestSubnet** | \<ResourceGroup_Name\>/\<VirtualNetwork_Name\>/\<SubnetName\> | Subnet associated with the destination IP in the flow. |
214+
| **SrcApplicationGateway** | \<SubscriptionID\>/\<ResourceGroupName\>/\<ApplicationGatewayName\> | Application gateway associated with the source IP in the flow. |
215+
| **DestApplicationGateway** | \<SubscriptionID\>/\<ResourceGroupName\>/\<ApplicationGatewayName\> | Application gateway associated with the destination IP in the flow. |
216+
| **SrcExpressRouteCircuit** | \<SubscriptionID\>/\<ResourceGroupName\>/\<ExpressRouteCircuitName\> | ExpressRoute circuit ID - when flow is sent from site via ExpressRoute. |
217+
| **DestExpressRouteCircuit** | \<SubscriptionID\>/\<ResourceGroupName\>/\<ExpressRouteCircuitName\> | ExpressRoute circuit ID - when flow is received from cloud by ExpressRoute. |
218+
| **ExpressRouteCircuitPeeringType** | - AzurePrivatePeering <br> - AzurePublicPeering <br> - MicrosoftPeering | ExpressRoute peering type involved in the flow. |
219+
| **SrcLoadBalancer** | \<SubscriptionID\>/\<ResourceGroupName\>/\<LoadBalancerName\> | Load balancer associated with the source IP in the flow. |
220+
| **DestLoadBalancer** | \<SubscriptionID\>/\<ResourceGroupName\>/\<LoadBalancerName\> | Load balancer associated with the destination IP in the flow. |
221+
| **SrcLocalNetworkGateway** | \<SubscriptionID\>/\<ResourceGroupName\>/\<LocalNetworkGatewayName\> | Local network gateway associated with the source IP in the flow. |
222+
| **DestLocalNetworkGateway** | \<SubscriptionID\>/\<ResourceGroupName\>/\<LocalNetworkGatewayName\> | Local network gateway associated with the destination IP in the flow. |
223+
| **ConnectionType** | - VNetPeering <br> - VpnGateway <br> - ExpressRoute | The connection type. |
224+
| **ConnectionName** | \<SubscriptionID\>/\<ResourceGroupName\>/\<ConnectionName\> | The connection name. For flow type P2S, it's formatted as \<GatewayName>_\<VPNClientIP> |
215225
| **ConnectingVNets** | Space separated list of virtual network names. | In hub and spoke topology, hub virtual networks are populated here. |
216226
| **Country** | Two-letter country code (ISO 3166-1 alpha-2) | Populated for flow type ExternalPublic. All IP addresses in PublicIPs field share the same country code. |
217227
| **AzureRegion** | Azure region locations | Populated for flow type AzurePublic. All IP addresses in PublicIPs field share the Azure region. |
@@ -225,15 +235,14 @@ The following table lists the fields in the schema and what they signify.
225235
| **BytesDestToSrc** | Represents bytes sent from the destination to the source of the flow | Populated only for the Version 2 of NSG flow log schema. |
226236
| **BytesSrcToDest** | Represents bytes sent from the source to the destination of the flow | Populated only for the Version 2 of NSG flow log schema. |
227237
| **CompletedFlows** | - | Populated with nonzero value only for the Version 2 of NSG flow log schema. |
228-
| **SrcPublicIPs** | <SOURCE_PUBLIC_IP>\|\<FLOW_STARTED_COUNT>\|\<FLOW_ENDED_COUNT>\|\<OUTBOUND_PACKETS>\|\<INBOUND_PACKETS>\|\<OUTBOUND_BYTES>\|\<INBOUND_BYTES> | Entries separated by bars. |
238+
| **SrcPublicIPs** | \<SOURCE_PUBLIC_IP\>\|\<FLOW_STARTED_COUNT\>\|\<FLOW_ENDED_COUNT\>\|\<OUTBOUND_PACKETS\>\|\<INBOUND_PACKETS\>\|\<OUTBOUND_BYTES\>\|\<INBOUND_BYTES\> | Entries separated by bars. |
229239
| **DestPublicIPs** | <DESTINATION_PUBLIC_IP>\|\<FLOW_STARTED_COUNT>\|\<FLOW_ENDED_COUNT>\|\<OUTBOUND_PACKETS>\|\<INBOUND_PACKETS>\|\<OUTBOUND_BYTES>\|\<INBOUND_BYTES> | Entries separated by bars. |
230240
| **FlowEncryption** | - Encrypted <br>- Unencrypted <br>- Unsupported hardware <br>- Software not ready <br>- Drop due to no encryption <br>- Discovery not supported <br>- Destination on same host <br>- Fall back to no encryption. | Encryption level of flows. |
241+
| **IsFlowCapturedAtUDRHop** | - True <br> - False | If the flow was captured at a UDR hop, the value is True. |
231242

232243
> [!NOTE]
233244
> *NTANetAnalytics* in VNet flow logs replaces *AzureNetworkAnalytics_CL* used in NSG flow logs.
234245
235-
---
236-
237246
## Public IP details schema
238247

239248
Traffic analytics provides WHOIS data and geographic location for all public IPs in your environment. For a malicious IP, traffic analytics provides DNS domain, threat type and thread descriptions as identified by Microsoft security intelligence solutions. IP Details are published to your Log Analytics workspace so you can create custom queries and put alerts on them. You can also access prepopulated queries from the traffic analytics dashboard.

0 commit comments

Comments
 (0)