Merge pull request #8816 from v-lianna/CI_5271

AB#5271 Create diagnose-packet-loss.md

Author: Simonx Xu

support/windows-client/networking/diagnose-packet-loss.md

@@ -0,0 +1,120 @@
+---
+title: Diagnose Packet Loss
+description: Learn how to troubleshoot TCP/IP packet loss.
+ms.date: 04/30/2025
+ms.topic: troubleshooting
+manager: dcscontentpm
+ms.collection: highpri
+ms.custom:
+- sap:network connectivity and file sharing\tcp/ip connectivity (tcp protocol,nla,winhttp)
+- pcy:WinComm Networking
+ms.reviewer: kaushik, mifriese, guhetier, v-lianna
+audience: itpro
+---
+# Diagnose packet loss
+
+Packet loss occurs whenever a network packet doesn't reach its intended destination. Some packet loss is normal, and doesn't always cause higher-level networking issues. At other times, packet loss can reduce performance, and cause application programming interfaces (APIs) or applications to fail.
+
+## Capture and diagnose packet loss
+
+The first step when conducting a packet loss investigation is to capture [Packet Monitor (Pktmon)](/windows-server/networking/technologies/pktmon/pktmon) traces, typically by using the pktmon command-line workflow. Pktmon can capture packet traces, attribute local packet loss to specific reasons and code locations, and collect packet loss statistics. When combined with [Wireshark](https://www.wireshark.org/) analysis of protocol-level behavior, pktmon traces are sufficient to identify the root causes of many cases of packet loss.
+
+If pktmon diagnostics are inconclusive, more exhaustive component-level traces can be captured using the `netsh.exe trace start scenario=InternetClient` or `netsh.exe trace start scenario=InternetServer` command for client and server scenarios respectively. After the events are captured, stop the trace by using the `netsh.exe trace stop` command. These component-level traces are noisy and not clear, but often contain additional context before or after a local packet is dropped. For remote loss, they might indicate that the local system infers the occurrence of packet loss. The traces can be converted to text using `netsh.exe trace convert nettrace.etl`, opened in [Windows Performance Analyzer](/windows-hardware/test/wpt/windows-performance-analyzer), or used with any other Event Tracing for Windows (ETW) tool.
+
+If the network interface card (NIC) is suspected as a cause of the packet loss, you can monitor its discard counters through any [performance counters](/windows-server/networking/technologies/network-subsystem/net-sub-performance-counters) interface or the [Get-NetAdapterStatistics](/powershell/module/netadapter/get-netadapterstatistics) cmdlet.
+
+## Common causes of local packet loss
+
+Local packet loss is fully observable and can be caused by various internal and external factors.
+
+- Local policy
+
+    Inspection software might cause packets from remote machines to be dropped by default, for example, when the Windows Firewall rejects inbound connection attempts. Cybersecurity or anti-malware software on the system can also cause these issues.
+- Low resources
+
+    If the system or socket runs out of resources to handle the packet, the packet is dropped. Examples of resource limits include physical memory on the system and socket send or receive buffers. Depending on the resource limit, these events might last only microseconds, for example, when the system's CPU can't react quickly enough to a full receive buffer.
+- ARP or ND failure
+
+    If the next hop for an outbound packet doesn't respond to Address Resolution Protocol (ARP) or neighbor discovery (ND) requests, then packets sent to that next hop are dropped on the local system. Packets might also be dropped during ARP or ND processes if the ARP or ND packet queue limit is exceeded. The ARP or ND packets themselves are typically not dropped locally and belong to the remote packet loss category.
+- No route
+
+    If the network layer can't find a valid route to the destination, packets might be dropped.
+- Invalid packet
+
+    If the packet headers are invalid, the packet might be dropped. For example, the packet headers contain an invalid checksum or field value.
+
+## Common causes of remote packet loss
+
+Remote packet loss isn't directly observable to the local machine when the packet is dropped. Internet Protocol (IP) and most layers below it are "best effort" and not reliable. The [end-to-end principle](https://en.wikipedia.org/wiki/End-to-end_principle) requires endpoints to implement reliability within their protocols if resilience to packet loss is required. In some scenarios, the network or remote endpoint sends a protocol-specific error message indicating the reason for the loss. However, in many cases, the only indication of packet loss is a lack of response.
+
+- Congestion
+
+    Loss-based congestion control algorithms send faster and faster until a packet is lost. If the algorithm infers the loss is caused by [congestion](https://en.wikipedia.org/wiki/Network_congestion), it temporarily reduces the rate of sending in response. These algorithms require a small amount of loss to provide a feedback signal.
+- Remote policy
+
+    The network or remote machine might drop packets according to its own policy.
+- Destination unreachable
+
+    This can occur if the remote machine doesn't have a socket bound to the remote port, remote machine is offline, or the network can't find a path to the remote machine.
+- Session loss
+
+    If the network (including stateful Network Address Translation (NAT), firewalls, and load balancers) or the remote machine is reset or doesn't receive a packet recently, its session context might expire, and subsequent packets are dropped.
+- Maximum Transmission Unit (MTU) drops
+
+    If the size of the packet exceeds the maximum transmission size of a network link along the path to the remote machine, MTU drops might produce an error: Internet Control Message Protocol (ICMP) fragmentation required or packet too big.
+
+## Example of Packet Monitor traces
+
+Run the following commands:
+
+```console
+pktmon.exe start -c
+pktmon.exe stop
+pktmon.exe etl2txt PktMon.etl
+```
+
+The resulting **PktMon.txt** file contains lines that resemble the following:
+
+```output
+[30]0000.0000::<DateTime> [Microsoft-Windows-PktMon] Drop: PktGroupId 8444249301423149, PktNumber 1, Appearance 0, Direction Rx , Type IP , Component 49, Filter 1, DropReason INET: transport endpoint was not found , DropLocation 0xE000460A, OriginalSize 402, LoggedSize 148
+       Drop: ip: 192.168.5.88.50005 > 192.168.5.68.50005: UDP, length 374
+```
+
+This information indicates the inbound UDP packet destined to port 50005 is dropped because no local socket is bound to the port.
+
+## Example of Network Shell traces
+
+Run the following commands:
+
+```console
+netsh.exe trace start scenario=InternetClient
+netsh.exe trace stop
+netsh.exe trace convert NetTrace.etl
+```
+
+The resulting **NetTrace.txt** file contains lines that resemble the following:
+
+```output
+[30]0000.0000::<DateTime> [Microsoft-Windows-TCPIP]TCPIP: Network layer (Protocol 1(ICMP), AddressFamily = 2(IPV4)) dropped 1 packet(s) on interface 13. SourceAddress = 192.168.5.68. DestAddress = 192.168.5.88. Reason = 9(Inspection drop). Direction = 0(Send). NBL = 0xFFFFE189BEAF3AC0.
+```
+
+This information indicates the outbound ICMP packet is dropped due to Windows Filtering Platform (WFP) inspection. The next step for WFP is to follow the [WFP live drops troubleshooting steps](/windows/security/operating-system-security/network-security/windows-firewall/troubleshooting-uwp-firewall#debugging-live-drops).
+
+In another scenario, a previously sent TCP segment isn't acknowledged by the remote endpoint, and eventually a local retransmit timer fires, causing TCP to resend some of the potentially lost data:
+
+```output
+[31]0000.0000::<DateTime> [Microsoft-Windows-TCPIP]TCP: Connection 0xFFFFE189BD811AA0 0(RetransmitTimer) timer has expired.
+[31]0000.0000::<DateTime> [Microsoft-Windows-TCPIP]TCP: Tail Loss Probe Event Connection = 0xFFFFE189BD811AA0, Event = 2(TimerFired).
+[31]0000.0000::<DateTime> [Microsoft-Windows-TCPIP]TCP: Tail Loss Probe Send Connection = 0xFFFFE189BD811AA0 SndUna = 2526318360, SndMax = 2526321759, SendAvailable = 3399, TailProbeSeq = 2526320299, TailProbeLast = 2526321759, ControlsToSend = 0, ThFlags = 16.
+[31]0000.0000::<DateTime> [Microsoft-Windows-TCPIP]TCP: connection 0xFFFFE189BD811AA0 (local=192.168.5.68:55330 remote=6.6.0.27:443): TCP send event, SeqNo = 2526320299, BytesSent = 1460, CWnd = 18538, SndWnd = 197632, SRtt = 17631, RttVar = 4947, RTO = 300, RcvWnd = 65535, PacingRate = 0, State = 4(EstablishedState), CongestionState = 0, SndUna = 2526318360, SndMax = 2526321759, RecoveryMax = 0, RcvBufSet = 0(FALSE), MaxRcvBuf = 65535.
+[31]0000.0000::<DateTime> [Microsoft-Windows-TCPIP]TCP: connection = 0xFFFFE189BD811AA0 send tracker marked a transmit as rexmit. Start = 2526320299, End = 2526321759, Timestamp = 467744252, InFlightCount = 2, SackedBytes = 0, BytesInFlight = 4859.
+[31]0000.0000::<DateTime> [Microsoft-Windows-TCPIP]TCP: Connection 0xFFFFE189BD811AA0 0(RetransmitTimer) timer started. Scheduled to expire in 300 ms. Processor 31: LastInterruptTime 305324952689 100-ns ticks; LastMicrosecondCount 30532515324 msec
+```
+
+## More information
+
+- [Troubleshoot TCP/IP connectivity](./tcp-ip-connectivity-issues-troubleshooting.md)
+- [How to capture a NetLog dump](https://www.chromium.org/for-testers/providing-network-details)
+- [How to collect a network trace](/azure/azure-web-pubsub/howto-troubleshoot-network-trace)
+- [Inspect network activity](/microsoft-edge/devtools-guide-chromium/network/)
+- [Trouble Shooting Guide](https://microsoft.github.io/msquic/msquicdocs/docs/TSG.html)

support/windows-client/networking/tcp-ip-connectivity-issues-troubleshooting.md

@@ -1,14 +1,14 @@
 ---
 title: TCP/IP connectivity issues troubleshooting
 description: Learn how to troubleshoot TCP/IP connectivity and what you should do if you come across TCP reset in a network capture.
-ms.date: 01/15/2025
+ms.date: 04/30/2025
 ms.topic: troubleshooting
 manager: dcscontentpm
 ms.collection: highpri
 ms.custom:
 - sap:network connectivity and file sharing\tcp/ip connectivity (tcp protocol,nla,winhttp)
 - pcy:WinComm Networking
-ms.reviewer: dansimp
+ms.reviewer: dansimp, mifriese, guhetier
 audience: itpro
 ---
 # Troubleshoot TCP/IP connectivity
@@ -18,7 +18,7 @@ audience: itpro
 
 _Applies to:_   Supported versions of Windows client and Windows Server  
 
-This article provides a comprehensive guide for troubleshooting TCP/IP connectivity errors.
+This article provides a comprehensive guide for troubleshooting Transmission Control Protocol (TCP)/Internet Protocol (IP) connectivity errors.
 
 ## Symptoms and analysis
 
@@ -40,9 +40,9 @@ During the troubleshooting process, you might encounter a TCP RESET in the netwo
 TCP is characterized as a connection-oriented and reliable protocol. It ensures reliability through the handshake process. A TCP session initiates with a three-way handshake, followed by data transfer, and concludes with a four-way closure.
 
 - A four-way closure, where both the sender and receiver agree to close the session, is known as a graceful closure. This is identified by the FIN flag in the TCP header being set to 1.
-
 - After the four-way closure, the machine waits for 4 minutes (by default) before releasing the port. This is termed as TIME_WAIT state. During this TIME_WAIT state, any pending packets for the TCP connection can be processed. Once the TIME_WAIT state completes, all resources allocated for the connection are released.  
-- A TCP reset is an abrupt session closure, causing the immediate release of allocated resources and erasure of all connection information. This is identified by the RESET flag in the TCP header being set to 1. 
+- A TCP reset is an abrupt session closure, causing the immediate release of allocated resources and erasure of all connection information. This is identified by setting the RESET flag in the TCP header to 1.
+
 A network trace collected simultaneously on the source and the destination helps you to determine the flow of the traffic and identify the point of failure.
 
 The following sections outline scenarios in which a RESET could occur.
@@ -53,6 +53,8 @@ When a TCP peer sends packets without receiving a response, the peer retransmits
 
 Simultaneous network traces at both the source and destination can verify this behavior. On the source side, you can see the retransmitted packets. On the destination side, these packets don't be present. This scenario indicates that a network device between the source and destination is dropping the packets.
 
+For more information about diagnosing packet loss issues, see [Diagnose packet loss](diagnose-packet-loss.md).
+
 ### Scenario 1: Packet loss during initial TCP handshake
 
 If the initial TCP handshake fails due to packet drops, the TCP SYN packet is retransmitted three times by default.
@@ -68,7 +70,7 @@ The same TCP conversation seen in the network trace collected on the destination
 
 :::image type="content" source="media/tcp-ip-connectivity-issues-troubleshooting/destination-side-same-filter.png" alt-text="Screenshot of frame summary with filter in Network Monitor.":::
 
-If the TCP SYN packets are reaching the destination, but the destination doesn't response, verify whether the TCP port that is connected to is in the LISTENING state on destination machine. This can be checked in the output of the command `netstat -anob`.
+If the TCP SYN packets reach the destination, but the destination doesn't respond, verify whether the connected TCP port on destination machine is in the LISTENING state. This can be checked in the output of the command `netstat -anob`.
 
 If the port is listening and there's still no response, there could be a drop at the Windows Filtering Platform (WFP).
 
@@ -117,15 +119,15 @@ An ACK+RST flagged TCP packet can also occur when a TCP SYN packet is sent out.
 :::image type="content" source="media/tcp-ip-connectivity-issues-troubleshooting/ack-rst-flag-packet.png" alt-text="Screenshot of packet with ACK RSK flag.":::In such cases, it's essential to investigate the application causing the reset (identified by port numbers) to understand why the connection is being reset.
 
 > [!NOTE]
-> The above information pertains to resets from a TCP perspective and not UDP. UDP is a connectionless protocol, and packets are sent unreliably. Consequently, retransmissions or resets are not observed when using UDP as a transport protocol. However, UDP utilizes ICMP as an error reporting protocol. When a UDP packet is sent to a port that is not listed at the destination, the destination will immediately respond with an ICMP "**Destination Host Unreachable: Port Unreachable**" message.
+> The preceding information pertains to resets from a TCP perspective and not UDP. UDP is a connectionless protocol, and packets are sent unreliably. Therefore, retransmissions or resets aren't observed when using UDP as a transport protocol. However, UDP utilizes ICMP as an error reporting protocol. When a UDP packet is sent to a port that isn't listed at the destination, the destination immediately responds with an ICMP "**Destination Host Unreachable: Port Unreachable**" message.
 
 ```output
 10.10.10.1  10.10.10.2  UDP UDP:SrcPort=49875,DstPort=3343
  
 10.10.10.2  10.10.10.1  ICMP    ICMP:Destination Unreachable Message, Port Unreachable,10.10.10.2:3343
 ```
 
-During the troubleshooting of TCP/IP connectivity issues, you may observe in the network trace that a machine receives packets but doesn't respond to them. This could indicate a drop at the destination server's network stack.
+During the troubleshooting of TCP/IP connectivity issues, you might observe in the network trace that a machine receives packets but doesn't respond to them. This might indicate a drop at the destination server's network stack.
 
 To determine whether the local Windows Firewall is dropping the packet, enable auditing for the Windows Filtering Platform (WFP) on the machine using the following command.
 

support/windows-client/toc.yml

@@ -383,6 +383,8 @@ items:
       href: ./networking/collect-data-using-network-monitor.md
     - name: Device can't connect to mobile broadband
       href: ./networking/device-cant-connect-mobile-broadband-over-the-air.md
+    - name: Diagnose packet loss
+      href: ./networking/diagnose-packet-loss.md
     - name: Domain-joined machines can't detect the domain profile
       href: ./networking/domain-joined-machines-cannot-detect-domain-profile.md
     - name: Enterprise APN lost after SIM change or MBN adapter error