-
Notifications
You must be signed in to change notification settings - Fork 244
Description
### Problem Description:
When using the AzCopy copy command to transfer a 3TB VHD disk file from an Azure virtual machine to a local Windows Server, the customer experiences frequent failures mid-transfer.
### Troubleshooting:
-
Network Connection Errors Identified in Failure Logs:
The logs show numerous network connection errors (A connection attempt failed). Specifically, there are many instances ofread tcp 10.xx.xx.xx:63854->40.xx.xx.xx:443: wsarecv: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
This indicates network connectivity issues between the local server and the Azure storage endpoint (40.xx.xx.xx), which is likely a significant factor contributing to the copy failures. -
Potential Disk Performance Constraints on the Local Server:
There are multiple occurrences of "Disk may be limiting speed," suggesting that AzCopy detected the local server's disk performance as a bottleneck. However, this might be misleading since network issues could cause the disk to wait, making it appear as though disk performance is the issue.
### Problem Analysis:
Theoretically, network connectivity between the Azure virtual machine and the local server should not be problematic, as the customer successfully copied an approximately 80GB VHD file to the local server previously. The current challenge lies with copying a much larger 3TB VHD file, where the extended duration of the transfer increases the likelihood of failure due to network instability.
### ASK:
Is there a way to optimize AzCopy for copying single large files (over 1TB) to improve efficiency and stability, thereby increasing the success rate of the copy operation? Alternatively, is there a true breakpoint resume capability available, such that if a copy fails at 50%, the next attempt would only need to copy the remaining 50%? According to Azure's official documentation, only a method to resume failed copies exists, but this does not constitute a breakpoint resume; instead, it restarts the entire copy process while generating a new log file.
https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-configure?toc=%2Fazure%2Fstorage%2Fblobs%2Ftoc.json&bc=%2Fazure%2Fstorage%2Fblobs%2Fbreadcrumb%2Ftoc.json