azcopy sync print failure logs for directory that has only subdirectory with files #3017
Description
Which version of the AzCopy was used?
AzcopyVersion 10.28.1
Which platform are you using? (ex: Windows, Mac, Linux)
Linux
What command did you run?
azcopy sync https://xxxxx.blob.core.windows.net/abc271/ /appl/prd/xyz/abc/ --log-level DEBUG --recursive=true --delete-destination true --compare-ha sh MD5 --local-hash-storage-mode=HiddenFiles --hash-meta-dir /appl/prd/xyz/home/.azcopy
What problem was encountered?
Eventhough the file is transferred , but the command status is in FAILED state. Below are logs that can be seen
2025/04/12 09:54:16 ERR: [P#0-T#138] DOWNLOADFAILED: https://xxxxx.blob.core.windows.net/abc271/lib/MIME/Type.pm : 000 : failed to create hash meta file: open /appl/prd/xyz/abc/home/.azcopy/lib/MIME/.Type.pm.azcopysyncmeta: no such file or directory. When saving MD5 data (writing alternate data stream). X-Ms-Request-Id:
Dst: /appl/prd/xyz/abc/lib/MIME/Type.pm
2025/04/12 09:54:16 DBG: [P#0-T#138] Finalizing Transfer Cancellation/Failure
2025/04/12 09:54:16 INFO: [P#0-T#138] https://xxxxx.blob.core.windows.net/abc271/lib/MIME/Type.pm Deleting incomplete destination file Dst: /appl/prd/xyz/abc/lib/MIME/Type.p
m
2025/04/12 09:54:16 ERR: [P#0-T#138] /appl/prd/xyz/abc/lib/MIME/Type.pm: 0: Delete File Error -remove /appl/prd/xyz/abc/lib/MIME/.azDownload-714f9ab3-8f15-254b-41cb-c9ec1ca22d22
-Type.pm: no such file or directory. X-Ms-Request-Id:
Because we execute the azcopy via ansible , this reports in the failure of the playbook becuase the azcopy command exist status code is not 0 .
"stdout_lines": [
"INFO: Login with identity succeeded.",
"INFO: Authenticating to source using Azure AD",
"INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support",
"",
"Job 714f9ab3-8f15-254b-41cb-c9ec1ca22d22 has started",
"Log file is located at: /appl/prd/xyz/abc/home/.azcopy/714f9ab3-8f15-254b-41cb-c9ec1ca22d22.log",
"",
"INFO: Deleting extra file: home/.bash_profile",
"INFO: Deleting extra file: home/.bashrc",
"INFO: Deleting extra file: home/.bash_logout",
"WARN: Failed to create hash data directory",
"",
"100.0 %, 138 Done, 1 Failed, 0 Pending, 139 Total, 2-sec Throughput (Mb/s): 105.0148",
"",
"Job 714f9ab3-8f15-254b-41cb-c9ec1ca22d22 Summary",
"Files Scanned at Source: 139",
"Files Scanned at Destination: 6",
"Elapsed Time (Minutes): 0.0334",
"Number of Copy Transfers for Files: 139",
"Number of Copy Transfers for Folder Properties: 0 ",
"Total Number of Copy Transfers: 139",
"Number of Copy Transfers Completed: 138",
"Number of Copy Transfers Failed: 1",
"Number of Deletions at Destination: 3",
"Total Number of Bytes Transferred: 26282060",
"Total Number of Bytes Enumerated: 26284789",
"Final Job Status: CompletedWithErrors"
This only happens if you have certain type of directory structure.
How can we reproduce the problem in the simplest way?
Have this kind of directory structure / blob that you are trying to sync
.
└── abc
├── home
│ ├── dfchk
│ ├── fix_eback.sh
│ └── static-ports.ini
├── lib
│ └── MIME
│ └── Type.pm
└── VERSIE
4 directories, 5 files
Since lib folder has only MIME as subfolder and no file , it will fail.
However if I put a dummy file inside lib folder this will be successful .
Have you found a mitigation/solution?
place a dummy file for every such folder.