Skip to content

AzCopy copy Blob to local with --preserve-symlinks fails because of not created directories when using wildcard in URL #3148

@wolfgang-desalvador

Description

@wolfgang-desalvador

Which version of the AzCopy was used?

AzCopy Version 10.29.1

Which platform are you using? (ex: Windows, Mac, Linux)

Linux

AlmaLinux 8.10 (Cerulean Leopard) - 4.18.0-553.16.1.el8_10.x86_64

What command did you run?

More documentation is below, in the section describing reproduce.

The command failing is the following, when copying from Blob to local using a wildcard in URL, as described in the azcopy copy command documentation:

azcopy copy --include-directory-stub --preserve-posix-properties --preserve-symlinks  --recursive

A note about using a wildcard character (*) in URLs:

There's only two supported ways to use a wildcard character in a URL. 
- You can use one just after the final forward slash (/) of a URL. This copies all of the files in a directory directly to the destination without placing them into a subdirectory. 
- You can also use one in the name of a container as long as the URL refers only to a container and not to a blob. You can use this approach to obtain files from a subset of containers. 

What problem was encountered?

Symlink creation fails with the following output:

2025/08/04 09:11:34 ERR: [P#0-T#44] DOWNLOADFAILED: https://XXXXXX.blob.core.windows.net/XXXXX/linux-6.16/scripts/dummy-tools/nm : 000 : symlink ld /mnt/recopy/scripts/dummy-tools/nm: no such file or directory. When creating destination symlink (check operation type, is it really download?). X-Ms-Request-Id:

How can we reproduce the problem in the simplest way?

To reproduce run the following:

wget "https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.16.tar.xz"
tar -xf linux-6.16.tar.xz
azcopy login --identity
azcopy copy linux-6.16 "https://<STORAGE_ACCOUNT_NAME>.blob.core.windows.net/<CONTAINER_NAME>" --include-directory-stub --preserve-posix-properties --preserve-symlinks  --recursive ## This will succeed
azcopy copy "https://<STORAGE_ACCOUNT_NAME>.blob.core.windows.net/<CONTAINER_NAME>/linux-6.16/*" recopy --include-directory-stub --preserve-posix-properties --preserve-symlinks  --recursive ## This will have failed entries

Have you found a mitigation/solution?

A call to CreateParentDirectoryIfNotExist should be added to xfer-remoteToLocal-symlink.go

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions