On the azure-cli docker image, 10.13.0 of azcopy is unavailable and so az storage copy fails

[hkethi002]

Describe the bug

the docker images

• mcr.microsoft.com/azure-cli:latest
• mcr.microsoft.com/azure-cli:2.67.0
  and presumably others which use the azure linux operating system, using tdnf as the package manager don't have azcopy version 10.13.0 available and so when trying to run:
  az storage copy or any other command that delegates out to azcopy fails.

Related command

docker run mcr.microsoft.com/azure-cli:latest az storage copy \
  --source "https://${storageAccountName}.blob.core.windows.net/${containerName}/" \
  --destination "https://${storageAccountName}.blob.core.windows.net/${containerName}/" \
  --recursive \
  --source-account-name "$storageAccountName" \
  --account-name "$storageAccountName" \
  --auth-mode "key" \
  --account-key "$STORAGE_AZURE_ACCOUNTKEY" \
  --source-account-key "$STORAGE_AZURE_ACCOUNTKEY"

reproduces the issue for me

Errors

WARNING: Positional argument '<EXTRA_OPTIONS>' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus
WARNING: Azcopy not found, installing at /root/.azure/bin/azcopy
ERROR: Connection error while attempting to download azcopy 10.13.0. You could also install the specified azcopy version to /root/.azure/bin manually. (<urlopen error [Errno -2] Name or service not known>)

Issue script & Debug output

DEBUG: cli.knack.cli: Command arguments: ['storage', 'copy', '--debug']
DEBUG: cli.knack.cli: init debug log:
Cannot enable color.
DEBUG: cli.knack.cli: Event: Cli.PreExecute []
DEBUG: cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0xffff831b5120>, <function OutputProducer.on_global_arguments at 0xffff82f20cc0>, <function CLIQuery.on_global_arguments at 0xffff82f72840>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
DEBUG: cli.azure.cli.core: Command index version or cloud profile is invalid or doesn't match the current command.
DEBUG: cli.azure.cli.core: Command index has been invalidated.
DEBUG: cli.azure.cli.core: No module found from index for '['storage', 'copy', '--debug']'
DEBUG: cli.azure.cli.core: Loading all modules and extensions
DEBUG: cli.azure.cli.core: Discovered command modules: ['acr', 'acs', 'advisor', 'ams', 'apim', 'appconfig', 'appservice', 'aro', 'backup', 'batch', 'batchai', 'billing', 'botservice', 'cdn', 'cloud', 'cognitiveservices', 'compute_recommender', 'computefleet', 'config', 'configure', 'consumption', 'container', 'containerapp', 'cosmosdb', 'databoxedge', 'dls', 'dms', 'eventgrid', 'eventhubs', 'extension', 'feedback', 'find', 'hdinsight', 'identity', 'interactive', 'iot', 'keyvault', 'lab', 'managedservices', 'maps', 'marketplaceordering', 'monitor', 'mysql', 'netappfiles', 'network', 'policyinsights', 'privatedns', 'profile', 'rdbms', 'redis', 'relay', 'resource', 'role', 'search', 'security', 'servicebus', 'serviceconnector', 'servicefabric', 'signalr', 'sql', 'sqlvm', 'storage', 'synapse', 'util', 'vm']
DEBUG: cli.azure.cli.core: Loading command modules:
DEBUG: cli.azure.cli.core: Name Load Time Groups Commands
DEBUG: cli.azure.cli.core: acr 0.102 36 149
DEBUG: cli.azure.cli.core: acs 0.008 14 77
DEBUG: cli.azure.cli.core: advisor 0.000 3 6
DEBUG: cli.azure.cli.core: ams 0.002 22 100
DEBUG: cli.azure.cli.core: apim 0.002 14 69
DEBUG: cli.azure.cli.core: appconfig 0.002 9 47
DEBUG: cli.azure.cli.core: appservice 0.011 79 270
DEBUG: cli.azure.cli.core: aro 0.004 1 10
DEBUG: cli.azure.cli.core: backup 0.002 16 60
DEBUG: cli.azure.cli.core: batch 0.026 34 102
DEBUG: cli.azure.cli.core: batchai 0.001 10 30
DEBUG: cli.azure.cli.core: billing 0.006 20 53
DEBUG: cli.azure.cli.core: botservice 0.001 12 42
DEBUG: cli.azure.cli.core: cdn 0.071 8 50
DEBUG: cli.azure.cli.core: cloud 0.001 1 7
DEBUG: cli.azure.cli.core: cognitiveservices 0.001 10 33
DEBUG: cli.azure.cli.core: compute_recommender 0.001 1 1
DEBUG: cli.azure.cli.core: computefleet 0.004 1 1
DEBUG: cli.azure.cli.core: config 0.001 2 7
DEBUG: cli.azure.cli.core: configure 0.000 2 5
DEBUG: cli.azure.cli.core: consumption 0.006 8 9
DEBUG: cli.azure.cli.core: container 0.004 2 17
DEBUG: cli.azure.cli.core: containerapp 0.045 37 123
DEBUG: cli.azure.cli.core: cosmosdb 0.005 58 199
DEBUG: cli.azure.cli.core: databoxedge 0.003 5 28
DEBUG: cli.azure.cli.core: dls 0.001 6 36
DEBUG: cli.azure.cli.core: dms 0.001 3 22
DEBUG: cli.azure.cli.core: eventgrid 0.002 25 96
DEBUG: cli.azure.cli.core: eventhubs 0.003 13 20
DEBUG: cli.azure.cli.core: extension 0.000 1 7
DEBUG: cli.azure.cli.core: feedback 0.000 1 2
DEBUG: cli.azure.cli.core: find 0.000 1 1
DEBUG: cli.azure.cli.core: hdinsight 0.003 9 42
DEBUG: cli.azure.cli.core: identity 0.001 2 11
DEBUG: cli.azure.cli.core: interactive 0.000 1 1
DEBUG: cli.azure.cli.core: iot 0.050 19 82
DEBUG: cli.azure.cli.core: keyvault 0.006 20 113
DEBUG: cli.azure.cli.core: lab 0.015 9 29
DEBUG: cli.azure.cli.core: managedservices 0.005 3 8
DEBUG: cli.azure.cli.core: maps 0.001 5 13
DEBUG: cli.azure.cli.core: marketplaceordering 0.001 1 2
DEBUG: cli.azure.cli.core: monitor 0.266 17 61
DEBUG: cli.azure.cli.core: mysql 0.048 16 56
DEBUG: cli.azure.cli.core: netappfiles 0.019 8 17
DEBUG: cli.azure.cli.core: network 0.059 103 338
DEBUG: cli.azure.cli.core: policyinsights 0.005 9 17
DEBUG: cli.azure.cli.core: privatedns 0.008 14 60
DEBUG: cli.azure.cli.core: profile 0.001 2 8
DEBUG: cli.azure.cli.core: rdbms 0.035 50 208
DEBUG: cli.azure.cli.core: redis 0.001 7 38
DEBUG: cli.azure.cli.core: relay 0.010 7 8
DEBUG: cli.azure.cli.core: resource 0.006 52 232
DEBUG: cli.azure.cli.core: role 0.001 17 61
DEBUG: cli.azure.cli.core: search 0.006 7 19
DEBUG: cli.azure.cli.core: security 0.006 47 98
DEBUG: cli.azure.cli.core: servicebus 0.049 14 17
DEBUG: cli.azure.cli.core: serviceconnector 0.011 20 315
DEBUG: cli.azure.cli.core: servicefabric 0.006 27 80
DEBUG: cli.azure.cli.core: signalr 0.001 10 42
DEBUG: cli.azure.cli.core: sql 0.006 56 215
DEBUG: cli.azure.cli.core: sqlvm 0.014 4 20
DEBUG: cli.azure.cli.core: storage 0.018 59 273
DEBUG: cli.azure.cli.core: synapse 0.005 54 246
DEBUG: cli.azure.cli.core: util 0.000 3 7
DEBUG: cli.azure.cli.core: vm 0.020 58 233
DEBUG: cli.azure.cli.core: Total (65) 1.002 1185 4649
DEBUG: cli.azure.cli.core: Loaded 1171 groups, 4649 commands.
DEBUG: cli.azure.cli.core: Updated command index in 0.002 seconds.
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0xffff81d1ae80>]
DEBUG: cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/root/.azure/commands/2025-01-09.21-07-22.storage_copy.9.log'.
INFO: az_command_data_logger: command args: storage copy --debug
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0xffff819a0360>]
DEBUG: cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/profiles/_shared.py", line 656, in _get_attr
op = getattr(op, part)
^^^^^^^^^^^^^^^^^
AttributeError: module 'azure.mgmt.storage.v2023_05_01.models' has no attribute 'ActiveDirectoryPropertiesAccountType'

DEBUG: cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/profiles/_shared.py", line 656, in _get_attr
op = getattr(op, part)
^^^^^^^^^^^^^^^^^
AttributeError: module 'azure.mgmt.storage.v2023_05_01.models' has no attribute 'ListKeyExpand'

DEBUG: cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/profiles/_shared.py", line 656, in _get_attr
op = getattr(op, part)
^^^^^^^^^^^^^^^^^
AttributeError: module 'azure.mgmt.storage.v2023_05_01.models' has no attribute 'CorsRuleAllowedMethodsItem'

DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0xffff819a0400>, <function register_cache_arguments..add_cache_arguments at 0xffff819a0540>, <function register_upcoming_breaking_change_info..update_breaking_change_info at 0xffff819a05e0>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0xffff800d8860>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0xffff82f20d60>, <function CLIQuery.handle_query_parameter at 0xffff82f728e0>, <function register_ids_argument..parse_ids_arguments at 0xffff819a04a0>]
WARNING: Positional argument '<EXTRA_OPTIONS>' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage auth_mode value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage account value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage connection_string value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage account value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage key value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage sas_token value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Cannot generate sas token. unsupported operand type(s) for +: 'NoneType' and 'str'
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage connection_string value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage account value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage key value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Try to get storage sas_token value from environment variables or config file.
INFO: cli.azure.cli.command_modules.storage._validators: Cannot generate sas token. unsupported operand type(s) for +: 'NoneType' and 'str'
WARNING: cli.azure.cli.command_modules.storage.azcopy.util: Azcopy not found, installing at /root/.azure/bin/azcopy
DEBUG: cli.azure.cli.core.azclierror: Traceback (most recent call last):
File "/usr/lib/python3.12/urllib/request.py", line 1344, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/usr/lib/python3.12/http/client.py", line 1336, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.12/http/client.py", line 1382, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.12/http/client.py", line 1331, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.12/http/client.py", line 1091, in _send_output
self.send(msg)
File "/usr/lib/python3.12/http/client.py", line 1035, in send
self.connect()
File "/usr/lib/python3.12/http/client.py", line 1470, in connect
super().connect()
File "/usr/lib/python3.12/http/client.py", line 1001, in connect
self.sock = self._create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/socket.py", line 828, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/socket.py", line 963, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/command_modules/storage/azcopy/util.py", line 76, in install_azcopy
_urlretrieve(file_url, install_location)
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/command_modules/storage/azcopy/util.py", line 224, in _urlretrieve
req = urlopen(url)
^^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 215, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 515, in open
response = self._open(req, data)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 532, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 492, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 1392, in https_open
return self.do_open(http.client.HTTPSConnection, req,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/urllib/request.py", line 1347, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 666, in execute
raise ex
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 733, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 725, in _run_job
return cmd_copy.exception_handler(ex)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/command_modules/storage/init.py", line 430, in new_handler
first(ex)
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/command_modules/storage/init.py", line 429, in new_handler
raise ex
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 703, in _run_job
result = cmd_copy(params)
^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 336, in call
return self.handler(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
return op(**command_args)
^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/command_modules/storage/operations/azcopy.py", line 15, in storage_copy
azcopy = AzCopy()
^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/command_modules/storage/azcopy/util.py", line 54, in init
self.install_azcopy(install_location)
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/command_modules/storage/azcopy/util.py", line 80, in install_azcopy
raise CLIError('Connection error while attempting to download azcopy {}. You could also install the '
knack.util.CLIError: Connection error while attempting to download azcopy 10.13.0. You could also install the specified azcopy version to /root/.azure/bin manually. (<urlopen error [Errno -2] Name or service not known>)

ERROR: cli.azure.cli.core.azclierror: Connection error while attempting to download azcopy 10.13.0. You could also install the specified azcopy version to /root/.azure/bin manually. (<urlopen error [Errno -2] Name or service not known>)
ERROR: az_command_data_logger: Connection error while attempting to download azcopy 10.13.0. You could also install the specified azcopy version to /root/.azure/bin manually. (<urlopen error [Errno -2] Name or service not known>)
DEBUG: cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0xffff81d1b100>]
INFO: az_command_data_logger: exit code: 1
INFO: cli.main: Command ran in 1.805 seconds (init: 0.482, invoke: 1.323)
DEBUG: cli.azure.cli.core._profile: Failed to load installationId from AzureRmSurvey.json. [Errno 2] No such file or directory: '/root/.azure/AzureRmContextSettings.json'
INFO: telemetry.main: Begin splitting cli events and extra events, total events: 1
INFO: telemetry.client: Accumulated 0 events. Flush the clients.
INFO: telemetry.main: Finish splitting cli events and extra events, cli events: 1
INFO: telemetry.save: Save telemetry record of length 3686 in cache file under /root/.azure/telemetry/20250109210723243
INFO: telemetry.main: Begin creating telemetry upload process.
INFO: telemetry.process: Creating upload process: "/usr/bin/python3.12 /usr/lib/az/lib/python3.12/site-packages/azure/cli/telemetry/init.py /root/.azure /root/.azure/telemetry/20250109210723243"
INFO: telemetry.process: Return from creating process 14
INFO: telemetry.main: Finish creating telemetry upload process.

Expected behavior

it should work

Environment Summary

azure-cli 2.52.0 *

core 2.52.0 *
telemetry 1.1.0

Extensions:
ml 2.20.0

Dependencies:
msal 1.24.0b1
azure-mgmt-resource 23.1.0b2

Python location '/opt/homebrew/Cellar/azure-cli/2.52.0_1/libexec/bin/python'
Extensions directory '/Users/harshakethineni/.azure/cliextensions'

Python (Darwin) 3.10.13 (main, Aug 24 2023, 22:36:46) [Clang 14.0.3 (clang-1403.0.22.14.1)]

Legal docs and information: aka.ms/AzureCliLegal

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[hkethi002]

I think bumping https://github.com/Azure/azure-cli/blob/dev/src/azure-cli/azure/cli/command_modules/storage/azcopy/util.py#L25 to a more recent version would work

[rspinabella]

Same thing is happening to me right now!!!! @hkethi002 @yonzhan

Run azure/cli@v1.0.0
with:
inlineScript: az storage blob sync -c '$web' -s 'public' --connection-string '***'

azcliversion: latest

Starting script execution via docker image mcr.microsoft.com/azure-cli:latest
WARNING: This command is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
WARNING: Positional argument '<EXTRA_OPTIONS>' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus
WARNING: Azcopy not found, installing at /root/.azure/bin/azcopy
ERROR: Connection error while attempting to download azcopy 10.13.0. You could also install the specified azcopy version to /root/.azure/bin manually. (<urlopen error [Errno -2] Name or service not known>)
Error: Error: az cli script failed.
cleaning up container...
MICROSOFT_AZURE_CLI_1736451680971_CONTAINER

[hkethi002]

Same thing is happening to me right now!!!! @hkethi002 @yonzhan

I managed to do a workaround by adding tdnf install azcopy -y to the start of my script I'm running in the container if that helps

[praiaJoaoMiranda]

Root Cause

Azure deprecated the previously Azure CDN (https://azureedge.net) and the Azure-CLI attempts to download the azcopy from CDN in case of not found locally

The previous address was built dynamically here and is exactly that which is not working anymore - checked doing this:

wget https://azcopyvnext.azureedge.net/release20211027/azcopy_linux_amd64_10.13.0.tar.gz

The azure-cli module install should be updated to use the new url:

https://azcopyvnext-awgzd8g7aagqhzhe.b02.azurefd.net/release20211027/azcopy_{}_{}_{}.{}

You can confirm that approach works by doing this:

wget https://azcopyvnext-awgzd8g7aagqhzhe.b02.azurefd.net/release20211027/azcopy_linux_amd64_10.13.0.tar.gz

---

@calvinhzy @evelyn-ys - Please open a PR and fix this, it could cause so much damage to many companies using azcopy inside azure-cli.

In this meanwhile, from our developer side, the best we can do is try to install azcopy manually where you need - like @hkethi002 suggest

[rspinabella]

@evelyn-ys @calvinhzy Can we please get an update when this is fixed. Im using Azure CLI and Azcopy through a github runner, theres no way for me to manually install it