az storage azcopy run-command

[dazinator]

Describe the bug

I am trying to use azcopy to clone a directory.

https://learn.microsoft.com/en-us/cli/azure/storage/azcopy?view=azure-cli-latest#az-storage-azcopy-run-command

I am using this approach

$sasToken = az storage share generate-sas --account-name myaccount --name myshare --account-key [my-key] -o tsv
az storage azcopy run-command azcopy copy "https://myaccount.file.core.windows.net/myshare/folder/?$sasToken" "https://myaccount.file.core.windows.net/myshare/folder-copy?$sasToken" --as-subdir=false --overwrite=prompt --from-to=FileFile --preserve-smb-info=true --recursive --log-level=INFO --trailing-dot=Enable

the problem is this gives an error:

unrecognized arguments: copy https://myaccount.file.core.windows.net/myshare/folder/?sv=2022-11-02

I can't seem to pass the correct arguments to use az copy. I tried wrapping the url's in quotes .g:

$sourceUrl = "https://myaccount.file.core.windows.net/myshare/folder/?$sasToken"
$destUrl = "https://myaccount.file.core.windows.net/myshare/folder-copy/?$sasToken" 

az storage azcopy run-command azcopy copy ""$sourceUrl"" ""$destUrl"" --as-subdir=false --overwrite=prompt --from-to=FileFile --preserve-smb-info=true --recursive --log-level=INFO --trailing-dot=Enable

same error

Related command

az storage azcopy run-command

Errors

unrecognized arguments: copy https://myaccount.file.core.windows.net/myshare/folder/?sv=2022-11-02

Issue script & Debug output

az storage azcopy run-command --debug azcopy "$sourceUrl" "$destUrl" --as-subdir=false --overwrite=prompt --from-to=FileFile --preserve-smb-info=true --recursive --log-level=INFO --trailing-dot=Enable --debug cli.knack.cli: Command arguments: ['storage', 'azcopy', 'run-command', '--debug', 'azcopy', 'https://myaccount.file.core.windows.net/myshare/alpha/?sv=2022-11-02']
cli.knack.cli: init debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x01A3A538>, <function OutputProducer.on_global_arguments at 0x01B3B778>, <function CLIQuery.on_global_arguments at 0x01C583D0>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'storage': ['azure.cli.command_modules.storage', 'azext_storage_preview']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name Load Time Groups Commands
cli.azure.cli.core: storage 0.064 58 272
cli.azure.cli.core: Total (1) 0.064 58 272
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name Load Time Groups Commands Directory
cli.knack.cli: Event: CommandLoader.OnLoadCommandTable []
cli.azure.cli.core: storage-preview 0.017 15 51 C:\Users\DarrellTunnell.azure\cliextensions\storage-preview
cli.azure.cli.core: Total (1) 0.017 15 51
cli.azure.cli.core: Loaded 64 groups, 294 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command : storage azcopy run-command
cli.azure.cli.core: Command table: storage azcopy run-command
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x03DFC580>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to 'C:\Users\DarrellTunnell.azure\commands\2023-07-21.13-27-56.storage_azcopy_run-command.29660.log'.
az_command_data_logger: command args: storage azcopy run-command --debug {} {}
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x03E23580>]
cli.knack.cli: Event: CommandLoader.OnLoadArguments []
cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/profiles/_shared.py", line 659, in _get_attr
AttributeError: module 'azext_storage_preview.vendored_sdks.azure_mgmt_storage.v2022_09_01.models' has no attribute 'ActiveDirectoryPropertiesAccountType'

cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0x03E344A8>, <function register_cache_arguments..add_cache_arguments at 0x03E346A0>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
urllib3.connectionpool: Starting new HTTPS connection (1): app.aladdin.microsoft.com:443
urllib3.connectionpool: https://app.aladdin.microsoft.com:443 "GET /api/v1.0/suggestions?query=%7B%22command%22%3A+%22storage+azcopy+run-command%22%2C+%22parameters%22%3A+%22%22%7D&clientType=AzureCli&context=%7B%22versionNumber%22%3A+%222.50.0%22%2C+%22errorType%22%3A+%22UnrecognizedArguments%22%2C+%22correlationId%22%3A+%2215271d5a-600f-404d-ad2b-0e732e8ae59c%22%2C+%22subscriptionId%22%3A+%22foobar%22%2C+%22eventId%22%3A+%222114ea9c-a852-4a2c-8ddc-bbf0a25e6437%22%7D HTTP/1.1" 200 None
cli.azure.cli.core.azclierror: NoneType: None

cli.azure.cli.core.azclierror: unrecognized arguments: https://myaccount.file.core.windows.net/myshare/alpha/?sv=2022-11-02
az_command_data_logger: unrecognized arguments: https://myaccount.file.core.windows.net/myshare/alpha/?sv=2022-11-02

Examples from AI knowledge base:
https://aka.ms/cli_ref
Read more about the command in reference docs
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x03DFC6A0>]
az_command_data_logger: exit code: 2
cli.main: Command ran in 0.984 seconds (init: 0.399, invoke: 0.585)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 3552 in cache
telemetry.check: Negative: The C:\Users\myname.azure\telemetry.txt was modified at 2023-07-21 13:19:37.207139, which in less than 600.000000 s
'sr' is not recognized as an internal or external command,
operable program or batch file.
'sig' is not recognized as an internal or external command,
operable program or batch file.
'sr' is not recognized as an internal or external command,
operable program or batch file.
'sig' is not recognized as an internal or external command,
operable program or batch file.

Expected behavior

I expect to be able to run azcopy and provide arguments for source and target.

Environment Summary

azure-cli 2.50.0

core 2.50.0
telemetry 1.0.8

Extensions:
azure-devops 0.22.0
bastion 0.2.4
ssh 1.1.3
storage-preview 0.9.0

Dependencies:
msal 1.22.0
azure-mgmt-resource 23.1.0b2

Python location 'C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python.exe'
Extensions directory 'C:\Users\myname.azure\cliextensions'

Python (Windows) 3.10.10 (tags/v3.10.10:aad5f6a, Feb 7 2023, 17:05:00) [MSC v.1929 32 bit (Intel)]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

I need to create a copy of a folder on a file share, e.g:

/foo-source
/foo-copy

I am doing this during an azure devops pipeline (yml) so am using an AzureCli task.
Note: There is an AzCopy task but it doesn't seem to support this scenario.
Note: There is an az storage file copy start-batch command but this can only copy files based on a pattern so for example I can copy foo-source/* to /foo-copy but then the result is /foo-copy/foo-source/somefile.txt rather than /foo-copy/somefile.txt. There is seemingly noo way around this.
So now I am exploring how to use azcopy directly during a devops build. Using the az cli makes it possible to use az copy but its unclear ow to pass these particular arguments.

[yonzhan]

Thank you for opening this issue, we will look into it.

[calvinhzy]

Can you run az storage azcopy run-command copy with the args?

[dazinator]

Nope I tried that also.