az storage fs file move not working with container files

[scoop96]

Describe the bug

Hi, I was looking for a way to move a file inside a storage account > container > folder to another folder in the same container.

thank you

Related command

this is the command

az storage fs file move -f $globalEnvironment.Storage.StorageContainer 
        --account-name $globalEnvironment.Storage.StorageAccountName 
        -path $globalEnvironment.Storage.StorageContainer  
        --new-path $("test/" +$globalEnvironment.Storage.BlobName)

I know im creating a bug, but could not open a simple issue for help for lack of documentation or understandment, sorry.

-f -> im using the container name
--account-name -> im using the storage account name
-path -> the file is in the container/file.csv so im simply putting the filename file.csv
--new-path -> the folder is created already (i saw the issue #28708 ) so im simply putting test/file.csv

Errors

Regarding the output that im getting is this one:

ERROR: unrecognized arguments: pingid-automation

as where pingid-automation is the container name.

Issue script & Debug output

DEBUG: cli.knack.cli: Command arguments: ['storage', 'fs', 'file', 'move', '-f', 'pingid-automation', '--account-name', 'identity02asaaaa001', '-path', 'pingid-automation', '--new-path', 'test/users_afc.csv', '--debug']
DEBUG: cli.knack.cli: __init__ debug log:
Cannot enable color.
DEBUG: cli.knack.cli: Event: Cli.PreExecute []
DEBUG: cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x02F22A78>, <function OutputProducer.on_global_arguments at 0x032609D8>, <function CLIQuery.on_global_arguments at 0x032787A8>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
DEBUG: cli.azure.cli.core: Modules found from index for 'storage': ['azure.cli.command_modules.storage']
DEBUG: cli.azure.cli.core: Loading command modules:
DEBUG: cli.azure.cli.core: Name                  Load Time    Groups  Commands
DEBUG: cli.azure.cli.core: storage                   0.750        59       273
DEBUG: cli.azure.cli.core: Total (1)                 0.750        59       273
DEBUG: cli.azure.cli.core: Loaded 59 groups, 273 commands.
DEBUG: cli.azure.cli.core: Found a match in the command table.
DEBUG: cli.azure.cli.core: Raw command  : storage fs file move
DEBUG: cli.azure.cli.core: Command table: storage fs file move
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x038213E8>]
DEBUG: cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to 'C:\Users\SE99236\.azure\commands\2024-11-07.18-51-07.storage_fs_file_move.18364.log'.
INFO: az_command_data_logger: command args: storage fs file move -f {} --account-name {} -p{} {} --new-path {} --debug
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x038473E8>]
DEBUG: cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/profiles/_shared.py", line 659, in _get_attr
AttributeError: module 'azure.mgmt.storage.v2023_05_01.models' has no attribute 'ActiveDirectoryPropertiesAccountType'

DEBUG: cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/profiles/_shared.py", line 659, in _get_attr
AttributeError: module 'azure.mgmt.storage.v2023_05_01.models' has no attribute 'ListKeyExpand'

DEBUG: cli.azure.cli.core.profiles._shared: Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/profiles/_shared.py", line 659, in _get_attr
AttributeError: module 'azure.mgmt.storage.v2023_05_01.models' has no attribute 'CorsRuleAllowedMethodsItem'

DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x0387DB18>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x0387DB68>, <function register_upcoming_breaking_change_info.<locals>.update_breaking_change_info at 0x0387DBB8>]
DEBUG: cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
DEBUG: cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
DEBUG: urllib3.connectionpool: Starting new HTTPS connection (1): app.aladdin.microsoft.com:443
DEBUG: urllib3.connectionpool: https://app.aladdin.microsoft.com:443 "GET /api/v1.0/suggestions?query=%7B%22command%22%3A+%22storage+fs+file+move%22%2C+%22parameters%22%3A+%22%22%7D&clientType=AzureCli&context=%7B%22versionNumber%22%3A+%222.65.0%22%2C+%22errorType%22%3A+%22UnrecognizedArguments%22%2C+%22correlationId%22%3A+%22d15b66dc-0a06-494b-9dd5-967e9efa6446%22%2C+%22subscriptionId%22%3A+%2254f6184f-874c-4082-b79a-269f5641cd1c%22%2C+%22eventId%22%3A+%229a39733c-16fe-473c-911c-4382a2bb6aee%22%7D HTTP/1.1" 200 None
DEBUG: cli.azure.cli.core.azclierror: NoneType: None

ERROR: cli.azure.cli.core.azclierror: unrecognized arguments: pingid-automation
ERROR: az_command_data_logger: unrecognized arguments: pingid-automation

Examples from AI knowledge base:
https://aka.ms/cli_ref
Read more about the command in reference docs
DEBUG: cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x03821528>]
INFO: az_command_data_logger: exit code: 2
INFO: cli.__main__: Command ran in 3.193 seconds (init: 0.520, invoke: 2.673)
INFO: telemetry.main: Begin splitting cli events and extra events, total events: 1
INFO: telemetry.client: Accumulated 0 events. Flush the clients.
INFO: telemetry.main: Finish splitting cli events and extra events, cli events: 1
INFO: telemetry.save: Save telemetry record of length 3787 in cache file under C:\Users\SE99236\.azure\telemetry\20241107185109626
INFO: telemetry.main: Begin creating telemetry upload process.
INFO: telemetry.process: Creating upload process: "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python.exe C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\azure\cli\telemetry\__init__.pyc C:\Users\SE99236\.azure C:\Users\SE99236\.azure\telemetry\20241107185109626"
INFO: telemetry.process: Return from creating process 21692
INFO: telemetry.main: Finish creating telemetry upload process.

Expected behavior

To simply move the folder

Environment Summary

azure-cli 2.65.0

core 2.65.0
telemetry 1.1.0

Dependencies:
msal 1.31.0
azure-mgmt-resource 23.1.1

Python location 'C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python.exe'
Extensions directory 'C:\Users\SE99236.azure\cliextensions'

Python (Windows) 3.11.8 (tags/v3.11.8:db85d51, Feb 6 2024, 21:52:07) [MSC v.1937 32 bit (Intel)]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[calvinhzy]

Hi @scoop96 can you try --path instead of -path

With -path the parameter got malformed.

[scoop96]

Hello @calvinhzy Thank you for the --path i didnt see that i was missing an slash.

The new error is regarding credentials.

For our implementation, we cannot use --auth-mode login , and we are behind of a proxy.

Regarding SAS, (correct me if im wrong) is not a valid solution, this script will be launched monthly to do some work over some files, and create a new file and move an existing file (Can I createa SAS for the new file from code?)

What can I do? thank you so much.

WARNING: 
There are no credentials provided in your command and environment, we will query for account key for your storage account.
It is recommended to provide --connection-string, --account-key or --sas-token in your command as credentials.

You also can add `--auth-mode login` in your command to use Azure Active Directory (Azure AD) for authorization if your login account is assigned required RBAC roles.
For more information about RBAC roles in storage, visit https://docs.microsoft.com/azure/storage/common/storage-auth-aad-rbac-cli.

In addition, setting the corresponding environment variables can avoid inputting credentials in your command. Please use --help to get more information about environment variable usage.
WARNING: 
Skip querying account key due to failure: (AuthorizationFailed) The client '49177ccc-7d27-45c1-9ee1-3f0573540971' with object id '49177ccc-7d27-45c1-9ee1-3f0573540971' does not have authorization to perform action 'Microsoft.Storage/storageAccounts/listKeys/action' over scope '/subscriptions/54f6184f-874c-4082-b79a-269f5641cd1c/resourceGroups/z0euw1acmrsg002/providers/Microsoft.Storage/storageAccounts/identity02asaaaa001' or the scope is invalid. If access was recently granted, please refresh your credentials.
Code: AuthorizationFailed
Message: The client '49177ccc-7d27-45c1-9ee1-3f0573540971' with object id '49177ccc-7d27-45c1-9ee1-3f0573540971' does not have authorization to perform action 'Microsoft.Storage/storageAccounts/listKeys/action' over scope '/subscriptions/54f6184f-874c-4082-b79a-269f5641cd1c/resourceGroups/z0euw1acmrsg002/providers/Microsoft.Storage/storageAccounts/identity02asaaaa001' or the scope is invalid. If access was recently granted, please refresh your credentials.
ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: please provide the sas token for the new file
Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/multiapi/storagev2/filedatalake/v2021_08_06/_data_lake_file_client.py", line 774, in rename_file
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 666, in execute
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 733, in _run_jobs_serially
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 725, in _run_job
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/storage/__init__.py", line 430, in new_handler
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/storage/__init__.py", line 429, in new_handler
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 703, in _run_job
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 336, in __call__
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
  File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/multiapi/storagev2/filedatalake/v2021_08_06/_data_lake_file_client.py", line 777, in rename_file
ValueError: please provide the sas token for the new file
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues

[calvinhzy]

Hi @scoop96, sorry for the late reply, you can try to generate sas with az storage fs generate-sas on the original file and use that. The file moving is actually doing a rename behind the scenes.