No module named 'distutils' on azure CLI docker image with `ml` extension

[andreavs]

Describe the bug

When running the azure cli docker image

docker run -it mcr.microsoft.com/azure-cli:2.67.0 /bin/bash

and adding the ml extension, I see that running az ml gives the error

No module named 'distutils'

Any other command using az ml seems to give errors.

Related command

az ml datastore show

Errors

root [ / ]# az ml datastore show
No module named 'distutils'
import error: 
No module named 'distutils'
The command failed with an unexpected error. Here is the traceback:
The operation 'azext_mlv2.custom#ml_datastore_show' is invalid.
Traceback (most recent call last):
  File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 61, in get_op_handler
    handler = getattr(handler, part)
              ^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'azext_mlv2.custom' has no attribute 'ml_datastore_show'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/az/lib/python3.12/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 564, in execute
    self.commands_loader.load_arguments(command)
  File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/__init__.py", line 526, in load_arguments
    self.command_table[command].load_arguments()  # this loads the arguments via reflection
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 321, in load_arguments
    super(AzCliCommand, self).load_arguments()
  File "/usr/lib64/az/lib/python3.12/site-packages/knack/commands.py", line 104, in load_arguments
    cmd_args = self.arguments_loader()
               ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 367, in arguments_loader
    cmd_args = self.load_getter_op_arguments(self.op_path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 70, in load_getter_op_arguments
    op = self.get_op_handler(getter_op_path)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 66, in get_op_handler
    raise ValueError("The operation '{}' is invalid.".format(op_path))
ValueError: The operation 'azext_mlv2.custom#ml_datastore_show' is invalid.

Issue script & Debug output

root [ / ]# az ml datastore show --debug
cli.knack.cli: Command arguments: ['ml', 'datastore', 'show', '--debug']
cli.knack.cli: init debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7507fb10a340>, <function OutputProducer.on_global_arguments at 0x7507fae71440>, <function CLIQuery.on_global_arguments at 0x7507faea6a20>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'ml': ['azext_mlv2']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name Load Time Groups Commands
cli.azure.cli.core: Total (0) 0.000 0 0
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name Load Time Groups Commands Directory
azext_mlv2.manual.vendored_curated_sdk.azure.ai.ml._azure_environments: Using the default cloud configuration: 'AzureCloud'.
azext_mlv2.manual.vendored_curated_sdk.azure.ai.ml._azure_environments: Using the default cloud configuration: 'AzureCloud'.
azext_mlv2.manual.vendored_curated_sdk.azure.ai.ml._azure_environments: Using the default cloud configuration: 'AzureCloud'.
No module named 'distutils'
cli.azure.cli.core: ml 3.567 24 161 /root/.azure/cliextensions/ml
cli.azure.cli.core: Total (1) 3.567 24 161
cli.azure.cli.core: Loaded 24 groups, 161 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command : ml datastore show
cli.azure.cli.core: Command table: ml datastore show
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7507f9ee2fc0>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/root/.azure/commands/2024-12-03.13-07-07.ml_datastore_show.41.log'.
az_command_data_logger: command args: ml datastore show --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x7507f9f304a0>]
import error:
No module named 'distutils'
cli.azure.cli.core.azclierror: Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 61, in get_op_handler
handler = getattr(handler, part)
^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'azext_mlv2.custom' has no attribute 'ml_datastore_show'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 564, in execute
self.commands_loader.load_arguments(command)
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/init.py", line 526, in load_arguments
self.command_table[command].load_arguments() # this loads the arguments via reflection
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 321, in load_arguments
super(AzCliCommand, self).load_arguments()
File "/usr/lib64/az/lib/python3.12/site-packages/knack/commands.py", line 104, in load_arguments
cmd_args = self.arguments_loader()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 367, in arguments_loader
cmd_args = self.load_getter_op_arguments(self.op_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 70, in load_getter_op_arguments
op = self.get_op_handler(getter_op_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 66, in get_op_handler
raise ValueError("The operation '{}' is invalid.".format(op_path))
ValueError: The operation 'azext_mlv2.custom#ml_datastore_show' is invalid.

cli.azure.cli.core.azclierror: The command failed with an unexpected error. Here is the traceback:
az_command_data_logger: The command failed with an unexpected error. Here is the traceback:
cli.azure.cli.core.azclierror: The operation 'azext_mlv2.custom#ml_datastore_show' is invalid.
Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 61, in get_op_handler
handler = getattr(handler, part)
^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'azext_mlv2.custom' has no attribute 'ml_datastore_show'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 564, in execute
self.commands_loader.load_arguments(command)
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/init.py", line 526, in load_arguments
self.command_table[command].load_arguments() # this loads the arguments via reflection
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 321, in load_arguments
super(AzCliCommand, self).load_arguments()
File "/usr/lib64/az/lib/python3.12/site-packages/knack/commands.py", line 104, in load_arguments
cmd_args = self.arguments_loader()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 367, in arguments_loader
cmd_args = self.load_getter_op_arguments(self.op_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 70, in load_getter_op_arguments
op = self.get_op_handler(getter_op_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 66, in get_op_handler
raise ValueError("The operation '{}' is invalid.".format(op_path))
ValueError: The operation 'azext_mlv2.custom#ml_datastore_show' is invalid.
az_command_data_logger: The operation 'azext_mlv2.custom#ml_datastore_show' is invalid.
Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 61, in get_op_handler
handler = getattr(handler, part)
^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'azext_mlv2.custom' has no attribute 'ml_datastore_show'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.12/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 564, in execute
self.commands_loader.load_arguments(command)
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/init.py", line 526, in load_arguments
self.command_table[command].load_arguments() # this loads the arguments via reflection
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 321, in load_arguments
super(AzCliCommand, self).load_arguments()
File "/usr/lib64/az/lib/python3.12/site-packages/knack/commands.py", line 104, in load_arguments
cmd_args = self.arguments_loader()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 367, in arguments_loader
cmd_args = self.load_getter_op_arguments(self.op_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 70, in load_getter_op_arguments
op = self.get_op_handler(getter_op_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/az/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 66, in get_op_handler
raise ValueError("The operation '{}' is invalid.".format(op_path))
ValueError: The operation 'azext_mlv2.custom#ml_datastore_show' is invalid.
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7507f9ee3240>]
az_command_data_logger: exit code: 1
cli.main: Command ran in 3.815 seconds (init: 0.223, invoke: 3.592)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 7460 in cache file under /root/.azure/telemetry/20241203130707109
telemetry.main: Begin creating telemetry upload process.
telemetry.process: Creating upload process: "/usr/bin/python3.12 /usr/lib/az/lib/python3.12/site-packages/azure/cli/telemetry/init.py /root/.azure /root/.azure/telemetry/20241203130707109"
telemetry.process: Return from creating process 45
telemetry.main: Finish creating telemetry upload process.

Expected behavior

The command should run without throwing Python exceptions.

Environment Summary

azure-cli 2.67.0

core 2.67.0
telemetry 1.1.0

Extensions:
ml 2.32.4

Dependencies:
msal 1.31.0
azure-mgmt-resource 23.1.1

Python location '/usr/bin/python3.12'
Extensions directory '/root/.azure/cliextensions'

Python (Linux) 3.12.3 (main, Oct 5 2024, 02:16:09) [GCC 13.2.0]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

I can make it work correctly by running

/usr/bin/python3.12 -m ensurepip --upgrade
/usr/bin/python3.12 -m pip install setuptools

[yonzhan]

Thank you for opening this issue, we will look into it.

[microsoft-github-policy-service]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github.

[jayesh-tanna]

Please install ml extension version 2.36.3, it is officially released. this version contains the fix.

[microsoft-github-policy-service]

Hi @@andreavs. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text “/unresolve” to remove the “issue-addressed” label and continue the conversation.