Azure Gov - az container commands result in API version errors

[andyhoffmanlifelens]

Describe the bug

After upgrading azure-cli to version 2.68.0 on MacOS (installed via brew), I am unable to connect to running containers instances (ACI resources) with Azure US Government cloud subscriptions. The API version "2024-05-01-preview" does not appear to be available in the government cloud.

We need to determine if there is either a way to DOWNGRADE back to prior version or manually set the API version within the az container command to move past this error and connect to our running container instances.

Related command

az container list -g

Similar results for:
az container exec -g -n --exec-command "/bin/bash"

Errors

(InvalidResourceType) The resource type 'containerGroups' could not be found in the namespace 'Microsoft.ContainerInstance' for api version '2024-05-01-preview'. The supported api-versions are '2017-08-01-preview,2017-10-01-preview,2017-12-01-preview,2018-02-01-preview,2018-04-01,2018-06-01,2018-07-01,2018-09-01,2018-10-01,2019-12-01,2020-11-01,2021-03-01,2021-07-01,2021-09-01,2021-10-01,2022-04-01-preview,2022-09-01,2022-10-01-preview,2023-02-01-preview,2023-05-01'.
Code: InvalidResourceType
Message: The resource type 'containerGroups' could not be found in the namespace 'Microsoft.ContainerInstance' for api version '2024-05-01-preview'. The supported api-versions are '2017-08-01-preview,2017-10-01-preview,2017-12-01-preview,2018-02-01-preview,2018-04-01,2018-06-01,2018-07-01,2018-09-01,2018-10-01,2019-12-01,2020-11-01,2021-03-01,2021-07-01,2021-09-01,2021-10-01,2022-04-01-preview,2022-09-01,2022-10-01-preview,2023-02-01-preview,2023-05-01'.

Issue script & Debug output

cli.knack.cli: Command arguments: ['container', 'list', '-g', 'rg-servers-dev-va', '--debug']
cli.knack.cli: init debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x102f431a0>, <function OutputProducer.on_global_arguments at 0x103114040>, <function CLIQuery.on_global_arguments at 0x103139800>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'container': ['azure.cli.command_modules.container']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name Load Time Groups Commands
cli.azure.cli.core: container 0.052 2 17
cli.azure.cli.core: Total (1) 0.052 2 17
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name Load Time Groups Commands Directory
cli.azure.cli.core: Total (0) 0.000 0 0
cli.azure.cli.core: Loaded 2 groups, 17 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command : container list
cli.azure.cli.core: Command table: container list
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x1044c9ee0>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/Users/ahoffman/.azure/commands/2025-01-24.14-08-29.container_list.30375.log'.
az_command_data_logger: command args: container list -g {} --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x104539940>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0x1045399e0>, <function register_cache_arguments..add_cache_arguments at 0x104539b20>, <function register_upcoming_breaking_change_info..update_breaking_change_info at 0x104539bc0>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x1031140e0>, <function CLIQuery.handle_query_parameter at 0x1031398a0>, <function register_ids_argument..parse_ids_arguments at 0x104539a80>]
cli.azure.cli.core.commands.client_factory: Getting management service client client_type=ContainerInstanceManagementClient
cli.azure.cli.core.auth.persistence: build_persistence: location='/Users/ahoffman/.azure/msal_token_cache.json', encrypt=False
cli.azure.cli.core.auth.binary_cache: load: /Users/ahoffman/.azure/msal_http_cache.bin
urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
msal.authority: Initializing with Entra authority: https://login.microsoftonline.us/52c02c3e-ef12-4b14-87d3-e6b844ee41dd
msal.authority: openid_config("https://login.microsoftonline.us/52c02c3e-ef12-4b14-87d3-e6b844ee41dd/v2.0/.well-known/openid-configuration") = {'token_endpoint': 'https://login.microsoftonline.us/52c02c3e-ef12-4b14-87d3-e6b844ee41dd/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.us/52c02c3e-ef12-4b14-87d3-e6b844ee41dd/discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.us/52c02c3e-ef12-4b14-87d3-e6b844ee41dd/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.us/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.us/52c02c3e-ef12-4b14-87d3-e6b844ee41dd/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.us/52c02c3e-ef12-4b14-87d3-e6b844ee41dd/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.us/52c02c3e-ef12-4b14-87d3-e6b844ee41dd/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.us/52c02c3e-ef12-4b14-87d3-e6b844ee41dd/kerberos', 'tenant_region_scope': 'USGov', 'tenant_region_sub_scope': 'DODCON', 'cloud_instance_name': 'microsoftonline.us', 'cloud_graph_host_name': 'graph.microsoftazure.us', 'msgraph_host': 'graph.microsoft.us', 'rbac_url': 'https://pasff.usgovcloudapi.net'}
msal.application: Broker enabled? None
cli.azure.cli.core.auth.credential_adaptor: CredentialAdaptor.get_token: scopes=('https://management.core.usgovcloudapi.net//.default',), kwargs={}
cli.azure.cli.core.auth.msal_credentials: UserCredential.get_token: scopes=('https://management.core.usgovcloudapi.net//.default',), claims=None, kwargs={}
msal.application: Cache hit an AT
msal.telemetry: Generate or reuse correlation_id: a9c73395-88ac-4e55-b495-6df0e50d57ed
cli.azure.cli.core.sdk.policies: Request URL: 'https://management.usgovcloudapi.net/subscriptions/ae64bc19-ee4b-4396-88b8-21a8b5341217/resourceGroups/rg-servers-dev-va/providers/Microsoft.ContainerInstance/containerGroups?api-version=2024-05-01-preview'
cli.azure.cli.core.sdk.policies: Request method: 'GET'
cli.azure.cli.core.sdk.policies: Request headers:
cli.azure.cli.core.sdk.policies: 'Accept': 'application/json'
cli.azure.cli.core.sdk.policies: 'x-ms-client-request-id': '987cac8c-da86-11ef-b32a-3240947a595d'
cli.azure.cli.core.sdk.policies: 'CommandName': 'container list'
cli.azure.cli.core.sdk.policies: 'ParameterSetName': '-g --debug'
cli.azure.cli.core.sdk.policies: 'User-Agent': 'AZURECLI/2.68.0 (HOMEBREW) azsdk-python-core/1.31.0 Python/3.12.8 (macOS-15.2-arm64-arm-64bit)'
cli.azure.cli.core.sdk.policies: 'Authorization': '*****'
cli.azure.cli.core.sdk.policies: Request body:
cli.azure.cli.core.sdk.policies: This request has no body
urllib3.connectionpool: Starting new HTTPS connection (1): management.usgovcloudapi.net:443
urllib3.connectionpool: https://management.usgovcloudapi.net:443 "GET /subscriptions/ae64bc19-ee4b-4396-88b8-21a8b5341217/resourceGroups/rg-servers-dev-va/providers/Microsoft.ContainerInstance/containerGroups?api-version=2024-05-01-preview HTTP/1.1" 404 502
cli.azure.cli.core.sdk.policies: Response status: 404
cli.azure.cli.core.sdk.policies: Response headers:
cli.azure.cli.core.sdk.policies: 'Cache-Control': 'no-cache'
cli.azure.cli.core.sdk.policies: 'Pragma': 'no-cache'
cli.azure.cli.core.sdk.policies: 'Content-Type': 'application/json; charset=utf-8'
cli.azure.cli.core.sdk.policies: 'Expires': '-1'
cli.azure.cli.core.sdk.policies: 'x-ms-failure-cause': 'gateway'
cli.azure.cli.core.sdk.policies: 'x-ms-request-id': '37e9fe61-cb7b-44aa-9eca-0d01dccd6a0d'
cli.azure.cli.core.sdk.policies: 'x-ms-correlation-request-id': '37e9fe61-cb7b-44aa-9eca-0d01dccd6a0d'
cli.azure.cli.core.sdk.policies: 'x-ms-routing-request-id': 'USDODEAST:20250124T190829Z:37e9fe61-cb7b-44aa-9eca-0d01dccd6a0d'
cli.azure.cli.core.sdk.policies: 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
cli.azure.cli.core.sdk.policies: 'X-Content-Type-Options': 'nosniff'
cli.azure.cli.core.sdk.policies: 'Date': 'Fri, 24 Jan 2025 19:08:29 GMT'
cli.azure.cli.core.sdk.policies: 'Content-Length': '502'
cli.azure.cli.core.sdk.policies: Response content:
cli.azure.cli.core.sdk.policies: {"error":{"code":"InvalidResourceType","message":"The resource type 'containerGroups' could not be found in the namespace 'Microsoft.ContainerInstance' for api version '2024-05-01-preview'. The supported api-versions are '2017-08-01-preview,2017-10-01-preview,2017-12-01-preview,2018-02-01-preview,2018-04-01,2018-06-01,2018-07-01,2018-09-01,2018-10-01,2019-12-01,2020-11-01,2021-03-01,2021-07-01,2021-09-01,2021-10-01,2022-04-01-preview,2022-09-01,2022-10-01-preview,2023-02-01-preview,2023-05-01'."}}
cli.azure.cli.core.azclierror: Traceback (most recent call last):
File "/opt/homebrew/Cellar/azure-cli/2.68.0/libexec/lib/python3.12/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/azure-cli/2.68.0/libexec/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 666, in execute
raise ex
File "/opt/homebrew/Cellar/azure-cli/2.68.0/libexec/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 734, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/azure-cli/2.68.0/libexec/lib/python3.12/site-packages/azure/cli/core/commands/init.py", line 716, in _run_job
result = list(result)
^^^^^^^^^^^^
File "/opt/homebrew/Cellar/azure-cli/2.68.0/libexec/lib/python3.12/site-packages/azure/core/paging.py", line 123, in next
return next(self._page_iterator)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/azure-cli/2.68.0/libexec/lib/python3.12/site-packages/azure/core/paging.py", line 75, in next
self._response = self._get_next(self.continuation_token)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/azure-cli/2.68.0/libexec/lib/python3.12/site-packages/azure/mgmt/containerinstance/operations/_container_groups_operations.py", line 545, in get_next
map_error(status_code=response.status_code, response=response, error_map=error_map)
File "/opt/homebrew/Cellar/azure-cli/2.68.0/libexec/lib/python3.12/site-packages/azure/core/exceptions.py", line 161, in map_error
raise error
azure.core.exceptions.ResourceNotFoundError: (InvalidResourceType) The resource type 'containerGroups' could not be found in the namespace 'Microsoft.ContainerInstance' for api version '2024-05-01-preview'. The supported api-versions are '2017-08-01-preview,2017-10-01-preview,2017-12-01-preview,2018-02-01-preview,2018-04-01,2018-06-01,2018-07-01,2018-09-01,2018-10-01,2019-12-01,2020-11-01,2021-03-01,2021-07-01,2021-09-01,2021-10-01,2022-04-01-preview,2022-09-01,2022-10-01-preview,2023-02-01-preview,2023-05-01'.
Code: InvalidResourceType
Message: The resource type 'containerGroups' could not be found in the namespace 'Microsoft.ContainerInstance' for api version '2024-05-01-preview'. The supported api-versions are '2017-08-01-preview,2017-10-01-preview,2017-12-01-preview,2018-02-01-preview,2018-04-01,2018-06-01,2018-07-01,2018-09-01,2018-10-01,2019-12-01,2020-11-01,2021-03-01,2021-07-01,2021-09-01,2021-10-01,2022-04-01-preview,2022-09-01,2022-10-01-preview,2023-02-01-preview,2023-05-01'.

cli.azure.cli.core.azclierror: (InvalidResourceType) The resource type 'containerGroups' could not be found in the namespace 'Microsoft.ContainerInstance' for api version '2024-05-01-preview'. The supported api-versions are '2017-08-01-preview,2017-10-01-preview,2017-12-01-preview,2018-02-01-preview,2018-04-01,2018-06-01,2018-07-01,2018-09-01,2018-10-01,2019-12-01,2020-11-01,2021-03-01,2021-07-01,2021-09-01,2021-10-01,2022-04-01-preview,2022-09-01,2022-10-01-preview,2023-02-01-preview,2023-05-01'.
Code: InvalidResourceType
Message: The resource type 'containerGroups' could not be found in the namespace 'Microsoft.ContainerInstance' for api version '2024-05-01-preview'. The supported api-versions are '2017-08-01-preview,2017-10-01-preview,2017-12-01-preview,2018-02-01-preview,2018-04-01,2018-06-01,2018-07-01,2018-09-01,2018-10-01,2019-12-01,2020-11-01,2021-03-01,2021-07-01,2021-09-01,2021-10-01,2022-04-01-preview,2022-09-01,2022-10-01-preview,2023-02-01-preview,2023-05-01'.
az_command_data_logger: (InvalidResourceType) The resource type 'containerGroups' could not be found in the namespace 'Microsoft.ContainerInstance' for api version '2024-05-01-preview'. The supported api-versions are '2017-08-01-preview,2017-10-01-preview,2017-12-01-preview,2018-02-01-preview,2018-04-01,2018-06-01,2018-07-01,2018-09-01,2018-10-01,2019-12-01,2020-11-01,2021-03-01,2021-07-01,2021-09-01,2021-10-01,2022-04-01-preview,2022-09-01,2022-10-01-preview,2023-02-01-preview,2023-05-01'.
Code: InvalidResourceType
Message: The resource type 'containerGroups' could not be found in the namespace 'Microsoft.ContainerInstance' for api version '2024-05-01-preview'. The supported api-versions are '2017-08-01-preview,2017-10-01-preview,2017-12-01-preview,2018-02-01-preview,2018-04-01,2018-06-01,2018-07-01,2018-09-01,2018-10-01,2019-12-01,2020-11-01,2021-03-01,2021-07-01,2021-09-01,2021-10-01,2022-04-01-preview,2022-09-01,2022-10-01-preview,2023-02-01-preview,2023-05-01'.
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x1044ca160>]
az_command_data_logger: exit code: 3
cli.main: Command ran in 0.374 seconds (init: 0.063, invoke: 0.311)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 4845 in cache file under /Users/ahoffman/.azure/telemetry/20250124140829390
telemetry.main: Begin creating telemetry upload process.
telemetry.process: Creating upload process: "/opt/homebrew/Cellar/azure-cli/2.68.0/libexec/bin/python /opt/homebrew/Cellar/azure-cli/2.68.0/libexec/lib/python3.12/site-packages/azure/cli/telemetry/init.py /Users/ahoffman/.azure /Users/ahoffman/.azure/telemetry/20250124140829390"
telemetry.process: Return from creating process 30392
telemetry.main: Finish creating telemetry upload process.

Expected behavior

Command should succeed and display a list of container instance/groups within the specified resource group.

Environment Summary

azure-cli 2.68.0

core 2.68.0
telemetry 1.1.0

Extensions:
account 0.2.5
init 0.1.0
rdbms-connect 1.0.6
ssh 2.0.6

Dependencies:
msal 1.31.1
azure-mgmt-resource 23.1.1

Python location '/opt/homebrew/Cellar/azure-cli/2.68.0/libexec/bin/python'
Extensions directory '/Users/ahoffman/.azure/cliextensions'

Python (Darwin) 3.12.8 (main, Dec 3 2024, 18:42:41) [Clang 16.0.0 (clang-1600.0.26.4)]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

az container commands were successful prior to updating to 2.68.0. We have been using container exec to access the internals of running containers in Azure Gov for past year without issues until recently.

[yonzhan]

Thank you for opening this issue, we will look into it.

[github-actions]

Here are some similar issues that might help you. Please check if they can solve your problem.

• In sovereign clouds, azure-cli version 2.67.0 fails when listing container instances (AzureUSGovernment) #30530
• azure container requires unavailable api version in sovereign clouds (e.g., Azure Government) #25791

---

Possible solution (Extracted from existing issue, might be incorrect; please verify carefully)

Solution 1:

The new API version is available in the US Gov Arizona region. It will be available in the US Gov Virginia region by Monday.

Reference:

• azure container requires unavailable api version in sovereign clouds (e.g., Azure Government) #25791 (comment)

Solution 2:

Thanks @navba-MSFT - I just tested and its working. Thanks for the quick turn!

Reference:

• azure container requires unavailable api version in sovereign clouds (e.g., Azure Government) #25791 (comment)

[andyhoffmanlifelens]

API versions for azure-cli 2.68.0 are still not available in Azure Government (primarily Virginia region). This is now hampering our deployment processes as well, as we utilize azure-cli commands during our automated deployment scripts. There also does not appear to be a way to downgrade the homebrew-installed azure-cli back to verison 2.67.0 (prior version that worked successfully).

Please give us an update on when the updated API versions will be available in Azure Government, as this will soon become a critical issue for us. Should we open a support ticket within our Azure Gov tenant?

[jason-neumann]

I am having this same issue. @navba-MSFT you resolved the parity issue as part of #25791 2 years ago. Any chance you can help resolve it again?

[Flightdeck73]

any updates on this? we are having issues trying to update a few different resources now on az gov.

NoRegisteredProviderFound: No registered resource provider found for location 'usgovvirginia' and API version '2024-05-01' for type 'firewallPolicies'.

[AHarman]

I think this is a duplicate of #23985