Fails while running prepdocs.py from Azure Cloud Shell

[lineshgajerams]

Please provide us with the following information:

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Run azd up command. It will provision all required services; however it will fail while running prepdocs.py consistently.

Any log messages given by the failure

Running "prepdocs.py"
AzureDeveloperCliCredential.get_token failed: {"type":"consoleMessage","timestamp":"2023-09-13T18:17:14.557034123Z","data":{"message":"fetching token: invalid CloudShell token API response code: 400, content: {"error":{"code":"AudienceNotSupported","message":"Audience https://cognitiveservices.azure.com is not a supported MSI token audience. Supported audiences: https://management.core.windows.net/,https://management.azure.com/,https://graph.windows.net/,https://vault.azure.net,https://datalake.azure.net/,https://outlook.office365.com/,https://graph.microsoft.com/,https://batch.core.windows.net/,https://analysis.windows.net/powerbi/api,https://storage.azure.com/,https://rest.media.azure.net,https://api.loganalytics.io,https://ossrdbms-aad.database.windows.net,https://www.yammer.com,https://digitaltwins.azure.net,0b07f429-9f4b-4714-9392-cc5e8e80c8b0,822c8694-ad95-4735-9c55-256f7db2f9b4,https://dev.azuresynapse.net,https://database.windows.net,https://quantum.microsoft.com,https://iothubs.azure.net,2ff814a6-3304-4ab8-85cb-cd0e6f879c1d,https://azuredatabricks.net/,ce34e7e5-485f-4d76-964f-b3d2b16d1e4f,https://azure-devices-provisioning.net,https://managedhsm.azure.net,499b84ac-1321-427f-aa17-267ca6975798,https://api.adu.microsoft.com/,https://purview.azure.net/,6dae42f8-4368-4678-94ff-3960e28e3630\"}}\n"}}

Traceback (most recent call last):
File "/home/linesh/azure-search-openai-demo/scripts/.venv/lib/python3.9/site-packages/azure/identity/_credentials/azd_cli.py", line 191, in _run_command
return subprocess.check_output(args, **kwargs)
File "/usr/lib/python3.9/subprocess.py", line 424, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/usr/lib/python3.9/subprocess.py", line 528, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/bin/sh', '-c', 'azd auth token --output json --scope https://cognitiveservices.azure.com/.default --tenant-id 16b3c013-d300-468d-ac64-7eda0820b6d3']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/linesh/azure-search-openai-demo/./scripts/prepdocs.py", line 385, in
openai.api_key = azd_credential.get_token("https://cognitiveservices.azure.com/.default").token
File "/home/linesh/azure-search-openai-demo/scripts/.venv/lib/python3.9/site-packages/azure/identity/_internal/decorators.py", line 32, in wrapper
token = fn(*args, **kwargs)
File "/home/linesh/azure-search-openai-demo/scripts/.venv/lib/python3.9/site-packages/azure/identity/_credentials/azd_cli.py", line 122, in get_token
output = _run_command(command, self._process_timeout)
File "/home/linesh/azure-search-openai-demo/scripts/.venv/lib/python3.9/site-packages/azure/identity/_credentials/azd_cli.py", line 205, in _run_command
raise ClientAuthenticationError(message=message)
azure.core.exceptions.ClientAuthenticationError: {"type":"consoleMessage","timestamp":"2023-09-13T18:17:14.557034123Z","data":{"message":"fetching token: invalid CloudShell token API response code: 400, content: {"error":{"code":"AudienceNotSupported","message":"Audience https://cognitiveservices.azure.com is not a supported MSI token audience. Supported audiences: https://management.core.windows.net/,https://management.azure.com/,https://graph.windows.net/,https://vault.azure.net,https://datalake.azure.net/,https://outlook.office365.com/,https://graph.microsoft.com/,https://batch.core.windows.net/,https://analysis.windows.net/powerbi/api,https://storage.azure.com/,https://rest.media.azure.net,https://api.loganalytics.io,https://ossrdbms-aad.database.windows.net,https://www.yammer.com,https://digitaltwins.azure.net,0b07f429-9f4b-4714-9392-cc5e8e80c8b0,822c8694-ad95-4735-9c55-256f7db2f9b4,https://dev.azuresynapse.net,https://database.windows.net,https://quantum.microsoft.com,https://iothubs.azure.net,2ff814a6-3304-4ab8-85cb-cd0e6f879c1d,https://azuredatabricks.net/,ce34e7e5-485f-4d76-964f-b3d2b16d1e4f,https://azure-devices-provisioning.net,https://managedhsm.azure.net,499b84ac-1321-427f-aa17-267ca6975798,https://api.adu.microsoft.com/,https://purview.azure.net/,6dae42f8-4368-4678-94ff-3960e28e3630\"}}\n"}}

ERROR: failed running post hooks: 'postprovision' hook failed with exit code: '1', Path: './scripts/prepdocs.sh'. : exit code: 1

Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

azd version?

run azd version and copy paste here.

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

[vhvb1989]

related to: Azure/azure-dev#2410 (comment)

[danieljurek]

Thanks for filing, @lineshgajerams. I'm able to reproduce this locally in my Cloud Shell instance and we think this behavior is caused by behavior in Cloud Shell.

One can use azd itself to get the same error output (when running in Cloud Shell):

azd auth token --output json --scope https://cognitiveservices.azure.com

Which works out to a request to fetch an access token using the Cloud Shell metadata API. The minimal repro is (in Cloud Shell):

curl http://localhost:50342/oauth2/token --data "resource=https://cognitiveservices.azure.com" -H Metadata:true -s

I'm in communication with the Cloud Shell team about this to find a way forward.

I'm also checking on some possible workarounds.

[danieljurek]

@lineshgajerams -- I also have a workaround that you can use in the short term while we dig in on the issue with Cloud Shell:

In this case, use device-code authentication:

azd auth login --use-device-code
azd up

The azd up command will succeed end to end.

[lineshgajerams]

@danieljurek Thank you for the work around. I will try today and see if it works from Azure Cloud Shell. I already have a customer engagement to enable Azure Cognitive Seach with OpenAI integration and I want to use this accelerator. I have been doing hands on to make sure It works in customer subscription

[lineshgajerams]

@danieljurek

After successfully completed the login with device code, getting following error. I replaced my subscription with *****

Cloud Shell is automatically authenticated under the initial account used to sign in. Run 'azd auth login' only if you need to use a different account.
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code EJVYJ8N5G to authenticate.

Device code authentication completed.
Logged in to Azure.
PS /home/linesh/azure-search-openai-demo> azd up
(✓) Done: Retrieving locations...

ERROR: listing locations: failed retrieving Azure location for account '': failed to resolve user access to subscription with ID '***'. If you recently gained access to this subscription, run azd auth login again to reload subscriptions.
Otherwise, visit this subscription in Azure Portal using the browser, then run azd auth login

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this issue will be closed.

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this issue will be closed.

[aropenai]

Do we have any workaround for this?

[pamelafox]

I do not believe this is currently an active issue, as I recently worked with the Cloud Shell team on a fix to this repo for Cloud Shell compatibility (Node version), so I assume that it is working otherwise. Closing this issue. Please open a new issue with details if you encounter again.