Avoid read response again when LRO operation raises HttpResponseError

[msyyc]

Describe the bug

Start from last year, Python SDK set strea=True when making http request (e.g. here). If HttpResponseError raised, SDK will read the response before raise it so that SDK users don't need to read it to get complete response error info. But I notice some CLI commands try to read the response again (e.g. here) which may meet StreamConsumedError/StreamClosedError since the response has already consumed innter SDK.

Related command

az deployment group create --resource-group <resource-group> --template-file main.bicep

Errors

see #31581 (comment)

Issue script & Debug output

see #31581 (comment)

Expected behavior

see #31581 (comment)

Environment Summary

see #31581 (comment)

Additional context

This issue may be the root cause of #31581

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[microsoft-github-policy-service]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @Azure/deployments-owners.

[msyyc]

My two cents for quick fix:

def _build_http_response_error_message(http_error):
    from azure.core.exceptions import StreamClosedError, StreamConsumedError
    try:
        error_txt = http_error.response.internal_response.text

        error_info = json.loads(error_txt)['error']
    except (StreamClosedError, StreamConsumedError):
        error_info = http_error.response.json()['error']
    error_details = error_info.pop('details') if 'details' in error_info else []
    err_messages = [f'{json.dumps(error_info)}']

    for detail in error_details:
        err_messages.append(_build_error_details(detail))

    return '\n\n'.join(err_messages)

[anthony-c-martin]

Another problem that I think is caused by the change to set _stream = True is that the logging you get from enabling --verbose --debug no longer includes the response bodies on certain API calls. This is something we use a lot to troubleshoot customer issues.

Instead of seeing the API response body logged with --debug --verbose, you see:

Response content:
Body is streamable

[anthony-c-martin]

On the _build_http_response_error_message error, I ran into the same issue on #31684 - here is the fix I made:

def _build_http_response_error_message(http_error):
    if not isinstance(http_error, HttpResponseError):
        raise TypeError("http_error must be an instance of HttpResponseError")

    error_info = http_error.response.json()['error']
    error_details = error_info.pop('details') if 'details' in error_info else []
    err_messages = [f'{json.dumps(error_info)}']

    for detail in error_details:
        err_messages.append(_build_error_details(detail))

    return '\n\n'.join(err_messages)

The naming of internal_response implies to me that it's not really intended for access outside of the SDK code, which may be why this caused a breaking change.