Skip to content

HTTP Errors are not Correctly Exposed by the Python Client SDK #865

@gzxu

Description

@gzxu

When using the tritonclient.http or tritonclient.http.aio package to call a Triton server behind a gateway, and the gateway returns a non-200 status code with a message, the message might not be correctly raised to callers.

As shown in https://github.com/triton-inference-server/client/blob/main/src/python/library/tritonclient/http/_utils.py#L35C1-L59C20, the library assumes the response body to be a JSON with an error field, but it's not always the case when the Triton server is behind a gateway.

    if response.status_code != 200:
        body = None
        try:
            body = response.read().decode("utf-8")
            error_response = (
                json.loads(body)
                if len(body)
                else {"error": "client received an empty response from the server."}
            )
            return InferenceServerException(
                msg=error_response["error"], status=str(response.status_code)
            )
        except Exception as e:
            return InferenceServerException(
                msg=f"an exception occurred in the client while decoding the response: {e}",
                status=str(response.status_code),
                debug_details=body,
            )

I would suggest including the whole response body even if it cannot be decoded as a JSON

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions