-
Notifications
You must be signed in to change notification settings - Fork 251
Closed
Description
When using the tritonclient.http or tritonclient.http.aio package to call a Triton server behind a gateway, and the gateway returns a non-200 status code with a message, the message might not be correctly raised to callers.
As shown in https://github.com/triton-inference-server/client/blob/main/src/python/library/tritonclient/http/_utils.py#L35C1-L59C20, the library assumes the response body to be a JSON with an error field, but it's not always the case when the Triton server is behind a gateway.
if response.status_code != 200:
body = None
try:
body = response.read().decode("utf-8")
error_response = (
json.loads(body)
if len(body)
else {"error": "client received an empty response from the server."}
)
return InferenceServerException(
msg=error_response["error"], status=str(response.status_code)
)
except Exception as e:
return InferenceServerException(
msg=f"an exception occurred in the client while decoding the response: {e}",
status=str(response.status_code),
debug_details=body,
)I would suggest including the whole response body even if it cannot be decoded as a JSON
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels