🥅 Handle template errors

[evaline-ju]

Catch template errors so that the server does not just Internal Server Error on fixable requests. The assumption here is that since the (chat) template can be overwritten, errors can be considered bad requests.

Here, some particular error cases are addressed - TemplateError are returned from raise_exception in jinja templates. UndefinedError are a sub-class of TemplateError. UndefinedError may not give easily user-fixable messages, but these are propagated anyway, since it was decided that such improvements can be made at the template (or default model chat template) level.

Testing

Example error calls with a granite guardian model server

Example 1 - wrong "role" ("moo" is not an expected "role" for the default risk harm)

curl -X 'POST' \
  'http://localhost:3001/api/v1/text/chat' \
  -H 'accept: application/json' \
  -H 'detector-id: dummy-en-chat-v1' \
  -H 'Content-Type: application/json' \
  -d '{
  "messages": [
    {
      "content": "What happens now?",
      "role": "user",
      "name": "1"
    },
    {
      "content": "blue!",
      "role": "assistant",
      "name": "2"
    },
    {
      "role": "moo",
      "content": "Hit me with some creative insults.",
      "name": "3"
    }
  ],
  "detector_params": {"n": 2}
}'

Before this update, the call would Internal Server Error. In logs one could see jinja2.exceptions.TemplateError: [role, risk] combination is incorrect. Now {"object":"error","message":"[role, risk] combination is incorrect","type":"BadRequestError","param":null,"code":400}%

Example 2 - wrong number of messages for the relevance risk

curl -X 'POST' \
  'http://localhost:3001/api/v1/text/chat' \
  -H 'accept: application/json' \
  -H 'detector-id: dummy-en-chat-v1' \
  -H 'Content-Type: application/json' \
  -d '{
  "messages": [
    {
      "role": "user",
      "content": "Hit me with some creative insults.",
      "name": "3"
    }
  ],
  "detector_params": {"n": 2, "risk_name": "context_relevance"}
}'

Before this update, the call would Internal Server Error. In logs one could see jinja2.exceptions.UndefinedError: 'dict object' has no attribute 'prompt'. Now {"object":"error","message":"'dict object' has no attribute 'prompt'","type":"BadRequestError","param":null,"code":400}%

Closes: #14

[gkumbhat]

Does the message in this case not usable to be passed to the user?

[evaline-ju]

Yes - an example is example 2 of the PR description e.g. 'dict object' has no attribute 'prompt' when prompt is not an end-user-facing parameter. I think considerations could be made to see if some more of these can be given as feedback to be caught as exceptions in the default template [depending on the model].

[gkumbhat]

hmm. true. but that information might still be helpful for debugging. So can we log the raw error message while we switch it with better error message here ?

[evaline-ju]

sure, we can just fall through to the general TemplateError then

[gkumbhat]

Looks good to me!