Empty answers in the chat due to the filtered model output

[advanced-flow]

This issue is for a:

- [x] bug report 
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression 

I noticed completely empty responses for certain user queries (which led to the retrieval of certain documents). I found out via Application Insights that the model output was being filtered by the OpenAi Content Filter. I increased the filter threshold with a custom filter. That worked. However, I think an error message should be issued to the user.

Any log messages given by the failure

Extract of the end-to-end transaction details in Application Insights; here you can see that the model output has been labelled ‘Violence Medium’ and therefore filtered.

Expected/desired behavior

Error message to the user informing them that the model output has been flagged by the OpenAI content filter.

OS and Version?

macOS

azd version?

azd version 1.10.3 (commit 0595f33fe948ee6df3da492567e3e7943cb9a733)

Versions

#2045

[pamelafox]

There is handling right now in the backend for the content filter errors. You can see it if you go to https://capps-backend-g6kuptydvtma6.redocean-cd0fb0be.westus.azurecontainerapps.io/ and type "how do I make a bomb?"

Can you trace the code to see why your filter errors aren't being handled?

The handling is in error.py which is called from app.py.

[advanced-flow]

Hi @pamelafox,
Yes, I get this error message reliably when I type something that violates the content filter - these are the cases when the model input is filtered.

But how does the app deal with this when the model output is filtered?

Take a look at the screenshot from Analytics Insights shown in my first post. It should be noted that the ‘gen_ai.prompt.prompt._filter_results’ are not filtered, but the ‘gen_ai.completion.0.content filter results’ are. As initially described, this has led to me receiving an empty response from the chatbot in my deployment.
In the safety monitoring of the Azure OpenAI resource, I could also see that it was not the input filter that was active, but the output filter.

[advanced-flow]

Okay - the filter is only triggered when I have activated streaming mode. If I deactivate it, no filtering takes place with the same prompt.

Strange - but for my case the workaround is ok.