Including Citations leads to Incomplete Responses

[ram-atea]

I have encountered an issue where the responses are overly short and incomplete, even though the source documents contain much more detailed information about the topic. Initially, I thought the problem was with the prompt I had written. However, whether I used a detailed prompt or a very simple one, the answers didn't change much at all.

I then noticed that when I used the same prompt I had written from "chat_answer_question.prompty" in the "override prompt" feature, the responses became much fuller and complete.

I found that the problem for me was the citations. Whenever I remove any mention of citations from the prompt in "chat_answer_question.prompty", the responses are fuller and more accurate. But whenever I write any instructions about citations, the responses go back to being short and incomplete.

Are there any known issues with citation handling that might explain this behavior?
Is there something in the way citations are handled that's causing the responses to be incomplete?

[pamelafox]

Are you using the Thought Process tab to debug the answer quality, as discussed in the answer improvement guide?

If so, in the Thought Process tab:

1. Are the retrieved results always the same?
2. Do you see the full expected system message in the final call to the model (the last step in the thought process)? I'm wondering if there's a bug with the way you're overriding the prompt. If you look at that final step, you'll see exactly what the model sees.