Ensure that GPT never answers from outside its sources

[viru2206]

Is there any way we can get answer only from the provided sources. I tried giving a proper command in the system message of chatreadretrieveread script but it still provides me with the answers and it provides citations also. Yeah it happens less now but its still happening sometimes.

[GauravSingh1A]

Yeah, I would also love to know how we can do that.

[pamelafox]

Can you share examples of where it's answering questions that aren't in the provided sources? Have you tried using GPT-4? (It is generally better at sticking to instructions).

[viru2206]

Yeah sure, I have attached an image in which you can see that it provided reponse that are not from PDF and also gave citations. The citation does not have any information regarding the query. Also, the GPT knows that the answers are not from the sources. No, I haven't tried GPT-4, I have applied for GPT-4 access but hasn't got yet.

[shamprasadrh]

can you try setting temperature to 0 ?

[viru2206]

Yeah, I already tried that.

[Kesavaraja]

I am also facing the same issue. I got the content and citations from Wikipedia. Which is weird for me.

[sabrab]

Same, any update on this?

[FranciscoMMendes]

I am also facing the same issue on my side. I have tried to improve the prompt but it does not change the reply and the citations provided.
When I ask the chat a question about some personality or celebrity for example, the chat fetches the information from Wikipedia instead of searching the documents provided.

[chriskabat63]

I added a little more to the question prompt and seem to get pretty good results. It seems to still honor the system message, but I think this helps reinforce that it should use the included sources.

Here is kind of what the prompt looks like now:

Answer the following question only using the sources listed below:

Question: 1933 superbowl winner

Sources:

Source 1: ….

Source 2: …

I basically just replaced line 179:
https://gist.github.com/chriskabat63/59da4f18657ab2b2f043cedd18d1c9ed

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this issue will be closed.

[pamelafox]

Are you all still seeing this? We may move to a slightly tweaked prompt that emphasizes coherence to sources more, you could try this prompt:

https://github.com/Azure-Samples/azure-search-openai-demo/pull/967/files#diff-d6c03003d61bd0a593ea552fb76573df32d24fe6ac495da5ac2f7250cc372b96