Bring Your Own Data AND General Search?

[HBKarlHolzinger]

Is it possible to search on your own data and the generic completion service at the same time? I've got the bring your own data working via AzureSearchChatExtensionConfiguration. The use case is as follows:

1. Upload a bunch of domain specific KB articles and index them.
2. Allow users to search for answers derived from those articles.
3. If the answer isn't available in those articles, allow the generic completion to provide an answer.

I suppose I could just do a search on the index and check to see if the answer is some flavor of "The requested information is not available in the retrieved data," and if so, send the request again without an index. But I was hoping I was just missing a flag somewhere that would just do this combined search.

[sophialagerkranspandey]

You might be able to get around this by using prompt engineering - you can include in the prompts if the provided context doesn't answer the question to try and answer from what the model knows.

[HBKarlHolzinger]

That sort of works, but breaks easily.

For example, in our curremt KB, we have an article on updating your Outlook signature (because there are some steps specific to our enviroment). We don't have an article on pivoting data in Excel, because that doesn't have any special circumstances we need to account for.

If I ask the Outlook question first, I get a response sourced from the article in the index with citations. If I then ask the Excel question, I get a general response from the model's capabilities. Working exactly as you suggested.

If I reverse the order and ask the Excel question first, I get the general response from the model's capabilities. When I then ask the Outlook question, I get a general response from the model's capabilities rather than sourced from the article in the index. The behaviour seems to be that as soon as I open the door to non-indexed knowledge, the indexed knowledge gets ignored.

For what it's worth, I tried updating the system message, and that did not change the behaviour at all. When I appened the following language to each prompt, that's when the behaviour changed: "If the requested information is not available in the retrieved data, try and answer general knowledge from the model."

Is there different language you'd recommend?