Exploration of Query Size and Model Configuration in Vector-Based Searches

[zhongshuai-cao]

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [x] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Expected/desired behavior

Description:
I'm currently delving into different search scenarios that rely exclusively on vector-based searches. A couple of questions arise:

Does enlarging the query size improve the accuracy of results?
Would utilizing a quicker, albeit possibly less capable model like gpt-3.5-turbo for queries and smarter gpt-4 for answers be effective?
To implement this approach, there's a need for:

Configurability between the GPT model used for queries and the one used for answers.
The ability to set query limits tailored for different scenarios, such as text-based versus vector-based searches.
Any insights or suggestions on these matters would be greatly appreciated.

---

Thanks! We'll be in touch soon.

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this issue will be closed.

[pamelafox]

This is a great discussion, I recommend looking at this blog post for some research into search quality:
https://aka.ms/ragrelevance

We also did a video about the effects of vector compression options on quality:
https://techcommunity.microsoft.com/blog/azure-ai-foundry-blog/rag-time-journey-3-optimize-your-vector-index-for-scale/4395134

If you have updated questions specifically after reading through this, please open a new issue.