Need some guidance with building a project for my organization

[mohit19906]

We are creating a chat based AI model for our organization. This will help the call center team to look answers for user's queries. Each call center personnel has their own set of 100s of PDFs to find the answers from. Our initial design look as below -

1. Store all the PDFs in the Vector store DB using the Embedding model.
2. A call center personnel logs in to the system. S/he queries the chatbot. Chatbot calls the LLM using the prestored PDF vectors and the user query and get the answer back. We display the answer to the user.

We want it to flow in certain way as described below -

1. We do not want to feed all the PDFs to LLM. The PDFs specific to a particular call center personnel should be fed to LLM.
2. We do not want to load the PDFs in the vector store DB on the run time which is when the user is logging in or querying. We want the PDFs to be stored in the vector store DB before hand.

We have couple of challenges and questions with the implementation of our chatbot -

1. How to store the user specific PDFs in the vector store.
2. How to retrieve the user specific PDFs data at the time of query.
3. How to improve the performance of the LLM when dealing with large set of PDFs.
4. Which vector store DB is best to perform this task.
5. Till now, we have designed a simple chatbot using chroma. We fed couple of PDFs to it and asked some simple questions. It was able to answer correctly. But now, we have to make it work with 100s of PDFs and other types of docs. How can we optimize the performance and results of our bot.

This is our first AI project. Any help will be highly appreciated. Thank you in advance.