A question about faiss acceleration

[mc112611]

I am using this tutorial: Tutorial: Better Retrieval with Embedded Retrieval
When I want to loop through the query using while True, I always need Inferencing Samples,
This process is very time-consuming, and I would like to know if there is a way to accelerate it.
This process can only be accelerated by improving computational power.

When I search for problems on a PDF document, the following step usually takes several minutes：

[mc112611]

By the way, my code is completely consistent with the Better Retrieval with Embedding Retrieval tutorial

[mayankjobanputra]

@mc112611 are you using GPU for inference? That should speed things up for you. There are many other techniques such as using a smaller Retriever model, different FAISS index, and a smaller Reader model but again all these choices might affect your performance.