Is it possible to run multiple queries in a query pipeline in Haystack version >2.0 ?

[kathy-lee]

Hi all,
I'm a beginner in Haystack :) May I ask in this tutorial sentencetransformerstextembedder in pipeline, is it possible to run the pipeline with multiple queries at once? By which I mean the following run of the pipeline in this tutorial:

result = query_pipeline.run({"text_embedder":{"text": query}})

I see 'run_batch' feature, but seems it was in Haystack version < 2.0.
And I see in the __init__() interface of class SentenceTransformersTextEmbedder there is a parameter batch_size: int = 32, but I'm not sure if it's exactly what I'm looking for, and I could not find a tutorial with its usage either.

Thanks for any help!

[Amnah199]

Hi @kathy-lee, Welcome to Haystack! 😊

In Haystack 2.0, processing multiple queries simultaneously with run_batch is not directly supported in the same way as older version.

The batch_size parameter in the SentenceTransformersTextEmbedder can optimize the embedding process by controlling how many queries are processed together during embedding. However, this does not translate into the pipeline accepting multiple queries.

[kathy-lee]

Thank you for answering my question! @Amnah199
Will the processing of multiple queries be a feature in future version, or is such a PR welcome?

[edwinlim0919]

I am also wondering if the Haystack team is considering query batching as a feature, as it seems a lot of pipeline performance is left off of the table without support for batching. Please let us know.

Thanks!