Similarity search accuracy

[jais001]

Hi Team,
I am using Haystack with ElasticsearchDocumentStore and EmbeddingRetriever in my application for semantic search. My doubt is the following, the content field is created by using Issue and Solution columns from the dataset,
eg: Issue: Error code #32
Solution: error occurred due to problem with xyz
content: "{ "Issue: Error code #32", "Solution: error occurred due to problem with xyz"}"
But when we are performing a search, with the query as "Error code #32" this particular result is coming at 8th position, I think this is because the model ignores #32 and brings results related to "Error code". Is there a way to increase the priority for #32 during the search?

[julian-risch]

Hi @JaisVJ if some of the queries are not full sentences but something like ""Error code #32" then this looks to me more like a keyword search, with "error", "code" and "32" being the keywords. You could check whether the results for this query get better if you use the BM25Retriever instead of the EmbeddingRetriever. If that's the case, you could change your pipeline such that it contains two retrievers. Our tutorial 11 contains an example. Just search for "CustomQueryClassifier" in that tutorial.
Another idea would be to store "issue" and "solution" in two different fields of the document when you create it from your dataset. "solution" could be still stored in "content" but "issue" could be stored as metadata instead. When you perform a search, you could use the metadata field to filter for documents containing "error code 32".

[jais001]

Hey @julian-risch !! Thank you for your support, will try these options.