[Azure AI Search] Vector search top and k_nearest_neighbors values should be able to receive different values

[pepenotti]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

Azure AI Search vector search top and k_nearest_neighbors values should be able to receive different values since both of them have different purposes.

langchain/libs/community/langchain_community/vectorstores/azuresearch.py

Lines 419 to 445 in bcb8ab5

	def vector_search_with_score(
	self, query: str, k: int = 4, filters: Optional[str] = None
	) -> List[Tuple[Document, float]]:
	"""Return docs most similar to query.
	Args:
	query: Text to look up documents similar to.
	k: Number of Documents to return. Defaults to 4.
	Returns:
	List of Documents most similar to the query and score for each
	"""
	from azure.search.documents.models import VectorizedQuery
	results = self.client.search(
	search_text="",
	vector_queries=[
	VectorizedQuery(
	vector=np.array(self.embed_query(query), dtype=np.float32).tolist(),
	k_nearest_neighbors=k,
	fields=FIELDS_CONTENT_VECTOR,
	)
	],
	filter=filters,
	top=k,
	)

Motivation

I'm working on an index where large content is chunked and indexed. This means that later on I'll be needing to search lots of documents (up to 50) when doing the similarity search.

At the same time, I might have smaller documents with less chunks.

The problem is that setting k as 50 will end up getting 50 k_nearest_neighbors . In my test cases, I ended up having a lot of noise when the document I was looking was small, and its nearest neighbors weren't as relevant as it should.

Proposal (If applicable)

No response

[ksood1996]

Hi @pepenotti I am facing the same issue at the moment. Were you able to find a potential workaround to independently set the nearest Neighbours parameter?

[pepenotti]

Hey @ksood1996, we ended up making our own extension of the Azure Search vector store and retriever, wrapping up the original where we could.

It's not ideal, but it's stable enough.