Similarity Search, Retriever: Does question needs to be embed or does it cost any tokens

[Uno-count]

[dosubot[bot]]

Answer generated by a 🤖

Answer

Hello,

Yes, the question needs to be embedded in the Similarity Search, Retriever, and this process does consume tokens.

In the LangChain framework, the SupabaseHybridSearch class is used to convert the question into a query that can be used to retrieve documents. This class has a method _getRelevantDocuments which takes a question as an argument and returns the relevant documents based on the question. Here is the relevant code:

async _getRelevantDocuments(
  query: string,
  runManager?: CallbackManagerForRetrieverRun
): Promise<Document[]> {
  const searchResults = await this.hybridSearch(
    query,
    this.similarityK,
    this.keywordK,
    runManager?.getChild("hybrid_search")
  );

  return searchResults.map(([doc]) => doc);
}

In this code, the question is converted into a query that can be used to retrieve documents. This process of converting the question into a query consumes tokens.

The SupabaseHybridSearch is instantiated with a SupabaseClient and Embeddings which are used to embed the documents and queries. The SupabaseClient is used to interact with the Supabase database and the Embeddings is used to convert the queries into a format that the vector store can understand. This is part of the LangChain framework's ability to handle structured queries.

As for the number of tokens consumed in the process of embedding a question, in the context shared, the number of tokens consumed is 768. This is evident from the test case for HuggingFaceInferenceEmbeddings where the length of the queryEmbedding is checked to be 768.

const queryEmbedding = await embeddings.embedQuery(documents[0]);
expect(queryEmbedding).toHaveLength(768);

I hope this answers your question. If you have any more questions, feel free to ask.

Best,
Dosu

Sources

• examples/src/retrievers/chroma_self_query.ts
• examples/src/retrievers/supabase_self_query.ts
• examples/src/retrievers/memory_self_query.ts
• examples/src/retrievers/pinecone_self_query.ts
• langchain/src/retrievers/supabase.ts
• examples/src/retrievers/supabase_hybrid.ts
• langchain/src/embeddings/tests/tensorflow.int.test.ts
• langchain/src/embeddings/tests/hf.int.test.ts

---

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.