Allow specifying different fields for similarity search and prompt context

[omihirofumi]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

Requirement

Currently, LangChain uses the same fields/content for both similarity search (e.g., with VectorStore) and generating the prompt context. However, there are cases where we may want to use different fields for these two operations.

For example, with a CSV dataset having fields like "Title", "Description", "Category", etc., we may want to perform the similarity search only on the "Title" and "Description" fields for better relevance. But when generating the prompt context, we want to include all fields like "Title", "Description", "Category" to provide more comprehensive information.

Motivation

Use Case

I'm working on a question-answering system using LangChain.js with a CSV dataset. The goal is to find the most relevant documents based on the question, but provide additional context from all fields when generating the final prompt for the language model.

Currently, I have to either:

1. Use all fields for similarity search (reducing relevance)
2. Use all fields for prompt context (lacking some information)

Being able to specify different fields for search and prompt context would allow me to optimize both relevance and information coverage.

Proposal (If applicable)

Potential Solution

One potential solution could be to provide additional options when creating the VectorStore, something like:

const vectorStore = await HNSWLib.fromDocuments(
  docs, 
  embeddings,
  {
    searchFields: ['Title', 'Description'], // Fields for similarity search
    promptFields: ['Title', 'Description', 'Category'] // Fields for prompt context
    ...
  }
);