Make a dynamic prompt based on the context documents length

[sneko]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

const promptCanvas = ChatPromptTemplate.fromMessages([
  [
    'system',
    `...
---
CONTEXTE :
{context}
---
`,
  ],
  new MessagesPlaceholder('chat_history'),
  ['human', '{input}'],
]);

const combineDocsChain = await createStuffDocumentsChain({
  llm: this.mistralaiClient,
  prompt: promptCanvas,
  documentSeparator: '\n',
});

const chain = await createRetrievalChain({
  retriever: this.initiativesVectorStore.asRetriever(5),
  combineDocsChain: combineDocsChain,
});

const stream = await chain.stream(
  {
    chat_history: await session.history.chatHistory.getMessages(),
    input: input,
  },
  ...
);

Description

With the assistant, when users ask for listing a few things it will always returns <= 5 items in the context, and so in the streamed answer when the assistant displays them as a list.

I would like my assistant to tell the user there is more available, and that by precising its request the user will see other items. My first idea was:

1. Instead of doing asRetriever(5), to do asRetriever(6)
2. If there is a 6th item, replace its content by a sentence there is more items in the database, tell the user to precise the request to see others

Unfortunately by looking at the code

langchainjs/langchain/src/chains/combine_documents/stuff.ts

Line 31 in 0ea9f92

* @param documentSeparator String separator to use between formatted document strings.

and

langchainjs/langchain/src/chains/combine_documents/base.ts

Line 15 in 0ea9f92

documentSeparator,

it seems my only choice is to rewrite createStuffDocumentsChain() to have a custom formatDocuments() function.

Do you see another proper way to do so? Maybe I'm thinking this problem the wrong way?

I also tought about not chaining the retriever and doing manual call aside, but if doing the embeddings at first, passing them to the retriever is possible, but I'm unaware to pass those embeddings as the input in the .stream() function (input of a raw text input).

Any help appreciated, thank you,

cc @nfcampos @jacoblee93

System Info

EDIT: I ended duplicating and modifying https://github.com/langchain-ai/langchainjs/blob/main/langchain/src/chains/combine_documents/stuff.ts and https://github.com/langchain-ai/langchainjs/blob/main/langchain/src/chains/combine_documents/base.ts to make the modification of the formatter. I tried at first to replace my whole logic of using langchain helpers by using a raw RunnableSequence.from([...]) as in the example but I was having typings issue due to chat history... so just took the facility of duplicating 👍

[dosubot[bot]]

Based on your requirements, it seems like you're on the right track. You can indeed modify the formatDocuments() function to check the number of documents and return a specific message if there are more than 5 documents. Here's a conceptual example of how you could modify the formatDocuments() function:

function formatDocuments({ documents, documentPrompt, documentSeparator, config }) {
  // Check if there are more than 6 documents
  if (documents.length > 6) {
    // Replace the 6th document with a message informing the user
    documents[5] = "There are more items available. Please make a more specific request to see additional items.";
  }
  // Proceed with the original formatting logic
  return documents.map(doc => formatSingleDocument(doc, documentPrompt)).join(documentSeparator);
}

In this modification, we're checking if there are more than 6 documents. If there are, we replace the 6th document with a message informing the user that there are more items available and that they should make a more specific request to see the additional items. The rest of the documents are formatted as before.

As for the createStuffDocumentsChain() function, it seems like no further modifications are necessary, as it already handles the output from formatDocuments() appropriately. The key change is in how formatDocuments() processes and returns the documents or the message about more specific requests.

Please note, the actual implementation details might vary based on the rest of your codebase and how formatDocuments() is implemented. The provided modifications are conceptual and should be adapted to fit the actual structure and logic of your application.

Regarding your consideration of not chaining the retriever and making a manual call aside, it's hard to provide a concrete suggestion without more details about your implementation. However, the .stream() function typically expects a stream of data as input, so you would need to ensure that the embeddings are provided in a format that .stream() can handle.

I hope this helps! If you have further questions or need more clarification, feel free to ask.

Sources

• langchain/src/chains/combine_documents/stuff.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.