How to get metadata from PrismaVectorStore when using it asRetriever and as a tool for an agent?

[max-sym]

I have the following code:

/**
 * Get a vector store retriever.
 */
export const getVSRetriever = (
  vectorStore: ReturnType<typeof getVectorStore>,
  name: string,
  description: string
) =>
  createRetrieverTool(vectorStore.asRetriever(), {
    name,
    description
  })


  const retrieverToolEvents = getVSRetriever(
    vectorStoreEvents,
    "events",
    "Similarity search tool for events."
  )

const executor = await initializeAgentExecutorWithOptions(tools, model, {
    agentType: "openai-functions",
    verbose: true,
    agentArgs: { prefix },
    memory
  })

  await executor.stream({ input: lastMessage }).catch(console.error)

The LLM agent doesn't get the metadata when using the events tool. The LLM agent needs metadata to do other queries using other tools.
How to fix this?

[dosubot[bot]]

🤖

Hello,

To allow the LLM agent to access the metadata when using the events tool, you need to modify the plan method in the XMLAgent class to include metadata in the _inputs object. This will ensure that metadata is passed to the call method of the llmChain object, which in turn will be accessible to the LLM agent.

Here's how you can modify the plan method:

async plan(
  steps: AgentStep[],
  inputs: ChainValues,
  callbackManager?: CallbackManager
): Promise<AgentAction | AgentFinish> {
  let log = "";
  for (const { action, observation } of steps) {
    log += `<tool>${action.tool}</tool><tool_input>${action.toolInput}</tool_input><observation>${observation}</observation>`;
  }
  let tools = "";
  for (const tool of this.tools) {
    tools += `${tool.name}: ${tool.description}\n`;
  }
  const _inputs = {
    intermediate_steps: log,
    tools,
    question: inputs.input,
    metadata: inputs.metadata, // Include metadata in the inputs
    stop: ["</tool_input>", "</final_answer>"],
  };
  const response = await this.llmChain.call(_inputs, callbackManager);
  return parseOutput(response[this.llmChain.outputKey]);
}

Please note that this assumes metadata is available in the inputs object. If it's not, you'll need to adjust your code accordingly to ensure it is.

You can find the plan method in the XMLAgent class in the langchain/src/agents/xml/index.ts file.

As for how metadata is currently being handled in the getVSRetriever function, it is being handled as a property of each item in the response from the Tavily Search API. It is then passed as an argument to the Document constructor when creating new Document instances.

Here is the relevant code:

const docs: Document[] = json.results.map((result: any) => {
  const pageContent = this.includeRawContent
    ? result.raw_content
    : result.content;
  const metadata = {
    title: result.title,
    source: result.url,
    ...Object.fromEntries(
      Object.entries(result).filter(
        ([k]) => !["content", "title", "url", "raw_content"].includes(k)
      )
    ),
    images: json.images,
  };
  return new Document({ pageContent, metadata });
});

You can find this code in the langchain/src/retrievers/metal.ts and langchain/src/retrievers/tavily_search_api.ts files.

I hope this helps! Let me know if you have any other questions.

Sources

• langchain/src/agents/xml/index.ts
• langchain/src/retrievers/metal.ts
• langchain/src/retrievers/tavily_search_api.ts

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.