How to run tools with Ollama?

[gbarton]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import dotenv from "dotenv";
import log from './lib/log/Logger';

dotenv.config();

import { ChatOllama } from "@langchain/community/chat_models/ollama";
import { OllamaFunctions } from "langchain/experimental/chat_models/ollama_functions";
import { OllamaEmbeddings } from "@langchain/community/embeddings/ollama";
import { createStuffDocumentsChain } from "langchain/chains/combine_documents";
import { createRetrievalChain } from "langchain/chains/retrieval";
import { AgentExecutor } from "langchain/agents";
import { createRetrieverTool } from "langchain/tools/retriever";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { pull } from "langchain/hub";
import { createOpenAIFunctionsAgent } from "langchain/agents";
// import { StringOutputParser } from "@langchain/core/output_parsers";
import { CheerioWebBaseLoader } from "langchain/document_loaders/web/cheerio";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { MemoryVectorStore } from "langchain/vectorstores/memory";

const baseUrl = process.env.ollama_api || "";
const model = "mist"; 

async function run() {
  const loader = new CheerioWebBaseLoader(
    "https://docs.smith.langchain.com/user_guide"
  );
  const rawDocs = await loader.load();
  log.info(rawDocs.length, 'loaded website');
  
  const splitter = new RecursiveCharacterTextSplitter({
    chunkSize: 1000,
    chunkOverlap: 200,
  });
  const docs = await splitter.splitDocuments(rawDocs);
  log.info(docs.length, 'Split into docs');
  
  const vectorstore = await MemoryVectorStore.fromDocuments(
    docs,
    new OllamaEmbeddings({ baseUrl, model }
    )
  );
  const retriever = vectorstore.asRetriever();
  log.info('loaded retriever');
  
  const retrieverResult = await retriever.getRelevantDocuments(
    "how to upload a dataset"
  );
  log.info(retrieverResult[0], "Retriever test 'how to upload a dataset'");

  const retrieverTool = createRetrieverTool(retriever, {
    name: "langsmith_search",
    description:
      "Search for information about LangSmith. For any questions about LangSmith, you must use this tool!",
  });

  // tool list created
  const tools = [retrieverTool];
  log.info('tools created');

  const functionModel = new OllamaFunctions({ temperature: 0.1, model, baseUrl });
  const prompt = await pull<ChatPromptTemplate>(
    "hwchase17/openai-functions-agent"
  );
  log.info(prompt.toJSON(), "prompt");

  const agent = await createOpenAIFunctionsAgent({
    llm: functionModel,
    tools,
    prompt,
  });

  const agentExecutor = new AgentExecutor({
    agent,
    tools,
  });

  const input = "how can langsmith help with testing?";
  const result2 = await agentExecutor.invoke({
    input,
  });
  
  log.info(result2, input);

}

run();

Description

Am trying to merge together the agent quickstart: https://js.langchain.com/docs/modules/agents/quick_start with Ollama Functions https://js.langchain.com/docs/integrations/chat/ollama_functions to be able to run tools with Ollama. No luck so far having read #3942 I thought it be possible to make it work maybe?

Thanks in advance!

OUTPUT BELOW:

[04:11:29.780] INFO (394149): 1
[04:11:29.793] INFO (394149): 47
[04:15:28.704] INFO (394149): loaded retriever
[04:15:29.329] INFO (394149): Retriever test 'how to upload a dataset'
    pageContent: "to the most interesting runs and highlight edge cases that are causing problematic responses. LangSmith allows you to attach feedback scores to logged traces (oftentimes, this is hooked up to a feedback button in your app), then filter on traces that have a specific feedback tag and score. A common workflow is to filter on traces that receive a poor user feedback score, then drill down into problematic points using the detailed trace view.Annotating Traces​LangSmith also supports sending runs to annotation queues, which allow annotators to closely inspect interesting traces and annotate them with respect to different criteria. Annotators can be PMs, engineers, or even subject matter experts. This allows users to catch regressions across important evaluation criteria.Adding Runs to a Dataset​As your application progresses through the beta testing phase, it's essential to continue collecting data to refine and improve its performance. LangSmith enables you to add runs as examples to"
    metadata: {
      "source": "https://docs.smith.langchain.com/user_guide",
      "loc": {
        "lines": {
          "from": 11,
          "to": 11
        }
      }
    }
[04:15:29.331] INFO (394149): tools created
[04:15:30.983] INFO (394149): prompt
    lc: 1
    type: "constructor"
    id: [
      "langchain_core",
      "prompts",
      "chat",
      "ChatPromptTemplate"
    ]
    kwargs: {
      "messages": [
        {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain_core",
            "prompts",
            "chat",
            "SystemMessagePromptTemplate"
          ],
          "kwargs": {
            "prompt": {
              "lc": 1,
              "type": "constructor",
              "id": [
                "langchain_core",
                "prompts",
                "prompt",
                "PromptTemplate"
              ],
              "kwargs": {
                "template": "You are a helpful assistant",
                "input_variables": [],
                "template_format": "f-string"
              }
            }
          }
        },
        {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain_core",
            "prompts",
            "chat",
            "MessagesPlaceholder"
          ],
          "kwargs": {
            "optional": true,
            "variable_name": "chat_history"
          }
        },
        {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain_core",
            "prompts",
            "chat",
            "HumanMessagePromptTemplate"
          ],
          "kwargs": {
            "prompt": {
              "lc": 1,
              "type": "constructor",
              "id": [
                "langchain_core",
                "prompts",
                "prompt",
                "PromptTemplate"
              ],
              "kwargs": {
                "template": "{input}",
                "input_variables": [
                  "input"
                ],
                "template_format": "f-string"
              }
            }
          }
        },
        {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain_core",
            "prompts",
            "chat",
            "MessagesPlaceholder"
          ],
          "kwargs": {
            "optional": false,
            "variable_name": "agent_scratchpad"
          }
        }
      ],
      "input_variables": [
        "agent_scratchpad",
        "chat_history",
        "input"
      ]
    }
/workspaces/mila/node_modules/@langchain/community/dist/chat_models/ollama.cjs:436
        return messages.map((message) => {
                        ^
Error: Unsupported message type for Ollama: function
    at /workspaces/mila/node_modules/@langchain/community/dist/chat_models/ollama.cjs:448:23
    at Array.map (<anonymous>)
    at ChatOllama._convertMessagesToOllamaMessages (/workspaces/mila/node_modules/@langchain/community/dist/chat_models/ollama.cjs:436:25)
    at /workspaces/mila/node_modules/@langchain/community/dist/chat_models/ollama.cjs:394:32
    at onFailedAttempt (/workspaces/mila/node_modules/@langchain/core/dist/utils/async_caller.cjs:92:66)
    at RetryOperation._fn (/workspaces/mila/node_modules/p-retry/index.js:50:18)
    at Timeout._onTimeout (/workspaces/mila/node_modules/retry/lib/retry_operation.js:85:10)
    at listOnTimeout (node:internal/timers:573:17)
    at processTimers (node:internal/timers:514:7) {
  attemptNumber: 7,
  retriesLeft: 0
}

System Info

node ➜ /workspaces/mila (master) $ npm info langchain
[email protected] | MIT | deps: 18 | versions: 259

node ➜ /workspaces/mila (master) $ node --version
v20.10.0

[dosubot[bot]]

Hey there, @gbarton! Great to see you back with another intriguing challenge for us.

Thank you for providing the detailed information. The error message "Unsupported message type for Ollama: function" suggests that the Ollama Functions model is receiving a message type that it doesn't support.

From the error stack trace, it seems like the error is thrown when the ChatOllama class tries to convert the messages to Ollama messages. This conversion process involves mapping over the messages and converting each one to an Ollama message. If it encounters a message type that it doesn't recognize, it throws the "Unsupported message type for Ollama: function" error.

In your code, you're creating an OllamaFunctions instance and then trying to use it to create an OpenAIFunctionsAgent. The OpenAIFunctionsAgent is then used to create an AgentExecutor, which is invoked with a HumanMessage. This seems to be the correct way to use these classes according to the LangChain JS documentation.

However, the error message suggests that somewhere in this process, a function is being passed as a message, which is not supported. It's possible that this is happening in the createOpenAIFunctionsAgent function or the AgentExecutor class.

Unfortunately, without more information about these classes and their methods, it's hard to say exactly what's causing the error. It might be helpful to look at the source code of these classes to see if you can identify where the function is being passed as a message.

One possible solution could be to ensure that the functionModel passed to createOpenAIFunctionsAgent is an instance of OllamaFunctions and not a function. If functionModel is a function, it could be causing the error when it's passed as a message.

Another possible solution could be to check the invoke method of the AgentExecutor class. If this method is expecting a HumanMessage but is receiving a function, it could be causing the error.

I hope this helps! If you have any other questions or need further clarification, please let me know.

Sources

• Error in handler Handler, handleLLMStart: Error: Got unsupported message type: [object Object]
• examples/src/models/chat/ollama_functions/function_calling.ts

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[divyanshverma]

@gbarton have you been able to get this to work ?

[robegamesios]

Same experience, I’m using OllamaEmbeddings. If I pass the Ollama model in createOpenAIFunctionsAgent, it does not work. But if I use the OpenAI model, it works. I tried using the OpenAI model constructor using Ollama baseUrl and apiKey. I read somewhere that I can use any api key, but unless I use a real openAI api key, I get an invalid apu key error. So I’m guessing it still accesses OpenAi endpoint instead of using ollama? any tips? Thanks