open source models with function calling that are better than 70b but less in size Ollama models will be preferred

[ehsanahmadkhan525]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

.

Description

I am looking for open source models with function calling that are better than 70b but less in size as I need better speed. ollama models will be preferred

System Info

.

[dosubot[bot]]

Hey @ehsanahmadkhan525! I'm here to help you with any questions or issues you have. I'm a bot designed to assist with bug fixes, answer questions, and guide you on becoming a contributor. Let me know how I can assist you!

LangChain supports several open-source language models with function calling capabilities that are smaller than 70 billion parameters. Here are some of the supported models:

• ChatAnthropic
• ChatMistralAI
• ChatFireworks
• AzureChatOpenAI
• ChatOpenAI
• ChatTogether
• ChatVertexAI
• ChatGoogleGenerativeAI
• ChatGroq
• ChatCohere
• ChatBedrock
• ChatHuggingFace
• ChatNVIDIA
• ChatOllama
• ChatLlamaCpp
• ChatAI21
• ChatUpstage

All these models have the tool_calling capability set to true ^[1].

For integrating Ollama models specifically, you can follow the detailed guidelines provided in the LangChain documentation. Here are the key steps:

1. Installation:

   pip install -U langchain-ollama

2. Setup:

   • Download and run the Ollama server locally from here.
   • Ollama will start as a background service automatically. If this is disabled, run:

     # export OLLAMA_HOST=127.0.0.1 # environment variable to set ollama host
     # export OLLAMA_PORT=11434 # environment variable to set the ollama port
     ollama serve

   • Download a model from the Ollama model library using:

     ollama pull llama3.1

3. Usage:

   from langchain_core.prompts import ChatPromptTemplate
   from langchain_ollama.llms import OllamaLLM
   
   template = """Question: {question}
   
   Answer: Let's think step by step."""
   
   prompt = ChatPromptTemplate.from_template(template)
   
   model = OllamaLLM(model="llama3.1")
   
   chain = prompt | model
   
   chain.invoke({"question": "What is LangChain?"})

For more details, you can refer to the LangChain documentation on integrating Ollama models ^[2][3][4][5].

To continue talking to Dosu, mention @dosu.