How can we use langgraph with open source llms?

[NomanSaleem4]

Searched docs, couldn't find anything

[hinthornw]

Hiya! I think we could definitely create more examples you could use off the shelf. In general, LangGraph doesn't have the concept of an "LLM" so the trite answer is "same way you'd uses open source LLMs in any other software", but I'm guessing that's not very helpful to hear.

We DO have a couple of examples with ollama:
The RAG examples in the docs use ollama (e.g.: https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_crag_local/?h=#setup)
Open Deep Research has some examples of using ollama within it: https://github.com/langchain-ai/open_deep_research

And the prebuilts support any ChatModel that supports tool calling.

[NomanSaleem4]

Exactly that's not helpful. Also not interested in getting the inference from the ollama.