Sharing LLM Instances Between Nodes in LangGraph

[BharathKumarRavichandran]

Hello,

I'm currently exploring LangGraph and have a couple of questions. Suppose I have a graph with two nodes (agents) that perform different operations, but I'm using the same LLM model for both agents. Should I use a single LLM instance for both agents or create separate instances for each? Does sharing the same LLM instance between the two nodes potentially cause interference between them?

[hinthornw]

In the vast majority of cases it's fine to share between nodes. I do that fairly often.

[BharathKumarRavichandran]

Got it, thanks for the clarification! I'm curious to know what resources are shared when using the same LLM instance across multiple nodes. Specifically, does the second agent have access to the input fed to the first agent, or are they completely isolated?

I'm also interested in understanding the use cases where it's not recommended to share LLM instances between nodes.

As a side note, I'm running the models on my local machine, which is different from the cloud-deployed models used in most of the examples I've come across. In the cloud, initializing the same model twice might not be a concern, as the cost is typically based on input tokens and API calls. However, on my local machine, initializing the same model twice consumes almost twice as much memory as initializing it once. In terms of performance, I'm wondering if there are any trade-offs to consider when deciding whether to share or not share LLM instances.

[hinthornw]

Oh this is helpful context! Will try to write up a more specific response today - thank you for your patience!

LLMs typically would be stateless, so you wouldn't be sharing context, just compute resources / connection resources.

[BharathKumarRavichandran]

Understood, thanks for the update!

Looking forward to your more detailed response, and thanks again for your time!

[Bayleaf2]

Would love to know the answer to this too. I did some limited testing and it seemed that using separate instances was faster when there were several operations going on concurrently.