🧵 Memory Leak Report: .stream() and .invoke() Both Cause Memory Growth When Input Is Empty

[Venkatesh-Zebra]

✅ Context

We observe memory growth when using both .stream() and .invoke() with a Runnable composed of a PromptTemplate and an LLM (e.g., OpenAI) — only when the conversation_history input is empty ("") or None.

---

🔍 Reproduction Snippet

stream = (prompt | self.llm.get("transform_query")).stream({
    "conversation_history": history_text,  # empty string or None
    "current_question": original_q
})
# Memory leak starts immediately upon getting the stream object, before even iterating over it.
content = "".join(getattr(chunk, "content", str(chunk)) for chunk in stream)

Similarly, with .invoke():

response = (prompt | self.llm.get("transform_query")).invoke({
    "conversation_history": None,
    "current_question": "moscow mule"
})

We see a smaller but still noticeable memory increase.

---

📌 Observations

Line	Mem Usage	Increment	Description
233	366.4 MiB	+0.9 MiB	Memory grows immediately upon stream creation (pre-iter)
238	367.3 MiB	+0.0 MiB	Memory remains stable while reading chunks

• Memory leak triggers right at the point .stream() is called — even before iteration starts.

• Leak happens only when conversation_history is empty or None.

• Leak occurs with both .stream() and .invoke()

• Memory growth accumulates with repeated calls, requiring process restart.

---

❓ Questions for LangChain / LangGraph Team

1. Why does calling .stream() with an empty input cause immediate memory growth even before iteration?

2. Is this a known issue related to input processing, caching, or internal buffers?

3. Could there be internal objects or retry logic that are not cleaned up properly for empty inputs?

4. What workarounds or fixes do you recommend to avoid this leak?

Looking forward to your insights!

---

Description

Using LangChain’s ChatVertexAI with the invoke() and stream() methods causes memory growth on each call. The leak happens even after deleting the response and forcing garbage collection. This occurs when the input dictionary is empty ({}), particularly when the conversation_history parameter is empty or omitted.

Minimal Reproducible Example

To reproduce the issue, install the following packages:

pip install langchain langchain-google-vertexai vertexai memory-profiler

import gc
from memory_profiler import profile
import vertexai
from langchain_google_vertexai.chat_models import ChatVertexAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

# Initialize Vertex AI
vertexai.init(project="your-project-id", location="us-central1")

# Setup LangChain prompt and model
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant."),
    ("human", "Tell me a joke about elephants.")
])

chat_model = ChatVertexAI(
    model="gemini-2.5-flash",
    temperature=0.7,
    max_output_tokens=256
)

chain = prompt | chat_model | StrOutputParser()

@profile
def run_invoke():
    print("\n[invoke()] Response:")
    result = chain.invoke({})
    print(result)
    del result
    gc.collect()

@profile
def run_stream():
    print("\n[stream()] Response:")
    for chunk in chain.stream({}):
        print(chunk, end="")

if __name__ == "__main__":
    run_invoke()
    print("\n" + "-" * 50)
    run_stream()

Observed Output from memory_profiler

[invoke()] Response:
What time is it when an elephant sits on a fence?

Time to fix the fence!

Filename: test.py

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
    27    226.6 MiB    226.6 MiB           1   @profile
    28                                         def run_invoke():
    29    226.6 MiB      0.0 MiB           1       print("\n[invoke()] Response:")
    30    235.8 MiB      9.2 MiB           1       result = chain.invoke({})
    31    235.8 MiB      0.0 MiB           1       print(result)
    32    235.8 MiB      0.0 MiB           1       del result
    33    235.8 MiB      0.0 MiB           1       gc.collect()


--------------------------------------------------

[stream()] Response:
What time is it when an elephant sits on a fence?

Time to fix the fence!

Filename: test.py

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
    36    235.8 MiB    235.8 MiB           1   @profile
    37                                         def run_stream():
    38    235.8 MiB      0.0 MiB           1       print("\n[stream()] Response:")
    39    235.9 MiB      0.1 MiB           4       for chunk in chain.stream({}):
    40    235.9 MiB      0.0 MiB           3           print(chunk, end="")

Notes

• Memory increases ~9 MiB immediately after invoke() call and does not drop.
• stream() method shows a smaller but persistent memory increase.(Note: if we run stream method first then it is taking up ~9 MiB)
• Deleting variables and forcing garbage collection (gc.collect()) does not reclaim memory.
• The leak appears linked to empty input or missing conversation_history.

Environment

Package	Version
Python	3.11.4
langchain	0.3.27
langchain-google-vertexai	2.0.27
langchain-core	0.3.72
vertexai	1.43.0
memory-profiler	0.61.0

Operating System:

Linux (Ubuntu 22.04 LTS or equivalent)

Request

Could you please investigate this potential memory leak in ChatVertexAI’s invoke() and stream()? It’s blocking usage in long-running applications due to gradual memory bloat.

Thank you!

---