StateGraph doesn't work on LangGraph server

[khteh]

Checked other resources

• This is a bug, not a usage question. For questions, please use GitHub Discussions.
• I added a clear and detailed title that summarizes the issue.
• I read what a minimal reproducible example is (https://stackoverflow.com/help/minimal-reproducible-example).
• I included a self-contained, minimal example that demonstrates the issue INCLUDING all the relevant imports. The code run AS IS to reproduce the issue.

Example Code

def BuildCheckpointedGraph(config: RunnableConfig) -> StateGraph:
    # Compile application and test
    print(f"\n=== {BuildCheckpointedGraph.__name__} ===")
    graph_builder = StateGraph(MessagesState)
    graph_builder.add_node("query_or_respond", query_or_respond)
    graph_builder.add_node("tools", ToolNode([retrieve])) # Execute the retrieval.
    graph_builder.add_node("generate", generate)
    graph_builder.set_entry_point("query_or_respond")
    graph_builder.add_conditional_edges(
        "query_or_respond",
        tools_condition,
        {END: END, "tools": "tools"},
    )
    graph_builder.add_edge("tools", "generate")
    graph_builder.add_edge("generate", END)
    return graph_builder.compile(store=InMemoryStore(), checkpointer=MemorySaver(), name="Checkedpoint StateGraph")

async def CheckpointedGraph():
    config = RunnableConfig(run_name="CheckpointGraph_RAG_Conversation")
    checkpoint_graph = BuildCheckpointedGraph(config) # config input parameter is required by langgraph.json to define the graph
    graph = checkpoint_graph.get_graph().draw_mermaid_png()
    # Save the PNG data to a file
    with open("/tmp/checkpoint_graph.png", "wb") as f:
        f.write(graph)
    img = Image.open("/tmp/checkpoint_graph.png")
    img.show()
    await Chat(checkpoint_graph, datetime.now(), ["What is Task Decomposition?", "Can you look up some common ways of doing it?"])

async def main():
    #await SimpleGraph()
    await CheckpointedGraph()

if __name__ == "__main__":
    docs = LoadDocuments("https://lilianweng.github.io/posts/2023-06-23-agent/")
    subdocs = SplitDocuments(docs)
    IndexChunks(subdocs)
    asyncio.run(main())


{
    "dependencies": [
        "langchain-text-splitters",
        "langchain-community",
        "langgraph",
        "langchain-openai",
        "python-dotenv",
        "langchain-google-vertexai",
        "beautifulsoup4",
        "langchain-google-genai",
        "langchain-core",
        "./src/rag_agent"
    ],
    "graphs": {
        "rag_agent": "./src/rag_agent/RAG_Conversation.py:BuildCheckpointedGraph"
    },
    "env": "./.env",
    "python_version": "3.12",
    "pip_config_file": "Pipfile"
}

Error Message and Stack Trace (if applicable)

Description

This is what I see on the BASH shell when I run python <filename>.py:

================================ Human Message =================================

What is Task Decomposition?
state: {'messages': [HumanMessage(content='What is Task Decomposition?', additional_kwargs={}, response_metadata={}, id='eb319019-b3dc-4a3f-a242-076a79cdd26b')]}
================================== Ai Message ==================================
Tool Calls:
  retrieve (764fae9b-480e-4434-be6f-88ddaa396a5a)
 Call ID: 764fae9b-480e-4434-be6f-88ddaa396a5a
  Args:
    query: Task Decomposition
================================= Tool Message =================================
Name: retrieve

Source: {'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/'}
Content: Fig. 1. Overview of a LLM-powered autonomous agent system.
Component One: Planning#
A complicated task usually involves many steps. An agent needs to know what they are and plan ahead.
Task Decomposition#
Chain of thought (CoT; Wei et al. 2022) has become a standard prompting technique for enhancing model performance on complex tasks. The model is instructed to “think step by step” to utilize more test-time computation to decompose hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the model’s thinking process.

Source: {'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/'}
Content: Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote.
Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.
================================== Ai Message ==================================

Task decomposition is a method of breaking down a complex task into smaller, more manageable steps. This can be achieved through prompting the model to "think step by step" or by providing task-specific instructions. The goal is to transform large tasks into multiple, simpler tasks, which can help to improve the model's performance and provide insights into its thinking process.
================================ Human Message =================================

Can you look up some common ways of doing it?
state: {'messages': [HumanMessage(content='What is Task Decomposition?', additional_kwargs={}, response_metadata={}, id='eb319019-b3dc-4a3f-a242-076a79cdd26b'), AIMessage(content='', additional_kwargs={'function_call': {'name': 'retrieve', 'arguments': '{"query": "Task Decomposition"}'}}, response_metadata={'is_blocked': False, 'safety_ratings': [], 'usage_metadata': {'prompt_token_count': 15, 'candidates_token_count': 4, 'total_token_count': 19, 'prompt_tokens_details': [{'modality': 1, 'token_count': 15}], 'candidates_tokens_details': [{'modality': 1, 'token_count': 4}], 'cached_content_token_count': 0, 'cache_tokens_details': []}, 'finish_reason': 'STOP', 'avg_logprobs': -0.24030499160289764}, id='run-404e917a-067b-4a7f-8b7e-148520a0da3a-0', tool_calls=[{'name': 'retrieve', 'args': {'query': 'Task Decomposition'}, 'id': '764fae9b-480e-4434-be6f-88ddaa396a5a', 'type': 'tool_call'}], usage_metadata={'input_tokens': 15, 'output_tokens': 4, 'total_tokens': 19}), ToolMessage(content='Source: {\'source\': \'https://lilianweng.github.io/posts/2023-06-23-agent/\'}\nContent: Fig. 1. Overview of a LLM-powered autonomous agent system.\nComponent One: Planning#\nA complicated task usually involves many steps. An agent needs to know what they are and plan ahead.\nTask Decomposition#\nChain of thought (CoT; Wei et al. 2022) has become a standard prompting technique for enhancing model performance on complex tasks. The model is instructed to “think step by step” to utilize more test-time computation to decompose hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the model’s thinking process.\n\nSource: {\'source\': \'https://lilianweng.github.io/posts/2023-06-23-agent/\'}\nContent: Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote.\nTask decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.', name='retrieve', id='a300a497-50b7-4282-b794-e32fd80d2d18', tool_call_id='764fae9b-480e-4434-be6f-88ddaa396a5a', artifact=[{'id': '672ffa61-0ecf-4082-8845-1c9478d02e4a', 'metadata': {'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/'}, 'page_content': 'Fig. 1. Overview of a LLM-powered autonomous agent system.\nComponent One: Planning#\nA complicated task usually involves many steps. An agent needs to know what they are and plan ahead.\nTask Decomposition#\nChain of thought (CoT; Wei et al. 2022) has become a standard prompting technique for enhancing model performance on complex tasks. The model is instructed to “think step by step” to utilize more test-time computation to decompose hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the model’s thinking process.', 'type': 'Document'}, {'id': 'fe071ebd-1724-4860-936d-b9228c40941a', 'metadata': {'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/'}, 'page_content': 'Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote.\nTask decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.', 'type': 'Document'}]), AIMessage(content='Task decomposition is a method of breaking down a complex task into smaller, more manageable steps. This can be achieved through prompting the model to "think step by step" or by providing task-specific instructions. The goal is to transform large tasks into multiple, simpler tasks, which can help to improve the model\'s performance and provide insights into its thinking process.\n', additional_kwargs={}, response_metadata={'is_blocked': False, 'safety_ratings': [], 'usage_metadata': {'prompt_token_count': 429, 'candidates_token_count': 73, 'total_token_count': 502, 'prompt_tokens_details': [{'modality': 1, 'token_count': 429}], 'candidates_tokens_details': [{'modality': 1, 'token_count': 73}], 'cached_content_token_count': 0, 'cache_tokens_details': []}, 'finish_reason': 'STOP', 'avg_logprobs': -0.16595640574416068}, id='run-de5c3571-d01b-4efb-b4aa-0402358d0239-0', usage_metadata={'input_tokens': 429, 'output_tokens': 73, 'total_tokens': 502}), HumanMessage(content='Can you look up some common ways of doing it?', additional_kwargs={}, response_metadata={}, id='a413a751-3e12-4cae-ac36-7a642f9664bb')]}
================================== Ai Message ==================================
Tool Calls:
  retrieve (03d931e0-c674-4ef0-a8f1-f5da52eadf30)
 Call ID: 03d931e0-c674-4ef0-a8f1-f5da52eadf30
  Args:
    query: common methods for task decomposition
================================= Tool Message =================================
Name: retrieve

Source: {'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/'}
Content: Fig. 1. Overview of a LLM-powered autonomous agent system.
Component One: Planning#
A complicated task usually involves many steps. An agent needs to know what they are and plan ahead.
Task Decomposition#
Chain of thought (CoT; Wei et al. 2022) has become a standard prompting technique for enhancing model performance on complex tasks. The model is instructed to “think step by step” to utilize more test-time computation to decompose hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the model’s thinking process.

Source: {'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/'}
Content: Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote.
Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs.
================================== Ai Message ==================================

Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." for writing a novel, or (3) with human inputs. Tree of Thoughts extends CoT by exploring multiple reasoning possibilities at each step and generating multiple thoughts per step, creating a tree structure.

LangGraph server:

LangSmith trace:

The follow-up question doesn't work.

System Info

System Information
------------------
> OS:  Linux
> OS Version:  #19-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 12 21:43:43 UTC 2025
> Python Version:  3.12.7 (main, Feb  4 2025, 14:46:03) [GCC 14.2.0]

Package Information
-------------------
> langchain_core: 0.3.43
> langchain: 0.3.20
> langchain_community: 0.3.19
> langsmith: 0.3.13
> langchain_google_genai: 2.0.11
> langchain_google_vertexai: 2.0.14
> langchain_openai: 0.3.8
> langchain_text_splitters: 0.3.6
> langgraph_api: 0.0.28
> langgraph_cli: 0.1.75
> langgraph_license: Installed. No version info available.
> langgraph_sdk: 0.1.55
> langgraph_storage: Installed. No version info available.

Optional packages not installed
-------------------------------
> langserve

Other Dependencies
------------------
> aiohttp<4.0.0,>=3.8.3: Installed. No version info available.
> anthropic[vertexai]: Installed. No version info available.
> async-timeout<5.0.0,>=4.0.0;: Installed. No version info available.
> click: 8.1.7
> cryptography: 43.0.3
> dataclasses-json<0.7,>=0.5.7: Installed. No version info available.
> filetype: 1.2.0
> google-ai-generativelanguage: 0.6.16
> google-cloud-aiplatform: 1.83.0
> google-cloud-storage: 2.19.0
> httpx: 0.28.1
> httpx-sse: 0.4.0
> httpx-sse<1.0.0,>=0.4.0: Installed. No version info available.
> jsonpatch<2.0,>=1.33: Installed. No version info available.
> jsonschema-rs: 0.20.0
> langchain-anthropic;: Installed. No version info available.
> langchain-aws;: Installed. No version info available.
> langchain-cohere;: Installed. No version info available.
> langchain-community;: Installed. No version info available.
> langchain-core<1.0.0,>=0.3.34: Installed. No version info available.
> langchain-core<1.0.0,>=0.3.41: Installed. No version info available.
> langchain-core<1.0.0,>=0.3.42: Installed. No version info available.
> langchain-deepseek;: Installed. No version info available.
> langchain-fireworks;: Installed. No version info available.
> langchain-google-genai;: Installed. No version info available.
> langchain-google-vertexai;: Installed. No version info available.
> langchain-groq;: Installed. No version info available.
> langchain-huggingface;: Installed. No version info available.
> langchain-mistralai: Installed. No version info available.
> langchain-mistralai;: Installed. No version info available.
> langchain-ollama;: Installed. No version info available.
> langchain-openai;: Installed. No version info available.
> langchain-text-splitters<1.0.0,>=0.3.6: Installed. No version info available.
> langchain-together;: Installed. No version info available.
> langchain-xai;: Installed. No version info available.
> langchain<1.0.0,>=0.3.20: Installed. No version info available.
> langgraph: 0.3.5
> langgraph-checkpoint: 2.0.18
> langsmith-pyo3: Installed. No version info available.
> langsmith<0.4,>=0.1.125: Installed. No version info available.
> langsmith<0.4,>=0.1.17: Installed. No version info available.
> numpy<3,>=1.26.2: Installed. No version info available.
> openai<2.0.0,>=1.58.1: Installed. No version info available.
> orjson: 3.10.15
> packaging: 24.1
> packaging<25,>=23.2: Installed. No version info available.
> pydantic: 2.10.6
> pydantic-settings<3.0.0,>=2.4.0: Installed. No version info available.
> pydantic<3.0.0,>=2.5.2;: Installed. No version info available.
> pydantic<3.0.0,>=2.7.4: Installed. No version info available.
> pydantic<3.0.0,>=2.7.4;: Installed. No version info available.
> pyjwt: 2.10.1
> pytest: 8.3.2
> python-dotenv: 1.0.1
> PyYAML>=5.3: Installed. No version info available.
> requests: 2.32.3
> requests-toolbelt: 1.0.0
> requests<3,>=2: Installed. No version info available.
> rich: 13.9.4
> SQLAlchemy<3,>=1.4: Installed. No version info available.
> sse-starlette: 2.1.3
> starlette: 0.46.1
> structlog: 25.1.0
> tenacity: 9.0.0
> tenacity!=8.4.0,<10,>=8.1.0: Installed. No version info available.
> tenacity!=8.4.0,<10.0.0,>=8.1.0: Installed. No version info available.
> tiktoken<1,>=0.7: Installed. No version info available.
> typing-extensions>=4.7: Installed. No version info available.
> uvicorn: 0.34.0
> watchfiles: 1.0.4
> zstandard: 0.23.0

[hinthornw]

@khteh it looks like the graph is running as intended but your vector store is incorrectly initialized (just returning --).

The server doesn't run the code block in __main__, as that's not how servers typically work.

[khteh]

I am well aware of this. I just showed the difference that running it directly from the BASH shell with python <file>.py works perfectly. langgraph dev doesn't work as expected.

[hinthornw]

Going to convert this to a discussion since it isn't a langgraph bug

[khteh]

Why is this not a bug since langgraph dev doesn't work as shown?