How to use memory function when launch LangGraph Server

[goreng2]

Hi, I want to make "chatbot with memory" API with LangGraph Server.

I'm using New LangGraph Project template (refer to document)

Then, I write src/agent/graph.py like the below

from langgraph.graph import StateGraph, START, END, MessagesState
from langchain_openai import ChatOpenAI
from langgraph.checkpoint.memory import MemorySaver

# I'm serving OpenAI-compatible API through vLLM
llm = ChatOpenAI(
    model="LGAI-EXAONE/EXAONE-3.5-32B-Instruct-AWQ",
    openai_api_key="EMPTY",
    openai_api_base="http://127.0.0.1:8000/v1",
    max_tokens=512,
    temperature=0
)

def chatbot(state: MessagesState):
    response = llm.invoke(state["messages"])
    return {"messages": response}


# Define a new graph
workflow = StateGraph(MessagesState)

# Add the node to the graph
workflow.add_node("chatbot", chatbot)

# Set the entrypoint
workflow.add_edge(START, "chatbot")
workflow.add_edge("chatbot", END)

# Add memory
memory = MemorySaver()

# Compile the workflow into an executable graph
graph = workflow.compile(checkpointer=memory)
graph.name = "New Graph"  # This defines the custom name in LangSmith

Launch LangGraph Server: $ langgraph dev.

I request with the below JSON body. (of course requested content: Hi, My Name is Bob. before)

{
    "assistant_id": "agent",
    "input": {
        "messages": [
            {
                "role": "human",
                // "content": "Hi, My Name is Bob."
                "content": "Remember my name?"
            }
        ]
    },
    "config": {
        "thread_id": "1"
    },
    "stream_mode": "updates"
}

But It's not working Memory function...
Because The below response doesn't remember first msg..

{
    "chatbot": {
        "messages": {
            "content": "I don't have the ability to remember specific names or personal information from previous interactions unless provided within the current context. If you'd like to share your name or any other details, feel free to do so, and I'll address you accordingly!",
            "additional_kwargs": {
                "refusal": null
            },
            "response_metadata": {
                "token_usage": {
                    "completion_tokens": 53,
                    "prompt_tokens": 23,
                    "total_tokens": 76,
                    "completion_tokens_details": null,
                    "prompt_tokens_details": null
                },
                "model_name": "LGAI-EXAONE/EXAONE-3.5-32B-Instruct-AWQ",
                "system_fingerprint": null,
                "finish_reason": "stop",
                "logprobs": null
            },
            "type": "ai",
            "name": null,
            "id": "run-31d9b3e9-5c1e-4eae-ad41-7cea6e69f20a-0",
            "example": false,
            "tool_calls": [],
            "invalid_tool_calls": [],
            "usage_metadata": {
                "input_tokens": 23,
                "output_tokens": 53,
                "total_tokens": 76,
                "input_token_details": {},
                "output_token_details": {}
            }
        }
    }
}

What's wrong..?

[hinthornw]

Hiya! So you sent two requests or just the one?

(Also FYI, the checkpointer is managed by LangGrah platform, so your MemorySaver is ignored)

[goreng2]

The first request: content: Hi, My Name is Bob.
second request: content: Remember my name?

So, Memory function is utilized automatically..?
How to specify thread_id?

[hinthornw]

Needs to be through the threads/runs endpoint if you're sending via the server

CURL http://localhost:2024/threads/{thread_id}/runs/wait
...

https://langchain-ai.github.io/langgraph/cloud/reference/api/api_ref.html#tag/thread-runs/POST/threads/%7Bthread_id%7D/runs/wait

[goreng2]

Well... I can't understand how work it is.

I will just combine with FastAPI