Doubts: Best Practices in LangGraph

[Yairama]

Hello everyone!

I know that there are no established "best practices" for such a new framework like LangGraph yet, but I would like to share some of the approaches I've been using to see if they're correct, and at the same time open up a discussion to define best practices we should follow.

Some of mi doubts:

1. Handling Pydantic BaseModel

When using Pydantic models, is it better to work with the pure class (BaseModel) in the message chain or to split the conversation into two separate threads: one for messages and another for artifacts?

For instance, I have the following Pydantic class to create SQL queries:

class QueryCreator(BaseModel):
    """
    Stores the creation of the query, including the need, the SQL query, the DataFrame name, and the observations.
    """

    need: Annotated[str, Field(..., description="The main need the query should solve.")]
    query: Annotated[str, Field(..., description="The SQL query, must be in valid MS SQL syntax.")]
    df_name: Annotated[str, Field(..., description="The snake_case name of the DataFrame that will store the query result.")]
    description: Annotated[str, Field(..., description="A detailed description of how the query works.")]

    def text(self) -> str:
        prompt_parts = []

        # Add the user's need
        if self.need:
            prompt_parts.append(f"The user's need is:\n{self.need}")

        # Add the SQL query
        if self.query:
            prompt_parts.append(f"The SQL query created is:\n{self.query}")

        # Add the DataFrame name
        if self.df_name:
            prompt_parts.append(f"Name of the DataFrame where the data will be stored:\n{self.df_name}")

        # Add the description
        if self.description:
            prompt_parts.append(f"Description of how the query works:\n{self.description}")

        # Join all parts into one explanatory text
        prompt = "\n\n".join(prompt_parts)

        return prompt if prompt_parts else "Not enough information provided."

In this case, I use the .text() method to generate a well-structured and more explanatory text that I pass into the messages chain, while the BaseModel is sent in the artifacts.

The state in my graph is structured like this:

class SQLState(TypedDict):
    messages: Annotated[list[AnyMessage], add_messages]
    artifacts: Annotated[list[dict], operator.add]

2. Node Flow in the Graph

Here’s an example of the flow between two nodes in my graph:

Node 1: Query Generation

def generate_query(state):
    # System prompt logic to instruct the LLM to create the query
    chain = self.llm.with_structured_output(QueryCreator)

    # Passing the 'messages' state, not the 'artifacts'
    artifact_response = chain.invoke({'messages': state['messages']})  # artifact_response is a QueryCreator model
    text_response = AIMessage(content=artifact_response.text())

    return {'messages': [text_response], 'artifacts': [artifact_response]}

Node 2: Query Execution

def call_query(state):
    last_artifact = state['artifacts'][-1]
    query_created = last_artifact.query

    query_response = db.run_no_throw(query_created, include_columns=True)
    
    # Generate a prompt for the LLM based on the query and its result
    order = AIMessage(content=f"Draw conclusions from this query: {query_created} and this result: {query_response}")

    # Passing the 'messages' state, not the 'artifacts'
    response = llm.invoke({'messages': state['messages'] + [order]})

    # Here I have a big question: should I use the entire response with all its metadata, or just send the AIMessage content without the extra metadata?
    return {'messages': [AIMessage(content=response.content())]}

3. Metadata in Messages

One of my main questions is whether it’s necessary to keep all possible metadata in the message chain. When function calls are made, a lot of metadata can be generated. How useful is this metadata for the model? Should we clean it up or continue sending all the information without filtering?

4. Plain Text vs. Structured Models

Is it better to send well-structured plain text (like the one generated by the .text() method), or is it preferable to work with BaseModel objects in their raw form during interactions? I’d like to know if LLMs take better advantage of structured models or if, on the contrary, it's more efficient to send well-structured plain text that is easier to interpret.

---

Conclusion

I would like this thread to serve as a space where we can all share ideas and approaches regarding what best practices we should follow when working with LangGraph. I’m particularly interested in avoiding overly theoretical or directionless debates.

Any comments or suggestions are very welcome!

[Yairama]

Is it a good idea to store the compiled graph in a global variable within a backend?

This question arose from the following thread:
Langchain Discussion #23630

While the discussion mentions that it is thread-safe, does this guarantee that it is safe to use as a global variable?

ChatGPT's perspective:

In web applications like FastAPI, it’s generally advisable to avoid the indiscriminate use of global variables. Global variables can introduce challenges when dealing with concurrency or scalability, especially in environments where multiple instances of your application may run in parallel.

Instead, solutions such as FastAPI dependencies offer greater flexibility and control over shared objects between routes. Dependencies also provide better management when it comes to concurrent access, making your application more scalable and maintainable.

However, for simpler use cases—like compiling a graph once and needing to access it across multiple routes—a carefully controlled global variable might be sufficient. This could especially work if:

• The graph only needs to be compiled once.
• There’s no need for frequent updates to the graph.
• You ensure that the variable is read-only after initialization, preventing race conditions or unintended mutations.

Conclusion:

While using a global variable can be "safe enough" for specific, low-complexity scenarios, it's always important to assess the trade-offs in terms of scalability, maintainability, and concurrency. For larger applications, utilizing more robust patterns like FastAPI dependencies is often the better choice.