Passing System Message and Examples in LangChain's MessagePlaceholder (Reference Examples)

[Rov7]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.pydantic_v1 import BaseModel, Field
from typing import Optional
from langchain_google_vertexai import ChatVertexAI
from langchain_core.globals import set_debug
set_debug(True)

llm = ChatVertexAI(
     model_name=MODEL_NAME #GEMINI_PRO
    ,max_output_tokens=N_TOKENS # Default 0
    ,temperature=0.0 
    ,safety_settings=safety_config 
    ,project=PROJECT_ID
    ,location=REGION
)
examples_match = [[SystemMessage(content="You are an expert in name matching. Your tasks are as follows: 1. Match the recipient's name with a list that contains recipient names and their associated id. 2. Return only one matching candidate from the provided list. 3. If no exact or approximate match is found, return 'None'. The list is: {'7643': 'Dorothy Lebanon','1021': 'Jorglein Clain','7212': 'Nirvan Moyer' }. "),
  HumanMessage(content='The recipient to match is: dorothy lebanon'),
  AIMessage(content='', additional_kwargs={'tool_calls': [{'id': '920d0237-23c9-46d8-af78-1bbae871e80a', 'type': 'function', 'function': {'name': 'MatchName', 'arguments': '{"recipient_name": "Dorothy Lebanon", "recipient_id": "7643"}'}}]}, tool_calls=[{'name': 'MatchName', 'args': {'recipient_name': 'Dorothy Lebanon', 'recipient_id': '7643'}, 'id': '920d0237-23c9-46d8-af78-1bbae871e80a', 'type': 'tool_call'}]),
  ToolMessage(content='You have correctly called this tool.', tool_call_id='920d0237-23c9-46d8-af78-1bbae871e80a'),
  AIMessage(content='ok')],]

match_name_prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You are an expert in name matching. "
            "Your tasks are as follows: "
            "1. Match the recipient's name with a list that contains recipient names and their associated id. "
            "2. Return only one matching candidate from the provided list. "
            "3. If no exact or approximate match is found, return 'None'. "
            "The list is: {destinatarios_dict}. ",
        ),
        MessagesPlaceholder("examples_match"),
        ("human", "The recipient to match is: {recipient}"),
    ]
)

class MatchName(BaseModel):
    """
    Contains the recipient information after matching it with a list containing the full names and associated ids. 
    
    Attributes:
        recipient_name (Optional[str]): The full name of the recipient that was matched from the list.   
        recipient_id (Optional[str]): The id associated with the matched recipient_name from the list.   
    """ 
    recipient_name: Optional[str] = Field('None', description= "The full name of the recipient that was matched from the list. ")
    recipient_id: Optional[str] = Field('None', description= "The id associated with the matched recipient_name from the list. ")
    
class MatchNameSchema(BaseModel):
    """
    Schema for the matched recipient name and id for a money transference. 
    
    Attributes:
        data (MatchName): Contains the matched recipient name and id information. 
    """
    data: MatchName

class MatchNameAssistant:
    def __init__(self, runnable):
        self.runnable = runnable

    def __call__(self, state, config):
        print("Ejecutando [Match Name Assistant]...",len(state.keys()))
        configuration = config.get("configurable", {})
        result = self.runnable.invoke(state)
        print(result)
        extracted_recipient_name = result.data.recipient_name if result.data.recipient_name != 'None' else 'None'
        extracted_recipient_id = result.data.recipient_id if result.data.recipient_id != 'None' else 'None'
        return {
            "recipient_name": extracted_recipient_name,
            "recipient_id": extracted_recipient_id}

match_name_runnable = match_name_prompt | llm.with_structured_output(schema=MatchNameSchema,include_raw=False)

destinatarios_dict =     {
        '1234': 'John Doe',
        '9090': 'Mcmarty Mirtha',
        '0912': 'Paul River',
    }

match_name_runnable.invoke({'recipient':'John',
                           'destinatarios_dict': destinatarios_dict,
                           'examples_match': examples_match[0]})

Description

### Description:

I'm working on a project using LangChain, and I need to pass examples to the LLM in the form of a dictionary, such as destinatarios_dict. However, I'm facing a limitation where I can only pass one system message to the MessagePlaceholder, and it doesn't seem to support the inclusion of multiple system messages or additional example data.

### Context:

System Message: I'm using a system message to set up initial instructions or context.
Examples: I want to pass a dictionary (e.g., destinatarios) as structured data to serve as examples for the model to understand the expected inputs/outputs.
Problem: The MessagePlaceholder only accepts a single system message, which restricts me from including both the structured data and additional system instructions simultaneously. When I try to include the dictionary as part of the system message, it causes conflicts or isn't processed as intended. I'm basing my code on this tutorial

### Error:
281 prev_ai_message = None
282 if i != 0:
--> 283 raise ValueError("SystemMessage should be the first in the history.")
284 if system_instruction is not None:
285 raise ValueError(
286 "Detected more than one SystemMessage in the list of messages."
287 "Gemini APIs support the insertion of only one SystemMessage."
288 )

ValueError: SystemMessage should be the first in the history.

### Goal:

I would like to know how I can pass structured data (like the destinatarios dictionary) as examples to the LLM, while also retaining my primary system message for setting up the context. Is there a recommended approach for passing multiple system messages or combining the system message with structured examples within LangChain?

Any guidance or suggestions on how to structure these messages effectively would be greatly appreciated!

System Info

langchain-core 0.2.4
langgraph 0.2.22
python3.12.4

[dosubot[bot]]

Hello, @Rov7! I'm here to help you with your technical questions and bug fixes. Let me know how I can assist you today!

To pass structured data, like a dictionary, as examples to an LLM in LangChain while retaining a primary system message for context, you can use the tool_example_to_messages function to convert your examples into a list of messages. This allows you to include both the system message and the examples in the prompt. Here's how you can do it:

1. Define your system message and examples.
2. Use the tool_example_to_messages function to convert each example into a list of messages.
3. Use MessagesPlaceholder to include the examples in the prompt.

Here's a code snippet demonstrating this:

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables import RunnablePassthrough
from langchain_openai import ChatOpenAI
from langchain_core.messages import (
    AIMessage,
    BaseMessage,
    HumanMessage,
    SystemMessage,
    ToolMessage,
)
import uuid
from typing import Dict, List

# Define the system message
system = """You are an expert at converting user questions into database queries. \
You have access to a database of tutorial videos about a software library for building LLM-powered applications. \
Given a question, return a list of database queries optimized to retrieve the most relevant results.

If there are acronyms or words you are not familiar with, do not try to rephrase them."""

# Define examples
examples = [
    {
        "input": "What's chat langchain, is it a langchain template?",
        "tool_calls": [
            Search(
                query="What is chat langchain and is it a langchain template?",
                sub_queries=["What is chat langchain", "What is a langchain template"],
            )
        ],
    },
    # Add more examples as needed
]

# Convert examples to messages
def tool_example_to_messages(example: Dict) -> List[BaseMessage]:
    messages: List[BaseMessage] = [HumanMessage(content=example["input"])]
    openai_tool_calls = []
    for tool_call in example["tool_calls"]:
        openai_tool_calls.append(
            {
                "id": str(uuid.uuid4()),
                "type": "function",
                "function": {
                    "name": tool_call.__class__.__name__,
                    "arguments": tool_call.json(),
                },
            }
        )
    messages.append(
        AIMessage(content="", additional_kwargs={"tool_calls": openai_tool_calls})
    )
    tool_outputs = example.get("tool_outputs") or [
        "You have correctly called this tool."
    ] * len(openai_tool_calls)
    for output, tool_call in zip(tool_outputs, openai_tool_calls):
        messages.append(ToolMessage(content=output, tool_call_id=tool_call["id"]))
    return messages

example_msgs = [msg for ex in examples for msg in tool_example_to_messages(ex)]

# Create the prompt with examples
prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system),
        MessagesPlaceholder("examples", optional=True),
        ("human", "{question}"),
    ]
)

# Initialize the LLM and structured output
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
structured_llm = llm.with_structured_output(Search)

# Create the query analyzer with examples
query_analyzer_with_examples = (
    {"question": RunnablePassthrough()}
    | prompt.partial(examples=example_msgs)
    | structured_llm
)

# Invoke the query analyzer
query_analyzer_with_examples.invoke(
    "what's the difference between web voyager and reflection agents? do both use langgraph?"
)

This approach allows you to retain the primary system message while including structured examples in the prompt ^[1].

To continue talking to Dosu, mention @dosu.

[Rov7]

Hey Dosu, thanks for your help! I’m currently using that function to generate structured messages, but I’m encountering an issue. As shown in the prompt, I’m passing the dictionary where it should look up the names. To address this, I need to include the dictionary as a system message within the structured example. However, when I do this, I get an error for having multiple system messages.

[dosubot[bot]]

LangChain enforces that only one SystemMessage can be included in a prompt, and it must be the first message. If you need to include structured data like a dictionary in your prompt, you can incorporate it into the content of the existing SystemMessage or pass it as part of the input data to be formatted into the prompt.

Here's how you can include the dictionary in the SystemMessage:

1. Embed the Dictionary in the System Message: You can format the dictionary as a string and include it in the SystemMessage content. This way, it remains part of the initial context without adding another SystemMessage.

2. Use Input Variables: Pass the dictionary as an input variable and format it into the SystemMessage content. This allows you to dynamically include the dictionary without violating the single SystemMessage rule.

Here's an example of how you might adjust your code:

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

# Define the system message with a placeholder for the dictionary
system_message_content = (
    "You are an expert in name matching. Your tasks are as follows: "
    "1. Match the recipient's name with a list that contains recipient names and their associated id. "
    "2. Return only one matching candidate from the provided list. "
    "3. If no exact or approximate match is found, return 'None'. "
    "The list is: {destinatarios_dict}. "
)

# Create the prompt with the system message and examples
match_name_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system_message_content),
        MessagesPlaceholder("examples_match"),
        ("human", "The recipient to match is: {recipient}"),
    ]
)

# Pass the dictionary as an input variable
destinatarios_dict = {
    '1234': 'John Doe',
    '9090': 'Mcmarty Mirtha',
    '0912': 'Paul River',
}

# Invoke the prompt with the dictionary and other inputs
match_name_runnable.invoke({
    'recipient': 'John',
    'destinatarios_dict': destinatarios_dict,
    'examples_match': examples_match[0]
})

This approach ensures that the dictionary is included in the prompt without adding additional SystemMessage instances, thus avoiding the error you're encountering.

To continue talking to Dosu, mention @dosu.

[Rov7]

I know, but the question relys on how can i pass examples that address the dictionaries as a system prompt to replicate the original flow.

[dosubot[bot]]

To pass examples that address dictionaries as part of the system prompt while maintaining the original flow, you can embed the dictionary directly into the SystemMessage content. This way, you can provide the necessary context and structured data without violating the single SystemMessage rule. Here's how you can do it:

1. Embed the Dictionary in the System Message: Format the dictionary as a string and include it in the SystemMessage content. This keeps the dictionary as part of the initial context.

2. Use Input Variables: Pass the dictionary as an input variable and format it into the SystemMessage content dynamically.

Here's an example:

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

# Define the system message with a placeholder for the dictionary
system_message_content = (
    "You are an expert in name matching. Your tasks are as follows: "
    "1. Match the recipient's name with a list that contains recipient names and their associated id. "
    "2. Return only one matching candidate from the provided list. "
    "3. If no exact or approximate match is found, return 'None'. "
    "The list is: {destinatarios_dict}. "
)

# Create the prompt with the system message and examples
match_name_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system_message_content),
        MessagesPlaceholder("examples_match"),
        ("human", "The recipient to match is: {recipient}"),
    ]
)

# Pass the dictionary as an input variable
destinatarios_dict = {
    '1234': 'John Doe',
    '9090': 'Mcmarty Mirtha',
    '0912': 'Paul River',
}

# Invoke the prompt with the dictionary and other inputs
match_name_runnable.invoke({
    'recipient': 'John',
    'destinatarios_dict': destinatarios_dict,
    'examples_match': examples_match[0]
})

This method allows you to include the dictionary in the prompt as part of the SystemMessage, ensuring that the structured data is available for the model to use without adding additional SystemMessage instances.

To continue talking to Dosu, mention @dosu.