How to take in JSON string as user input using ChatPromptTemplate and MessagesPlaceholder

[Yuxuan1998]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder

translator_prompt_template = """
Translate to English, do not add any extra information
"""

# create prompt
translator_prompt_f_string = ChatPromptTemplate.from_messages(
    [("system", translator_prompt_template), 
    MessagesPlaceholder(variable_name="messages"),],
)

translator_prompt_jinja2 = ChatPromptTemplate.from_messages(
    [("system", translator_prompt_template), 
    MessagesPlaceholder(variable_name="messages"),],
    template_format="jinja2"
)

# Define the chain
translator_agent_f_string = translator_prompt_f_string | llm
translator_agent_jinja2 = translator_prompt_jinja2 | llm

# invoke
translator_agent_f_string.invoke({"messages":[
    ('user', "[{{'happy':'脸', 'high':'温度'}}]")
    ]})
translator_agent_jinja2.invoke({"messages":[
    ('user', "[{'happy':'脸', 'high':'温度'}]")
    ]})

Description

I'm trying to translate JSON string to English using ChatPromptTemplate and MessagesPlaceholder, for instance:

• input JSON string: "[{'happy':'脸', 'high':'温度'}]"
• output translated JSON string: "[{'happy':'face', 'high':'temperature'}]"

I tried 2 template_format, but encountered error with both approaches:

1. template_format="f_string" (default) with double {}: TypeError: unhashable type: 'dict'
2. template_format="jinja2": KeyError: 'type'

Can someone enlighten me how to solve the problem? Thanks in advance

System Info

System Information

OS: Darwin
OS Version: Darwin Kernel Version 23.6.0: Wed Jul 31 20:49:39 PDT 2024; root:xnu-10063.141.1.700.5~1/RELEASE_ARM64_T6000
Python Version: 3.11.3 (main, Apr 19 2023, 18:49:55) [Clang 14.0.6 ]

Package Information

langchain_core: 0.3.5
langchain: 0.3.0
langchain_community: 0.3.0
langsmith: 0.1.125
langchain_experimental: 0.3.0
langchain_google_vertexai: 2.0.1
langchain_text_splitters: 0.3.0
langgraph: 0.2.23

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.10.5
anthropic[vertexai]: Installed. No version info available.
async-timeout: Installed. No version info available.
dataclasses-json: 0.6.4
google-cloud-aiplatform: 1.67.1
google-cloud-storage: 2.18.2
httpx: 0.27.2
httpx-sse: 0.4.0
jsonpatch: 1.33
langchain-mistralai: Installed. No version info available.
langgraph-checkpoint: 1.0.10
numpy: 1.26.4
orjson: 3.10.7
packaging: 23.2
pydantic: 2.9.2
pydantic-settings: 2.5.2
PyYAML: 6.0.2
requests: 2.31.0
SQLAlchemy: 2.0.35
tenacity: 8.5.0
typing-extensions: 4.9.0

[dosubot[bot]]

I found a similar unsolved discussion that might be relevant to your issue: LangGraph - Plan-and-Execute Error - Need Help ^[1].

To take in a JSON string as user input using ChatPromptTemplate and MessagesPlaceholder in LangChain, and resolve the errors encountered with both f_string and jinja2 template formats, you can follow the example provided below:

1. Define the System and Human Prompts:

   • The system prompt provides the context for the assistant.
   • The human prompt includes placeholders for tools, tool names, and user input.

2. Create the ChatPromptTemplate:

   • Use MessagesPlaceholder for chat_history and agent_scratchpad.

3. Handle JSON Input:

   • Ensure the JSON input is correctly formatted and passed into the template.

Here is the code example:

import base64
import json

from langchain_community.chat_models import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder, SystemMessagePromptTemplate
from langchain_core.pydantic_v1 import Field
from langserve import CustomUserType
from langchain.agents import AgentExecutor, create_json_chat_agent
from langchain import hub

# Define the system prompt
system = '''Assistant is a large language model trained by OpenAI.

Assistant is designed to be able to assist with a wide range of tasks, from answering \
simple questions to providing in-depth explanations and discussions on a wide range of \
topics. As a language model, Assistant is able to generate human-like text based on \
the input it receives, allowing it to engage in natural-sounding conversations and \
provide responses that are coherent and relevant to the topic at hand.
Assistant is constantly learning and improving, and its capabilities are constantly \
evolving. It is able to process and understand large amounts of text, and can use this \
knowledge to provide accurate and informative responses to a wide range of questions. \
Additionally, Assistant is able to generate its own text based on the input it \
receives, allowing it to engage in discussions and provide explanations and \
descriptions on a wide range of topics.

Overall, Assistant is a powerful system that can help with a wide range of tasks \
and provide valuable insights and information on a wide range of topics. Whether \
you need help with a specific question or just want to have a conversation about \
a particular topic, Assistant is here to assist.'''

# Define the human prompt
human = '''TOOLS
------
Assistant can ask the user to use tools to look up information that may be helpful in \
answering the users original question. The tools the human can use are:

{tools}

RESPONSE FORMAT INSTRUCTIONS
----------------------------

When responding to me, please output a response in one of two formats:

**Option 1:**
Use this if you want the human to use a tool.
Markdown code snippet formatted in the following schema:

```json
{{
    "action": string, \\ The action to take. Must be one of {tool_names}
    "action_input": string \\ The input to the action
}}

Option #2:
Use this if you want to respond directly to the human. Markdown code snippet formatted
in the following schema:

{{
    "action": "Final Answer",
    "action_input": string \\ You should put what you want to return to use here
}}

USER'S INPUT

Here is the user's input (remember to respond with a markdown code snippet of a json
blob with a single action, and NOTHING else):

{input}'''

Create the ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages(
[
SystemMessagePromptTemplate.from_template(system),
MessagesPlaceholder("chat_history", optional=True),
("human", human),
MessagesPlaceholder("agent_scratchpad"),
]
)

Pull the prompt from the hub

prompt = hub.pull("hwchase17/react-chat-json")

Initialize the model and tools

model = ChatOpenAI()
tools = [...] # Define your tools here

Create the JSON chat agent

agent = create_json_chat_agent(model, tools, prompt)

Create the agent executor

agent_executor = AgentExecutor(agent=agent, tools=tools)

Invoke the agent with user input

user_input = '{"input": "hi"}'
agent_executor.invoke(user_input)

This code sets up the `ChatPromptTemplate` with the necessary placeholders and handles JSON input correctly. Ensure that the JSON input is properly formatted and passed into the `invoke` method. The `MessagesPlaceholder` class allows for flexible handling of message lists, and the `ChatPromptTemplate` can be configured to use either `f-string` or `jinja2` template formats as needed <sup>[[2]](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/agents/json_chat/base.py)</sup><sup>[[3]](https://github.com/langchain-ai/langchain/blob/master/libs/core/langchain_core/prompts/chat.py)</sup><sup>[[4]](https://github.com/langchain-ai/langchain/blob/master/templates/plate-chain/plate_chain/chain.py)</sup>.


<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*