how can i add custom prompt in create_pandas_dataframe_agent

[deepak-hl]

            agent = create_pandas_dataframe_agent(
                llm, df, verbose=True, allow_dangerous_code=True, PREFIX=PREFIX, FORMAT_INSTRUCTIONS=FORMAT_INSTRUCTIONS,SUFFIX=SUFFIX
            )

After using above code I am getting below error

/home/deepak/summarizer_bot/.venv/lib/python3.12/site-packages/langchain_experimental/agents/agent_toolkits/pandas/base.py:283: UserWarning: Received additional kwargs {'PREFIX': 'Answer the following questions as best you can. You have access to the following tools:', 'FORMAT_INSTRUCTIONS': 'Use the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction: the action to take, should be one of [{tool_names}]\nAction Input: the input to the action\nObservation: the result of the action\n... (this Thought/Action/Action Input/Observation can repeat N times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question in JSON format, where the keys are entities and the values are their corresponding data', 'SUFFIX': 'Begin!\n\nQuestion: {input}\nThought:{agent_scratchpad}'} which are no longer supported.

[dosubot[bot]]

The error you're encountering is due to the use of unsupported keyword arguments (PREFIX, FORMAT_INSTRUCTIONS, SUFFIX) in the create_pandas_dataframe_agent function. To customize prompts, you should use the CustomPromptTemplate class instead.

Here's how you can do it:

1. Define a Custom Prompt Template:

   Use the CustomPromptTemplate class to format your prompt with the necessary variables.

   from langchain.prompts import StringPromptTemplate
   
   class CustomPromptTemplate(StringPromptTemplate):
       template: str
       tools_getter: Callable
   
       def format(self, **kwargs) -> str:
           intermediate_steps = kwargs.pop("intermediate_steps")
           thoughts = ""
           for action, observation in intermediate_steps:
               thoughts += action.log
               thoughts += f"\nObservation: {observation}\nThought: "
           kwargs["agent_scratchpad"] = thoughts
           tools = self.tools_getter(kwargs["input"])
           kwargs["tools"] = "\n".join(
               [f"{tool.name}: {tool.description}" for tool in tools]
           )
           kwargs["tool_names"] = ", ".join([tool.name for tool in tools])
           return self.template.format(**kwargs)

2. Use the Custom Prompt Template:

   Pass this custom prompt template to your agent setup.

   custom_template = CustomPromptTemplate(
       template="Your custom template here",
       tools_getter=lambda x: []  # Define how to get tools based on input
   )
   
   agent = create_pandas_dataframe_agent(
       llm, df, verbose=True, prompt_template=custom_template
   )

This approach allows you to customize the prompt while adhering to the current API requirements ^[1][2].

To continue talking to Dosu, mention @dosu.