Training of Agent_Executor on dataset possible?

[mraguth]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

Here my agent executor is using python repl.

# Define the prompt
prompt = ChatPromptTemplate.from_messages(
    [
        ("system", template),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
        ("human", "{input}"),
        MessagesPlaceholder("chat_history", optional=True),
    ]
)

repl = PythonAstREPLTool(
    locals={"df": df},
    name="python_repl",
    description="Runs Python code and returns the output of the final line",
    args_schema=PythonInputs,
    prompt = prompt,
)


tools = [repl, retriever_tool]
agent = create_openai_functions_agent(
    llm=llm, prompt=prompt, tools=tools
)
agent_executor = AgentExecutor(
    agent=agent, 
    tools=tools, 
    max_iterations=10, 
    early_stopping_method="force",    ##### Changed from generate due to Generate error.
    allow_dangerous_code=True,
    allow_dangerous_requests=True,
) | (lambda x: x["output"])

Description

Similar to this post #9591, is it possible to train my agent_executor who is using python repl (datafram qery)? I like to train on my dataset columns and data before asking a question.

System Info

Name: langchain
Version: 0.3.0

[dosubot[bot]]

Yes, it is possible to train your agent_executor using a Python REPL on specific dataset columns and data before querying. The provided code snippet shows that the agent_executor is set up with a PythonAstREPLTool which allows running code snippets on the dataframe df. This means you can perform exploratory data analysis and manipulate specific columns of the dataset using the REPL tool before making queries.

Here is the relevant part of the code:

repl = PythonAstREPLTool(
    locals={"df": df},
    name="python_repl",
    description="Runs code and returns the output of the final line",
    args_schema=PythonInputs,
)
tools = [repl, retriever_tool]
agent = create_openai_functions_agent(
    llm=llm, prompt=prompt, tools=tools
)
agent_executor = AgentExecutor(
    agent=agent, tools=tools, max_iterations=10, early_stopping_method="force",
    allow_dangerous_code=True, allow_dangerous_requests=True
) | (lambda x: x["output"])

This setup allows the agent_executor to use the python_repl tool to run code on the dataframe df, enabling training and manipulation of specific dataset columns and data before querying.

Additionally, the LangChain framework provides a comprehensive setup for integrating various tools, including a Python REPL, with language models. The langchain_experimental.utilities module includes a PythonREPL class that can be used to create a REPL tool for executing Python commands. This can be particularly useful for tasks such as data analysis and manipulation within a dataframe.

Here is an example of how to set up the Python REPL tool using LangChain:

from langchain.agents import Tool
from langchain_experimental.utilities import PythonREPL

python_repl = PythonREPL()

repl_tool = Tool(
    name="python_repl",
    description="A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.",
    func=python_repl.run,
)

This setup allows you to execute Python commands interactively, making it easier to explore and manipulate your dataset before querying.

For more advanced use cases and flexibility, LangChain also offers other tools and agents, such as LangGraph Agents, which provide more control and customization options. You can refer to the LangChain documentation for more details on setting up and using these advanced agents ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[mraguth]

@dosu How I use query function after training? after your comment This setup allows the agent_executor to use the python_repl tool to run code on the dataframe df, enabling training and manipulation of specific dataset columns and data before querying.

[dosubot[bot]]

To use the query function after training the agent_executor with the python_repl tool on the dataframe df, you can simply call the agent_executor with the query string. Here is an example:

result = agent_executor.invoke({"input": "your query here"})
print(result)

For instance, if you want to find out the average age of passengers in the dataframe df, you can do:

result = agent_executor.invoke({"input": "What is the average age of passengers?"})
print(result)

This will use the python_repl tool to execute the necessary code to answer the query based on the dataframe df.

Additionally, you can use the run method to execute queries directly, similar to how it is done with the PowerBI toolkit:

result = agent_executor.run("What is the average age of passengers?")
print(result)

This approach leverages the language model to interpret and execute the query on the dataframe df.

Here is an example from the templates/csv-agent/main.py file in the LangChain framework:

from csv_agent.agent import agent_executor

if __name__ == "__main__":
    question = "who was in cabin c28?"
    print(agent_executor.invoke({"input": question}))

This demonstrates how to use the agent_executor to query information from a CSV file, which is similar to querying a dataframe ^[1][2][3][4].

To continue talking to Dosu, mention @dosu.

[mraguth]

@dosu Thank you. One last question... what is the difference between what I used below for python repl vs. what you suggested?

repl = PythonAstREPLTool(
    locals={"df": df},
    name="python_repl",
    description="Runs Python code and returns the output of the final line",
    args_schema=PythonInputs,
    prompt = prompt,
)

Your suggestion:

from langchain.agents import Tool
from langchain_experimental.utilities import PythonREPL

python_repl = PythonREPL()

repl_tool = Tool(
    name="python_repl",
    description="A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.",
    func=python_repl.run,
)

[dosubot[bot]]

The key differences between PythonAstREPLTool and PythonREPL are in their capabilities and limitations:

PythonAstREPLTool:

• Capabilities:
  • Can handle both single-line and multi-line inputs.
  • Supports execution of code within backticks (`) and triple backticks (```).
  • Can return values directly from expressions (e.g., 1 + 1 returns 2).
  • Can handle exceptions and return appropriate error messages (e.g., IndexError, KeyError).
  • Requires Python version >= 3.9.
• Limitations:
  • Requires Python version >= 3.9 to run.
  • May need additional context (e.g., local variables) to execute some code snippets correctly.

PythonREPL:

• Capabilities:
  • Can handle single-line and multi-line inputs.
  • Executes code and captures printed output (e.g., print statements).
  • Does not require a specific Python version.
• Limitations:
  • Primarily captures printed output and may not return values directly from expressions.
  • Does not handle code within backticks or triple backticks as PythonAstREPLTool does.

Example Code:

# Using PythonAstREPLTool
repl = PythonAstREPLTool(
    locals={"df": df},
    name="python_repl",
    description="Runs Python code and returns the output of the final line",
    args_schema=PythonInputs,
    prompt=prompt,
)

# Using PythonREPL
from langchain.agents import Tool
from langchain_experimental.utilities import PythonREPL

python_repl = PythonREPL()

repl_tool = Tool(
    name="python_repl",
    description="A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.",
    func=python_repl.run,
)

In summary, PythonAstREPLTool offers more advanced capabilities such as direct value returns and exception handling but requires a newer Python version, while PythonREPL is more straightforward and captures printed output without such version constraints ^[1].

To continue talking to Dosu, mention @dosu.