E2B Code Interpreter with LangGraph with pandas df possible?

[mraguth]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

https://github.com/e2b-dev/e2b-cookbook/blob/main/examples/langgraph-python/langgraph_code_interpreter.ipynb

I am explorng e2b python sandbox and if possible I like to ingest pandas df and in the Langgraph and execute my query. Please help.

Description

Can I bring my own pandas df and ask questions? Is that possible?

# 4. Invoke the app
result = app.invoke([("human", "plot and show sinus")])

code_interpreter.close()

def display_results(messages):
    for message in result:
        if hasattr(message, 'raw_output'):
            if message.raw_output["results"]:
                rs = message.raw_output["results"]
                for r in rs:
                    display(r)

display_results(result)

System Info

Name: langchain
Version: 0.3.0

[dosubot[bot]]

Yes, it is possible to use a pandas DataFrame with the E2B Code Interpreter and LangGraph to execute queries. E2B's Data Analysis sandbox allows for safe code execution in a sandboxed environment, which is ideal for building tools such as code interpreters or advanced data analysis. You can run Python code, generate charts via matplotlib, install Python packages dynamically during runtime, and upload and download files. Specifically, you can install the pandas package and use it within the sandbox to manipulate and query DataFrames ^[1][2].

Here are the specific steps to integrate a pandas DataFrame with LangGraph in the E2B Code Interpreter:

1. Install Required Dependencies:

   %pip install --upgrade --quiet langchain e2b langchain-community pandas

2. Load Your Data into a pandas DataFrame:

   import pandas as pd
   
   df = pd.read_csv("path_to_your_csv_file.csv")

3. Create a Python Tool for DataFrame Interaction:

   from langchain_community.tools import E2BDataAnalysisTool
   
   # Artifacts are charts created by matplotlib when `plt.show()` is called
   def save_artifact(artifact):
       print("New matplotlib chart generated:", artifact.name)
       file = artifact.download()
       basename = os.path.basename(artifact.name)
       with open(f"./charts/{basename}", "wb") as f:
           f.write(file)
   
   e2b_data_analysis_tool = E2BDataAnalysisTool(
       env_vars={"MY_SECRET": "secret_value"},
       on_stdout=lambda stdout: print("stdout:", stdout),
       on_stderr=lambda stderr: print("stderr:", stderr),
       on_artifact=save_artifact,
   )

4. Upload Your Data to the Sandbox:

   with open("./path_to_your_csv_file.csv") as f:
       remote_path = e2b_data_analysis_tool.upload_file(
           file=f,
           description="Description of your data",
       )
       print(remote_path)

5. Bind the Tool to the LLM:

   from langchain.agents import AgentType, initialize_agent
   from langchain_openai import ChatOpenAI
   
   llm = ChatOpenAI(model="gpt-4", temperature=0)
   tools = [e2b_data_analysis_tool.as_tool()]
   agent = initialize_agent(
       tools,
       llm,
       agent=AgentType.OPENAI_FUNCTIONS,
       verbose=True,
       handle_parsing_errors=True,
   )

6. Create a Prompt Template:

   from langchain_core.prompts import ChatPromptTemplate
   
   system = f"""You have access to a pandas dataframe `df`. \
   Here is the output of `df.head().to_markdown()`:

   {df.head().to_markdown()}

   
   Given a user question, write the Python code to answer it. \
   Return ONLY the valid Python code and nothing else. \
   Don't assume you have access to any libraries other than built-in Python ones and pandas."""
   prompt = ChatPromptTemplate.from_messages([("system", system), ("human", "{question}")])
   

7. Create a Chain to Execute the Code:

   from langchain_core.output_parsers.openai_tools import JsonOutputKeyToolsParser
   
   parser = JsonOutputKeyToolsParser(key_name=e2b_data_analysis_tool.name, first_tool_only=True)
   chain = prompt | agent | parser | e2b_data_analysis_tool

8. Invoke the Chain with a Question:

   response = chain.invoke({"question": "What's the correlation between age and fare?"})
   print(response)

By following these steps, you can integrate a pandas DataFrame with LangGraph in the E2B Code Interpreter and perform data analysis using natural language queries ^[1][2].

To continue talking to Dosu, mention @dosu.

[mraguth]

@dosu. This is great example for langchain. Do you have similar example for "LangGraph"? Really interested in LangGraph.

[dosubot[bot]]

Yes, here is an example of using "LangGraph" to implement a Corrective-RAG framework:

! pip install langchain-chroma langchain_community tiktoken langchain-openai langchainhub langchain langgraph tavily-python

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_chroma import Chroma
from langchain_community.document_loaders import WebBaseLoader
from langchain_openai import OpenAIEmbeddings

urls = [
    "https://lilianweng.github.io/posts/2023-06-23-agent/",
    "https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/",
    "https://lilianweng.github.io/posts/2023-10-25-adv-attack-llm/",
]

docs = [WebBaseLoader(url).load() for url in urls]
docs_list = [item for sublist in docs for item in sublist]

text_splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(
    chunk_size=250, chunk_overlap=0
)
doc_splits = text_splitter.split_documents(docs_list)

# Add to vectorDB
vectorstore = Chroma.from_documents(
    documents=doc_splits,
    collection_name="rag-chroma",
    embedding=OpenAIEmbeddings(),
)
retriever = vectorstore.as_retriever()

from typing import Dict, TypedDict

from langchain_core.messages import BaseMessage

class GraphState(TypedDict):
    keys: Dict[str, any]

import json
import operator
from typing import Annotated, Sequence, TypedDict

from langchain import hub
from langchain.output_parsers import PydanticOutputParser
from langchain.output_parsers.openai_tools import PydanticToolsParser
from langchain.prompts import PromptTemplate
from langchain.schema import Document
from langchain_chroma import Chroma
from langchain_community.tools.tavily_search import TavilySearchResults
from langchain_core.messages import BaseMessage, FunctionMessage
from langchain_core.output_parsers import StrOutputParser
from langchain_core.pydantic_v1 import BaseModel, Field
from langchain_core.runnables import RunnablePassthrough
from langchain_core.utils.function_calling import convert_to_openai_tool
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langgraph.prebuilt import ToolInvocation

def retrieve(state):
    print("---RETRIEVE---")
    state_dict = state["keys"]
    question = state_dict["question"]
    documents = retriever.invoke(question)
    return {"keys": {"documents": documents, "question": question}}

def generate(state):
    print("---GENERATE---")
    state_dict = state["keys"]
    question = state_dict["question"]
    documents = state_dict["documents"]

    prompt = hub.pull("rlm/rag-prompt")
    llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0, streaming=True)

    def format_docs(docs):
        return "\n\n".join(doc.page_content for doc in docs)

    rag_chain = prompt | llm | StrOutputParser()
    generation = rag_chain.invoke({"context": documents, "question": question})
    return {
        "keys": {"documents": documents, "question": question, "generation": generation}
    }

def grade_documents(state):
    print("---CHECK RELEVANCE---")
    state_dict = state["keys"]
    question = state_dict["question"]
    documents = state_dict["documents"]

    class grade(BaseModel):
        binary_score: str = Field(description="Relevance score 'yes' or 'no'")

    model = ChatOpenAI(temperature=0, model="gpt-4-0125-preview", streaming=True)
    grade_tool_oai = convert_to_openai_tool(grade)
    llm_with_tool = model.bind(
        tools=[convert_to_openai_tool(grade_tool_oai)],
        tool_choice={"type": "function", "function": {"name": "grade"}},
    )
    parser_tool = PydanticToolsParser(tools=[grade])
    prompt = PromptTemplate(
        template="""You are a grader assessing relevance of a retrieved document to a user question. \n 
        Here is the retrieved document: \n\n {context} \n\n
        Here is the user question: {question} \n
        If the document contains keyword(s) or semantic meaning related to the user question, grade it as relevant. \n
        Give a binary score 'yes' or 'no' score to indicate whether the document is relevant to the question.""",
        input_variables=["context", "question"],
    )
    chain = prompt | llm_with_tool | parser_tool

    filtered_docs = []
    search = "No"
    for d in documents:
        score = chain.invoke({"question": question, "context": d.page_content})
        grade = score[0].binary_score
        if grade == "yes":
            print("---GRADE: DOCUMENT RELEVANT---")
            filtered_docs.append(d)
        else:
            print("---GRADE: DOCUMENT NOT RELEVANT---")
            search = "Yes"
            continue

    return {
        "keys": {
            "documents": filtered_docs,
            "question": question,
            "run_web_search": search,
        }
    }

def transform_query(state):
    print("---TRANSFORM QUERY---")
    state_dict = state["keys"]
    question = state_dict["question"]
    documents = state_dict["documents"]

    prompt = PromptTemplate(
        template="""You are generating questions that is well optimized for retrieval. \n 
        Look at the input and try to reason about the underlying sematic intent / meaning. \n 
        Here is the initial question:
        \n ------- \n
        {question} 
        \n ------- \n
        Formulate an improved question: """,
        input_variables=["question"],
    )

    model = ChatOpenAI(temperature=0, model="gpt-4-0125-preview", streaming=True)
    chain = prompt | model | StrOutputParser()
    better_question = chain.invoke({"question": question})

    return {"keys": {"documents": documents, "question": better_question}}

def web_search(state):
    print("---WEB SEARCH---")
    state_dict = state["keys"]
    question = state_dict["question"]
    documents = state_dict["documents"]

    tool = TavilySearchResults()
    docs = tool.invoke({"query": question})
    web_results = "\n".join([d["content"] for d in docs])
    web_results = Document(page_content=web_results)
    documents.append(web_results)

    return {"keys": {"documents": documents, "question": question}}

def decide_to_generate(state):
    print("---DECIDE TO GENERATE---")
    state_dict = state["keys"]
    question = state_dict["question"]
    filtered_documents = state_dict["documents"]
    search = state_dict["run_web_search"]

    if search == "Yes":
        print("---DECISION: TRANSFORM QUERY and RUN WEB SEARCH---")
        return "transform_query"
    else:
        print("---DECISION: GENERATE---")
        return "generate"

import pprint

from langgraph.graph import END, StateGraph

workflow = StateGraph(GraphState)

workflow.add_node("retrieve", retrieve)
workflow.add_node("grade_documents", grade_documents)
workflow.add_node("generate", generate)
workflow.add_node("transform_query", transform_query)
workflow.add_node("web_search", web_search)

workflow.set_entry_point("retrieve")
workflow.add_edge("retrieve", "grade_documents")
workflow.add_conditional_edges(
    "grade_documents",
    decide_to_generate,
    {
        "transform_query": "transform_query",
        "generate": "generate",
    },
)
workflow.add_edge("transform_query", "web_search")
workflow.add_edge("web_search", "generate")
workflow.add_edge("generate", END)

app = workflow.compile()

inputs = {"keys": {"question": "Explain how the different types of agent memory work?"}}
for output in app.stream(inputs):
    for key, value in output.items():
        pprint.pprint(f"Output from node '{key}':")
        pprint.pprint("---")
        pprint.pprint(value["keys"], indent=2, width=80, depth=None)
    pprint.pprint("\n---\n")

inputs = {"keys": {"question": "What is the approach taken in the AlphaCodium paper?"}}
for output in app.stream(inputs):
    for key, value in output.items():
        pprint.pprint(f"Output from node '{key}':")
        pprint.pprint("---")
        pprint.pprint(value["keys"], indent=2, width=80, depth=None)
    pprint.pprint("\n---\n")

This example demonstrates how to use LangGraph to create a Corrective-RAG framework, including document retrieval, grading, query transformation, and web search ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[mraguth]

@dosu I am looking for example of 'E2B Code Interpreter with LangGraph with pandas df'

[dosubot[bot]]

Here is an example of using the E2B Code Interpreter with LangGraph and a pandas DataFrame:

1. Install Required Dependencies:

   %pip install -qU langchain langchain-openai langchain-community langchain-experimental pandas e2b

2. Load Your Data into a pandas DataFrame:

   import pandas as pd
   
   df = pd.read_csv("path_to_your_csv_file.csv")

3. Create a Python Tool for DataFrame Interaction:

   from langchain_experimental.tools import PythonAstREPLTool
   
   tool = PythonAstREPLTool(locals={"df": df})

4. Initialize the LLM and Bind the Tool:

   from langchain_openai import ChatOpenAI
   
   llm = ChatOpenAI()
   llm_with_tools = llm.bind_tools([tool], tool_choice=tool.name)

5. Create a Prompt Template:

   from langchain_core.prompts import ChatPromptTemplate
   
   system = f"""You have access to a pandas dataframe `df`. \
   Here is the output of `df.head().to_markdown()`:

   {df.head().to_markdown()}

   
   Given a user question, write the Python code to answer it. \
   Return ONLY the valid Python code and nothing else. \
   Don't assume you have access to any libraries other than built-in Python ones and pandas."""
   prompt = ChatPromptTemplate.from_messages([("system", system), ("human", "{question}")])
   

6. Create a Parser and Chain:

   from langchain_core.output_parsers.openai_tools import JsonOutputKeyToolsParser
   
   parser = JsonOutputKeyToolsParser(key_name=tool.name, first_tool_only=True)
   chain = prompt | llm_with_tools | parser | tool

7. Invoke the Chain with a Question:

   result = chain.invoke({"question": "What's the correlation between age and fare?"})
   print(result)

8. Example of Using E2B Data Analysis Tool:

   from langchain_community.tools import E2BDataAnalysisTool
   
   e2b_data_analysis_tool = E2BDataAnalysisTool(
       env_vars={"MY_SECRET": "secret_value"},
       on_stdout=lambda stdout: print("stdout:", stdout),
       on_stderr=lambda stderr: print("stderr:", stderr),
       on_artifact=lambda artifact: print("New artifact generated:", artifact.name)
   )
   
   # Upload the Titanic dataset to the E2B sandbox
   with open("path_to_your_csv_file.csv") as f:
       remote_path = e2b_data_analysis_tool.upload_file(
           file=f,
           description="Description of your data"
       )
       print(remote_path)
   
   # Create a Tool object and initialize the Langchain agent
   from langchain.agents import AgentType, initialize_agent
   
   tools = [e2b_data_analysis_tool.as_tool()]
   agent = initialize_agent(
       tools,
       llm,
       agent=AgentType.OPENAI_FUNCTIONS,
       verbose=True,
       handle_parsing_errors=True,
   )
   
   # Ask the agent a question about the uploaded CSV file
   agent.run("What's the correlation between age and fare?")

This example demonstrates how to integrate a pandas DataFrame with LangGraph in the E2B Code Interpreter and perform data analysis using natural language queries ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[mraguth]

@dosu...in above code, pasted the code below...how I upload my pandas dataframe instead of 'titanic dataset'. Please help.

# Upload the Titanic dataset to the E2B sandbox
with open("path_to_your_csv_file.csv") as f:
    remote_path = e2b_data_analysis_tool.upload_file(
        file=f,
        description="Description of your data"
    )
    print(remote_path)