Performance Issue with Langchain 0.2.6

[herbertmoses]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

#Customized a tool to generate, create, get, update, and delete a ticket in Rally in the following sample manner.

class GenerateUserStoryInput(BaseModel):
    """Input for generating user stories based on user query.Note: There is another tool for create"""
   
    user_query: str = Field(..., description="Scenario from the user to generate user story")
    application: str = Field(..., description="Application setting from user")
    tool: str = Field(..., description="Tool input from user like Rally,JIRA")
 
 
class GenerateUserStoryTool(BaseTool):
    name = "generate_userstory"
    description = (
        "Useful for Generating User Stories based on User input. This is just for generation of User Story"
        "Use this tool if user is specifically looking to generate user story"
        "Tool output:"
        "Do not auto generate random value for any field"
        "Provide the tool output as is to the user."
        "As part of the response, check with the user if the use story can be created in Rally with this"
        )
 
    def _run(self, user_query: str,application: str, tool :str):
        try:
            print("Tool MEMORY: ", memory.load_memory_variables({}))
            response = generate_userstory(user_query,application,tool)
           
            #response = "User Story Successfully Created with No:U4578899"
            #ext_step= "Would you like to create test cases for this userstory?(Yes/no):"
            return response
        except Exception as e:
            print(f"An error occurred ")
            # Optionally, return a specific error message or re-raise the exception
            # depending on your application's error handling policy.
            return {"error": "An unexpected error occurred. Please try again later."}
 
    args_schema: Optional[Type[BaseModel]] = GenerateUserStoryInput
 


#Below are the code steps for invoking the agent, creating the runnable and invoking the same

open_ai_agent = initialize_agent(tools, llm, agent=AgentType.OPENAI_FUNCTIONS, verbose=True,agent_kwargs=agent_kwargs)

agent_executor_with_memory = RunnableWithMessageHistory(
                        open_ai_agent,
                        get_session_history,
                        input_messages_key="input",
                        history_messages_key="chat_history"
                        )
 
response = agent_executor_with_memory.invoke(
                        {"input": userquestion},
                   
                    config={
                        "configurable": {"session_id": session_id}
                    }
                    )
                    print( "Print:",response["output"])

Description

Creating tools using LCEL and Pydantic to perform the following tasks in Rally:
i. Generate and create a ticket
ii. Get details of an existing ticket
iii. Update an existing ticket
iv. Delete an existing ticket

While invoking the Runnable, we are facing lagging issues some occasion

System Info

Pydantic = 2.8.2
Langchain = 0.2.6

[dosubot[bot]]

Hey there, @herbertmoses! I'm here to help you with any bugs, questions, or contributions you may have. Let's tackle this issue together.

To resolve performance issues when invoking a Runnable with Langchain 0.2.6 and Pydantic 2.8.2 for generating, creating, getting, updating, and deleting tickets in Rally, you can leverage the parallel execution capabilities provided by RunnableParallel. This allows you to run multiple tasks concurrently, which can significantly improve performance.

Here is an example of how you can use RunnableParallel to run tasks in parallel:

from langchain_core.runnables import RunnableLambda, RunnableParallel

# Define your functions for generating, creating, getting, updating, and deleting tickets
def generate_ticket(data):
    # Implementation for generating a ticket
    pass

def create_ticket(data):
    # Implementation for creating a ticket
    pass

def get_ticket(data):
    # Implementation for getting a ticket
    pass

def update_ticket(data):
    # Implementation for updating a ticket
    pass

def delete_ticket(data):
    # Implementation for deleting a ticket
    pass

# Wrap the functions in RunnableLambda
generate_runnable = RunnableLambda(generate_ticket)
create_runnable = RunnableLambda(create_ticket)
get_runnable = RunnableLambda(get_ticket)
update_runnable = RunnableLambda(update_ticket)
delete_runnable = RunnableLambda(delete_ticket)

# Create a RunnableParallel to run the tasks in parallel
parallel_runnable = RunnableParallel({
    "generate": generate_runnable,
    "create": create_runnable,
    "get": get_runnable,
    "update": update_runnable,
    "delete": delete_runnable,
})

# Invoke the parallel runnable
result = parallel_runnable.invoke(data)
# Or asynchronously
result = await parallel_runnable.ainvoke(data)

By using RunnableParallel, you can ensure that the tasks for generating, creating, getting, updating, and deleting tickets are executed concurrently, which should help in resolving performance issues ^[1].

Additionally, you can take advantage of the Chain class in LangChain to create structured sequences of calls to components. This class provides methods like invoke and ainvoke for synchronous and asynchronous execution, respectively. The Chain class also supports stateful operations, observability through callbacks, and composability with other components, which can further enhance the performance and manageability of your tasks.

from langchain.chains.base import Chain

class TicketChain(Chain):
    def __init__(self):
        super().__init__()
        self.input_keys = ["data"]
        self.output_keys = ["result"]

    def _call(self, inputs, run_manager=None):
        # Implement the sequence of operations here
        data = inputs["data"]
        # Example: generate, create, get, update, delete tickets
        generate_ticket(data)
        create_ticket(data)
        ticket = get_ticket(data)
        update_ticket(data)
        delete_ticket(data)
        return {"result": ticket}

    async def _acall(self, inputs, run_manager=None):
        # Implement the sequence of operations here
        data = inputs["data"]
        # Example: generate, create, get, update, delete tickets
        generate_ticket(data)
        create_ticket(data)
        ticket = get_ticket(data)
        update_ticket(data)
        delete_ticket(data)
        return {"result": ticket}

# Create an instance of the chain
ticket_chain = TicketChain()

# Invoke the chain
result = ticket_chain.invoke({"data": data})
# Or asynchronously
result = await ticket_chain.ainvoke({"data": data})

By structuring your tasks within a Chain, you can leverage its built-in capabilities for input validation, output preparation, and memory management, further optimizing the performance and reliability of your ticket management operations in Rally ^[2].

To continue talking to Dosu, mention @dosu.

[eyurtsev]

This is complete nonsense

[eyurtsev]

In the provided example, it's impossible to tell whether issues are in user code, user database configuration, or langchain. You should use a tool to benchmark the code and identify where lagging issues come from.

We have made some improvements that are available as of langchain 0.3 and should speed up some common use cases in terms of performance.