RunnableParallel: run chain for each element in a list in parralel

[VolodymyrGnennyi]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain_core.prompts import PromptTemplate
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables.base import RunnableSequence
from langchain_core.runnables import RunnableParallel

sentenses = ["The USA is the fourth largest country in the world in area", "CA, constituent state of the United States of America."]
prompt = PromptTemplate(template="Rephrase input sentense resolving abbreviations. {sentence}")
llm = ChatOpenAI(temperature=0, model="gpt-4o")
chain: RunnableSequence = prompt | llm | StrOutputParser()

# create a map for chains with different invoke input parameter
runnable_map = { key: chain.bind(sentence=s) for s, key in enumerate(sentenses) }

paralel_chain = RunnableParallel(runnable_map)
paralel_chain.invoke()

Description

I'm trying to run a Runnable for each element in a list. My example contains list of sentenses with abbreviations (e.g. USA, CA). I would like LLM to rephrase each sentense to get full meaning for each abbreviation.
I would like to run this in parallel. So I created a chain definition. Then I created a map { key : chain_with_invokation_parameter } which is required argument for RunnableParallel

Unfortunately, when I run code it complains: "RunnableParallel.invoke() missing 1 required positional argument: 'input'"
If I modify list line of the code as the following:
paralel_chain.invoke({'input': 'bla-bla-bla'})
then I get error as the following: "TypeError: BasePromptTemplate.invoke() got an unexpected keyword argument 'sentence'"

System Info

System Information

OS: Windows
OS Version: 10.0.22631
Python Version: 3.11.4 (tags/v3.11.4:d2340ef, Jun 7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)]

Package Information

langchain_core: 0.2.23
langchain: 0.2.11
langchain_community: 0.2.10
langsmith: 0.1.93
langchain_openai: 0.1.17
langchain_postgres: 0.0.9
langchain_text_splitters: 0.2.2
langchainhub: 0.1.15
langgraph: 0.1.4

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langserve

[dosubot[bot]]

Hey there, @VolodymyrGnennyi! I'm a bot designed to help you with bugs, questions, and becoming a contributor. Let's squash this bug together! 🤖

To run a RunnableParallel for each element in a list and resolve the error "RunnableParallel.invoke() missing 1 required positional argument: 'input'", you need to ensure that you are passing the required input argument to the invoke method. Here is an example of how you can do this:

from langchain_core.prompts import PromptTemplate
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables.base import RunnableSequence
from langchain_core.runnables import RunnableParallel

sentences = ["The USA is the fourth largest country in the world in area", "CA, constituent state of the United States of America."]
prompt = PromptTemplate(template="Rephrase input sentence resolving abbreviations. {sentence}")
llm = ChatOpenAI(temperature=0, model="gpt-4o")
chain = prompt | llm | StrOutputParser()

# Create a list of chains with different input parameters
runnable_list = [chain.bind(sentence=s) for s in sentences]

parallel_chain = RunnableParallel(runnable_list)
results = parallel_chain.batch([{} for _ in sentences])

print(results)

In this example, RunnableParallel is created with a list of chain instances, each bound to a different sentence. The batch method is then used to run the parallel_chain for each element in the sentences list. This should resolve the error you encountered ^[1].

To continue talking to Dosu, mention @dosu.

[dosubot[bot]]

The error "TypeError: BasePromptTemplate.invoke() got an unexpected keyword argument 'sentense'" indicates that there is a mismatch between the keyword argument you are using and the expected parameter in your PromptTemplate. It seems like there is a typo in your template. The correct keyword should be sentence instead of sentense.

Here's the corrected version of your code:

from langchain_core.prompts import PromptTemplate
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables.base import RunnableSequence
from langchain_core.runnables import RunnableParallel

sentences = ["The USA is the fourth largest country in the world in area", "CA, constituent state of the United States of America."]
prompt = PromptTemplate(template="Rephrase input sentence resolving abbreviations. {sentence}")
llm = ChatOpenAI(temperature=0, model="gpt-4o")
chain = prompt | llm | StrOutputParser()

# Create a dictionary of chains with different input parameters
runnable_map = {f"chain_{i}": chain.bind(sentence=s) for i, s in enumerate(sentences)}

parallel_chain = RunnableParallel(runnable_map)
results = parallel_chain.invoke({})

print(results)

Make sure that the keyword argument sentence matches the one defined in your PromptTemplate ^[1].

To continue talking to Dosu, mention @dosu.

[VolodymyrGnennyi]

@dosu, I paid attention to the differense of "sentence" and "sentense" from the bigining. But I double-checked, even now it still complains: BasePromptTemplate.invoke() got an unexpected keyword argument 'sentence'

[dosubot[bot]]

The error "BasePromptTemplate.invoke() got an unexpected keyword argument 'sentence'" occurs because the PromptTemplate you defined does not include 'sentence' as one of its input_variables.

Here's the corrected version of your code:

1. Ensure that the PromptTemplate includes 'sentence' as an input variable.
2. Pass the correct input when invoking the RunnableParallel.

Here's the updated code:

from langchain_core.prompts import PromptTemplate
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables.base import RunnableSequence
from langchain_core.runnables import RunnableParallel

sentences = ["The USA is the fourth largest country in the world in area", "CA, constituent state of the United States of America."]
prompt = PromptTemplate(template="Rephrase input sentence resolving abbreviations. {sentence}", input_variables=["sentence"])
llm = ChatOpenAI(temperature=0, model="gpt-4o")
chain = prompt | llm | StrOutputParser()

# Create a dictionary of chains with different input parameters
runnable_map = {f"chain_{i}": chain.bind(sentence=s) for i, s in enumerate(sentences)}

parallel_chain = RunnableParallel(runnable_map)
results = parallel_chain.invoke({})

print(results)

Make sure that the input_variables in the PromptTemplate include 'sentence' ^[1]. This should resolve the error you are encountering.

To continue talking to Dosu, mention @dosu.

[VolodymyrGnennyi]

@dosu, I tested that approach as well - still same issue.

[dosubot[bot]]

To resolve the "BasePromptTemplate.invoke() got an unexpected keyword argument 'sentence'" error, you need to ensure that the keyword arguments you pass to the invoke method match the expected parameters of the PromptTemplate.

Here's a revised version of your code that should work:

1. Ensure the PromptTemplate includes 'sentence' as an input variable.
2. Use the correct input when invoking the RunnableParallel.

from langchain_core.prompts import PromptTemplate
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables.base import RunnableSequence
from langchain_core.runnables import RunnableParallel

sentences = ["The USA is the fourth largest country in the world in area", "CA, constituent state of the United States of America."]
prompt = PromptTemplate(template="Rephrase input sentence resolving abbreviations. {sentence}", input_variables=["sentence"])
llm = ChatOpenAI(temperature=0, model="gpt-4o")
chain = prompt | llm | StrOutputParser()

# Create a dictionary of chains with different input parameters
runnable_map = {f"chain_{i}": chain.bind(sentence=s) for i, s in enumerate(sentences)}

parallel_chain = RunnableParallel(runnable_map)
results = parallel_chain.invoke({})

print(results)

Ensure that the input_variables in the PromptTemplate include 'sentence' and that the input data matches the expected parameters of the PromptTemplate ^[1].

This is a tough nut to crack, could really use your help on this, @eyurtsev.

To continue talking to Dosu, mention @dosu.

[VolodymyrGnennyi]

@eyurtsev , in addition to above here is a full error message and it looks like a bug somewhere in runnable:

[ronaldarifin]

from langchain_core.prompts import PromptTemplate
from langchain_openai import ChatOpenAI
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables.base import RunnableSequence
from langchain_core.runnables import RunnableParallel

sentences = ["The USA is the fourth largest country in the world in area", "CA, constituent state of the United States of America."]
prompt = PromptTemplate(template="Rephrase input sentence resolving abbreviations. {sentence}", input_variables=["sentence"])
llm = ChatOpenAI(temperature=0, model="gpt-4o")
chain = prompt | llm | StrOutputParser()

Create a dictionary of chains with different input parameters

runnable_map = {}
for i, s in enumerate(sentences):
prompt = PromptTemplate(template=f"Rephrase input sentence resolving abbreviations. {s}")
llm = ChatOpenAI(temperature=0, model="gpt-4o")
chain = prompt | llm | StrOutputParser()
runnable_map[f"a_{i}"] = chain

parallel_chain = RunnableParallel(runnable_map)
results = parallel_chain.invoke({})

print(results)

This should work

[ronaldarifin]

Got the same problem....