Router Chain to Route Requests to API Chain

[mukunds97]

I have a requirement where in I need to route the user input to one of the 2 chains. These destination chains are API chains as I need these chains to post a request and then retrieve the appropriate answer from an API.
Currently I have made use of a router chain with destination chains being the API chains, I would like to get suggestions if this is the best approach and understand if there are any limitations which should be taken care of?
Would like to know if there are any other alternatives.

[dosubot[bot]]

🤖

Your approach of using a router chain with destination chains as API chains is indeed a valid one for routing user input to one of the two chains. This method allows you to define a routing logic in the router chain and then direct the input to the appropriate destination chain based on the result of the routing logic.

However, there are a few limitations to consider:

1. If the router chain returns a destination name that does not exist in the destination chains, an error will be raised. This can be mitigated by setting silent_errors to True, which will cause the default chain to be used when an invalid destination name is provided.

2. The router chain and destination chains need to be carefully designed to ensure that the inputs and outputs match up correctly. If the router chain outputs a set of inputs that the destination chain cannot handle, an error will occur.

3. This approach requires that the routing logic be defined in the router chain. If the routing logic needs to be changed, the router chain will need to be modified.

In addition to these, the router chain in LangChain also provides methods for batch processing (both synchronous and asynchronous), which can be useful if you need to process multiple inputs at once.

As for alternative methods, it really depends on the specific requirements of your application. If the routing logic is simple, you could potentially use a simple if-else statement to route the input to the appropriate chain. If the routing logic is more complex, you might consider using a decision tree or a rule-based system.

The router chain in LangChain, specifically the EmbeddingRouterChain, handles the routing of user input to the destination chains by using embeddings to route between options. It has a vectorstore attribute and routing_keys attribute which defaults to ["query"].

In the _call method, it takes the inputs and an optional run_manager. It joins the inputs based on the routing_keys and performs a similarity search on the vectorstore with the joined inputs. The result of the similarity search, specifically the name in the metadata of the first result, is used as the destination.

There is also a convenience constructor from_names_and_descriptions that takes a sequence of names and descriptions, a vectorstore_cls, embeddings, and any additional arguments. It creates a list of Document objects from the names and descriptions, creates a vectorstore from the documents and the embeddings, and returns an instance of EmbeddingRouterChain with the created vectorstore.

You can find more details in the source code:

• libs/langchain/langchain/chains/router/base.py
• libs/langchain/langchain/schema/runnable/router.py
• libs/langchain/langchain/chains/router/embedding_router.py

As for the silent_errors attribute, I wasn't able to find specific information within the repository. It might be beneficial to provide more context or details about where you encountered this attribute.

Sources

• libs/langchain/langchain/chains/router/base.py
• libs/langchain/langchain/schema/runnable/router.py
• libs/langchain/langchain/chains/router/embedding_router.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[mukunds97]

@dosu-bot
For the default chain, I want the router chain to first get the answer by sending API request to chain2. If chain2 does not provide appropriate answer then I need to provide the answer from chain1.
How can this be implemented?

[dosubot[bot]]

🤖

To implement a fallback mechanism in the LangChain Python framework that first sends an API request to chain2 and then to chain1 if chain2 does not provide an appropriate answer, you can use the RunnableWithFallbacks class. This class allows you to define a sequence of fallbacks that will be tried in order until one succeeds or all fail.

Here is an example of how you can use it:

from langchain.schema.runnable.fallbacks import RunnableWithFallbacks

# Assuming chain1 and chain2 are instances of Runnable
fallback_chain = RunnableWithFallbacks(runnable=chain2, fallbacks=[chain1])

# Now you can use fallback_chain just like any other Runnable
# If chain2 fails, it will automatically fallback to chain1
result = fallback_chain.invoke('your input')

In this example, chain2 is the runnable to run first and chain1 is the fallback. If chain2 fails to provide an appropriate answer, the framework will automatically try chain1.

This is supported by the invoke method in the RunnableWithFallbacks class, which iterates over the runnables (the initial runnable and the fallbacks) and tries to invoke them. If an exception that is a subclass of exceptions_to_handle is raised during the invocation of a runnable, it will try the next one. If all runnables fail, it will raise the first exception that was caught. You can find more details in the source code.

The LangChain framework considers a chain to have "failed" under the following conditions:

1. If there are missing keys in the input or output. This is checked by the _validate_inputs and _validate_outputs methods. If any of the keys specified in input_keys or output_keys are not present in the input or output dictionaries, a ValueError is raised with a message indicating the missing keys.

2. If an exception is raised during the execution of the chain. This is handled in the __call__ and acall methods. If an exception is raised during the execution of the _call or _acall methods, the exception is caught and the on_chain_error method of the run_manager is called with the exception as an argument. The exception is then re-raised.

The fallback mechanism is triggered by the on_chain_error method of the run_manager, which is called when an exception is caught during the execution of the chain. The specific behavior of the fallback mechanism would depend on the implementation of the on_chain_error method in the CallbackManager or AsyncCallbackManager used. You can find more details in the source code.

I hope this helps! If you have any more questions, feel free to ask.

Sources

• [Feature Request] Fallback from one provider to another
• libs/langchain/langchain/schema/runnable/fallbacks.py
• libs/langchain/langchain/chains/base.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[mukunds97]

@dosu_bot How can this be achieved using the Legacy method and if using the current method, how can I specify the condition myself?

[dosubot[bot]]

🤖

To implement a routing mechanism in the LangChain Python framework using the Legacy method, you can use the MultiPromptChain class. This class allows you to create a chain that dynamically selects the next chain to use for a given input.

First, you need to define your destination chains. These are the chains that the router chain can route to. In your case, you want user input to be first sent to chain2 and if an appropriate answer is not provided, it falls back to chain1.

Here is an example of how you can define your destination chains:

from langchain.chains.llm import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate

llm = OpenAI()

chain1_prompt = PromptTemplate.from_template(chain1_template)
chain1 = LLMChain(llm=llm, prompt=chain1_prompt)

chain2_prompt = PromptTemplate.from_template(chain2_template)
chain2 = LLMChain(llm=llm, prompt=chain2_prompt)

destination_chains = {"chain1": chain1, "chain2": chain2}

Next, you need to define your router chain. This is the chain that is responsible for selecting the next chain to call. You can use the LLMRouterChain class to create a router chain that uses an LLM to determine how to route things. Here is an example:

from langchain.chains.router.llm_router import LLMRouterChain, RouterOutputParser
from langchain.chains.router.multi_prompt_prompt import MULTI_PROMPT_ROUTER_TEMPLATE

router_template = MULTI_PROMPT_ROUTER_TEMPLATE.format(destinations="chain1: description1\nchain2: description2")
router_prompt = PromptTemplate(template=router_template, input_variables=["input"], output_parser=RouterOutputParser())
router_chain = LLMRouterChain.from_llm(llm, router_prompt)

Finally, you can create your MultiPromptChain:

from langchain.chains.router import MultiPromptChain
from langchain.chains import ConversationChain

default_chain = ConversationChain(llm=llm, output_key="text")

chain = MultiPromptChain(
    router_chain=router_chain,
    destination_chains=destination_chains,
    default_chain=default_chain,
    verbose=True,
)

To specify the condition for an appropriate answer when using the RunnableWithFallbacks class, you can define a function that takes the output of a chain as input and returns a boolean value. If the function returns True, the output is considered appropriate and the chain stops. If the function returns False, the chain falls back to the next chain. Here is an example:

def is_appropriate(output):
    # Define your condition here
    return True

chain_with_fallbacks = RunnableWithFallbacks(chain2, chain1, is_appropriate)

Please note that the RunnableWithFallbacks class and the is_appropriate function are not defined in the provided context. You may need to define them yourself or use similar functionality provided by the LangChain Python framework.

For more information, you can refer to the LangChain documentation.

Sources

• docs/docs/modules/chains/foundational/router.ipynb
• libs/langchain/langchain/chains/router/multi_prompt.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[Remya-Ramachandran]

I have a requirement where in I need to route the user input to one of the 2 chains. These destination chains are API chains as I need these chains to post a request and then retrieve the appropriate answer from an API. Currently I have made use of a router chain with destination chains being the API chains, I would like to get suggestions if this is the best approach and understand if there are any limitations which should be taken care of? Would like to know if there are any other alternatives.

Hi, @mukunds97 , could you please mention how did you generate the API documentation.

[amitMialo]

Hi @mukunds97 I have two chains one is RAG Chain and other is API Chain. I am facing TypeError: unhashable type: APIChain while putting both chains in destination chains. Were you able to solve this issue or you are using some different mechanism to route instead of using MultiPromptChain?

[joaograndotto]

Hi @mukunds97, could you mention the example code?