Langserve RemoteRunnable Error

[conaku]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

I am invoking RemoteRunnable to remote langserve instance that hosts a chain_with_guardrail and facing error. When I call the invoke method on chain without guardrail then it works fine.

Server Side
===========

chain = (
        {"context": reranking_retriever, "question": RunnablePassthrough()}
        | prompt_template
        | llm
        | StrOutputParser()
    )

chain_with_guardrails = guardrails | chain

add_routes(
        app,
        chain_with_guardrails,
        path="/conversational_ai_with_rag_text_chat"
)

Client Side
============
llm = RemoteRunnable("http://127.0.0.1:90000") | StrOutputParser()
llm.invoke("Tell me something about Langchain")

Description

this invoke method is erroring out if the langserve chain is having "chain_with_guardrails" but not erroring with just chain

System Info

pip freeze|grep -i langchain
langchain==0.1.11
langchain-community==0.0.25
langchain-core==0.2.33
langchain-nvidia-ai-endpoints==0.1.4
langchain-text-splitters==0.2.2

[dosubot[bot]]

The invoke method errors out when using chain_with_guardrails in Langserve RemoteRunnable but not with just chain because chain_with_guardrails likely specifies an exception_key. When exception_key is specified, the input to the invoke method must be a dictionary. If the input is not a dictionary, a ValueError is raised.

Here is the relevant code snippet from the invoke method:

def invoke(
        self, input: Input, config: Optional[RunnableConfig] = None, **kwargs: Any
    ) -> Output:
        if self.exception_key is not None and not isinstance(input, dict):
            raise ValueError(
                "If 'exception_key' is specified then input must be a dictionary."
                f"However found a type of {type(input)} for input"
            )
        # setup callbacks
        config = ensure_config(config)
        callback_manager = get_callback_manager_for_config(config)
        # start the root run
        run_manager = callback_manager.on_chain_start(
            dumpd(self),
            input,
            name=config.get("run_name"),
            run_id=config.pop("run_id", None),
        )
        ...

In contrast, when using just chain, the exception_key is not specified, so this check is not performed, and the method does not raise a ValueError for non-dictionary inputs.

Additionally, the Chain class in LangChain framework expects inputs to be a dictionary, especially when multiple inputs are involved. If the input is not a dictionary, it attempts to convert it to one, but this conversion is only straightforward when there is a single input key. If there are multiple input keys, a non-dictionary input will result in a ValueError.

Here is the relevant code snippet from the Chain class:

def _validate_inputs(self, inputs: Dict[str, Any]) -> None:
    """Check that all inputs are present."""
    if not isinstance(inputs, dict):
        _input_keys = set(self.input_keys)
        if self.memory is not None:
            _input_keys = _input_keys.difference(self.memory.memory_variables)
        if len(_input_keys) != 1:
            raise ValueError(
                f"A single string input was passed in, but this chain expects "
                f"multiple inputs ({_input_keys}). When a chain expects "
                f"multiple inputs, please call it by passing in a dictionary, "
                "eg `chain({'foo': 1, 'bar': 2})`"
            )
    missing_keys = set(self.input_keys).difference(inputs)
    if missing_keys:
        raise ValueError(f"Missing some input keys: {missing_keys}")

Therefore, the error occurs because chain_with_guardrails enforces stricter input validation by requiring the input to be a dictionary when exception_key is specified, whereas chain does not have this requirement ^[1][2].

To continue talking to Dosu, mention @dosu.

[conaku]

Is there any example of how RemoteRunnable can call the invoke function with dictionary as input to remote chain_with_guardrails

[dosubot[bot]]

Yes, there is an example of how RemoteRunnable can call the invoke function with a dictionary to a remote chain_with_guardrails. Here is a relevant example:

from langchain_core.runnables import RunnableLambda

def add_one(x: int) -> int:
    return x + 1

def mul_two(x: int) -> int:
    return x * 2

def mul_five(x: int) -> int:
    return x * 5

sequence = RunnableLambda(add_one) | {
    'mul_2': RunnableLambda(mul_two),
    'mul_5': RunnableLambda(mul_five)
}

result = sequence.invoke(1)  # {'mul_2': 4, 'mul_5': 10}
print(result)

In this example, sequence is a RunnableSequence that contains a RunnableParallel constructed using a dictionary literal. The invoke function is called with an input value of 1, and the result is a dictionary with the outputs of the parallel runnables.

Additionally, the Chain class from the LangChain library provides a way to handle structured sequences of calls to components, which can be useful when dealing with remote invocations. Here is an example of how to use it:

from langchain_core.runnables import RunnableConfig
from langchain_core.chains.base import Chain

class MyChain(Chain):
    input_keys = ['input']
    output_keys = ['output']

    def _call(self, inputs, run_manager=None):
        # Your chain logic here
        return {'output': inputs['input'] * 2}

chain_with_guardrails = MyChain()

result = chain_with_guardrails.invoke({'input': 5})
print(result)  # {'output': 10}

In this example, chain_with_guardrails is a chain that doubles the input value. The invoke function is called with a dictionary input, and the result is a dictionary with the output ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[conaku]

I changed my code to

output = llm.invoke({'query': 'How easy is it to implement RAG?'})

Now, I am getting error on the remote server side

File "/stage/ashish/venv310/lib/python3.10/site-packages/nemoguardrails/rails/llm/utils.py", line 43, in get_history_cache_key
history_cache_key = ":".join(key_items)
TypeError: sequence item 2: expected str instance, NoneType found

I have not done any change on the server side code.