Issue with Serializing and Deserializing LLM Chain vs Conversational Retrieval Chain in langchain_core.load.loads

[KalyaniBogala]

I'm encountering an issue while serializing and deserializing two chains: one is a standard LLM chain, and the other is a conversational retrieval chain. When I use langchain_core.load.loads to serialize the normal LLM chain, the object is of type "constructor" and works as expected. However, when I try to serialize the conversational retrieval chain using the same method, the object is of type "NotImplemented". This prevents me from deserializing the conversational chain after serialization. Could you provide insight into why this discrepancy occurs and how to resolve it?

conversational retrieval chain after serialization.
{
"lc": 1,
"type": "not_implemented",
"id": [
"langchain",
"chains",
"conversational_retrieval",
"base",
"ConversationalRetrievalChain"
]

Normal llm chain after serialization.
{
"lc": 1,
"type": "constructor",
"id": [
"langchain",
"schema",
"runnable",
"RunnableSequence"
]

error encountering when try to deserialize conversational retrieval chain :

NotImplementedError: Trying to load an object that doesn't implement serialization: {'lc': 1, 'type': 'not_implemented', 'id': ['langchain', 'chains', 'conversational_retrieval', 'base', 'ConversationalRetrievalChain']

[dosubot[bot]]

The discrepancy occurs because the ConversationalRetrievalChain class is not marked as serializable by default. In the LangChain framework, the Serializable base class has a method is_lc_serializable that returns False by default. This method determines whether a class is serializable. If a class does not override this method to return True, the to_json method will return a SerializedNotImplemented object instead of a SerializedConstructor object.

To resolve this issue, you need to ensure that the ConversationalRetrievalChain class overrides the is_lc_serializable method to return True. Here is how you can do it:

class ConversationalRetrievalChain(Serializable):
    @classmethod
    def is_lc_serializable(cls) -> bool:
        """Return whether this model can be serialized by Langchain."""
        return True

    # Your existing implementation of the class

By overriding the is_lc_serializable method to return True, you indicate that instances of ConversationalRetrievalChain are serializable within the LangChain framework, thus resolving the issue of the chain being serialized as a SerializedNotImplemented object.

For more details, you can refer to the implementation of the is_lc_serializable method in the ConversationalRetrievalChain class ^[1].

To continue talking to Dosu, mention @dosu.

[KalyaniBogala]

I am using conversational buffer memory inside the conversational retrieval chain. After following the previous suggestion, I am now encountering a "not implemented" error for conversational buffer memory.

[dosubot[bot]]

You are encountering a "not implemented" error for conversational buffer memory because the ConversationBufferMemory class does not have an implementation for the save_context method, which is likely being called by the conversational retrieval chain.

To resolve this, you need to implement the save_context method in the ConversationBufferMemory class. Here is an example implementation:

class ConversationBufferMemory(BaseChatMemory):
    """Buffer for storing conversation memory."""

    # ... other methods ...

    def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None:
        """Save context from this conversation to buffer."""
        if self.input_key is None:
            prompt_input_key = get_prompt_input_key(inputs, self.memory_variables)
        else:
            prompt_input_key = self.input_key
        if self.output_key is None:
            if len(outputs) != 1:
                raise ValueError(f"One output key expected, got {outputs.keys()}")
            output_key = list(outputs.keys())[0]
        else:
            output_key = self.output_key
        human = f"{self.human_prefix}: " + inputs[prompt_input_key]
        ai = f"{self.ai_prefix}: " + outputs[output_key]
        self.chat_memory.messages.append(human)
        self.chat_memory.messages.append(ai)

    async def asave_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None:
        """Save context from this conversation to buffer."""
        return self.save_context(inputs, outputs)

Alternatively, you can consider using the ConversationVectorStoreTokenBufferMemory class, which already has the save_context method implemented along with additional features like token limit and vector store backing ^[1][2].

To continue talking to Dosu, mention @dosu.

[KalyaniBogala]

hello @dosu Even though I have changed the above method in the ConversationBufferMemory, I am still encountering the same error.

class CustomConversationalRetrievalChain(ConversationalRetrievalChain, Serializable):
    @classmethod
    def is_lc_serializable(cls) -> bool:
        """Return whether this model can be serialized by Langchain."""
        return True

    
class CustomConversationBufferMemory(ConversationBufferMemory, BaseChatMemory):
    """Buffer for storing conversation memory."""

    def save_context(self, inputs: dict[str, any], outputs: dict[str, str]) -> None:
        """Save context from this conversation to buffer."""
        if self.input_key is None:
            prompt_input_key = get_prompt_input_key(inputs, self.memory_variables)
        else:
            prompt_input_key = self.input_key
        if self.output_key is None:
            if len(outputs) != 1:
                raise ValueError(f"One output key expected, got {outputs.keys()}")
            output_key = list(outputs.keys())[0]
        else:
            output_key = self.output_key
        human = f"{self.human_prefix}: " + inputs[prompt_input_key]
        ai = f"{self.ai_prefix}: " + outputs[output_key]
        self.chat_memory.messages.append(human)
        self.chat_memory.messages.append(ai)

    async def asave_context(self, inputs: dict[str, any], outputs: dict[str, str]) -> None:
        """Save context from this conversation to buffer."""
        return self.save_context(inputs, outputs)

ERROR: NotImplementedError: Trying to load an object that doesn't implement serialization: {'lc': 1, 'type': 'not_implemented', 'id': ['main', 'CustomConversationBufferMemory'], 'repr': "CustomConversationBufferMemory(return_messages=True, memory_key='chat_history')"}

[dosubot[bot]]

The error persists because the CustomConversationBufferMemory class also needs to be marked as serializable. You need to override the is_lc_serializable method in the CustomConversationBufferMemory class as well. Here is how you can do it:

class CustomConversationBufferMemory(ConversationBufferMemory, BaseChatMemory, Serializable):
    """Buffer for storing conversation memory."""

    @classmethod
    def is_lc_serializable(cls) -> bool:
        """Return whether this model can be serialized by Langchain."""
        return True

    def save_context(self, inputs: dict[str, any], outputs: dict[str, str]) -> None:
        """Save context from this conversation to buffer."""
        if self.input_key is None:
            prompt_input_key = get_prompt_input_key(inputs, self.memory_variables)
        else:
            prompt_input_key = self.input_key
        if self.output_key is None:
            if len(outputs) != 1:
                raise ValueError(f"One output key expected, got {outputs.keys()}")
            output_key = list(outputs.keys())[0]
        else:
            output_key = self.output_key
        human = f"{self.human_prefix}: " + inputs[prompt_input_key]
        ai = f"{self.ai_prefix}: " + outputs[output_key]
        self.chat_memory.messages.append(human)
        self.chat_memory.messages.append(ai)

    async def asave_context(self, inputs: dict[str, any], outputs: dict[str, str]) -> None:
        """Save context from this conversation to buffer."""
        return self.save_context(inputs, outputs)

By adding the is_lc_serializable method to the CustomConversationBufferMemory class, you indicate that this class is also serializable within the LangChain framework. This should resolve the "not implemented" error you are encountering ^[1].

To continue talking to Dosu, mention @dosu.