feat(core): Model Fallback Support for BaseChatModel

[keenborder786]

• Description: This is a major change, where I’ve added fallback support for BaseChatModel. The purpose of this change is to ensure that users can use BaseChatModel with fallbacks in middleware or react agents in LangChain v1.
• Issue: create_agent() fails with RunnableWithFallbacks when using middleware #33129
• Usage will be something like this

from langchain.agents import create_agent
from langchain.agents.middleware import SummarizationMiddleware
from langchain_openai import ChatOpenAI
from langchain.tools import tool

# Setup models
main_model = ChatOpenAI(model="gpt-4")
fallback_model = ChatOpenAI(model="gpt-3.5-turbo")
fallback_model = main_model.with_fallbacks([fallback_model])
# Create a model with fallback type - this returns BaseChatModel
model_with_fallback = main_model.set_fallback(fallback_model)

# Setup middleware for conversation summarization
middleware = [
    SummarizationMiddleware(
        model=main_model,
        max_tokens_before_summary=10000,
        messages_to_keep=20
    )
]

# Define a simple tool for the agent
@tool
def get_weather(city: str) -> str:
    """Get weather information for a city."""
    return f"The weather in {city} is sunny."

# This fails with AssertionError
agent = create_agent(
    model=model_with_fallback,  # RunnableWithFallbacks type
    prompt="You are a helpful assistant that can provide weather information.",
    tools=[get_weather],
    middleware=middleware  # When middleware exists, assertion fails
)

# # Run the agent
result = agent.invoke({"input": "What's the weather in New York?"})
print(result)

[keenborder786]

@sydney-runkle @eyurtsev please review, and let me know if I need to make any changes.

[codspeed-hq]

CodSpeed Performance Report

Merging #33281 will not alter performance

_{Comparing keenborder786:fix/fallback_base_chat_model (a171d9f) with master (010ed5d)¹}

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

✅ 13 untouched
⏩ 21 skipped²

Footnotes

1. No successful run was found on master (a9aa3f2) during the generation of this report, so 010ed5d was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

2. 21 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[eyurtsev]

@keenborder786 thanks for the PR, we're going to handle fallbacks through middleware