@@ -1752,3 +1752,141 @@ If set, it will override the `tools_strict` parameter set during component initi
17521752A dictionary with the following key:
17531753- `replies` : A list containing the generated responses as ChatMessage instances.
17541754
1755+ < a id = " chat/fallback" >< / a>
1756+
1757+ # Module chat/fallback
1758+
1759+ < a id = " chat/fallback.FallbackChatGenerator" >< / a>
1760+
1761+ # # FallbackChatGenerator
1762+
1763+ A chat generator wrapper that tries multiple chat generators sequentially.
1764+
1765+ It forwards all parameters transparently to the underlying chat generators and returns the first successful result.
1766+ Calls chat generators sequentially until one succeeds. Falls back on any exception raised by a generator.
1767+ If all chat generators fail, it raises a RuntimeError with details.
1768+
1769+ Timeout enforcement is fully delegated to the underlying chat generators. The fallback mechanism will only
1770+ work correctly if the underlying chat generators implement proper timeout handling and raise exceptions
1771+ when timeouts occur. For predictable latency guarantees, ensure your chat generators:
1772+ - Support a `timeout` parameter in their initialization
1773+ - Implement timeout as total wall- clock time (shared deadline for both streaming and non- streaming)
1774+ - Raise timeout exceptions (e.g., TimeoutError , asyncio.TimeoutError, httpx.TimeoutException) when exceeded
1775+
1776+ Note: Most well- implemented chat generators (OpenAI, Anthropic, Cohere, etc.) support timeout parameters
1777+ with consistent semantics. For HTTP - based LLM providers, a single timeout value (e.g., `timeout=30 ` )
1778+ typically applies to all connection phases: connection setup, read, write, and pool. For streaming
1779+ responses, read timeout is the maximum gap between chunks. For non- streaming, it' s the time limit for
1780+ receiving the complete response.
1781+
1782+ Failover is automatically triggered when a generator raises any exception, including:
1783+ - Timeout errors (if the generator implements and raises them)
1784+ - Rate limit errors (429 )
1785+ - Authentication errors (401 )
1786+ - Context length errors (400 )
1787+ - Server errors (500 + )
1788+ - Any other exception
1789+
1790+ < a id = " chat/fallback.FallbackChatGenerator.__init__" >< / a>
1791+
1792+ # ### FallbackChatGenerator.\_\_init\_\_
1793+
1794+ ```python
1795+ def __init__ (chat_generators: list[ChatGenerator])
1796+ ```
1797+
1798+ Creates an instance of FallbackChatGenerator.
1799+
1800+ ** Arguments** :
1801+
1802+ - `chat_generators` : A non- empty list of chat generator components to try in order.
1803+
1804+ < a id = " chat/fallback.FallbackChatGenerator.to_dict" >< / a>
1805+
1806+ # ### FallbackChatGenerator.to\_dict
1807+
1808+ ```python
1809+ def to_dict() -> dict[str , Any]
1810+ ```
1811+
1812+ Serialize the component, including nested chat generators when they support serialization.
1813+
1814+ < a id = " chat/fallback.FallbackChatGenerator.from_dict" >< / a>
1815+
1816+ # ### FallbackChatGenerator.from\_dict
1817+
1818+ ```python
1819+ @ classmethod
1820+ def from_dict(cls , data: dict[str , Any]) -> FallbackChatGenerator
1821+ ```
1822+
1823+ Rebuild the component from a serialized representation, restoring nested chat generators.
1824+
1825+ < a id = " chat/fallback.FallbackChatGenerator.run" >< / a>
1826+
1827+ # ### FallbackChatGenerator.run
1828+
1829+ ```python
1830+ @ component.output_types(replies = list[ChatMessage], meta = dict[str , Any])
1831+ def run(
1832+ messages: list[ChatMessage],
1833+ generation_kwargs: Union[dict[str , Any], None ] = None ,
1834+ tools: Union[list[Tool], Toolset, None ] = None ,
1835+ streaming_callback: Union[StreamingCallbackT,
1836+ None ] = None ) -> dict[str , Any]
1837+ ```
1838+
1839+ Execute chat generators sequentially until one succeeds.
1840+
1841+ ** Arguments** :
1842+
1843+ - `messages` : The conversation history as a list of ChatMessage instances.
1844+ - `generation_kwargs` : Optional parameters for the chat generator (e.g., temperature, max_tokens).
1845+ - `tools` : Optional Tool instances or Toolset for function calling capabilities.
1846+ - `streaming_callback` : Optional callable for handling streaming responses.
1847+
1848+ ** Raises** :
1849+
1850+ - `RuntimeError ` : If all chat generators fail.
1851+
1852+ ** Returns** :
1853+
1854+ A dictionary with :
1855+ - " replies" : Generated ChatMessage instances from the first successful generator.
1856+ - " meta" : Execution metadata including successful_chat_generator_index, successful_chat_generator_class,
1857+ total_attempts, failed_chat_generators, plus any metadata from the successful generator.
1858+
1859+ < a id = " chat/fallback.FallbackChatGenerator.run_async" >< / a>
1860+
1861+ # ### FallbackChatGenerator.run\_async
1862+
1863+ ```python
1864+ @ component.output_types(replies = list[ChatMessage], meta = dict[str , Any])
1865+ async def run_async(
1866+ messages: list[ChatMessage],
1867+ generation_kwargs: Union[dict[str , Any], None ] = None ,
1868+ tools: Union[list[Tool], Toolset, None ] = None ,
1869+ streaming_callback: Union[StreamingCallbackT,
1870+ None ] = None ) -> dict[str , Any]
1871+ ```
1872+
1873+ Asynchronously execute chat generators sequentially until one succeeds.
1874+
1875+ ** Arguments** :
1876+
1877+ - `messages` : The conversation history as a list of ChatMessage instances.
1878+ - `generation_kwargs` : Optional parameters for the chat generator (e.g., temperature, max_tokens).
1879+ - `tools` : Optional Tool instances or Toolset for function calling capabilities.
1880+ - `streaming_callback` : Optional callable for handling streaming responses.
1881+
1882+ ** Raises** :
1883+
1884+ - `RuntimeError ` : If all chat generators fail.
1885+
1886+ ** Returns** :
1887+
1888+ A dictionary with :
1889+ - " replies" : Generated ChatMessage instances from the first successful generator.
1890+ - " meta" : Execution metadata including successful_chat_generator_index, successful_chat_generator_class,
1891+ total_attempts, failed_chat_generators, plus any metadata from the successful generator.
1892+
0 commit comments