Skip to content

Conversation

mbutler-aws
Copy link

Description

This PR implements FallbackModel, a new model wrapper that provides automatic failover between any two model instances with circuit breaker protection.

Related Issues

N/A

Documentation PR

Not yet prepared.

Type of Change

New feature

Testing

Yes. PR includes both unit tests (29) with 91% code coverage, and integration tests (7):

Unit Tests

  • Fallback triggering on throttle and connection errors
  • Circuit breaker behavior (opening, closing, cooldown)
  • Configuration and statistics tracking
  • Streaming and structured output with fallback
  • Error classification and re-raising

Integration Tests

  • Same-provider fallback (Bedrock→Bedrock)

  • Cross-provider fallback (OpenAI→Bedrock, Anthropic→Bedrock)

  • Agent integration with FallbackModel

  • Tool calling with real models

  • Statistics tracking with real API calls

  • I ran hatch run prepare

Checklist

  • [X ] I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • [X ] My changes generate no new warnings
  • [ X] Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant