Advanced Retry and Fallbacks Mechanisms #9770
Unanswered
winningcode
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Lets say there are 10 deployments for a model with different regions . First two have been given preference using weight and order .
Now if there is an issue with first two deployments , Retry is trying with first two deployments only due to weight and order set.
How to fallback to rest of the 8 healthy deployments ? since model name is same we can not provide the fallbacks in Router instance . is there any way to setup fallback to use rest of the 8 deployments ?
Also Will the retry mechanisms exclude the failed deployment ? I don't think so
Beta Was this translation helpful? Give feedback.
All reactions