[Enhancement]: Dynamic API Key Randomization and Failover Strategy for AI Providers

[chalitbkb]

What features would you like to see added?

Is it possible to support specifying multiple API keys for GEMINI or other AI providers by separating them with commas, such as "sk-1, sk-2, sk-3, ..."? The system would then randomly select one of the user-defined API keys each time the user interacts with the chatbot. If any of the listed keys are unusable, it would randomly choose another available key. This is because, in some of my personal tasks, I need to use multiple API keys, and some of them may not be functional, so I want to randomly select from the remaining available keys.

More details

Below is a summary of the proposal and requirements in bullet-point format:

• Specifying Multiple API Keys:
  Users can specify multiple API keys in a single string by separating them with commas, for example:
  sk-1, sk-2, sk-3, ...

• Random Selection of an API Key:
  Each time the user interacts with the chatbot, the system will randomly select one API key from the provided list.

• Fallback Mechanism for Unusable API Keys:

  • If the selected API key is unusable or encounters an error, the system will randomly select another key from the remaining available ones.
  • This mechanism ensures continuity even when one or more keys fail.

• Use Case for Personal Tasks:

  • Some users may need to use multiple API keys because certain keys might become non-functional.
  • This solution allows for seamless switching between keys to maintain uninterrupted service.

This approach increases the flexibility and reliability of the AI service, especially in scenarios where multiple API keys are used and some may fail during actual use.

Which components are impacted by your request?

No response

Pictures

No response

Code of Conduct

• I agree to follow this project's Code of Conduct