Refactor multiple flows to enhance performance, boost scalability, and ensure stability.#131
Refactor multiple flows to enhance performance, boost scalability, and ensure stability.#131luuquangvu wants to merge 275 commits intoNativu5:mainfrom
Conversation
…nd update response handling for consistency
…mat for compatibility
…ck to default size
…andling, and improved extension determination
- Introduced `model_strategy` configuration for "append" (default + custom models) or "overwrite" (custom models only). - Enhanced `/v1/models` endpoint to return models based on the configured strategy. - Improved model loading with environment variable overrides and validation. - Refactored model handling logic for improved modularity and error handling.
…eld support - Enhanced `extract_gemini_models_env` to handle nested fields within environment variables. - Updated type hints for more flexibility in model overrides. - Improved `_merge_models_with_env` to better support field-level updates and appending new models.
- Moved utility functions like `strip_code_fence`, `extract_tool_calls`, and `iter_stream_segments` to a centralized helper module. - Removed unused and redundant private methods from `chat.py`, including `_strip_code_fence`, `_strip_tagged_blocks`, and `_strip_system_hints`. - Updated imports and references across modules for consistency. - Simplified tool call and streaming logic by replacing inline implementations with shared helper functions.
- Replaced unused model placeholder in `config.yaml` with an empty list. - Added JSON parsing validators for `model_header` and `models` to enhance flexibility and error handling. - Improved validation to filter out incomplete model configurations.
…N support - Replaced prefix-based parsing with a root key approach. - Added JSON parsing to handle list-based model configurations. - Improved handling of errors and cleanup of environment variables.
…to Python literals - Added `ast.literal_eval` as a fallback for parsing environment variables when JSON decoding fails. - Improved error handling and logging for invalid configurations. - Ensured proper cleanup of environment variables post-parsing.
- Adjusted `TOOL_CALL_RE` regex pattern for better accuracy.
…nvironment variable setup
…nvironment variables; enhance error logging in config validation
…tring or list structure for enhanced flexibility in automated environments
…s found in either the raw or cleaned history.
… for better Gemini compatibility.
I don't know how you tested it, but it works fine for me. Note that to keep conversations running continuously through restarts, you need to enable Gemini Activities. |
|
Gemini activity is enabled, but I think it is using temporary chat (the best option to not saturate my chat history with all the users request) I think it should work regardless the original code worked |
|
The library's author is preparing to release a major 2.0 update. Therefore, this PR will also have to wait for it, as some features are being developed based on the latest code from the library. |
|
I think one improvement you could make is stripping unnecessary tokens before sending messages to the chat. For example, when using Open WebUI, I see messages formatted like this looking at them from the gemini web ui: As you can see, it includes all the special tags. However, these tags are already reintroduced by Gemini on the backend, since the message is sent as part of a web ui chat request. Removing them would have a few benefits:
Overall, stripping these redundant tokens seems like a simple optimization with multiple advantages, i write it here since opening a pull request for this really small feature seems a bit pointless and also i wanted a second opinion on the matter |
Optimize the codebase by applying Sourcery suggestions
… on exponential backoff
…r a certain period of time
…r a certain period of time
…r a certain period of time
2 participants
This PR is still a work in progress and uses features that aren't yet officially available in the Gemini-API library, so we'll need to wait for the library's official update before merging. Feel free to try it out and share any feedback or report any issues you encounter. Thanks!
Here are some highlights of the changes: