Refactor/plugin #14
Refactor/plugin #14
Conversation
CaddyGlow
force-pushed
the
refactor/plugin
branch
3 times, most recently
from
ecbeaa7 to
6eac0eb
Compare
WIP: quick save - 2025-08-28 21:02:18 WIP: quick save - 2025-08-28 21:18:00 WIP: quick save - 2025-08-28 22:42:38
…and deprecated code files from the codebase. The deleted files include old middleware, services, UI modules, validators, proxy implementations, and observability components that have been replaced by the plugin architecture. ``` refactor: remove deprecated modules and unused code - Remove obsolete API middleware, services, and UI modules - Delete legacy proxy and validator implementations - Clean up deprecated observability and logging components - Remove unused imports and commented code ``` WIP: quick save - 2025-08-28 23:20:30
- Create detailed plan for completing hooks system integration - Define migration strategy from observer pattern to hooks - Outline request_tracer plugin refactoring - Include testing and rollout strategy
context-based emission
WIP: quick save - 2025-08-28 21:02:18 WIP: quick save - 2025-08-28 21:18:00 WIP: quick save - 2025-08-28 22:42:38
- Remove obsolete API middleware, services, and UI modules - Delete legacy proxy and validator implementations - Clean up deprecated observability and logging components - Remove unused imports and commented code
…dule-level logger
…ce_container_startup
…shgateway to metrics plugin
- Replace conditional import with direct import in AdapterShim - Inherit from BaseAPIAdapter instead of LegacyBaseAPIAdapter - Add fallback handling for adapter failures in streaming - Create SimpleStreamingAdapter for direct dict-based streaming - Add model_dump conversion for Pydantic models in serialization - Fix newline at end of legacy_base.py
…or lightweight format conversion
… validation with stream pair checks
- Move imports to module level to avoid repeated imports - Add explicit type annotations and improve type safety - Simplify variable names and remove redundant casting - Standardize error handling patterns - Remove deprecated format parameter usage - Fix format chain decorator parameter alignment
|
@CaddyGlow are you no longer pursuing the plugin-based refactor? Edit: Nevermind, found the new branch :) |
|
@izzoa Yeah, it should be done very soon. I'm fixing the CLI, tests, and docs, but most of the code is done. I also added support for the Copilot API. You can already try it with the #dev/v0.2 branch. I just finished a good refactor of the formatter/adapter for conversion between APIs. I updated the CLI. I will push it to the #dev/v0.2 branch later. If everything works well, v0.2 should be ready for release. Let me know if you try it and run into any issues or need help. |
|
@CaddyGlow Nice, looking forward to it. One thing I noticed that might be useful (and am happy to help with my own contribution / making a PR...actually already working on it) is adjusting to dynamic model/token limit fetching from LiteLLM instead of hard-coding. Since you've already set it up to poll for pricing, some of the the infra is already there. But would help when providers change window sizes or release new models. Just a thought :) Keep up the great work. |
|
@izzoa I normally don't want to mess with the token limit—that’s usually the client's responsibility—but the model mapping idea could make sense. I could implement it so it’s set when the client doesn’t specify it in the request. During model mapping, if model A has a 60k max and we map to model B with a 200k max, we could bump it to 200k. Otherwise, that feels too invasive. In that case, we could add a config flag. |
|
@izzoa Oh yeah, don’t work too much on your code. The next push will probably introduce breaking changes if you touch anything related to the formatter or the unit tests. It’ll also include a plugin scaffold. If you want to implement it as a plugin, that should be pretty easy too. |
Refactor to plugin based architecture