-
Notifications
You must be signed in to change notification settings - Fork 144
Closed
Description
Several providers (huggingface, sambanova, portkey) don't handle the reasoning tags when returning streaming and non-streaming responses. Because of this I created a utility file to parse out these tags, and then also needed to override the default parsing logic from the base openai behavior. This initially resulted in some copied code. This ticket is to track the evaluation of whether or not we should consolidate this logic to avoid copied code.
Options as I see it:
- Do nothing because the providers will likely eventually change their implementations to better handle reasoning content
- Implement the xml tag parsing into the baseopenai class itself, so that they don't need to override any behavior (I don't love this option)
- Make a new child class from the BaseOpenAI class called something like XMLParsingBaseOpenAI class, which the providers can inherit from, and this new class will have all the necessary override logic
- Something else
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels