fix: add NVIDIA API compatibility for minimax-M2 model #9001
Conversation
- Added _isNvidiaApi() method to detect NVIDIA API endpoints - Excluded stream_options parameter for NVIDIA endpoints (similar to Grok XAI) - Fixes issue #8998 where NVIDIA API returns body errors with stream_options
- Changed from broad .includes("nvidia.com") to specific hostnames
- Now checks for integrate.api.nvidia.com, build.nvidia.com, or *.api.nvidia.com
- Follows review feedback for more precise endpoint detection
dosubot
bot
added
size:S
|
Review completed. The implementation correctly adds NVIDIA API compatibility by excluding unsupported parameters, following the same pattern as the existing Grok xAI workaround. One issue identified:
Mention @roomote in a comment to trigger your PR Fixer agent and make changes to this pull request. |
![roomote[bot]](https://avatars.githubusercontent.com/in/1546624?s=60&v=4){kind=small}
| private _isNvidiaApi(baseUrl?: string): boolean { | ||
| const urlHost = this._getUrlHost(baseUrl) | ||
| // NVIDIA API endpoints for AI models | ||
| return ( | ||
| urlHost === "integrate.api.nvidia.com" || | ||
| urlHost === "build.nvidia.com" || | ||
| urlHost.endsWith(".api.nvidia.com") | ||
| ) | ||
| } |
There was a problem hiding this comment.
The new NVIDIA API detection functionality lacks test coverage. Similar to the existing Grok xAI tests (lines 527-565 in the test file), this change should include tests that verify: (1) _isNvidiaApi() correctly identifies NVIDIA endpoints, (2) stream_options is excluded when using NVIDIA API in both streaming and non-streaming modes, and (3) the behavior works correctly with O3 family models. Without tests, future changes could inadvertently break this compatibility fix.
hannesrudolph
added
the
Issue/PR - Triage
Fixes #8998
Problem
Users were unable to use the minimax-M2 model through NVIDIA API endpoints (integrate.api.nvidia.com) when using the OpenAI Compatible provider. The API was returning body errors when attempting to stream responses.
Root Cause
The OpenAI handler was sending
stream_options: { include_usage: true }parameter with streaming requests, which is not supported by NVIDIA API endpoints.Solution
_isNvidiaApi()method to detect NVIDIA API endpointsstream_optionsparameter for NVIDIA endpoints (similar to existing Grok XAI workaround)Testing
Additional Context
The user provided a Python example showing they were using
https://integrate.api.nvidia.com/v1as the base URL with the minimax-M2 model. This fix ensures compatibility by not sending unsupported parameters to NVIDIA endpoints.Important
Adds NVIDIA API compatibility to
OpenAiHandlerby excluding unsupportedstream_optionsin requests.stream_optionsparameter for NVIDIA API endpoints inOpenAiHandler._isNvidiaApi()method to detect NVIDIA API endpoints._isNvidiaApi()checks forintegrate.api.nvidia.com,build.nvidia.com, and*.api.nvidia.com.createMessage()andhandleO3FamilyMessage()to excludestream_optionsfor NVIDIA APIs.This description was created by
for ff2ec78. You can customize this summary. It will automatically update as commits are pushed.