Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions src/api/providers/__tests__/openai.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,55 @@ describe("OpenAiHandler", () => {
const callArgs = mockCreate.mock.calls[0][0]
expect(callArgs.temperature).toBe(0.6)
})

it("should detect DeepSeek V3 models with reasoning effort as reasoning models", async () => {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests look good, but what about edge cases like "DeepSeek-V3" (uppercase) or "deepseek-v3.1"? Should we add tests to ensure the case-insensitive matching works correctly for all variations?

const deepseekV3Options: ApiHandlerOptions = {
...mockOptions,
openAiModelId: "deepseek-v3",
openAiCustomModelInfo: {
...openAiModelInfoSaneDefaults,
supportsReasoningEffort: true,
},
reasoningEffort: "medium",
}
const deepseekHandler = new OpenAiHandler(deepseekV3Options)
const stream = deepseekHandler.createMessage(systemPrompt, messages)
for await (const _chunk of stream) {
// consume stream
}
// Assert the mockCreate was called with R1 format messages
expect(mockCreate).toHaveBeenCalled()
const callArgs = mockCreate.mock.calls[0][0]
// When DeepSeek is detected as a reasoning model, it uses R1 format
// which combines system and user messages
expect(callArgs.messages[0].role).toBe("user")
expect(callArgs.messages[0].content).toContain("You are a helpful assistant.")
expect(callArgs.reasoning_effort).toBe("medium")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also verify that there's no separate system message in the converted messages? This would make the R1 format usage more explicit:

})

it("should detect DeepSeek-chat models with reasoning effort as reasoning models", async () => {
const deepseekChatOptions: ApiHandlerOptions = {
...mockOptions,
openAiModelId: "deepseek-chat",
openAiCustomModelInfo: {
...openAiModelInfoSaneDefaults,
supportsReasoningEffort: true,
},
reasoningEffort: "high",
}
const deepseekHandler = new OpenAiHandler(deepseekChatOptions)
const stream = deepseekHandler.createMessage(systemPrompt, messages)
for await (const _chunk of stream) {
// consume stream
}
// Assert the mockCreate was called with R1 format messages
expect(mockCreate).toHaveBeenCalled()
const callArgs = mockCreate.mock.calls[0][0]
// When DeepSeek is detected as a reasoning model, it uses R1 format
expect(callArgs.messages[0].role).toBe("user")
expect(callArgs.messages[0].content).toContain("You are a helpful assistant.")
expect(callArgs.reasoning_effort).toBe("high")
})
})

describe("error handling", () => {
Expand Down
7 changes: 6 additions & 1 deletion src/api/providers/openai.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,12 @@ export class OpenAiHandler extends BaseProvider implements SingleCompletionHandl
const enabledR1Format = this.options.openAiR1FormatEnabled ?? false
const enabledLegacyFormat = this.options.openAiLegacyFormat ?? false
const isAzureAiInference = this._isAzureAiInference(modelUrl)
const deepseekReasoner = modelId.includes("deepseek-reasoner") || enabledR1Format
// Check if this is a DeepSeek model with reasoning enabled
const isDeepSeekWithReasoning =
(modelId.toLowerCase().includes("deepseek") && reasoning) ||
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this intentional to keep the detection logic inline? Consider extracting modelId.toLowerCase().includes("deepseek") && reasoning into a helper method like isDeepSeekModelWithReasoning() for better readability and potential reuse.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue comment mentions GLM-4.5 has the same problem. Should we consider adding similar detection for GLM models in this PR, or would that be better as a separate fix to keep this PR focused?

modelId.includes("deepseek-reasoner") ||
enabledR1Format
const deepseekReasoner = isDeepSeekWithReasoning
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variable naming here could be clearer. We have isDeepSeekWithReasoning and deepseekReasoner which seem redundant. Could we simplify to use just one consistent variable name?

const ark = modelUrl.includes(".volces.com")

if (modelId.includes("o1") || modelId.includes("o3") || modelId.includes("o4")) {
Expand Down
Loading