Skip to content

fix: append /v1 to OpenAI Compatible base URLs for llama.cpp compatibility #7066

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
+52 −1
Closed

fix: append /v1 to OpenAI Compatible base URLs for llama.cpp compatibility #7066

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions src/api/providers/__tests__/openai-native.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,41 @@ describe("OpenAiNativeHandler", () => {
})
expect(handlerWithoutKey).toBeInstanceOf(OpenAiNativeHandler)
})

it("should append /v1 to base URL when it's a plain URL without path", () => {
// Test with plain URL (like llama.cpp server)
const handler1 = new OpenAiNativeHandler({
apiModelId: "gpt-4.1",
openAiNativeApiKey: "test-api-key",
openAiNativeBaseUrl: "http://127.0.0.1:8080",
})
expect(handler1).toBeInstanceOf(OpenAiNativeHandler)
// The client should have the /v1 appended internally
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tests verify that the handler instantiates correctly, but they don't actually verify that the /v1 is appended to the client's baseURL. Since the OpenAI client is mocked, we can't directly inspect its configuration. Could we consider adding a way to verify the actual URL being used, perhaps by checking the mock calls or adding integration tests?


// Test with URL that already has /v1
const handler2 = new OpenAiNativeHandler({
apiModelId: "gpt-4.1",
openAiNativeApiKey: "test-api-key",
openAiNativeBaseUrl: "http://localhost:8080/v1",
})
expect(handler2).toBeInstanceOf(OpenAiNativeHandler)

// Test with URL that has a different path
const handler3 = new OpenAiNativeHandler({
apiModelId: "gpt-4.1",
openAiNativeApiKey: "test-api-key",
openAiNativeBaseUrl: "https://api.openai.com/v1",
})
expect(handler3).toBeInstanceOf(OpenAiNativeHandler)

// Test with URL with trailing slash
const handler4 = new OpenAiNativeHandler({
apiModelId: "gpt-4.1",
openAiNativeApiKey: "test-api-key",
openAiNativeBaseUrl: "http://localhost:11434/",
})
expect(handler4).toBeInstanceOf(OpenAiNativeHandler)
})
})

describe("createMessage", () => {
Expand Down
18 changes: 17 additions & 1 deletion src/api/providers/openai-native.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,23 @@ export class OpenAiNativeHandler extends BaseProvider implements SingleCompletio
this.options.enableGpt5ReasoningSummary = true
}
const apiKey = this.options.openAiNativeApiKey ?? "not-provided"
this.client = new OpenAI({ baseURL: this.options.openAiNativeBaseUrl, apiKey })

// Handle base URL - append /v1 if it's a plain URL without a path
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding a comment explaining why /v1 is appended and which servers (like llama.cpp) expect this format. This would help future maintainers understand the reasoning.

Suggested change
// Handle base URL - append /v1 if it's a plain URL without a path
// Handle base URL - append /v1 if it's a plain URL without a path
// This is needed for compatibility with llama.cpp and other OpenAI-compatible servers
// that expose their endpoints at /v1 but users typically provide just the base URL

let baseURL = this.options.openAiNativeBaseUrl
if (baseURL && !baseURL.includes("/v1")) {
// Check if it's a URL without any path (just protocol://host:port)
try {
const url = new URL(baseURL)
// If the pathname is just '/', append /v1
if (url.pathname === "/") {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current implementation only appends /v1 when the pathname is exactly /. What happens with URLs like http://localhost:8080/api or http://localhost:8080/openai? These might also be base URLs that need /v1 appended. Should we consider a more flexible approach or document the expected URL format clearly?

baseURL = baseURL.replace(/\/$/, "") + "/v1"
}
} catch {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When URL parsing fails, the code silently continues with the original URL. Would it be helpful to log a warning so users know their URL might not be processed as expected?

Suggested change
} catch {
} catch (error) {
// If URL parsing fails, leave it as is but warn the user
console.warn('Failed to parse OpenAI base URL, using as-is:', baseURL, error);

// If URL parsing fails, leave it as is
}
}

this.client = new OpenAI({ baseURL, apiKey })
}

private normalizeGpt5Usage(usage: any, model: OpenAiNativeModel): ApiStreamUsageChunk | undefined {
Expand Down
Loading