Skip to content

feat: Add IBM watsonx AI provider support #8091

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

feat: Add IBM watsonx AI provider support #8091

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c69452c
feat: add IBM watsonx AI provider support
roomote Sep 17, 2025
b0ff385
fix: address review feedback for watsonx provider
roomote Sep 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions packages/types/src/provider-settings.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import {
vscodeLlmModels,
xaiModels,
internationalZAiModels,
watsonxModels,
} from "./providers/index.js"

/**
Expand Down Expand Up @@ -68,6 +69,7 @@ export const providerNames = [
"io-intelligence",
"roo",
"vercel-ai-gateway",
"watsonx",
] as const

export const providerNamesSchema = z.enum(providerNames)
Expand Down Expand Up @@ -343,6 +345,13 @@ const vercelAiGatewaySchema = baseProviderSettingsSchema.extend({
vercelAiGatewayModelId: z.string().optional(),
})

const watsonxSchema = apiModelIdProviderModelSchema.extend({
watsonxApiKey: z.string().optional(),
watsonxProjectId: z.string().optional(),
watsonxBaseUrl: z.string().optional(),
watsonxRegion: z.string().optional(),
})

const defaultSchema = z.object({
apiProvider: z.undefined(),
})
Expand Down Expand Up @@ -384,6 +393,7 @@ export const providerSettingsSchemaDiscriminated = z.discriminatedUnion("apiProv
qwenCodeSchema.merge(z.object({ apiProvider: z.literal("qwen-code") })),
rooSchema.merge(z.object({ apiProvider: z.literal("roo") })),
vercelAiGatewaySchema.merge(z.object({ apiProvider: z.literal("vercel-ai-gateway") })),
watsonxSchema.merge(z.object({ apiProvider: z.literal("watsonx") })),
defaultSchema,
])

Expand Down Expand Up @@ -425,6 +435,7 @@ export const providerSettingsSchema = z.object({
...qwenCodeSchema.shape,
...rooSchema.shape,
...vercelAiGatewaySchema.shape,
...watsonxSchema.shape,
...codebaseIndexProviderSchema.shape,
})

Expand Down Expand Up @@ -578,6 +589,7 @@ export const MODELS_BY_PROVIDER: Record<
unbound: { id: "unbound", label: "Unbound", models: [] },
deepinfra: { id: "deepinfra", label: "DeepInfra", models: [] },
"vercel-ai-gateway": { id: "vercel-ai-gateway", label: "Vercel AI Gateway", models: [] },
watsonx: { id: "watsonx", label: "IBM watsonx", models: Object.keys(watsonxModels) },
}

export const dynamicProviders = [
Expand All @@ -589,6 +601,7 @@ export const dynamicProviders = [
"unbound",
"deepinfra",
"vercel-ai-gateway",
"watsonx",
Copy link
Contributor Author

@roomote roomote bot Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While watsonx is correctly added to dynamicProviders, it's worth noting (maybe in a comment) that unlike other dynamic providers, watsonx doesn't actually fetch models dynamically - it uses a static list. This is already documented in the getWatsonxModels function, but a note here might help future maintainers understand the distinction.

] as const satisfies readonly ProviderName[]

export type DynamicProvider = (typeof dynamicProviders)[number]
Expand Down
1 change: 1 addition & 0 deletions packages/types/src/providers/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,4 @@ export * from "./xai.js"
export * from "./vercel-ai-gateway.js"
export * from "./zai.js"
export * from "./deepinfra.js"
export * from "./watsonx.js"
148 changes: 148 additions & 0 deletions packages/types/src/providers/watsonx.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
import type { ModelInfo } from "../model.js"

// IBM watsonx.ai models
// https://www.ibm.com/products/watsonx-ai
export type WatsonxModelId = keyof typeof watsonxModels

export const watsonxDefaultModelId: WatsonxModelId = "ibm/granite-3-8b-instruct"

export const watsonxModels = {
// Granite models - IBM's foundation models
"ibm/granite-3-8b-instruct": {
maxTokens: 8192,
contextWindow: 8192,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.0002,
outputPrice: 0.0006,
description: "IBM Granite 3.0 8B Instruct - Optimized for enterprise tasks",
},
"ibm/granite-3-2b-instruct": {
maxTokens: 4096,
contextWindow: 4096,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.0001,
outputPrice: 0.0003,
description: "IBM Granite 3.0 2B Instruct - Lightweight model for simple tasks",
},
"ibm/granite-20b-multilingual": {
maxTokens: 8192,
contextWindow: 8192,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.0006,
outputPrice: 0.0018,
description: "IBM Granite 20B Multilingual - Supports multiple languages",
},
"ibm/granite-13b-chat-v2": {
maxTokens: 8192,
contextWindow: 8192,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.0004,
outputPrice: 0.0012,
description: "IBM Granite 13B Chat v2 - Optimized for conversational AI",
},
"ibm/granite-13b-instruct-v2": {
maxTokens: 8192,
contextWindow: 8192,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.0004,
outputPrice: 0.0012,
description: "IBM Granite 13B Instruct v2 - General purpose instruction following",
},
"ibm/granite-7b-lab": {
maxTokens: 4096,
contextWindow: 4096,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.0002,
outputPrice: 0.0006,
description: "IBM Granite 7B Lab - Experimental model for research",
},
// Granite Code models - specialized for code generation
"ibm/granite-34b-code-instruct": {
maxTokens: 8192,
contextWindow: 8192,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.001,
outputPrice: 0.003,
description: "IBM Granite 34B Code Instruct - Specialized for code generation and understanding",
},
"ibm/granite-20b-code-instruct": {
maxTokens: 8192,
contextWindow: 8192,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.0006,
outputPrice: 0.0018,
description: "IBM Granite 20B Code Instruct - Code generation model",
},
"ibm/granite-8b-code-instruct": {
maxTokens: 4096,
contextWindow: 4096,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.0002,
outputPrice: 0.0006,
description: "IBM Granite 8B Code Instruct - Lightweight code model",
},
"ibm/granite-3b-code-instruct": {
maxTokens: 2048,
contextWindow: 2048,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.0001,
outputPrice: 0.0003,
description: "IBM Granite 3B Code Instruct - Fast code completion",
},
// Third-party models available on watsonx
"meta-llama/llama-3-70b-instruct": {
maxTokens: 8192,
contextWindow: 8192,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.0029,
outputPrice: 0.0087,
description: "Meta Llama 3 70B Instruct on watsonx",
},
"meta-llama/llama-3-8b-instruct": {
maxTokens: 8192,
contextWindow: 8192,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.0002,
outputPrice: 0.0006,
description: "Meta Llama 3 8B Instruct on watsonx",
},
"mistralai/mixtral-8x7b-instruct-v01": {
maxTokens: 4096,
contextWindow: 32768,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.0005,
outputPrice: 0.0015,
description: "Mistral Mixtral 8x7B Instruct on watsonx",
},
"mistralai/mistral-large": {
maxTokens: 8192,
contextWindow: 32768,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0.003,
outputPrice: 0.009,
description: "Mistral Large on watsonx",
},
} as const satisfies Record<string, ModelInfo>

export const watsonxModelInfoSaneDefaults: ModelInfo = {
Copy link
Contributor Author

@roomote roomote bot Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This export is defined but never used anywhere. The PR description mentions it was removed, but it's still here. Should we actually remove it?

maxTokens: 4096,
contextWindow: 8192,
supportsImages: false,
supportsPromptCache: false,
inputPrice: 0,
outputPrice: 0,
}
3 changes: 3 additions & 0 deletions src/api/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ import {
FeatherlessHandler,
VercelAiGatewayHandler,
DeepInfraHandler,
WatsonxHandler,
} from "./providers"
import { NativeOllamaHandler } from "./providers/native-ollama"

Expand Down Expand Up @@ -165,6 +166,8 @@ export function buildApiHandler(configuration: ProviderSettings): ApiHandler {
return new FeatherlessHandler(options)
case "vercel-ai-gateway":
return new VercelAiGatewayHandler(options)
case "watsonx":
return new WatsonxHandler(options)
default:
apiProvider satisfies "gemini-cli" | undefined
return new AnthropicHandler(options)
Expand Down
1 change: 1 addition & 0 deletions src/api/providers/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,4 @@ export { RooHandler } from "./roo"
export { FeatherlessHandler } from "./featherless"
export { VercelAiGatewayHandler } from "./vercel-ai-gateway"
export { DeepInfraHandler } from "./deepinfra"
export { WatsonxHandler } from "./watsonx"
Loading
Loading