-
Notifications
You must be signed in to change notification settings - Fork 2.8k
add support for custom anthropic models #16673
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
add support for custom anthropic models #16673
Conversation
Signed-off-by: Udit Kapoor <[email protected]>
sdirix
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the contribution ❤️
I tested it successfully by defining a custom model against the official API endpoint 👍
However there is an issue with the "apiKey"-less path, see my comments.
| @@ -1,5 +1,5 @@ | |||
| // ***************************************************************************** | |||
| // Copyright (C) 2024 EclipseSource GmbH. | |||
| // Copyright (C) 2024, 2025 EclipseSource GmbH. | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't update the year in the Theia project and just leave the header alone once a file is created.
|
|
||
| export const API_KEY_PREF = 'ai-features.anthropic.AnthropicApiKey'; | ||
| export const MODELS_PREF = 'ai-features.anthropic.AnthropicModels'; | ||
| export const CUSTOM_ENDPOINTS_PREF = 'ai-features.AnthropicCustom.customAnthropicModels'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| export const CUSTOM_ENDPOINTS_PREF = 'ai-features.AnthropicCustom.customAnthropicModels'; | |
| export const CUSTOM_ENDPOINTS_PREF = 'ai-features.anthropicCustom.customAnthropicModels'; |
| [CUSTOM_ENDPOINTS_PREF]: { | ||
| type: 'array', | ||
| title: AI_CORE_PREFERENCES_TITLE, | ||
| markdownDescription: nls.localize('theia/ai/anthropic/useResponseApi/mdDescription', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| markdownDescription: nls.localize('theia/ai/anthropic/useResponseApi/mdDescription', | |
| markdownDescription: nls.localize('theia/ai/anthropic/customEndpoints/mdDescription', |
| // Always mark custom models (models with url) as ready for now as we do not know about API Key requirements | ||
| if (modelDescription.url) { | ||
| return { status: 'ready' }; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change is not sufficient. Within the AnthropicModel we check for the api key and if it does not exist, the model will immediately throw an error before even trying to use the SDK, see here.
If we allow apiKey-less models to report ready, then there must be a successful path for them.
Please check what is required for this. In the OpenAI SDK we always need to hand over a key, so for keyless models we hand over a "stub key". I don't know whether this is also required for the Anthropic SDK or whether it actually supports not handing over a key if the endpoint does not need one.
|
Hi @udit-329, just a gentle ping, the review is done and there are a few points that would need your input before we can proceed. TIA! |
What it does
Adds support for custom anthropic models. This works similar to how user can configure custom openAI models for situations such as when the model is hosted on a custom url.
Addresses #16666
How to test
If a user has an Anthropic model that is accessed from a custom endpoint, they can add it in settings.json similar to how it can already be done for openAI models.
Follow-ups
Breaking changes
Attribution
Review checklist
nlsservice (for details, please see the Internationalization/Localization section in the Coding Guidelines)Reminder for reviewers