-
Notifications
You must be signed in to change notification settings - Fork 25.6k
Add Google Model Garden's Anthropic support to Inference Plugin #134080
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
jonathan-buttner
merged 64 commits into
elastic:main
from
Jan-Kazlouski-elastic:google-model-garden-integration
Sep 29, 2025
Merged
Changes from 20 commits
Commits
Show all changes
64 commits
Select commit
Hold shift + click to select a range
ef50c43
Add Google Model Garden Anthropic integration
Jan-Kazlouski-elastic 1de1067
Clean up AnthropicChatCompletionStreamingProcessor
Jan-Kazlouski-elastic ae4576c
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic ea9f933
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic 9f61342
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic c3ce521
Enhance GoogleVertexAiChatCompletionServiceSettings to support option…
Jan-Kazlouski-elastic fc136a5
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic 4e6ccee
Add extractOptionalUri method and corresponding tests for URI extraction
Jan-Kazlouski-elastic 07eed54
Add GoogleModelGardenProvider support to chat completion models and t…
Jan-Kazlouski-elastic 1adf958
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic a4ad1c5
Enhance AnthropicChatCompletionStreamingProcessor and related classes…
Jan-Kazlouski-elastic b143661
Refactor AnthropicChatCompletionResponseHandler to use a custom error…
Jan-Kazlouski-elastic 0712434
Add unit tests for AnthropicChatCompletionStreamingProcessor to valid…
Jan-Kazlouski-elastic fa44bd4
Add unit tests for GoogleModelGardenAnthropicChatCompletionRequestEnt…
Jan-Kazlouski-elastic 8a1e710
Add support for Anthropic provider in Google Vertex AI chat completio…
Jan-Kazlouski-elastic ff63315
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic 93cb5ad
Add changelog
Jan-Kazlouski-elastic 54ff2ff
Refactor switch case in GoogleVertexAiActionCreator to handle null case
Jan-Kazlouski-elastic 67d5d8a
Validate service settings for Google Vertex AI model configuration
Jan-Kazlouski-elastic a4540d3
Enhance Anthropic model tests to validate URI handling and provider r…
Jan-Kazlouski-elastic 4d1a6fa
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic 8f60605
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic f4be58b
[CI] Auto commit changes from spotless
d926c71
Refactor switch case in GoogleVertexAiService to handle null case
Jan-Kazlouski-elastic 954c423
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic fbb0cf7
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic 386eedf
Simplify version check in GoogleVertexAiChatCompletionServiceSettings
Jan-Kazlouski-elastic 32b44b2
Make GOOGLE provider default for GoogleModelGarden integration
Jan-Kazlouski-elastic 63ea4b8
Update anthropic_version to vertex-2024-10-22 in request entity and t…
Jan-Kazlouski-elastic 76b27a5
Refactor Google Vertex AI request handling to improve provider manage…
Jan-Kazlouski-elastic eb4c328
Enhance validation for Google Model Garden settings to ensure require…
Jan-Kazlouski-elastic c61b069
Remove uri streamingUri and provider from rate limit grouping hash ca…
Jan-Kazlouski-elastic c9631fb
Refactor null and empty checks for projectId, location, and modelId i…
Jan-Kazlouski-elastic 4072327
Refactor Google Model Garden integration to include task settings in …
Jan-Kazlouski-elastic 5e5c985
Revert "Update anthropic_version to vertex-2024-10-22 in request enti…
Jan-Kazlouski-elastic 6987c00
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic 001670c
Refactor Google Vertex AI settings to utilize GoogleVertexAiUtils for…
Jan-Kazlouski-elastic 1d27867
[CI] Update transport version definitions
7c6940b
Update anthropic_version in tests and enhance validation logic for Go…
Jan-Kazlouski-elastic c14b8c6
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic b715a83
Update versions
Jan-Kazlouski-elastic 01c7348
Enhance task settings validation in GoogleVertexAiChatCompletionModel
Jan-Kazlouski-elastic af5c93a
Address comments regarding anthropic version and configuration
Jan-Kazlouski-elastic 9fe4bdc
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic 398992d
[CI] Update transport version definitions
d0d2e57
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic bd43f4f
Add nullable annotation for maxTokens parameter in GoogleVertexAiChat…
Jan-Kazlouski-elastic df27cc0
[CI] Update transport version definitions
b1823ee
Clarify URI handling logic in GoogleVertexAiChatCompletionModel comments
Jan-Kazlouski-elastic 9fd5e03
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic 370c6dc
Make maxTokens nullable
Jan-Kazlouski-elastic 87c1464
[CI] Update transport version definitions
3e498ce
Fixed unit tests
Jan-Kazlouski-elastic 066b396
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic 617c090
[CI] Update transport version definitions
ffc8e4f
Fix validation logic for Google Model Garden and Vertex AI settings
Jan-Kazlouski-elastic ac47f22
[CI] Update transport version definitions
e47e311
Add validation tests for Google Vertex AI and Model Garden settings
Jan-Kazlouski-elastic 2852695
Refactor validation logic for Google Vertex AI and Model Garden settings
Jan-Kazlouski-elastic 23b86ec
Add comment
Jan-Kazlouski-elastic d1b63bf
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic 89a744e
Update Google Vertex AI Task Settings parsing logic and AnthropicChat…
Jan-Kazlouski-elastic 2847a1f
Merge remote-tracking branch 'origin/main' into google-model-garden-i…
Jan-Kazlouski-elastic 55da91c
Merge branch 'main' into google-model-garden-integration
jonathan-buttner File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
pr: 134080 | ||
summary: Added Google Model Garden Anthropic Completion and Chat Completion support to the Inference Plugin | ||
area: Machine Learning | ||
type: enhancement | ||
issues: [] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
60 changes: 60 additions & 0 deletions
60
...sticsearch/xpack/inference/services/anthropic/AnthropicChatCompletionResponseHandler.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
/* | ||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
* or more contributor license agreements. Licensed under the Elastic License | ||
* 2.0; you may not use this file except in compliance with the Elastic License | ||
* 2.0. | ||
*/ | ||
|
||
package org.elasticsearch.xpack.inference.services.anthropic; | ||
|
||
import org.elasticsearch.inference.InferenceServiceResults; | ||
import org.elasticsearch.xpack.core.inference.results.StreamingUnifiedChatCompletionResults; | ||
import org.elasticsearch.xpack.core.inference.results.UnifiedChatCompletionException; | ||
import org.elasticsearch.xpack.inference.external.http.HttpResult; | ||
import org.elasticsearch.xpack.inference.external.http.retry.ChatCompletionErrorResponseHandler; | ||
import org.elasticsearch.xpack.inference.external.http.retry.ResponseParser; | ||
import org.elasticsearch.xpack.inference.external.http.retry.UnifiedChatCompletionErrorParserContract; | ||
import org.elasticsearch.xpack.inference.external.http.retry.UnifiedChatCompletionErrorResponseUtils; | ||
import org.elasticsearch.xpack.inference.external.request.Request; | ||
import org.elasticsearch.xpack.inference.external.response.streaming.ServerSentEventParser; | ||
import org.elasticsearch.xpack.inference.external.response.streaming.ServerSentEventProcessor; | ||
import org.elasticsearch.xpack.inference.services.anthropic.response.AnthropicChatCompletionResponseEntity; | ||
|
||
import java.util.concurrent.Flow; | ||
|
||
/** | ||
* Handles streaming chat completion responses and error parsing for Anthropic inference endpoints. | ||
* Adapts the AnthropicResponseHandler to support chat completion schema. | ||
*/ | ||
public class AnthropicChatCompletionResponseHandler extends AnthropicResponseHandler { | ||
private static final String ANTHROPIC_ERROR = "anthropic_error"; | ||
private static final UnifiedChatCompletionErrorParserContract ANTHROPIC_ERROR_PARSER = UnifiedChatCompletionErrorResponseUtils | ||
.createErrorParserWithStringify(ANTHROPIC_ERROR); | ||
|
||
private final ChatCompletionErrorResponseHandler chatCompletionErrorResponseHandler; | ||
|
||
public AnthropicChatCompletionResponseHandler(String requestType) { | ||
this(requestType, AnthropicChatCompletionResponseEntity::fromResponse); | ||
} | ||
|
||
private AnthropicChatCompletionResponseHandler(String requestType, ResponseParser parseFunction) { | ||
super(requestType, parseFunction, true); | ||
this.chatCompletionErrorResponseHandler = new ChatCompletionErrorResponseHandler(ANTHROPIC_ERROR_PARSER); | ||
} | ||
|
||
@Override | ||
public InferenceServiceResults parseResult(Request request, Flow.Publisher<HttpResult> flow) { | ||
var serverSentEventProcessor = new ServerSentEventProcessor(new ServerSentEventParser()); | ||
var anthropicProcessor = new AnthropicChatCompletionStreamingProcessor( | ||
(m, e) -> chatCompletionErrorResponseHandler.buildMidStreamChatCompletionError(request.getInferenceEntityId(), m, e) | ||
); | ||
flow.subscribe(serverSentEventProcessor); | ||
serverSentEventProcessor.subscribe(anthropicProcessor); | ||
return new StreamingUnifiedChatCompletionResults(anthropicProcessor); | ||
} | ||
|
||
@Override | ||
protected UnifiedChatCompletionException buildError(String message, Request request, HttpResult result) { | ||
return chatCompletionErrorResponseHandler.buildChatCompletionError(message, request, result); | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let me know if this needs to be removed. I haven't seen backports in a while. But Google Vertex AI is there for quite some time, so probably we'd require one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's remove this, we won't be backporting the changes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed.