Add Amazon Bedrock Unified Chat Completions support #133697
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
elasticsearchmachine
added
v9.2.0
external-contributor
Evgenii-Kazannik
force-pushed
the
Add-Amazon-Bedrock-Unified-Chat-Completions-support
branch
5 times, most recently
from
141c154 to
736ce22
Compare
| var chatCompletionInput = inferenceInputs.castTo(ChatCompletionInput.class); | ||
| var inputs = chatCompletionInput.getInputs(); | ||
| var stream = chatCompletionInput.stream(); | ||
| switch (inferenceInputs) { |
There was a problem hiding this comment.
How about we create a new class that handles the UnifiedChatInput directly?
Once we've done that we can use that class directly in doUnifiedCompletionInfer.
| protected void executeClientRequest(AmazonBedrockBaseClient awsBedrockClient) { | ||
| if (chatCompletionRequest.isStreaming()) { | ||
| var publisher = chatCompletionRequest.executeStreamChatCompletionRequest(awsBedrockClient); | ||
| inferenceResultsListener.onResponse(new StreamingChatCompletionResults(publisher)); |
There was a problem hiding this comment.
Instead of StreamingChatCompletionResults I think we want to use StreamingUnifiedChatCompletionResults. I think we should just be able to pass in publisher 🤔 Let me know if that works.
Evgenii-Kazannik
force-pushed
the
Add-Amazon-Bedrock-Unified-Chat-Completions-support
branch
from
2cd928c to
a6df6f0
Compare
Evgenii-Kazannik
force-pushed
the
Add-Amazon-Bedrock-Unified-Chat-Completions-support
branch
from
3f4df8f to
158734b
Compare
Evgenii-Kazannik
force-pushed
the
Add-Amazon-Bedrock-Unified-Chat-Completions-support
branch
from
5157eb3 to
0f83043
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
DRAFT