-
Notifications
You must be signed in to change notification settings - Fork 184
Fix #4168: Support Claude V3 response format in DefaultLLMImpl #4275
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Fix #4168: Support Claude V3 response format in DefaultLLMImpl #4275
Conversation
aa64626
to
540a52e
Compare
Hi @mingshl @rithin-pullela-aws , Could you please review my PR and suggest changes. Thanks :) |
oh great, this is going to use the same interface string, and it will auto detect these two formats and parse accordingly. Can you also add in ITs? this is the best way we can monitor and keep track when the model interface failed. I ran your CIs and also I added the resolved issues and link the issue in the PR descriptions. |
|
Hi @rithin-pullela-aws , so sorry to spam your inbox. Just want to work here more, please help me with reviewing and suggesting changes. Thanks :) |
Thanks for fixing this part. I think we should include this in the coming 3.3.2. Add a backport label |
Hi @sonianuj287, I added some comments, can you please address them? |
plugin/src/test/java/org/opensearch/ml/rest/RestMLRAGSearchProcessorIT.java
Outdated
Show resolved
Hide resolved
5139cff
to
641ff7d
Compare
Hi @mingshl , resolved the comments. Thanks for pointing |
cea2567
to
f17840c
Compare
Signed-off-by: Anuj Soni <[email protected]>
…ntent, text) Signed-off-by: Anuj Soni <[email protected]>
Signed-off-by: Anuj Soni <[email protected]>
Signed-off-by: Anuj Soni <[email protected]>
Signed-off-by: Anuj Soni <[email protected]>
Signed-off-by: Anuj Soni <[email protected]>
Signed-off-by: Anuj Soni <[email protected]>
f17840c
to
2fa3e62
Compare
This PR updates the DefaultLlmImpl class to properly parse responses from the Claude V3 family of Bedrock models. Previously, only the V2 response format (completion field) was supported. With this change, the parser now supports the V3 format (content -> text) while maintaining backward compatibility with V2.
Changes Made
Detects "content" field for Claude V3 responses.
Extracts text from the first element of the content list.
Adds extracted text to answers.
Added a new unit test in DefaultLlmImplTests.java:
Mocks a V3-style Bedrock response and validates output parsing.
Reformatted test files to satisfy Spotless formatting rules.
Why this change is needed
Claude V2 models are deprecated; V3 is the current standard.
Without this fix, requests to "bedrock/anthropic-claude" with V3 models fail to extract answers, causing RAG pipelines to break.
Ensures users don’t need to pass llmResponseField manually for default Bedrock Claude models.
How to test
Run all existing unit tests:
./gradlew clean build
resolves #4168 (comment)