Skip to content

Conversation

sonianuj287
Copy link

@sonianuj287 sonianuj287 commented Oct 7, 2025

This PR updates the DefaultLlmImpl class to properly parse responses from the Claude V3 family of Bedrock models. Previously, only the V2 response format (completion field) was supported. With this change, the parser now supports the V3 format (content -> text) while maintaining backward compatibility with V2.

Changes Made

  • Updated buildChatCompletionOutput():
    Detects "content" field for Claude V3 responses.
    Extracts text from the first element of the content list.
    Adds extracted text to answers.
    Added a new unit test in DefaultLlmImplTests.java:
  • testChatCompletionApiForBedrockClaudeV3()
    Mocks a V3-style Bedrock response and validates output parsing.
    Reformatted test files to satisfy Spotless formatting rules.

Why this change is needed
Claude V2 models are deprecated; V3 is the current standard.
Without this fix, requests to "bedrock/anthropic-claude" with V3 models fail to extract answers, causing RAG pipelines to break.
Ensures users don’t need to pass llmResponseField manually for default Bedrock Claude models.

How to test
Run all existing unit tests:
./gradlew clean build

resolves #4168 (comment)

@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 15:01 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 15:01 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 15:01 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 15:01 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 force-pushed the fix/claude-v3-bedrock-support branch from aa64626 to 540a52e Compare October 7, 2025 15:02
@sonianuj287 sonianuj287 had a problem deploying to ml-commons-cicd-env-require-approval October 7, 2025 15:03 — with GitHub Actions Failure
@sonianuj287 sonianuj287 had a problem deploying to ml-commons-cicd-env-require-approval October 7, 2025 15:03 — with GitHub Actions Error
@sonianuj287 sonianuj287 had a problem deploying to ml-commons-cicd-env-require-approval October 7, 2025 15:03 — with GitHub Actions Failure
@sonianuj287 sonianuj287 had a problem deploying to ml-commons-cicd-env-require-approval October 7, 2025 15:03 — with GitHub Actions Error
@sonianuj287
Copy link
Author

Hi @mingshl @rithin-pullela-aws , Could you please review my PR and suggest changes.

Thanks :)

@mingshl
Copy link
Collaborator

mingshl commented Oct 7, 2025

oh great, this is going to use the same interface string, and it will auto detect these two formats and parse accordingly. Can you also add in ITs? this is the best way we can monitor and keep track when the model interface failed.

I ran your CIs and also I added the resolved issues and link the issue in the PR descriptions.

@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 20:02 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 20:02 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 20:02 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 7, 2025 20:02 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 17, 2025 09:53 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 17, 2025 09:53 — with GitHub Actions Waiting
@sonianuj287
Copy link
Author

Small Comment about the IT. It should simple to have the IT fixed instead of writing a new one.
We can remove the BM25_SEARCH_REQUEST_WITH_CONVO_WITH_LLM_RESPONSE_TEMPLATE from plugin/src/test/java/org/opensearch/ml/rest/RestMLRAGSearchProcessorIT.java to have the IT hit the code change

Hi @rithin-pullela-aws , I understood your point. Made some changes, please review. Thanks

Hi @rithin-pullela-aws

@sonianuj287
Copy link
Author

Small Comment about the IT. It should simple to have the IT fixed instead of writing a new one.
We can remove the BM25_SEARCH_REQUEST_WITH_CONVO_WITH_LLM_RESPONSE_TEMPLATE from plugin/src/test/java/org/opensearch/ml/rest/RestMLRAGSearchProcessorIT.java to have the IT hit the code change

Hi @rithin-pullela-aws , I understood your point. Made some changes, please review. Thanks

Hi @rithin-pullela-aws

Hi @rithin-pullela-aws , so sorry to spam your inbox. Just want to work here more, please help me with reviewing and suggesting changes. Thanks :)

@sonianuj287 sonianuj287 had a problem deploying to ml-commons-cicd-env-require-approval October 21, 2025 17:25 — with GitHub Actions Error
@sonianuj287 sonianuj287 had a problem deploying to ml-commons-cicd-env-require-approval October 21, 2025 17:25 — with GitHub Actions Failure
@sonianuj287 sonianuj287 had a problem deploying to ml-commons-cicd-env-require-approval October 21, 2025 17:25 — with GitHub Actions Error
@sonianuj287 sonianuj287 had a problem deploying to ml-commons-cicd-env-require-approval October 21, 2025 17:25 — with GitHub Actions Failure
@ylwu-amzn
Copy link
Collaborator

Thanks for fixing this part. I think we should include this in the coming 3.3.2. Add a backport label

@rithin-pullela-aws
Copy link
Contributor

Hi @sonianuj287, I added some comments, can you please address them?

@sonianuj287 sonianuj287 force-pushed the fix/claude-v3-bedrock-support branch from 5139cff to 641ff7d Compare October 21, 2025 17:45
@sonianuj287
Copy link
Author

Hi @mingshl , resolved the comments. Thanks for pointing

@sonianuj287 sonianuj287 force-pushed the fix/claude-v3-bedrock-support branch from cea2567 to f17840c Compare October 21, 2025 18:04
@sonianuj287 sonianuj287 force-pushed the fix/claude-v3-bedrock-support branch from f17840c to 2fa3e62 Compare October 21, 2025 18:12
@sonianuj287 sonianuj287 requested a review from mingshl October 21, 2025 18:14
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 21, 2025 18:15 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 21, 2025 18:15 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 21, 2025 18:15 — with GitHub Actions Waiting
@sonianuj287 sonianuj287 requested a deployment to ml-commons-cicd-env-require-approval October 21, 2025 18:15 — with GitHub Actions Waiting
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Outdated Default Claude model parsing in Generative QA Processor

5 participants