client-bedrock-runtime: Same request via InvokeModelWithResponseStreamCommand throws different exceptions depending on the model used

[modcab]

Checkboxes for prior research

• I've gone through Developer Guide and API reference
• I've checked AWS Forums and StackOverflow.
• I've searched for previous similar issues and didn't find any solution.

Describe the bug

When sending a request that exceeds the allowed token count:

• anthropic.claude-3-haiku-20240307-v1:0 will throw a ValidationException with a descriptive message: ValidationException: Input is too long for requested model.
• anthropic.claude-3-opus-20240229-v1:0 will throw a ModelStreamErrorException with a generic message: The system encountered an unexpected error during processing. Try your request again.

SDK version number

@aws-sdk/client-bedrock-runtime@3.602.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v20.11.0

Reproduction Steps

Sending a request that exceeds the allowed token count and examine the exception thrown.

Observed Behavior

• anthropic.claude-3-haiku-20240307-v1:0 will throw a ValidationException with a descriptive message: ValidationException: Input is too long for requested model.
• anthropic.claude-3-opus-20240229-v1:0 will throw a ModelStreamErrorException with a generic message: The system encountered an unexpected error during processing. Try your request again.

Expected Behavior

The same exception type for both models, with information that allows us to respond correctly.

Possible Solution

No response

Additional Information/Context

No response

[aBurmeseDev]

Hi @modcab - thanks for reaching out.

To understand the issue better, can you please share your code snippet without any sensitive information? Are you seeing both error in the same request or is it thrown individually in different request? Note that these errors come from service side which means we'll have to reach out to Bedrock team for their input once you've shared the code.

[modcab]

Hi @aBurmeseDev , thanks for your quick response. They're individual, different requests. The actual code is of course much longer and scattered over different functions and files, so let me know if this helps.

First one:

//
// const params = {
//   body: Uint8Array(4485220) [
//     123,  34, 115, 121, 115, 116, 101, 109,  34,  58,  34,  34,
//       44,  34, 109, 101, 115, 115,  97, 103, 101, 115,  34,  58,
//       91, 123,  34, 114, 111, 108, 101,  34,  58,  34, 117, 115,
//       101, 114,  34,  44,  34,  99, 111, 110, 116, 101, 110, 116,
//       34,  58,  91, 123,  34, 116, 121, 112, 101,  34,  58,  34,
//       116, 101, 120, 116,  34,  44,  34, 116, 101, 120, 116,  34,
//       58,  34, 111, 107,  46,  32,  67, 111, 110, 116, 101, 110,
//       116, 115,  32, 111, 102,  32, 102, 105, 108, 101,  32, 111,
//       122,  95,  49,  46,
// ... 4485120 more items
// ],
//   modelId: 'anthropic.claude-3-haiku-20240307-v1:0',
//     contentType: 'application/json',
//   accept: 'application/json'
// }

const command = new InvokeModelWithResponseStreamCommand(params);
const data = await client.send(command);
const stream = data.body;
// ...
for await (const encodedChunk of stream) {

There, it breaks, and I can see this error in the logs:

{
    "timestamp": "---",
    "level": "ERROR",
    "requestId": "---",
    "message": {
        "errorType": "ValidationException",
        "errorMessage": "Input is too long for requested model.",
        "stackTrace": [
            "ValidationException: Input is too long for requested model.",
            "    at ----",
            "    at ----",
            "    at ----",
        ],
        "name": "ValidationException",
        "$fault": "client",
        "$metadata": {}
    }
}

Which is great. It has a descriptive message, and I can present it in a helpful way.

But then, if the code is the same, but the modelId is 'anthropic.claude-3-opus-20240229-v1:0' or anthropic.claude-3-5-sonnet-20240620-v1:0' then we get this one:

{
    "timestamp": "-----",
    "level": "ERROR",
    "requestId": "-----",
    "message": "An unexpected error occurred in the chat stream: ModelStreamErrorException: The system encountered an unexpected error during processing. Try your request again.\n    at ---",
    "errorType": "ModelStreamErrorException",
    "errorMessage": "The system encountered an unexpected error during processing. Try your request again.",
    "stackTrace": [
        "ModelStreamErrorException: The system encountered an unexpected error during processing. Try your request again.",
        "    at ----",
        "    at  ----",
        "    at ----",
    ]
}

[aBurmeseDev]

Apologies for the delay here. As previously mentioned, this was thrown by upstream service team and according to service team, this behavior was addressed in recent version. Please verify and feel free to create a new issue if issue persists.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.