Skip to content

Conversation

@jonathan343
Copy link
Contributor

@jonathan343 jonathan343 commented Oct 22, 2025

Summary

Member index was updated to use natural ordering in #510

When code generating the deserialize members, we skip members that are of union type and have the streaming trait (see here) without incrementing the index. These results in the index of subsequent members being off by 1.

This PR increments the index even in the case when a member is skipped.

Impact

This addresses the broken invoke_model_with_response_stream operation in the aws-sdk-bedrock-runtime.

Additionally, it unblocks us from releasing a aws-sdk-transcribe-streaming client since it addresses issues with the start_stream_transcription operation.

Testing

aws-sdk-bedrock-runtime

Example adapted from: https://docs.aws.amazon.com/nova/latest/userguide/using-invoke-api.html

import asyncio
import json

from smithy_aws_core.identity import EnvironmentCredentialsResolver
from aws_sdk_bedrock_runtime.client import BedrockRuntimeClient
from aws_sdk_bedrock_runtime.models import InvokeModelWithResponseStreamInput
from aws_sdk_bedrock_runtime.config import Config


MODEL_ID = "us.amazon.nova-lite-v1:0"


async def test_async():
    c = BedrockRuntimeClient(
        config=Config(
            endpoint_uri="https://bedrock-runtime.us-east-1.amazonaws.com",
            region="us-east-1",
            aws_credentials_identity_resolver=EnvironmentCredentialsResolver(),
        )
    )
    # Define your system prompt(s).
    system_list = [
        {
            "text": "Act as a creative writing assistant. When the user provides you with a topic, write a short story about that topic."
        }
    ]

    # Define one or more messages using the "user" and "assistant" roles.
    message_list = [{"role": "user", "content": [{"text": "A camping trip"}]}]

    # Configure the inference parameters.
    inf_params = {"maxTokens": 500, "topP": 0.9, "topK": 20, "temperature": 0.7}

    request_body = {
        "schemaVersion": "messages-v1",
        "messages": message_list,
        "system": system_list,
        "inferenceConfig": inf_params,
    }
    response = c.invoke_model_with_response_stream(
        InvokeModelWithResponseStreamInput(
            model_id=MODEL_ID,
            body=json.dumps(request_body).encode("utf-8"),
        )
    )

    async with await response as stream:
        output = ""
        async for r in stream.output_stream:
            try:
                text = json.loads(r.value.bytes_.decode())["contentBlockDelta"][
                    "delta"
                ]["text"]
                output += text
            except Exception:
                continue
        print(f"Response:\n------------\n{output}")


if __name__ == "__main__":
    asyncio.run(test_async())

When installing the latest aws-sdk-bedrock-runtime client from PyPI and running the example above, the following error is thrown:

smithy_core.exceptions.SmithyError: InvokeModelWithResponseStreamOutput.__init__() missing 1 required keyword-only argument: 'content_type'

With this fix I get the following response:

Response:
------------
The sun hung low in the sky, casting a golden glow over the dense forest as Mia, Jake, and their dog, Rusty, set up camp. They had hiked for hours, the trail winding through towering pines and past a babbling brook, until they found the perfect spot by a serene lake.

Mia pitched the tent while Jake gathered firewood. Rusty, ever the explorer, darted around, sniffing at the underbrush and occasionally bringing back a twig for Jake to add to the growing pile. The air was crisp and filled with the scent of pine needles and earth.

As the sun dipped below the horizon, painting the sky in hues of orange and purple, the trio gathered around the fire. They roasted marshmallows, the sweet, gooey treats melting perfectly over the crackling flames. The lake shimmered under the moonlight, reflecting the stars that began to twinkle above.

They shared stories, laughter echoing through the quiet forest. Mia spoke of her childhood camping trips with her family, while Jake recounted tales of his adventurous spirit and the places he had traveled. Rusty lay at their feet, occasionally letting out a contented sigh.

That night, as they settled into their sleeping bags, the sounds of the forest lulled them into a peaceful sleep. The gentle rustling of leaves and the distant hoot of an owl created a symphony of nature. In the morning, they awoke to the fresh scent of dew and the promise of another day of adventure.

The camping trip became a cherished memory, a blend of laughter, stories, and the simple joy of being surrounded by nature. They left the campsite as they found it, taking with them not just photographs, but the warmth of shared experiences and the bond of friendship.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@jonathan343 jonathan343 requested a review from a team as a code owner October 22, 2025 17:32
for (MemberShape member : members) {
var target = model.expectShape(member.getTarget());
if (target.hasTrait(StreamingTrait.class) && target.isUnionShape()) {
index++;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit; rather than incrementing in two places, should we just increment at the beginning of each loop once and use the current index value in the formatter on 435?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 26c742f

@jonathan343 jonathan343 merged commit 257e544 into develop Oct 27, 2025
6 checks passed
@jonathan343 jonathan343 deleted the member-indexing-bug branch October 27, 2025 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants