Skip to content

Commit 9805519

Browse files
julianwoodjulianwood
authored
Merge pull request #2865 from jojo786/jojo786-feature-apigw-restapi-lambda-python-response-streaming
New Feature: API Gateway REST API to AWS Lambda Python function with response streaming
2 parents 015b84c + 58f787e commit 9805519

File tree

6 files changed

+302
-0
lines changed

6 files changed

+302
-0
lines changed

‎apigw-rest-api-lambda-python-response-streaming/README.md‎

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# Amazon Gateway REST API to AWS Lambda Python function with response streaming
2+
3+
This pattern demonstrates how to use an Amazon API Gateway REST API with response streaming to an AWS Lambda Python function.
4+
5+
Learn more about this pattern at Serverless Land Patterns: https://serverlessland.com/patterns/apigw-rest-api-lambda-python-response-streaming
6+
7+
Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
8+
9+
## Requirements
10+
11+
* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
12+
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
13+
* [Git installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
14+
* [AWS Serverless Application Model](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html) (AWS SAM) installed
15+
* [Python 3.14 installed](https://www.python.org/downloads/)
16+
17+
## Deployment Instructions
18+
19+
1. Create a new directory, navigate to that directory in a terminal and clone the GitHub repository:
20+
```
21+
git clone https://github.com/aws-samples/serverless-patterns
22+
```
23+
1. Change directory to the pattern directory:
24+
```
25+
cd apigw-restapi-lambda-python-response-streaming
26+
```
27+
1. From the command line, use AWS SAM to deploy the AWS resources for the pattern as specified in the template.yml file:
28+
```
29+
sam deploy --guided
30+
```
31+
1. During the prompts:
32+
* Enter a stack name
33+
* Enter the desired AWS Region
34+
* Allow SAM CLI to create IAM roles with the required permissions
35+
* Allow API Gateway API without any authentication
36+
37+
Once you have run `sam deploy --guided` mode once and saved arguments to a configuration file (samconfig.toml), you can use `sam deploy` in future to use these defaults.
38+
39+
1. Note the outputs from the SAM deployment process. These contain the resource names and/or ARNs which are used for testing.
40+
41+
## How it works
42+
43+
Response streaming allows you to incrementally stream responses back to clients rather than waiting for the entire response to be buffered first, reducing Time to First Byte, and making your applications more responsive to users. [Amazon API Gateway REST APIs support response streaming](https://aws.amazon.com/blogs/compute/building-responsive-apis-with-amazon-api-gateway-response-streaming/).
44+
[AWS Lambda supports response streaming natively for Nodejs](https://aws.amazon.com/blogs/compute/introducing-aws-lambda-response-streaming/) (native support for [Python in the Lambda roadmap](https://github.com/orgs/aws/projects/286/views/1?pane=issue&itemId=129507898&issue=aws%7Caws-lambda-roadmap%7C39)), so to enable response streaming with an AWS Lambda Python function, we use [Lambda Web Adaptor](https://aws.amazon.com/blogs/compute/using-response-streaming-with-aws-lambda-web-adapter-to-optimize-performance/) and [Fast API](https://github.com/awslabs/aws-lambda-web-adapter/tree/main/examples/fastapi-response-streaming-zip). The Lambda function takes the topic from the API Gateway request, and sends a request to Bedrock, using the `InvokeModelWithResponseStream` call, to generate a bedtime story for that topic.
45+
46+
## Testing
47+
48+
To test response streaming, you can call the API Gateway REST API URL included in the SAM output. You may use `curl` with the `no-buffer` parameter to send in a topic, to which you will receive a bedtime story for. E.g.
49+
```
50+
curl --no-buffer --json '{"topic":"response streaming with AWS serverless"}' https://<abc123.execute-api.us-eas>t-1.amazonaws.com/prod/story
51+
```
52+
53+
## Cleanup
54+
55+
1. Delete the stack
56+
```bash
57+
sam delete
58+
```
59+
1. Confirm the stack has been deleted
60+
```bash
61+
aws cloudformation list-stacks --query "StackSummaries[?contains(StackName,'STACK_NAME')].StackStatus"
62+
```
63+
----
64+
Copyright 2025 Amazon.com, Inc. or its affiliates. All Rights Reserved.
65+
66+
SPDX-License-Identifier: MIT-0

‎apigw-rest-api-lambda-python-response-streaming/apigw-rest-api-lambda-python-response-streaming-pattern.json‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
{
2+
"title": "Response streaming with Amazon API Gateway Rest API and AWS Lambda",
3+
"description": "Creates an API Gateway REST API and Python Lambda function that provides a streaming response from the LLMs in Amazon Bedrock.",
4+
"language": "Python",
5+
"level": "200",
6+
"framework": "AWS SAM",
7+
"introBox": {
8+
"headline": "How it works",
9+
"text": [
10+
"This pattern demonstrates how to use an Amazon API Gateway REST API with response streaming to a AWS Lambda Python function.",
11+
"To enable response streaming with a Lambda Python function, we use Lambda Web Adapter and Fast API."
12+
]
13+
},
14+
"gitHub": {
15+
"template": {
16+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/apigw-rest-api-lambda-python-response-streaming",
17+
"templateURL": "serverless-patterns/apigw-rest-api-lambda-python-response-streaming",
18+
"projectFolder": "apigw-rest-api-lambda-python-response-streaming",
19+
"templateFile": "template.yaml"
20+
}
21+
},
22+
"resources": {
23+
"bullets": [
24+
{
25+
"text": "Responsive APIs with Amazon API Gateway REST API",
26+
"link": "https://docs.aws.amazon.com/apigateway/latest/developerguide/response-transfer-mode.html"
27+
},
28+
{
29+
"text": "AWS Lambda response streaming with Python and Lambda Web Adapter",
30+
"link": "https://docs.aws.amazon.com/lambda/latest/dg/configuration-response-streaming.html"
31+
},
32+
{
33+
"text": "Building responsive APIs with Amazon API Gateway response streaming",
34+
"link": "https://aws.amazon.com/blogs/compute/building-responsive-apis-with-amazon-api-gateway-response-streaming/"
35+
},
36+
{
37+
"text": "Using response streaming with AWS Lambda Web Adapter to optimize performance",
38+
"link": "https://aws.amazon.com/blogs/compute/using-response-streaming-with-aws-lambda-web-adapter-to-optimize-performance/"
39+
},
40+
{
41+
"text": "FastAPI Response Streaming",
42+
"link": "https://github.com/awslabs/aws-lambda-web-adapter/tree/main/examples/fastapi-response-streaming-zip"
43+
}
44+
]
45+
},
46+
"deploy": {
47+
"text": [
48+
"sam deploy"
49+
]
50+
},
51+
"testing": {
52+
"text": [
53+
"See the GitHub repo for detailed testing instructions."
54+
]
55+
},
56+
"cleanup": {
57+
"text": [
58+
"Delete the stack: <code>sam delete</code>."
59+
]
60+
},
61+
"authors": [
62+
{
63+
"name": "Yusuf Mayet",
64+
"image": "https://d2908q01vomqb2.cloudfront.net/9e6a55b6b4563e652a23be9d623ca5055c356940/2021/11/24/Yusuf-mayet-aws.jpg",
65+
"bio": "I am a Solutions Architect at AWS, where I help customers realise that true transformation lies at the intersection of Cloud, DevOps cultural practices, Agile principles, modular and scalable architectures, and efficient team structures.",
66+
"linkedin": "yusufmayet"
67+
}
68+
],
69+
"patternArch": {
70+
"icon1": {
71+
"x": 20,
72+
"y": 50,
73+
"service": "apigw",
74+
"label": "API Gateway REST API"
75+
},
76+
"icon2": {
77+
"x": 50,
78+
"y": 50,
79+
"service": "lambda",
80+
"label": "AWS Lambda"
81+
},
82+
"icon3": {
83+
"x": 80,
84+
"y": 50,
85+
"service": "bedrock",
86+
"label": "Amazon Bedrock"
87+
},
88+
"line1": {
89+
"from": "icon1",
90+
"to": "icon2",
91+
"label": ""
92+
},
93+
"line2": {
94+
"from": "icon2",
95+
"to": "icon3",
96+
"label": ""
97+
}
98+
}
99+
}

‎apigw-rest-api-lambda-python-response-streaming/src/app.py‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
# SPDX-License-Identifier: MIT-0
3+
4+
import boto3
5+
import json
6+
import os
7+
import uvicorn
8+
from fastapi import FastAPI, Request
9+
from fastapi.responses import StreamingResponse
10+
11+
app = FastAPI() #specified in run.sh
12+
bedrock = boto3.client('bedrock-runtime')
13+
14+
15+
@app.post("/story")
16+
async def api_story(request: Request):
17+
body = await request.json()
18+
topic = body.get("topic")
19+
print(f"Topic received: {topic}")
20+
return StreamingResponse(bedrock_stream(topic), media_type="text/html")
21+
22+
23+
def bedrock_stream(topic: str):
24+
instruction = f"""
25+
You are a world class writer. Please write a sweet bedtime story about {topic}.
26+
"""
27+
body = json.dumps({
28+
"anthropic_version": "bedrock-2023-05-31",
29+
"max_tokens": 1024,
30+
"messages": [
31+
{
32+
"role": "user",
33+
"content": instruction,
34+
}
35+
],
36+
})
37+
38+
response = bedrock.invoke_model_with_response_stream(
39+
modelId='global.anthropic.claude-sonnet-4-5-20250929-v1:0', #using Global CRIS Anthropic Claude Sonnet 4.5 on Bedrock
40+
body=body
41+
)
42+
43+
stream = response.get('body')
44+
if stream:
45+
for event in stream:
46+
chunk = event.get('chunk')
47+
if chunk:
48+
message = json.loads(chunk.get("bytes").decode())
49+
if message['type'] == "content_block_delta":
50+
yield message['delta']['text'] or ""
51+
elif message['type'] == "message_stop":
52+
yield "\n"
53+
54+
55+
if __name__ == "__main__":
56+
uvicorn.run(app, host="0.0.0.0", port=int(os.environ.get("PORT", "8080"))) #port specified in run.sh

‎apigw-rest-api-lambda-python-response-streaming/src/requirements.txt‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
boto3
2+
fastapi
3+
pydantic
4+
uvicorn

‎apigw-rest-api-lambda-python-response-streaming/src/run.sh‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
#!/bin/bash
2+
3+
PATH=$PATH:$LAMBDA_TASK_ROOT/bin PYTHONPATH=$PYTHONPATH:/opt/python:$LAMBDA_RUNTIME_DIR exec python -m uvicorn --port=$PORT app:app

‎apigw-rest-api-lambda-python-response-streaming/template.yaml‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
AWSTemplateFormatVersion: '2010-09-09'
2+
Transform: AWS::Serverless-2016-10-31
3+
Description: |
4+
A sample SAM template for streaming Bedrock responses with Lambda Response Streaming and Lambda Web Adapter.
5+
This template includes an response streaming enabled Amazon API Gateway REST API that invokes an AWS Lambda Python function
6+
that uses Lambda Web Adapter with Fast API to enable response streaming.
7+
8+
Resources:
9+
StreamingFunction:
10+
Type: AWS::Serverless::Function
11+
Properties:
12+
Environment:
13+
Variables:
14+
AWS_LAMBDA_EXEC_WRAPPER: /opt/bootstrap #required for Lambda Web Adapter
15+
PORT: 8080 #sets the port to be used in run.sh, for Lambda Web Adapter
16+
AWS_LWA_INVOKE_MODE: RESPONSE_STREAM #set Lambda Web Adapter to enable response streaming for Lambda functions
17+
CodeUri: src/
18+
Handler: run.sh #required for the Lambda Web Adapter
19+
Runtime: python3.14
20+
Architectures:
21+
- arm64 #AWS Graviton for better price performance
22+
Timeout: 60
23+
Tracing: Active
24+
Layers:
25+
- !Sub arn:aws:lambda:${AWS::Region}:753240598075:layer:LambdaAdapterLayerArm64:25 #Lambda Web Adapter Layer for arm64/graviton
26+
Policies:
27+
- Statement:
28+
- Effect: Allow
29+
Action:
30+
- bedrock:InvokeModelWithResponseStream #Lambda function permission to call Bedrock with response streaming
31+
Resource:
32+
- !Sub 'arn:aws:bedrock:${AWS::Region}::foundation-model/anthropic.claude-sonnet-4-5-20250929-v1:0'
33+
- !Sub 'arn:aws:bedrock:${AWS::Region}:${AWS::AccountId}:inference-profile/global.anthropic.claude-sonnet-4-5-20250929-v1:0'
34+
- !Sub 'arn:aws:bedrock:::foundation-model/anthropic.claude-sonnet-4-5-20250929-v1:0'
35+
Events: #connects the Lambda function to the API Gateway StreamingAPI
36+
StreamingApi:
37+
Type: Api
38+
Properties:
39+
RestApiId: !Ref StreamingApi
40+
Path: /story
41+
Method: post
42+
43+
StreamingApi:
44+
Type: AWS::Serverless::Api
45+
Description: |
46+
Please note: This API does not include authentication and should not be used in production.
47+
For securing API Gateway APIs, refer to other serverless patterns for authentication methods.
48+
Properties:
49+
StageName: prod
50+
DefinitionBody:
51+
openapi: 3.0.1
52+
info:
53+
title: Streaming API
54+
version: 1.0.0
55+
paths:
56+
/story:
57+
post:
58+
responses:
59+
'200':
60+
description: Success
61+
content:
62+
application/json:
63+
schema:
64+
type: object
65+
x-amazon-apigateway-integration:
66+
type: aws_proxy
67+
httpMethod: POST
68+
responseTransferMode: "STREAM" #enable response streaming on API Gateway
69+
uri: !Sub 'arn:aws:apigateway:${AWS::Region}:lambda:path/2021-11-15/functions/${StreamingFunction.Arn}/response-streaming-invocations' #the ARN for Lamba is different for response streaming
70+
71+
Outputs:
72+
StreamingApiUrl:
73+
Description: API Gateway endpoint URL
74+
Value: !Sub 'https://${StreamingApi}.execute-api.${AWS::Region}.amazonaws.com/prod/story'

0 commit comments

Comments
 (0)