Skip to content

Commit 8f4d800

Browse files
julianwoodjulianwood
authored
Merge pull request #2553 from kaustavbecs/kaustavbecs-feature-appsync-lambda-bedrock-async-stream-subscription-cdk
New serverless pattern - AppSync Async Bedrock Streaming with Lambda Event Mode
2 parents 432cffd + 9c7faa0 commit 8f4d800

15 files changed

+989
-0
lines changed

‎appsync-lambda-bedrock-async-stream-subscription-cdk/.gitignore‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
*.js
2+
!jest.config.js
3+
*.d.ts
4+
node_modules
5+
6+
# CDK asset staging directory
7+
.cdk.staging
8+
cdk.out

‎appsync-lambda-bedrock-async-stream-subscription-cdk/.npmignore‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
*.ts
2+
!*.d.ts
3+
4+
# CDK asset staging directory
5+
.cdk.staging
6+
cdk.out

‎appsync-lambda-bedrock-async-stream-subscription-cdk/README.md‎

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,131 @@
1+
# Long running invocations of Amazon Bedrock using Amazon AppSync and AWS Lambda streaming
2+
3+
This pattern demonstrates how to implement [long-running invocations](https://docs.aws.amazon.com/appsync/latest/devguide/resolver-reference-bedrock-js.html#long-running-invocations) with Amazon Bedrock using AWS AppSync subscriptions and AWS Lambda in Event Mode, following the official AWS AppSync documentation pattern.
4+
5+
Learn more about this pattern at [Serverless Land Patterns](https://serverlessland.com/patterns/appsync-lambda-bedrock-async-stream-subscription-cdk).
6+
7+
Important: this application uses various AWS services and there are costs associated with these services after the Free Tier usage - please see the [AWS Pricing page](https://aws.amazon.com/pricing/) for details. You are responsible for any AWS costs incurred. No warranty is implied in this example.
8+
9+
## Requirements
10+
11+
* [Create an AWS account](https://portal.aws.amazon.com/gp/aws/developer/registration/index.html) if you do not already have one and log in. The IAM user that you use must have sufficient permissions to make necessary AWS service calls and manage AWS resources.
12+
* [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html) installed and configured
13+
* [Git Installed](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)
14+
* [Node and NPM](https://nodejs.org/en/download/) installed
15+
* [AWS Cloud Development Kit](https://docs.aws.amazon.com/cdk/v2/guide/cli.html) (AWS CDK) installed
16+
* Enable the **Anthropic - Claude Sonnet 3.5 V2** model in **us-east-1** region through the [Bedrock console](https://console.aws.amazon.com/bedrock/home#/modelaccess). This implementation uses the [cross-region inference profile](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-support.html#inference-profiles-support-system) from us-east-1.
17+
18+
## How it works
19+
20+
The pattern implements an asynchronous [streaming architecture](https://docs.aws.amazon.com/appsync/latest/devguide/resolver-reference-bedrock-js.html#long-running-invocations) where:
21+
22+
1. Client initiates a WebSocket subscription and makes a request to AppSync
23+
2. AppSync invokes Lambda function in Event mode, enabling asynchronous processing
24+
3. Lambda function streams responses from Bedrock using ConverseStream
25+
4. Lambda sends updates via mutations to AppSync
26+
5. Updates are delivered to client through WebSocket subscription
27+
28+
![alt text](image.png)
29+
30+
**Key Benefits**
31+
- **Asynchronous Processing**: AppSync immediately returns a response while Lambda processes the request asynchronously, preventing timeouts for long-running operations
32+
- **Real-time Updates**: Clients receive progressive updates through WebSocket subscriptions as the model generates responses
33+
- **Scalable Architecture**: Event-driven design allows handling multiple concurrent requests without blocking
34+
- **Enhanced User Experience**: Progressive updates enable responsive interfaces even during lengthy AI model invocations
35+
36+
## Deployment Instructions
37+
38+
1. Clone the repository:
39+
```sh
40+
git clone https://github.com/aws-samples/serverless-patterns
41+
```
42+
2. Navigate to pattern directory:
43+
```sh
44+
cd appsync-lambda-bedrock-async-stream-subscription-cdk
45+
```
46+
47+
3. Install dependencies:
48+
```sh
49+
npm install
50+
```
51+
52+
4. Bootstrap CDK (if needed):
53+
```sh
54+
cdk bootstrap
55+
```
56+
57+
5. Deploy stack:
58+
```sh
59+
npm run deploy
60+
```
61+
62+
### Important:
63+
Note the GraphQL API URL and API Key from the stack outputs - you'll need these for testing.
64+
65+
## Testing
66+
67+
After deployment, you can test the Bedrock streaming integration using the provided test script. The script demonstrates:
68+
- WebSocket subscription initialization
69+
- Conversation start with Bedrock
70+
- Real-time streaming chunks display
71+
- Graceful cleanup on exit
72+
73+
1. Configure test credentials:
74+
```sh
75+
Open test/test.ts
76+
Replace APPSYNC_API_URL with the API URL from stack outputs
77+
Replace APPSYNC_API_KEY with the API Key from stack outputs
78+
```
79+
80+
2. Run the test:
81+
```sh
82+
npx tsx test/test.ts
83+
```
84+
85+
3. Expected Output:
86+
```sh
87+
Starting subscription...
88+
Starting conversation...
89+
StartConversation response: {
90+
data: {
91+
startConversation: {
92+
conversationId: '123e4567-e89b-12d3-a456-426614174000',
93+
status: 'STARTED'
94+
}
95+
}
96+
}
97+
Received chunk: {
98+
conversationId: '123e4567-e89b-12d3-a456-426614174000',
99+
chunk: "Here's a joke for you: Why don't scientists trust atoms? Because they make"
100+
}
101+
Received chunk: {
102+
conversationId: '123e4567-e89b-12d3-a456-426614174000',
103+
chunk: 'up everything!'
104+
}
105+
```
106+
107+
If you do not receive any response, please check your Bedrock Model access for Claude Sonnet 3.5 V2 in us-east-1 region.
108+
109+
4. Stop the test:
110+
```sh
111+
Press Ctrl+C to terminate the process
112+
```
113+
114+
115+
## Cleanup
116+
117+
1. Delete the stack
118+
```sh
119+
cdk destroy --all
120+
```
121+
122+
## Author bio
123+
Kaustav Dey,
124+
https://www.linkedin.com/in/kaustavbecs/
125+
Solution Architect
126+
127+
----
128+
Copyright 2024 Amazon.com, Inc. or its affiliates. All Rights Reserved.
129+
130+
SPDX-License-Identifier: MIT-0
131+

‎appsync-lambda-bedrock-async-stream-subscription-cdk/appsync-lambda-bedrock-async-stream-subscription-cdk.json‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
{
2+
"title": "Amazon Bedrock calls via AppSync & Lambda streaming for long tasks",
3+
"language": "TypeScript",
4+
"level": "300",
5+
"framework": "CDK",
6+
"introBox": {
7+
"headline": "How it works",
8+
"text": [
9+
"The pattern implements an asynchronous streaming architecture.",
10+
"Client initiates a WebSocket subscription and makes a request to AWS AppSync. AppSync invokes Lambda function in event mode, enabling asynchronous processing.",
11+
"Lambda function streams responses from Amazon Bedrock using ConverseStream. Lambda function sends updates via mutations to AppSync. Updates are delivered to client through WebSocket subscription."
12+
]
13+
},
14+
"gitHub": {
15+
"template": {
16+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/appsync-lambda-bedrock-async-stream-subscription-cdk",
17+
"templateURL": "serverless-patterns/appsync-lambda-bedrock-async-stream-subscription-cdk",
18+
"projectFolder": "appsync-lambda-bedrock-async-stream-subscription-cdk",
19+
"templateFile": "/lib/appsync-lambda-bedrock-async-stream-subscription-cdk-stack.ts"
20+
}
21+
},
22+
"resources": {
23+
"bullets": [
24+
{
25+
"text": "AWS AppSync JavaScript resolver and function reference for Amazon Bedrock runtime",
26+
"link": "https://docs.aws.amazon.com/appsync/latest/devguide/resolver-reference-bedrock-js.html#long-running-invocations"
27+
},
28+
{
29+
"text": "Bedrock ConverseStream API",
30+
"link": "https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html"
31+
}
32+
]
33+
},
34+
"deploy": {
35+
"text": ["cdk deploy"]
36+
},
37+
"testing": {
38+
"text": ["See the GitHub repo for detailed testing instructions."]
39+
},
40+
"cleanup": {
41+
"text": ["Delete the stack: <code>cdk destroy --all</code>."]
42+
},
43+
"authors": [
44+
{
45+
"name": "Kaustav Dey",
46+
"image": "https://avatars.githubusercontent.com/u/13236519",
47+
"bio": "Solution Architect at AWS",
48+
"linkedin": "kaustavbecs"
49+
}
50+
],
51+
"patternArch": {
52+
"icon1": {
53+
"x": 20,
54+
"y": 50,
55+
"service": "appsync",
56+
"label": "AWS AppSync"
57+
},
58+
"icon2": {
59+
"x": 50,
60+
"y": 50,
61+
"service": "lambda",
62+
"label": "AWS Lambda"
63+
},
64+
"icon3": {
65+
"x": 80,
66+
"y": 50,
67+
"service": "bedrock",
68+
"label": "Amazon Bedrock"
69+
},
70+
"line1": {
71+
"from": "icon1",
72+
"to": "icon2",
73+
"label": ""
74+
},
75+
"line2": {
76+
"from": "icon2",
77+
"to": "icon3",
78+
"label": ""
79+
}
80+
}
81+
}

‎appsync-lambda-bedrock-async-stream-subscription-cdk/bin/appsync-lambda-bedrock-async-stream-subscription-cdk.ts‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
#!/usr/bin/env node
2+
import * as cdk from 'aws-cdk-lib';
3+
import { AppsyncLambdaBedrockAsyncStreamSubscriptionCdkStack } from '../lib/appsync-lambda-bedrock-async-stream-subscription-cdk-stack';
4+
5+
const app = new cdk.App();
6+
new AppsyncLambdaBedrockAsyncStreamSubscriptionCdkStack(app, 'AppsyncLambdaBedrockAsyncStreamSubscriptionCdkStack', {
7+
/* If you don't specify 'env', this stack will be environment-agnostic.
8+
* Account/Region-dependent features and context lookups will not work,
9+
* but a single synthesized template can be deployed anywhere. */
10+
11+
/* Uncomment the next line to specialize this stack for the AWS Account
12+
* and Region that are implied by the current CLI configuration. */
13+
// env: { account: process.env.CDK_DEFAULT_ACCOUNT, region: process.env.CDK_DEFAULT_REGION },
14+
15+
/* Uncomment the next line if you know exactly what Account and Region you
16+
* want to deploy the stack to. */
17+
// env: { account: '123456789012', region: 'us-east-1' },
18+
19+
/* For more information, see https://docs.aws.amazon.com/cdk/latest/guide/environments.html */
20+
});

‎appsync-lambda-bedrock-async-stream-subscription-cdk/cdk.json‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
{
2+
"app": "npx ts-node --prefer-ts-exts bin/appsync-lambda-bedrock-async-stream-subscription-cdk.ts",
3+
"watch": {
4+
"include": [
5+
"**"
6+
],
7+
"exclude": [
8+
"README.md",
9+
"cdk*.json",
10+
"**/*.d.ts",
11+
"**/*.js",
12+
"tsconfig.json",
13+
"package*.json",
14+
"yarn.lock",
15+
"node_modules",
16+
"test"
17+
]
18+
},
19+
"context": {
20+
"@aws-cdk/aws-lambda:recognizeLayerVersion": true,
21+
"@aws-cdk/core:checkSecretUsage": true,
22+
"@aws-cdk/core:target-partitions": [
23+
"aws",
24+
"aws-cn"
25+
],
26+
"@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
27+
"@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
28+
"@aws-cdk/aws-ecs:arnFormatIncludesClusterName": true,
29+
"@aws-cdk/aws-iam:minimizePolicies": true,
30+
"@aws-cdk/core:validateSnapshotRemovalPolicy": true,
31+
"@aws-cdk/aws-codepipeline:crossAccountKeyAliasStackSafeResourceName": true,
32+
"@aws-cdk/aws-s3:createDefaultLoggingPolicy": true,
33+
"@aws-cdk/aws-sns-subscriptions:restrictSqsDescryption": true,
34+
"@aws-cdk/aws-apigateway:disableCloudWatchRole": true,
35+
"@aws-cdk/core:enablePartitionLiterals": true,
36+
"@aws-cdk/aws-events:eventsTargetQueueSameAccount": true,
37+
"@aws-cdk/aws-ecs:disableExplicitDeploymentControllerForCircuitBreaker": true,
38+
"@aws-cdk/aws-iam:importedRoleStackSafeDefaultPolicyName": true,
39+
"@aws-cdk/aws-s3:serverAccessLogsUseBucketPolicy": true,
40+
"@aws-cdk/aws-route53-patters:useCertificate": true,
41+
"@aws-cdk/customresources:installLatestAwsSdkDefault": false,
42+
"@aws-cdk/aws-rds:databaseProxyUniqueResourceName": true,
43+
"@aws-cdk/aws-codedeploy:removeAlarmsFromDeploymentGroup": true,
44+
"@aws-cdk/aws-apigateway:authorizerChangeDeploymentLogicalId": true,
45+
"@aws-cdk/aws-ec2:launchTemplateDefaultUserData": true,
46+
"@aws-cdk/aws-secretsmanager:useAttachedSecretResourcePolicyForSecretTargetAttachments": true,
47+
"@aws-cdk/aws-redshift:columnId": true,
48+
"@aws-cdk/aws-stepfunctions-tasks:enableEmrServicePolicyV2": true,
49+
"@aws-cdk/aws-ec2:restrictDefaultSecurityGroup": true,
50+
"@aws-cdk/aws-apigateway:requestValidatorUniqueId": true,
51+
"@aws-cdk/aws-kms:aliasNameRef": true,
52+
"@aws-cdk/aws-autoscaling:generateLaunchTemplateInsteadOfLaunchConfig": true,
53+
"@aws-cdk/core:includePrefixInUniqueNameGeneration": true,
54+
"@aws-cdk/aws-efs:denyAnonymousAccess": true,
55+
"@aws-cdk/aws-opensearchservice:enableOpensearchMultiAzWithStandby": true,
56+
"@aws-cdk/aws-lambda-nodejs:useLatestRuntimeVersion": true,
57+
"@aws-cdk/aws-efs:mountTargetOrderInsensitiveLogicalId": true,
58+
"@aws-cdk/aws-rds:auroraClusterChangeScopeOfInstanceParameterGroupWithEachParameters": true,
59+
"@aws-cdk/aws-appsync:useArnForSourceApiAssociationIdentifier": true,
60+
"@aws-cdk/aws-rds:preventRenderingDeprecatedCredentials": true,
61+
"@aws-cdk/aws-codepipeline-actions:useNewDefaultBranchForCodeCommitSource": true,
62+
"@aws-cdk/aws-cloudwatch-actions:changeLambdaPermissionLogicalIdForLambdaAction": true,
63+
"@aws-cdk/aws-codepipeline:crossAccountKeysDefaultValueToFalse": true,
64+
"@aws-cdk/aws-codepipeline:defaultPipelineTypeToV2": true,
65+
"@aws-cdk/aws-kms:reduceCrossAccountRegionPolicyScope": true,
66+
"@aws-cdk/aws-eks:nodegroupNameAttribute": true,
67+
"@aws-cdk/aws-ec2:ebsDefaultGp3Volume": true,
68+
"@aws-cdk/aws-ecs:removeDefaultDeploymentAlarm": true,
69+
"@aws-cdk/custom-resources:logApiResponseDataPropertyTrueDefault": false,
70+
"@aws-cdk/aws-s3:keepNotificationInImportedBucket": false,
71+
"@aws-cdk/aws-ecs:reduceEc2FargateCloudWatchPermissions": true,
72+
"@aws-cdk/aws-dynamodb:resourcePolicyPerReplica": true,
73+
"@aws-cdk/aws-ec2:ec2SumTImeoutEnabled": true,
74+
"@aws-cdk/aws-appsync:appSyncGraphQLAPIScopeLambdaPermission": true,
75+
"@aws-cdk/aws-rds:setCorrectValueForDatabaseInstanceReadReplicaInstanceResourceId": true,
76+
"@aws-cdk/core:cfnIncludeRejectComplexResourceUpdateCreatePolicyIntrinsics": true,
77+
"@aws-cdk/aws-lambda-nodejs:sdkV3ExcludeSmithyPackages": true,
78+
"@aws-cdk/aws-stepfunctions-tasks:fixRunEcsTaskPolicy": true,
79+
"@aws-cdk/aws-ec2:bastionHostUseAmazonLinux2023ByDefault": true
80+
}
81+
}

‎appsync-lambda-bedrock-async-stream-subscription-cdk/example.pattern.json‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
{
2+
"title": "Long running invocations of Amazon Bedrock using Amazon AppSync and AWS Lambda streaming",
3+
"language": "Typescript",
4+
"level": "300",
5+
"framework": "CDK",
6+
"introBox": {
7+
"headline": "How it works",
8+
"text": [
9+
"The pattern implements an asynchronous streaming architecture.",
10+
"Client initiates a WebSocket subscription and makes a request to AppSync. AppSync invokes Lambda function in Event mode, enabling asynchronous processing.",
11+
"Lambda function streams responses from Bedrock using ConverseStream. Lambda sends updates via mutations to AppSync. Updates are delivered to client through WebSocket subscription"
12+
]
13+
},
14+
"gitHub": {
15+
"template": {
16+
"repoURL": "https://github.com/aws-samples/serverless-patterns/tree/main/appsync-lambda-bedrock-async-stream-subscription-cdk",
17+
"templateURL": "serverless-patterns/appsync-lambda-bedrock-async-stream-subscription-cdk",
18+
"projectFolder": "appsync-lambda-bedrock-async-stream-subscription-cdk",
19+
"templateFile": "lib/appsync-lambda-bedrock-async-stream-subscription-cdk-stack.ts"
20+
}
21+
},
22+
"resources": {
23+
"bullets": [
24+
{
25+
"text": "AWS AppSync JavaScript resolver and function reference for Amazon Bedrock runtime",
26+
"link": "https://docs.aws.amazon.com/appsync/latest/devguide/resolver-reference-bedrock-js.html#long-running-invocations"
27+
},
28+
{
29+
"text": "Bedrock ConverseStream API",
30+
"link": "https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_ConverseStream.html"
31+
}
32+
]
33+
},
34+
"deploy": {
35+
"text": [
36+
"cdk deploy"
37+
]
38+
},
39+
"testing": {
40+
"text": [
41+
"See the GitHub repo for detailed testing instructions."
42+
]
43+
},
44+
"cleanup": {
45+
"text": [
46+
"Delete the stack: <code>cdk destroy --all</code>."
47+
]
48+
},
49+
"authors": [
50+
{
51+
"name": "Kaustav Dey",
52+
"image": "https://avatars.githubusercontent.com/u/13236519",
53+
"bio": "Solution Architect at AWS",
54+
"linkedin": "https://www.linkedin.com/in/kaustavbecs/"
55+
}
56+
]
57+
}
58+

‎appsync-lambda-bedrock-async-stream-subscription-cdk/image.png‎

213 KB
Loading

0 commit comments

Comments
 (0)