Virtual Try-On on AWS

Table of Contents

Required

1. Overview
   • Architecture Diagram
   • Cost
2. Prerequisites
3. Deployment Steps
4. Deployment Validation
5. Running the Guidance
6. Next Steps
7. Cleanup
8. FAQ, known issues, additional considerations, and limitations
9. Notices
10. Authors

Overview

This Guidance addresses a critical challenge in online retail: every fourth piece of clothing bought online is returned, contributing to America's $890 billion returns problem in 2024. Poor fit (65%), wrong size, and style mismatch are the primary drivers. Each return not only increases processing costs but adds 30% more carbon emissions than the original delivery. Virtual Try-On technology directly addresses these challenges by letting customers accurately visualize products before purchase.

The solution leverages Amazon Nova Canvas with its virtual try-on capability, which requires just two 2D inputs: a source image and product reference. The system combines automatic product placement through auto-masking with manual controls, while carefully preserving crucial details like logos, textures, and patterns during the try-on process.

Architecture Diagram

Architecture Flow:

1. Amazon S3 stores the uploaded customer model photos and product images.
2. Each upload generates a message sent to an Amazon SQS queue. The lambda creates the corresponding metadata, S3 path and stores it in the Amazon DynamoDB product table for later retrieval.
3. Amazon API Gateway manages the WebSocket connections for real-time status updates between the client and the VTO.
4. AWS Lambda processes initial requests by retrieving product information in the Amazon DynamoDB product table, and creating job entries in DynamoDB. Amazon DynamoDB:
5. The products table (vto- products) stores catalog items available for the VTO, notably the Amazon S3 picture location.
6. The Virtual Try-On Jobs DynamoDB Table (vto-jobs) tracks the state of each try-on request.
7. Amazon DynamoDB Streams asynchronously triggers AWS Step Functions workflows on job creation for processing try-on requests.
8. AWS Step Functions orchestrates the VTO generation. It triggers a Lambda function that calls the Amazon Nova Canvas model through Amazon Bedrock. The DynamoDB job table is updated with the VTO status.
9. Amazon S3 stores the generated try-on images with job ID metadata.
10. Amazon SQS handles Amazon S3 event notifications for completed try-on images.
11. AWS Lambda Function sends the Amazon S3 URL of the result back to the user via WebSocket.

Cost

You are responsible for the cost of the AWS services used while running this Guidance. As of January 2025, the cost for running this Guidance with the default settings in the US East (N. Virginia) Region is approximately $1,420 per month for processing 1,000,000 virtual try-on requests.

We recommend creating a Budget through AWS Cost Explorer to help manage costs. Prices are subject to change. For full details, refer to the pricing webpage for each AWS service used in this Guidance.

Sample Cost Table

For a single VTO query the total cost is $0.0403686. The following table details the price of a single query in dollars ($):

AWS Service	Price per single query in $	Link for more pricing information
Amazon Bedrock - Nova Canvas Standard 1024x1024	0.04	https://aws.amazon.com/bedrock/pricing/
Amazon Simple Storage Service (Amazon S3) - Storage(*)	0.0000152	https://aws.amazon.com/s3/pricing/
Amazon API Gateway	0.00000313	https://aws.amazon.com/api-gateway/pricing/
AWS Lambda(**)	0.0002229	https://aws.amazon.com/lambda/pricing/
Amazon DynamoDB (***)	0.0000104	https://aws.amazon.com/dynamodb/pricing/
AWS Step Functions	0.000124	https://aws.amazon.com/step-functions/pricing/
Amazon Simple Queue Service (Amazon SQS)(****)	0.0000016	https://aws.amazon.com/sqs/pricing/?nc1=h_ls
Total	0.0403586

Notes:

• *3M PUT & 3M SELECT. Outbound costs are taken into account.
• **Five Lambda invocations: 1 lambda runs for approximately 17 seconds and 4 lambda run for 1 sec each
• ***1 product stored in the vto-products table. 1 model stored in the vto-models table. 1 job stored in the vto-jobs table. 1 connection in the vto-connections table. Writes on the Job table are not deleted
• ****4 operations in total
• *****First 50,000 MAU (Monthly Active Users) are free with full features
• ******WAF cost includes Web ACL ($5/month), 2 rules ($2/month), and request processing ($0.60/1M requests) amortized per query

Prerequisites

Operating System

These deployment instructions are optimized to best work on Amazon Linux 2023 AMI or macOS. Deployment in another OS may require additional steps.

Required packages and tools:

• Python 3.12 or higher
• Node.js 18.19 or higher
• npm 8.x or higher
• AWS CLI v2
• Git

Tools

• AWS CDK v2.x or higher
• wscat (for WebSocket testing if needed)

Install commands:

# Install AWS CDK globally
npm install -g aws-cdk

# Install wscat for testing
npm install -g wscat

AWS account requirements

• AWS account with appropriate IAM permissions for creating resources
• Amazon Nova Canvas model access enabled in your account
• AWS CLI configured with credentials

aws cdk bootstrap

This Guidance uses aws-cdk. If you are using aws-cdk for the first time, please perform the following bootstrapping:

cdk bootstrap aws://ACCOUNT-NUMBER/REGION

Supported Regions

This Guidance uses the Amazon Canvas VTO model hosted on Amazon Bedrock, which is not currently available in all AWS Regions. You must launch this solution in an AWS Region where Amazon Nova Canvas model is available. For the most current availability, refer to the AWS Regional Services List.

Deployment Steps

1. Clone the repo using command:

   git clone https://github.com/aws-solutions-library-samples/guidance-for-virtual-try-ons-on-aws.git
   cd guidance-for-virtual-try-ons-on-aws

2. Navigate to the CDK directory:

   cd deployment/cdk

3. Create and activate a Python virtual environment:

   python3 -m venv venv
   source venv/bin/activate  
   
   

4. Install dependencies

   pip install -r requirements.txt

5. Verify installation

   cdk --version  # Should show 2.208.0 or higher

6. Deploy the CDK stack:

   cdk deploy

   Note: This deployment takes approximately 20 minutes.

7. Capture the CDK outputs including:

   • BucketName
   • WebSocketApiEndpoint
   • WebSocketApiKey (API Key ID)
   • WebSocketApiId

8. Get the actual API key value using the API Key ID from step 6:

   aws apigateway get-api-key --api-key <API_KEY_ID> --include-value --query 'value' --output text

9. (OPTIONAL) Deploy the S3 Storage Browser Frontend:

   The frontend provides a web interface for uploading and managing model/product images. To deploy it:

   a. Update the configuration file deployment/s3-browser-frontend/amplify/backend.ts:

   // Replace this line:
   bucketArn: 'arn:aws:s3:::vto-demo-bucket-{your-account-number}-{region}',
   // With actual bucket name from CDK output:
   bucketArn: 'arn:aws:s3:::vto-demo-bucket-123456789012-us-east-1',
   // And this with the region you used
   region: '{region}'

   b. Install dependencies and deploy the frontend:

   cd deployment/s3-browser-frontend
   npm install
   npm install @aws-amplify/backend-cli
   npx ampx sandbox

   Once deployment is complete, you can exit the cmd (CTRL+C).

   ✔️ Backend synthesized in 1.65 seconds
   ✔️  Type checks completed in 4.65 seconds
   ✔️  Built and published assets
   ✔️  Deployment completed in 86.549 seconds
   [Sandbox] Watching for file changes...
   File written: amplify_outputs.json

   This creates a temporary Amplify environment with S3 storage browser capabilities for easy file uploads. The npm run dev command starts the local development server, accessible at http://localhost:5173.

   c. Start the development server:

   npm run dev

   Once connected to the frontend you should get an Amazon Cognito interface inviting you to sign in. Create an account to sign in and access the bucket interface.

   Frontend Interface:

   The frontend provides an intuitive interface showing the S3 bucket structure with separate folders for models and products.

Deployment Validation

To validate successful deployment: Replace your-api-id, region, & api key value with your outputs

1. Test WebSocket connectivity:

   wscat -c "wss://{your-api-id}.execute-api.{region}.amazonaws.com/prod" \
     -H "x-api-key: your-actual-api-key-value"

   Expected: Connected successfully

Running the Guidance

Guidance inputs

1. Product images: Upload product images to S3 bucket under /products/ prefix
2. Model image: Customer photo or model image
3. WebSocket message: JSON request with action and image URLs

Upload sample products

We provide a model.png and black-top.png model and product examples in the assets folder which you can use to test the solution.

You first need to upload the assets to the S3 Bucket so they can be referenced in the API call. You can use the optional frontend S3 Storage Browser frontend to perform bulk uploads or upload manually via CLI:

Using the Frontend Interface:

Navigate to the products folder and click the upload button to add your product images.

After uploading the black-top.png to the products folder, repeat the same process in the models folder for the model image.

Alternatively, you can use the AWS CLI if you prefer: Replace with your bucket names

# Upload model images to S3 (use descriptive IDs as filenames)
aws s3 cp assets/model.png s3://your-bucket-name/models/model.png
# Upload product images to S3 (use descriptive IDs as filenames)
aws s3 cp assets/black-top.png s3://your-bucket-name/products/black-top.png

Test virtual try-on via WebSocket

Step 1: Connect to WebSocket Replace your-api-id, region, & api key value with your outputs

wscat -c "wss://{your-api-id}.execute-api.{us-east-1}.amazonaws.com/prod" -H "x-api-key: your-actual-api-key"

Step 2: Send VTO request (once connected)

{"action": "create_job","model_id": "model","product_id": "black-top","parameters": {"maskType": "GARMENT","mergeStyle": "BALANCED", "garmentClass": "UPPER_BODY","quality": "standard","cfgScale": 4}}

API Response Example:

The API will return a job ID immediately, and after a few seconds of processing, you'll receive an S3 pre-signed URL with the virtual try-on result.

WebSocket API Actions

The WebSocket API supports the following actions:

1. Ping Test

Test WebSocket connection health:

{"action": "ping"}

Response: {"action": "pong", "timestamp": "2025-01-15T10:30:00.000Z"}

2. Create VTO Job

Create a new virtual try-on job:

{
  "action": "create_job",
  "model_id": "model",
  "product_id": "black-top",
  "parameters": {
    "maskType": "GARMENT",
    "mergeStyle": "BALANCED",
    "garmentClass": "UPPER_BODY",
    "quality": "standard",
    "cfgScale": 4.0
  }
}

Response: {"action": "job_created", "jobId": "uuid", "status": "created", "message": "VTO job created successfully"}

3. Get Products List

Retrieve all available products from the catalog:

{"action": "get_products"}

Response: {"action": "products_list", "products": [...], "count": 5}

4. Subscribe to Job Updates

Subscribe to real-time updates for a specific job:

{"action": "subscribe", "jobId": "your-job-id-here"}

Response: {"action": "subscribed", "jobId": "your-job-id", "message": "Subscribed to job updates"}

5. Unsubscribe from Job Updates

Stop receiving updates for a specific job:

{"action": "unsubscribe", "jobId": "your-job-id-here"}

Response: {"action": "unsubscribed", "jobId": "your-job-id", "message": "Unsubscribed from job updates"}

Error Responses

All actions may return error responses:

{"action": "error", "error": "Description of what went wrong"}

API Parameters

Required Parameters:

• action: "create_job" - Action type
• model_id: String (1-50 chars, alphanumeric + underscore/dash) - ID of the model image
• product_id: String (1-50 chars, alphanumeric + underscore/dash) - ID of the product image

Optional VTO Parameters: All parameters are optional and will use defaults if not specified:

• maskType: String - Type of mask to generate

  • "GARMENT" (default) - Automatically detects and masks garment areas

• mergeStyle: String - How to blend the garment with the model

  • "BALANCED" (default) - Balanced blending approach

• garmentClass: String - Specific garment type

  • "UPPER_BODY" (default) - For tops, shirts, jackets, etc.
  • "LOWER_BODY" - For pants, skirts, shorts, etc.
  • "FULL_BODY" - For dresses, jumpsuits, etc.

  Check the Amazon Nova Canvas documentation for full parameter list.

• quality: String - Generation quality level

  • "standard" (default) - Standard quality generation
  • "premium" - Higher quality (longer processing time)

• cfgScale: Number - CFG scale for generation control (1.0-20.0)

  • 4.0 (default) - Balanced creativity vs adherence to input

File Organization:

• Model images: s3://bucket-name/models/{model_id}.jpg
• Product images: s3://bucket-name/products/{product_id}.jpg
• Results: s3://bucket-name/results/{job_id}-{timestamp}.jpg

Please refer to the Amazon Nova Canvas documentation to read more about the available parameters.

Expected output

{"status": "completed", "jobId": "test-123", "resultUrl": "s3://bucket/results/test-123-1234567890.jpg", "message": "Try-on ready for viewing"}

The entire process typically takes 7-11 seconds end-to-end.

Output description

The final output is a generated image showing the model wearing the selected product, with preserved brand elements and fabric patterns. The result URL provides access to download or display the virtual try-on image.

Virtual Try-On Results:

Model Image	Product Image	Virtual Try-On Result

The system seamlessly combines the model pose and body with the selected garment, maintaining realistic lighting, shadows, and fabric textures.

Next Steps

To enhance this Guidance for your specific requirements:

1. Implement custom product catalog: Replace the sample products table with your actual product inventory including metadata like sizes, colors, and descriptions.

2. Add custom masking strategy: Implement custom masking algorithms for specific garment types or accessories.

Cleanup

To remove all resources created by this Guidance:

1. Empty the S3 bucket (required before deletion):

   aws s3 rm s3://vto-demo-bucket-{account-id}-{region} --recursive
   aws s3 rm s3://vto-access-logs-{account-id}-{region} --recursive

2. Navigate to the CDK directory:

   cd deployment/cdk

3. Destroy the CDK stack:

   cdk destroy

4. Confirm deletion when prompted by typing y.

5. Remove the frontend resources:

   cd deployment/s3-browser-frontend
   npx ampx sandbox delete

Note: This will delete all resources including DynamoDB tables, Lambda functions, API Gateway endpoints, and S3 buckets. Ensure you have backed up any important data before proceeding.

FAQ, known issues, additional considerations, and limitations (optional)

Known issues

1. WebSocket connection timeout: If idle for more than 10 minutes, the WebSocket connection will close. Implement reconnection logic in production applications.

2. Large image processing: Images larger than 5MB may cause timeout issues. Resize images before uploading for optimal performance.

Additional considerations

• This Guidance creates public API endpoints protected by API key authentication. For production use, implement additional security measures such as request throttling and IP whitelisting and JWT Tokens authentication.
• The solution processes one try-on request at a time per connection. For bulk processing, implement queuing mechanisms.
• Amazon Nova Canvas model pricing is usage-based. Monitor your usage to manage costs effectively.

For any feedback, questions, or suggestions, please use the issues tab under this repo.

Notices

Customers are responsible for making their own independent assessment of the information in this Guidance. This Guidance: (a) is for informational purposes only, (b) represents AWS current product offerings and practices, which are subject to change without notice, and (c) does not create any commitments or assurances from AWS and its affiliates, suppliers or licensors. AWS products or services are provided "as is" without warranties, representations, or conditions of any kind, whether express or implied. AWS responsibilities and liabilities to its customers are controlled by AWS agreements, and this Guidance is not part of, nor does it modify, any agreement between AWS and its customers.

Authors

• Kevin Polossat
• Leopold Cheval
• Rania Khemiri
• Amandine Annoye

[1] Title: "Virtual Try-On Implementation Guide" URL: https://docs.aws.amazon.com/prescriptive-guidance/latest/virtual-try-on/ Sections: Overview, Features, Use Cases, Architecture, Cost, Security, Deployment, Contributors