Users need the ability to use custom ARNs (Amazon Resource Names) with AWS Bedrock for intelligent prompt routing.

[Smartsheet-JB-Brown]

Add support for custom ARNs in AWS Bedrock

Problem

Users need the ability to use custom ARNs (Amazon Resource Names) with AWS Bedrock for intelligent prompt routing. This is particularly useful for:

1. Using AWS Bedrock Intelligent Prompt Routing to dynamically route prompts to different models
2. Using custom models deployed in AWS Bedrock
3. Using provisioned throughput models
4. Accessing models through inference profiles

Currently, when attempting to use a custom ARN, users encounter the error:

You don't have access to the model with the specified model ID.

Solution

This PR adds support for custom ARNs in AWS Bedrock by:

1. Adding a "Use custom ARN..." option to the model dropdown for Bedrock in the UI
2. Adding a text field for entering the custom ARN when this option is selected
3. Adding validation for the ARN format and a warning if the region in the ARN doesn't match the selected AWS region
4. Enhancing the Bedrock provider to handle custom ARNs properly:
   • Skip cross-region inference for custom ARNs
   • Use the ARN directly as the model ID
   • Set appropriate model info for custom ARNs
5. Creating a test script for verifying AWS Bedrock Intelligent Prompt Routing functionality

Implementation Details

UI Changes

• Added a "Use custom ARN..." option to the model dropdown for Bedrock
• Added a text field for entering the custom ARN when "custom-arn" is selected
• Added validation for the ARN format with a regex pattern
• Added a warning if the region in the ARN doesn't match the selected AWS region

Backend Changes

• Added awsCustomArn to the API handler options and global state keys
• Enhanced the getModel method to handle custom ARNs
• Modified the createMessage and completePrompt methods to skip cross-region inference for custom ARNs

Testing

• Added tests for the custom ARN functionality
• Created a test script (test-custom-arn.js) for verifying AWS Bedrock Intelligent Prompt Routing:
  • Supports both standard model IDs and custom ARNs
  • Takes AWS profile as an input parameter via the AWS_PROFILE environment variable
  • Provides detailed error messages for troubleshooting
  • Usage examples:

    # Test with a standard model ID
    AWS_PROFILE="your-profile" CUSTOM_ARN="anthropic.claude-3-haiku-20240307-v1:0" node test-custom-arn.js
    
    # Test with a prompt router ARN
    AWS_PROFILE="your-profile" CUSTOM_ARN="arn:aws:bedrock:region:account-id:default-prompt-router/router-name" node test-custom-arn.js
    

Remaining Issue

• AWS API for intelligent prompt router ARN isn't returning costs, as such the cost label in Roo will not properly increment. Working with AWS account team to surface the bug to Bedrock engineering.

If you encounter errors while testing:

You may no have access to the ARN in the specified region. The AWS console may not make this obvious. It didn't for me and I only discovered
through testing.

Things to verify:

1. AWS permissions - The user's AWS credentials might not have access to the specified model
2. ARN format - The ARN might not be correctly formatted or might be for a resource that doesn't exist
3. Region mismatch - The region in the ARN might not match the region where the model is deployed

Implemented Fixes

This PR includes the following fixes to help diagnose and resolve the issue:

1. Enhanced Error Handling: Added specific error messages for common ARN-related issues:

   • Permission errors (when you don't have access to the model)
   • Invalid ARN format or non-existent model errors
   • Other ARN-related errors

2. Improved Logging: Added detailed logging when using custom ARNs:

   • Logs the custom ARN being used
   • Logs the complete payload being sent to AWS Bedrock
   • Logs detailed error information when errors occur

3. User-Friendly Error Messages: Provides more specific guidance to users when errors occur:

   • Suggests checking permissions
   • Recommends verifying ARN format
   • Advises ensuring region consistency

4. Test Script for Prompt Routing: Created a dedicated test script for verifying AWS Bedrock Intelligent Prompt Routing:

   • Allows testing with both standard model IDs and custom ARNs
   • Configurable AWS profile via environment variable
   • Provides detailed error messages for common issues
   • Helps users verify their AWS credentials and ARN configurations

These changes will help users diagnose and fix issues with custom ARNs more easily.

---

Important

Adds support for custom ARNs in AWS Bedrock, including UI, backend changes, and tests for intelligent prompt routing.

• Behavior:
  • Adds support for custom ARNs in AWS Bedrock, allowing users to specify custom models and handle intelligent prompt routing.
  • UI changes in ApiOptions.tsx to include a dropdown option for "Use custom ARN..." and a text field for ARN input.
  • Backend changes in bedrock.ts to handle custom ARNs, including region extraction and validation.
  • Enhanced error handling and logging for ARN-related issues.
• Testing:
  • Adds bedrock-custom-arn.test.ts for testing custom ARN functionality.
  • Includes test-custom-arn.js script for manual testing of custom ARNs.
• Misc:
  • Updates api.ts and globalState.ts to include awsCustomArn in configuration keys.

^{This description was created by for 3eb8cd8d0cf81070a73071a6f329c70950c9dccb. It will automatically update as commits are pushed.}

[changeset-bot]

⚠️ No Changeset found

Latest commit: 77186e8

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[Smartsheet-JB-Brown]

Knip is failing because of the utility js test script for custom ARNs. Let me know if you want me to handle it and how. Happy to, just need direction.

[mrubens]

Looks good to me! I just fixed one test and removed the test script. Thanks for the contribution 🚀