Skip to content

Commit 8767fb3

Browse files
Merge pull request #356 from Portkey-AI/chore/inference-profiles
2 parents 67d1f6a + 0c03e33 commit 8767fb3

File tree

1 file changed

+21
-0
lines changed

1 file changed

+21
-0
lines changed

integrations/llms/bedrock/aws-bedrock.mdx

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -631,6 +631,27 @@ Note that you will have to set [`strict_open_ai_compliance=False`](/product/ai-g
631631
```
632632
</CodeGroup>
633633

634+
## Inference Profiles
635+
636+
[Inference profiles](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles.html) are a resource in Amazon Bedrock that define a model and one or more Regions to which the inference profile can route model invocation requests.
637+
638+
To use inference profiles, your IAM role needs to have the following permissions:
639+
```json
640+
"Effect": "Allow",
641+
"Action": [
642+
"bedrock:GetInferenceProfile"
643+
],
644+
"Resource": [
645+
"arn:aws:bedrock:*:*:inference-profile/*",
646+
"arn:aws:bedrock:*:*:application-inference-profile/*"
647+
]
648+
```
649+
This is a pre-requisite for using inference profiles, as the gateway needs to fetch the foundation model to process the request.
650+
651+
For reference, see the following documentation:
652+
https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-prereq.html
653+
654+
634655
## Bedrock Converse API
635656

636657
Portkey uses the [AWS Converse API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) internally for making chat completions requests.

0 commit comments

Comments
 (0)