Skip to content

Commit 0c03e33

Browse files
committed
inference profiles
1 parent 5fb24f7 commit 0c03e33

File tree

1 file changed

+13
-0
lines changed

1 file changed

+13
-0
lines changed

integrations/llms/bedrock/aws-bedrock.mdx

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -636,6 +636,19 @@ Note that you will have to set [`strict_open_ai_compliance=False`](/product/ai-g
636636
[Inference profiles](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles.html) are a resource in Amazon Bedrock that define a model and one or more Regions to which the inference profile can route model invocation requests.
637637

638638
To use inference profiles, your IAM role needs to have the following permissions:
639+
```json
640+
"Effect": "Allow",
641+
"Action": [
642+
"bedrock:GetInferenceProfile"
643+
],
644+
"Resource": [
645+
"arn:aws:bedrock:*:*:inference-profile/*",
646+
"arn:aws:bedrock:*:*:application-inference-profile/*"
647+
]
648+
```
649+
This is a pre-requisite for using inference profiles, as the gateway needs to fetch the foundation model to process the request.
650+
651+
For reference, see the following documentation:
639652
https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-prereq.html
640653

641654

0 commit comments

Comments
 (0)