Skip to content

Commit 7773b41

Browse files
pagezyhfpagezyhf
committed
bedrock quickstart
1 parent e0ec150 commit 7773b41

File tree

1 file changed

+54
-81
lines changed

1 file changed

+54
-81
lines changed

‎docs/sagemaker/tutorials/bedrock/bedrock-quickstart.md‎

Lines changed: 54 additions & 81 deletions
Original file line numberDiff line numberDiff line change
@@ -1,103 +1,76 @@
1-
# Use Hugging Face models with Amazon Bedrock
1+
# Quickstart — Using Hugging Face Models with Amazon Bedrock Marketplace
22

3-
We are excited to announce that popular open models from Hugging Face are now available on Amazon Bedrock in the new Bedrock Marketplace! AWS customers can now deploy [83 open models](https://us-east-1.console.aws.amazon.com/bedrock/home?region=us-east-1#/model-catalog) with Bedrock Marketplace to build their Generative AI applications.
3+
## Why use Bedrock Marketplace for Hugging Face models?
4+
Amazon Bedrock now exposes 83 Hugging Face open-weight models—including Gemma, Llama 3, Mistral, and more—through a single catalog. You invoke them with the same Bedrock APIs you already use for Titan, Anthropic, Cohere, etc. Under the hood, Bedrock Marketplace model endpoints are managed by Amazon SageMaker AI. With Bedrock Marketplace, you can now combine the ease of use of SageMaker JumpStart with the fully managed infrastructure of Amazon Bedrock, including compatibility with high-level APIs such as Agents, Knowledge Bases, Guardrails and Model Evaluations.
45

5-
Under the hood, Bedrock Marketplace model endpoints are managed by Amazon Sagemaker Jumpstart. With Bedrock Marketplace, you can now combine the ease of use of SageMaker JumpStart with the fully managed infrastructure of Amazon Bedrock, including compatibility with high-level APIs such as Agents, Knowledge Bases, Guardrails and Model Evaluations.
6+
## 1 . Prerequisites
67

7-
When registering your Sagemaker Jumpstart endpoints in Amazon Bedrock, you only pay for the Sagemaker compute resources and regular Amazon Bedrock APIs prices are applicable.
8+
|  | Requirement | Notes |
9+
|---|-------------|-------|
10+
| **AWS account in a Bedrock Region** | Marketplace is regional; switch the console to one of the 14 supported Regions first. |
11+
| **Permissions** | For a quick trial, attach **AmazonBedrockFullAccess** and **AmazonSageMakerFullAccess**.|
12+
| **Service quotas** | The SageMaker endpoint uses GPU instances (for example **ml.g5**). Verify you have quota or request it. |
13+
| **JumpStart‑only** | If you choose path B, create a **SageMaker Studio domain** and **user profile** first (Console ▸ SageMaker ▸ Domains). Open Studio before continuing. |
814

9-
In this blog we will show you how to deploy [Gemma 2 27B Instruct](https://huggingface.co/google/gemma-2-27b-it) and use the model with Amazon Bedrock APIs. Learn how to:
15+
When registering your Sagemaker Jumpstart endpoints in Amazon Bedrock, you only pay for the SageMaker compute resources and regular Amazon Bedrock APIs prices are applicable.
1016

11-
1. Deploy Google Gemma 2 27B Instruct
12-
2. Send requests using the Amazon Bedrock APIs
13-
3. Clean Up
17+
## 2. Endpoint deployment
1418

15-
## Deploy Google Gemma 2 27B Instruct
19+
There are two equivalent paths to use a Hugging Face model with Amazon Bedrock Marketplace
20+
Path A is from the Bedrock *Model Catalog*:
21+
1. Console → Amazon Bedrock ▸ Foundation Models ▸ Model catalog
22+
2. Filter Provider → “Hugging Face”, then pick your model (e.g., Gemma 2 27B Instruct)
23+
3. If you see Subscribe, review pricing & terms, click Subscribe, then continue
24+
4. Click Deploy → name the endpoint → keep the recommended instance → accept the EULA → Deploy
25+
5. Wait for Foundation Models ▸ Marketplace deployments to show status In service (takes a few minutes)
26+
6. Click the deployment name and copy the SageMaker endpoint ARN — you’ll need it for API calls
1627

17-
There are two ways to deploy an open model to be used with Amazon Bedrock:
28+
Path B is from SageMaker JumpStart for the model that shows “Use with Bedrock”:
29+
1. In SageMaker Studio, open JumpStart
30+
2. Filter Bedrock Ready models → select the model card (e.g., Gemma 2 9B Instruct)
31+
3. Click Deploy, accept the EULA, keep defaults, Deploy
32+
4. Studio → Deployments ▸ Endpoints → wait for status In service
33+
5. Click the endpoint, choose Use with Bedrock
34+
6. In the Bedrock console, review and Register → a new entry appears under Marketplace deployments
35+
7. Open that entry and copy the SageMaker endpoint ARN for code samples
1836

19-
1. You can deploy your open model from the Bedrock Model Catalog.
20-
2. You can deploy your open model with Amazon Jumpstart and register it with Bedrock.
37+
## 3 . Test interactively
2138

22-
Both ways are similar, so we will guide you through the Bedrock Model catalog.
39+
To test the model interactively in the console, select the model under Marketplace deployments, open it in the playground, and send a prompt in Chat/Text mode to verify the model's response.
2340

24-
To get started, in the Amazon Bedrock console, make sure you are in one of the 14 regions where the Bedrock Marketplace is available. Then, you choose [“Model catalog”](https://us-east-1.console.aws.amazon.com/bedrock/home?region=us-east-1#/model-catalog) in the “Foundation models” section of the navigation pane. Here, you can search for both serverless models and models available in Amazon Bedrock Marketplace. You filter results by “Hugging Face” provider and you can browse through the 83 open models available.
41+
Alternatively, you can programmatically access your endpoint.
2542

26-
For example, let’s search and select Google Gemma 2 27B Instruct.
27-
28-
![model-catalog.png](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/bedrock-marketplace/model-catalog.png)
29-
30-
Choosing the model opens the model detail page where you can see more information from the model provider such as highlights about the model, and usage including sample API calls.
31-
32-
On the top right, let’s click on Deploy.
33-
34-
![model-card.png](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/bedrock-marketplace/model-card.png)
35-
36-
It brings you to the deployment page where you can select the endpoint name, the instance configuration and advanced settings related to networking configuration and service role used to perform the deployment in Sagemaker. Let’s use the default advanced settings and the recommended instance type.
37-
38-
You are also required to accept the End User License Agreement of the model provider.
39-
40-
On the bottom right, let’s click on Deploy.
41-
42-
![model-deploy.png](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/bedrock-marketplace/model-deploy.png)
43-
44-
We just launched the deployment of  GoogleGemma 2 27B Instruct model on a ml.g5.48xlarge instance, hosted in your Amazon Sagemaker tenancy, compatible with Amazon Bedrock APIs!
45-
46-
The endpoint deployment can take several minutes. It will appear in the “Marketplace deployments” page, which you can find in the “Foundation models” section of the navigation pane.
47-
48-
## Use the model with Amazon Bedrock APIs
49-
50-
You can quickly test the model in the Playground through the UI.  However, to invoke the deployed model programmatically with any Amazon Bedrock APIs, you need to get the endpoint ARN.
51-
52-
From the list of managed deployments, choose your model deployment to copy its endpoint ARN.
53-
54-
![model-arn.png](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/bedrock-marketplace/model-arn.png)
43+
```python
44+
import boto3
5545

56-
You can query your endpoint using the AWS SDK in your preferred language or with the AWS CLI.
46+
bedrock = boto3.client("bedrock-runtime")
5747

58-
Here is an example using Bedrock Converse API through the AWS SDK for Python (boto3):
48+
# Paste the endpoint ARN you copied above
49+
endpoint_arn = "arn:aws:sagemaker:<region>:<account‑id>:endpoint/<name>"
5950

60-
```python
61-
import boto3
51+
inference_cfg = {"maxTokens": 256, "temperature": 0.1, "topP": 0.95}
52+
extra = {"parameters": {"repetition_penalty": 1.05}}
6253

63-
bedrock_runtime = boto3.client("bedrock-runtime")
64-
65-
# Add your bedrock endpoint arn here.
66-
endpoint_arn = "arn:aws:sagemaker:<AWS::REGION>:<AWS::AccountId>:endpoint/<Endpoint_Name>"
67-
68-
# Base inference parameters to use.
69-
inference_config = {
70-
"maxTokens": 256,
71-
"temperature": 0.1,
72-
"topP": 0.999,
73-
}
74-
75-
# Additional inference parameters to use.
76-
additional_model_fields = {"parameters": {"repetition_penalty": 0.9, "top_k": 250, "do_sample": True}}
77-
response = bedrock_runtime.converse(
78-
modelId=endpoint_arn,
79-
messages=[
80-
{
81-
"role": "user",
82-
"content": [
83-
{
84-
"text": "What is Amazon doing in the field of generative AI?",
85-
},
86-
]
87-
},
88-
],
89-
inferenceConfig=inference_config,
90-
additionalModelRequestFields=additional_model_fields,
54+
response = bedrock.converse(
55+
modelId=endpoint_arn, # <- SageMaker endpoint ARN
56+
messages=[{
57+
"role": "user",
58+
"content": [{"text": "Give me three taglines for a serverless AI startup"}]
59+
}],
60+
inferenceConfig=inference_cfg,
61+
additionalModelRequestFields=extra,
9162
)
63+
9264
print(response["output"]["message"]["content"][0]["text"])
9365
```
9466

95-
```python
96-
"Amazon is making significant strides in the field of generative AI, applying it across various products and services. Here's a breakdown of their key initiatives:\n\n**1. Amazon Bedrock:**\n\n* This is their **fully managed service** that allows developers to build and scale generative AI applications using models from Amazon and other leading AI companies. \n* It offers access to foundational models like **Amazon Titan**, a family of large language models (LLMs) for text generation, and models from Cohere"
97-
```
67+
*Heads‑up*: the same `modelId=endpoint_arn` works with **InvokeModel**, **Knowledge Bases (RetrieveAndGenerate)**, **Agents**, and **Guardrails**—no code changes.
9868

99-
That’s it! If you want to go further, have a look at the [Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html).
69+
## 4 . Clean‑up (stop charges)
10070

101-
## Clean up
71+
| Resource | How to delete |
72+
|----------|---------------|
73+
| SageMaker endpoint | Console → Marketplace deployments → select → Delete (also de‑registers it) • *or* `boto3.client("sagemaker").delete_endpoint(...)` |
74+
| Optional extras | Delete Knowledge Base, Guardrail, or S3 vectors if you created them. |
10275

103-
Don’t forget to delete your endpoint at the end of your experiment to stop incurring costs! At the top right of the page where you grab the endpoint ARN, you can delete your endpoint by clicking on “Delete”.
76+
For more information, refer to the [Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html).

0 commit comments

Comments
 (0)