florent's review

pagezyhf · pagezyhf · commit 9542b9e8508b · 2025-05-13T18:40:15.000+02:00
diff --git a/docs/sagemaker/source/_toctree.yml b/docs/sagemaker/source/_toctree.yml
@@ -1,15 +1,11 @@
 - sections:
     - local: index
       title: Hugging Face on AWS
-    - local: resources
-      title: Other Resources
   title: Get Started
   isExpanded: true
 - sections:
     - local: dlcs/introduction
       title: Introduction
-    - local: dlcs/features
-      title: Features & benefits
     - local: dlcs/available
       title: Available DLCs on AWS
   title: Deep Learning Containers (DLCs)
@@ -62,5 +58,7 @@
 - sections:
     - local: reference/inference-toolkit
       title: Inference Toolkit API
+    - local: reference/resources
+      title: Other Resources
   title: Reference
   isExpanded: false
diff --git a/docs/sagemaker/source/dlcs/available.md b/docs/sagemaker/source/dlcs/available.md
@@ -8,6 +8,8 @@ For each supported combination of use-case (training, inference), accelerator ty
 
 Pytorch Training DLC: For training, our DLCs are available for PyTorch via Transformers. They include support for training on GPUs and AWS AI chips with libraries such as TRL, Sentence Transformers, or Diffusers.
 
+You can also keep track of the latest Pytorch Training DLC releases [here](https://github.com/aws/deep-learning-containers/releases?q=huggingface-training+AND+NOT+neuronx&expanded=true).
+
 | Container URI                                                                                                                    | Accelerator |
 | -------------------------------------------------------------------------------------------------------------------------------- | ----------- |
 | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-training:2.5.1-transformers4.49.0-gpu-py311-cu124-ubuntu22.04 | GPU         |
@@ -19,6 +21,8 @@ Pytorch Training DLC: For training, our DLCs are available for PyTorch via Trans
 
 For inference, we have a general-purpose PyTorch inference DLC, for serving models trained with any of those frameworks mentioned before on CPU, GPU, and AWS AI chips.
 
+You can also keep track of the latest Pytorch Inference DLC releases [here](https://github.com/aws/deep-learning-containers/releases?q=huggingface-inference+AND+NOT+tgi+AND+NOT+neuronx&expanded=true).
+
 | Container URI                                                                                                                    | Accelerator |
 | -------------------------------------------------------------------------------------------------------------------------------- | ----------- |
 | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:2.6.0-transformers4.49.0-cpu-py312-ubuntu22.04- | CPU         |
@@ -29,6 +33,8 @@ For inference, we have a general-purpose PyTorch inference DLC, for serving mode
 
 There is also the LLM Text Generation Inference (TGI) DLC for high-performance text generation of LLMs on GPU and AWS AI chips.
 
+You can also keep track of the latest LLM TGI DLC releases [here](https://github.com/aws/deep-learning-containers/releases?q=tgi+AND+gpu&expanded=true).
+
 | Container URI                                                                                                                    | Accelerator |
 | -------------------------------------------------------------------------------------------------------------------------------- | ----------- |
 | 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-tgi-inference:2.6.0-tgi3.2.3-gpu-py311-cu124-ubuntu22.04 | GPU         |
@@ -45,11 +51,11 @@ Finally, there is a Text Embeddings Inference (TEI) DLC for high-performance ser
 
 ## FAQ
 
-**How to choose the right container for my use case?**
+**How to choose the right inference container for my use case?**
 
-![dlc-decision-tree](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/sagemaker/dlc-decision-tree.png)
+![inference-dlc-decision-tree](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/sagemaker/inference-dlc-decision-tree.png)
 
-*Note:* See [here]((https://huggingface.co/docs/sagemaker/main/en/reference/inference-toolkit)) for the list of supported task in the inference toolkit.
+*Note:* See [here](https://huggingface.co/docs/sagemaker/main/en/reference/inference-toolkit) for the list of supported task in the inference toolkit.
 
 *Note:* Browse through the Hub to see if you model is tagged ["text-generation-inference"](https://huggingface.co/models?other=text-generation-inference) or ["text-embeddings-inference"](https://huggingface.co/models?other=text-embeddings-inference)
 
diff --git a/docs/sagemaker/source/dlcs/features.md b/docs/sagemaker/source/dlcs/features.md
@@ -4,11 +4,11 @@ The Hugging Face DLCs provide ready-to-use, tested environments to train and dep
 
 ## One command is all you need
 
-With the new Hugging Face DLCs, train and deploy cutting-edge Transformers-based NLP models in a single line of code. The Hugging Face PyTorch DLCs for training come with all the libraries installed to run a single command e.g. via TRL CLI to fine-tune LLMs on any setting, either single-GPU, single-node multi-GPU, and more.
+With the new Hugging Face DLCs, train and deploy cutting-edge Transformers-based NLP models in a single line of code. The Hugging Face PyTorch DLCs for training come with all the libraries installed to run a single command e.g. via [TRL CLI](https://huggingface.co/docs/trl/en/clis) to fine-tune LLMs on any setting, either single-GPU, single-node multi-GPU, and more.
 
 ## Accelerate machine learning from science to production
 
-In addition to Hugging Face DLCs, we created a first-class Hugging Face library for inference, huggingface-inference-toolkit, that comes with the Hugging Face PyTorch DLCs for inference, with full support on serving any PyTorch model on AWS.
+In addition to Hugging Face DLCs, we created a first-class Hugging Face library for inference, [`sagemaker-huggingface-inference-toolkit`](https://github.com/aws/sagemaker-huggingface-inference-toolkit/tree/main/src/sagemaker_huggingface_inference_toolkit), that comes with the Hugging Face PyTorch DLCs for inference, with full support on serving any PyTorch model on AWS.
 
 Deploy your trained models for inference with just one more line of code or select any of the ever growing publicly available models from the model Hub.
 
@@ -26,4 +26,4 @@ Additionally, these DLCs come with full support for AWS meaning that deploying m
 
 Hugging Face DLCs feature built-in performance optimizations for PyTorch to train models faster. The DLCs also give you the flexibility to choose a training infrastructure that best aligns with the price/performance ratio for your workload.
 
-Hugging Face Inference DLCs provide you with production-ready endpoints that scale quickly with your Google Cloud environment, built-in monitoring, and a ton of enterprise features.
+Hugging Face Inference DLCs provide you with production-ready endpoints that scale quickly with your AWS environment, built-in monitoring, and a ton of enterprise features.
diff --git a/docs/sagemaker/source/dlcs/introduction.md b/docs/sagemaker/source/dlcs/introduction.md
@@ -3,10 +3,40 @@
 Hugging Face built Deep Learning Containers (DLCs) for Amazon Web Services customers to run any of their machine learning workload in an optimized environment, with no configuration or maintenance on their part. These are Docker images pre-installed with deep learning frameworks and libraries such as 🤗 Transformers, 🤗 Datasets, and 🤗 Tokenizers. The DLCs allow you to directly serve and train any models, skipping the complicated process of building and optimizing your serving and training environments from scratch.
 
 The containers are publicly maintained, updated and released periodically by Hugging Face and the AWS team and available for all AWS customers within the AWS’s Elastic Container Registry. They can be used from any AWS service such as:
-* Amazon Sagemaker AI: Amazon SageMaker AI is a fully managed machine learning (ML) platform for data scientists and developers to quickly and confidently build, train, and deploy ML models into a production-ready hosted environment.
-* Amazon Bedrock: Amazon Bedrock is a fully managed service that makes high-performing foundation models (FMs) from leading AI companies and Amazon available for your use through a unified API to build generative AI applications.
-* Amazon Elastic Kubernetes Service (EKS): Amazon EKS is the premiere platform for running Kubernetes clusters in the AWS cloud.
-* Amazon Elastic Container Service (ECS): Amazon ECS is a fully managed container orchestration service that helps you easily deploy, manage, and scale containerized applications.
-* Amazon Elastic Compute Cloud (EC2): Amazon EC2 provides on-demand, scalable computing capacity in the Amazon Web Services (AWS) Cloud.
+* **Amazon Sagemaker AI**: Amazon SageMaker AI is a fully managed machine learning (ML) platform for data scientists and developers to quickly and confidently build, train, and deploy ML models into a production-ready hosted environment.
+* **Amazon Bedrock**: Amazon Bedrock is a fully managed service that makes high-performing foundation models (FMs) from leading AI companies and Amazon available for your use through a unified API to build generative AI applications.
+* **Amazon Elastic Kubernetes Service (EKS)**: Amazon EKS is the premiere platform for running Kubernetes clusters in the AWS cloud.
+* **Amazon Elastic Container Service (ECS)**: Amazon ECS is a fully managed container orchestration service that helps you easily deploy, manage, and scale containerized applications.
+* **Amazon Elastic Compute Cloud (EC2)**: Amazon EC2 provides on-demand, scalable computing capacity in the Amazon Web Services (AWS) Cloud.
 
-Hugging Face DLCs are open source and licensed under Apache 2.0. Feel free to reach out on our [community forum](https://discuss.huggingface.co/c/sagemaker/17) if you have any questions.
+Hugging Face DLCs are open source and licensed under Apache 2.0. Feel free to reach out on our [community forum](https://discuss.huggingface.co/c/sagemaker/17) if you have any questions.
+
+## Features & benefits
+
+The Hugging Face DLCs provide ready-to-use, tested environments to train and deploy Hugging Face models.
+
+### One command is all you need
+
+With the new Hugging Face DLCs, train and deploy cutting-edge Transformers-based NLP models in a single line of code. The Hugging Face PyTorch DLCs for training come with all the libraries installed to run a single command e.g. via [TRL CLI](https://huggingface.co/docs/trl/en/clis) to fine-tune LLMs on any setting, either single-GPU, single-node multi-GPU, and more.
+
+### Accelerate machine learning from science to production
+
+In addition to Hugging Face DLCs, we created a first-class Hugging Face library for inference, [`sagemaker-huggingface-inference-toolkit`](https://github.com/aws/sagemaker-huggingface-inference-toolkit/tree/main/src/sagemaker_huggingface_inference_toolkit), that comes with the Hugging Face PyTorch DLCs for inference, with full support on serving any PyTorch model on AWS.
+
+Deploy your trained models for inference with just one more line of code or select any of the ever growing publicly available models from the model Hub.
+
+### High-performance text generation and embedding
+
+Besides the PyTorch-oriented DLCs, Hugging Face also provides high-performance inference for both text generation and embedding models via the Hugging Face DLCs for both Text Generation Inference (TGI) and Text Embeddings Inference (TEI), respectively.
+
+The Hugging Face DLC for TGI enables you to deploy any of the +225,000 text generation inference supported models from the Hugging Face Hub, or any custom model as long as its architecture is supported within TGI.
+
+The Hugging Face DLC for TEI enables you to deploy any of the +12,000 embedding, re-ranking or sequence classification supported models from the Hugging Face Hub, or any custom model as long as its architecture is supported within TEI.
+
+Additionally, these DLCs come with full support for AWS meaning that deploying models from Amazon Simple Storage Service (S3) is also straight forward and requires no configuration.
+
+### Built-in performance
+
+Hugging Face DLCs feature built-in performance optimizations for PyTorch to train models faster. The DLCs also give you the flexibility to choose a training infrastructure that best aligns with the price/performance ratio for your workload.
+
+Hugging Face Inference DLCs provide you with production-ready endpoints that scale quickly with your AWS environment, built-in monitoring, and a ton of enterprise features.
diff --git a/docs/sagemaker/source/reference/inference-toolkit.md b/docs/sagemaker/source/reference/inference-toolkit.md
@@ -2,7 +2,7 @@
 
 ## Supported tasks
 
-The Inference Toolkit accepts inputs in the `inputs` key, and supports additional [`pipelines`](https://huggingface.co/docs/transformers/main_classes/pipelines) parameters in the `parameters` key. You can provide any of the supported `kwargs` from `pipelines` as `parameters`.
+The [Sagemaker Hugging Face Inference Toolkit](https://github.com/aws/sagemaker-huggingface-inference-toolkit/tree/main) accepts inputs in the `inputs` key, and supports additional [`pipelines`](https://huggingface.co/docs/transformers/main_classes/pipelines) parameters in the `parameters` key. You can provide any of the supported `kwargs` from `pipelines` as `parameters`.
 
 Tasks supported by the Inference Toolkit API include:
 
diff --git a/docs/sagemaker/source/reference/resources.md b/docs/sagemaker/source/reference/resources.md
diff --git a/docs/sagemaker/source/tutorials/bedrock/bedrock-quickstart.md b/docs/sagemaker/source/tutorials/bedrock/bedrock-quickstart.md
@@ -1,14 +1,14 @@
 # Quickstart — Using Hugging Face Models with Amazon Bedrock Marketplace
 
 ## Why use Bedrock Marketplace for Hugging Face models?
-Amazon Bedrock now exposes 83 Hugging Face open-weight models—including Gemma, Llama 3, Mistral, and more—through a single catalog. You invoke them with the same Bedrock APIs you already use for Titan, Anthropic, Cohere, etc. Under the hood, Bedrock Marketplace model endpoints are managed by Amazon SageMaker AI. With Bedrock Marketplace, you can now combine the ease of use of SageMaker JumpStart with the fully managed infrastructure of Amazon Bedrock, including compatibility with high-level APIs such as Agents, Knowledge Bases, Guardrails and Model Evaluations.
+Amazon Bedrock now exposes Hugging Face open-weight models—including Gemma, Llama 3, Mistral, and more—through a single catalog. You invoke them with the same Bedrock APIs you already use for Titan, Anthropic, Cohere, etc. Under the hood, Bedrock Marketplace model endpoints are managed by Amazon SageMaker AI. With Bedrock Marketplace, you can now combine the ease of use of SageMaker JumpStart with the fully managed infrastructure of Amazon Bedrock, including compatibility with high-level APIs such as Agents, Knowledge Bases, Guardrails and Model Evaluations.
 
 ## 1 . Prerequisites
 
 |  | Requirement | Notes |
 |---|-------------|
-| AWS account in a Bedrock Region | Marketplace is regional; switch the console to one of the 14 supported Regions first. |
-| Permissions | For a quick trial, attach AmazonBedrockFullAccess and AmazonSageMakerFullAccess.|
+| AWS account in a Bedrock Region | Marketplace is regional; switch the console to one of the 14 supported Regions first, for example `us-east-1`. |
+| Permissions | For a quick trial, attach `AmazonBedrockFullAccess` and `AmazonSageMakerFullAccess`.|
 | Service quotas | The SageMaker endpoint uses GPU instances (for example ml.g5). Verify you have quota or request it. |
 | JumpStart-only | If you choose path B, create a SageMaker Studio domain and user profile first (Console ▸ SageMaker ▸ Domains). Open Studio before continuing. |
 
@@ -24,7 +24,11 @@ Path A is from the Bedrock *Model Catalog*:
 3. If you see Subscribe, review pricing & terms, click Subscribe, then continue  
 4. Click Deploy → name the endpoint → keep the recommended instance → accept the EULA → Deploy  
 5. Wait for Foundation Models → Marketplace deployments to show status In service (takes a few minutes)  
-6. Click the deployment name and copy the SageMaker endpoint ARN — you’ll need it for API calls  
+6. Click the deployment name and copy the SageMaker endpoint ARN — you’ll need it for API calls
+
+<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/sagemaker/bedrock-marketplace-deployment.gif"
+     alt="Bedrock deployment demo"
+     width="500">
 
 Path B is from SageMaker JumpStart for the model that shows “Use with Bedrock”:
 1. In SageMaker Studio, open JumpStart  
diff --git a/docs/sagemaker/source/tutorials/jumpstart/jumpstart-quickstart.md b/docs/sagemaker/source/tutorials/jumpstart/jumpstart-quickstart.md
@@ -13,7 +13,7 @@ In this quickstart guide, we will deploy [Qwen/Qwen2.5-14B-Instruct](https://hug
 | AWS account with SageMaker enabled | An AWS account that will contain all your AWS resources. |
 | An IAM role to access SageMaker AI | Learn more about how IAM works with SageMaker AI in this [guide](https://docs.aws.amazon.com/sagemaker/latest/dg/security-iam.html). |
 | SageMaker Studio domain and user profile | We recommend using SageMaker Studio for straightforward deployment and inference. Follow this [guide](https://docs.aws.amazon.com/sagemaker/latest/dg/onboard-quick-start.html). |
-| Service quotas | Most LLMs need GPU instances (e.g. ml.g5). Verify you have quota for ml.g5.24xlarge or [request it](https://docs.aws.amazon.com/sagemaker/latest/dg/canvas-requesting-quota-increases.html). | 
+| Service quotas | Most LLMs need GPU instances (e.g. ml.g5). Verify you have quota for `ml.g5.24xlarge` or [request it](https://docs.aws.amazon.com/sagemaker/latest/dg/canvas-requesting-quota-increases.html). | 
 
 ## 2· Endpoint deployment
 
@@ -24,10 +24,19 @@ Let's explain how you would deploy a Hugging Face model to SageMaker browsing th
 4. Wait until Endpoints shows In service.  
 5. Copy the Endpoint name (or ARN) for later use.
 
+<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/sagemaker/jumpstart-deployment.gif"
+     alt="JumpStart deployment demo"
+     width="500">
+
 Alternatively, you can also browse through the Hugging Face Model Hub:
 1. Open the model page → Click Deploy → SageMaker → Jumpstart tab if model is available.
 2. Copy the code snippet and use it from a SageMaker Notebook instance.
 
+
+<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/sagemaker/hf-jumpstart-deployment.gif"
+     alt="JumpStart deployment demo"
+     width="500">
+
 ```python
 # SageMaker JumpStart provides APIs as part of SageMaker SDK that allow you to deploy and fine-tune models in network isolation using scripts that SageMaker maintains.
 
