diff --git a/docs/sagemaker/tgi.md b/docs/sagemaker/tgi.md index a28cd4284..3b9abf4d6 100644 --- a/docs/sagemaker/tgi.md +++ b/docs/sagemaker/tgi.md @@ -5,3 +5,31 @@ Below, you can find a list of the latest available images for TGI for use on AWS SageMaker. To find the latest supported versions of the HF DLCs, check out https://aws.amazon.com/releasenotes/dlc-support-policy/ + + +## huggingface-pytorch-tgi-inference + +| Framework Version | Image Type | Image URI | Size (GB) | Pushed At | Details | +| --- | --- | --- | --- | --- | --- | +| 2.4 | gpu | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-tgi-inference:2.4.0-tgi3.0.1-gpu-py311-cu124-ubuntu22.04-v2.1` | 6.49 | 2025-01-21 19:37:15 | [Details](https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-text-generation-inference-tgi-containers) | +| 2.3 | gpu | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-tgi-inference:2.3.0-tgi2.2.0-gpu-py310-cu121-ubuntu22.04-v2.1` | 4.92 | 2024-10-04 21:59:12 | [Details](https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-text-generation-inference-tgi-containers) | + + +### SM Example +``` +# create Hugging Face Model Class +huggingface_model = HuggingFaceModel( + image_uri=get_huggingface_llm_image_uri("huggingface",version="2.4"), + env=, + role=, +) + +# deploy model to SageMaker Inference +predictor = huggingface_model.deploy( + initial_instance_count=1, + instance_type="ml.g6.48xlarge", + container_startup_health_check_timeout=2400, +) +``` + + \ No newline at end of file diff --git a/docs/sagemaker/transformers.md b/docs/sagemaker/transformers.md index ee405debc..0e1522b23 100644 --- a/docs/sagemaker/transformers.md +++ b/docs/sagemaker/transformers.md @@ -5,3 +5,60 @@ Below, we include a list of the latest images available on AWS, which come pre-packaged with transformers and [datasets](https://huggingface.co/docs/datasets/en/index) libraries for your convenience. Check out some of the tutorials in the reference section for more information! To find the latest supported versions of the HF DLCs, check out https://aws.amazon.com/releasenotes/dlc-support-policy/ + + +## huggingface-pytorch-training + +| Framework Version | Image Type | Image URI | Size (GB) | Pushed At | Details | +| --- | --- | --- | --- | --- | --- | +| 2.3 | gpu | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-training:2.3.0-transformers4.46.1-gpu-py311-cu121-ubuntu20.04-v1.0` | 8.78 | 2024-11-12 19:49:17 | [Details](https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-training-containers) | + + +### SM Example +``` +# create Hugging Face Model Class +huggingface_model = HuggingFaceModel( + image_uri=get_huggingface_llm_image_uri("huggingface",version="2.3"), + env=, + role=, +) + +# deploy model to SageMaker Inference +predictor = huggingface_model.deploy( + initial_instance_count=1, + instance_type="ml.g6.48xlarge", + container_startup_health_check_timeout=2400, +) +``` + + +## huggingface-pytorch-inference + +| Framework Version | Image Type | Image URI | Size (GB) | Pushed At | Details | +| --- | --- | --- | --- | --- | --- | +| 2.3 | gpu | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-inference:2.3.0-transformers4.46.1-gpu-py311-cu121-ubuntu20.04-v1.0` | 9.06 | 2024-12-21 19:19:57 | [Details](https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-inference-containers) | +| 2.3 | cpu | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-inference:2.3.0-transformers4.46.1-cpu-py311-ubuntu22.04-v1.0` | 1.4 | 2024-12-21 19:11:54 | [Details](https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-inference-containers) | + + +### SM Example +``` +# create Hugging Face Model Class +huggingface_model = HuggingFaceModel( + image_uri=get_huggingface_llm_image_uri("huggingface",version="2.3"), + env=, + role=, +) + +# deploy model to SageMaker Inference +predictor = huggingface_model.deploy( + initial_instance_count=1, + instance_type="ml.g6.48xlarge", + container_startup_health_check_timeout=2400, +) +``` + + + + + + \ No newline at end of file