diff --git a/docs/sagemaker/_toctree.yml b/docs/sagemaker/_toctree.yml index d5464737d..c66be617c 100644 --- a/docs/sagemaker/_toctree.yml +++ b/docs/sagemaker/_toctree.yml @@ -7,4 +7,10 @@ - local: inference title: Deploy models to Amazon SageMaker - local: reference - title: Reference \ No newline at end of file + title: AWS Deep Learning Containers (DLCs) + isExpanded: true + sections: + - local: tgi + title: Text Generation Inference (TGI) + - local: transformers + title: Transformers diff --git a/docs/sagemaker/reference.md b/docs/sagemaker/reference.md index 42c5e5146..e2085721e 100644 --- a/docs/sagemaker/reference.md +++ b/docs/sagemaker/reference.md @@ -1,91 +1,8 @@ -# Reference +# AWS Deep Learning Containers (DLCs) ## Deep Learning Container -Below you can find a version table of currently available Hugging Face DLCs. The table doesn't include the full `image_uri` here are two examples on how to construct those if needed. - -**Manually construction the `image_uri`** - -`{dlc-aws-account-id}.dkr.ecr.{region}.amazonaws.com/huggingface-{framework}-{(training | inference)}:{framework-version}-transformers{transformers-version}-{device}-{python-version}-{device-tag}` - -- `dlc-aws-account-id`: The AWS account ID of the account that owns the ECR repository. You can find them in the [here](https://github.com/aws/sagemaker-python-sdk/blob/e0b9d38e1e3b48647a02af23c4be54980e53dc61/src/sagemaker/image_uri_config/huggingface.json#L21) -- `region`: The AWS region where you want to use it. -- `framework`: The framework you want to use, either `pytorch` or `tensorflow`. -- `(training | inference)`: The training or inference mode. -- `framework-version`: The version of the framework you want to use. -- `transformers-version`: The version of the transformers library you want to use. -- `device`: The device you want to use, either `cpu` or `gpu`. -- `python-version`: The version of the python of the DLC. -- `device-tag`: The device tag you want to use. The device tag can include os version and cuda version - -**Example 1: PyTorch Training:** -`763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-training:1.6.0-transformers4.4.2-gpu-py36-cu110-ubuntu18.04` -**Example 2: Tensorflow Inference:** -`763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-tensorflow-inference:2.4.1-transformers4.6.1-cpu-py37-ubuntu18.04` - -## Training DLC Overview - -The Training DLC overview includes all released and available Hugging Face Training DLCs. It includes PyTorch and TensorFlow flavored -versions for GPU. - -| 🤗 Transformers version | 🤗 Datasets version | PyTorch/TensorFlow version | type | device | Python Version | -| ----------------------- | ------------------- | -------------------------- | -------- | ------ | -------------- | -| 4.4.2 | 1.5.0 | PyTorch 1.6.0 | training | GPU | 3.6 | -| 4.4.2 | 1.5.0 | TensorFlow 2.4.1 | training | GPU | 3.7 | -| 4.5.0 | 1.5.0 | PyTorch 1.6.0 | training | GPU | 3.6 | -| 4.5.0 | 1.5.0 | TensorFlow 2.4.1 | training | GPU | 3.7 | -| 4.6.1 | 1.6.2 | PyTorch 1.6.0 | training | GPU | 3.6 | -| 4.6.1 | 1.6.2 | PyTorch 1.7.1 | training | GPU | 3.6 | -| 4.6.1 | 1.6.2 | TensorFlow 2.4.1 | training | GPU | 3.7 | -| 4.10.2 | 1.11.0 | PyTorch 1.8.1 | training | GPU | 3.6 | -| 4.10.2 | 1.11.0 | PyTorch 1.9.0 | training | GPU | 3.8 | -| 4.10.2 | 1.11.0 | TensorFlow 2.4.1 | training | GPU | 3.7 | -| 4.10.2 | 1.11.0 | TensorFlow 2.5.1 | training | GPU | 3.7 | -| 4.11.0 | 1.12.1 | PyTorch 1.9.0 | training | GPU | 3.8 | -| 4.11.0 | 1.12.1 | TensorFlow 2.5.1 | training | GPU | 3.7 | -| 4.12.3 | 1.15.1 | PyTorch 1.9.1 | training | GPU | 3.8 | -| 4.12.3 | 1.15.1 | TensorFlow 2.5.1 | training | GPU | 3.7 | -| 4.17.0 | 1.18.4 | PyTorch 1.10.2 | training | GPU | 3.8 | -| 4.17.0 | 1.18.4 | TensorFlow 2.6.3 | training | GPU | 3.8 | -| 4.26.0 | 2.9.0 | PyTorch 1.13.1 | training | GPU | 3.9 | - -## Inference DLC Overview - -The Inference DLC overview includes all released and available Hugging Face Inference DLCs. It includes PyTorch and TensorFlow flavored -versions for CPU, GPU & AWS Inferentia. - - -| 🤗 Transformers version | PyTorch/TensorFlow version | type | device | Python Version | -| ----------------------- | -------------------------- | --------- | ------ | -------------- | -| 4.6.1 | PyTorch 1.7.1 | inference | CPU | 3.6 | -| 4.6.1 | PyTorch 1.7.1 | inference | GPU | 3.6 | -| 4.6.1 | TensorFlow 2.4.1 | inference | CPU | 3.7 | -| 4.6.1 | TensorFlow 2.4.1 | inference | GPU | 3.7 | -| 4.10.2 | PyTorch 1.8.1 | inference | GPU | 3.6 | -| 4.10.2 | PyTorch 1.9.0 | inference | GPU | 3.8 | -| 4.10.2 | TensorFlow 2.4.1 | inference | GPU | 3.7 | -| 4.10.2 | TensorFlow 2.5.1 | inference | GPU | 3.7 | -| 4.10.2 | PyTorch 1.8.1 | inference | CPU | 3.6 | -| 4.10.2 | PyTorch 1.9.0 | inference | CPU | 3.8 | -| 4.10.2 | TensorFlow 2.4.1 | inference | CPU | 3.7 | -| 4.10.2 | TensorFlow 2.5.1 | inference | CPU | 3.7 | -| 4.11.0 | PyTorch 1.9.0 | inference | GPU | 3.8 | -| 4.11.0 | TensorFlow 2.5.1 | inference | GPU | 3.7 | -| 4.11.0 | PyTorch 1.9.0 | inference | CPU | 3.8 | -| 4.11.0 | TensorFlow 2.5.1 | inference | CPU | 3.7 | -| 4.12.3 | PyTorch 1.9.1 | inference | GPU | 3.8 | -| 4.12.3 | TensorFlow 2.5.1 | inference | GPU | 3.7 | -| 4.12.3 | PyTorch 1.9.1 | inference | CPU | 3.8 | -| 4.12.3 | TensorFlow 2.5.1 | inference | CPU | 3.7 | -| 4.12.3 | PyTorch 1.9.1 | inference | Inferentia | 3.7 | -| 4.17.0 | PyTorch 1.10.2 | inference | GPU | 3.8 | -| 4.17.0 | TensorFlow 2.6.3 | inference | GPU | 3.8 | -| 4.17.0 | PyTorch 1.10.2 | inference | CPU | 3.8 | -| 4.17.0 | TensorFlow 2.6.3 | inference | CPU | 3.8 | -| 4.26.0 | PyTorch 1.13.1 | inference | CPU | 3.9 | -| 4.26.0 | PyTorch 1.13.1 | inference | GPU | 3.9 | - - +There are several different types of AWS Deep Learning Containers. Feel free to explore further in the subheadings! ## Hugging Face Transformers Amazon SageMaker Examples diff --git a/docs/sagemaker/tgi.md b/docs/sagemaker/tgi.md new file mode 100644 index 000000000..829cad647 --- /dev/null +++ b/docs/sagemaker/tgi.md @@ -0,0 +1,14 @@ +# Available DLCs on Google Cloud + +Below you can find a listing of all the Deep Learning Containers (DLCs) available on Google Cloud. + + + +The listing below only contains the latest version of each one of the Hugging Face DLCs, the full listing of the available published containers in Google Cloud can be found either in the [Google Cloud Deep Learning Containers Documentation](https://cloud.google.com/deep-learning-containers/docs/choosing-container#hugging-face), in the [Google Cloud Artifact Registry](https://console.cloud.google.com/artifacts/docker/deeplearning-platform-release/us/gcr.io) or via the `gcloud container images list --repository="us-docker.pkg.dev/deeplearning-platform-release/gcr.io" | grep "huggingface-"` command. + + + +## Text Generation Inference (TGI) + +``` +``` diff --git a/docs/sagemaker/transformers.md b/docs/sagemaker/transformers.md new file mode 100644 index 000000000..e69de29bb