Skip to content

Commit aba1f85

Browse files
philschmidphilschmid
authored
add new container version (#514)
1 parent 3ec77b3 commit aba1f85

File tree

1 file changed

+72
-19
lines changed

1 file changed

+72
-19
lines changed

‎docs/sagemaker/reference.md‎

Lines changed: 72 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -6,19 +6,71 @@ title: Reference
66

77
## Deep Learning Container
88

9-
| 🤗 Transformers version | 🤗 Datasets version | PyTorch/TensorFlow version | type | device | Python Version | Example `image_uri` |
10-
| ----------------------- | ------------------- | -------------------------- | -------- | ------ | -------------- | --------------------------------------------------------------------------------------------------------------------------------- |
11-
| 4.4.2 | 1.5.0 | PyTorch 1.6.0 | training | GPU | 3.6 | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-training:1.6.0-transformers4.4.2-gpu-py36-cu110-ubuntu18.04` |
12-
| 4.4.2 | 1.5.0 | TensorFlow 2.4.1 | training | GPU | 3.7 | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-tensorflow-training:2.4.1-transformers4.4.2-gpu-py37-cu110-ubuntu18.04` |
13-
| 4.5.0 | 1.5.0 | PyTorch 1.6.0 | training | GPU | 3.6 | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-training:1.6.0-transformers4.4.2-gpu-py36-cu110-ubuntu18.04` |
14-
| 4.5.0 | 1.5.0 | TensorFlow 2.4.1 | training | GPU | 3.7 | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-tensorflow-training:2.4.1-transformers4.5.0-gpu-py37-cu110-ubuntu18.04` |
15-
| 4.6.1 | 1.6.2 | PyTorch 1.6.0 | training | GPU | 3.6 | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-training:1.6.0-transformers4.5.0-gpu-py36-cu110-ubuntu18.04` |
16-
| 4.6.1 | 1.6.2 | PyTorch 1.7.1 | training | GPU | 3.6 | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-training:1.7.1-transformers4.6.1-gpu-py36-cu110-ubuntu18.04` |
17-
| 4.6.1 | 1.6.2 | TensorFlow 2.4.1 | training | GPU | 3.7 | `763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-tensorflow-training:2.4.1-transformers4.6.1-gpu-py37-cu110-ubuntu18.04` |
18-
| 4.6.1 | 1.6.2 | PyTorch 1.7.1 | inference | CPU | 3.6 | `763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:1.7.1-transformers4.6.1-cpu-py36-ubuntu18.04` |
19-
| 4.6.1 | 1.6.2 | PyTorch 1.7.1 | inference | GPU | 3.6 | `763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-inference:1.7.1-transformers4.6.1-gpu-py36-cu110-ubuntu18.04` |
20-
| 4.6.1 | 1.6.2 | TensorFlow 2.4.1 | inference | CPU | 3.7 | `763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-tensorflow-inference:2.4.1-transformers4.6.1-cpu-py37-ubuntu18.04` |
21-
| 4.6.1 | 1.6.2 | TensorFlow 2.4.1 | inference | GPU | 3.7 | `763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-tensorflow-inference:2.4.1-transformers4.6.1-gpu-py37-cu110-ubuntu18.04` |
9+
Below you can find a version table of currently available Hugging Face DLCs. The table doesn't include the full `image_uri` here are two examples on how to construct those if needed.
10+
11+
**Manually construction the `image_uri`**
12+
13+
`{dlc-aws-account-id}.dkr.ecr.{region}.amazonaws.com/huggingface-{framework}-{(training | inference)}:{framework-version}-transformers{transformers-version}-{device}-{python-version}-{device-tag}`
14+
15+
- `dlc-aws-account-id`: The AWS account ID of the account that owns the ECR repository. You can find them in the [here](https://github.com/aws/sagemaker-python-sdk/blob/e0b9d38e1e3b48647a02af23c4be54980e53dc61/src/sagemaker/image_uri_config/huggingface.json#L21)
16+
- `region`: The AWS region where you want to use it.
17+
- `framework`: The framework you want to use, either `pytorch` or `tensorflow`.
18+
- `(training | inference)`: The training or inference mode.
19+
- `framework-version`: The version of the framework you want to use.
20+
- `transformers-version`: The version of the transformers library you want to use.
21+
- `device`: The device you want to use, either `cpu` or `gpu`.
22+
- `python-version`: The version of the python of the DLC.
23+
- `device-tag`: The device tag you want to use. The device tag can include os version and cuda version
24+
25+
**Example 1: PyTorch Training:**
26+
`763104351884.dkr.ecr.us-west-2.amazonaws.com/huggingface-pytorch-training:1.6.0-transformers4.4.2-gpu-py36-cu110-ubuntu18.04`
27+
**Example 2: Tensorflow Inference:**
28+
`763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-tensorflow-inference:2.4.1-transformers4.6.1-cpu-py37-ubuntu18.04`
29+
30+
## Training DLC Overview
31+
32+
| 🤗 Transformers version | 🤗 Datasets version | PyTorch/TensorFlow version | type | device | Python Version |
33+
| ----------------------- | ------------------- | -------------------------- | -------- | ------ | -------------- |
34+
| 4.4.2 | 1.5.0 | PyTorch 1.6.0 | training | GPU | 3.6 |
35+
| 4.4.2 | 1.5.0 | TensorFlow 2.4.1 | training | GPU | 3.7 |
36+
| 4.5.0 | 1.5.0 | PyTorch 1.6.0 | training | GPU | 3.6 |
37+
| 4.5.0 | 1.5.0 | TensorFlow 2.4.1 | training | GPU | 3.7 |
38+
| 4.6.1 | 1.6.2 | PyTorch 1.6.0 | training | GPU | 3.6 |
39+
| 4.6.1 | 1.6.2 | PyTorch 1.7.1 | training | GPU | 3.6 |
40+
| 4.6.1 | 1.6.2 | TensorFlow 2.4.1 | training | GPU | 3.7 |
41+
| 4.10.2 | 1.11.0 | PyTorch 1.8.1 | training | GPU | 3.6 |
42+
| 4.10.2 | 1.11.0 | PyTorch 1.9.0 | training | GPU | 3.8 |
43+
| 4.10.2 | 1.11.0 | TensorFlow 2.4.1 | training | GPU | 3.7 |
44+
| 4.10.2 | 1.11.0 | TensorFlow 2.5.1 | training | GPU | 3.7 |
45+
| 4.11.0 | 1.12.1 | PyTorch 1.9.0 | training | GPU | 3.8 |
46+
| 4.11.0 | 1.12.1 | TensorFlow 2.5.1 | training | GPU | 3.7 |
47+
| 4.12.3 | 1.15.1 | PyTorch 1.9.1 | training | GPU | 3.8 |
48+
| 4.12.3 | 1.15.1 | TensorFlow 2.5.1 | training | GPU | 3.7 |
49+
50+
## Inference DLC Overview
51+
52+
| 🤗 Transformers version | PyTorch/TensorFlow version | type | device | Python Version |
53+
| ----------------------- | -------------------------- | --------- | ------ | -------------- |
54+
| 4.6.1 | PyTorch 1.7.1 | inference | CPU | 3.6 |
55+
| 4.6.1 | PyTorch 1.7.1 | inference | GPU | 3.6 |
56+
| 4.6.1 | TensorFlow 2.4.1 | inference | CPU | 3.7 |
57+
| 4.6.1 | TensorFlow 2.4.1 | inference | GPU | 3.7 |
58+
| 4.10.2 | PyTorch 1.8.1 | inference | GPU | 3.6 |
59+
| 4.10.2 | PyTorch 1.9.0 | inference | GPU | 3.8 |
60+
| 4.10.2 | TensorFlow 2.4.1 | inference | GPU | 3.7 |
61+
| 4.10.2 | TensorFlow 2.5.1 | inference | GPU | 3.7 |
62+
| 4.10.2 | PyTorch 1.8.1 | inference | CPU | 3.6 |
63+
| 4.10.2 | PyTorch 1.9.0 | inference | CPU | 3.8 |
64+
| 4.10.2 | TensorFlow 2.4.1 | inference | CPU | 3.7 |
65+
| 4.10.2 | TensorFlow 2.5.1 | inference | CPU | 3.7 |
66+
| 4.11.0 | PyTorch 1.9.0 | inference | GPU | 3.8 |
67+
| 4.11.0 | TensorFlow 2.5.1 | inference | GPU | 3.7 |
68+
| 4.11.0 | PyTorch 1.9.0 | inference | CPU | 3.8 |
69+
| 4.11.0 | TensorFlow 2.5.1 | inference | CPU | 3.7 |
70+
| 4.12.3 | PyTorch 1.9.1 | inference | GPU | 3.8 |
71+
| 4.12.3 | TensorFlow 2.5.1 | inference | GPU | 3.7 |
72+
| 4.12.3 | PyTorch 1.9.1 | inference | CPU | 3.8 |
73+
| 4.12.3 | TensorFlow 2.5.1 | inference | CPU | 3.7 |
2274

2375
## Inference Toolkit API
2476

@@ -106,11 +158,11 @@ a deck (one of them the OWNER). Both times, we never heard from them. Not a ca
106158

107159
```json
108160
{
109-
"inputs": "Hugging Face, the winner of VentureBeat’s Innovation in Natural Language Process/Understanding Award for 2021, is looking to level the playing field. The team, launched by Clément Delangue and Julien Chaumond in 2016, was recognized for its work in democratizing NLP, the global market value for which is expected to hit $35.1 billion by 2026. This week, Google’s former head of Ethical AI Margaret Mitchell joined the team.",
110-
"paramters": {
111-
"repetition_penalty": 4.0,
112-
"length_penalty": 1.5
113-
}
161+
"inputs": "Hugging Face, the winner of VentureBeat’s Innovation in Natural Language Process/Understanding Award for 2021, is looking to level the playing field. The team, launched by Clément Delangue and Julien Chaumond in 2016, was recognized for its work in democratizing NLP, the global market value for which is expected to hit $35.1 billion by 2026. This week, Google’s former head of Ethical AI Margaret Mitchell joined the team.",
162+
"paramters": {
163+
"repetition_penalty": 4.0,
164+
"length_penalty": 1.5
165+
}
114166
}
115167
```
116168

@@ -125,6 +177,7 @@ The Inference Toolkit implements various additional environment variables to sim
125177
```bash
126178
HF_TASK="question-answering"
127179
```
180+
128181
**`HF_MODEL_ID`**
129182

130183
`HF_MODEL_ID` defines the model ID which is automatically loaded from [hf.co/models](https://huggingface.co/models) when creating a SageMaker endpoint. All of the 🤗 Hub's 10,000+ models are available through this environment variable.
@@ -147,4 +200,4 @@ HF_MODEL_REVISION="03b4d196c19d0a73c7e0322684e97db1ec397613"
147200

148201
```bash
149202
HF_API_TOKEN="api_XXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
150-
```
203+
```

0 commit comments

Comments
 (0)