Skip to content

Commit eeb8e01

Browse files
chuyang-dengchuyang-deng
authored
change: upgrade framework versions (#1315)
* change: upgrade framework versions
1 parent adeabe6 commit eeb8e01

File tree

8 files changed

+76
-61
lines changed

8 files changed

+76
-61
lines changed

‎src/sagemaker/mxnet/README.rst‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,11 +28,11 @@ The Docker images have the following dependencies installed:
2828
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+
2929
| CUDA (GPU image only) | 9.0 | 9.0 | 9.0 | 9.0 | 9.0 | 9.2 | 10.0 | 10.1 |
3030
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+
31-
| numpy | 1.13.3 | 1.13.3 | 1.13.3 | 1.14.5 | 1.14.6 | 1.16.3 | 1.14.5 | 1.17.2 |
31+
| numpy | 1.13.3 | 1.13.3 | 1.13.3 | 1.14.5 | 1.14.6 | 1.16.3 | 1.14.5 | 1.17.4 |
3232
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+
33-
| onnx | N/A | N/A | N/A | 1.2.1 | 1.2.1 | 1.4.1 | 1.4.1 | 1.6.0 |
33+
| onnx | N/A | N/A | N/A | 1.2.1 | 1.2.1 | 1.4.1 | 1.4.1 | 1.4.1 |
3434
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+
35-
| keras-mxnet | N/A | N/A | N/A | N/A | 2.2.2 | 2.2.4.1 | 2.2.4.1 | 2.2.4.2 |
35+
| keras-mxnet | N/A | N/A | N/A | N/A | 2.2.2 | 2.2.4.1 | 2.2.4.1 | 2.2.4.1 |
3636
+-------------------------+--------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+
3737

3838
The Docker images extend Ubuntu 16.04.

‎src/sagemaker/pytorch/README.rst‎

Lines changed: 40 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ SageMaker PyTorch Estimators and Models
44

55
With PyTorch Estimators and Models, you can train and host PyTorch models on Amazon SageMaker.
66

7-
Supported versions of PyTorch: ``0.4.0``, ``1.0.0``, ``1.1.0``, ``1.2.0``, ``1.3.1``.
7+
Supported versions of PyTorch: ``0.4.0``, ``1.0.0``, ``1.1.0``, ``1.2.0``, ``1.3.1``, ``1.4.0``.
88

99
We recommend that you use the latest supported version, because that's where we focus most of our development efforts.
1010

@@ -37,43 +37,45 @@ This Python version applies to both the Training Job, created by fit, and the En
3737

3838
The PyTorch Docker images have the following dependencies installed:
3939

40-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
41-
| Dependencies | pytorch 0.4.0 | pytorch 1.0.0 | pytorch 1.1.0 | pytorch 1.2.0 | pytorch 1.3.1 |
42-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
43-
| boto3 | >=1.7.35 | >=1.9.11 | 1.9.82 | 1.9.249 | 1.10.34 |
44-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
45-
| botocore | >=1.10.35 | >=1.12.11 | >= 1.12.11 | 1.12.249 | 1.13.34 |
46-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
47-
| CUDA (GPU image only) | 9.0 | 9.0 | 10.1 | 10.0 | 10.1 |
48-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
49-
| numpy | >=1.14.3 | >=1.15.2 | 1.16.4 | 1.16.4 | 1.16.4 |
50-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
51-
| Pillow | >=5.1.0 | >=5.2.0 | 6.0.0 | 5.4.1 | 6.2.1 |
52-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
53-
| pip | >=10.0.1 | >=18.0 | >=18.0 | 19.3 | 19.3.1 |
54-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
55-
| python-dateutil | >=2.7.3 | >=2.7.3 | >=2.7.3 | 2.8.0 | 2.8.0 |
56-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
57-
| retrying | >=1.3.3 | >=1.3.3 | 1.3.3 | 1.3.3 | 1.3.3 |
58-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
59-
| s3transfer | >=0.1.13 | >=0.1.13 | >=0.1.13 | 0.2.1 | 0.2.1 |
60-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
61-
| sagemaker-containers | >=2.1.0 | >=2.1.0 | 2.4.10.post0 | 2.5.4 | 2.5.4 |
62-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
63-
| sagemaker-inference (inference only) | N/A | N/A | N/A | N/A | 1.1.2 |
64-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
65-
| sagemaker-pytorch-container | 1.0 | 1.1 | 1.2 | 1.2 | 1.3 |
66-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
67-
| setuptools | >=39.2.0 | >=40.4.3 | >=40.4.3 | 41.4.0 | 42.0.2.post20191203 |
68-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
69-
| six | >=1.11.0 | >=1.11.0 | 1.12.0 | 1.12.0 | 1.12.0 |
70-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
71-
| torch | 0.4.0 | 1.0.0 | 1.1.0 | 1.2.0 | 1.3.1 |
72-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
73-
| torchvision | 0.2.1 | 0.2.1 | 0.3.0 | 0.4.0a0+9232c4a | 0.4.2 |
74-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
75-
| Python | 2.7 or 3.5 | 2.7 or 3.6 | 2.7 or 3.6 | 2.7 or 3.6 | 2.7 or 3.6 |
76-
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+
40+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
41+
| Dependencies | pytorch 0.4.0 | pytorch 1.0.0 | pytorch 1.1.0 | pytorch 1.2.0 | pytorch 1.3.1 | pytorch 1.4.0 |
42+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
43+
| boto3 | >=1.7.35 | >=1.9.11 | 1.9.82 | 1.9.249 | 1.10.34 | 1.12.4 |
44+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
45+
| botocore | >=1.10.35 | >=1.12.11 | >= 1.12.11 | 1.12.249 | 1.13.34 | 1.15.4 |
46+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
47+
| CUDA (GPU image only) | 9.0 | 9.0 | 10.1 | 10.0 | 10.1 | 10.1 |
48+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
49+
| numpy | >=1.14.3 | >=1.15.2 | 1.16.4 | 1.16.4 | 1.16.4 | 1.16.4 |
50+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
51+
| Pillow | >=5.1.0 | >=5.2.0 | 6.0.0 | 5.4.1 | 6.2.1 | 6.2.0 |
52+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
53+
| pip | >=10.0.1 | >=18.0 | >=18.0 | 19.3 | 19.3.1 | 20.0.2 |
54+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
55+
| python-dateutil | >=2.7.3 | >=2.7.3 | >=2.7.3 | 2.8.0 | 2.8.0 | 2.8.1 |
56+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
57+
| retrying | >=1.3.3 | >=1.3.3 | 1.3.3 | 1.3.3 | 1.3.3 | 1.3.3 |
58+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
59+
| s3transfer | >=0.1.13 | >=0.1.13 | >=0.1.13 | 0.2.1 | 0.2.1 | 0.3.3 |
60+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
61+
| sagemaker-containers | >=2.1.0 | >=2.1.0 | 2.4.10.post0 | 2.5.4 | 2.5.4 | 2.5.4 |
62+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
63+
| sagemaker-inference (inference only) | N/A | N/A | N/A | N/A | 1.1.2 | 1.1.2 |
64+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
65+
| sagemaker-pytorch-container | 1.0 | 1.1 | 1.2 | 1.2 | 1.3 | N/A |
66+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
67+
| sagemaker-pytorch-inference | N/A | N/A | N/A | N/A | N/A | 1.1.2 |
68+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
69+
| setuptools | >=39.2.0 | >=40.4.3 | >=40.4.3 | 41.4.0 | 42.0.2.post20191203 | N/A |
70+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
71+
| six | >=1.11.0 | >=1.11.0 | 1.12.0 | 1.12.0 | 1.12.0 | 1.12.0 |
72+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
73+
| torch | 0.4.0 | 1.0.0 | 1.1.0 | 1.2.0 | 1.3.1 | 1.4.0 |
74+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
75+
| torchvision | 0.2.1 | 0.2.1 | 0.3.0 | 0.4.0a0+9232c4a | 0.4.2 | 0.5.0 |
76+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
77+
| Python | 2.7 or 3.5 | 2.7 or 3.6 | 2.7 or 3.6 | 2.7 or 3.6 | 2.7 or 3.6 | 2.7 or 3.6 |
78+
+---------------------------------------+---------------+----------------+---------------+-----------------+---------------------+---------------------+
7779

7880
The Docker images extend Ubuntu 16.04.
7981

‎src/sagemaker/pytorch/defaults.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
The default version is no longer updated so as to not break existing workflows.
1919
"""
2020

21-
LATEST_VERSION = "1.3.1"
21+
LATEST_VERSION = "1.4.0"
2222
"""The latest version of PyTorch included in the SageMaker pre-built Docker images."""
2323

2424
PYTHON_VERSION = "py3"

‎src/sagemaker/tensorflow/README.rst‎

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -30,41 +30,41 @@ SageMaker TensorFlow Docker containers
3030
The latest containers include the following Python packages:
3131

3232
+--------------------------------+---------------+---------------+
33-
| Dependencies | TF 1.15 | TF 2.0 |
33+
| Dependencies | TF 1.15.2 | TF 2.1 |
3434
+--------------------------------+---------------+---------------+
35-
| awscli | 1.16.296 | 1.16.303 |
35+
| awscli | 1.18.1 | 1.18.3 |
3636
+--------------------------------+---------------+---------------+
37-
| boto3 | Latest | Latest |
37+
| boto3 | 1.12.1 | 1.12.3 |
3838
+--------------------------------+---------------+---------------+
39-
| botocore | Latest | Latest |
39+
| botocore | 1.15.1 | 1.15.3 |
4040
+--------------------------------+---------------+---------------+
41-
| h5py | 2.9.0 | 2.10.0 |
41+
| h5py | 2.10.0 | 2.10.0 |
4242
+--------------------------------+---------------+---------------+
4343
| horovod | 0.18.2 | 0.18.2 |
4444
+--------------------------------+---------------+---------------+
4545
| keras | 2.3.1 | 2.3.1 |
4646
+--------------------------------+---------------+---------------+
4747
| mpi4py | 3.0.2 | 3.0.3 |
4848
+--------------------------------+---------------+---------------+
49-
| numpy | 1.17.4 | 1.17.4 |
49+
| numpy | 1.18.1 | 1.18.1 |
5050
+--------------------------------+---------------+---------------+
51-
| pandas | 0.24.2 | 0.25.3 |
51+
| pandas | 0.24.2 | 1.0.1 |
5252
+--------------------------------+---------------+---------------+
53-
| pip | 19.3.1 | 19.3.1 |
53+
| pip | 20.0.2 | 20.0.2 |
5454
+--------------------------------+---------------+---------------+
55-
| Pillow | 6.1.0 | 6.2.1 |
55+
| Pillow | 6.2.1 | 7.0.0 |
5656
+--------------------------------+---------------+---------------+
5757
| Python | 2.7 or 3.6 | 2.7 or 3.6 |
5858
+--------------------------------+---------------+---------------+
5959
| requests | 2.22.0 | 2.22.0 |
6060
+--------------------------------+---------------+---------------+
61-
| sagemaker-containers | >=2.6.1 | >=2.6.2 |
61+
| sagemaker-containers | 2.7.0 | 2.8.0 |
6262
+--------------------------------+---------------+---------------+
63-
| sagemaker-tensorflow-container | 1.15.0.1.0.0 | 2.0.0.1.0.0 |
63+
| sagemaker-tensorflow-container | 1.15.0.1.1.0 | 2.0.0.1.1.0 |
6464
+--------------------------------+---------------+---------------+
65-
| scipy | 1.2.2 | 1.2.2 |
65+
| scipy | 1.2.2 | 1.4.1 |
6666
+--------------------------------+---------------+---------------+
67-
| tensorflow | 1.15.0 | 2.0.0 |
67+
| tensorflow | 1.15.2 | 2.1.0 |
6868
+--------------------------------+---------------+---------------+
6969

7070
Script Mode TensorFlow Docker images support both Python 2.7 and Python 3.6. The Docker images extend Ubuntu 16.04.

‎src/sagemaker/tensorflow/defaults.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@
1818
This is no longer updated so as to not break existing workflows.
1919
"""
2020

21-
LATEST_VERSION = "2.0.0"
21+
LATEST_VERSION = "2.1.0"
2222
"""The latest version of TensorFlow included in the SageMaker pre-built Docker images."""
2323

24-
LATEST_PY2_VERSION = "2.0.0"
24+
LATEST_PY2_VERSION = "2.1.0"

‎src/sagemaker/tensorflow/estimator.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -199,12 +199,12 @@ class TensorFlow(Framework):
199199

200200
LATEST_VERSION = defaults.LATEST_VERSION
201201

202-
_LATEST_1X_VERSION = "1.15.0"
202+
_LATEST_1X_VERSION = "1.15.2"
203203

204204
_LOWEST_SCRIPT_MODE_ONLY_VERSION = [1, 13]
205205
# 2.0.0 still supports py2
206206
# we will need to update this version number if future versions still support py2
207-
_HIGHEST_PYTHON_2_VERSION = [2, 0, 0]
207+
_HIGHEST_PYTHON_2_VERSION = [2, 1, 0]
208208

209209
def __init__(
210210
self,

‎tests/integ/test_pytorch_train.py‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121

2222
from sagemaker.pytorch.estimator import PyTorch
2323
from sagemaker.pytorch.model import PyTorchModel
24+
from sagemaker.pytorch.defaults import LATEST_PY2_VERSION
2425
from sagemaker.utils import sagemaker_timestamp
2526

2627
MNIST_DIR = os.path.join(DATA_DIR, "pytorch_mnist")
@@ -38,6 +39,10 @@ def fixture_training_job(sagemaker_session, pytorch_full_version, cpu_instance_t
3839

3940
@pytest.mark.canary_quick
4041
@pytest.mark.regional_testing
42+
@pytest.mark.skipif(
43+
PYTHON_VERSION == "py2",
44+
reason="Python 2 is supported by PyTorch {} and lower versions.".format(LATEST_PY2_VERSION),
45+
)
4146
def test_sync_fit_deploy(pytorch_training_job, sagemaker_session, cpu_instance_type):
4247
# TODO: add tests against local mode when it's ready to be used
4348
endpoint_name = "test-pytorch-sync-fit-attach-deploy{}".format(sagemaker_timestamp())
@@ -55,6 +60,10 @@ def test_sync_fit_deploy(pytorch_training_job, sagemaker_session, cpu_instance_t
5560

5661

5762
@pytest.mark.local_mode
63+
@pytest.mark.skipif(
64+
PYTHON_VERSION == "py2",
65+
reason="Python 2 is supported by PyTorch {} and lower versions.".format(LATEST_PY2_VERSION),
66+
)
5867
def test_fit_deploy(sagemaker_local_session, pytorch_full_version):
5968
pytorch = PyTorch(
6069
entry_point=MNIST_SCRIPT,
@@ -79,6 +88,10 @@ def test_fit_deploy(sagemaker_local_session, pytorch_full_version):
7988
predictor.delete_endpoint()
8089

8190

91+
@pytest.mark.skipif(
92+
PYTHON_VERSION == "py2",
93+
reason="Python 2 is supported by PyTorch {} and lower versions.".format(LATEST_PY2_VERSION),
94+
)
8295
def test_deploy_model(pytorch_training_job, sagemaker_session, cpu_instance_type):
8396
endpoint_name = "test-pytorch-deploy-model-{}".format(sagemaker_timestamp())
8497

0 commit comments

Comments
 (0)