Merge branch 'master' into fix-duplicate-transformer-docstring

Author: admivsn

.github/PULL_REQUEST_TEMPLATE.md

@@ -22,5 +22,6 @@ _Put an `x` in the boxes that apply. You can also fill these out after creating
 - [ ] I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes
 - [ ] I have checked that my tests are not configured for a specific region or account (if appropriate)
 - [ ] I have used [`unique_name_from_base`](https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/utils.py#L77) to create resource names in integ tests (if appropriate)
+- [ ] If adding any dependency in requirements.txt files, I have spell checked and ensured they exist in PyPi
 
 By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

.pylintrc

@@ -310,7 +310,7 @@ ignore-mixin-members=yes
 # (useful for modules/projects where namespaces are manipulated during runtime
 # and thus existing member attributes cannot be deduced by static analysis. It
 # supports qualified module names, as well as Unix pattern matching.
-ignored-modules=distutils
+ignored-modules=
 
 # List of class names for which member attributes should not be checked (useful
 # for classes with dynamically set attributes). This supports the use of
@@ -384,7 +384,7 @@ max-returns=6
 max-branches=12
 
 # Maximum number of statements in function / method body
-max-statements=100
+max-statements=105
 
 # Maximum number of parents for a class (see R0901).
 max-parents=7

CHANGELOG.md

@@ -1,5 +1,120 @@
 # Changelog
 
+## v2.232.3 (2024-10-30)
+
+### Bug Fixes and Other Changes
+
+ * update image_uri_configs  10-29-2024 07:17:56 PST
+ * Skip pytorch tests incompatible with latest version 2.4.0
+ * adding eu-central-2 bucket info to JS constants
+ * update image_uri_configs  10-23-2024 11:26:03 PST
+ * update image_uri_configs  10-17-2024 07:17:55 PST
+ * update image_uri_configs  10-03-2024 07:17:59 PST
+ * update image_uri_configs  09-27-2024 07:18:01 PST
+ * modified pull request template
+ * fixing typo in dependecy setup
+ * release: huggingface tgi neuronx 0.0.25 image
+ * Revert "update cloudpickle version to >=2.2.1 in pyproject.toml (#4899)"
+ * update cloudpickle version to >=2.2.1 in pyproject.toml
+ * update cloudpickle version to >=2.2.1
+ * chore(deps): bump pyspark from 3.3.1 to 3.3.2 in /requirements/extras
+ * changes for PT 2.4 currency upgrade
+ * chore: add lmi image config in me-central-1
+ * tests: Implement integration tests covering JumpStart PrivateHub workflows
+ * Use Miniforge to replace MambaForge
+
+## v2.232.2 (2024-10-03)
+
+### Bug Fixes and Other Changes
+
+ * Pass kwargs to HuggingFaceModel.deploy()
+ * improve logging and exception messages
+ * remove deprecated distutils
+ * update image_uri_configs  09-24-2024 07:18:00 PST
+
+## v2.232.1 (2024-09-19)
+
+### Bug Fixes and Other Changes
+
+ * update image_uri_configs  09-17-2024 07:17:54 PST
+ * support latest container version in image_uris and DJLModel for lmi c…
+
+## v2.232.0 (2024-09-12)
+
+### Features
+
+ * add deployment config name in modelbuilder telemetry
+ * add Clarify image URIs for us-isof
+
+### Bug Fixes and Other Changes
+
+ * chore: add flaky test markers & skip region with low P3 instance capacity
+ * update image_uri_configs  09-11-2024 11:54:11 PST
+ * update image_uri_configs  09-10-2024 07:18:01 PST
+ * [change] add us-gov and cn region repo accounts to djl and hugging face image metadata
+ * update image_uri_configs  09-06-2024 07:17:55 PST
+ * add us-gov region repo accounts to djl image metadata
+ * pass name from modelbuilder constructor to created model
+
+## v2.231.0 (2024-08-30)
+
+### Features
+
+ * Add SageMaker Core to the dependency
+
+### Bug Fixes and Other Changes
+
+ * Disable test_mnist_async
+ * SMP v2.5
+ * update image_uri_configs  08-29-2024 07:17:59 PST
+
+## v2.230.0 (2024-08-28)
+
+### Features
+
+ * FastAPI integration for In_Process Mode (2/2)
+
+### Bug Fixes and Other Changes
+
+ * chore: add HF LLM neuronx 0.0.24 image
+ * TF-2.16 test modification and handling
+ * fix test fail
+ * Add troubleshooting links to exceptions
+ * cross account private hub model fine-tuning
+ * chore: cleanup jumpstart factory
+ * disable failing integration tests
+
+## v2.229.0 (2024-08-15)
+
+### Features
+
+ * Support for ModelBuilder In_Process Mode (1/2)
+ * Pulling in dependencies (in_process mode) using conda environment
+ * Add optional CodeArtifact login to FrameworkProcessing job script
+ * implemented security-monitoring to send metrics to CW #1510
+
+### Bug Fixes and Other Changes
+
+ * alt configs model deployment and training issues
+ * fix keras extension in integ test
+ * update image_uri_configs  08-13-2024 07:17:54 PST
+ * trn1 instance family does not support volume size
+ * Update model.py
+ * removed log statement
+ * update image_uri_configs  08-09-2024 07:18:00 PST
+ * Added torchrun compatibility for distributet training across multiple GPUs in a single node (single instance)
+ * BiasConfig type hint
+ * add model monitor image accounts for ap-southeast-5 and eu-central-2
+ * aligned UTC times with PST
+ * ensure hpt jobs inherit tags from config
+ * add JumpStart PDT and OSU regions
+ * chore(deps): bump certifi in /src/sagemaker/serve/utils
+ * Updates for DJL 0.29.0 release
+ * chore(deps): bump apache-airflow from 2.9.2 to 2.9.3 in /requirements/extras
+ * chore(deps): bump torch from 2.0.1 to 2.2.0 in /tests/data/serve_resources/mlflow/pytorch
+ * avoided printing stack trace and escaped input
+ * removing kwargs as this is breaking predictor_cls param for mode…
+
 ## v2.228.0 (2024-08-06)
 
 ### Features

VERSION

@@ -1 +1 @@
-2.228.1.dev0
+2.232.4.dev0

pyproject.toml

@@ -47,6 +47,8 @@ dependencies = [
   "psutil",
   "PyYAML~=6.0",
   "requests",
+  "sagemaker-core>=1.0.0,<2.0.0",
+  "sagemaker-mlflow",
   "schema",
   "smdebug_rulesconfig==1.0.1",
   "tblib>=1.7.0,<4",

requirements/extras/feature-processor_requirements.txt

@@ -1,2 +1,2 @@
-pyspark==3.3.1
+pyspark==3.3.2
 sagemaker-feature-store-pyspark-3.3

requirements/extras/huggingface_requirements.txt

@@ -1,2 +1,5 @@
 accelerate>=0.24.1,<=0.27.0
 sagemaker_schema_inference_artifacts>=0.0.5
+uvicorn>=0.30.1
+fastapi>=0.111.0
+nest-asyncio

requirements/extras/test_requirements.txt

@@ -1,4 +1,5 @@
 tox==3.24.5
+numpy>=1.24.0
 build[virtualenv]==1.2.1
 flake8==4.0.1
 pytest==6.2.5
@@ -23,7 +24,7 @@ Jinja2==3.1.4
 pyvis==0.2.1
 pandas>=1.3.5,<1.5
 scikit-learn==1.3.0
-cloudpickle==2.2.1
+cloudpickle>=2.2.1
 PyYAML==6.0
 # TODO find workaround
 xgboost>=1.6.2,<=1.7.6
@@ -40,3 +41,6 @@ schema==0.7.5
 tensorflow>=2.1,<=2.16
 mlflow>=2.12.2,<2.13
 huggingface_hub>=0.23.4
+uvicorn>=0.30.1
+fastapi>=0.111.0
+nest-asyncio

src/sagemaker/algorithm.py

@@ -157,6 +157,20 @@ def __init__(
                 available (default: ``None``).
             **kwargs: Additional kwargs. This is unused. It's only added for AlgorithmEstimator
                 to ignore the irrelevant arguments.
+
+        Raises:
+            ValueError:
+            - If an AWS IAM Role is not provided.
+            - Bad value for instance type.
+            RuntimeError:
+            - When setting up custom VPC, both subnets and security_group_ids are not provided
+            - If instance_count > 1 (distributed training) with instance type local or local gpu
+            - If LocalSession is not used with instance type local or local gpu
+            - file:// output path used outside of local mode
+            botocore.exceptions.ClientError:
+            - algorithm arn is incorrect
+            - insufficient permission to access/ describe algorithm
+            - algorithm is in a different region
         """
         self.algorithm_arn = algorithm_arn
         super(AlgorithmEstimator, self).__init__(

src/sagemaker/base_predictor.py

@@ -430,6 +430,8 @@ def update_endpoint(
                 - If ``initial_instance_count``, ``instance_type``, or ``accelerator_type`` is
                   specified and either ``model_name`` is ``None`` or there are multiple models
                   associated with the endpoint.
+            botocore.exceptions.ClientError: If SageMaker throws an error while creating
+            endpoint config, describing endpoint or updating endpoint
         """
         production_variants = None
         current_model_names = self._get_model_names()