Skip to content

Commit 0473a8e

Browse files
rubanhbalajisankar15Balaji SankarRuban HussainIvy Bazan
authored
feature: Support for SageMaker Intelligent Defaults (#3757)
* feature: Added Config parser for SageMaker Python SDK (#840) Co-authored-by: Balaji Sankar <[email protected]> * intelligent defaults - tags and encryption (#842) * feature: sagemaker config - support tags for all APIs * feature: sagemaker config - support EnableInterContainerTrafficEncryption for relevant APIs --------- Co-authored-by: Ruban Hussain <[email protected]> * intelligent defaults - custom parameters and small fixes (#845) * fix: sagemaker-config - S3 session, tuning tags, config schema test side-effects * feature: sagemaker-config - support for custom parameters in config schema --------- Co-authored-by: Ruban Hussain <[email protected]> * feature: Added support for VPC Config, EnableNetworkIsolation, KMS Key ID, Volume KMS Key ID, IAM role to be fetched from Config (#846) Co-authored-by: Balaji Sankar <[email protected]> * fix: Make Key, Value as required fields for each "Tags" entry in the config file. * fix: Make 'role' as Optional for ModelQualityMonitor and DefaultModelMonitor, and fixed PROCESSING_CONFIG_PATH (#849) Co-authored-by: Balaji Sankar <[email protected]> * Fix: Certain unit tests aren't passing sagemaker_session. Modify the logic to accommodate that case (#850) Co-authored-by: Balaji Sankar <[email protected]> * fix: Sagemaker Config - KeyError: 'MonitoringJobDefinition' in model_monitoring * change: Sagemaker Config - improved readability of print statements and simplified its code * fix: Sagemaker Config - Reduce duplicate and misleading config-related print statements * fix: Sagemaker Config - add function description * fix: Sagemaker Config - Fix failing Integ tests, fix backwards incompatible behavior, and improved some unit tests * change: new integ test for sagemaker_config * fix: Sagemaker Config - fleshed out unit tests and fixed bugs * fix: Sagemaker Config - Removed hard coded config values in the unit tests * fix: inject from config into existing ProductionVariants inside create_endpoint_config_from_existing * change: added unit test for verifying yaml safe_load method * change: addressed PR comments for SageMaker Config * change: Sagemaker Config - minor clarification * change: ModelMonitoring and Processing now use helper methods for updating NetworkConfig * change: Refactoring session.py and added additional schema validation for ValidationProfiles * update: expand one unit test * update: new integ test for cross context injection * change: remove unwanted method and replace it with a different method for config injection * fix: Address documentation errors and removed unnecessary properties and setters * fix: moving certain config file helper methods to utils.py * change: Add a separate helper to merge list of objects * fix: Documentation updates for SageMakerConfig * fix: bubble up exceptions from S3 while fetching the Config * fix: Added additional test cases for config helper methods. Also made minor documentation updates. * fix: small bug fix to print statements for update_list_of_dicts_with_values_from_config * fix: Replace SageMakerConfig class with just method invocations * fix: fix broken unit tests due to refactoring * fix: bug where a user-provided sagemaker_config wasnt set * change: rename fetch_sagemaker_config to load_sagemaker_config * fix: update Schema to match exactly with APIs * add documentation for default configuration support * fix linting errors * fix link lint * fix lint --------- Co-authored-by: Balaji Sankar <[email protected]> Co-authored-by: Balaji Sankar <[email protected]> Co-authored-by: Ruban Hussain <[email protected]> Co-authored-by: Ivy Bazan <[email protected]>
1 parent 3fed846 commit 0473a8e

File tree

130 files changed

+7152
-285
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

130 files changed

+7152
-285
lines changed

‎doc/api/utility/config.rst

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
Config
2+
-------
3+
4+
.. automodule:: sagemaker.config.config
5+
:members:
6+
:undoc-members:
7+
:show-inheritance:

‎doc/overview.rst

Lines changed: 572 additions & 0 deletions
Large diffs are not rendered by default.

‎setup.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,9 @@ def read_requirements(filename):
5959
"pandas",
6060
"pathos",
6161
"schema",
62+
"PyYAML==5.4.1",
63+
"jsonschema",
64+
"platformdirs",
6265
]
6366

6467
# Specific use case dependencies

‎src/sagemaker/algorithm.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ class AlgorithmEstimator(EstimatorBase):
4646
def __init__(
4747
self,
4848
algorithm_arn: str,
49-
role: str,
49+
role: str = None,
5050
instance_count: Optional[Union[int, PipelineVariable]] = None,
5151
instance_type: Optional[Union[str, PipelineVariable]] = None,
5252
volume_size: Union[int, PipelineVariable] = 30,

‎src/sagemaker/amazon/amazon_estimator.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ class AmazonAlgorithmEstimatorBase(EstimatorBase):
5050

5151
def __init__(
5252
self,
53-
role: str,
53+
role: Optional[Union[str, PipelineVariable]] = None,
5454
instance_count: Optional[Union[int, PipelineVariable]] = None,
5555
instance_type: Optional[Union[str, PipelineVariable]] = None,
5656
data_location: Optional[str] = None,

‎src/sagemaker/amazon/factorization_machines.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ class FactorizationMachines(AmazonAlgorithmEstimatorBase):
8787

8888
def __init__(
8989
self,
90-
role: str,
90+
role: Optional[Union[str, PipelineVariable]] = None,
9191
instance_count: Optional[Union[int, PipelineVariable]] = None,
9292
instance_type: Optional[Union[str, PipelineVariable]] = None,
9393
num_factors: Optional[int] = None,
@@ -326,7 +326,7 @@ class FactorizationMachinesModel(Model):
326326
def __init__(
327327
self,
328328
model_data: Union[str, PipelineVariable],
329-
role: str,
329+
role: Optional[str] = None,
330330
sagemaker_session: Optional[Session] = None,
331331
**kwargs
332332
):

‎src/sagemaker/amazon/ipinsights.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ class IPInsights(AmazonAlgorithmEstimatorBase):
6363

6464
def __init__(
6565
self,
66-
role: str,
66+
role: Optional[Union[str, PipelineVariable]] = None,
6767
instance_count: Optional[Union[int, PipelineVariable]] = None,
6868
instance_type: Optional[Union[str, PipelineVariable]] = None,
6969
num_entity_vectors: Optional[int] = None,
@@ -229,7 +229,7 @@ class IPInsightsModel(Model):
229229
def __init__(
230230
self,
231231
model_data: Union[str, PipelineVariable],
232-
role: str,
232+
role: Optional[str] = None,
233233
sagemaker_session: Optional[Session] = None,
234234
**kwargs
235235
):

‎src/sagemaker/amazon/kmeans.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ class KMeans(AmazonAlgorithmEstimatorBase):
6262

6363
def __init__(
6464
self,
65-
role: str,
65+
role: Optional[Union[str, PipelineVariable]] = None,
6666
instance_count: Optional[Union[int, PipelineVariable]] = None,
6767
instance_type: Optional[Union[str, PipelineVariable]] = None,
6868
k: Optional[int] = None,
@@ -255,7 +255,7 @@ class KMeansModel(Model):
255255
def __init__(
256256
self,
257257
model_data: Union[str, PipelineVariable],
258-
role: str,
258+
role: Optional[str] = None,
259259
sagemaker_session: Optional[Session] = None,
260260
**kwargs
261261
):

‎src/sagemaker/amazon/knn.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ class KNN(AmazonAlgorithmEstimatorBase):
7373

7474
def __init__(
7575
self,
76-
role: str,
76+
role: Optional[Union[str, PipelineVariable]] = None,
7777
instance_count: Optional[Union[int, PipelineVariable]] = None,
7878
instance_type: Optional[Union[str, PipelineVariable]] = None,
7979
k: Optional[int] = None,
@@ -246,7 +246,7 @@ class KNNModel(Model):
246246
def __init__(
247247
self,
248248
model_data: Union[str, PipelineVariable],
249-
role: str,
249+
role: Optional[str] = None,
250250
sagemaker_session: Optional[Session] = None,
251251
**kwargs
252252
):

‎src/sagemaker/amazon/lda.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ class LDA(AmazonAlgorithmEstimatorBase):
5252

5353
def __init__(
5454
self,
55-
role: str,
55+
role: Optional[Union[str, PipelineVariable]] = None,
5656
instance_type: Optional[Union[str, PipelineVariable]] = None,
5757
num_topics: Optional[int] = None,
5858
alpha0: Optional[float] = None,
@@ -230,7 +230,7 @@ class LDAModel(Model):
230230
def __init__(
231231
self,
232232
model_data: Union[str, PipelineVariable],
233-
role: str,
233+
role: Optional[str] = None,
234234
sagemaker_session: Optional[Session] = None,
235235
**kwargs
236236
):

0 commit comments

Comments
 (0)