Broken instructions in "Local Mode" section #4394
Description
What did you find confusing? Please describe.
The instructions in the "Local Mode section" are imprecise and is not working properly.
Describe how documentation can be improved
- "The local mode uses a YAML configuration file located at ~/.sagemaker/config.yaml to define the default values that are automatically passed to the config attribute of LocalSession."
That is not (totally?) true.
❯ python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from sagemaker.local import LocalSession
sagemaker.config INFO - Not applying SDK defaults from location: /etc/xdg/xdg-ubuntu/sagemaker/config.yaml
sagemaker.config INFO - Not applying SDK defaults from location: /home/tapyu/.config/sagemaker/config.yaml
>>>
For user wide, Python looks config.yaml on $XDG_CONFIG_HOME, which is ~/.config. sagemaker simply ignores ~/.sagemaker/config.yaml.
local:
local_code: true # Using everything locally
region_name: "us-west-2" # Name of the region
container_config: # Additional docker container config
shm_size: "128MThe double quote of 128M is incomplete
from sagemaker.local import LocalSession
sagemaker_session = LocalSession()
sagemaker_session.config = {'local': {'local_code': True}}This code is not working, even is I correct the .yaml file and put it in ~/.config:
❯ python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from sagemaker.local import LocalSession
sagemaker.config INFO - Not applying SDK defaults from location: /etc/xdg/xdg-ubuntu/sagemaker/config.yaml
Unable to create default JumpStart SageMaker Session due to the following error: 'SchemaVersion' is a required property
Failed validating 'required' in schema:
{'$schema': 'https://json-schema.org/draft/2020-12/schema',
'additionalProperties': False,
'definitions': {'container': {'additionalProperties': False,
'properties': {'Environment': {'$ref': '#/definitions/environment-Length1024-Properties16'}},
'type': 'object'},
'containers': {'items': {'$ref': '#/definitions/container'},
'maxItems': 15,
'minItems': 0,
'type': 'array'},
'environment-Length1024-Properties16': {'additionalProperties': False,
'maxProperties': 16,
'patternProperties': {'([a-zA-Z_][a-zA-Z0-9_]*){1,1024}': {'maxLength': 1024,
'pattern': '[\\S\\s]*',
'type': 'string'}},
'type': 'object'},
'environment-Length10240-Properties16': {'additionalProperties': False,
'maxProperties': 16,
'patternProperties': {'[a-zA-Z_][a-zA-Z0-9_]{0,1023}': {'maxLength': 10240,
'pattern': '[\\S\\s]*',
'type': 'string'}},
'type': 'object'},
'environment-Length256-Properties100': {'additionalProperties': False,
'maxProperties': 100,
'patternProperties': {'([a-zA-Z_][a-zA-Z0-9_]*){1,256}': {'maxLength': 256,
'pattern': '[\\S\\s]*',
'type': 'string'}},
'type': 'object'},
'environment-Length256-Properties50': {'additionalProperties': False,
'maxProperties': 50,
'patternProperties': {'([a-zA-Z_][a-zA-Z0-9_]*){1,256}': {'maxLength': 256,
'pattern': '[\\S\\s]*',
'type': 'string'}},
'type': 'object'},
'environment-Length512-Properties48': {'additionalProperties': False,
'maxProperties': 48,
'patternProperties': {'([a-zA-Z_][a-zA-Z0-9_]*){1,512}': {'maxLength': 512,
'pattern': '[\\S\\s]*',
'type': 'string'}},
'type': 'object'},
'environmentVariables': {'additionalProperties': False,
'maxProperties': 48,
'patternProperties': {'([a-zA-Z_][a-zA-Z0-9_]*){1,512}': {'maxLength': 512,
'pattern': '[\\S\\s]*',
'type': 'string'}},
'type': 'object'},
'kmsKeyId': {'maxLength': 2048, 'type': 'string'},
'modelPackageContainers': {'items': {'$ref': '#/definitions/container'},
'maxItems': 15,
'minItems': 1,
'type': 'array'},
'preExecutionCommand': {'pattern': '.*',
'type': 'string'},
'processingInput': {'additionalProperties': False,
'properties': {'DatasetDefinition': {'additionalProperties': False,
'properties': {'AthenaDatasetDefinition': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'},
'RedshiftDatasetDefinition': {'additionalProperties': False,
'properties': {'ClusterRoleArn': {'$ref': '#/definitions/roleArn'},
'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'}},
'type': 'object'}},
'type': 'object'},
'productionVariant': {'additionalProperties': False,
'properties': {'CoreDumpConfig': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'}},
'type': 'object'},
'roleArn': {'maxLength': 2048,
'minLength': 20,
'pattern': '^arn:aws[a-z\\-]*:iam::\\d{12}:role/?[a-zA-Z_0-9+=,.@\\-_/]+$',
'type': 'string'},
's3Bucket': {'maxLength': 63,
'minLength': 3,
'pattern': '^[a-z0-9][\\.\\-a-z0-9]{1,61}[a-z0-9]$',
'type': 'string'},
's3Uri': {'maxLength': 1024,
'pattern': '^(https|s3)://([^/]+)/?(.*)$',
'type': 'string'},
'securityGroupId': {'maxLength': 32,
'pattern': '[-0-9a-zA-Z]+',
'type': 'string'},
'subnet': {'maxLength': 32,
'pattern': '[-0-9a-zA-Z]+',
'type': 'string'},
'tags': {'items': {'additionalProperties': False,
'properties': {'Key': {'maxLength': 128,
'minLength': 1,
'pattern': '^[\\w\\s\\d_.:/=+\\-@]*$',
'type': 'string'},
'Value': {'maxLength': 256,
'minLength': 0,
'pattern': '^[\\w\\s\\d_.:/=+\\-@]*$',
'type': 'string'}},
'required': ['Key', 'Value'],
'type': 'object'},
'maxItems': 50,
'minItems': 0,
'type': 'array'},
'validationProfile': {'additionalProperties': False,
'properties': {'TransformJobDefinition': {'additionalProperties': False,
'properties': {'Environment': {'$ref': '#/definitions/environment-Length10240-Properties16'},
'TransformOutput': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'},
'TransformResources': {'additionalProperties': False,
'properties': {'VolumeKmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'}},
'type': 'object'}},
'type': 'object'},
'vpcConfig': {'additionalProperties': False,
'properties': {'SecurityGroupIds': {'items': {'$ref': '#/definitions/securityGroupId'},
'maxItems': 5,
'minItems': 1,
'type': 'array'},
'Subnets': {'items': {'$ref': '#/definitions/subnet'},
'maxItems': 16,
'minItems': 1,
'type': 'array'}},
'type': 'object'}},
'properties': {'CustomParameters': {'additionalProperties': False,
'patternProperties': {'^[\\w\\s\\d_.:/=+\\-@]+$': {'type': 'string'}},
'type': 'object'},
'SageMaker': {'additionalProperties': False,
'properties': {'AutoMLJob': {'additionalProperties': False,
'properties': {'AutoMLJobConfig': {'additionalProperties': False,
'properties': {'SecurityConfig': {'additionalProperties': False,
'properties': {'EnableInterContainerTrafficEncryption': {'type': 'boolean'},
'VolumeKmsKeyId': {'$ref': '#/definitions/kmsKeyId'},
'VpcConfig': {'$ref': '#/definitions/vpcConfig'}},
'type': 'object'}},
'type': 'object'},
'OutputDataConfig': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'},
'RoleArn': {'$ref': '#/definitions/roleArn'},
'Tags': {'$ref': '#/definitions/tags'}},
'type': 'object'},
'CompilationJob': {'additionalProperties': False,
'properties': {'OutputConfig': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'},
'RoleArn': {'$ref': '#/definitions/roleArn'},
'Tags': {'$ref': '#/definitions/tags'},
'VpcConfig': {'$ref': '#/definitions/vpcConfig'}},
'type': 'object'},
'EdgePackagingJob': {'additionalProperties': False,
'properties': {'OutputConfig': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'},
'ResourceKey': {'$ref': '#/definitions/kmsKeyId'},
'RoleArn': {'$ref': '#/definitions/roleArn'},
'Tags': {'$ref': '#/definitions/tags'}},
'type': 'object'},
'Endpoint': {'additionalProperties': False,
'properties': {'Tags': {'$ref': '#/definitions/tags'}},
'type': 'object'},
'EndpointConfig': {'additionalProperties': False,
'properties': {'AsyncInferenceConfig': {'additionalProperties': False,
'properties': {'OutputConfig': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'}},
'type': 'object'},
'DataCaptureConfig': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'},
'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'},
'ProductionVariants': {'items': {'$ref': '#/definitions/productionVariant'},
'type': 'array'},
'Tags': {'$ref': '#/definitions/tags'}},
'type': 'object'},
'FeatureGroup': {'additionalProperties': False,
'properties': {'OfflineStoreConfig': {'additionalProperties': False,
'properties': {'S3StorageConfig': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'}},
'type': 'object'},
'OnlineStoreConfig': {'additionalProperties': False,
'properties': {'SecurityConfig': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'}},
'type': 'object'},
'RoleArn': {'$ref': '#/definitions/roleArn'},
'Tags': {'$ref': '#/definitions/tags'}},
'type': 'object'},
'Model': {'additionalProperties': False,
'properties': {'Containers': {'$ref': '#/definitions/containers'},
'EnableNetworkIsolation': {'type': 'boolean'},
'ExecutionRoleArn': {'$ref': '#/definitions/roleArn'},
'PrimaryContainer': {'$ref': '#/definitions/container'},
'Tags': {'$ref': '#/definitions/tags'},
'VpcConfig': {'$ref': '#/definitions/vpcConfig'}},
'type': 'object'},
'ModelPackage': {'additionalProperties': False,
'properties': {'InferenceSpecification': {'additionalProperties': False,
'properties': {'Containers': {'$ref': '#/definitions/modelPackageContainers'}},
'type': 'object'},
'ValidationSpecification': {'additionalProperties': False,
'properties': {'ValidationProfiles': {'items': {'$ref': '#/definitions/validationProfile'},
'maxItems': 1,
'minItems': 1,
'type': 'array'},
'ValidationRole': {'$ref': '#/definitions/roleArn'}},
'type': 'object'}},
'type': 'object'},
'MonitoringSchedule': {'additionalProperties': False,
'properties': {'MonitoringScheduleConfig': {'additionalProperties': False,
'properties': {'MonitoringJobDefinition': {'additionalProperties': False,
'properties': {'Environment': {'$ref': '#/definitions/environment-Length256-Properties50'},
'MonitoringOutputConfig': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'},
'MonitoringResources': {'additionalProperties': False,
'properties': {'ClusterConfig': {'additionalProperties': False,
'properties': {'VolumeKmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'}},
'type': 'object'},
'NetworkConfig': {'additionalProperties': False,
'properties': {'EnableInterContainerTrafficEncryption': {'type': 'boolean'},
'EnableNetworkIsolation': {'type': 'boolean'},
'VpcConfig': {'$ref': '#/definitions/vpcConfig'}},
'type': 'object'},
'RoleArn': {'$ref': '#/definitions/roleArn'}},
'type': 'object'}},
'type': 'object'},
'Tags': {'$ref': '#/definitions/tags'}},
'type': 'object'},
'Pipeline': {'additionalProperties': False,
'properties': {'RoleArn': {'$ref': '#/definitions/roleArn'},
'Tags': {'$ref': '#/definitions/tags'}},
'type': 'object'},
'ProcessingJob': {'additionalProperties': False,
'properties': {'Environment': {'$ref': '#/definitions/environment-Length256-Properties100'},
'NetworkConfig': {'additionalProperties': False,
'properties': {'EnableInterContainerTrafficEncryption': {'type': 'boolean'},
'EnableNetworkIsolation': {'type': 'boolean'},
'VpcConfig': {'$ref': '#/definitions/vpcConfig'}},
'type': 'object'},
'ProcessingInputs': {'items': {'$ref': '#/definitions/processingInput'},
'maxItems': 10,
'minItems': 0,
'type': 'array'},
'ProcessingOutputConfig': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'},
'ProcessingResources': {'additionalProperties': False,
'properties': {'ClusterConfig': {'additionalProperties': False,
'properties': {'VolumeKmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'}},
'type': 'object'},
'RoleArn': {'$ref': '#/definitions/roleArn'},
'Tags': {'$ref': '#/definitions/tags'}},
'type': 'object'},
'PythonSDK': {'additionalProperties': False,
'properties': {'Modules': {'additionalProperties': False,
'properties': {'Estimator': {'additionalProperties': False,
'properties': {'DebugHookConfig': {'description': 'Sets '
'a '
'boolean '
'for '
'`debugger_hook_config` '
'ofEstimator '
'which '
'will '
'be '
'then '
'used '
'for '
'training '
'jobAPI '
'call. '
'Today, '
'the '
'config_schema '
"doesn't "
'supporta '
'dictionary '
'as '
'a '
'valid '
'value '
'to '
'be '
'provided.In '
'the '
'future '
'to '
'add '
'support '
'for '
'DebugHookConfigas '
'a '
'dictionary, '
'schema '
'should '
'be '
'added '
'underthe '
'config '
'path '
'`SageMaker.TrainingJob` '
'instead '
'ofhere, '
'since '
'the '
'TrainingJob '
'API '
'supportsDebugHookConfig '
'as '
'a '
'dictionary, '
'we '
'can '
'adda '
'schema '
'for '
'it '
'at '
'API '
'level.',
'type': 'boolean'}},
'type': 'object'},
'NotebookJob': {'additionalProperties': False,
'properties': {'RoleArn': {'$ref': '#/definitions/roleArn'},
'S3KmsKeyId': {'$ref': '#/definitions/kmsKeyId'},
'S3RootUri': {'$ref': '#/definitions/s3Uri'},
'VolumeKmsKeyId': {'$ref': '#/definitions/kmsKeyId'},
'VpcConfig': {'$ref': '#/definitions/vpcConfig'}},
'type': 'object'},
'RemoteFunction': {'additionalProperties': False,
'properties': {'CustomFileFilter': {'additionalProperties': False,
'properties': {'IgnoreNamePatterns': {'items': {'type': 'string'},
'type': 'array'}},
'type': 'object'},
'Dependencies': {'type': 'string'},
'EnableInterContainerTrafficEncryption': {'type': 'boolean'},
'EnvironmentVariables': {'$ref': '#/definitions/environmentVariables'},
'ImageUri': {'type': 'string'},
'IncludeLocalWorkDir': {'type': 'boolean'},
'InstanceType': {'type': 'string'},
'JobCondaEnvironment': {'type': 'string'},
'PreExecutionCommands': {'items': {'$ref': '#/definitions/preExecutionCommand'},
'type': 'array'},
'PreExecutionScript': {'type': 'string'},
'RoleArn': {'$ref': '#/definitions/roleArn'},
'S3KmsKeyId': {'$ref': '#/definitions/kmsKeyId'},
'S3RootUri': {'$ref': '#/definitions/s3Uri'},
'Tags': {'$ref': '#/definitions/tags'},
'VolumeKmsKeyId': {'$ref': '#/definitions/kmsKeyId'},
'VpcConfig': {'$ref': '#/definitions/vpcConfig'}},
'type': 'object'},
'Serve': {'additionalProperties': False,
'properties': {'EnvironmentVariables': {'$ref': '#/definitions/environmentVariables'},
'InstanceType': {'type': 'string'},
'RoleArn': {'$ref': '#/definitions/roleArn'},
'S3ModelDataUri': {'$ref': '#/definitions/s3Uri'}},
'type': 'object'},
'Session': {'additionalProperties': False,
'properties': {'DefaultS3Bucket': {'$ref': '#/definitions/s3Bucket',
'description': 'sets '
'`default_bucket` '
'of '
'Session'},
'DefaultS3ObjectKeyPrefix': {'description': 'sets '
'`default_bucket_prefix` '
'of '
'Session',
'maxLength': 1024,
'minLength': 1,
'type': 'string'}},
'type': 'object'},
'TelemetryOptOut': {'type': 'boolean'}},
'type': 'object'}},
'type': 'object'},
'TrainingJob': {'additionalProperties': False,
'properties': {'EnableInterContainerTrafficEncryption': {'type': 'boolean'},
'EnableNetworkIsolation': {'type': 'boolean'},
'Environment': {'$ref': '#/definitions/environment-Length512-Properties48'},
'OutputDataConfig': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'},
'ProfilerConfig': {'additionalProperties': False,
'properties': {'DisableProfiler': {'type': 'boolean'}},
'type': 'object'},
'ResourceConfig': {'additionalProperties': False,
'properties': {'VolumeKmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'},
'RoleArn': {'$ref': '#/definitions/roleArn'},
'Tags': {'$ref': '#/definitions/tags'},
'VpcConfig': {'$ref': '#/definitions/vpcConfig'}},
'type': 'object'},
'TransformJob': {'additionalProperties': False,
'properties': {'DataCaptureConfig': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'},
'Environment': {'$ref': '#/definitions/environment-Length10240-Properties16'},
'Tags': {'$ref': '#/definitions/tags'},
'TransformOutput': {'additionalProperties': False,
'properties': {'KmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'},
'TransformResources': {'additionalProperties': False,
'properties': {'VolumeKmsKeyId': {'$ref': '#/definitions/kmsKeyId'}},
'type': 'object'}},
'type': 'object'}},
'type': 'object'},
'SchemaVersion': {'description': 'The schema version '
'of the document.',
'enum': ['1.0'],
'type': 'string'}},
'required': ['SchemaVersion'],
'type': 'object'}
On instance:
{'local': {'container_config': {'shm_size': '128M'},
'local_code': True,
'region_name': 'us-west-2'}}.
Additional context
1.
❯ pip list | rg 'sagemaker'
sagemaker 2.205.0
❯ python3 --version
Python 3.10.12