add inference pipelines and SparkML serving

Author: orchidmajumder

CHANGELOG.rst

@@ -363,4 +363,4 @@ CHANGELOG
 1.0.0
 =====
 
-* Initial commit
+* Initial commit

README.rst

@@ -32,13 +32,15 @@ Table of Contents
 4. `TensorFlow SageMaker Estimators <#tensorflow-sagemaker-estimators>`__
 5. `Chainer SageMaker Estimators <#chainer-sagemaker-estimators>`__
 6. `PyTorch SageMaker Estimators <#pytorch-sagemaker-estimators>`__
-7. `AWS SageMaker Estimators <#aws-sagemaker-estimators>`__
-8. `BYO Docker Containers with SageMaker Estimators <#byo-docker-containers-with-sagemaker-estimators>`__
-9. `SageMaker Automatic Model Tuning <#sagemaker-automatic-model-tuning>`__
-10. `SageMaker Batch Transform <#sagemaker-batch-transform>`__
-11. `Secure Training and Inference with VPC <#secure-training-and-inference-with-vpc>`__
-12. `BYO Model <#byo-model>`__
-13. `SageMaker Workflow <#sagemaker-workflow>`__
+7. `SageMaker SparkML Serving <#sagemaker-sparkml-serving>`__
+8. `AWS SageMaker Estimators <#aws-sagemaker-estimators>`__
+9. `BYO Docker Containers with SageMaker Estimators <#byo-docker-containers-with-sagemaker-estimators>`__
+10. `SageMaker Automatic Model Tuning <#sagemaker-automatic-model-tuning>`__
+11. `SageMaker Batch Transform <#sagemaker-batch-transform>`__
+12. `Secure Training and Inference with VPC <#secure-training-and-inference-with-vpc>`__
+13. `BYO Model <#byo-model>`__
+14. `Inference Pipelines <#inference-pipelines>`__
+15. `SageMaker Workflow <#sagemaker-workflow>`__
 
 
 Installing the SageMaker Python SDK
@@ -374,7 +376,7 @@ For more information, see `TensorFlow SageMaker Estimators and Models`_.
 
 
 Chainer SageMaker Estimators
--------------------------------
+----------------------------
 
 By using Chainer SageMaker ``Estimators``, you can train and host Chainer models on Amazon SageMaker.
 
@@ -390,7 +392,7 @@ For more information about  Chainer SageMaker ``Estimators``, see `Chainer SageM
 
 
 PyTorch SageMaker Estimators
--------------------------------
+----------------------------
 
 With PyTorch SageMaker ``Estimators``, you can train and host PyTorch models on Amazon SageMaker.
 
@@ -408,6 +410,39 @@ For more information about PyTorch SageMaker ``Estimators``, see `PyTorch SageMa
 .. _PyTorch SageMaker Estimators and Models: src/sagemaker/pytorch/README.rst
 
 
+SageMaker SparkML Serving
+-------------------------
+
+With SageMaker SparkML Serving, you can now perform predictions against a SparkML Model in SageMaker.
+In order to host a SparkML model in SageMaker, it should be serialized with ``MLeap`` library.
+
+For more information on MLeap, see https://github.com/combust/mleap .
+
+Supported major version of Spark: 2.2 (MLeap version - 0.9.6)
+
+Here is an example on how to create an instance of  ``SparkMLModel`` class and use ``deploy()`` method to create an
+endpoint which can be used to perform prediction against your trained SparkML Model.
+
+.. code:: python
+
+    sparkml_model = SparkMLModel(model_data='s3://path/to/model.tar.gz', env={'SAGEMAKER_SPARKML_SCHEMA': schema})
+    model_name = 'sparkml-model'
+    endpoint_name = 'sparkml-endpoint'
+    predictor = sparkml_model.deploy(initial_instance_count=1, instance_type='ml.c4.xlarge', endpoint_name=endpoint_name)
+
+Once the model is deployed, we can invoke the endpoint with a ``CSV`` payload like this:
+
+.. code:: python
+
+    payload = 'field_1,field_2,field_3,field_4,field_5'
+    predictor.predict(payload)
+
+
+For more information about the different ``content-type`` and ``Accept`` formats as well as the structure of the
+``schema`` that SageMaker SparkML Serving recognizes, please see `SageMaker SparkML Serving Container`_.
+
+.. _SageMaker SparkML Serving Container: https://github.com/aws/sagemaker-sparkml-serving-container
+
 AWS SageMaker Estimators
 ------------------------
 Amazon SageMaker provides several built-in machine learning algorithms that you can use to solve a variety of problems.
@@ -709,11 +744,45 @@ This returns a predictor the same way an ``Estimator`` does when ``deploy()`` is
 A full example is available in the `Amazon SageMaker examples repository <https://github.com/awslabs/amazon-sagemaker-examples/tree/master/advanced_functionality/mxnet_mnist_byom>`__.
 
 
+Inference Pipelines
+-------------------
+You can create a Pipeline for realtime or batch inference comprising of one or multiple model containers. This will help
+you to deploy an ML pipeline behind a single endpoint and you can have one API call perform pre-processing, model-scoring
+and post-processing on your data before returning it back as the response.
+
+For this, you have to create a ``PipelineModel`` which will take a list of ``Model`` objects. Calling ``deploy()`` on the
+``PipelineModel`` will provide you with an endpoint which can be invoked to perform the prediction on a data point against
+the ML Pipeline.
+
+.. code:: python
+
+   xgb_image = get_image_uri(sess.boto_region_name, 'xgboost', repo_version="latest")
+   xgb_model = Model(model_data='s3://path/to/model.tar.gz', image=xgb_image)
+   sparkml_model = SparkMLModel(model_data='s3://path/to/model.tar.gz', env={'SAGEMAKER_SPARKML_SCHEMA': schema})
+
+   model_name = 'inference-pipeline-model'
+   endpoint_name = 'inference-pipeline-endpoint'
+   sm_model = PipelineModel(name=model_name, role=sagemaker_role, models=[sparkml_model, xgb_model])
+
+This will define a ``PipelineModel`` consisting of SparkML model and an XGBoost model stacked sequentially. For more
+information about how to train an XGBoost model, please refer to the XGBoost notebook here_.
+
+.. _here: https://docs.aws.amazon.com/sagemaker/latest/dg/xgboost.html#xgboost-sample-notebooks
+
+.. code:: python
+
+   sm_model.deploy(initial_instance_count=1, instance_type='ml.c5.xlarge', endpoint_name=endpoint_name)
+
+This returns a predictor the same way an ``Estimator`` does when ``deploy()`` is called. Whenever you make an inference
+request using this predictor, you should pass the data that the first container expects and the predictor will return the
+output from the last container.
+
+
 SageMaker Workflow
 ------------------
 
 You can use Apache Airflow to author, schedule and monitor SageMaker workflow.
 
 For more information, see `SageMaker Workflow in Apache Airflow`_.
 
-.. _SageMaker Workflow in Apache Airflow: src/sagemaker/workflow/README.rst
+.. _SageMaker Workflow in Apache Airflow: src/sagemaker/workflow/README.rst

doc/pipeline.rst

@@ -0,0 +1,7 @@
+PipelineModel
+-------------
+
+.. autoclass:: sagemaker.pipeline.PipelineModel
+    :members:
+    :undoc-members:
+    :show-inheritance:

doc/sagemaker.sparkml.rst

@@ -0,0 +1,18 @@
+SparkML Serving
+===============
+
+SparkML Model
+-------------
+
+.. autoclass:: sagemaker.sparkml.model.SparkMLModel
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+SparkML Predictor
+-----------------
+
+.. autoclass:: sagemaker.sparkml.model.SparkMLPredictor
+    :members:
+    :undoc-members:
+    :show-inheritance:

src/sagemaker/__init__.py

@@ -30,9 +30,10 @@
 from sagemaker.local.local_session import LocalSession  # noqa: F401
 
 from sagemaker.model import Model  # noqa: F401
+from sagemaker.pipeline import PipelineModel  # noqa: F401
 from sagemaker.predictor import RealTimePredictor  # noqa: F401
 from sagemaker.session import Session  # noqa: F401
-from sagemaker.session import container_def  # noqa: F401
+from sagemaker.session import container_def, pipeline_container_def  # noqa: F401
 from sagemaker.session import production_variant  # noqa: F401
 from sagemaker.session import s3_input  # noqa: F401
 from sagemaker.session import get_execution_role  # noqa: F401

src/sagemaker/fw_registry.py

@@ -0,0 +1,86 @@
+# Copyright 2017-2018 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License"). You
+# may not use this file except in compliance with the License. A copy of
+# the License is located at
+#
+#     http://aws.amazon.com/apache2.0/
+#
+# or in the "license" file accompanying this file. This file is
+# distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF
+# ANY KIND, either express or implied. See the License for the specific
+# language governing permissions and limitations under the License.
+from __future__ import absolute_import
+import logging
+
+image_registry_map = {
+    "us-west-1": {
+        "sparkml-serving": "746614075791",
+        "scikit-learn": "746614075791"
+    },
+    "us-west-2": {
+        "sparkml-serving": "246618743249",
+        "scikit-learn": "246618743249"
+    },
+    "us-east-1": {
+        "sparkml-serving": "683313688378",
+        "scikit-learn": "683313688378"
+    },
+    "us-east-2": {
+        "sparkml-serving": "257758044811",
+        "scikit-learn": "257758044811"
+    },
+    "ap-northeast-1": {
+        "sparkml-serving": "354813040037",
+        "scikit-learn": "354813040037"
+    },
+    "ap-northeast-2": {
+        "sparkml-serving": "366743142698",
+        "scikit-learn": "366743142698"
+    },
+    "ap-southeast-1": {
+        "sparkml-serving": "121021644041",
+        "scikit-learn": "121021644041"
+    },
+    "ap-southeast-2": {
+        "sparkml-serving": "783357654285",
+        "scikit-learn": "783357654285"
+    },
+    "ap-south-1": {
+        "sparkml-serving": "720646828776",
+        "scikit-learn": "720646828776"
+    },
+    "eu-west-1": {
+        "sparkml-serving": "141502667606",
+        "scikit-learn": "141502667606"
+    },
+    "eu-west-2": {
+        "sparkml-serving": "764974769150",
+        "scikit-learn": "764974769150"
+    },
+    "eu-central-1": {
+        "sparkml-serving": "492215442770",
+        "scikit-learn": "492215442770"
+    },
+    "ca-central-1": {
+        "sparkml-serving": "341280168497",
+        "scikit-learn": "341280168497"
+    },
+    "us-gov-west-1": {
+        "sparkml-serving": "414596584902",
+        "scikit-learn": "414596584902"
+    }
+}
+
+
+def registry(region_name, framework=None):
+    """
+    Return docker registry for the given AWS region for the given framework.
+    This is only used for SparkML and Scikit-learn for now.
+    """
+    try:
+        account_id = image_registry_map[region_name][framework]
+        return "{}.dkr.ecr.{}.amazonaws.com".format(account_id, region_name)
+    except KeyError:
+        logging.error("The specific image or region does not exist")
+        raise

src/sagemaker/model.py

@@ -15,17 +15,13 @@
 import logging
 
 import sagemaker
-
-from sagemaker import local
-from sagemaker import fw_utils
-from sagemaker import session
-from sagemaker import utils
+from sagemaker import fw_utils, local, session, utils
 
 
 class Model(object):
     """A SageMaker ``Model`` that can be deployed to an ``Endpoint``."""
 
-    def __init__(self, model_data, image, role, predictor_cls=None, env=None, name=None, vpc_config=None,
+    def __init__(self, model_data, image, role=None, predictor_cls=None, env=None, name=None, vpc_config=None,
                  sagemaker_session=None):
         """Initialize an SageMaker ``Model``.
 
@@ -34,8 +30,9 @@ def __init__(self, model_data, image, role, predictor_cls=None, env=None, name=N
             image (str): A Docker image URI.
             role (str): An AWS IAM role (either name or full ARN). The Amazon SageMaker training jobs and APIs
                 that create Amazon SageMaker endpoints use this role to access training data and model artifacts.
-                After the endpoint is created, the inference code might use the IAM role,
-                if it needs to access an AWS resource.
+                After the endpoint is created, the inference code might use the IAM role if it needs to access some AWS
+                resources. It can be null if this is being used to create a Model to pass to a ``PipelineModel`` which
+                has its own Role field. (default: None)
             predictor_cls (callable[string, sagemaker.session.Session]): A function to call to create
                a predictor (default: None). If not None, ``deploy`` will return the result of invoking
                this function on the created endpoint name.
@@ -89,6 +86,7 @@ def deploy(self, initial_instance_count, instance_type, endpoint_name=None, tags
                 ``Endpoint`` created from this ``Model``.
             endpoint_name (str): The name of the endpoint to create (default: None).
                 If not specified, a unique endpoint name will be created.
+            tags(List[dict[str, str]]): The list of tags to attach to this specific endpoint.
 
         Returns:
             callable[string, sagemaker.session.Session] or None: Invocation of ``self.predictor_cls`` on
@@ -102,6 +100,8 @@ def deploy(self, initial_instance_count, instance_type, endpoint_name=None, tags
 
         container_def = self.prepare_container_def(instance_type)
         self.name = self.name or utils.name_from_image(container_def['Image'])
+        if self.role is None:
+            raise ValueError("Role can not be null for deploying a model")
         self.sagemaker_session.create_model(self.name, self.role, container_def, vpc_config=self.vpc_config)
         production_variant = sagemaker.production_variant(self.name, instance_type, initial_instance_count)
         self.endpoint_name = endpoint_name or self.name