Moving code integration guide into this repo (#243)

* Moving code integration guide into this repo

* dtzar feedback

* add recommendation

Author: jotaylo

README.md

@@ -34,7 +34,7 @@ The build pipelines include DevOps tasks for data sanity tests, unit tests, mode
 
 ## Getting Started
 
-To deploy this solution in your subscription, follow the manual instructions in the [getting started](docs/getting_started.md) doc
+To deploy this solution in your subscription, follow the manual instructions in the [getting started](docs/getting_started.md) doc. Then optionally follow the guide for [integrating your own code](docs/custom_model.md) with this repository template.
 
 ### Repo Details
 

bootstrap/README.md

@@ -1,28 +1,18 @@
 # Bootstrap from MLOpsPython repository
 
-To use this existing project structure and scripts for your new ML project, you can quickly get started from the existing repository, bootstrap and create a template that works for your ML project. Bootstrapping will prepare a similar directory structure for your project which includes renaming files and folders, deleting and cleaning up some directories and fixing imports and absolute path based on your project name. This will enable reusing various resources like pre-built pipelines and scripts for your new project.
+To use this existing project structure and scripts for your new ML project, you can quickly get started from the existing repository, bootstrap and create a template that works for your ML project.
 
-## Generating the project structure
+Bootstrapping will prepare a directory structure for your project which includes:
 
-To bootstrap from the existing MLOpsPython repository clone this repository, ensure Python is installed locally, and run bootstrap.py script as below
+* renaming files and folders from the base project name `diabetes` to your project name
+* fixing imports and absolute path based on your project name
+* deleting and cleaning up some directories
 
-`python bootstrap.py --d [dirpath] --n [projectname]`
-
-Where `[dirpath]` is the absolute path to the root of your directory where MLOps repo is cloned and `[projectname]` is the name of your ML project.
-
-The script renames folders, files and files' content from the base project name `diabetes` to your project name. However, you might need to manually rename variables defined in a variable group and their values.
-
-[This article](https://docs.microsoft.com/azure/machine-learning/tutorial-convert-ml-experiment-to-production#use-your-own-model-with-mlopspython-code-template) will also assist to use this code template for your own ML project.
-
-### Using an existing dataset
+To bootstrap from the existing MLOpsPython repository:
 
-The training ML pipeline uses a [sample diabetes dataset](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_diabetes.html) as training data. To use your own data, you need to [create a Dataset](https://docs.microsoft.com/azure/machine-learning/how-to-create-register-datasets) in your workspace and add a DATASET_NAME variable in the ***devopsforai-aml-vg*** variable group with the Dataset name. You'll also need to modify the test cases in the **ml_service/util/smoke_test_scoring_service.py** script to match the schema of the training features in your dataset.
-
-## Customizing the CI and AML environments
-
-In your project you will want to customize your own Docker image and Conda environment to use only the dependencies and tools required for your use case. This requires you to edit the following environment definition files:
-
-- The Azure ML training and scoring Conda environment defined in [conda_dependencies.yml](diabetes_regression/conda_dependencies.yml).
-- The CI Docker image and Conda environment used by the Azure DevOps build agent. See [instructions for customizing the Azure DevOps job container](../docs/custom_container.md).
-
-You will want to synchronize dependency versions as appropriate between both environment definitions (for example, ML libraries used both in training and in unit tests).
+1. Ensure Python 3 is installed locally
+1. Clone this repository locally
+1. Run bootstrap.py script  
+`python bootstrap.py --d [dirpath] --n [projectname]`
+    * `[dirpath]` is the absolute path to the root of the directory where MLOpsPython is cloned
+    * `[projectname]` is the name of your ML project

docs/code_description.md

@@ -67,7 +67,8 @@ The repository provides a template with folders structure suitable for maintaini
 
 ### Training Step
 
-- `diabetes_regression/training/train.py` : a training step of an ML training pipeline.
+- `diabetes_regression/training/train_aml.py`: a training step of an ML training pipeline.
+- `diabetes_regression/training/train.py` : ML functionality called by train_aml.py
 - `diabetes_regression/training/R/r_train.r` : training a model with R basing on a sample dataset (weight_data.csv).
 - `diabetes_regression/training/R/train_with_r.py` : a python wrapper (ML Pipeline Step) invoking R training script on ML Compute
 - `diabetes_regression/training/R/train_with_r_on_databricks.py` : a python wrapper (ML Pipeline Step) invoking R training script on Databricks Compute

docs/custom_model.md

@@ -0,0 +1,80 @@
+# Bring your own code with the MLOpsPython repository template
+
+This document provides steps to follow when using this repository as a template to train models and deploy the models with real-time inference in Azure ML with your own scripts and data.
+
+1. Follow the MLOpsPython [Getting Started](https://github.com/microsoft/MLOpsPython/blob/master/docs/getting_started.md) guide
+1. Follow the MLOpsPython [bootstrap instructions](https://github.com/microsoft/MLOpsPython/blob/master/bootstrap/README.md) to create your project starting point
+1. Configure training data
+1. [If necessary] Convert your ML experimental code into production ready code
+1. Replace the training code
+1. Update the evaluation code
+1. Customize the build agent environment
+1. [If appropriate] Replace the score code
+
+## Follow the Getting Started guide
+
+Follow the [Getting Started](https://github.com/microsoft/MLOpsPython/blob/master/docs/getting_started.md) guide to set up the infrastructure and pipelines to execute MLOpsPython.
+
+## Follow the Bootstrap instructions
+
+The [Bootstrap from MLOpsPython repository](https://github.com/microsoft/MLOpsPython/blob/master/bootstrap/README.md) guide will help you to quickly prepare the repository for your project.
+
+**Note:** Since the bootstrap script will rename the `diabetes_regression` folder to the project name of your choice, we'll refer to your project as `[project name]` when paths are involved.
+
+## Configure training data
+
+The training ML pipeline uses a [sample diabetes dataset](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_diabetes.html) as training data.
+
+To use your own data:
+
+1. [Create a Dataset](https://docs.microsoft.com/azure/machine-learning/how-to-create-register-datasets) in your Azure ML workspace
+1. Update the `DATASET_NAME` and `DATASTORE_NAME` variables in `.pipelines/[project name]-variables-template.yml`
+
+## Convert your ML experimental code into production ready code
+
+The MLOpsPython template creates an Azure Machine Learning (ML) pipeline that invokes a set of [Azure ML pipeline steps](https://docs.microsoft.com/python/api/azureml-pipeline-steps/azureml.pipeline.steps) (see `ml_service/pipelines/[project name]_build_train_pipeline.py`). If your experiment is currently in a Jupyter notebook, it will need to be refactored into scripts that can be run independantly and dropped into the template which the existing Azure ML pipeline steps utilize.
+
+1. Refactor your experiment code into scripts
+1. [Recommended] Prepare unit tests
+
+Examples of all these scripts are provided in this repository.
+See the [Convert ML experimental code to production code tutorial](https://docs.microsoft.com/azure/machine-learning/tutorial-convert-ml-experiment-to-production) for a step by step guide and additional details.
+
+## Replace training code
+
+The template contains three scripts in the `[project name]/training` folder. Update these scripts for your experiment code.
+
+* `train.py` contains the platform-agnostic logic required to do basic data preparation and train the model. This script can be invoked against a static data file for local development.
+* `train_aml.py` is the entry script for the ML pipeline step. It invokes the functions in `train.py` in an Azure ML context and adds logging. `train_aml.py` loads parameters for training from `[project name]/parameters.json` and passes them to the training function in `train.py`. If your experiment code can be refactored to match the function signatures in `train.py`, this file shouldn't need many changes.
+* `test_train.py` contains tests that guard against functional regressions in `train.py`. Remove this file if you have no tests for your own code.
+
+Add any dependencies required by training to `[project name]/conda_dependencies.yml]`. This file will be used to generate the environment that the pipeline steps will run in.
+
+## Update evaluation code
+
+The MLOpsPython template uses the evaluate_model script to compare the performance of the newly trained model and the current production model based on Mean Squared Error. If the performance of the newly trained model is better than the current production model, then the pipelines continue. Otherwise, the pipelines are canceled.
+
+To keep the evaluation step, replace all instances of `mse` in `[project name]/evaluate/evaluate_model.py` with the metric that you want.
+
+To disable the evaluation step, either:
+
+* set the DevOps pipeline variable `RUN_EVALUATION` to `false`
+* uncomment `RUN_EVALUATION` in `.pipelines/[project name]-variables-template.yml` and set the value to `false`
+
+## Customize the build agent environment
+
+The DevOps pipeline definitions in the MLOpsPython template run several steps in a Docker container that contains the dependencies required to work through the Getting Started guide. If additional dependencies are required to run your unit tests or generate your Azure ML pipeline, there are a few options:
+
+* Add a pipeline step to install dependencies required by unit tests to `.pipelines/code-quality-template.yml`. Recommended if you only have a small number of test dependencies.
+* Create a new Docker image containing your dependencies. See [docs/custom_container.md](custom_container.md). Recommended if you have a larger number of dependencies, or if the overhead of installing additional dependencies on each run is too high.
+* Remove the container references from the pipeline definition files and run the pipelines on self hosted agents with dependencies pre-installed.
+
+## Replace score code
+
+For the model to provide real-time inference capabilities, the score code needs to be replaced. The MLOpsPython template uses the score code to deploy the model to do real-time scoring on ACI, AKS, or Web apps.
+
+If you want to keep scoring:
+
+1. Update or replace `[project name]/scoring/score.py`
+1. Add any dependencies required by scoring to `[project name]/conda_dependencies.yml`
+1. Modify the test cases in the `ml_service/util/smoke_test_scoring_service.py` script to match the schema of the training features in your data

docs/getting_started.md

@@ -263,7 +263,7 @@ To remove the resources created for this project, use the [/environment_setup/ia
 
 ## Next Steps: Integrating your project
 
-* Follow the [bootstrap instructions](../bootstrap/README.md) to create a starting point for your project use case. This guide includes information on bringing your own code to this repository template.
+* The [custom model](custom_model.md) guide includes information on bringing your own code to this repository template.
 * Consider using [Azure Pipelines self-hosted agents](https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops&tabs=browser#install) to speed up your Azure ML pipeline execution. The Docker container image for the Azure ML pipeline is sizable, and having it cached on the agent between runs can trim several minutes from your runs.
 
 ### Additional Variables and Configuration