Skip to content

Commit a88ab43

Browse files
praneet22praneet22
authored
Merge pull request #1 from Microsoft/feature/initial-code
Feature/initial code
2 parents 35c0f6c + c41807d commit a88ab43

38 files changed

+2110
-0
lines changed

‎README.md‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,94 @@
1+
### Author: Praneet Singh Solanki
2+
3+
# DevOps For AI
4+
5+
[DevOps for AI template](https://azuredevopsdemogenerator.azurewebsites.net/?name=azure%20machine%20learning) will help you to understand how to build the Continuous Integration and Continuous Delivery pipeline for a ML/AI project. We will be using the Azure DevOps Project for build and release pipelines along with Azure ML services for ML/AI model management and operationalization.
6+
7+
This template contains code and pipeline definition for a machine learning project demonstrating how to automate the end to end ML/AI project. The build pipelines include DevOps tasks for data sanity test, unit test, model training on different compute targets, model version management, model evaluation/model selection, model deployment as realtime web service, staged deployment to QA/prod, integration testing and functional testing.
8+
9+
## Prerequisite
10+
- Active Azure subscription
11+
- Minimum contributor access to Azure subscription
12+
13+
## Getting Started:
14+
15+
### Import the DevOps for AI solution template from Azure DevOps Demo Generator: [Click here](https://azuredevopsdemogenerator.azurewebsites.net/?name=azure%20machine%20learning)
16+
17+
Skip above step if already done.
18+
19+
Once the template is imported for personal Azure DevOps account using DevOps demo generator, you need to follow below steps to get the pipeline running:
20+
21+
### Update Pipeline Config:
22+
23+
#### Build Pipeline
24+
1. Go to the **Pipelines -> Builds** on the newly created project and click **Edit** on top right
25+
![EditPipeline1](/docs/images/EditPipeline1.png)
26+
2. Click on **Create or Get Workspace** task, select the Azure subscription where you want to deploy and run the solution, and click **Authorize**
27+
![EditPipeline2](/docs/images/EditPipeline2.png)
28+
3. Click all other tasks below it and select the same subscription (no need to authorize again)
29+
4. Once the tasks are updated with subscription, click on **Save & queue** and select **Save**
30+
![EditPipeline3](/docs/images/EditPipeline3.png)
31+
32+
#### Release Pipeline
33+
1. Go to the **Pipelines -> Releases** and click **Edit** on top
34+
![EditPipeline4](/docs/images/EditPipeline4.png)
35+
2. Click on **1 job, 4 tasks** to open the tasks in **QA stage**
36+
![EditPipeline5](/docs/images/EditPipeline5.png)
37+
3. Update the subscription details in two tasks
38+
![EditPipeline6](/docs/images/EditPipeline6.png)
39+
4. Click on **Tasks** on the top to switch to the Prod stage, update the subscription details for the two tasks in prod
40+
![EditPipeline7](/docs/images/EditPipeline7.png)
41+
5. Once you fix all the missing subscription, the **Save** is no longer grayed, click on save to save the changes in release pepeline
42+
![EditPipeline8](/docs/images/EditPipeline8.png)
43+
44+
### Update Repo config:
45+
1. Go to the **Repos** on the newly created Azure DevOps project
46+
2. Open the config file [/aml_config/config.json](/aml_config/config.json) and edit it
47+
3. Put your Azure subscription ID in place of <>
48+
4. Change resource group and AML workspace name if you want
49+
5. Put the location where you want to deploy your Azure ML service workspace
50+
6. Save the changes and commit these changes to master branch
51+
7. The commit will trigger the build pipeline to run deploying AML end to end solution
52+
8. Go to **Pipelines -> Builds** to see the pipeline run
53+
54+
## Steps Performed in the Build Pipeline:
55+
56+
1. Prepare the python environment
57+
2. Get or Create the workspace
58+
3. Submit Training job on the remote DSVM / Local Python Env
59+
4. Register model to workspace
60+
5. Create Docker Image for Scoring Webservice
61+
6. Copy and Publish the Artifacts to Release Pipeline
62+
63+
## Steps Performed in the Release Pipeline
64+
In Release pipeline we deploy the image created from the build pipeline to Azure Container Instance and Azure Kubernetes Services
65+
66+
### Deploy on ACI - QA Stage
67+
1. Prepare the python environment
68+
2. Create ACI and Deploy webservice image created in Build Pipeline
69+
3. Test the scoring image
70+
71+
### Deploy on AKS - PreProd/Prod Stage
72+
1. Prepare the python environment
73+
2. Deploy on AKS
74+
- Create AKS and create a new webservice on AKS with the scoring docker image
75+
76+
OR
77+
78+
- Get the existing AKS and update the webservice with new image created in Build Pipeline
79+
3. Test the scoring image
80+
81+
### Repo Details
82+
83+
You can find the details of the code ans scripts in the repository [here](/docs/code_description.md)
84+
85+
### References
86+
87+
- [Azure Machine Learning(Azure ML) Service Workspace](https://docs.microsoft.com/en-us/azure/machine-learning/service/overview-what-is-azure-ml)
88+
89+
- [Azure ML Samples](https://docs.microsoft.com/en-us/azure/machine-learning/service/samples-notebooks)
90+
- [Azure ML Python SDK Quickstart](https://docs.microsoft.com/en-us/azure/machine-learning/service/quickstart-create-workspace-with-python)
91+
- [Azure DevOps](https://docs.microsoft.com/en-us/azure/devops/?view=vsts)
192

293
# Contributing
394

‎aml_config/conda_dependencies.yml‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# Conda environment specification. The dependencies defined in this file will
2+
3+
# be automatically provisioned for managed runs. These include runs against
4+
5+
# the localdocker, remotedocker, and cluster compute targets.
6+
7+
8+
# Note that this file is NOT used to automatically manage dependencies for the
9+
10+
# local compute target. To provision these dependencies locally, run:
11+
12+
# conda env update --file conda_dependencies.yml
13+
14+
15+
# Details about the Conda environment file format:
16+
17+
# https://conda.io/docs/using/envs.html#create-environment-file-by-hand
18+
19+
20+
# For managing Spark packages and configuration, see spark_dependencies.yml.
21+
22+
23+
# Version of this configuration file's structure and semantics in AzureML.
24+
25+
# This directive is stored in a comment to preserve the Conda file structure.
26+
27+
# [AzureMlVersion] = 2
28+
29+
30+
name: project_environment
31+
dependencies:
32+
# The python interpreter version.
33+
34+
# Currently Azure ML Workbench only supports 3.5.2 and later.
35+
36+
- python=3.6.2
37+
# Required by azureml-defaults, installed separately through Conda to
38+
39+
# get a prebuilt version and not require build tools for the install.
40+
41+
- psutil=5.3
42+
43+
- pip:
44+
# Required packages for AzureML execution, history, and data preparation.
45+
- azureml-sdk[notebooks]
46+
- pynacl==1.2.1
47+
- scipy==1.0.0
48+
- scikit-learn==0.19.1
49+
- pandas==0.23.1
50+
- numpy==1.14.5

‎aml_config/config.json‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"subscription_id": "<>",
3+
"resource_group": "DevOps_AzureML_Demo",
4+
"workspace_name": "AzureML_Demo_ws",
5+
"location": "southcentralus"
6+
}

‎aml_config/security_config.json‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"sp_user" : "<>",
3+
"sp_password" : "<>",
4+
"sp_tenant_id" : "<>",
5+
"remote_vm_name" : "<>",
6+
"remote_vm_username" : "<>",
7+
"remote_vm_password" : "<>",
8+
"remote_vm_ip" : "<>"
9+
}

‎aml_service/00-WorkSpace.py‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
"""
2+
Copyright (C) Microsoft Corporation. All rights reserved.​
3+
​
4+
Microsoft Corporation (“Microsoft”) grants you a nonexclusive, perpetual,
5+
royalty-free right to use, copy, and modify the software code provided by us
6+
("Software Code"). You may not sublicense the Software Code or any use of it
7+
(except to your affiliates and to vendors to perform work on your behalf)
8+
through distribution, network access, service agreement, lease, rental, or
9+
otherwise. This license does not purport to express any claim of ownership over
10+
data you may have shared with Microsoft in the creation of the Software Code.
11+
Unless applicable law gives you more rights, Microsoft reserves all other
12+
rights not expressly granted herein, whether by implication, estoppel or
13+
otherwise. ​
14+
​
15+
THE SOFTWARE CODE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS
16+
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
18+
MICROSOFT OR ITS LICENSORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
19+
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
20+
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
21+
BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
22+
IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
23+
ARISING IN ANY WAY OUT OF THE USE OF THE SOFTWARE CODE, EVEN IF ADVISED OF THE
24+
POSSIBILITY OF SUCH DAMAGE.
25+
"""
26+
from azureml.core import Workspace
27+
import os, json
28+
import azureml.core
29+
30+
print("SDK Version:", azureml.core.VERSION)
31+
# print('current dir is ' +os.curdir)
32+
with open("aml_config/config.json") as f:
33+
config = json.load(f)
34+
35+
workspace_name = config["workspace_name"]
36+
resource_group = config["resource_group"]
37+
subscription_id = config["subscription_id"]
38+
location = config["location"]
39+
# location = 'southcentralus'
40+
try:
41+
ws = Workspace.get(
42+
name=workspace_name,
43+
subscription_id=subscription_id,
44+
resource_group=resource_group,
45+
)
46+
47+
except:
48+
# this call might take a minute or two.
49+
print("Creating new workspace")
50+
ws = Workspace.create(
51+
name=workspace_name,
52+
subscription_id=subscription_id,
53+
resource_group=resource_group,
54+
# create_resource_group=True,
55+
location=location,
56+
)
57+
58+
# print Workspace details
59+
print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep="\n")

‎aml_service/01-Experiment.py‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
"""
2+
Copyright (C) Microsoft Corporation. All rights reserved.​
3+
​
4+
Microsoft Corporation (“Microsoft”) grants you a nonexclusive, perpetual,
5+
royalty-free right to use, copy, and modify the software code provided by us
6+
("Software Code"). You may not sublicense the Software Code or any use of it
7+
(except to your affiliates and to vendors to perform work on your behalf)
8+
through distribution, network access, service agreement, lease, rental, or
9+
otherwise. This license does not purport to express any claim of ownership over
10+
data you may have shared with Microsoft in the creation of the Software Code.
11+
Unless applicable law gives you more rights, Microsoft reserves all other
12+
rights not expressly granted herein, whether by implication, estoppel or
13+
otherwise. ​
14+
​
15+
THE SOFTWARE CODE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS
16+
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
18+
MICROSOFT OR ITS LICENSORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
19+
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
20+
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
21+
BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
22+
IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
23+
ARISING IN ANY WAY OUT OF THE USE OF THE SOFTWARE CODE, EVEN IF ADVISED OF THE
24+
POSSIBILITY OF SUCH DAMAGE.
25+
"""
26+
import os
27+
from azureml.core import Experiment
28+
from azureml.core import Workspace
29+
30+
31+
def getExperiment():
32+
ws = Workspace.from_config()
33+
script_folder = "."
34+
experiment_name = "devops-ai-demo"
35+
exp = Experiment(workspace=ws, name=experiment_name)
36+
print(exp.name, exp.workspace.name, sep="\n")
37+
return exp
38+
39+
40+
if __name__ == "__main__":
41+
exp = getExperiment()

‎aml_service/02-AttachTrainingVM.py‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
"""
2+
Copyright (C) Microsoft Corporation. All rights reserved.​
3+
​
4+
Microsoft Corporation (“Microsoft”) grants you a nonexclusive, perpetual,
5+
royalty-free right to use, copy, and modify the software code provided by us
6+
("Software Code"). You may not sublicense the Software Code or any use of it
7+
(except to your affiliates and to vendors to perform work on your behalf)
8+
through distribution, network access, service agreement, lease, rental, or
9+
otherwise. This license does not purport to express any claim of ownership over
10+
data you may have shared with Microsoft in the creation of the Software Code.
11+
Unless applicable law gives you more rights, Microsoft reserves all other
12+
rights not expressly granted herein, whether by implication, estoppel or
13+
otherwise. ​
14+
​
15+
THE SOFTWARE CODE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS
16+
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
18+
MICROSOFT OR ITS LICENSORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
19+
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
20+
PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
21+
BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
22+
IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
23+
ARISING IN ANY WAY OUT OF THE USE OF THE SOFTWARE CODE, EVEN IF ADVISED OF THE
24+
POSSIBILITY OF SUCH DAMAGE.
25+
"""
26+
27+
from azureml.core import Workspace
28+
from azureml.core import Run
29+
from azureml.core import Experiment
30+
from azureml.core.conda_dependencies import CondaDependencies
31+
from azureml.core.runconfig import RunConfiguration
32+
import os, json
33+
from azureml.core.compute import RemoteCompute
34+
from azureml.core.compute import DsvmCompute
35+
from azureml.core.compute_target import ComputeTargetException
36+
37+
38+
# Get workspace
39+
ws = Workspace.from_config()
40+
41+
# Read the New VM Config
42+
with open("aml_config/security_config.json") as f:
43+
config = json.load(f)
44+
45+
remote_vm_name = config["remote_vm_name"]
46+
remote_vm_username = config["remote_vm_username"]
47+
remote_vm_password = config["remote_vm_password"]
48+
remote_vm_ip = config["remote_vm_ip"]
49+
50+
try:
51+
dsvm_compute = RemoteCompute.attach(
52+
ws,
53+
name=remote_vm_name,
54+
username=remote_vm_username,
55+
address=remote_vm_ip,
56+
ssh_port=22,
57+
password=remote_vm_password,
58+
)
59+
dsvm_compute.wait_for_completion(show_output=True)
60+
61+
except Exception as e:
62+
print("Caught = {}".format(e.message))
63+
print("Compute config already attached.")
64+
65+
66+
## Create VM if not available
67+
# compute_target_name = remote_vm_name
68+
69+
# try:
70+
# dsvm_compute = DsvmCompute(workspace=ws, name=compute_target_name)
71+
# print('found existing:', dsvm_compute.name)
72+
# except ComputeTargetException:
73+
# print('creating new.')
74+
# dsvm_config = DsvmCompute.provisioning_configuration(vm_size="Standard_D2_v2")
75+
# dsvm_compute = DsvmCompute.create(ws, name=compute_target_name, provisioning_configuration=dsvm_config)
76+
# dsvm_compute.wait_for_completion(show_output=True)

0 commit comments

Comments
 (0)