You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-differential-privacy.md
+12-10Lines changed: 12 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,27 +1,27 @@
1
1
---
2
2
title: Differential privacy in machine learning (preview)
3
3
titleSuffix: Azure Machine Learning
4
-
description: Learn what differential privacy is and how you can implement differentially private systems that preserve data privacy.
4
+
description: Learn what differential privacy is and how differentially private systems preserve data privacy.
5
5
author: luisquintanilla
6
6
ms.author: luquinta
7
7
ms.date: 10/21/2021
8
8
services: machine-learning
9
9
ms.service: machine-learning
10
10
ms.subservice: enterprise-readiness
11
11
ms.topic: conceptual
12
-
ms.custom: responsible-ml
12
+
ms.custom: responsible-ml, mktng-kw-nov2021
13
13
#Customer intent: As a data scientist, I want to know what differential privacy is and how I can implement a differentially private systems.
14
14
---
15
15
16
-
# What is differential privacy in machine learning (preview)
16
+
# What is differential privacy in machine learning (preview)?
17
17
18
18
Learn about differential privacy in machine learning and how it works.
19
19
20
-
As the amount of data that an organization collects and uses for analyses increases, so do concerns of privacy and security. Analyses require data. Typically, the more data used to train models, the more accurate they are. When personal information is used for these analyses, it's especially important that the data remains private throughout its use.
20
+
As the amount of data that an organization collects and uses for analyses increases, so do concerns of privacy and security. Analyses require data. Typically, the more data used to train machine learning models, the more accurate they are. When personal information is used for these analyses, it's especially important that the data remains private throughout its use.
21
21
22
22
## How differential privacy works
23
23
24
-
Differential privacy is a set of systems and practices that help keep the data of individuals safe and private.
24
+
Differential privacy is a set of systems and practices that help keep the data of individuals safe and private. In machine learning solutions, differential privacy may be required for regulatory compliance.
@@ -32,9 +32,9 @@ In differentially private systems, data is shared through requests called **quer
32
32
33
33
## Differential privacy metrics
34
34
35
-
Differential privacy tries to protect against the possibility that a user can produce an indefinite number of reports to eventually reveal sensitive data. A value known as **epsilon** measures how noisy or private a report is. Epsilon has an inverse relationship to noise or privacy. The lower the epsilon, the more noisy (and private) the data is.
35
+
Differential privacy tries to protect against the possibility that a user can produce an indefinite number of reports to eventually reveal sensitive data. A value known as **epsilon** measures how noisy, or private, a report is. Epsilon has an inverse relationship to noise or privacy. The lower the epsilon, the more noisy (and private) the data is.
36
36
37
-
Epsilon values are non-negative. Values below 1 provide full plausible deniability. Anything above 1 comes with a higher risk of exposure of the actual data. As you implement differentially private systems, you want to produce reports with epsilon values between 0 and 1.
37
+
Epsilon values are non-negative. Values below 1 provide full plausible deniability. Anything above 1 comes with a higher risk of exposure of the actual data. As you implement machine learning solutions with differential privacy, you want to data with epsilon values between 0 and 1.
38
38
39
39
Another value directly correlated to epsilon is **delta**. Delta is a measure of the probability that a report is not fully private. The higher the delta, the higher the epsilon. Because these values are correlated, epsilon is used more often.
40
40
@@ -48,7 +48,7 @@ Although the preservation of privacy should be the goal, there is a tradeoff whe
48
48
49
49
## Open-source differential privacy libraries
50
50
51
-
SmartNoise is an open-source project that contains different components for building global differentially private systems. SmartNoise is made up of the following top-level components:
51
+
SmartNoise is an open-source project that contains components for building machine learning solutions with differential privacy. SmartNoise is made up of the following top-level components:
52
52
53
53
- SmartNoise Core library
54
54
- SmartNoise SDK library
@@ -76,6 +76,8 @@ The system library provides the following tools and services for working with ta
76
76
77
77
## Next steps
78
78
79
-
[How to build a differentially private system](how-to-differential-privacy.md) in Azure Machine Learning.
79
+
Learn more about differentialprivacy in machine learning:
80
80
81
-
To learn more about the components of SmartNoise, check out the GitHub repositories for [SmartNoise Core](https://github.com/opendifferentialprivacy/smartnoise-core), [SmartNoise SDK](https://github.com/opendifferentialprivacy/smartnoise-sdk), and [SmartNoise samples](https://github.com/opendifferentialprivacy/smartnoise-samples).
81
+
-[How to build a differentially private system](how-to-differential-privacy.md) in Azure Machine Learning.
82
+
83
+
- To learn more about the components of SmartNoise, check out the GitHub repositories for [SmartNoise Core](https://github.com/opendifferentialprivacy/smartnoise-core), [SmartNoise SDK](https://github.com/opendifferentialprivacy/smartnoise-sdk), and [SmartNoise samples](https://github.com/opendifferentialprivacy/smartnoise-samples).
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-model-management-and-deployment.md
+10-7Lines changed: 10 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,20 +1,20 @@
1
1
---
2
2
title: 'MLOps: ML model management'
3
3
titleSuffix: Azure Machine Learning
4
-
description: 'Learn about model management with Azure Machine Learning (MLOps). Deploy, manage, track lineage and monitor your models to continuously improve them. '
4
+
description: 'Learn about model management (MLOps) with Azure Machine Learning . Deploy, manage, track lineage and monitor your models to continuously improve them. '
5
5
services: machine-learning
6
6
ms.service: machine-learning
7
7
ms.subservice: mlops
8
8
ms.topic: conceptual
9
9
author: jpe316
10
10
ms.author: jordane
11
-
ms.date: 10/21/2021
12
-
ms.custom: seodec18
11
+
ms.custom: seodec18, mktng-kw-nov2021
12
+
ms.date: 11/04/2021
13
13
---
14
14
15
15
# MLOps: Model management, deployment, lineage, and monitoring with Azure Machine Learning
16
16
17
-
In this article, learn about how to use Azure Machine Learning to manage the lifecycle of your models. Azure Machine Learning uses a Machine Learning Operations (MLOps) approach. MLOps improves the quality and consistency of your machine learning solutions.
17
+
In this article, learn about how do Machine Learning Operations (MLOps) in Azure Machine Learning to manage the lifecycle of your models. MLOps improves the quality and consistency of your machine learning solutions.
18
18
19
19
## What is MLOps?
20
20
@@ -24,6 +24,8 @@ Machine Learning Operations (MLOps) is based on [DevOps](https://azure.microsoft
24
24
* Faster deployment of models into production
25
25
* Quality assurance and end-to-end lineage tracking
26
26
27
+
## MLOps in Azure Machine Learning
28
+
27
29
Azure Machine Learning provides the following MLOps capabilities:
28
30
29
31
-**Create reproducible ML pipelines**. Machine Learning pipelines allow you to define repeatable and reusable steps for your data preparation, training, and scoring processes.
@@ -35,6 +37,7 @@ Azure Machine Learning provides the following MLOps capabilities:
35
37
-**Automate the end-to-end ML lifecycle with Azure Machine Learning and Azure Pipelines**. Using pipelines allows you to frequently update models, test new models, and continuously roll out new ML models alongside your other applications and services.
36
38
37
39
For more information on MLOps, see [Machine Learning DevOps (MLOps)](/azure/cloud-adoption-framework/ready/azure-best-practices/ai-machine-learning-mlops).
40
+
38
41
## Create reproducible ML pipelines
39
42
40
43
Use ML pipelines from Azure Machine Learning to stitch together all of the steps involved in your model training process.
@@ -136,7 +139,7 @@ For more information, see [Controlled rollout of ML models](how-to-deploy-azure-
136
139
137
140
Microsoft Power BI supports using machine learning models for data analytics. For more information, see [Azure Machine Learning integration in Power BI (preview)](/power-bi/service-machine-learning-integration).
138
141
139
-
## Capture the governance data required for capturing the end-to-end ML lifecycle
142
+
## Capture the governance data required for MLOps
140
143
141
144
Azure ML gives you the capability to track the end-to-end audit trail of all of your ML assets by using metadata.
142
145
@@ -190,8 +193,8 @@ The [Azure Machine Learning extension](https://marketplace.visualstudio.com/item
190
193
For more information on using Azure Pipelines with Azure Machine Learning, see the following links:
191
194
192
195
*[Continuous integration and deployment of ML models with Azure Pipelines](/azure/devops/pipelines/targets/azure-machine-learning)
You can also use Azure Data Factory to create a data ingestion pipeline that prepares data for use with training. For more information, see [Data ingestion pipeline](how-to-cicd-data-ingestion.md).
Copy file name to clipboardExpand all lines: articles/machine-learning/concept-open-source.md
+17-14Lines changed: 17 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,28 +8,21 @@ ms.subservice: mlops
8
8
ms.topic: conceptual
9
9
author: luisquintanilla
10
10
ms.author: luquinta
11
-
ms.date: 10/21/2021
11
+
ms.custom: mktng-kw-nov2021
12
+
ms.date: 11/04/2021
12
13
---
13
14
14
-
# Open-source integration with Azure Machine Learning projects
15
+
# Use open-source machine learning libraries and platforms with Azure Machine Learning
15
16
16
-
You can train, deploy, and manage the end-to-end machine learning process in Azure Machine Learning by using open-source Python machine learning libraries and platforms. Use development tools, like Jupyter Notebooks and Visual Studio Code, to leverage your existing models and scripts in Azure Machine Learning.
17
-
18
-
In this article, learn more about these open-source libraries and platforms.
17
+
In this article, learn about open-source Python machine learning libraries and platforms you can use with Azure Machine Learning. Train, deploy, and manage the end-to-end machine learning process using open source projects you prefer. Use development tools, like Jupyter Notebooks and Visual Studio Code, to leverage your existing models and scripts in Azure Machine Learning.
19
18
20
19
## Train open-source machine learning models
21
20
22
-
The machine learning training process involves the application of algorithms to your data in order to achieve a task or solve a problem. Depending on the problem, you may choose different algorithms that best fit the task and your data. For more information on the different branches of machine learning, see the [deep learning vs machine learning article](./concept-deep-learning-vs-machine-learning.md) and the [machine learning algorithm cheat sheet](algorithm-cheat-sheet.md).
23
-
24
-
### Preserve data privacy using differential privacy
25
-
26
-
To train a machine learning model, you need data. Sometimes that data is sensitive, and it's important to make sure that the data is secure and private. Differential privacy is a technique of preserving the confidentiality of information in a dataset. To learn more, see the article on [preserving data privacy](concept-differential-privacy.md).
27
-
28
-
Open-source differential privacy toolkits like [SmartNoise](https://github.com/opendifferentialprivacy/smartnoise-core-python) help you [preserve the privacy of data](how-to-differential-privacy.md) in Azure Machine Learning solutions.
21
+
The machine learning training process involves the application of algorithms to your data in order to achieve a task or solve a problem. Depending on the problem, you may choose different algorithms that best fit the task and your data. For more information on what you can solve with machine learning, see the [deep learning vs machine learning article](./concept-deep-learning-vs-machine-learning.md) and the [machine learning algorithm cheat sheet](algorithm-cheat-sheet.md).
29
22
30
23
### Classical machine learning: scikit-learn
31
24
32
-
For training tasks involving classical machine learning algorithms tasks such classification, clustering, and regression you might use something like Scikit-learn. To learn how to train a flower classification model, see the [how to train with Scikit-learn article](how-to-train-scikit-learn.md).
25
+
For training tasks involving classical machine learning algorithms tasks such classification, clustering, and regression you might use something like scikit-learn. To learn how to train a flower classification model, see the [how to train with scikit-learn article](how-to-train-scikit-learn.md).
33
26
34
27
### Neural networks: PyTorch, TensorFlow, Keras
35
28
@@ -41,6 +34,8 @@ Open-source deep learning frameworks and how-to guides include:
41
34
*[TensorFlow](https://github.com/tensorflow/tensorflow): [Recognize handwritten digits using TensorFlow](how-to-train-tensorflow.md)
42
35
*[Keras](https://github.com/keras-team/keras): [Build a neural network to analyze images using Keras](how-to-train-keras.md)
43
36
37
+
### Transfer learning
38
+
44
39
Training a deep learning model from scratch often requires large amounts of time, data, and compute resources. You can shortcut the training process by using transfer learning. Transfer learning is a technique that applies knowledge gained from solving one problem to a different but related problem. This means you can take an existing model repurpose it. See the [deep learning vs machine learning article](concept-deep-learning-vs-machine-learning.md#what-is-transfer-learning) to learn more about transfer learning.
45
40
46
41
### Reinforcement learning: Ray RLLib
@@ -55,6 +50,14 @@ To learn how to use Ray RLLib with Azure Machine Learning, see the [how to train
55
50
56
51
Training a single or multiple models requires the visualization and inspection of desired metrics to make sure the model performs as expected. You can [use TensorBoard in Azure Machine Learning to track and visualize experiment metrics](./how-to-monitor-tensorboard.md)
57
52
53
+
## Responsible ML: Privacy and fairness
54
+
55
+
### Preserve data privacy with differential privacy
56
+
57
+
To train a machine learning model, you need data. Sometimes that data is sensitive, and it's important to make sure that the data is secure and private. Differential privacy is a technique of preserving the confidentiality of information in a dataset. To learn more, see the article on [preserving data privacy](concept-differential-privacy.md).
58
+
59
+
Open-source differential privacy toolkits like [SmartNoise](https://github.com/opendifferentialprivacy/smartnoise-core-python) help you [preserve the privacy of data](how-to-differential-privacy.md) in Azure Machine Learning solutions.
60
+
58
61
### Frameworks for interpretable and fair models
59
62
60
63
Machine learning systems are used in different areas of society such as banking, education, and healthcare. As such, it's important for these systems to be accountable for the predictions and recommendations they make to prevent unintended consequences.
@@ -90,7 +93,7 @@ Container technologies such as Docker are one way to deploy models as web servic
90
93
91
94
Securing deployments is an important part of the deployment process. To [deploy encrypted inferencing services](how-to-homomorphic-encryption-seal.md), use the `encrypted-inference` open-source Python library. The `encrypted inferencing` package provides bindings based on [Microsoft SEAL](https://github.com/Microsoft/SEAL), a homomorphic encryption library.
92
95
93
-
## Machine Learning Operations (MLOps)
96
+
## Machine learning operations (MLOps)
94
97
95
98
Machine Learning Operations (MLOps), commonly thought of as DevOps for machine learning allows you to build more transparent, resilient, and reproducible machine learning workflows. See the [what is MLOps article](./concept-model-management-and-deployment.md) to learn more about MLOps.
@@ -19,15 +19,17 @@ Learn how to deploy your machine learning or deep learning model as a web servic
19
19
> [!TIP]
20
20
> Managed online endpoints (preview) provide a way to deploy your trained model without your having to create and manage the underlying infrastructure. For more information, see [Deploy and score a machine learning model with a managed online endpoint (preview)](how-to-deploy-managed-online-endpoints.md).
21
21
22
+
## Workflow for deploying a model
23
+
22
24
The workflow is similar no matter where you deploy your model:
23
25
24
-
1. Register the model
25
-
1. Prepare an entry script
26
-
1. Prepare an inference configuration
27
-
1. Deploy the model locally to ensure everything works
28
-
1. Choose a compute target
29
-
1. Re-deploy the model to the cloud
30
-
1. Test the resulting web service
26
+
1. Register the model.
27
+
1. Prepare an entry script.
28
+
1. Prepare an inference configuration.
29
+
1. Deploy the model locally to ensure everything works.
30
+
1. Choose a compute target.
31
+
1. Re-deploy the model to the cloud.
32
+
1. Test the resulting web service.
31
33
32
34
For more information on the concepts involved in the machine learning deployment workflow, see [Manage, deploy, and monitor models with Azure Machine Learning](concept-model-management-and-deployment.md).
33
35
@@ -78,7 +80,7 @@ For more information on using the SDK to connect to a workspace, see the [Azure
78
80
79
81
---
80
82
81
-
## <aid="registermodel"></a> Register your model
83
+
## <aid="registermodel"></a> Register the model
82
84
83
85
A typical situation for a deployed machine learning service is that you need the following components:
0 commit comments