Skip to content

Commit 8b2d3e2

Browse files
Merge pull request #219683 from fbsolo-ms1/updates-for-YP
Update files as requested by Yogi P . . .
2 parents 23e02ca + cca9866 commit 8b2d3e2

11 files changed

+48
-45
lines changed

articles/machine-learning/.openpublishing.redirection.machine-learning.json

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
{
2-
"redirections": [
2+
"redirections": [
3+
{
4+
"source_path_from_root": "/articles/machine-learning/how-to-link-synapse-ml-workspaces.md",
5+
"redirect_url": "/azure/machine-learning/v1/how-to-link-synapse-ml-workspaces",
6+
"redirect_document_id": true
7+
},
38
{
49
"source_path_from_root": "/articles/machine-learning/how-to-use-environments.md",
510
"redirect_url": "/azure/machine-learning/how-to-manage-environments-v2",

articles/machine-learning/concept-compute-target.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -159,7 +159,7 @@ Azure Machine Learning supports the following unmanaged compute types:
159159
* Azure HDInsight
160160
* Azure Databricks
161161
* Azure Data Lake Analytics
162-
* [Azure Synapse Spark pool](how-to-link-synapse-ml-workspaces.md) (preview)
162+
* [Azure Synapse Spark pool](v1/how-to-link-synapse-ml-workspaces.md) (preview)
163163

164164
> [!TIP]
165165
> Currently this requires the Azure Machine Learning SDK v1.

articles/machine-learning/how-to-private-endpoint-integration-synapse.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ ms.date: 11/16/2022
1818
In this article, learn how to securely integrate with Azure Machine Learning from Azure Synapse. This integration enables you to use Azure Machine Learning from notebooks in your Azure Synapse workspace. Communication between the two workspaces is secured using an Azure Virtual Network.
1919

2020
> [!TIP]
21-
> You can also perform integration in the opposite direction, using Azure Synapse spark pool from Azure Machine Learning. For more information, see [Link Azure Synapse and Azure Machine Learning](how-to-link-synapse-ml-workspaces.md).
21+
> You can also perform integration in the opposite direction, using Azure Synapse spark pool from Azure Machine Learning. For more information, see [Link Azure Synapse and Azure Machine Learning](v1/how-to-link-synapse-ml-workspaces.md).
2222
2323
## Prerequisites
2424

@@ -141,4 +141,4 @@ To verify that the integration between Azure Synapse and Azure Machine Learning
141141
## Next steps
142142

143143
* [Quickstart: Create a new Azure Machine Learning linked service in Synapse](../synapse-analytics/machine-learning/quickstart-integrate-azure-machine-learning.md).
144-
* [Link Azure Synapse Analytics and Azure Machine Learning workspaces](how-to-link-synapse-ml-workspaces.md).
144+
* [Link Azure Synapse Analytics and Azure Machine Learning workspaces](v1/how-to-link-synapse-ml-workspaces.md).

articles/machine-learning/v1/how-to-attach-compute-targets.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,7 @@ Azure Machine Learning also supports attaching an Azure Virtual Machine. The VM
130130
131131
## <a id="synapse"></a>Apache Spark pools
132132

133-
The Azure Synapse Analytics integration with Azure Machine Learning (preview) allows you to attach an Apache Spark pool backed by Azure Synapse for interactive data exploration and preparation. With this integration, you can have a dedicated compute for data wrangling at scale. For more information, see [How to attach Apache Spark pools powered by Azure Synapse Analytics](../how-to-link-synapse-ml-workspaces.md#attach-synapse-spark-pool-as-a-compute).
133+
The Azure Synapse Analytics integration with Azure Machine Learning (preview) allows you to attach an Apache Spark pool backed by Azure Synapse for interactive data exploration and preparation. With this integration, you can have a dedicated compute for data wrangling at scale. For more information, see [How to attach Apache Spark pools powered by Azure Synapse Analytics](how-to-link-synapse-ml-workspaces.md#attach-synapse-spark-pool-as-a-compute).
134134

135135
## Azure HDInsight
136136

articles/machine-learning/v1/how-to-data-prep-synapse-spark-pool.md

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Data wrangling with Apache Spark pools (preview)
2+
title: Data wrangling with Apache Spark pools (deprecated)
33
titleSuffix: Azure Machine Learning
44
description: Learn how to attach and launch Apache Spark pools for data wrangling with Azure Synapse Analytics and Azure Machine Learning.
55
services: machine-learning
@@ -9,25 +9,24 @@ ms.topic: how-to
99
author: ynpandey
1010
ms.author: franksolomon
1111
ms.reviewer: franksolomon
12-
ms.date: 08/17/2022
12+
ms.date: 11/28/2022
1313
ms.custom: devx-track-python, data4ml, synapse-azureml, contperf-fy21q4, sdkv1, event-tier1-build-2022
1414
#Customer intent: As a data scientist, I want to prepare my data at scale, and to train my machine learning models from a single notebook using Azure Machine Learning.
1515
---
1616

17-
# Data wrangling with Apache Spark pools (preview)
17+
# Data wrangling with Apache Spark pools (deprecated)
1818

1919
[!INCLUDE [sdk v1](../../../includes/machine-learning-sdk-v1.md)]
2020

21+
> [!WARNING]
22+
> The Azure Synapse Analytics integration with Azure Machine Learning available in Python SDK v1 is deprecated. Users can continue using Synapse workspace registered with Azure Machine Learning as a linked service. However, a new Synapse workspace can no longer be registered with Azure Machine Learning as a linked service. We recommend using Managed (Automatic) Synapse compute and attached Synapse Spark pools available in CLI v2 and Python SDK v2. Please see [https://aka.ms/aml-spark](https://aka.ms/aml-spark) for more details.
2123
2224
In this article, you learn how to perform data wrangling tasks interactively within a dedicated Synapse session, powered by [Azure Synapse Analytics](../../synapse-analytics/overview-what-is.md), in a Jupyter notebook using the [Azure Machine Learning Python SDK](/python/api/overview/azure/ml/).
2325

2426
If you prefer to use Azure Machine Learning pipelines, see [How to use Apache Spark (powered by Azure Synapse Analytics) in your machine learning pipeline (preview)](how-to-use-synapsesparkstep.md).
2527

2628
For guidance on how to use Azure Synapse Analytics with a Synapse workspace, see the [Azure Synapse Analytics get started series](../../synapse-analytics/get-started.md).
2729

28-
>[!IMPORTANT]
29-
> The Azure Machine Learning and Azure Synapse Analytics integration is in preview. The capabilities presented in this article employ the `azureml-synapse` package which contains [experimental](/python/api/overview/azure/ml/#stable-vs-experimental) preview features that may change at any time.
30-
3130
## Azure Machine Learning and Azure Synapse Analytics integration
3231

3332
The Azure Synapse Analytics integration with Azure Machine Learning (preview) allows you to attach an Apache Spark pool backed by Azure Synapse for interactive data exploration and preparation. With this integration, you can have a dedicated compute for data wrangling at scale, all within the same Python notebook you use for training your machine learning models.
@@ -50,9 +49,9 @@ The Azure Synapse Analytics integration with Azure Machine Learning (preview) al
5049
pip install azureml-synapse
5150
```
5251

53-
* Link your Azure Machine Learning workspace and Azure Synapse Analytics workspace with the [Azure Machine Learning Python SDK](../how-to-link-synapse-ml-workspaces.md#link-sdk) or via the [Azure Machine Learning studio](../how-to-link-synapse-ml-workspaces.md#link-studio)
52+
* Link your Azure Machine Learning workspace and Azure Synapse Analytics workspace with the [Azure Machine Learning Python SDK](how-to-link-synapse-ml-workspaces.md#link-sdk) or via the [Azure Machine Learning studio](how-to-link-synapse-ml-workspaces.md#link-studio)
5453

55-
* [Attach a Synapse Spark pool](../how-to-link-synapse-ml-workspaces.md#attach-synapse-spark-pool-as-a-compute) as a compute target.
54+
* [Attach a Synapse Spark pool](how-to-link-synapse-ml-workspaces.md#attach-synapse-spark-pool-as-a-compute) as a compute target.
5655

5756
## Launch Synapse Spark pool for data wrangling tasks
5857

@@ -251,7 +250,7 @@ input1 = train_ds.as_mount()
251250

252251
## Use a `ScriptRunConfig` to submit an experiment run to a Synapse Spark pool
253252

254-
If you're ready to automate and productionize your data wrangling tasks, you can submit an experiment run to [an attached Synapse Spark pool](../how-to-link-synapse-ml-workspaces.md#attach-a-pool-with-the-python-sdk) with the [ScriptRunConfig](/python/api/azureml-core/azureml.core.scriptrunconfig) object.
253+
If you're ready to automate and productionize your data wrangling tasks, you can submit an experiment run to [an attached Synapse Spark pool](how-to-link-synapse-ml-workspaces.md#attach-a-pool-with-the-python-sdk) with the [ScriptRunConfig](/python/api/azureml-core/azureml.core.scriptrunconfig) object.
255254

256255
Similarly, if you have an Azure Machine Learning pipeline, you can use the [SynapseSparkStep to specify your Synapse Spark pool as the compute target](how-to-use-synapsesparkstep.md) for the data preparation step in your pipeline.
257256

articles/machine-learning/how-to-link-synapse-ml-workspaces.md renamed to articles/machine-learning/v1/how-to-link-synapse-ml-workspaces.md

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Create a linked service with Synapse and Azure Machine Learning workspaces (preview)
2+
title: Create a linked service with Synapse and Azure Machine Learning workspaces (deprecated)
33
titleSuffix: Azure Machine Learning
44
description: Learn how to link Azure Synapse and Azure Machine Learning workspaces, and attach Apache Spark pools for a unified data wrangling experience.
55
services: machine-learning
@@ -8,34 +8,34 @@ ms.subservice: mldata
88
ms.topic: how-to
99
ms.author: jhirono
1010
author: jhirono
11-
ms.reviewer: larryfr
11+
ms.reviewer: franksolomon
1212
ms.date: 11/04/2022
1313
ms.custom: devx-track-python, data4ml, synapse-azureml, contperf-fy21q4, sdkv1, event-tier1-build-2022
1414
#Customer intent: As a workspace administrator, I want to link Azure Synapse workspaces and Azure Machine Learning workspaces and attach Apache Spark pools for a unified data wrangling experience.
1515
---
1616

17-
# Link Azure Synapse Analytics and Azure Machine Learning workspaces and attach Apache Spark pools(preview)
17+
# Link Azure Synapse Analytics and Azure Machine Learning workspaces and attach Apache Spark pools(deprecated)
1818

19-
[!INCLUDE [sdk v1](../../includes/machine-learning-sdk-v1.md)]
19+
[!INCLUDE [sdk v1](../../../includes/machine-learning-sdk-v1.md)]
2020

21-
In this article, you learn how to create a linked service that links your [Azure Synapse Analytics](../synapse-analytics/overview-what-is.md) workspace and [Azure Machine Learning workspace](concept-workspace.md).
21+
> [!WARNING]
22+
> The Azure Synapse Analytics integration with Azure Machine Learning available in Python SDK v1 is deprecated. Users can continue using Synapse workspace registered with Azure Machine Learning as a linked service. However, a new Synapse workspace can no longer be registered with Azure Machine Learning as a linked service. We recommend using Managed (Automatic) Synapse compute and attached Synapse Spark pools available in CLI v2 and Python SDK v2. Please see [https://aka.ms/aml-spark](https://aka.ms/aml-spark) for more details.
23+
24+
In this article, you learn how to create a linked service that links your [Azure Synapse Analytics](../../synapse-analytics/overview-what-is.md) workspace and [Azure Machine Learning workspace](../concept-workspace.md).
2225

2326
With your Azure Machine Learning workspace linked with your Azure Synapse workspace, you can attach an Apache Spark pool, powered by Azure Synapse Analytics, as a dedicated compute for data wrangling at scale or conduct model training all from the same Python notebook.
2427

2528
You can link your ML workspace and Synapse workspace via the [Python SDK](#link-sdk) or the [Azure Machine Learning studio](#link-studio).
2629

2730
You can also link workspaces and attach a Synapse Spark pool with a single [Azure Resource Manager (ARM) template](https://github.com/Azure/azure-quickstart-templates/blob/master/quickstarts/microsoft.machinelearningservices/machine-learning-linkedservice-create/azuredeploy.json).
2831

29-
>[!IMPORTANT]
30-
> The Azure Machine Learning and Azure Synapse integration is in public preview. The functionalities presented from the `azureml-synapse` package are [experimental](/python/api/overview/azure/ml/#stable-vs-experimental) preview features, and may change at any time.
31-
3232
## Prerequisites
3333

34-
* [Create an Azure Machine Learning workspace](quickstart-create-resources.md).
34+
* [Create an Azure Machine Learning workspace](../quickstart-create-resources.md).
3535

36-
* [Create a Synapse workspace in Azure portal](../synapse-analytics/quickstart-create-workspace.md).
36+
* [Create a Synapse workspace in Azure portal](../../synapse-analytics/quickstart-create-workspace.md).
3737

38-
* [Create Apache Spark pool using Azure portal, web tools, or Synapse Studio](../synapse-analytics/quickstart-create-apache-spark-pool-studio.md)
38+
* [Create Apache Spark pool using Azure portal, web tools, or Synapse Studio](../../synapse-analytics/quickstart-create-apache-spark-pool-studio.md)
3939

4040
* Install the [Azure Machine Learning Python SDK](/python/api/overview/azure/ml/intro)
4141

@@ -74,7 +74,7 @@ linked_service = LinkedService.register(workspace = ws,
7474
```
7575

7676
> [!IMPORTANT]
77-
> A managed identity, `system_assigned_identity_principal_id`, is created for each linked service. This managed identity must be granted the **Synapse Apache Spark Administrator** role of the Synapse workspace before you start your Synapse session. [Assign the Synapse Apache Spark Administrator role to the managed identity in the Synapse Studio](../synapse-analytics/security/how-to-manage-synapse-rbac-role-assignments.md).
77+
> A managed identity, `system_assigned_identity_principal_id`, is created for each linked service. This managed identity must be granted the **Synapse Apache Spark Administrator** role of the Synapse workspace before you start your Synapse session. [Assign the Synapse Apache Spark Administrator role to the managed identity in the Synapse Studio](../../synapse-analytics/security/how-to-manage-synapse-rbac-role-assignments.md).
7878
>
7979
> To find the `system_assigned_identity_principal_id` of a specific linked service, use `LinkedService.get('<your-mlworkspace-name>', '<linked-service-name>')`.
8080
@@ -145,7 +145,7 @@ Follow these steps:
145145
1. Select **Attach**.
146146
1. Select your Apache Spark pool from the list and provide a name.
147147
1. This list identifies the available Synapse Spark pools that can be attached to your compute.
148-
1. To create a new Synapse Spark pool, see [Create Apache Spark pool with the Synapse Studio](../synapse-analytics/quickstart-create-apache-spark-pool-portal.md)
148+
1. To create a new Synapse Spark pool, see [Create Apache Spark pool with the Synapse Studio](../../synapse-analytics/quickstart-create-apache-spark-pool-portal.md)
149149
1. Select **Attach selected**.
150150

151151
### Attach a pool with the Python SDK
@@ -188,7 +188,7 @@ ws.compute_targets['Synapse Spark pool alias']
188188

189189
## Next steps
190190

191-
* [How to data wrangle with Azure Synapse (preview)](v1/how-to-data-prep-synapse-spark-pool.md).
192-
* [How to use Apache Spark in your machine learning pipeline with Azure Synapse (preview)](v1/how-to-use-synapsesparkstep.md)
193-
* [Train a model](v1/how-to-set-up-training-targets.md).
194-
* [How to securely integrate Azure Synapse and Azure Machine Learning workspaces](how-to-private-endpoint-integration-synapse.md).
191+
* [How to data wrangle with Azure Synapse (preview)](how-to-data-prep-synapse-spark-pool.md).
192+
* [How to use Apache Spark in your machine learning pipeline with Azure Synapse (preview)](how-to-use-synapsesparkstep.md)
193+
* [Train a model](how-to-set-up-training-targets.md).
194+
* [How to securely integrate Azure Synapse and Azure Machine Learning workspaces](../how-to-private-endpoint-integration-synapse.md).

articles/machine-learning/v1/how-to-use-synapsesparkstep.md

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,30 +1,29 @@
11
---
2-
title: Use Apache Spark in a machine learning pipeline (preview)
2+
title: Use Apache Spark in a machine learning pipeline (deprecated)
33
titleSuffix: Azure Machine Learning
44
description: Link your Azure Synapse Analytics workspace to your Azure machine learning pipeline to use Apache Spark for data manipulation.
55
services: machine-learning
66
ms.service: machine-learning
77
ms.subservice: mldata
88
ms.author: zhanxia
99
author: xiaoharper
10-
ms.date: 11/04/2022
10+
ms.reviewer: franksolomon
11+
ms.date: 11/28/2022
1112
ms.topic: how-to
1213
ms.custom: synapse-azureml, sdkv1, event-tier1-build-2022
1314
#Customer intent: As a user of both Azure Machine Learning pipelines and Azure Synapse Analytics, I'd like to use Apache Spark for the data preparation of my pipeline
1415
---
1516

16-
# How to use Apache Spark (powered by Azure Synapse Analytics) in your machine learning pipeline (preview)
17+
# How to use Apache Spark (powered by Azure Synapse Analytics) in your machine learning pipeline (deprecated)
1718

1819

1920
[!INCLUDE [sdk v1](../../../includes/machine-learning-sdk-v1.md)]
2021

21-
22-
[!INCLUDE [preview disclaimer](../../../includes/machine-learning-preview-generic-disclaimer.md)]
22+
> [!WARNING]
23+
> The Azure Synapse Analytics integration with Azure Machine Learning available in Python SDK v1 is deprecated. Users can continue using Synapse workspace registered with Azure Machine Learning as a linked service. However, a new Synapse workspace can no longer be registered with Azure Machine Learning as a linked service. We recommend using Managed (Automatic) Synapse compute and attached Synapse Spark pools available in CLI v2 and Python SDK v2. Please see [https://aka.ms/aml-spark](https://aka.ms/aml-spark) for more details.
2324
2425
In this article, you'll learn how to use Apache Spark pools powered by Azure Synapse Analytics as the compute target for a data preparation step in an Azure Machine Learning pipeline. You'll learn how a single pipeline can use compute resources suited for the specific step, such as data preparation or training. You'll see how data is prepared for the Spark step and how it's passed to the next step.
2526

26-
27-
2827
## Prerequisites
2928

3029
* Create an [Azure Machine Learning workspace](../quickstart-create-resources.md) to hold all your pipeline resources.
@@ -35,10 +34,10 @@ In this article, you'll learn how to use Apache Spark pools powered by Azure Syn
3534

3635
## Link your Azure Machine Learning workspace and Azure Synapse Analytics workspace
3736

38-
You create and administer your Apache Spark pools in an Azure Synapse Analytics workspace. To integrate an Apache Spark pool with an Azure Machine Learning workspace, you must [link to the Azure Synapse Analytics workspace](../how-to-link-synapse-ml-workspaces.md).
37+
You create and administer your Apache Spark pools in an Azure Synapse Analytics workspace. To integrate an Apache Spark pool with an Azure Machine Learning workspace, you must [link to the Azure Synapse Analytics workspace](how-to-link-synapse-ml-workspaces.md).
3938

4039
Once your Azure Machine Learning workspace and your Azure Synapse Analytics workspaces are linked, you can attach an Apache Spark pool via
41-
* [Azure Machine Learning studio](../how-to-link-synapse-ml-workspaces.md#attach-a-pool-via-the-studio)
40+
* [Azure Machine Learning studio](how-to-link-synapse-ml-workspaces.md#attach-a-pool-via-the-studio)
4241
* Python SDK ([as elaborated below](#attach-your-apache-spark-pool-as-a-compute-target-for-azure-machine-learning))
4342
* Azure Resource Manager (ARM) template (see this [Example ARM template](https://github.com/Azure/azure-quickstart-templates/blob/master/quickstarts/microsoft.machinelearningservices/machine-learning-linkedservice-create/azuredeploy.json)).
4443
* You can use the command line to follow the ARM template, add the linked service, and attach the Apache Spark pool with the following code:

articles/machine-learning/v1/toc.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -125,7 +125,7 @@
125125
displayName: AKS, inference
126126
href: how-to-create-attach-kubernetes.md
127127
- name: Link to Azure Synapse Analytics workspace
128-
href: ../how-to-link-synapse-ml-workspaces.md
128+
href: ../v1/how-to-link-synapse-ml-workspaces.md
129129
- name: Security
130130
items:
131131
- name: Use managed identities for access control

articles/sentinel/notebooks-with-synapse-export-data.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,5 +126,5 @@ For more information, see:
126126
- [Blog post: Export Historical Log Data from Microsoft Sentinel](https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/export-historical-log-data-from-microsoft-sentinel/ba-p/3413418)
127127
- [Use Jupyter notebooks to hunt for security threats](notebooks.md)
128128
- [Tutorial: Get started with Jupyter notebooks and MSTICPy in Microsoft Sentinel](notebook-get-started.md)
129-
- [Link Azure Synapse Analytics and Azure Machine Learning workspaces and attach Apache Spark pools(preview)](../machine-learning/how-to-link-synapse-ml-workspaces.md)
129+
- [Link Azure Synapse Analytics and Azure Machine Learning workspaces and attach Apache Spark pools(preview)](../machine-learning/v1/how-to-link-synapse-ml-workspaces.md)
130130
- [Create your first Microsoft Sentinel notebook](https://techcommunity.microsoft.com/t5/microsoft-sentinel-blog/creating-your-first-microsoft-sentinel-notebook/ba-p/2977745) (Blog series)

0 commit comments

Comments
 (0)