You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/how-to-migrate-from-v1.md
+20-21Lines changed: 20 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,10 +8,11 @@ ms.subservice: core
8
8
ms.topic: how-to
9
9
author: sdgilley
10
10
ms.author: sgilley
11
-
ms.date: 02/22/2024
12
-
ms.reviewer: balapv
11
+
ms.date: 10/14/2024
12
+
ms.reviewer: paulsh
13
13
ms.custom: devplatv2, devx-track-python
14
14
monikerRange: 'azureml-api-2 || azureml-api-1'
15
+
#customer intent: As a developer, I want to understand the differences between v1 and v2 of Azure Machine Learning REST APIs, CLI extension, and Python SDK, to decide which I should use.
15
16
---
16
17
17
18
# Upgrade to v2
@@ -36,7 +37,7 @@ A new v2 project can reuse existing v1 resources like workspaces and compute and
36
37
37
38
Some feature gaps in v2 include:
38
39
39
-
- Spark support in jobs - this is currently in preview in v2.
40
+
- Spark support in jobs - currently in preview in v2.
40
41
- Publishing jobs (pipelines in v1) as endpoints. You can however, schedule pipelines without publishing.
41
42
- Support for SQL/database datastores.
42
43
- Ability to use classic prebuilt components in the designer with v2.
@@ -100,13 +101,13 @@ For a comparison of SDK v1 and v2 code, see [Workspace management in SDK v1 and
100
101
101
102
### Datastore
102
103
103
-
Object storage datastore types created with v1 are fully available for use in v2. Database datastores are not supported; export to object storage (usually Azure Blob) is the recommended migration path.
104
+
Object storage datastore types created with v1 are fully available for use in v2. Database datastores aren't supported; export to object storage (usually Azure Blob) is the recommended migration path.
104
105
105
106
For a comparison of SDK v1 and v2 code, see [Datastore management in SDK v1 and SDK v2](migrate-to-v2-resource-datastore.md).
106
107
107
108
### Data (datasets in v1)
108
109
109
-
Datasets are renamed to data assets. *Backwards compatibility* is provided, which means you can use V1 Datasets in V2. When you consume a V1 Dataset in a V2 job you will notice they are automatically mapped into V2 types as follows:
110
+
Datasets are renamed to data assets. *Backwards compatibility* is provided, which means you can use V1 Datasets in V2. When you consume a V1 Dataset in a V2 job, you'll notice they're automatically mapped into V2 types as follows:
110
111
111
112
* V1 FileDataset = V2 Folder (`uri_folder`)
112
113
* V1 TabularDataset = V2 Table (`mltable`)
@@ -126,7 +127,7 @@ For a comparison of SDK v1 and v2 code, see [Compute management in SDK v1 and SD
126
127
127
128
### Jobs (experiments, runs, pipelines in v1)
128
129
129
-
In v2, "experiments", "runs", and "pipelines" are consolidated into jobs. A job has a type. Most jobs are `command` jobs that run a command, like `python main.py`. What runs in a job is agnostic to any programming language, so you can run `bash` scripts, invoke `python` interpreters, run a bunch of `curl` commands, or anything else. Another common type of job is `pipeline`, which defines child jobs that may have input/output relationships, forming a directed acyclic graph (DAG).
130
+
In v2, "experiments," "runs," and "pipelines" are consolidated into jobs. A job has a type. Most jobs are `command` jobs that run a command, like `python main.py`. What runs in a job is agnostic to any programming language, so you can run `bash` scripts, invoke `python` interpreters, run a bunch of `curl` commands, or anything else. Another common type of job is `pipeline`, which defines child jobs that might have input/output relationships, forming a directed acyclic graph (DAG).
130
131
131
132
For a comparison of SDK v1 and v2 code, see
132
133
*[Run a script](migrate-to-v2-command-job.md)
@@ -140,9 +141,9 @@ For a comparison of SDK v1 and v2 code, see
140
141
141
142
You can use designer to build pipelines using your own v2 custom components and the new prebuilt components from registry. In this situation, you can use v1 or v2 data assets in your pipeline.
142
143
143
-
You can continue to use designer to build pipelines using classic prebuilt components and v1 dataset types (tabular, file). You cannot use existing designer classic prebuilt components with v2 data asset.
144
+
You can continue to use designer to build pipelines using classic prebuilt components and v1 dataset types (tabular, file). You can't use existing designer classic prebuilt components with v2 data asset.
144
145
145
-
You cannot build a pipeline using both existing designer classic prebuilt components and v2 custom components.
146
+
You can't build a pipeline using both existing designer classic prebuilt components and v2 custom components.
146
147
147
148
148
149
### Model
@@ -153,7 +154,7 @@ For a comparison of SDK v1 and v2 code, see [Model management in SDK v1 and SDK
153
154
154
155
### Endpoint and deployment (endpoint and web service in v1)
155
156
156
-
With SDK/CLI v1, you can deploy models on ACI or AKS as web services. Your existing v1 model deployments and web services will continue to function as they are, but Using SDK/CLI v1 to deploy models on ACI or AKS as web services is now considered as **legacy**. For new model deployments, we recommend upgrading to v2. In v2, we offer [managed endpoints or Kubernetes endpoints](./concept-endpoints.md?view=azureml-api-2&preserve-view=true). The following table guides our recommendation:
157
+
With SDK/CLI v1, you can deploy models on ACI or AKS as web services. Your existing v1 model deployments and web services continue to function as they are, but Using SDK/CLI v1 to deploy models on ACI or AKS as web services is now considered as **legacy**. For new model deployments, we recommend upgrading to v2. In v2, we offer [managed endpoints or Kubernetes endpoints](./concept-endpoints.md?view=azureml-api-2&preserve-view=true). The following table guides our recommendation:
157
158
158
159
|Endpoint type in v2|Upgrade from|Notes|
159
160
|-|-|-|
@@ -172,7 +173,7 @@ Environments created from v1 can be used in v2. In v2, environments have new fea
172
173
173
174
## Managing secrets
174
175
175
-
The management of Key Vault secrets differs significantly in V2 compared to V1. The V1 set_secret and get_secret SDK methods are not available in V2. Instead, direct access using Key Vault client libraries should be used. When accessing secrets from a training script, you can use either the managed identity of the compute or your identity.
176
+
The management of Key Vault secrets differs significantly in V2 compared to V1. The V1 set_secret and get_secret SDK methods aren't available in V2. Instead, direct access using Key Vault client libraries should be used. When accessing secrets from a training script, you can use either the managed identity of the compute or your identity.
176
177
177
178
For details about Key Vault, see [Use authentication credential secrets in Azure Machine Learning training jobs](how-to-use-secrets-in-runs.md?view=azureml-api-2&preserve-view=true).
178
179
@@ -184,11 +185,11 @@ There are a few scenarios that are common across the machine learning lifecycle
184
185
185
186
Azure recommends Azure Resource Manager templates (often via Bicep for ease of use) to create resources. The same is a good approach for creating Azure Machine Learning resources as well.
186
187
187
-
If your team is only using Azure Machine Learning, you may consider provisioning the workspace and any other resources via YAML files and CLI instead.
188
+
If your team is only using Azure Machine Learning, you might consider provisioning the workspace and any other resources via YAML files and CLI instead.
188
189
189
190
### Prototyping models
190
191
191
-
We recommend v2 for prototyping models. You may consider using the CLI for an interactive use of Azure Machine Learning, while your model training code is Python or any other programming language. Alternatively, you may adopt a full-stack approach with Python solely using the Azure Machine Learning SDK or a mixed approach with the Azure Machine Learning Python SDK and YAML files.
192
+
We recommend v2 for prototyping models. You might consider using the CLI for an interactive use of Azure Machine Learning, while your model training code is Python or any other programming language. Alternatively, you might adopt a full-stack approach with Python solely using the Azure Machine Learning SDK or a mixed approach with the Azure Machine Learning Python SDK and YAML files.
192
193
193
194
### Production model training
194
195
@@ -210,30 +211,28 @@ The solution accelerator for MLOps with v2 is being developed at https://github.
210
211
211
212
### A note on GitOps with v2
212
213
213
-
A key paradigm with v2 is serializing machine learning entities as YAML files for source control with `git`, enabling better GitOps approaches than were possible with v1. For instance, you could enforce policy by which only a service principal used in CI/CD pipelines can create/update/delete some or all entities, ensuring changes go through a governed process like pull requests with required reviewers. Since the files in source control are YAML, they're easy to diff and track changes over time. You and your team may consider shifting to this paradigm as you upgrade to v2.
214
+
A key paradigm with v2 is serializing machine learning entities as YAML files for source control with `git`, enabling better GitOps approaches than were possible with v1. For instance, you could enforce policy by which only a service principal used in CI/CD pipelines can create/update/delete some or all entities, ensuring changes go through a governed process like pull requests with required reviewers. Since the files in source control are YAML, they're easy to diff and track changes over time. You and your team might consider shifting to this paradigm as you upgrade to v2.
214
215
215
-
You can obtain a YAML representation of any entity with the CLI via `az ml <entity> show --output yaml`. Note that this output will have system-generated properties, which can be ignored or deleted.
216
+
You can obtain a YAML representation of any entity with the CLI via `az ml <entity> show --output yaml`. This output has system-generated properties, which can be ignored or deleted.
216
217
217
218
## Should I upgrade existing v1 code to v2
218
219
219
-
You can reuse your existing v1 assets in your v2 workflows. For instance a model created in v1 can be used to perform Managed Inferencing in v2.
220
+
You can reuse your existing v1 assets in your v2 workflows. For instance, a model created in v1 can be used to perform Managed Inferencing in v2.
220
221
221
-
Optionally, if you want to upgrade specific parts of your existing v1 code to v2, please refer to the comparison links provided in this document.
222
+
Optionally, if you want to upgrade specific parts of your existing v1 code to v2, refer to the comparison links provided in this document.
222
223
223
224
## Can I use v1 and v2 together?
224
225
225
-
v1 and v2 can co-exist in a workspace. You can reuse your existing assets in your v2 workflows. For instance a model created in v1 can be used to perform Managed Inferencing in v2. Resources like workspace, compute, and datastore work across v1 and v2, with exceptions. A user can call the v1 Python SDK to change a workspace's description, then using the v2 CLI extension change it again. Jobs (experiments/runs/pipelines in v1) can be submitted to the same workspace from the v1 or v2 Python SDK. A workspace can have both v1 and v2 model deployment endpoints.
226
+
v1 and v2 can coexist in a workspace. You can reuse your existing assets in your v2 workflows. For instance, a model created in v1 can be used to perform Managed Inferencing in v2. Resources like workspace, compute, and datastore work across v1 and v2, with exceptions. A user can call the v1 Python SDK to change a workspace's description, then using the v2 CLI extension change it again. Jobs (experiments/runs/pipelines in v1) can be submitted to the same workspace from the v1 or v2 Python SDK. A workspace can have both v1 and v2 model deployment endpoints.
226
227
227
228
### Using v1 and v2 code together
228
229
229
-
We do not recommend using the v1 and v2 SDKs together in the same code. It is technically possible to use v1 and v2 in the same code because they use different Azure namespaces. However, there are many classes with the same name across these namespaces (like Workspace, Model) which can cause confusion and make code readability and debuggability challenging.
230
+
We don't recommend using the v1 and v2 SDKs together in the same code. It's technically possible to use v1 and v2 in the same code because they use different Azure namespaces. However, there are many classes with the same name across these namespaces (like Workspace, Model) which can cause confusion and make code readability and debuggability challenging.
230
231
231
232
> [!IMPORTANT]
232
233
> If your workspace uses a private endpoint, it will automatically have the `v1_legacy_mode` flag enabled, preventing usage of v2 APIs. See [how to configure network isolation with v2](how-to-configure-network-isolation-with-v2.md?view=azureml-api-2&preserve-view=true) for details.
233
234
234
-
235
-
236
-
## Next steps
235
+
## Related content
237
236
238
237
-[Get started with the CLI (v2)](how-to-configure-cli.md?view=azureml-api-2&preserve-view=true)
239
238
-[Get started with the Python SDK (v2)](https://aka.ms/sdk-v2-install)
0 commit comments