Skip to content

Commit 0718933

Browse files
authored
Merge pull request #225089 from Blackmist/fix-v2-v1
moving v1 articles to v1 folder
2 parents bbf0c2c + b9ec597 commit 0718933

12 files changed

+23
-20
lines changed

articles/machine-learning/.openpublishing.redirection.machine-learning.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1386,8 +1386,8 @@
13861386
"redirect_document_id": true
13871387
},
13881388
{
1389-
"source_path_from_root": "/articles/machine-learning/service/how-to-save-write-experiment-files.md",
1390-
"redirect_url": "/azure/machine-learning/how-to-save-write-experiment-files",
1389+
"source_path_from_root": "/articles/machine-learning/how-to-save-write-experiment-files.md",
1390+
"redirect_url": "/azure/machine-learning/v1/how-to-save-write-experiment-files",
13911391
"redirect_document_id": true
13921392
},
13931393
{

articles/machine-learning/concept-train-machine-learning-model.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,11 @@ Machine learning pipelines can use the previously mentioned training methods. Pi
7777

7878
The Azure training lifecycle consists of:
7979

80-
1. Zipping the files in your project folder, ignoring those specified in _.amlignore_ or _.gitignore_
80+
1. Zipping the files in your project folder and upload to the cloud.
81+
82+
> [!TIP]
83+
> [!INCLUDE [amlinclude-info](../../includes/machine-learning-amlignore-gitignore.md)]
84+
8185
1. Scaling up your compute cluster
8286
1. Building or downloading the dockerfile to the compute node
8387
1. The system calculates a hash of:

articles/machine-learning/how-to-train-keras.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -180,7 +180,7 @@ It's now time to submit the job to run in AzureML. This time, you'll use `create
180180
Once completed, the job will register a model in your workspace (as a result of training) and output a link for viewing the job in AzureML studio.
181181

182182
> [!WARNING]
183-
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use a [.ignore file](how-to-save-write-experiment-files.md#storage-limits-of-experiment-snapshots) or don't include it in the source directory.
183+
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use a [.ignore file](concept-train-machine-learning-model.md#understand-what-happens-when-you-submit-a-training-job) or don't include it in the source directory.
184184
185185
### What happens during job execution
186186
As the job is executed, it goes through the following stages:

articles/machine-learning/how-to-train-pytorch.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -140,7 +140,7 @@ It's now time to submit the job to run in AzureML. This time, you'll use `create
140140
Once completed, the job will register a model in your workspace (as a result of training) and output a link for viewing the job in AzureML studio.
141141

142142
> [!WARNING]
143-
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use a [.ignore file](how-to-save-write-experiment-files.md#storage-limits-of-experiment-snapshots) or don't include it in the source directory.
143+
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use a [.ignore file](concept-train-machine-learning-model.md#understand-what-happens-when-you-submit-a-training-job) or don't include it in the source directory.
144144
145145
### What happens during job execution
146146
As the job is executed, it goes through the following stages:

articles/machine-learning/how-to-train-scikit-learn.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -164,7 +164,7 @@ It's now time to submit the job to run in AzureML. This time you'll use `create_
164164
Once completed, the job will register a model in your workspace (as a result of training) and output a link for viewing the job in AzureML studio.
165165

166166
> [!WARNING]
167-
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use a [.ignore file](how-to-save-write-experiment-files.md#storage-limits-of-experiment-snapshots) or don't include it in the source directory.
167+
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use a [.ignore file](concept-train-machine-learning-model.md#understand-what-happens-when-you-submit-a-training-job) or don't include it in the source directory.
168168
169169
### What happens during job execution
170170
As the job is executed, it goes through the following stages:

articles/machine-learning/how-to-train-tensorflow.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -166,7 +166,7 @@ It's now time to submit the job to run in AzureML. This time, you'll use `create
166166
Once completed, the job will register a model in your workspace (as a result of training) and output a link for viewing the job in AzureML studio.
167167

168168
> [!WARNING]
169-
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use a [.ignore file](how-to-save-write-experiment-files.md#storage-limits-of-experiment-snapshots) or don't include it in the source directory.
169+
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use a [.ignore file](concept-train-machine-learning-model.md#understand-what-happens-when-you-submit-a-training-job) or don't include it in the source directory.
170170
171171
### What happens during job execution
172172
As the job is executed, it goes through the following stages:

articles/machine-learning/how-to-train-with-custom-image.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -171,7 +171,7 @@ run.wait_for_completion(show_output=True)
171171
```
172172

173173
> [!WARNING]
174-
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use an [.ignore file](how-to-save-write-experiment-files.md#storage-limits-of-experiment-snapshots) or don't include it in the source directory. Instead, access your data by using a [datastore](/python/api/azureml-core/azureml.data).
174+
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use an [.ignore file](concept-train-machine-learning-model.md#understand-what-happens-when-you-submit-a-training-job) or don't include it in the source directory. Instead, access your data by using a [datastore](/python/api/azureml-core/azureml.data).
175175
176176
## Next steps
177177
In this article, you trained a model by using a custom Docker image. See these other articles to learn more about Azure Machine Learning:

articles/machine-learning/how-to-save-write-experiment-files.md renamed to articles/machine-learning/v1/how-to-save-write-experiment-files.md

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -6,19 +6,18 @@ services: machine-learning
66
author: rastala
77
ms.author: roastala
88
manager: danielsc
9-
ms.reviewer: nibaccam
109
ms.service: machine-learning
1110
ms.subservice: core
1211
ms.topic: how-to
13-
ms.date: 03/10/2020
12+
ms.date: 01/25/2023
1413

1514
---
1615
# Where to save and write files for Azure Machine Learning experiments
17-
16+
[!INCLUDE [sdk v1](../../../includes/machine-learning-sdk-v1.md)]
1817

1918
In this article, you learn where to save input files, and where to write output files from your experiments to prevent storage limit errors and experiment latency.
2019

21-
When launching training jobs on a [compute target](concept-compute-target.md), they are isolated from outside environments. The purpose of this design is to ensure reproducibility and portability of the experiment. If you run the same script twice, on the same or another compute target, you receive the same results. With this design, you can treat compute targets as stateless computation resources, each having no affinity to the jobs that are running after they are finished.
20+
When launching training jobs on a [compute target](../concept-compute-target.md), they are isolated from outside environments. The purpose of this design is to ensure reproducibility and portability of the experiment. If you run the same script twice, on the same or another compute target, you receive the same results. With this design, you can treat compute targets as stateless computation resources, each having no affinity to the jobs that are running after they are finished.
2221

2322
## Where to save input files
2423

@@ -30,7 +29,7 @@ The storage limit for experiment snapshots is 300 MB and/or 2000 files.
3029

3130
For this reason, we recommend:
3231

33-
* **Storing your files in an Azure Machine Learning [dataset](/python/api/azureml-core/azureml.data).** This prevents experiment latency issues, and has the advantages of accessing data from a remote compute target, which means authentication and mounting are managed by Azure Machine Learning. Learn more about how to specify a dataset as your input data source in your training script with [Train with datasets](v1/how-to-train-with-datasets.md).
32+
* **Storing your files in an Azure Machine Learning [dataset](/python/api/azureml-core/azureml.data).** This prevents experiment latency issues, and has the advantages of accessing data from a remote compute target, which means authentication and mounting are managed by Azure Machine Learning. Learn more about how to specify a dataset as your input data source in your training script with [Train with datasets](how-to-train-with-datasets.md).
3433

3534
* **If you only need a couple data files and dependency scripts and can't use a datastore,** place the files in the same folder directory as your training script. Specify this folder as your `source_directory` directly in your training script, or in the code that calls your training script.
3635

@@ -50,15 +49,15 @@ To resolve this error, store your experiment files on a datastore. If you can't
5049
Experiment description|Storage limit solution
5150
---|---
5251
Less than 2000 files & can't use a datastore| Override snapshot size limit with <br> `azureml._restclient.snapshots_client.SNAPSHOT_MAX_SIZE_BYTES = 'insert_desired_size'`<br> This may take several minutes depending on the number and size of files.
53-
Must use specific script directory| [!INCLUDE [amlinclude-info](../../includes/machine-learning-amlignore-gitignore.md)]
52+
Must use specific script directory| [!INCLUDE [amlinclude-info](../../../includes/machine-learning-amlignore-gitignore.md)]
5453
Pipeline|Use a different subdirectory for each step
5554
Jupyter notebooks| Create a `.amlignore` file or move your notebook into a new, empty, subdirectory and run your code again.
5655

5756
## Where to write files
5857

5958
Due to the isolation of training experiments, the changes to files that happen during jobs are not necessarily persisted outside of your environment. If your script modifies the files local to compute, the changes are not persisted for your next experiment job, and they're not propagated back to the client machine automatically. Therefore, the changes made during the first experiment job don't and shouldn't affect those in the second.
6059

61-
When writing changes, we recommend writing files to storage via an Azure Machine Learning dataset with an [OutputFileDatasetConfig object](/python/api/azureml-core/azureml.data.output_dataset_config.outputfiledatasetconfig). See [how to create an OutputFileDatasetConfig](v1/how-to-train-with-datasets.md#where-to-write-training-output).
60+
When writing changes, we recommend writing files to storage via an Azure Machine Learning dataset with an [OutputFileDatasetConfig object](/python/api/azureml-core/azureml.data.output_dataset_config.outputfiledatasetconfig). See [how to create an OutputFileDatasetConfig](how-to-train-with-datasets.md#where-to-write-training-output).
6261

6362
Otherwise, write files to the `./outputs` and/or `./logs` folder.
6463

@@ -73,4 +72,4 @@ Otherwise, write files to the `./outputs` and/or `./logs` folder.
7372

7473
* Learn more about [accessing data from storage](how-to-access-data.md).
7574

76-
* Learn more about [Create compute targets for model training and deployment](how-to-create-attach-compute-studio.md)
75+
* Learn more about [Create compute targets for model training and deployment](../how-to-create-attach-compute-studio.md)

articles/machine-learning/v1/how-to-train-pytorch.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -211,7 +211,7 @@ src = ScriptRunConfig(source_directory=project_folder,
211211
```
212212

213213
> [!WARNING]
214-
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use a [.ignore file](../how-to-save-write-experiment-files.md#storage-limits-of-experiment-snapshots) or don't include it in the source directory . Instead, access your data using an Azure ML [dataset](how-to-train-with-datasets.md).
214+
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use a [.ignore file](how-to-save-write-experiment-files.md#storage-limits-of-experiment-snapshots) or don't include it in the source directory . Instead, access your data using an Azure ML [dataset](how-to-train-with-datasets.md).
215215

216216
For more information on configuring jobs with ScriptRunConfig, see [Configure and submit training runs](how-to-set-up-training-targets.md).
217217

articles/machine-learning/v1/how-to-train-scikit-learn.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,7 +141,7 @@ run.wait_for_completion(show_output=True)
141141
```
142142

143143
> [!WARNING]
144-
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use a [.ignore file](../how-to-save-write-experiment-files.md#storage-limits-of-experiment-snapshots) or don't include it in the source directory . Instead, access your data using an Azure ML [dataset](how-to-train-with-datasets.md).
144+
> Azure Machine Learning runs training scripts by copying the entire source directory. If you have sensitive data that you don't want to upload, use a [.ignore file](how-to-save-write-experiment-files.md#storage-limits-of-experiment-snapshots) or don't include it in the source directory . Instead, access your data using an Azure ML [dataset](how-to-train-with-datasets.md).
145145
146146
### What happens during run execution
147147
As the run is executed, it goes through the following stages:

0 commit comments

Comments
 (0)