You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/service/how-to-save-write-experiment-files.md
+5-3Lines changed: 5 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -29,6 +29,8 @@ Azure Machine Learning executes training scripts by copying the entire script fo
29
29
30
30
***For large data and dependency files:** Store your files in an Azure Machine Learning [datastore](https://docs.microsoft.com/python/api/azureml-core/azureml.data?view=azure-ml-py). It has the advantages of accessing data from a remote compute target, which means things like authentication and mounting are managed by Azure Machine Learning service. Learn more about specifying a datastore as your source directory, and uploading files to your datastore in the [Access data from your datastores](how-to-access-data.md) article.
31
31
32
+
<aname="limits"></a>
33
+
32
34
### Storage limits of experiment snapshots
33
35
34
36
For experiments, Azure Machine Learning also makes an experiment snapshot of your code based on the directory you suggest when you configure the run. This has a total limit of 300 MB and/or 2000 files. If you exceed this limit, you'll see the following error:
@@ -40,11 +42,11 @@ Your total snapshot size exceeds the limit of 300.0 MB
40
42
41
43
Try one of the following solutions to resolve the error.
Very large files| Move files to a datastore, and specify the datastore as your source_directory to prevent latency issues when the script folder is copied to the compute target environment at runtime.
46
48
Many/ large files, but don't want a datastore| Override limit by setting SNAPSHOT_MAX_SIZE_BYTES to whatever your experiment needs. <br> `azureml._restclient.snapshots_client.SNAPSHOT_MAX_SIZE_BYTES = 'insert_desired_size'`
47
-
Must use specific script directory| Make an ignore file (`.gitignore` or `.amlignore`) to prevent files from being included in the experiment snapshot that are not really a part of the source code. Place this file in the directory and add the files names to ignore to it. The `.amlignore` file uses the same syntax and patterns as the `.gitignore` file. If both files exist, the `.amlignore` file takes precedence.
49
+
Must use specific script directory| Make an ignore file (`.gitignore` or `.amlignore`) to prevent files from being included in the experiment snapshot that are not really a part of the source code. Place this file in the directory and add the files names to ignore in it. The `.amlignore` file uses the same syntax and patterns as the `.gitignore` file. If both files exist, the `.amlignore` file takes precedence.
48
50
Pipeline|Use a different subdirectory for each step or create an ignore file.
49
51
Jupyter notebooks| You are likely using a directory that has more than 300 MB worth of data or files inside. Move your notebook into a new, empty, subdirectory with the following steps. <br> 1. Create a new folder.<br> 2. Move Jupyter notebook into empty folder. <br> 3. Run the code again.
50
52
@@ -67,4 +69,4 @@ Write files to one of the following:
67
69
68
70
* Learn more about [accessing data from your datastores](how-to-access-data.md).
69
71
70
-
* Learn more about [How to Set Up Training Targets](how-to-set-up-training-targets.md).
72
+
* Learn more about [How to Set Up Training Targets](how-to-set-up-training-targets.md).
0 commit comments