You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/machine-learning/tutorial-horovod-tensorflow.md
+13-13Lines changed: 13 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,7 +13,7 @@ ms.author: midesa
13
13
14
14
[Horovod](https://github.com/horovod/horovod) is a distributed training framework for libraries like TensorFlow and PyTorch. With Horovod, users can scale up an existing training script to run on hundreds of GPUs in just a few lines of code.
15
15
16
-
Within Azure Synapse Analytics, users can quickly get started with Horovod using the default Apache Spark 3 runtime. For Spark ML pipeline applications using TensorFlow, users can use ```HorovodRunner```. This notebook uses an Apache Spark dataframe to perform distributed training of a distributed neural network (DNN) model on MNIST dataset. This tutorial leverages TensorFlow and the ```HorovodRunner``` to run the training process.
16
+
Within Azure Synapse Analytics, users can quickly get started with Horovod using the default Apache Spark 3 runtime. For Spark ML pipeline applications using TensorFlow, users can use ```HorovodRunner```. This notebook uses an Apache Spark dataframe to perform distributed training of a distributed neural network (DNN) model on MNIST dataset. This tutorial uses TensorFlow and the ```HorovodRunner``` to run the training process.
17
17
18
18
## Prerequisites
19
19
@@ -27,9 +27,9 @@ Within Azure Synapse Analytics, users can quickly get started with Horovod using
27
27
28
28
## Configure the Apache Spark session
29
29
30
-
At the start of the session, we will need to configure a few Apache Spark settings. In most cases, we only need to set the ```numExecutors``` and ```spark.rapids.memory.gpu.reserve```. For very large models, users may also need to configure the ```spark.kryoserializer.buffer.max``` setting. For TensorFlow models, users will need to set the ```spark.executorEnv.TF_FORCE_GPU_ALLOW_GROWTH``` to be true.
30
+
At the start of the session, we need to configure a few Apache Spark settings. In most cases, we only need to set the ```numExecutors``` and ```spark.rapids.memory.gpu.reserve```. For very large models, users may also need to configure the ```spark.kryoserializer.buffer.max``` setting. For TensorFlow models, users need to set the ```spark.executorEnv.TF_FORCE_GPU_ALLOW_GROWTH``` to be true.
31
31
32
-
In the example below, you can see how the Spark configurations can be passed with the ```%%configure``` command. The detailed meaning of each parameter is explained in the [Apache Spark configuration documentation](https://spark.apache.org/docs/latest/configuration.html). The values provided below are the suggested, best practice values for Azure Synapse GPU-large pools.
32
+
In the example, you can see how the Spark configurations can be passed with the ```%%configure``` command. The detailed meaning of each parameter is explained in the [Apache Spark configuration documentation](https://spark.apache.org/docs/latest/configuration.html). The values provided are the suggested, best practice values for Azure Synapse GPU-large pools.
33
33
34
34
```spark
35
35
@@ -48,7 +48,7 @@ In the example below, you can see how the Spark configurations can be passed wit
48
48
}
49
49
```
50
50
51
-
For this tutorial, we will use the following configurations:
51
+
For this tutorial, we use the following configurations:
52
52
53
53
```python
54
54
@@ -67,9 +67,9 @@ For this tutorial, we will use the following configurations:
67
67
68
68
## Setup primary storage account
69
69
70
-
We will need the Azure Data Lake Storage (ADLS) account for storing intermediate and model data. If you are using an alternative storage account, be sure to set up the [linked service](../../data-factory/concepts-linked-services.md) to automatically authenticate and read from the account.
70
+
We need the Azure Data Lake Storage (ADLS) account for storing intermediate and model data. If you are using an alternative storage account, be sure to set up the [linked service](../../data-factory/concepts-linked-services.md) to automatically authenticate and read from the account.
71
71
72
-
In this example, we will read from the primary Azure Synapse Analytics storage account. To do this, you will need to modify the following properties below: ```remote_url```.
72
+
In this example, we read data from the primary Azure Synapse Analytics storage account. To read the results you need to modify the following properties: ```remote_url```.
Next, we will prepare the dataset for training. In this tutorial, we will use the MNIST dataset from [Azure Open Datasets](../../open-datasets/dataset-mnist.md?tabs=azureml-opendatasets).
87
+
Next, we prepare the dataset for training. In this tutorial, we use the MNIST dataset from [Azure Open Datasets](../../open-datasets/dataset-mnist.md?tabs=azureml-opendatasets).
Once we have finished processing our dataset, we can now define our TensorFlow model. The same code could also be used to train a single-node TensorFlow model.
134
+
Once our dataset is processed, we can define our TensorFlow model. The same code could also be used to train a single-node TensorFlow model.
135
135
136
136
```python
137
137
# Define the TensorFlow model without any Horovod-specific parameters
@@ -158,7 +158,7 @@ def get_model():
158
158
159
159
## Define a training function for a single node
160
160
161
-
First, we will train our TensorFlow model on the driver node of the Apache Spark pool. Once we have finished the training process, we will evaluate the model and print the loss and accuracy scores.
161
+
First, we train our TensorFlow model on the driver node of the Apache Spark pool. Once the training process is complete, we evaluate the model and print the loss and accuracy scores.
162
162
163
163
```python
164
164
@@ -208,7 +208,7 @@ Next, we will take a look at how the same code could be re-run using ```HorovodR
208
208
209
209
### Define training function
210
210
211
-
To do this, we will first define a training function for ```HorovodRunner```.
211
+
To train a model, we first define a training function for ```HorovodRunner```.
Copy file name to clipboardExpand all lines: articles/synapse-analytics/machine-learning/tutorial-load-data-petastorm.md
+7-7Lines changed: 7 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@ ms.author: midesa
11
11
12
12
# Load data with Petastorm (Preview)
13
13
14
-
Petastorm is an open source data access library which enables single-node or distributed training of deep learning models. This library enables training directly from datasets in Apache Parquet format and datasets that have already been loaded as an Apache Spark DataFrame. Petastorm supports popular training frameworks such as Tensorflow and PyTorch.
14
+
Petastorm is an open source data access library, which enables single-node or distributed training of deep learning models. This library enables training directly from datasets in Apache Parquet format and datasets that are loaded as an Apache Spark DataFrame. Petastorm supports popular training frameworks such as Tensorflow and PyTorch.
15
15
16
16
For more information about Petastorm, you can visit the [Petastorm GitHub page](https://github.com/uber/petastorm) or the [Petastorm API documentation](https://petastorm.readthedocs.io/en/latest).
17
17
@@ -28,7 +28,7 @@ For more information about Petastorm, you can visit the [Petastorm GitHub page](
28
28
29
29
## Configure the Apache Spark session
30
30
31
-
At the start of the session, we will need to configure a few Apache Spark settings. In most cases, we only need to set the ```numExecutors``` and ```spark.rapids.memory.gpu.reserve```. In the example below, you can see how the Spark configurations can be passed with the ```%%configure``` command. The detailed meaning of each parameter is explained in the [Apache Spark configuration documentation](https://spark.apache.org/docs/latest/configuration.html).
31
+
At the start of the session, we need to configure a few Apache Spark settings. In most cases, we only need to set the ```numExecutors``` and ```spark.rapids.memory.gpu.reserve```. In the example, you can see how the Spark configurations can be passed with the ```%%configure``` command. The detailed meaning of each parameter is explained in the [Apache Spark configuration documentation](https://spark.apache.org/docs/latest/configuration.html).
32
32
33
33
```python
34
34
%%configure -f
@@ -44,7 +44,7 @@ At the start of the session, we will need to configure a few Apache Spark settin
44
44
45
45
A dataset created using Petastorm is stored in an Apache Parquet format. On top of a Parquet schema, Petastorm also stores higher-level schema information that makes multidimensional arrays into a native part of a Petastorm dataset.
46
46
47
-
In the sample below, we create a dataset using PySpark. We write the dataset to an Azure Data Lake Storage Gen2 account.
47
+
In the sample, we create a dataset using PySpark. We write the dataset to an Azure Data Lake Storage Gen2 account.
The ```petastorm.reader.Reader``` class is the main entry point for user code that accesses the data from an ML framework such as Tensorflow or Pytorch. You can read a dataset using the ```petastorm.reader.Reader``` class and the ```petastorm.make_reader``` factory method.
104
104
105
-
In the example below, you can see how you can pass an ```abfs``` URL protocol.
105
+
In the example, you can see how you can pass an ```abfs``` URL protocol.
106
106
107
107
```python
108
108
from petastorm import make_reader
@@ -115,7 +115,7 @@ with make_reader('abfs://<container_name>/<data directory path>/') as reader:
115
115
116
116
### Read dataset from secondary storage account
117
117
118
-
If you are using an alternative storage account, be sure to set up the [linked service](../../data-factory/concepts-linked-services.md) to automatically authenticate and read from the account. In addition, you will need to modify the following properties below: ```remote_url```, ```account_name```, and ```linked_service_name```.
118
+
If you are using an alternative storage account, be sure to set up the [linked service](../../data-factory/concepts-linked-services.md) to automatically authenticate and read from the account. In addition, you need to modify the following properties: ```remote_url```, ```account_name```, and ```linked_service_name```.
119
119
120
120
```python
121
121
from petastorm import make_reader
@@ -134,7 +134,7 @@ with make_reader('{}/data_directory'.format(remote_url), storage_options = {'sas
134
134
135
135
### Read dataset in batches
136
136
137
-
In the example below, you can see how you can pass an ```abfs``` URL protocol to read data in batches. This example uses the ```make_batch_reader``` class.
137
+
In the example, you can see how you can pass an ```abfs``` URL protocol to read data in batches. This example uses the ```make_batch_reader``` class.
138
138
139
139
```python
140
140
from petastorm import make_batch_reader
@@ -146,7 +146,7 @@ with make_batch_reader('abfs://<container_name>/<data directory path>/', schema_
146
146
147
147
## PyTorch API
148
148
149
-
To read a Petastorm dataset from PyTorch, you can use the adapter ```petastorm.pytorch.DataLoader``` class. This allows for custom PyTorch collating functions and transforms to be supplied.
149
+
To read a Petastorm dataset from PyTorch, you can use the adapter ```petastorm.pytorch.DataLoader``` class. This adapter allows for custom PyTorch collating functions and transforms to be supplied.
150
150
151
151
In this example, we will show how Petastorm DataLoader can be used to load a Petastorm dataset with the help of make_reader API. This first section creates the definition of a ```Net``` class and ```train``` and ```test``` function.
0 commit comments