You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/blog/production-ready-object-detection-model-training-workflow-with-hpe-machine-learning-development-environment.md
+15-15Lines changed: 15 additions & 15 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -181,40 +181,40 @@ Here, you will be using the SAHI library to slice our large satellite images. Sa
181
181
182
182
## 4. Upload to S3 bucket to support distributed training
183
183
184
-
We will now upload our exported data to a publically accessible AWS S3 bucket. This will enable for a largescale distributed experiment to have access to the dataset without installing the dataset on device.
184
+
Now, you can upload your exported data to a publicly accessible AWS S3 bucket. For a large-scale distributed experiment, this will enable you to access the dataset without installing the dataset on the device.
185
185
View [Determined Documentation](<* https://docs.determined.ai/latest/training/load-model-data.html#streaming-from-object-storage>) and [AWS instructions](<* https://codingsight.com/upload-files-to-aws-s3-with-the-aws-cli/>) to learn how to upload your dataset to an S3 bucket. Review the `S3Backend` class in `data.py`
186
186
187
-
Once you create an S3 bucket that is publically accessible, here are example commands to upload the preprocessed dataset to S3:
187
+
Once you create an S3 bucket that is publicly accessible, here are example commands to upload the preprocessed dataset to S3:
Our satellite imagery data is in an S3 bucket and is prepped for distributed training, so now we can progress to model training and inference via the NGC Container.
192
+
Now that the satellite imagery data is in an S3 bucket and is prepped for distributed training, you can progress to model training and inferencing via the NGC container.
193
193
194
-
# Part 3: End-to-End example training object detection model using NVIDIA PyTorch Container from NGC
194
+
# Part 3: End-to-End example training object detection model using NVIDIA PyTorch container from NGC
195
195
196
-
## Training and Inference via NGC Container
196
+
## Training and inference via NGC Container
197
197
198
-
This notebook walks you each step to train a model using containers from the NGC Catalog. We chose the GPUoptimized Pytorch container as an example. The basics of working with docker containers apply to all NGC containers.
198
+
This notebook walks you through each step to train a model using containers from the NGC Catalog. I chose the GPU-optimized PyTorch container for this example. The basics of working with Docker containers apply to all NGC containers.
199
199
200
200
We will show you how to:
201
201
202
-
* Execute training a object detection on satellite imagery using TensorFlow and Jupyter Notebook
202
+
* Execute training an object detection model on satellite imagery using TensorFlow and Jupyter Notebook
203
203
* Run inference on a trained object detection model using the SAHI library
204
204
205
-
Note this Object Detection demo is based on https://github.com/pytorch/vision/tree/v0.11.3 and ngc docker image `nvcr.io/nvidia/pytorch:21.11-py3`
205
+
Note this object detection demo is based on [this PyTorch repo](https://github.com/pytorch/vision/tree/v0.11.3) and ngc docker image `nvcr.io/nvidia/pytorch:21.11-py3`
206
206
207
-
We assume you completed step 2 of dataset preprocessing and have your tiled satellite imagery dataset completed and in the local directory `train_images_rgb_no_neg/train_images_300_02`
207
+
It is assumed that, by now, you have completed step 2 of dataset preprocessing and have your tiled satellite imagery dataset completed and in the local directory `train_images_rgb_no_neg/train_images_300_02`
208
208
209
-
Let's get started!
209
+
Let's get started!
210
210
211
-
## Execute docker run to create NGC environment for Data Prep
211
+
## Execute Docker run to create NGC environment for data prep
212
212
213
-
Make sure to map host directory to docker directory, we will use the host directory again to
213
+
Make sure to map host directory to Docker directory. You will use the host directory again to:
214
214
215
215
*`docker run --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 -v /home/ubuntu:/home/ubuntu -p 8008:8888 -it nvcr.io/nvidia/pytorch:21.11-py3 /bin/bash`
216
216
217
-
## Run Jupyter notebook command within docker container to access it on your local browser
217
+
## Run Jupyter Notebook command within Docker container to access it on your local browser
The below cell will run a multi-gpu training job. This job will train an object detection model (faster-rcnn) on a dataset of satellite imagery images that contain 61 classes of objects
229
+
The below cell will run a multi-gpu training job. This job will train an object detection model (faster-rcnn) on a dataset of satellite imagery images that contain 61 classes of objects.
230
230
231
231
* Change `nproc_per_node` argument to specify the number of GPUs available on your server
0 commit comments