Skip to content

Commit c0735f5

Browse files
committed
Update Blog “production-ready-object-detection-model-training-workflow-with-hpe-machine-learning-development-environment”
1 parent 8fb9a55 commit c0735f5

File tree

1 file changed

+13
-13
lines changed

1 file changed

+13
-13
lines changed

content/blog/production-ready-object-detection-model-training-workflow-with-hpe-machine-learning-development-environment.md

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,7 @@ Now that you have your Docker engine installed and the PyTorch Container running
8181

8282
This notebook walks you through each step required to train a model using containers from the NGC catalog. We chose the GPU optimized PyTorch container as an example. The basics of working with docker containers apply to all NGC containers.
8383

84-
We will show you how to:
84+
Here, I will show you how to:
8585

8686
* Download the Xview Dataset
8787
* How to convert labels to coco format
@@ -92,25 +92,25 @@ Let's get started!
9292

9393
- - -
9494

95-
## Pre-reqs, set up Jupyter notebook environment using NGC container
95+
## Pre-reqs, set up Jupyter Notebook environment using NGC container
9696

97-
### Execute docker run to create NGC environment for data preparation
97+
### Execute Docker run to create NGC environment for data preparation
9898

99-
Make sure to map host directory to docker directory, we will use the host directory again to
99+
Make sure to map host directory to Docker directory. You will use the host directory again to do the following:
100100

101101
* `docker run --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 -v /home/ubuntu:/home/ubuntu -p 8008:8888 -it nvcr.io/nvidia/pytorch:21.11-py3 /bin/bash`
102102

103-
### Run Jupyter notebook command within Docker container to access it on your local browser
103+
### Run Jupyter Notebook command within Docker container to access it on your local browser
104104

105105
* `cd /home/ubuntu`
106106
* `jupyter lab --ip=0.0.0.0 --port=8888 --NotebookApp.token='' --NotebookApp.password=''`
107107
* `git clone https://github.com/interactivetech/e2e_blogposts.git`
108108

109-
### Download the Xview Dataset
109+
### Download the xView dataset
110110

111111
The dataset we will be using is from the [DIUx xView 2018 Challenge](https://challenge.xviewdataset.org) by U.S. National Geospatial-Intelligence Agency (NGA). You will need to [create an account](https://challenge.xviewdataset.org/welcome), agree to the terms and conditions, and download the dataset manually.
112112

113-
You can also [download the dataset](https://challenge.xviewdataset.org/data-download).
113+
You can also [download the dataset](https://challenge.xviewdataset.org/data-download).
114114

115115
```python
116116
# run pip install to get the SAHI library
@@ -146,9 +146,9 @@ You can also [download the dataset](https://challenge.xviewdataset.org/data-down
146146
--out_dir xview_dataset/train_images_rgb/
147147
```
148148

149-
## 2. How to convert labels to coco format
149+
## 2. How to convert labels to COCO format
150150

151-
Here we run a script to convert the dataset labels from .geojson format to COCO format. [Read more details about the COCO format at this link.](https://www.immersivelimit.com/tutorials/create-coco-annotations-from-scratch)
151+
Run a script to convert the dataset labels from .geojson format to COCO format. [Read more details about the COCO format at this link.](https://www.immersivelimit.com/tutorials/create-coco-annotations-from-scratch)
152152

153153
The result will be two files (in COCO formal) generated `train.json` and `val.json`
154154

@@ -165,7 +165,7 @@ The result will be two files (in COCO formal) generated `train.json` and `val.js
165165

166166
## 3. Slicing/Tiling the Dataset
167167

168-
Here we are using the SAHI library to slice our large satellite images. Satellite images can be up to 50k^2 pixels in size, which wouldn't fit in GPU memory. We alleviate this problem by slicing the image.
168+
Here, you will be using the SAHI library to slice our large satellite images. Satellite images can be up to 50k^2 pixels in size, which wouldn't fit in GPU memory. You can alleviate this problem by slicing the image.
169169

170170
```python
171171
!python data_utils/slice_coco.py --image_dir xview_dataset/train_images_rgb/ \
@@ -179,9 +179,9 @@ Here we are using the SAHI library to slice our large satellite images. Satellit
179179
--output_val_dir xview_dataset/val_images_rgb_no_neg/
180180
```
181181

182-
## 4. Upload to s3 bucket to support distributed training
182+
## 4. Upload to S3 bucket to support distributed training
183183

184-
We will now upload our exported data to a publically accessible S3 bucket. This will enable for a large scale distributed experiment to have access to the dataset without installing the dataset on device.
184+
We will now upload our exported data to a publically accessible AWS S3 bucket. This will enable for a large scale distributed experiment to have access to the dataset without installing the dataset on device.
185185
View [Determined Documentation](<* https://docs.determined.ai/latest/training/load-model-data.html#streaming-from-object-storage>) and [AWS instructions](<* https://codingsight.com/upload-files-to-aws-s3-with-the-aws-cli/>) to learn how to upload your dataset to an S3 bucket. Review the `S3Backend` class in `data.py`
186186

187187
Once you create an S3 bucket that is publically accessible, here are example commands to upload the preprocessed dataset to S3:
@@ -191,7 +191,7 @@ Once you create an S3 bucket that is publically accessible, here are example com
191191

192192
Our satellite imagery data is in an S3 bucket and is prepped for distributed training, so now we can progress to model training and inference via the NGC Container.
193193

194-
# Part 3: End-to-End Example training object detection model using NVIDIA Pytorch Container from NGC
194+
# Part 3: End-to-End example training object detection model using NVIDIA PyTorch Container from NGC
195195

196196
## Training and Inference via NGC Container
197197

0 commit comments

Comments
 (0)