You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<a href="#Quick-Start-with-Docker">Quick-Start with Docker</a> •
22
+
<a href="#Quick-Start">Quick Start</a> •
22
23
<a href="#Performance">Performance</a> •
23
24
<a href="#Accuracy">Accuracy</a> •
24
25
<a href="#License">License</a>
25
26
</p>
26
27
27
28
# HyperPose
28
29
29
-
HyperPose is a library for building high-performance custom pose estimation systems.
30
+
HyperPose is a library for building high-performance custom pose estimation applications.
30
31
31
32
## Features
32
33
33
34
HyperPose has two key features:
34
35
35
-
-**High-performance pose estimation with CPUs/GPUs**: HyperPose achieves real-time pose estimation through a high-performance pose estimation engine. This engine implements numerous system optimisations: pipeline parallelism, model inference with TensorRT, CPU/GPU hybrid scheduling, and many others. These optimisations contribute to up to 10x higher FPS compared to OpenPose and TF-Pose.
36
+
-**High-performance pose estimation with CPUs/GPUs**: HyperPose achieves real-time pose estimation through a high-performance pose estimation engine. This engine implements numerous system optimisations: pipeline parallelism, model inference with TensorRT, CPU/GPU hybrid scheduling, and many others. These optimisations contribute to up to 10x higher FPS compared to OpenPose, TF-Pose and OpenPifPaf.
36
37
-**Flexibility for developing custom pose estimation models**: HyperPose provides high-level Python APIs to develop pose estimation models. HyperPose users can:
37
38
* Customise training, evaluation, visualisation, pre-processing and post-processing in pose estimation.
38
39
* Customise model architectures (e.g., OpenPose, Pifpaf, PoseProposal Network) and training datasets.
@@ -59,18 +60,17 @@ The HyperPose library contains two parts:
59
60
60
61
The easiest way to use the inference library is through a [Docker image](https://hub.docker.com/r/tensorlayer/hyperpose). Pre-requisites for this image:
Once pre-requisites are ready, pull the HyperPose docker:
74
74
75
75
```bash
76
76
docker pull tensorlayer/hyperpose
@@ -102,16 +102,19 @@ xhost +; docker run --rm --gpus all -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/t
102
102
# docker run --rm --gpus all -it --entrypoint /bin/bash tensorlayer/hyperpose
103
103
```
104
104
105
-
More information of the Docker image is[here](https://hyperpose.readthedocs.io/en/latest/markdown/quick_start/prediction.html#table-of-flags-for-hyperpose-cli).
105
+
For more usage regarding the command line flags, please visit[here](https://hyperpose.readthedocs.io/en/latest/markdown/quick_start/prediction.html#table-of-flags-for-hyperpose-cli).
106
106
107
107
### Python training library
108
108
109
-
We recommend to use the Python training library within an [Anaconda](https://www.anaconda.com/products/individual) environment. The below quick-start has been tested with these environments:<br>
110
-
* Ubuntu 18.04, Tesla V100-DGX, NVIDIA Driver 440.33.01, CUDA 10.2
111
-
* Ubuntu 18.04, Tesla V100-DGX, NVIDIA Driver 410.79, CUDA 10.0
112
-
* Ubuntu 18.04, TITAN RTX, NVIDIA Driver 430.64, CUDA 10.1
113
-
* Ubuntu 18.04, TITAN XP, NVIDIA Driver 430.26, CUDA 10.2
114
-
* Ubuntu 16.04, RTX 2080Ti, NVIDIA Driver 430.50, CUDA 10.1
109
+
We recommend using the Python training library within an [Anaconda](https://www.anaconda.com/products/individual) environment. The below quick-start has been tested with these environments:
We then install the dependencies listed in [requirements.txt](https://github.com/tensorlayer/hyperpose/blob/master/requirements.txt):
131
+
We then clone the repository and install the dependencies listed in [requirements.txt](https://github.com/tensorlayer/hyperpose/blob/master/requirements.txt):
We demonstrate how to train a custom pose estimation model with HyperPose. HyperPose APIs contain three key modules: *Config*, *Model* and *Dataset*, and their basic usages are shown below.
135
139
136
140
```python
137
-
import tensorflow as tf
138
-
import tensorlayer as tl
139
-
tf.test.is_gpu_available()
140
-
141
141
from hyperpose import Config, Model, Dataset
142
+
142
143
# Set model name to distinguish models (necessary)
The full training program is [here](https://github.com/tensorlayer/hyperpose/blob/master/train.py). To evaluate the trained model, you can use an evaluation program [here](https://github.com/tensorlayer/hyperpose/blob/master/eval.py). More information about the training library is[here](https://hyperpose.readthedocs.io/en/latest/markdown/quick_start/training.html).
162
+
The full training program is listed [here](https://github.com/tensorlayer/hyperpose/blob/master/train.py). To evaluate the trained model, you can use the evaluation program [here](https://github.com/tensorlayer/hyperpose/blob/master/eval.py). More information about the training library can be found[here](https://hyperpose.readthedocs.io/en/latest/markdown/quick_start/training.html).
159
163
160
164
161
165
## Documentation
162
166
163
-
The APIs of the HyperPose training library and the inference library are described in [Documentation](https://hyperpose.readthedocs.io/en/latest/).
167
+
The APIs of the HyperPose training library and the inference library are described in the [Documentation](https://hyperpose.readthedocs.io/en/latest/).
164
168
165
169
## Performance
166
170
167
-
We compare the prediction performance of HyperPose with [OpenPose 1.6](https://github.com/CMU-Perceptual-Computing-Lab/openpose) and [TF-Pose](https://github.com/ildoonet/tf-pose-estimation). We implement the OpenPose algorithms with different configurations in HyperPose. The test-bed has Ubuntu18.04, 1070Ti GPU, Intel i7 CPU (12 logic cores).
171
+
We compare the prediction performance of HyperPose with [OpenPose 1.6](https://github.com/CMU-Perceptual-Computing-Lab/openpose), [TF-Pose](https://github.com/ildoonet/tf-pose-estimation) and [OpenPifPaf 0.12](https://github.com/openpifpaf/openpifpaf). The test-bed has Ubuntu18.04, 1070Ti GPU, Intel i7 CPU (12 logic cores).
[OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose) is the state-of-the-art hyperpose estimation algorithm.
206
-
In its Caffe [codebase](https://github.com/ZheC/Realtime_Multi-Person_Pose_Estimation),
207
-
data augmentation, training, and neural networks are most hard-coded. They are difficult
208
-
to be customized. In addition,
209
-
key performance features such as embedded platform supports and parallel GPU training are missing.
210
-
All these limitations makes OpenPose, in these days, hard to
211
-
be deployed in the wild. To resolve this, we develop **OpenPose-Plus**, a high-performance yet flexible hyperpose estimation framework that offers many powerful features:
212
-
213
-
- Flexible combination of standard training dataset with your own custom labelled data.
214
-
- Customizable data augmentation pipeline without compromising performance
215
-
- Deployment on embedded platforms using TensorRT
216
-
- Switchable neural networks (e.g., changing VGG to MobileNet for minimal memory consumption)
217
-
- High-performance training using multiple GPUs
218
-
219
-
## Custom Model Training
220
-
221
-
Training the model is implemented using TensorFlow. To run `train.py`, you would need to install packages, shown
222
-
in [requirements.txt](https://github.com/tensorlayer/openpose-plus/blob/master/requirements.txt), in your virtual environment (**Python 3**):
223
-
224
-
```bash
225
-
pip3 install -r requirements.txt
226
-
pip3 install pycocotools
227
-
```
228
-
229
-
`train.py` automatically download MSCOCO 2017 dataset into `dataset/coco17`.
230
-
The default model is VGG19 used in the OpenPose paper.
231
-
To customize the model, simply changing it in `models.py`.
232
-
233
-
You can use `train_config.py` to configure the training. `config.DATA.train_data` can be:
234
-
* `coco`: training data is COCO dataset only (default)
235
-
* `custom`: training data is your dataset specified by `config.DATA.your_xxx`
236
-
* `coco_and_custom`: training data is COCO and your dataset
237
-
238
-
`config.MODEL.name` can be:
239
-
* `vgg`: VGG19 version (default), slow
240
-
* `vggtiny`: VGG tiny version, faster
241
-
* `mobilenet`: MobileNet version, faster
242
-
243
-
Train your model by running:
244
-
245
-
```bash
246
-
python3 train.py
247
-
```
248
-
249
-
### Additional steps for training on Windows
250
-
251
-
There are a few extra steps to follow with Windows. Please make sure you have the following prerequisites installed:
252
-
* [git](https://git-scm.com/downloads)
253
-
* [Visual C++ Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/)
254
-
* [wget](https://eternallybored.org/misc/wget/)
255
-
256
-
Download the wget executable and copy it into one of your folders in System path to use the wget command from anywhere. Use the `path` command in command line to find the folders. Paste the wget.exe in one of the folders given by `path`. An example folder is `C:\Windows`.
257
-
258
-
pycocotools is not supported by default on Windows. Use the pycocotools build for Windows at [here](https://github.com/philferriere/cocoapi). Instead of `pip install pycocotools`, using:
Real-time inference on resource-constrained embedded platforms is always challenging. To resolve this, we provide a TensorRT-compatible inference engine.
295
-
The engine has two C++ APIs, both defined in `include/openpose-plus.hpp`.
296
-
They are for running the TensorFlow model with TensorRT and post-processing respectively.
297
-
298
-
For details of inference(dependencies/quick start), please refer to [**cpp-inference**](doc/markdown-doc/cpp-inference.md).
299
-
300
-
We are improving the performance of the engine.
301
-
Initial benchmark results for running the full OpenPose model are as follows.
302
-
On Jetson TX2, the inference speed is 13 frames / second (the mobilenet variant is even faster).
303
-
On Jetson TX1, the speed is 10 frames / second. On Titan 1050, the speed is 38 frames / second.
304
-
305
-
After our first optimization, we achieved 50FPS(float32) on 1070Ti.
306
-
307
-
We also have a Python binding for the engine. The current binding relies on
308
-
the external tf-hyperpose-estimation project. We are working on providing the Python binding for our high-performance
309
-
C++ implementation. For now, to enable the binding, please build C++ library for post processing by:
310
-
311
-
```bash
312
-
./scripts/install-pafprocess.sh
313
-
# swig is required. Run `conda install -c anaconda swig` to install swig.
314
-
```
315
-
316
-
See [tf-hyperpose](https://github.com/ildoonet/tf-hyperpose-estimation/tree/master/tf_pose/pafprocess) for details.
317
-
318
-
## Live Camera Example
319
-
320
-
You can look at the examples in the `examples` folder to see how to use the inference C++ APIs.
321
-
Running `./scripts/live-camera.sh` will give you a quick review of how it works.
322
-
323
-
## License
324
-
325
-
You can use the project code under a free [Apache 2.0 license](https://github.com/tensorlayer/tensorlayer/blob/master/LICENSE.rst) ONLY IF you:
326
-
- Cite the [TensorLayer paper](https://github.com/tensorlayer/tensorlayer#cite) and this project in your research article if you are an **academic user**.
327
-
- Acknowledge TensorLayer and this project in your project websites/articles if you are a **commercial user**.
- Post-Processing: Part Association Field(PAF), Pose Proposal Networks, PifPaf.
13
13
14
14
-**Fast Prediction**
15
15
- Rich operator APIs for you to do fast DNN inference and post-processing.
16
16
- 2 API styles:
17
-
- Operator API(Imperative): HyperPose provides basic operators to do DNN inference and post processing.
18
-
- Stream API(Declarative): HyperPose provides a streaming processing runtime scheduler where users only need to specify the engine, post-processing methods and input/output streams.
17
+
- Operator API(Imperative): HyperPose provides basic operators to do DNN inference and post processing.
18
+
- Stream API(Declarative): HyperPose provides a streaming processing runtime scheduler where users only need to specify the engine, post-processing methods and input/output streams.
19
19
- Model format supports:
20
20
- Uff.
21
21
- ONNX.
22
22
- Cuda Engine Protobuf.
23
23
- Good performance. (see [here](../performance/prediction.md))
0 commit comments