Skip to content

Commit e9f2f1a

Browse files
DefTruthDefTruth
authored
Merge pull request #6 from DefTruth/dev
fix(autodtypes): fixed un-convert bug in autodtype wrapper
2 parents d05fd90 + 439825d commit e9f2f1a

27 files changed

+685
-104
lines changed

‎.gitignore‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
.idea
22
__pycache__
33
.DS_Store
4-
logs
5-
debug.py
4+
debug.py
5+
build
6+
dist

‎README.md‎

Lines changed: 215 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,216 @@
11
![torchlm-logo](docs/res/logo.png)
2+
3+
<div align='center'>
4+
<img src=https://img.shields.io/badge/PRs-welcome-9cf.svg >
5+
<img src=https://img.shields.io/badge/slack-chat-ffa.svg?logo=slack >
6+
<img src=https://img.shields.io/pypi/v/torchlm?color=aff >
7+
<img src=https://img.shields.io/pypi/pyversions/torchlm?color=dfd >
8+
<img src=https://img.shields.io/badge/macos|linux|windows-pass-skyblue.svg >
9+
<img src=https://img.shields.io/badge/license-MIT-lightblue.svg >
10+
</div>
11+
12+
13+
## 🤗 Introduction
14+
**torchlm** is a PyTorch landmarks-only library with **100+ data augmentations**, **training** and **inference**. **torchlm** is only focus on any landmarks detection, such as face landmarks, hand keypoints and body keypoints, etc. It provides **30+** native data augmentations and compatible with **80+** torchvision and albumations's transforms, no matter the input is a np.ndarray or a torch Tensor, **torchlm** will **automatically** be compatible with different data types through a **autodtype** wrapper. Further, in the future **torchlm** will add modules for **training** and **inference**.
15+
16+
# 🆕 What's New
17+
18+
* [2022/02/13]: Add **30+** native data augmentations and **bind** 80+ torchvision and albumations's transforms.
19+
20+
## 🛠️ Usage
21+
22+
### Requirements
23+
* opencv-python-headless>=4.5.2
24+
* numpy>=1.14.4
25+
* torch>=1.6.0
26+
* torchvision>=0.9.0
27+
* albumentations>=1.1.0
28+
29+
### Installation
30+
you can install **torchlm** directly from pip.
31+
```shell
32+
pip3 install torchlm
33+
# install from specific pypi mirrors use '-i'
34+
pip3 install torchlm -i https://pypi.org/simple/
35+
```
36+
37+
### Data Augmentation
38+
**torchlm** provides 30+ native data augmentations for landmarks and is compatible with 80+ transforms from torchvision and albumations, no matter the input is a np.ndarray or a torch Tensor, torchlm will automatically be compatible with different data types through a autodtype wrapper.
39+
* use native torchlm's transforms
40+
```python
41+
import torchlm
42+
transform = torchlm.LandmarksCompose([
43+
# use native torchlm transforms
44+
torchlm.LandmarksRandomScale(prob=0.5),
45+
torchlm.LandmarksRandomTranslate(prob=0.5),
46+
torchlm.LandmarksRandomShear(prob=0.5),
47+
torchlm.LandmarksRandomMask(prob=0.5),
48+
torchlm.LandmarksRandomBlur(kernel_range=(5, 25), prob=0.5),
49+
torchlm.LandmarksRandomBrightness(prob=0.),
50+
torchlm.LandmarksRandomRotate(40, prob=0.5, bins=8),
51+
torchlm.LandmarksRandomCenterCrop((0.5, 1.0), (0.5, 1.0), prob=0.5),
52+
torchlm.LandmarksResize((256, 256)),
53+
torchlm.LandmarksNormalize(),
54+
torchlm.LandmarksToTensor(),
55+
torchlm.LandmarksToNumpy(),
56+
torchlm.LandmarksUnNormalize()
57+
])
58+
```
59+
* **bind** torchvision and albumations's transform, using **torchlm.bind**
60+
```python
61+
import torchvision
62+
import albumentations
63+
import torchlm
64+
transform = torchlm.LandmarksCompose([
65+
# use native torchlm transforms
66+
torchlm.LandmarksRandomScale(prob=0.5),
67+
# ...
68+
# bind torchvision image only transforms
69+
torchlm.bind(torchvision.transforms.GaussianBlur(kernel_size=(5, 25))),
70+
torchlm.bind(torchvision.transforms.RandomAutocontrast(p=0.5)),
71+
torchlm.bind(torchvision.transforms.RandomAdjustSharpness(sharpness_factor=3, p=0.5)),
72+
# bind albumentations image only transforms
73+
torchlm.bind(albumentations.ColorJitter(p=0.5)),
74+
torchlm.bind(albumentations.GlassBlur(p=0.5)),
75+
torchlm.bind(albumentations.RandomShadow(p=0.5)),
76+
# bind albumentations dual transforms
77+
torchlm.bind(albumentations.RandomCrop(height=200, width=200, p=0.5)),
78+
torchlm.bind(albumentations.RandomScale(p=0.5)),
79+
torchlm.bind(albumentations.Rotate(p=0.5)),
80+
torchlm.LandmarksResize((256, 256)),
81+
torchlm.LandmarksNormalize(),
82+
torchlm.LandmarksToTensor(),
83+
torchlm.LandmarksToNumpy(),
84+
torchlm.LandmarksUnNormalize()
85+
])
86+
```
87+
* **bind** custom callable array or Tensor functions, using **torchlm.bind**
88+
89+
```python
90+
# First, defined your custom functions
91+
def callable_array_noop(
92+
img: np.ndarray,
93+
landmarks: np.ndarray
94+
) -> Tuple[np.ndarray, np.ndarray]:
95+
# Do some transform here ...
96+
return img.astype(np.uint32), landmarks.astype(np.float32)
97+
98+
99+
def callable_tensor_noop(
100+
img: Tensor,
101+
landmarks: Tensor
102+
) -> Tuple[Tensor, Tensor]:
103+
# Do some transform here ...
104+
return img, landmarks
105+
```
106+
107+
```python
108+
# Then, bind your functions and put it into transforms pipeline.
109+
transform = torchlm.LandmarksCompose([
110+
# use native torchlm transforms
111+
torchlm.LandmarksRandomScale(prob=0.5),
112+
# ...
113+
# bind torchvision image only transforms
114+
torchlm.bind(torchvision.transforms.GaussianBlur(kernel_size=(5, 25))),
115+
torchlm.bind(torchvision.transforms.RandomAutocontrast(p=0.5)),
116+
torchlm.bind(torchvision.transforms.RandomAdjustSharpness(sharpness_factor=3, p=0.5)),
117+
# bind albumentations image only transforms
118+
torchlm.bind(albumentations.ColorJitter(p=0.5)),
119+
torchlm.bind(albumentations.GlassBlur(p=0.5)),
120+
torchlm.bind(albumentations.RandomShadow(p=0.5)),
121+
# bind albumentations dual transforms
122+
torchlm.bind(albumentations.RandomCrop(height=200, width=200, p=0.5)),
123+
torchlm.bind(albumentations.RandomScale(p=0.5)),
124+
torchlm.bind(albumentations.Rotate(p=0.5)),
125+
# bind custom callable array functions
126+
torchlm.bind(callable_array_noop, bind_type=torchlm.BindEnum.Callable_Array),
127+
# bind custom callable Tensor functions
128+
torchlm.bind(callable_tensor_noop, bind_type=torchlm.BindEnum.Callable_Tensor),
129+
torchlm.LandmarksResize((256, 256)),
130+
torchlm.LandmarksNormalize(),
131+
torchlm.LandmarksToTensor(),
132+
torchlm.LandmarksToNumpy(),
133+
torchlm.LandmarksUnNormalize()
134+
])
135+
```
136+
<div align='center'>
137+
<img src='docs/res/124.jpg' height="100px" width="100px">
138+
<img src='docs/res/158.jpg' height="100px" width="100px">
139+
<img src='docs/res/386.jpg' height="100px" width="100px">
140+
<img src='docs/res/478.jpg' height="100px" width="100px">
141+
<img src='docs/res/537.jpg' height="100px" width="100px">
142+
<img src='docs/res/605.jpg' height="100px" width="100px">
143+
<img src='docs/res/802.jpg' height="100px" width="100px">
144+
</div>
145+
146+
147+
* setup logging mode as `True` globally might help you figure out the runtime details
148+
```python
149+
import torchlm
150+
# some global setting
151+
torchlm.set_transforms_debug(True)
152+
torchlm.set_transforms_logging(True)
153+
torchlm.set_autodtype_logging(True)
154+
```
155+
Some details logs will show you at each runtime, just like the follows
156+
```shell
157+
LandmarksRandomHorizontalFlip() AutoDtype Info: AutoDtypeEnum.Array_InOut
158+
LandmarksRandomHorizontalFlip() Execution Flag: True
159+
LandmarksRandomScale() AutoDtype Info: AutoDtypeEnum.Array_InOut
160+
LandmarksRandomScale() Execution Flag: False
161+
...
162+
BindTorchVisionTransform(GaussianBlur())() AutoDtype Info: AutoDtypeEnum.Tensor_InOut
163+
BindTorchVisionTransform(GaussianBlur())() Execution Flag: True
164+
...
165+
BindAlbumentationsTransform(ColorJitter())() AutoDtype Info: AutoDtypeEnum.Array_InOut
166+
BindAlbumentationsTransform(ColorJitter())() Execution Flag: True
167+
...
168+
BindArrayCallable(callable_array_noop())() AutoDtype Info: AutoDtypeEnum.Array_InOut
169+
BindArrayCallable(callable_array_noop())() Execution Flag: True
170+
BindTensorCallable(callable_tensor_noop())() AutoDtype Info: AutoDtypeEnum.Tensor_InOut
171+
BindTensorCallable(callable_tensor_noop())() Execution Flag: True
172+
...
173+
LandmarksUnNormalize() AutoDtype Info: AutoDtypeEnum.Array_InOut
174+
LandmarksUnNormalize() Execution Flag: True
175+
```
176+
* Execution Flag: True means current transform was executed successful, False means it was not executed because of the random probability or some Runtime Exceptions(torchlm will should the error infos if debug mode is True).
177+
* AutoDtype Info:
178+
* Array_InOut means current transform need a np.ndnarray as input and then output a np.ndarray.
179+
* Tensor_InOut means current transform need a torch Tensor as input and then output torch Tensor.
180+
* Array_In means current transform needs a np.ndarray input and then output a torch Tensor.
181+
* Tensor_In means current transform needs a torch Tensor input and then output a np.ndarray.
182+
183+
But, is ok if your pass a Tensor to a np.ndarray like transform, **torchlm** will automatically be compatible with different data types and then wrap back to the original type through a autodtype wrapper.
184+
185+
186+
* Supported Transforms Sets, see [transforms.md](docs/api/transfroms.md). A detail example can be found at [test/transforms.py](test/transforms.py).
187+
188+
### Training(TODO)
189+
* [ ] YOLOX
190+
* [ ] YOLOv5
191+
* [ ] NanoDet
192+
* [ ] PIPNet
193+
* [ ] ResNet
194+
* [ ] MobileNet
195+
* [ ] ShuffleNet
196+
* [ ] ...
197+
198+
### Inference(TODO)
199+
* [ ] ONNXRuntime
200+
* [ ] MNN
201+
* [ ] NCNN
202+
* [ ] TNN
203+
* [ ] ...
204+
205+
## 📖 Documentations
206+
* [ ] Native Data Augmentation's API (TODO)
207+
* [ ] ...
208+
209+
## 🎓 License
210+
The code of torchlm is released under the MIT License.
211+
212+
## 👋 Contributing
213+
If you like this project please consider ⭐ this repo, as it is the simplest way to support me.
214+
215+
## 🎓 Acknowledgement
216+
The implementation of torchlm's transforms borrow the code from [Paperspace](ttps://github.com/Paperspace/DataAugmentationForObjectDetection/blob/master/data_aug/bbox_util.py).

‎docs/api/.gitignore‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
11
.idea
22
__pycache__
33
.DS_Store
4-
logs

‎docs/api/transfroms.md‎

Lines changed: 139 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,139 @@
1+
## Supported Transforms Set
2+
3+
### native torchlm's transforms
4+
5+
```python
6+
__all__ = [
7+
"LandmarksCompose",
8+
"LandmarksNormalize",
9+
"LandmarksUnNormalize",
10+
"LandmarksToTensor",
11+
"LandmarksToNumpy",
12+
"LandmarksResize",
13+
"LandmarksClip",
14+
"LandmarksAlign",
15+
"LandmarksRandomAlign",
16+
"LandmarksRandomCenterCrop",
17+
"LandmarksRandomHorizontalFlip",
18+
"LandmarksHorizontalFlip",
19+
"LandmarksRandomScale",
20+
"LandmarksRandomTranslate",
21+
"LandmarksRandomRotate",
22+
"LandmarksRandomShear",
23+
"LandmarksRandomHSV",
24+
"LandmarksRandomMask",
25+
"LandmarksRandomBlur",
26+
"LandmarksRandomBrightness",
27+
"LandmarksRandomPatches",
28+
"LandmarksRandomBackground",
29+
"LandmarksRandomPatchesWithAlpha",
30+
"LandmarksRandomBackgroundWithAlpha",
31+
"LandmarksRandomMaskWithAlpha",
32+
"BindAlbumentationsTransform",
33+
"BindTorchVisionTransform",
34+
"BindArrayCallable",
35+
"BindTensorCallable",
36+
"BindEnum",
37+
"bind",
38+
"set_transforms_logging",
39+
"set_transforms_debug"
40+
]
41+
```
42+
43+
### transforms from torchvision
44+
45+
```python
46+
# torchvision >= 0.9.0
47+
_Supported_Image_Only_Transform_Set: Tuple = (
48+
torchvision.transforms.Normalize,
49+
torchvision.transforms.ColorJitter,
50+
torchvision.transforms.Grayscale,
51+
torchvision.transforms.RandomGrayscale,
52+
torchvision.transforms.RandomErasing,
53+
torchvision.transforms.GaussianBlur,
54+
torchvision.transforms.RandomInvert,
55+
torchvision.transforms.RandomPosterize,
56+
torchvision.transforms.RandomSolarize,
57+
torchvision.transforms.RandomAdjustSharpness,
58+
torchvision.transforms.RandomAutocontrast,
59+
torchvision.transforms.RandomEqualize
60+
)
61+
```
62+
63+
### transforms from albumentations
64+
65+
```python
66+
# albumentations >= v 1.1.0
67+
_Supported_Image_Only_Transform_Set: Tuple = (
68+
albumentations.Blur,
69+
albumentations.CLAHE,
70+
albumentations.ChannelDropout,
71+
albumentations.ChannelShuffle,
72+
albumentations.ColorJitter,
73+
albumentations.Downscale,
74+
albumentations.Emboss,
75+
albumentations.Equalize,
76+
albumentations.FDA,
77+
albumentations.FancyPCA,
78+
albumentations.FromFloat,
79+
albumentations.GaussNoise,
80+
albumentations.GaussianBlur,
81+
albumentations.GlassBlur,
82+
albumentations.HistogramMatching,
83+
albumentations.HueSaturationValue,
84+
albumentations.ISONoise,
85+
albumentations.ImageCompression,
86+
albumentations.InvertImg,
87+
albumentations.MedianBlur,
88+
albumentations.MotionBlur,
89+
albumentations.Normalize,
90+
albumentations.PixelDistributionAdaptation,
91+
albumentations.Posterize,
92+
albumentations.RGBShift,
93+
albumentations.RandomBrightnessContrast,
94+
albumentations.RandomFog,
95+
albumentations.RandomGamma,
96+
albumentations.RandomRain,
97+
albumentations.RandomShadow,
98+
albumentations.RandomSnow,
99+
albumentations.RandomSunFlare,
100+
albumentations.RandomToneCurve,
101+
albumentations.Sharpen,
102+
albumentations.Solarize,
103+
albumentations.Superpixels,
104+
albumentations.TemplateTransform,
105+
albumentations.ToFloat,
106+
albumentations.ToGray
107+
)
108+
109+
_Supported_Dual_Transform_Set: Tuple = (
110+
albumentations.Affine,
111+
albumentations.CenterCrop,
112+
albumentations.CoarseDropout,
113+
albumentations.Crop,
114+
albumentations.CropAndPad,
115+
albumentations.CropNonEmptyMaskIfExists,
116+
albumentations.Flip,
117+
albumentations.HorizontalFlip,
118+
albumentations.Lambda,
119+
albumentations.LongestMaxSize,
120+
albumentations.NoOp,
121+
albumentations.PadIfNeeded,
122+
albumentations.Perspective,
123+
albumentations.PiecewiseAffine,
124+
albumentations.RandomCrop,
125+
albumentations.RandomCropNearBBox,
126+
albumentations.RandomGridShuffle,
127+
albumentations.RandomResizedCrop,
128+
albumentations.RandomRotate90,
129+
albumentations.RandomScale,
130+
albumentations.RandomSizedCrop,
131+
albumentations.Resize,
132+
albumentations.Rotate,
133+
albumentations.SafeRotate,
134+
albumentations.ShiftScaleRotate,
135+
albumentations.SmallestMaxSize,
136+
albumentations.Transpose,
137+
albumentations.VerticalFlip
138+
)
139+
```

‎docs/res/.gitignore‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
.idea
22
__pycache__
33
.DS_Store
4-
logs
4+

‎docs/res/124.jpg‎

16.5 KB
Loading

‎docs/res/158.jpg‎

19.6 KB
Loading

‎docs/res/386.jpg‎

13.1 KB
Loading

‎docs/res/478.jpg‎

14.7 KB
Loading

‎docs/res/537.jpg‎

12.1 KB
Loading

0 commit comments

Comments
 (0)