Skip to content

Commit fe1c432

Browse files
committed
update readme for portrait image matting commercial solution
1 parent a1ea72f commit fe1c432

File tree

3 files changed

+90
-39
lines changed

3 files changed

+90
-39
lines changed

README.md

Lines changed: 90 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,56 +1,100 @@
1-
<h2 align="center">MODNet: Is a Green Screen Really Necessary for Real-Time Portrait Matting?</h2>
1+
<h2 align="center">MODNet: Trimap-Free Portrait Matting in Real Time</h2>
22

33
<img src="doc/gif/homepage_demo.gif" width="100%">
44

5+
<div align="center">MODNet is a model for <b>real-time</b> portrait matting with <b>only RGB image input</b>.</div>
6+
<div align="center">MODNet是一个<b>仅需RGB图片输入</b>的<b>实时</b>人像抠图模型。</div>
7+
8+
<br />
9+
510
<p align="center">
11+
<a href="#commercial-solution-商用方案">Commercial Solution (商用方案)</a> |
12+
<a href="#research-demo">Research Demo</a> |
613
<a href="https://arxiv.org/pdf/2011.11961.pdf">Arxiv Preprint</a> |
714
<a href="https://youtu.be/PqJ3BRHX3Lc">Supplementary Video</a>
815
</p>
916

1017
<p align="center">
11-
WebCam Video Demo [<a href="demo/video_matting/webcam">Offline</a>][<a href="https://colab.research.google.com/drive/1Pt3KDSc2q7WxFvekCnCLD8P0gBEbxm6J?usp=sharing">Colab</a>] | Custom Video Demo [<a href="demo/video_matting/custom">Offline</a>] |
12-
Image Demo [<a href="https://www.gradio.app/hub/aliabd/modnet">WebGUI</a>][<a href="https://colab.research.google.com/drive/1GANpbKT06aEFiW-Ssx0DQnnEADcXwQG6?usp=sharing">Colab</a>]
18+
<a href="#community">Community</a> |
19+
<a href="#code">Code</a> |
20+
<a href="#ppm-benchmark">PPM Benchmark</a> |
21+
<a href="#license">License</a> |
22+
<a href="#acknowledgement">Acknowledgement</a> |
23+
<a href="#citation">Citation</a> |
24+
<a href="#contact">Contact</a>
1325
</p>
1426

15-
<div align="center">This is the official project of our paper <b>Is a Green Screen Really Necessary for Real-Time Portrait Matting?</b></div>
16-
<div align="center">MODNet is a <b>trimap-free</b> model for portrait matting in <b>real time</b> under <b>changing scenes</b>.</div>
17-
1827

1928
---
2029

2130

22-
## News
23-
- [Mar 12 2021] Support [TorchScript version](torchscript) of MODNet (from the community).
24-
- [Feb 19 2021] Support [ONNX version](onnx) of MODNet (from the community).
25-
- [Jan&nbsp; 28 2021] Release the [code](src/trainer.py) of MODNet training iteration.
26-
- [Dec 25 2020] ***Merry Christmas!*** :christmas_tree: Release Custom Video Matting Demo [[Offline](demo/video_matting/custom)] for user videos.
27-
- [Dec 10 2020] Release WebCam Video Matting Demo [[Offline](demo/video_matting/webcam)][[Colab](https://colab.research.google.com/drive/1Pt3KDSc2q7WxFvekCnCLD8P0gBEbxm6J?usp=sharing)] and Image Matting Demo [[Colab](https://colab.research.google.com/drive/1GANpbKT06aEFiW-Ssx0DQnnEADcXwQG6?usp=sharing)].
28-
- [Nov 24 2020] Release [Arxiv Preprint](https://arxiv.org/pdf/2011.11961.pdf) and [Supplementary Video](https://youtu.be/PqJ3BRHX3Lc).
31+
## Commercial Solution (商用方案)
2932

33+
Our commercial solution for portrait matting is coming!
34+
我们的人像抠图商用方案来了!
3035

31-
## Demos
36+
### Portrait Image Matting Solution (图片抠像方案)
37+
38+
A **Single** Model! Only **7M**! Process **2K** resolution image with a **Fast** speed on common PCs or Mobiles!
39+
**单个**模型!大小仅为**7M**!可以在普通PC或移动设备上**快速**处理具有**2K**分辨率的图像!
40+
41+
<!-- Here are two example videos processed (frame independently) via our **portrait image matting** model:
42+
我们**图片抠像**模型逐帧单独处理的两个示例视频:
43+
44+
<img src="doc/gif/commercial_image_matting_model_result.gif" width='100%'>
45+
46+
<br>
47+
<br> -->
48+
49+
Now you can try the Beta version of our **portrait image matting** online via [this website](https://sight-x.cn/portrait_matting).
50+
(As our current server for testing is hosted in China, your access may be delayed.)
51+
现在,您可以通过[此网站](https://sight-x.cn/portrait_matting)在线使用我们的**图片抠像**测试版。
52+
53+
<img src="doc/gif/commercial_image_matting_website.gif" width='100%'>
54+
55+
<!-- You can also scan the QR code below to try the WeChat Mini-Program based on our model.
56+
您也可以扫描下方二维码尝试基于我们模型的微信小程序。 -->
57+
58+
The commercial API interface for **portrait image matting** will be available soon.
59+
用于**图片抠像**的商用API接口即将推出。
60+
61+
If you are interest in a **portrait image matting SDK**, please contact `[email protected]`.
62+
如果您对**图片抠像SDK**感兴趣,请联系`[email protected]`
3263

33-
### Video Matting
34-
We provide two real-time portrait video matting demos based on WebCam. When using the demo, you can move the WebCam around at will.
35-
If you have an Ubuntu system, we recommend you to try the [offline demo](demo/video_matting/webcam) to get a higher *fps*. Otherwise, you can access the [online Colab demo](https://colab.research.google.com/drive/1Pt3KDSc2q7WxFvekCnCLD8P0gBEbxm6J?usp=sharing).
36-
We also provide an [offline demo](demo/video_matting/custom) that allows you to process custom videos.
3764

38-
<img src="doc/gif/video_matting_demo.gif" width='60%'>
65+
### Portrait Video Matting Solution (视频抠像方案)
3966

67+
Stay tuned.
68+
敬请期待。
4069

41-
### Image Matting
70+
71+
---
72+
73+
74+
## Research Demo
75+
76+
All the models behind the following demos are trained on the datasets mentioned in [our paper](https://arxiv.org/pdf/2011.11961.pdf).
77+
78+
### Portrait Image Matting
4279
We provide an [online Colab demo](https://colab.research.google.com/drive/1GANpbKT06aEFiW-Ssx0DQnnEADcXwQG6?usp=sharing) for portrait image matting.
4380
It allows you to upload portrait images and predict/visualize/download the alpha mattes.
4481

45-
<img src="doc/gif/image_matting_demo.gif" width='40%'>
82+
<!-- <img src="doc/gif/image_matting_demo.gif" width='40%'> -->
4683

84+
### Portrait Video Matting
85+
We provide two real-time portrait video matting demos based on WebCam. When using the demo, you can move the WebCam around at will.
86+
If you have an Ubuntu system, we recommend you to try the [offline demo](demo/video_matting/webcam) to get a higher *fps*. Otherwise, you can access the [online Colab demo](https://colab.research.google.com/drive/1Pt3KDSc2q7WxFvekCnCLD8P0gBEbxm6J?usp=sharing).
87+
We also provide an [offline demo](demo/video_matting/custom) that allows you to process custom videos.
88+
89+
<!-- <img src="doc/gif/video_matting_demo.gif" width='60%'> -->
90+
91+
92+
## Community
4793

48-
### Community
49-
Here we share some cool applications/extentions of MODNet built by the community.
94+
We share some cool applications/extentions of MODNet built by the community.
5095

51-
- **WebGUI for Image Matting**
52-
You can try [this WebGUI](https://www.gradio.app/hub/aliabd/modnet) (hosted on [Gradio](https://www.gradio.app/)) for portrait matting from your browser without code!
53-
<!-- <img src="https://i.ibb.co/9gLxFXF/modnet.gif" width='40%'> -->
96+
- **WebGUI for Portrait Image Matting**
97+
You can try [this WebGUI](https://www.gradio.app/hub/aliabd/modnet) (hosted on [Gradio](https://www.gradio.app/)) for portrait image matting from your browser without code!
5498

5599
- **Colab Demo of Bokeh (Blur Background)**
56100
You can try [this Colab demo](https://colab.research.google.com/github/eyaler/avatars4all/blob/master/yarok.ipynb) (built by [@eyaler](https://github.com/eyaler)) to blur the backgroud based on MODNet!
@@ -61,31 +105,37 @@ You can convert the pre-trained MODNet to an ONNX model by using [this code](onn
61105
- **TorchScript Version of MODNet**
62106
You can convert the pre-trained MODNet to an TorchScript model by using [this code](torchscript) (provided by [@yarkable](https://github.com/yarkable)).
63107

108+
- **TensorRT Version of MODNet**
109+
You can access [this Github repository](https://github.com/jkjung-avt/tensorrt_demos) to try the TensorRT version of MODNet (provided by [@jkjung-avt](https://github.com/jkjung-avt)).
110+
111+
112+
There are some resources about MODNet from the community.
113+
- [Video from What's AI YouTube Channel](https://youtu.be/rUo0wuVyefU)
114+
- [Article from Louis Bouchard's Blog](https://www.louisbouchard.ai/remove-background/)
115+
64116

65117
## Code
66118
We provide the [code](src/trainer.py) of MODNet training iteration, including:
67119
- **Supervised Training**: Train MODNet on a labeled matting dataset
68120
- **SOC Adaptation**: Adapt a trained MODNet to an unlabeled dataset
69121

70-
In the function comments, we provide examples of how to call the function.
122+
In the code comments, we provide examples for using the functions.
71123

72124

73-
## TODO
74-
- Release the code of One-Frame Delay
75-
- Release PPM-100 validation benchmark (**Delayed, But On The Way...**)
76-
**NOTE**: PPM-100 is a **validation set**. Our training set will not be published.
125+
## PPM Benchmark
126+
The PPM benchmark will be released in a separate repository [PPM](https://github.com/ZHKKKe/PPM).
77127

78128

79129
## License
80-
This project (**code, pre-trained models, demos, *etc.***) is released under the [Creative Commons Attribution NonCommercial ShareAlike 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode) license.
130+
All resources in this repository (code, models, demos, *etc.*) are released under the [Creative Commons Attribution NonCommercial ShareAlike 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode) license.
131+
The license will be changed to allow commercial use after our paper is accepted.
81132

82-
**NOTE**: The license will be changed to allow commercial use after this work is accepted by a conference or a journal.
83133

84-
## Acknowledgement
85-
- We thank [City University of Hong Kong](https://www.cityu.edu.hk/) and [SenseTime](https://www.sensetime.com/) for their support to this project.
134+
## Acknowledgement
86135
- We thank
87-
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[the Gradio team](https://github.com/gradio-app/gradio), [@eyaler](https://github.com/eyaler), [@manthan3C273](https://github.com/manthan3C273), [@yarkable](https://github.com/yarkable),
88-
for their contributions to this repository or their cool applications based on MODNet.
136+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[@eyaler](https://github.com/eyaler), [@manthan3C273](https://github.com/manthan3C273), [@yarkable](https://github.com/yarkable), [@jkjung-avt](https://github.com/jkjung-avt),
137+
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[the Gradio team](https://github.com/gradio-app/gradio), [What's AI YouTube Channel](https://www.youtube.com/channel/UCUzGQrN-lyyc0BWTYoJM_Sg), [Louis Bouchard's Blog](https://www.louisbouchard.ai),
138+
for their contributions to this repository or their cool applications/extentions/resources of MODNet.
89139

90140

91141
## Citation
@@ -103,5 +153,6 @@ If this work helps your research, please consider to cite:
103153

104154

105155
## Contact
106-
This project is currently maintained by Zhanghan Ke ([@ZHKKKe](https://github.com/ZHKKKe)).
107-
If you have any questions, please feel free to contact `[email protected]`.
156+
This repository is currently maintained by Zhanghan Ke ([@ZHKKKe](https://github.com/ZHKKKe)).
157+
For commercial questions, please contact `[email protected]`.
158+
For research questions, please contact `[email protected]`.
11.1 MB
Loading
1.54 MB
Loading

0 commit comments

Comments
 (0)