Skip to content

Commit abfbf90

Browse files
WovchenaWovchena
authored
Merge pull request #2992 from DmitriySidnev/ds/bgr_subtraction
Add background subtraction demo
2 parents d276b46 + 985e681 commit abfbf90

File tree

8 files changed

+791
-0
lines changed

8 files changed

+791
-0
lines changed

‎demos/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ The Open Model Zoo includes the following demos:
1111
- [3D Human Pose Estimation Python\* Demo](./human_pose_estimation_3d_demo/python/README.md) - 3D human pose estimation demo.
1212
- [3D Segmentation Python\* Demo](./3d_segmentation_demo/python/README.md) - Segmentation demo segments 3D images using 3D convolutional networks.
1313
- [Action Recognition Python\* Demo](./action_recognition_demo/python/README.md) - Demo application for Action Recognition algorithm, which classifies actions that are being performed on input video.
14+
- [Background Subtraction Python\* Demo](./background_subtraction_demo/python/README.md) - Background subtraction using instance segmentation based models.
1415
- [BERT Named Entity Recognition Python\* Demo](./bert_named_entity_recognition_demo/python/README.md) - NER Demo application that uses a CONLL2003-tuned BERT model for inference.
1516
- [BERT Question Answering Python\* Demo](./bert_question_answering_demo/python/README.md)
1617
- [BERT Question Answering Embedding Python\* Demo](./bert_question_answering_embedding_demo/python/README.md) - The demo demonstrates how to run BERT based models for question answering task.

‎demos/background_subtraction_demo/python/README.md

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
# Background subtraction Python\* Demo
2+
3+
![example](./background_subtraction.gif)
4+
5+
This demo shows how to perform background subtraction using OpenVINO.
6+
7+
> **NOTE**: Only batch size of 1 is supported.
8+
9+
## How It Works
10+
11+
The demo application expects an instance segmentation model in the Intermediate Representation (IR) format with the following constraints:
12+
1. for instance segmentation models based on `Mask RCNN` approach:
13+
* One input: `image` for input image.
14+
* At least three outputs including:
15+
* `boxes` with absolute bounding box coordinates of the input image and its score
16+
* `labels` with object class IDs for all bounding boxes
17+
* `masks` with fixed-size segmentation heat maps for all classes of all bounding boxes
18+
2. for instance segmentation models based on `YOLACT` approach:
19+
* Single input for input image.
20+
* At least four outputs including:
21+
* `boxes` with normalized in [0, 1] range bounding box coordinates
22+
* `conf` with confidence scores for each class for all boxes
23+
* `mask` with fixed-size mask channels for all boxes.
24+
* `proto` with fixed-size segmentation heat maps prototypes for all boxes.
25+
26+
The use case for the demo is an online conference where is needed to show only foreground - people and, respectively, to hide or replace background.
27+
Based on this an instance segmentation model must be trained at least for person class.
28+
29+
As input, the demo application accepts a path to a single image file, a video file or a numeric ID of a web camera specified with a command-line argument `-i`
30+
31+
The demo workflow is the following:
32+
33+
1. The demo application reads image/video frames one by one, resizes them to fit into the input image blob of the network (`image`).
34+
2. The demo visualizes the resulting background subtraction. Certain command-line options affect the visualization:
35+
* If you specify `--target_bgr`, background will be replaced by a chosen image or video. By default background replaced by green field.
36+
* If you specify `--blur_bgr`, background will be blurred according to a set value. By default equal to zero and is not applied.
37+
* If you specify `--show_with_original_frame`, the result image will be merged with an input one.
38+
39+
> **NOTE**: By default, Open Model Zoo demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the demo application or reconvert your model using the Model Optimizer tool with the `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model Using General Conversion Parameters](https://docs.openvino.ai/latest/openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model.html#general-conversion-parameters).
40+
41+
## Preparing to Run
42+
43+
For demo input image or video files, refer to the section **Media Files Available for Demos** in the [Open Model Zoo Demos Overview](../../README.md).
44+
The list of models supported by the demo is in `<omz_dir>/demos/instance_segmentation_demo/python/models.lst` file.
45+
This file can be used as a parameter for [Model Downloader](../../../tools/model_tools/README.md) and Converter to download and, if necessary, convert models to OpenVINO Inference Engine format (\*.xml + \*.bin).
46+
47+
An example of using the Model Downloader:
48+
49+
```sh
50+
omz_downloader --list models.lst
51+
```
52+
53+
An example of using the Model Converter:
54+
55+
```sh
56+
omz_converter --list models.lst
57+
```
58+
59+
### Supported Models
60+
61+
* instance-segmentation-person-0007
62+
* yolact-resnet50-fpn-pytorch
63+
64+
> **NOTE**: Refer to the tables [Intel's Pre-Trained Models Device Support](../../../models/intel/device_support.md) and [Public Pre-Trained Models Device Support](../../../models/public/device_support.md) for the details on models inference support at different devices.
65+
66+
## Running
67+
68+
Run the application with the `-h` option to see the following usage message:
69+
70+
```
71+
usage: background_subtraction_demo.py [-h] -m MODEL
72+
[--adapter {openvino,remote}] -i INPUT
73+
[-d DEVICE] [-t PROB_THRESHOLD]
74+
[--resize_type {crop,standard,fit_to_window,fit_to_window_letterbox}]
75+
[--labels LABELS]
76+
[--target_bgr TARGET_BGR]
77+
[--blur_bgr BLUR_BGR]
78+
[-nireq NUM_INFER_REQUESTS]
79+
[-nstreams NUM_STREAMS]
80+
[-nthreads NUM_THREADS] [--loop]
81+
[-o OUTPUT] [-limit OUTPUT_LIMIT]
82+
[--no_show] [--show_with_original_frame]
83+
[--output_resolution OUTPUT_RESOLUTION]
84+
[-u UTILIZATION_MONITORS] [-r]
85+
86+
Options:
87+
-h, --help Show this help message and exit.
88+
-m MODEL, --model MODEL
89+
Required. Path to an .xml file with a trained model.
90+
--adapter {openvino,remote}
91+
Optional. Specify the model adapter. Default is
92+
openvino.
93+
-i INPUT, --input INPUT
94+
Required. An input to process. The input must be a
95+
single image, a folder of images, video file or camera
96+
id.
97+
-d DEVICE, --device DEVICE
98+
Optional. Specify the target device to infer on; CPU,
99+
GPU, HDDL or MYRIAD is acceptable. The demo will look
100+
for a suitable plugin for device specified. Default
101+
value is CPU.
102+
-t PROB_THRESHOLD, --prob_threshold PROB_THRESHOLD
103+
Optional. Probability threshold for detections
104+
filtering.
105+
--resize_type {crop,standard,fit_to_window,fit_to_window_letterbox}
106+
Optional. A resize type for model preprocess. By
107+
defauld used model predefined type.
108+
--labels LABELS Optional. Labels mapping file.
109+
--target_bgr TARGET_BGR
110+
Optional. Background onto which to composite the
111+
output (by default to green field).
112+
--blur_bgr BLUR_BGR Optional. Background blur strength (by default with
113+
value 0 is not applied).
114+
115+
Inference options:
116+
-nireq NUM_INFER_REQUESTS, --num_infer_requests NUM_INFER_REQUESTS
117+
Optional. Number of infer requests.
118+
-nstreams NUM_STREAMS, --num_streams NUM_STREAMS
119+
Optional. Number of streams to use for inference on
120+
the CPU or/and GPU in throughput mode (for HETERO and
121+
MULTI device cases use format
122+
<device1>:<nstreams1>,<device2>:<nstreams2> or just
123+
<nstreams>).
124+
-nthreads NUM_THREADS, --num_threads NUM_THREADS
125+
Optional. Number of threads to use for inference on
126+
CPU (including HETERO cases).
127+
128+
Input/output options:
129+
--loop Optional. Enable reading the input in a loop.
130+
-o OUTPUT, --output OUTPUT
131+
Optional. Name of the output file(s) to save.
132+
-limit OUTPUT_LIMIT, --output_limit OUTPUT_LIMIT
133+
Optional. Number of frames to store in output. If 0 is
134+
set, all frames are stored.
135+
--no_show Optional. Don't show output.
136+
--show_with_original_frame
137+
Optional. Merge the result frame with the original
138+
one.
139+
--output_resolution OUTPUT_RESOLUTION
140+
Optional. Specify the maximum output window resolution
141+
in (width x height) format. Example: 1280x720. Input
142+
frame size used by default.
143+
-u UTILIZATION_MONITORS, --utilization_monitors UTILIZATION_MONITORS
144+
Optional. List of monitors to show initially.
145+
146+
Debug options:
147+
-r, --raw_output_message
148+
Optional. Output inference results as mask histogram.
149+
```
150+
151+
Running the application with an empty list of options yields the short version of the usage message and an error message.
152+
153+
To run the demo, please provide paths to the model in the IR format, and to an input video, image, or folder with images:
154+
155+
```bash
156+
python3 background_subtraction_demo/python/background_subtraction_demo.py \
157+
-m <path_to_model>/instance-segmentation-person-0007.xml \
158+
-i 0
159+
```
160+
161+
>**NOTE**: If you provide a single image as an input, the demo processes and renders it quickly, then exits. To continuously visualize inference results on the screen, apply the `loop` option, which enforces processing a single image in a loop.
162+
163+
You can save processed results to a Motion JPEG AVI file or separate JPEG or PNG files using the `-o` option:
164+
165+
* To save processed results in an AVI file, specify the name of the output file with `avi` extension, for example: `-o output.avi`.
166+
* To save processed results as images, specify the template name of the output image file with `jpg` or `png` extension, for example: `-o output_%03d.jpg`. The actual file names are constructed from the template at runtime by replacing regular expression `%03d` with the frame number, resulting in the following: `output_000.jpg`, `output_001.jpg`, and so on.
167+
To avoid disk space overrun in case of continuous input stream, like camera, you can limit the amount of data stored in the output file(s) with the `limit` option. The default value is 1000. To change it, you can apply the `-limit N` option, where `N` is the number of frames to store.
168+
169+
>**NOTE**: Windows\* systems may not have the Motion JPEG codec installed by default. If this is the case, you can download OpenCV FFMPEG back end using the PowerShell script provided with the OpenVINO &trade; install package and located at `<INSTALL_DIR>/opencv/ffmpeg-download.ps1`. The script should be run with administrative privileges if OpenVINO &trade; is installed in a system protected folder (this is a typical case). Alternatively, you can save results as images.
170+
171+
## Demo Output
172+
173+
The application uses OpenCV to display resulting images.
174+
The demo reports
175+
176+
* **FPS**: average rate of video frame processing (frames per second).
177+
* **Latency**: average time required to process one frame (from reading the frame to displaying the results).
178+
You can use both of these metrics to measure application-level performance.
179+
180+
## See Also
181+
182+
* [Open Model Zoo Demos](../../README.md)
183+
* [Model Optimizer](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)
184+
* [Model Downloader](../../../tools/model_tools/README.md)

‎demos/background_subtraction_demo/python/background_subtraction.gif

6.94 MB
Loading

0 commit comments

Comments
 (0)