Merge branch 'master' into merge-releases/2022/SCv1.1-into-master

Author: Wovchena

README.md

@@ -1,5 +1,5 @@
 # [OpenVINO™ Toolkit](https://docs.openvino.ai/latest/index.html) - Open Model Zoo repository
-[![Stable release](https://img.shields.io/badge/version-2022.1-green.svg)](https://github.com/openvinotoolkit/open_model_zoo/releases/tag/2022.1)
+[![Stable release](https://img.shields.io/badge/version-2022.2.0-green.svg)](https://github.com/openvinotoolkit/open_model_zoo/releases/tag/2022.2.0)
 [![Gitter chat](https://badges.gitter.im/gitterHQ/gitter.png)](https://gitter.im/open_model_zoo/community)
 [![Apache License Version 2.0](https://img.shields.io/badge/license-Apache_2.0-green.svg)](LICENSE)
 

ci/dependencies.yml

@@ -1,6 +1,6 @@
-opencv_linux: '20220228_0602-4.5.5_079'
-opencv_windows: '20220228_0602-4.5.5_079'
-openvino_linux: '2022.1.0.612'
-openvino_windows: '2022.1.0.612'
-wheel_linux: '2022.1.0.dev20220228-6910'
-wheel_windows: '2022.1.0.dev20220228-6910'
+opencv_linux: '20220311_0602-4.5.5_090'
+opencv_windows: '20220311_0602-4.5.5_090'
+openvino_linux: '2022.1.0.643'
+openvino_windows: '2022.1.0.643'
+wheel_linux: '2022.1.0-7019'
+wheel_windows: '2022.1.0-7019'

ci/requirements-openvino-dev.in

@@ -30,9 +30,9 @@ numpy (<1.20,>=1.16.6) ; extra == 'kaldi'
 numpy (<1.20,>=1.16.6) ; extra == 'mxnet'
 numpy (<1.20,>=1.16.6) ; extra == 'onnx'
 numpy (<1.20,>=1.16.6) ; extra == 'tensorflow2'
-onnx (>=1.8.1) ; extra == 'caffe2'
-onnx (>=1.8.1) ; extra == 'onnx'
-onnx (>=1.8.1) ; extra == 'pytorch'
+onnx (<=1.12,>=1.8.1) ; extra == 'caffe2'
+onnx (<=1.12,>=1.8.1) ; extra == 'onnx'
+onnx (<=1.12,>=1.8.1) ; extra == 'pytorch'
 opencv-python (==4.5.*)
 openvino (==2021.4.2)
 pandas (~=1.1.5)

data/dataset_definitions.yml

@@ -1254,7 +1254,7 @@ datasets:
     dataset_meta: antispoofing.json
 
   - name: sound_classification
-    data_source: audio_dataset
+    data_source: audio_dataset/data
     annotation_conversion:
       converter: sound_classification
       annotation_file: audio_dataset/validation.csv
@@ -1424,8 +1424,8 @@ datasets:
       image_folder: Raw-data/Single-channel/Val
       reconstructed_folder: accuracy_checker_preprocess/Single-channel/Reconstructed
       sampled_folder: accuracy_checker_preprocess/Single-channel/Sampled
-      mask_file: hybrid-cs-model-mri/sampling_mask_20perc.npy
-      stats_file: hybrid-cs-model-mri/stats_fs_unet_norm_20.npy
+      mask_file: sampling_mask_20perc.npy
+      stats_file: stats_fs_unet_norm_20.npy
 
   - name: WikiText_2_raw_gpt2
     annotation: wikitext_2_raw.pickle
@@ -1493,3 +1493,14 @@ datasets:
       annotation_file: object_detection/streams_1/high/annotations/instances_glb2bcls3.json
     annotation: mscoco_detection_high_3cls.pickle
     dataset_meta: mscoco_detection_high_3cls.json
+
+  - name: HumanMattingDataset
+    data_source:  human_matting_dataset/clip_img/1803151818/clip_00000000
+    additional_data_source: human_matting_dataset/matting/1803151818/matting_00000000
+    annotation_conversion:
+      converter: background_matting
+      images_dir: human_matting_dataset/clip_img/1803151818/clip_00000000
+      masks_dir: human_matting_dataset/matting/1803151818/matting_00000000
+      image_postfix: '.jpg'
+    annotation: human_matting.pickle
+    dataset_meta: human_matting.json

data/datasets.md

@@ -137,12 +137,16 @@ macro(add_demo)
     endif()
 endmacro()
 
-find_package(OpenCV REQUIRED COMPONENTS core highgui videoio imgproc imgcodecs gapi)
+find_package(OpenCV REQUIRED COMPONENTS core highgui videoio imgproc imgcodecs)
 find_package(OpenVINO REQUIRED COMPONENTS Runtime)
 
 add_subdirectory(thirdparty/gflags)
 add_subdirectory(common/cpp)
-add_subdirectory(common/cpp_gapi)
+# TODO: remove wrapping if after OpenCV3 is dropped
+if(OpenCV_VERSION VERSION_GREATER_EQUAL 4.5.3)
+    find_package(OpenCV REQUIRED COMPONENTS gapi)
+    add_subdirectory(common/cpp_gapi)
+endif()
 add_subdirectory(multi_channel_common/cpp)
 
 # collect all samples subdirectories

demos/CMakeLists.txt

@@ -75,14 +75,12 @@
 @endsphinxdirective
 -->
 
-The Open Model Zoo demo applications are console applications that provide robust application templates to help you implement specific deep learning scenarios. These applications involve increasingly complex processing pipelines that gather analysis data from several models that run inference simultaneously, such as detecting a person in a video stream along with detecting the person's physical attributes, such as age, gender, and emotional state
+Open Model Zoo demos are console applications that provide templates to help implement specific deep learning inference scenarios. These applications show how to preprocess and postrpocess data for model inference and organize processing pipelines. Some pipelines collect analysis data from several models being inferred simultaneously. For example, [detecting a person in a video stream along with detecting the person's physical attributes, such as age, gender, and emotional state](./interactive_face_detection_demo/cpp/README.md).
 
 Source code of the demos can be obtained from the Open Model Zoo [GitHub repository](https://github.com/openvinotoolkit/open_model_zoo/).
 
 ```sh
-git clone https://github.com/openvinotoolkit/open_model_zoo.git
-cd open_model_zoo
-git submodule update --init --recursive
+git clone --recurse-submodules https://github.com/openvinotoolkit/open_model_zoo.git
 ```
 
 C++, C++ G-API and Python\* versions are located in the `cpp`, `cpp_gapi` and `python` subdirectories respectively.
@@ -156,7 +154,7 @@ The Open Model Zoo includes the following demos:
 
 ## Media Files Available for Demos
 
-To run the demo applications, you can use images and videos from the media files collection available at https://github.com/intel-iot-devkit/sample-videos.
+To run the demo applications, you can use videos from https://storage.openvinotoolkit.org/data/test_data/videos.
 
 ## Demos that Support Pre-Trained Models
 
@@ -166,11 +164,10 @@ You can download the [Intel pre-trained models](../models/intel/index.md) or [pu
 
 ## Build the Demo Applications
 
-To build the demos, you need to source OpenVINO™ and OpenCV environment. You can install the OpenVINO™ toolkit using the installation package for [Intel® Distribution of OpenVINO™ toolkit](https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit-download.html) or build the open-source version available in the [OpenVINO GitHub repository](https://github.com/openvinotoolkit/openvino) using the [build instructions](https://github.com/openvinotoolkit/openvino/wiki/BuildingCode).
+To build the demos, you need to source OpenVINO™ environment and [get OpenCV](https://github.com/opencv/opencv/wiki/BuildOpenCV4OpenVINO). You can install the OpenVINO™ toolkit using the installation package for [Intel® Distribution of OpenVINO™ toolkit](https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit-download.html) or build the open-source version available in the [OpenVINO GitHub repository](https://github.com/openvinotoolkit/openvino) using the [build instructions](https://github.com/openvinotoolkit/openvino/wiki/BuildingCode).
 For the Intel® Distribution of OpenVINO™ toolkit installed to the `<INSTALL_DIR>` directory on your machine, run the following commands to download prebuilt OpenCV and set environment variables before building the demos:
 
 ```sh
-<INSTALL_DIR>/extras/scripts/download_opencv.sh
 source <INSTALL_DIR>/setupvars.sh
 ```
 
@@ -253,7 +250,7 @@ build_demos_msvc.bat VS2019
 ```
 
 By default, the demo applications binaries are build into the `C:\Users\<username>\Documents\Intel\OpenVINO\omz_demos_build\intel64\Release` directory.
-The default build folder can be changed with `-b` option. For example, following command will buid Open Model Zoo demos into `c:\temp\omz-demos-build` folder:
+The default build folder can be changed with `-b` option. For example, following command will build Open Model Zoo demos into `c:\temp\omz-demos-build` folder:
 
 ```bat
 build_demos_msvc.bat -b c:\temp\omz-demos-build
@@ -298,7 +295,7 @@ python -mpip install --user -r <omz_dir>/demos/requirements.txt
 
 ### <a name="python_model_api"></a>Python\* model API package
 
-To run Python demo applications, you need to install the Python* Model API package. Refer to [Python* Model API documentation](common/python/openvino/model_zoo/model_api/README.md#installing-python*-model-api-package) to learn about its installation.
+Python* ModelAPI is factored out as a sepparate package. Refer to the [Python Model API documentation](https://github.com/openvinotoolkit/open_model_zoo/blob/master/demos/common/python/openvino/model_zoo/model_api/README.md#installing-python-model-api-package)) to learn about its installation. At the same time demos can find this package on their own. It's not required to install ModelAPI for demos.
 
 ### <a name="build_python_extensions"></a>Build the Native Python\* Extension Modules
 

demos/README.md

@@ -22,6 +22,8 @@
 import cv2
 import numpy as np
 
+from openvino.runtime import AsyncInferQueue
+
 
 def center_crop(frame, crop_size):
     img_h, img_w, _ = frame.shape
@@ -93,22 +95,31 @@ def __init__(self, model_path, core, target_device, num_requests, model_type):
             log.error("Demo supports only models with 1 output")
             sys.exit(1)
 
+        self.outputs = {}
         compiled_model = core.compile_model(self.model, target_device)
         self.output_tensor = compiled_model.outputs[0]
         self.input_name = self.model.inputs[0].get_any_name()
         self.input_shape = self.model.inputs[0].shape
 
         self.num_requests = num_requests
-        self.infer_requests = [compiled_model.create_infer_request() for _ in range(self.num_requests)]
+        self.infer_queue = AsyncInferQueue(compiled_model, num_requests)
+        self.infer_queue.set_callback(self.completion_callback)
         log.info('The {} model {} is loaded to {}'.format(model_type, model_path, target_device))
 
+    def completion_callback(self, infer_request, id):
+        self.outputs[id] = infer_request.results[self.output_tensor]
+
     def async_infer(self, frame, req_id):
         input_data = {self.input_name: frame}
-        self.infer_requests[req_id].start_async(inputs=input_data)
+        self.infer_queue.start_async(input_data, req_id)
 
     def wait_request(self, req_id):
-        self.infer_requests[req_id].wait()
-        return self.infer_requests[req_id].results[self.output_tensor]
+        self.infer_queue[req_id].wait()
+        return self.outputs.pop(req_id, None)
+
+    def cancel(self):
+        for ireq in self.infer_queue:
+            ireq.cancel()
 
 
 class DummyDecoder:
@@ -126,3 +137,6 @@ def async_infer(self, model_input, req_id):
     def wait_request(self, req_id):
         assert req_id in self.requests
         return self.requests.pop(req_id)
+
+    def cancel(self):
+        pass

demos/action_recognition_demo/python/action_recognition_demo/models.py

@@ -101,6 +101,9 @@ def __init__(self, encoder):
         self.encoder = encoder
         self.async_model = AsyncWrapper(self.encoder, self.encoder.num_requests)
 
+    def __del__(self):
+        self.encoder.cancel()
+
     def process(self, frame):
         preprocessed = preprocess_frame(frame)
         preprocessed = preprocessed[np.newaxis, ...]  # add batch dimension
@@ -121,6 +124,9 @@ def __init__(self, decoder, sequence_size=16):
         self.async_model = AsyncWrapper(self.decoder, self.decoder.num_requests)
         self._embeddings = deque(maxlen=self.sequence_size)
 
+    def __del__(self):
+        self.decoder.cancel()
+
     def process(self, item):
         if item is None:
             return None

demos/action_recognition_demo/python/action_recognition_demo/steps.py

@@ -5,7 +5,7 @@ This demo shows how to perform background subtraction using G-API.
 > **NOTE**: Only batch size of 1 is supported.
 
 ## How It Works
-The demo application expects an instance-segmentation-security-???? or trimap free background matting based on pixel-level segmentation approach model in the Intermediate Representation (IR) format.
+The demo application expects an instance-segmentation-security-???? or trimap free background matting based on pixel-level segmentation approach model in the Intermediate Representation (IR) format. Please note, that there aren't background matting models in `OpenModelZoo` collection.
 
 1. for instance segmentation models based on `Mask RCNN` approach:
     * One input: `image` for input image.
@@ -54,6 +54,34 @@ omz_converter --list models.lst
 
 > **NOTE**: Refer to the tables [Intel's Pre-Trained Models Device Support](../../../models/intel/device_support.md) and [Public Pre-Trained Models Device Support](../../../models/public/device_support.md) for the details on models inference support at different devices.
 
+
+### OneVPL Support
+
+Demo provides functionality to use [OneVPL](https://github.com/oneapi-src/oneVPL#-video-processing-library) video decoding.
+Example:
+```sh
+./background_subtraction_demo_gapi/ -m <path_to_model> -i <path_to_video_file> -use_onevpl
+```
+
+In order to provide additional configuration paramaters use `-onevpl_params`:
+```sh
+./background_subtraction_demo_gapi/ -m <path_to_model> -i <path_to_raw_file> -use_onevpl -onevpl_params="mfxImplDescription.mfxDecoderDescription.decoder.CodecID:MFX_CODEC_HEVC"
+```
+>**NOTE**: Only raw formats such as `h264`, `h265` etc are supported on Linux.
+Working with raw formats user always must specify `codec` type via `-onevpl_params`. See example below.
+
+To build OpenCV G-API with `oneVPL` support follow instruction:
+[Building G-API with oneVPL Toolkit support](https://github.com/opencv/opencv/wiki/Graph-API#building-with-onevpl-toolkit-support)
+
+#### Troubleshooting
+During execution `oneVPL` might report warnings that tell the user that source can be configurable more accurate.
+
+For example:
+```
+ cv::gapi::wip::onevpl::VPLLegacyDecodeEngine::process_error [000001CED3851C70] error: cv::gapi::wip::onevpl::CachedPool::find_free - cannot get free surface from pool, size: 5
+```
+This might be fixed by increasing pool size using `-onevpl_pool_size` parameter.
+
 ## Running
 
 Run the application with the `-h` option to see the following usage message:
@@ -82,6 +110,9 @@ Options:
     -blur_bgr                  Optional. Blur background.
     -target_bgr                Optional. Background onto which to composite the output (by default to green field).
     -u                         Optional. List of monitors to show initially.
+    -use_onevpl                Optional. Use onevpl video decoding.
+    -onevpl_params             Optional. Parameters for onevpl video decoding. OneVPL source can be fine-grained by providing configuration parameters. Format: <prop name>:<value>,<prop name>:<value> Several important configuration parameters: 'mfxImplDescription.mfxDecoderDescription.decoder.CodecID' values: https://spec.oneapi.io/onevpl/2.7.0/API_ref/VPL_enums.html?highlight=mfx_codec_hevc#codecformatfourcc and 'mfxImplDescription.AccelerationMode' values: https://spec.oneapi.io/onevpl/2.7.0/API_ref/VPL_disp_api_enum.html?highlight=d3d11#mfxaccelerationmode(see `MFXSetConfigFilterProperty` by https://spec.oneapi.io/versions/latest/elements/oneVPL/source/index.html)
+    -onevpl_pool_size          OneVPL source applies this parameter as preallocated frames pool size. 0 leaves frames pool size default for your system. This parameter doesn't have a god default value. It must be adjusted for specific execution (video, model, system ...).
 
 Available target devices:  <targets>
 ```