Question: run inference directly from raw_image

[hoangtnm]

Hi, as I understand from the face_mask_and_eyes_closeness example, it requires input for masked_face_estimator.estimate to be detected faces from the capturer. However, there are several cases where I want to use my own face detection models and use the SDK's masked_face_estimator only. For this scenario, is there any way to bypass the capturer.capture(raw_img) and pass the raw_img = CVRawImage(img) into the estimator?

[leonid-leshukov]

Hi @hoangtnm

Thank you for your interest in 3DiVi Face SDK.

You can use your own detector to create a sample by doing the following:

1. Create a Capturer object using FacerecService.create_service method and configuration file manual_capturer_fda.xml
2. Create a RawSample object by passing eye points to the Capturer.manual_capture method.
3. Use the RawSample object to evaluate the appearance of the mask.

Please let me know if this works for you.

[hoangtnm]

Hi @leonid-leshukov

I think capturer.manual_capture(raw_img) could fix the problem. However,
in my case, I would like to try age_gender_estimator given raw image, is it
possible?

def detect(img_path):
    img = cv2.imread(img_path)
    assert img is not None
    raw_img = CVRawImage(img)
    detected = capturer.manual_capture(raw_img)
    print(f"On image detected {len(detected)} faces")
    return detected

samples = detect(os.path.join(face_sdk_dir, "bin/set1", "01100.jpg"))
for i, sample in enumerate(samples):
    age_gender_res = age_gender_estimator.estimate_age_gender(sample)
    print(age_gender_res)

but it raises

TypeError                                 Traceback (most recent call last)
Cell In [6], line 1
----> 1 samples = detect(os.path.join(face_sdk_dir, "bin/set1", "01100.jpg"))
      2 for i, sample in enumerate(samples):
      3     # raw_img = sample.cut_face_raw_image(Format.FORMAT_GRAY)
      4     # img_crop = np.frombuffer(raw_img.data, dtype=np.uint8).reshape(
      5     #     [raw_img.height, raw_img.width]
      6     # )
      7     age_gender_res = age_gender_estimator.estimate_age_gender(sample)

Cell In [5], line 6, in detect(img_path)
      4 raw_img = CVRawImage(img)
      5 # detected = capturer.capture(raw_img)
----> 6 detected = capturer.manual_capture(raw_img)
      7 print(f"On image detected {len(detected)} faces")
      8 return detected

TypeError: manual_capture() missing 4 required positional arguments: 'left_eye_x', 'left_eye_y', 'right_eye_x', and 'right_eye_y'

[leonid-leshukov]

Hi @hoangtnm

capturer.manual_capture(raw_img) requires eyes position to create RawSample object. Do you have eyes position in your detector output?

[hoangtnm]

Hi @hoangtnm

capturer.manual_capture(raw_img) requires eyes position to create RawSample object. Do you have eyes position in your detector output?

Hi @leonid-leshukov, I don't have eye positions in your detector output. Actually, I only have face-bounding boxes from the detector. Therefore, I only want to create a RawSample object from a numpy array.

[leonid-leshukov]

Hi @hoangtnm
In such case, you can use fda_fake_detector_signleface.xml. It uses a bounding box to find face landmarks and returns the RawSample object.

To use fda_fake_detector_signleface.xml you should specify the relative position of the bounding box center and the relative size of the bounding box on the image. See example below

fda_fake_detector_signleface example

from face_sdk_3divi import FacerecService, Config
from face_sdk_3divi.modules.raw_image import Format
from face_sdk_3divi.example import CVRawImage

import os
import cv2
import numpy as np


def detect(img_path):
    img = cv2.imread(img_path)
    assert img is not None
    raw_img = CVRawImage(img)
    detected = capturer.capture(raw_img)
    print(f'On image detected {len(detected)} faces')
    return detected


face_sdk_dir = "../.."
service = FacerecService.create_service(
    os.path.join(face_sdk_dir, "bin/facerec.dll"),
    os.path.join(face_sdk_dir, "conf/facerec"))
print('Service created')

config = Config("fda_fake_detector_singleface.xml")
config.override_parameter("fake_rect_center_x0", 0.628)
config.override_parameter("fake_rect_center_y0", 0.354)
config.override_parameter("fake_rect_size0", 0.1)
capturer = service.create_capturer(config)
print('Capturer created')

samples = detect(os.path.join(face_sdk_dir, "bin/1/8.jpg"))
for i, sample in enumerate(samples):
    raw_img = sample.cut_face_raw_image(Format.FORMAT_GRAY)