Merge pull request #73 from HyperInspire/dev/attribute

Dev/attribute

Former-commit-id: dcc4d87

Author: tunmx

.github/workflows/sample_ubuntu_x86_Pikachu_python.yaml

@@ -0,0 +1,45 @@
+# This GitHub Actions workflow is designed for a CMake project running on a single platform (Ubuntu-x86).
+# For multi-platform testing, see the link provided.
+# Refer to: https://github.com/actions/starter-workflows/blob/main/ci/cmake-multi-platform.yml
+name: Run Ubuntu-x86 Test Pikachu from Python Native
+
+# Trigger this workflow on push or pull request to the "feature/sub" branch
+on:
+  push:
+    branches: ["master"]
+  pull_request:
+    branches: ["master"]
+
+# Define environment variables shared across jobs
+env:
+  # Set the CMake build type (e.g., Release, Debug, RelWithDebInfo, etc.)
+  BUILD_TYPE: Release
+
+# Jobs section defines all individual tasks for the CI workflow
+jobs:
+  build:
+    # Specify that this job should run on the latest Ubuntu environment provided by GitHub
+    runs-on: ubuntu-latest
+
+    # Define steps for this job
+    steps:
+      # Step 1: Check out the code from the repository
+      - uses: actions/checkout@v4
+
+      # Step 2: Update Git submodules recursively
+      - name: Update submodules
+        run: |
+          git clone --recurse-submodules https://github.com/HyperInspire/3rdparty.git
+
+      # Step 3: Install necessary dependencies for building the CMake project
+      - name: Install dependencies
+        run: |
+          sudo apt-get update  # Update package lists
+          # Install build tools and required libraries for video processing
+          sudo apt-get install -y build-essential libgtk-3-dev libavcodec-dev libavformat-dev libjpeg-dev libswscale-dev
+
+      # Step 4: Run a separate script for CMake configuration and building
+      - name: Download Dataset And Configure CMake
+        # Execute a pre-existing script to handle CMake configuration and building
+        # The script is assumed to be located at `ci/quick_test_linux_x86_usual.sh`
+        run: bash ci/quick_test_linux_x86_usual_python_native_interface.sh

CMakeLists.txt

@@ -9,7 +9,7 @@ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -O3")
 # Current version
 set(INSPIRE_FACE_VERSION_MAJOR 1)
 set(INSPIRE_FACE_VERSION_MINOR 1)
-set(INSPIRE_FACE_VERSION_PATCH 3)
+set(INSPIRE_FACE_VERSION_PATCH 4)
 
 # Converts the version number to a string
 string(CONCAT INSPIRE_FACE_VERSION_MAJOR_STR ${INSPIRE_FACE_VERSION_MAJOR})

README.md

@@ -1,6 +1,7 @@
 # InspireFace
 [![GitHub release](https://img.shields.io/github/v/release/HyperInspire/InspireFace.svg?style=for-the-badge&color=blue)](https://github.com/HyperInspire/InspireFace/releases/latest)
 [![build](https://img.shields.io/github/actions/workflow/status/HyperInspire/InspireFace/release-sdks.yaml?&style=for-the-badge&label=build)](https://img.shields.io/github/actions/workflow/status/HyperInspire/InspireFace/release-sdks.yaml?&style=for-the-badge&label=build)
+[![test](https://img.shields.io/github/actions/workflow/status/HyperInspire/InspireFace/release-sdks.yaml?&style=for-the-badge&label=test)](https://img.shields.io/github/actions/workflow/status/HyperInspire/InspireFace/test_ubuntu_x86_Pikachu.yaml?&style=for-the-badge&label=test)
 
 InspireFace is a cross-platform face recognition SDK developed in C/C++, supporting multiple operating systems and various backend types for inference, such as CPU, GPU, and NPU.
 
@@ -10,6 +11,10 @@ Please contact [contact@insightface.ai](mailto:contact@insightface.ai?subject=In
 
 ## Change Logs
 
+**`2024-07-05`** Fixed some bugs in the python ctypes interface.
+
+**`2024-07-03`** Add the blink detection algorithm of face interaction module.
+
 **`2024-07-02`** Fixed several bugs in the face detector with multi-level input.
 
 **`2024-06-27`** Verified iOS usability and fixed some bugs.
@@ -52,7 +57,7 @@ You can download the model package files containing models and configurations ne
 If you intend to use the SDK locally or on a server, ensure that OpenCV is installed on the host device beforehand to enable successful linking during the compilation process. For cross-compilation targets like Android or ARM embedded boards, you can use the pre-compiled OpenCV libraries provided by **3rdparty/inspireface-precompile/opencv/**.
 
 ### 1.4. Installing MNN
-The '3rdparty' directory already includes the MNN library and specifies a particular version as the stable version. If you need to enable or disable additional configuration options during compilation, you can refer to the CMake Options provided by MNN. If you need to use your own precompiled version, feel free to replace it.
+The '**3rdparty**' directory already includes the MNN library and specifies a particular version as the stable version. If you need to enable or disable additional configuration options during compilation, you can refer to the CMake Options provided by MNN. If you need to use your own precompiled version, feel free to replace it.
 
 ### 1.5. Requirements
 
@@ -298,14 +303,14 @@ In the project, there is a subproject called cpp/test. To compile it, you need t
 ```bash
 cmake -DISF_BUILD_WITH_TEST=ON ..
 ```
-If you need to run test cases, you will need to download the required [resource files](https://drive.google.com/file/d/1i4uC-dZTQxdVgn2rP0ZdfJTMkJIXgYY4/view?usp=sharing), which are **test_res** and **Model Package** respectively. Unzip the pack file into the test_res folder. The directory structure of test_res should be prepared as follows before testing:
+If you need to run test cases, you will need to download the required [resource files](https://drive.google.com/file/d/1i4uC-dZTQxdVgn2rP0ZdfJTMkJIXgYY4/view?usp=sharing): **test_res**. Unzip the test_res folder. The directory structure of test_res should be prepared as follows before testing:
 
 ```bash
 
 test_res
 ├── data
 ├── images
-├── pack		<- unzip pack.zip
+├── pack	<-- The model package files are here
 ├── save
 ├── valid_lfw_funneled.txt
 ├── video
@@ -352,17 +357,17 @@ The following functionalities and technologies are currently supported.
 | 6 | Silent Liveness Detection | ![Static Badge](https://img.shields.io/badge/STABLE-blue?style=for-the-badge) | MiniVision |
 | 7 | Face Quality Detection | ![Static Badge](https://img.shields.io/badge/STABLE-blue?style=for-the-badge) |  |
 | 8 | Face Pose Estimation | ![Static Badge](https://img.shields.io/badge/STABLE-blue?style=for-the-badge) |  |
-| 9 | Age Prediction | ![Static Badge](https://img.shields.io/badge/PENDING-yellow?style=for-the-badge) |  |
-| 10 | Cooperative Liveness Detection | ![Static Badge](https://img.shields.io/badge/PENDING-yellow?style=for-the-badge) |  |
+| 9 | Face Attribute Prediction | ![Static Badge](https://img.shields.io/badge/STABLE-blue?style=for-the-badge) | Age, Race, Gender |
+| 10 | Cooperative Liveness Detection | ![Static Badge](https://img.shields.io/badge/DEVELOP-green?style=for-the-badge) | Blink |
 
 
 ## 6. Models Package List
 
-For different scenarios, we currently provide several Packs, each containing multiple models and configurations.
+For different scenarios, we currently provide several Packs, each containing multiple models and configurations.The package file is placed in the **pack** subdirectory under the **test_res** directory.
 
 | Name | Supported Devices | Note | Link |
 | --- | --- | --- | --- |
-| Pikachu | CPU | Lightweight edge-side model | [GDrive](https://drive.google.com/drive/folders/1krmv9Pj0XEZXR1GRPHjW_Sl7t4l0dNSS?usp=sharing) |
-| Megatron | CPU, GPU | Local or server-side model | [GDrive](https://drive.google.com/drive/folders/1krmv9Pj0XEZXR1GRPHjW_Sl7t4l0dNSS?usp=sharing) |
+| Pikachu | CPU | Lightweight edge-side models | [GDrive](https://drive.google.com/drive/folders/1krmv9Pj0XEZXR1GRPHjW_Sl7t4l0dNSS?usp=sharing) |
+| Megatron | CPU, GPU | Mobile and server models | [GDrive](https://drive.google.com/drive/folders/1krmv9Pj0XEZXR1GRPHjW_Sl7t4l0dNSS?usp=sharing) |
 | Gundam-RV1109 | RKNPU | Supports RK1109 and RK1126 | [GDrive](https://drive.google.com/drive/folders/1krmv9Pj0XEZXR1GRPHjW_Sl7t4l0dNSS?usp=sharing) |
 

ci/quick_test_linux_x86_usual.sh

@@ -54,8 +54,14 @@ cmake -DCMAKE_BUILD_TYPE=Release \
 # Compile the project using 4 parallel jobs
 make -j4
 
-# Create a symbolic link to the extracted test data directory
-ln -s ${FULL_TEST_DIR} .
+# Check if the symbolic link or directory already exists
+if [ ! -e "$(basename ${FULL_TEST_DIR})" ]; then
+    # Create a symbolic link to the extracted test data directory
+    ln -s ${FULL_TEST_DIR} .
+    echo "Symbolic link to '${TARGET_DIR}' created."
+else
+    echo "Symbolic link or directory '$(basename ${FULL_TEST_DIR})' already exists. Skipping creation."
+fi
 
 # Check if the test executable file exists
 if [ ! -f "$TEST_EXECUTABLE" ]; then

ci/quick_test_linux_x86_usual_python_native_interface.sh

@@ -0,0 +1,71 @@
+#!/bin/bash
+
+# Exit immediately if any command exits with a non-zero status
+set -e
+
+ROOT_DIR="$(pwd)"
+TARGET_DIR="test_res"
+DOWNLOAD_URL="https://github.com/tunmx/inspireface-store/raw/main/resource/test_res-lite.zip"
+ZIP_FILE="test_res-lite.zip"
+BUILD_DIRNAME="ubuntu18_shared"
+
+# Check if the target directory already exists
+if [ ! -d "$TARGET_DIR" ]; then
+    echo "Directory '$TARGET_DIR' does not exist. Downloading..."
+
+    # Download the dataset zip file
+    wget -q "$DOWNLOAD_URL" -O "$ZIP_FILE"
+
+    echo "Extracting '$ZIP_FILE' to '$TARGET_DIR'..."
+    # Unzip the downloaded file
+    unzip "$ZIP_FILE"
+
+    # Remove the downloaded zip file and unnecessary folders
+    rm "$ZIP_FILE"
+    rm -rf "__MACOSX"
+
+    echo "Download and extraction complete."
+else
+    echo "Directory '$TARGET_DIR' already exists. Skipping download."
+fi
+
+# Get the absolute path of the target directory
+FULL_TEST_DIR="$(realpath ${TARGET_DIR})"
+
+# Create the build directory if it doesn't exist
+mkdir -p build/${BUILD_DIRNAME}/
+
+# Change directory to the build directory
+# Disable the shellcheck warning for potential directory changes
+# shellcheck disable=SC2164
+cd build/${BUILD_DIRNAME}/
+
+# Configure the CMake build system
+cmake -DCMAKE_BUILD_TYPE=Release \
+  -DISF_BUILD_WITH_SAMPLE=OFF \
+  -DISF_BUILD_WITH_TEST=OFF \
+  -DISF_ENABLE_BENCHMARK=OFF \
+  -DISF_ENABLE_USE_LFW_DATA=OFF \
+  -DISF_ENABLE_TEST_EVALUATION=OFF \
+  -DOpenCV_DIR=3rdparty/inspireface-precompile/opencv/4.5.1/opencv-ubuntu18-x86/lib/cmake/opencv4 \
+  -DISF_BUILD_SHARED_LIBS=ON ../../
+
+# Compile the project using 4 parallel jobs
+make -j4
+
+# Come back to project root dir
+cd ${ROOT_DIR}
+
+# Important: You must copy the compiled dynamic library to this path!
+cp build/${BUILD_DIRNAME}/lib/libInspireFace.so python/inspireface/modules/core/
+
+# Install dependency
+pip install opencv-python
+pip install click
+pip install loguru
+
+cd python/
+
+# Run sample
+python sample_face_detection.py ../test_res/pack/Pikachu ../test_res/data/bulk/woman.png
+

ci/quick_test_local.sh

@@ -66,3 +66,5 @@ else
     echo "Test executable found. Running tests..."
     "$TEST_EXECUTABLE"
 fi
+
+# Executing python scripts

cpp/inspireface/c_api/inspireface.cc

@@ -100,13 +100,13 @@ HResult HFReleaseInspireFaceSession(HFSession handle) {
 HResult HFCreateInspireFaceSession(HFSessionCustomParameter parameter, HFDetectMode detectMode, HInt32 maxDetectFaceNum, HInt32 detectPixelLevel, HInt32 trackByDetectModeFPS, HFSession *handle) {
     inspire::ContextCustomParameter param;
     param.enable_mask_detect = parameter.enable_mask_detect;
-    param.enable_age = parameter.enable_age;
+    param.enable_face_attribute = parameter.enable_face_quality;
     param.enable_liveness = parameter.enable_liveness;
     param.enable_face_quality = parameter.enable_face_quality;
-    param.enable_gender = parameter.enable_gender;
     param.enable_interaction_liveness = parameter.enable_interaction_liveness;
     param.enable_ir_liveness = parameter.enable_ir_liveness;
     param.enable_recognition = parameter.enable_recognition;
+    param.enable_face_attribute = parameter.enable_face_attribute;
     inspire::DetectMode detMode = inspire::DETECT_MODE_ALWAYS_DETECT;
     if (detectMode == HF_DETECT_MODE_LIGHT_TRACK) {
         detMode = inspire::DETECT_MODE_LIGHT_TRACK;
@@ -138,11 +138,8 @@ HResult HFCreateInspireFaceSessionOptional(HOption customOption, HFDetectMode de
     if (customOption & HF_ENABLE_IR_LIVENESS) {
         param.enable_ir_liveness = true;
     }
-    if (customOption & HF_ENABLE_AGE_PREDICT) {
-        param.enable_age = true;
-    }
-    if (customOption & HF_ENABLE_GENDER_PREDICT) {
-        param.enable_gender = true;
+    if (customOption & HF_ENABLE_FACE_ATTRIBUTE) {
+        param.enable_face_attribute = true;
     }
     if (customOption & HF_ENABLE_MASK_DETECT) {
         param.enable_mask_detect = true;
@@ -508,13 +505,13 @@ HResult HFMultipleFacePipelineProcess(HFSession session, HFImageStream streamHan
     }
     inspire::ContextCustomParameter param;
     param.enable_mask_detect = parameter.enable_mask_detect;
-    param.enable_age = parameter.enable_age;
+    param.enable_face_attribute = parameter.enable_face_quality;
     param.enable_liveness = parameter.enable_liveness;
     param.enable_face_quality = parameter.enable_face_quality;
-    param.enable_gender = parameter.enable_gender;
     param.enable_interaction_liveness = parameter.enable_interaction_liveness;
     param.enable_ir_liveness = parameter.enable_ir_liveness;
     param.enable_recognition = parameter.enable_recognition;
+    param.enable_face_attribute = parameter.enable_face_attribute;
 
     HResult ret;
     std::vector<inspire::HyperFaceData> data;
@@ -562,11 +559,8 @@ HResult HFMultipleFacePipelineProcessOptional(HFSession session, HFImageStream s
     if (customOption & HF_ENABLE_IR_LIVENESS) {
         param.enable_ir_liveness = true;
     }
-    if (customOption & HF_ENABLE_AGE_PREDICT) {
-        param.enable_age = true;
-    }
-    if (customOption & HF_ENABLE_GENDER_PREDICT) {
-        param.enable_gender = true;
+    if (customOption & HF_ENABLE_FACE_ATTRIBUTE) {
+        param.enable_face_attribute = true;
     }
     if (customOption & HF_ENABLE_MASK_DETECT) {
         param.enable_mask_detect = true;
@@ -675,6 +669,23 @@ HResult HFGetFaceIntereactionResult(HFSession session, PHFFaceIntereactionResult
     return HSUCCEED;
 }
 
+HResult HFGetFaceAttributeResult(HFSession session, PHFFaceAttributeResult results) {
+    if (session == nullptr) {
+        return HERR_INVALID_CONTEXT_HANDLE;
+    }
+    HF_FaceAlgorithmSession *ctx = (HF_FaceAlgorithmSession* ) session;
+    if (ctx == nullptr) {
+        return HERR_INVALID_CONTEXT_HANDLE;
+    }
+
+    results->num = ctx->impl.GetFaceAgeBracketResultsCache().size();
+    results->race = (HPInt32 )ctx->impl.GetFaceRaceResultsCache().data();
+    results->gender = (HPInt32 )ctx->impl.GetFaceGenderResultsCache().data();
+    results->ageBracket = (HPInt32 )ctx->impl.GetFaceAgeBracketResultsCache().data();
+
+    return HSUCCEED;
+}
+
 HResult HFFeatureHubGetFaceCount(HInt32* count) {
     *count = FEATURE_HUB->GetFaceFeatureCount();
     return HSUCCEED;

cpp/inspireface/c_api/inspireface.h

@@ -29,8 +29,8 @@ extern "C" {
 #define HF_ENABLE_LIVENESS                  0x00000004  ///< Flag to enable RGB liveness detection feature.
 #define HF_ENABLE_IR_LIVENESS               0x00000008  ///< Flag to enable IR (Infrared) liveness detection feature.
 #define HF_ENABLE_MASK_DETECT               0x00000010  ///< Flag to enable mask detection feature.
-#define HF_ENABLE_AGE_PREDICT               0x00000020  ///< Flag to enable age prediction feature.
-#define HF_ENABLE_GENDER_PREDICT            0x00000040  ///< Flag to enable gender prediction feature.
+#define HF_ENABLE_FACE_ATTRIBUTE            0x00000020  ///< Flag to enable face attribute prediction feature.
+#define HF_ENABLE_PLACEHOLDER_              0x00000040  ///< - 
 #define HF_ENABLE_QUALITY                   0x00000080  ///< Flag to enable face quality assessment feature.
 #define HF_ENABLE_INTERACTION               0x00000100  ///< Flag to enable interaction feature.
 
@@ -125,9 +125,8 @@ typedef struct HFSessionCustomParameter {
     HInt32 enable_liveness;                  ///< Enable RGB liveness detection feature.
     HInt32 enable_ir_liveness;               ///< Enable IR liveness detection feature.
     HInt32 enable_mask_detect;               ///< Enable mask detection feature.
-    HInt32 enable_age;                       ///< Enable age prediction feature.
-    HInt32 enable_gender;                    ///< Enable gender prediction feature.
     HInt32 enable_face_quality;              ///< Enable face quality detection feature.
+    HInt32 enable_face_attribute;            ///< Enable face attribute prediction feature.
     HInt32 enable_interaction_liveness;      ///< Enable interaction for liveness detection feature.
 } HFSessionCustomParameter, *PHFSessionCustomParameter;
 
@@ -149,7 +148,7 @@ typedef enum HFDetectMode {
  * @param detectMode Detection mode to be used.
  * @param maxDetectFaceNum Maximum number of faces to detect.
  * @param detectPixelLevel Modify the input resolution level of the detector, the larger the better, 
- *          the need to input a multiple of 160, such as 160, 320, 640, the default value -1 is 160.
+ *          the need to input a multiple of 160, such as 160, 320, 640, the default value -1 is 320.
  * @param trackByDetectModeFPS If you are using the MODE_TRACK_BY_DETECTION tracking mode, 
  *          this value is used to set the fps frame rate of your current incoming video stream, which defaults to -1 at 30fps.
  * @param handle Pointer to the context handle that will be returned.
@@ -647,6 +646,47 @@ typedef struct HFFaceIntereactionResult {
 
 HYPER_CAPI_EXPORT extern HResult HFGetFaceIntereactionResult(HFSession session, PHFFaceIntereactionResult result);
 
+/**
+ * @brief Struct representing face attribute results.
+ *
+ * This struct holds the race, gender, and age bracket attributes for a detected face.
+ */
+typedef struct HFFaceAttributeResult {
+    HInt32 num;             ///< Number of faces detected.
+    HPInt32 race;           ///< Race of the detected face.
+                            ///< 0: Black; 
+                            ///< 1: Asian; 
+                            ///< 2: Latino/Hispanic; 
+                            ///< 3: Middle Eastern; 
+                            ///< 4: White;
+    HPInt32 gender;         ///< Gender of the detected face.
+                            ///< 0: Female; 
+                            ///< 1: Male;
+    HPInt32 ageBracket;     ///< Age bracket of the detected face.
+                            ///< 0: 0-2 years old; 
+                            ///< 1: 3-9 years old; 
+                            ///< 2: 10-19 years old; 
+                            ///< 3: 20-29 years old; 
+                            ///< 4: 30-39 years old; 
+                            ///< 5: 40-49 years old; 
+                            ///< 6: 50-59 years old; 
+                            ///< 7: 60-69 years old; 
+                            ///< 8: more than 70 years old;
+} HFFaceAttributeResult, *PHFFaceAttributeResult;
+
+/**
+ * @brief Get the face attribute results.
+ *
+ * This function retrieves the attribute results such as race, gender, and age bracket
+ * for faces detected in the current context.
+ *
+ * @param session Handle to the session.
+ * @param results Pointer to the structure where face attribute results will be stored.
+ * @return HResult indicating the success or failure of the operation.
+ */
+HYPER_CAPI_EXPORT extern HResult HFGetFaceAttributeResult(HFSession session, PHFFaceAttributeResult results);
+
+
 /************************************************************************
 * System Function
 ************************************************************************/