Fetch RAISR internal work with upstream main.

.dockerignore

@@ -0,0 +1,5 @@
+*.md
+./Jenkinsfile*
+./license*
+./test*
+*Dockerfile*

.gitignore

@@ -57,3 +57,4 @@ build
 *.*proj
 *.*advi*
 config/
+raisr/

.shellcheckrc

@@ -0,0 +1 @@
+external-sources=true

CMakeLists.txt

@@ -45,8 +45,8 @@ SET(CMAKE_REQUIRED_FLAGS "-march=native")
 check_cxx_source_runs("
     #include <immintrin.h>
     int main(int argc, char** argv) {
-        _Float16 data[16] = {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15};
-        _Float16 output[16];
+        _Float16 data[32] = {0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31};
+        _Float16 output[32];
         __m512h a = _mm512_loadu_ph(data); // avx512fp16
         __m512h b = _mm512_loadu_ph(data);
         __mmask32 mask = _mm512_cmp_ph_mask(a, b, _CMP_EQ_OQ); // avx512fp16

How to build.md

@@ -3,24 +3,24 @@ To build this project you will need:
 - Linux based OS
     - For CPU tested and validated on Ubuntu 18.04 LTS, Ubuntu 22.04 and CentOS 7.9
     - For GPU tested and validated on Intel® Data Center GPU Flex 170 with Ubuntu 22.04 LTS(5.15 LTS kernel)
-- [Docker](https://www.docker.com/) 
+- [Docker](https://www.docker.com/)
 - Intel Xeon hardware which supports Intel AVX512 (Skylake generation or later)
 - Compiler (clang++, g++, icc), and enabling AVX512-FP16 on Sapphire Rapiads needs:
     - clang++ version 14.0.0 or later
     - g++ version 12.1.0 with binutils 2.38 or later
     - icc version 2021.2 or later
-- Cmake version 3.14 or later 
+- Cmake version 3.14 or later
 - Intel® Integrated Performance Primitives (Intel® IPP) (Stand-Alone Version is the minimum requirement)
 - zlib1g-dev, pkg-config (The pkg-config is used to find x264.pc/x265.pc in specific pkgconfig path.)
 
 We provide 3 ways to build the Intel VSR with FFmpeg environment:
-- build docker images with dockerfiles(only for CPU).
-- build via [scripts](https://github.com/OpenVisualCloud/Video-Super-Resolution-Library/tree/master/scripts)(only for CPU).
+- build docker images with dockerfiles(both CPU and GPU).
+- build via [scripts](https://github.com/OpenVisualCloud/Video-Super-Resolution-Library/tree/main/scripts)(only for CPU).
 - build manually(both CPU and GPU).
 
 # Build Docker Images.
 
-We provide 3 Dockerfile for Intel Xeon platforms: Ubuntu18.04, Ubuntu22.04 and CentOS7.9. You can refer to below steps to build docker images.
+We provide 3 Dockerfile for Intel Xeon platforms: Ubuntu18.04, Ubuntu22.04 and CentOS7.9, and 1 Dockerfile with Ubuntu22.04 for Intel Flex GPU. You can refer to below steps to build docker images.
 ## Setup docker proxy as follows if you are behind a firewall:
 ```
 sudo mkdir -p /etc/systemd/system/docker.service.d
@@ -32,23 +32,25 @@ sudo systemctl restart docker
 ## build docker image via docker_build.sh
 The usage of the script is as follows
 ```
-./docker_build.sh <OS> <OS_VERSION>
+./docker_build.sh <PLATFORM> <OS> <OS_VERSION>
 
-./docker_build.sh ubuntu 22.04 #for building Ubuntu22.04
-./docker_build.sh ubuntu 18.04 #for building Ubuntu18.04
-./docker_build.sh centos 7.9 #for building CentOS7.9
+./docker_build.sh xeon ubuntu 22.04 #for building Xeon platform with Ubuntu22.04
+./docker_build.sh xeon ubuntu 18.04 #for building Xeon platform with Ubuntu18.04
+./docker_build.sh xeon centos 7.9 #for building Xeon platform with CentOS7.9
+./docker_build.sh flex ubuntu 22.04 #for building Flex platform with Ubuntu22.04
 ```
-If the image is built successfully, you can find a docker image named `raisr:ubuntu22.04` or `raisr:ubuntu18.04` or `raisr:centos7.9` with command `docker images`
+If the image is built successfully, you can find a docker image named `raisr-xeon:ubuntu22.04` or `raisr-xeon:ubuntu18.04` or `raisr-xeon:centos7.9` or `raisr-flex:ubuntu22.04` with command `docker images`
+Please note it needs to add `--privileged  --device /dev/dri` option to access GPU hardware in docker container during run raisr-flex docker container.
 
-# Build via [scripts](https://github.com/OpenVisualCloud/Video-Super-Resolution-Library/tree/master/scripts)
+# Build via [scripts](https://github.com/OpenVisualCloud/Video-Super-Resolution-Library/tree/main/scripts)
 If the user would prefer not to use Docker, you can follow the steps below to setup enviroment: \
     `cd Video-Super-Resolution-Library/scripts` \
     `./01_pull_resources.sh` \
     `./02_install_prerequisites.sh /xxx/raisr.tar.gz` \
     `./03_build_raisr_ffmpeg.sh /xxx/raisr/Video-Super-Resolution-Library`
-- [01_pull_resources.sh](https://github.com/OpenVisualCloud/Video-Super-Resolution-Library/blob/master/scripts/01_pull_resources.sh): Download the resources used for build Intel Library for VSR and FFmpeg(cmake 3.14, nasm, x264, x265, ipp, Intel Library for VSR and FFmpeg) and package these resource to      raisr.tar.gz.
-- [02_install_prerequisites.sh](https://github.com/OpenVisualCloud/Video-Super-Resolution-Library/blob/master/scripts/02_install_prerequisites.sh): Extract the tarball raisr.tar.gz of resources and build and install the libraries required by building Intel Library for VSR and FFmpeg.
-- [03_build_raisr_ffmpeg.sh](https://github.com/OpenVisualCloud/Video-Super-Resolution-Library/blob/master/scripts/03_build_raisr_ffmpeg.sh): Build Intel Library for VSR and FFmpeg.
+- [01_pull_resources.sh](https://github.com/OpenVisualCloud/Video-Super-Resolution-Library/blob/main/scripts/01_pull_resources.sh): Download the resources used for build Intel Library for VSR and FFmpeg(cmake 3.14, nasm, x264, x265, ipp, Intel Library for VSR and FFmpeg) and package these resource to      raisr.tar.gz.
+- [02_install_prerequisites.sh](https://github.com/OpenVisualCloud/Video-Super-Resolution-Library/blob/main/scripts/02_install_prerequisites.sh): Extract the tarball raisr.tar.gz of resources and build and install the libraries required by building Intel Library for VSR and FFmpeg.
+- [03_build_raisr_ffmpeg.sh](https://github.com/OpenVisualCloud/Video-Super-Resolution-Library/blob/main/scripts/03_build_raisr_ffmpeg.sh): Build Intel Library for VSR and FFmpeg.
 
 # Build manually following the steps below
 ## Install Intel IPP
@@ -66,7 +68,7 @@ The x264/x265 libraries can be installed via apt on Ubuntu OS or built and insta
 
 ### Build and install x264/x265 from source code(Option-2)
 
-#### Build and install x264 
+#### Build and install x264
 
 `git clone https://github.com/mirror/x264 -b stable --depth 1` \
 `cd x264` \

Library/Raisr.cpp

@@ -73,7 +73,7 @@ static bool machine_supports_feature(MachineVendorType vendor, ASMType type)
     bool ret = false;
     unsigned int eax = 0, ebx = 0, ecx = 0, edx = 0;
 
-    if (vendor == INTEL ) {
+    if (vendor == INTEL || vendor == AMD ) {
         __get_cpuid_count(0x7, 0x0, &eax, &ebx, &ecx, &edx);
 
         if (type == AVX512_FP16) {
@@ -98,20 +98,6 @@ static bool machine_supports_feature(MachineVendorType vendor, ASMType type)
             }
         }
     }
-    else if (vendor == AMD)
-    {
-        __get_cpuid_count(0x7, 0x0, &eax, &ebx, &ecx, &edx);
-        if (type == AVX512_FP16) {
-            ret = false;
-        } else if (type == AVX512) {
-            ret = false;
-        } else if (type == AVX2) {
-            if ( (ebx >> 5) & 0x1)
-            {
-                ret = true;
-            }
-        }
-    }
     return ret;
 }
 
@@ -898,7 +884,7 @@ __m256i inline modulo_imm( __m256i a, int b) {
 void inline write_pixeltype(int c, __m256i gPatchMargin_epi32, __m256i partone, int* out) {
                         __m256i b = _mm256_sub_epi32( _mm256_add_epi32(_mm256_set1_epi32(c), _mm256_setr_epi32(0,1,2,3,4,5,6,7)), gPatchMargin_epi32);
                         __m256i pixelType_epi32 = _mm256_add_epi32( partone, modulo_imm(b, gRatio) );
-                        _mm256_storeu_epi32(out, pixelType_epi32);
+                        _mm256_storeu_si256((__m256i *)(out), pixelType_epi32);
 }
 
 RNLERRORTYPE processSegment(VideoDataType *srcY, VideoDataType *final_outY, BlendingMode blendingMode, int threadIdx)

README.md

@@ -14,6 +14,7 @@ Enhanced RAISR is provided as an FFmpeg plugin inside of a Docker container(Dock
 
 - Feb 2024 : AWS and Intel announced collaboration to release Intel Library for VSR on AWS Cloud at the Mile High Video 2024 conference, technical details available at  https://dl.acm.org/doi/10.1145/3638036.3640290  
 
+We have enhanced the public RAISR algorithm to achieve better visual quality and beyond real-time performance for 2x and 1.5x upscaling on Intel® Xeon® platforms and Intel® GPUs. The Intel Library for VSR is provided as an FFmpeg plugin inside of a Docker container to help ease testing and deployment burdens. This project is developed using C++ and takes advantage of Intel® Advanced Vector Extension 512 (Intel® AVX-512) where available and newly added Intel® AVX-512FP16 support on Intel® Xeon® 4th Generation (Sapphire Rapids) and added OpenCL support on Intel® GPUs.
 
 ## How to build
 Please see "How to build.md" to build via scripts or manually.
@@ -73,6 +74,15 @@ ffmpeg -init_hw_device vaapi=va -init_hw_device qsv=qs@va -init_hw_device opencl
 ffmpeg -init_hw_device vaapi=va -init_hw_device opencl=ocl@va -hwaccel vaapi -hwaccel_output_format vaapi -i input.264 -vf "hwmap=derive_device=opencl,format=opencl,raisr_opencl,hwmap=derive_device=vaapi:reverse=1:extra_hw_frames=16" -c:v hevc_vaapi output.mp4
 ```
 
+**Even output**
+
+There are certain codecs that support only even resolution, the `evenoutput` parameter will support users to choose whether to make the output an even number
+
+Set `evenoutput=1` to make output size as even number, the following command will get 632x632 output.
+```
+ffmpeg -i input.mp4 -an -vf scale=422x422,raisr=ratio=1.5:filterfolder=filters_1.5x/filters_highres:threadcount=1:evenoutput=1 output.mp4
+```
+It will keep the output resolution as the input resolution multiply by the upscaling ratio if set `evenoutput=0` or not set the parameter, will get 633x633 output with 422x422 input.
 
 ## To see help on the RAISR filter
 `./ffmpeg -h filter=raisr`
@@ -89,6 +99,7 @@ ffmpeg -init_hw_device vaapi=va -init_hw_device opencl=ocl@va -hwaccel vaapi -hw
       asm               <string>     ..FV....... x86 asm type: (avx512fp16, avx512, avx2 or opencl) (default "avx512fp16")
       platform          <int>        ..FV....... select the platform (from 0 to INT_MAX) (default 0)
       device            <int>        ..FV....... select the device (from 0 to INT_MAX) (default 0)
+      evenoutput        <int>        ..FV....... make output size as even number (0: ignore, 1: subtract 1px if needed) (from 0 to 1) (default 0)
 
 
 # How to Contribute

build.sh

@@ -1,11 +1,21 @@
-#!/bin/sh
+#!/bin/bash
+
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright 2024-2025 Intel Corporation
+
+# Fails the script if any of the commands error (Other than if and some others)
+set -ex -o pipefail
+
+SCRIPT_DIR="$(readlink -f "$(dirname -- "${BASH_SOURCE[0]}")")"
+. "${SCRIPT_DIR}/scripts/common.sh"
+nproc="${nproc:-$(nproc)}"
+
 # Helpful when copying and pasting functions and debuging.
 if printf '%s' "$0" | grep -q '\.sh'; then
     IN_SCRIPT=true
 fi
 
-# Fails the script if any of the commands error (Other than if and some others)
-set -e
+
 
 cd_safe() {
     if (cd "$1"); then
@@ -20,7 +30,7 @@ cd_safe() {
 # Usage: build [test]
 build() (
     build_type=Release
-    echo "Create folder: build, build type: $build_type"
+    log_info "Create folder: build, build type: $build_type"
     mkdir -p build > /dev/null 2>&1
     cd_safe build
 
@@ -32,8 +42,8 @@ build() (
     #cmake .. -DCMAKE_BUILD_TYPE="RelWithDebInfo" $CMAKE_EXTRA_FLAGS "$@"
 
     if [ -f Makefile ]; then
-        # make -j
-	    make install -j
+        make -j "${nproc}"
+        make install -j "${nproc}"
     fi
 
     cd ..
@@ -74,4 +84,4 @@ else
 fi
 export CXX
 
-build $@
+build "$@"