[MME] MME Performance Optimizations

[sun1lach]

Description

Refactored Pre-Processing & Inference

• Added AsyncInferQueue based OV inference.
• Static shape model compilation for iGPU
• ThreadPool for open_clip image parallel pre-processing.
• Batching and padding of input tensors to allow optimal path through OV inference.

Video Decode

• PyAV based video decode and video_decoder abstraction for usage from MME wrapper.
• Support for Keyframes and Uniform sampled frames extraction.
• Batching of decoded frames for parallel pipelined pre-processing and inference.
• producer-consumer pattern for parallel decode and frame translation to PIL.

TODO

• Functional for CLIP model, extend the optimization for other model handlers.
• Remove unnecessary imports, add type hints, formatting etc.
• README update for iGPU drivers, pre-requisites and usage.
• Unit Tests

Partially backward compatible. Revert this patch to check/validate existing functionality.

Any Newly Introduced Dependencies

PyAV==16.1.0 (https://github.com/PyAV-Org/PyAV)

How Has This Been Tested?

ARL - Ultra 7 255H + iGPU (Development)

Checklist:

• I agree to use the APACHE-2.0 license for my code changes.
• I have not introduced any 3rd party components incompatible with APACHE-2.0.
• I have not included any company confidential information, trade secret, password or security token.
• I have performed a self-review of my code.