Merging v2.99-R3 into main (#132)

* Fix issue with Vulkan and OV giving more than 1 GPU.

* light refactoring, and removing NPU from SuperRes device list (for now).

* created function for handling inference result

* aligned on lowercase sd_, SD UI tweaks.

* Deleting unneeded files.

* Reorganzing tree to make installation easier.

* Add workaround for caching issue in GPU for SR

* updated path due to dir structure changes.

* updated to reflect case changes.

* Updated windows install. Plugin path is no longer needed to be modified.

* Refactor install.sh

* Updated documentation to reflect Windows install changes.

* Major overhall of installation. Blobs are now created after model download.

* Updated windows install documenation

* More documenation updates

* updated UI elements

* bug fixes for SR

* Renamed files to be consistent with pythonic module nameing convention.

* Add model setup back to install

* Fixed logo paths for inclusion

* more dash to underscore fixes

* updated complete install

* updating linux installer

* Removing unused files

* Fixed logo paths

* Enabling multiple models for download.

* V2.99 r3 staging (#126)

* Adding multithreading for compile and load

* tweaks for perf

* adding back in exit

* Fixing naming

* Syncing with internal snapshot

* Syncing with internal snapshot

* Correcting capitalization

* Doc update

* Update user_guide_for_windows_users.md

* Fixed model download bug.

* Fix issue with torch 2.4.0

* Auto exit if a user chooses option 12.

* Remove debug statements

* V2.99 r3 staging update (#131)

* Initial commit

* Initial commit of gimp-ml with semantic seg powered by OV

* Added OV based deblur plugin

* Added super-resolution OV based plugin supports esrgan & sr_1033

* Initial version for OV Inpainting

* Replaced checkboxes with drop down list & added edsr model support to superres

* Added ov style transfer filter

* Super-res changes

* Adding weights folder

* Added initial version of stable diffusion

* automated stable difussion model paths

* Initial commit of gimp-ov

* Removed unwanted files

* Removed unwanted files

* Fixed semseg and removed unwanted code

* Removed unwanted dependencies

* Added screenshot and removed unwanted files

* Updated Readme

* Updated Readme

* Updated Readme

* Updated Readme

* Updated Readme

* Updated Readme

* Updated Readme

* Delete LICENSE

* Added thirdparty license and copyrights notices

* Create LICENSE.md

* Updated Readme

* Delete testscases/.ipynb_checkpoints directory

* Updated Readme

* Added missing gdown during install

* Added missing gdown during install

* Changed license to Apache

* Removed VPU device from mainline and deleted deblur plugin

* Removed unused function and files after checkmarx scan

* Commented out the exc_info to fix the Information_Exporsure issue in checkmarx

* Replaced gimpov from the entire code base

* Updated the GUI screen shot for readme

* Renamed gimpov folder

* Replaced GIMP-OV to GIMP-OpenVINO & gimpopenvino

* Removed md5 and html related blacklisted items for bandit

* Removed use of pickle from the plugin infrastructure

* Replace readlines from text with json format for semantic seg

* Replaced readlines method with json

* Updated OpenVino to 2022.3.0

* Updated transformer version

* Added copyright header to all source files

* staging for final upstream part1

* final staging part2

* Modify readme

* modify readme

* Added more gifs

* replaced the links

* updated readme

* updated readme

* updated readme

* updated sem-seg gif

* updated readme

* updated readme

* updated readme

* updated readme

* Added new web

* Final push

* Added in-painting to main branch of open-source

* Added model conversion support for SD1.5 Inpainting

* Added inital image from canvas support

* Modified mask creation step, fixed bug in load model, updated readme

* small readme update

* small readme update

* small readme update

* small readme update

* Added controlnet-openpose

* First version of updated internal version with external & internal controlnet, external inpainting

* Optimized pipeline - moving infer_request.wait()

* updated pipeline to N=2 controlnet-pose IR

* Added Super-Res workflow in SD for acer

* Latest optimized pipeline wirh share_outputs = True etc.

* updated pipeline to use ergan and set SR resolution to landscape 2240x1400

* keeping final cache.png image

* updated to run esrgan on GPU

* Added inpainting internal version and updated device list to show NPU instead of VPUX

* updated transformer version and removed unwanted debug prints

* Update stable-diffusion-ov-server.py

* Added model caching for SR model to reduce the load time

* Added model caching for SR model to reduce the load time

* Added padding as input image is square and output landscape

* fixed a bug in SR standalone plugin

* Fixed a bug in style transfer and semseg

* fixed progress bar update in controlnet-pose, added model caching for other plugins and updated model setup

* Updates for VPUX/VPU to NPU transition

* Removed direct SR support from SD gui for release

* Added controlnet-canny internal blob support

* Updates to the path specification to be cross platform compatible

* Updating paths to be cross platform compatible

* Update to OV 2023.0.2

* Cleanup of _NEW and _org files

* Combined StableDiffusion engines into one module

* Run Inference becomes Generate, Internal becomes Advanced.

* Replaced model conversion with model download from hugging face, some change in directory structure, added canny-edge batch2, scribble batch1 and batch2

* fixed a bug in repo-id for HF download

* bug fix around shutil.copytree

* uncomment the delete HF model cache

* Fixed few more bugs

* Fixed a typo

* Removed UniPCM scheduler from local file for all batch-size 2 controlnet

* Fixed corrupt image issue with landscape and portrait models:

* Moved NPU to INT8 and added LCM

* Updated import of diffusion pipeline for diffusers 0.23.0

* Initial checkin of working LCM

* Updated to OV 2023.1.0 and diffusers 0.23.0. Added accelerate

* removed some debug statements.

* Moving install to use requirements.txt to ease linux/windows parity
Added activation/deactivation of gimpenv3 on Windows during install

* Updates for the UI to hide non-needed elements for LCM

* Removed old class, other LCM updates

* Adding LCM Scheduler and removing unneeded prints.

* Updated Latent_Consistency to SD_1.5_lcm

* Removed HF pull of scheduler for LCM. Commented out scheduler log print
for now, until i find a better way to do it.

* updated LCM model dir to be in square_lcm

* added lcm in model download

* update tensorflow version for scan issue

* update onnx version for scan issue

* update README

* Fixed tabs v spaces

* Added model_setup to Linux install script.

* Updated to be linux compatible. added functions, and put back the "all" option

* Update README.md

Fixed Linux instructions, other edits to add LCM as supported.

* Update model_setup.py to remove token

Models are now public and downloads tested with access_token = None

* Fixed device list parsing due to bug found in systmed with dGPU

* updated to match public repo

* updated to match public repo

* Initial commit

* Rebase to http://github.com/intel/openvino-ai-plugins-gimp

* removed gimpml dir, not needed.

* Fixed installer location bug.

* initial set of changes to support config files for models to determine
device list

* Adding driver detection for NPU

* more updates for powermodes

* fixed few issues, including revision

* Adding LCM under my idsid

* Updating LCM to use config.json and load blobs for NPU

* Updating setup to point to my private repo for testing.

* Removed blob and swap flags, as they are no longer needed.

* Removed blob and swap flags, as they are no longer needed.

* Updated links in README.md

* Removing current direcotry echo as not needed.

* Initial checkin of SD3 support

* Update power mode support checking. Still needs work, though...

* Fixing typo in MTL NPU arch naming

* Adding config file and code to download models from the revision
adding ubuntu distro check

* Updating model setup for LNL.

* SAI Logo adding for SD3.0

* Bug fixes for Linux.

* Added default for config.json, and better dGPU handling

* trimming down plugin requirements

* Pulling updates from opensource.

* Rolled back SAI logo addition

* Moved spinner to avoid horizontal shifts in the UI. Moved the license to over by the new logo location.

* Update model_setup.py with spectral comment

* Update model_setup.py

* Update model_setup.py

* Update model_setup.py

* Update model_setup.py

* Update model_setup_config.json

* fixed typo

* Removed repeated code.

* Update for testing configs

* Fixed issues with platform detection.

* fixing tab alignment

* Fixing case issue with SD which was root cause of neg prompt still showing up in LCM menu.

* update for merge to main.

* Updating testcases

* removing unneeded files

* Updated test case with all SD models.

* Reduced redunant redundancies.

* Aligned around image instead of init_image to be consistent with other controlnet engines

* Added more engine support

* Added FP16 support for NPU

* fixing powermode ui logic

* Added Decoder info for LCM model.

* updating result naming to make comparing the same seed images easier.

* Adding blob compilation to model setup

* remove key from model setup

* Removing chmod call for deprecated features.

* update model_setup to compile 2T for Unet INT8

* add CLI options for model weights directory location

* removed reliance on driver versions.

* removed win32com which is no longer needed.

* Create README.md

* Update README.md

* Model Loading and Compiling are multithreaded, mainly on SD1.5 INT8

* Adding multithreading for compile and load

* tweaks for perf

* adding back in exit

* Fixing naming

* Syncing with internal snapshot

* Syncing with internal snapshot

* Merge WW30 Snapshot into Main (#38)

* Fixing initilization bug

* Adding better default config.json support and fixed a bug in the device_list parsing in the case there are more than one GPU in the system.

* updating dgpu device choosing logic

* Bug fixes for Linux.

* Fixed paths

* update branch info for LCM for Linux

* Fix issue with Vulkan and OV giving more than 1 GPU.

* light refactoring, and removing NPU from SuperRes device list (for now).

* created function for handling inference result

* aligned on lowercase sd_, SD UI tweaks.

* Deleting unneeded files.

* Reorganzing tree to make installation easier.

* Add workaround for caching issue in GPU for SR

* updated path due to dir structure changes.

* updated to reflect case changes.

* Updated windows install. Plugin path is no longer needed to be modified.

* Refactor install.sh

* Updated documentation to reflect Windows install changes.

* Major overhall of installation. Blobs are now created after model download.

* Updated windows install documenation

* More documenation updates

* updated UI elements

* bug fixes for SR

* syncing with v2.99-R3

* Adding support for comma separated list

* Spaces are better than commas

* Simplifying flow by not giving option to skip setup. Users can just exit right away, if needed

* Updated install flow

* switched from xcopy to robocopy

* removed unneeded lines

* removing cls for now

* whitespace changes

* remove files not needed.

* Renamed files to be consistent with pythonic module nameing convention.

* Add model setup back to install

* Fixed logo paths for inclusion

* more dash to underscore fixes

* updated complete install

* updating linux installer

* fixed missed merge conflict

* Whitespace changes.

* added back exit

* Removing tile specific config for compilation. now using default.

* Needed to be recongnized as a module

* Removing unused files

* Fixed logo paths

* updating testcase to align with new module organization.

* Updated documentation to reflect 2.99.16 and new model setup flow.

* Updated documentation to reflect 2.99.16 and new model setup flow.

* updating screenshot

* Resizing and removing figs not needed.

* Update user_guide_for_windows_users.md

* Fixing Super Resolution naming.

* Model Loading and Compiling are multithreaded, mainly on SD1.5 INT8

* Adding multithreading for model compilation and load.

* removed blank line at the start of the file.

* Removed unused function.

* fixed botched merge.

* fixed botched merge

* Bug fixes

* Removing redundant print

* minor update

* Fixed FP16 powermode issue

---------

Co-authored-by: arisha07 <[email protected]>
Co-authored-by: Siwoo Park <[email protected]>

* Correcting capitalization

* Npu turbo feature (#40)

* Fixing initilization bug

* Adding better default config.json support and fixed a bug in the device_list parsing in the case there are more than one GPU in the system.

* updating dgpu device choosing logic

* Bug fixes for Linux.

* Fixed paths

* update branch info for LCM for Linux

* Fix issue with Vulkan and OV giving more than 1 GPU.

* light refactoring, and removing NPU from SuperRes device list (for now).

* created function for handling inference result

* aligned on lowercase sd_, SD UI tweaks.

* Deleting unneeded files.

* Reorganzing tree to make installation easier.

* Add workaround for caching issue in GPU for SR

* updated path due to dir structure changes.

* updated to reflect case changes.

* Updated windows install. Plugin path is no longer needed to be modified.

* Refactor install.sh

* Updated documentation to reflect Windows install changes.

* Major overhall of installation. Blobs are now created after model download.

* Updated windows install documenation

* More documenation updates

* updated UI elements

* bug fixes for SR

* syncing with v2.99-R3

* Adding support for comma separated list

* Spaces are better than commas

* Simplifying flow by not giving option to skip setup. Users can just exit right away, if needed

* Updated install flow

* switched from xcopy to robocopy

* removed unneeded lines

* removing cls for now

* whitespace changes

* remove files not needed.

* Renamed files to be consistent with pythonic module nameing convention.

* Add model setup back to install

* Fixed logo paths for inclusion

* more dash to underscore fixes

* updated complete install

* updating linux installer

* fixed missed merge conflict

* Whitespace changes.

* added back exit

* Removing tile specific config for compilation. now using default.

* Needed to be recongnized as a module

* Removing unused files

* Fixed logo paths

* updating testcase to align with new module organization.

* Updated documentation to reflect 2.99.16 and new model setup flow.

* Updated documentation to reflect 2.99.16 and new model setup flow.

* updating screenshot

* Resizing and removing figs not needed.

* Update user_guide_for_windows_users.md

* Fixing Super Resolution naming.

* Model Loading and Compiling are multithreaded, mainly on SD1.5 INT8

* Adding multithreading for model compilation and load.

* removed blank line at the start of the file.

* Removed unused function.

* fixed botched merge.

* fixed botched merge

* Bug fixes

* Removing redundant print

* minor update

* Fixed FP16 powermode issue

* Fixing issue with pytorch 2.4.0

* Added exit after opton 12. fixed ref-only download.

* Adding turbo mode.

* relocated art.

* Adding wmi for driver version detection on Windows

* Added platform info logging for easier tracking

* Add back initial SD3.0 support.

* Updated Turbo mode and removed Artwork which could cause issues on some systems.

* Moving OV version to 2024.3.0

* Gating NPU turbo DEVICE_ARCHITECTURE.

* Test case tweaks to add timestamps to images generated.

* removing pip upgrade. may cause SSL errors.

---------

Co-authored-by: arisha07 <[email protected]>
Co-authored-by: Siwoo Park <[email protected]>

* Fixing default steps and min guidance.

* fixing NPU turbo to not fail if not enabled in the plugin

* Adding flash SD3 and hardcoding dev version of OV (for now).

* Fix sys.path extend calls to for openvino_utils/tools folder

* Updating sd3 implementation from Alpha to sd3-medium

* update default prompt

* Set openvino release

* Removing unused files.

* freezing requirements.txt

---------

Co-authored-by: Arisha Kumar <[email protected]>
Co-authored-by: RyanMetcalfeInt8 <[email protected]>
Co-authored-by: Luke Lin <[email protected]>
Co-authored-by: ymangukiya <[email protected]>
Co-authored-by: nschaper-intel <[email protected]>
Co-authored-by: Siwoo Park <[email protected]>
Co-authored-by: Metcalfe, Ryan <[email protected]>

* Add tile configuration to model setup

* updating testcases with new default prompt and random seeds beyond the first image.

* Adjusting model to IP mapping

* Added guidance scale param

* Removed unnecessary print.

* Fixing git clone command.

* updating threshold

* updating tools path

* remove unneeded print.

* fixed model configs for FP16 and INT8

* typo and bug fix in controlnet models

* Removing unneeded lines

* I am once again updating the profile IP mapping.

* fixed seed for SD3

* Fixed missing zero

* Update model configs.

* Adjust init image checkbox to not display for LCM.

* freezing torch and torchvision versions.

* Adapted stable_diffusion_engine.py and stable_diffusion_engine_tc.py for basic Linux (Ubuntu 22) support

* Adding W8A16 support

---------

Co-authored-by: Arisha Kumar <[email protected]>
Co-authored-by: RyanMetcalfeInt8 <[email protected]>
Co-authored-by: Luke Lin <[email protected]>
Co-authored-by: ymangukiya <[email protected]>
Co-authored-by: nschaper-intel <[email protected]>
Co-authored-by: Siwoo Park <[email protected]>
Co-authored-by: Metcalfe, Ryan <[email protected]>
Co-authored-by: Rafael Lago <[email protected]>

Author: 8 people