Merge pull request #36 from intel/version_1_4

Added SD-1.5-Inpainting and modified init image support in GUI

Author: arisha07

README.md

@@ -9,7 +9,7 @@ This branch is currently under development. <br>Dedicated for GIMP 3, Python 3 a
 [2] Style-Transfer <br>
 [3] Inpainting <br>
 [4] Semantic-Segmentation <br>
-[5] Stable-Diffusion <br>
+[5] Stable-Diffusion (Suppports - SD 1.4, SD 1.5 (landscape & portrait), SD 1.5 Inpainting) <br>
 
 # Objectives
 [1] Provides a set of OpenVino based plugins that add AI features to GIMP. <br>
@@ -26,14 +26,26 @@ Please raise a PR for any new features, modifactions or bug fixes.
 ## Installation Steps
 
 ### Windows
+Skip steps 1 and 2 if you already have Python3 and Git on Windows
+
+#### 1. Install Python
+- Download a Python installer from python.org. Choose Python 3.7, 3.8, 3.9 or 3.10 and make sure to pick a 64 bit version. For example, this 3.8 installer: https://www.python.org/ftp/python/3.8.8/python-3.8.8-amd64.exe <br>
+- Double click on the installer to run it, and follow the steps in the installer. Check the box to add Python to your PATH, and to install py. At the end of the installer, there is an option to disable the PATH length limit. It is recommended to click this. <br>
+
+#### 2. Install Git
+- Download and install [GIT](https://git-scm.com/)
+
+#### 3. Install the GIMP Plugin
+
 1. Install [GIMP 2.99.10 (revision 2)](https://download.gimp.org/gimp/v2.99/windows/gimp-2.99.10-setup-2.exe) or Install [GIMP 2.99.14](https://download.gimp.org/gimp/v2.99/windows/gimp-2.99.14-setup.exe) <br>
-2. Clone, run install script: <br>
+2. Clone and run install script: <br>
 
    - clone this repo: <br>
    ```git clone https://github.com/intel/openvino-ai-plugins-gimp.git``` <br>
 
     - run install script - this will create the virtual environment "gimpenv3", install all required packages and will also walk you through models setup. <br>
    ```openvino-ai-plugins-gimp\install.bat``` <br>
+   *You can re-run "run install script" step later again to install & setup models that you may have missed.* <br>
 
 3. Start the GIMP application, and add the gimpenv3 path that was printed when running the above step to the list of plugin folders [Edit-> Preferences-> Folders-> Plugins]. <br>
    Example:  ```Plug-ins in GIMP :  <path\to>\gimpenv3\lib\site-packages\gimpopenvino\plugins``` Add this path to [Edit-> Preferences-> Folders-> Plugins] in GIMP <br>
@@ -47,20 +59,38 @@ Please raise a PR for any new features, modifactions or bug fixes.
    # clone this repo:
    git clone https://github.com/intel/openvino-ai-plugins-gimp.git
 
-   # run install script - this will create the virtual environment "gimpenv3", install all required packages and will also walk you through models setup 
+   # run install script - this will create the virtual environment "gimpenv3", install all required packages and will also walk you through models setup. 
    openvino-ai-plugins-gimp/install.sh
    ```
+   *You can re-run "run install script" step later again to install & setup models that you may have missed.* <br>
+
 3. Start the GIMP application (```flatpak run org.gimp.GIMP```), and add the gimpenv3 path that was printed when running the above step to the list of plugin folders  [Edit-> Preferences-> Folders-> Plugins]. <br>
 4. Restart GIMP, and you should see 'OpenVINO-AI-Plugins' show up in 'Layer' menu <br>
 
 ### OpenVINO™ Image Generator Plugin with Stable Diffusion - This GIF doesn't represent the current GUI
+#### A. Prompt to Image 
 1. Create or choose a layer  <br>
 2. Select Stable Diffusion from the drop down list in layers -> OpenVINO-AI-Plugins <br>
 3. Choose the desired model and device from the drop down list.<br>
 4. Click on "Load Models" to compile & load the model on the selected device. Wait for it to complete. Please note that you need to perform this step only if you change the model or device or both. For any subsequent runs just click "Run Inference" <br>
-3. Enter prompt and other parameters <br>
-4. Click on “Run Inference”. Wait for the total inference steps to get completed. <br>
-6. If create gif option is selected, please note that performance will reduce. The generated gif is located in below path. You can play it in GIMP by going to Filters -> Animations -> Playback <br>
+5. Enter prompt and other parameters <br>
+6. Click on “Run Inference”. Wait for the total inference steps to get completed. <br>
+
+#### B. Image to Image
+1. Create or choose a layer or open an image  <br>
+2. Follow steps 2,3,4,5 from section A. <br> 
+3. Select "Use Initial Image"
+4. By default the opened image in canvas will be used as initial image to the model. You can also select a different image by browsing from files.
+5. Click on “Run Inference”. Wait for the total inference steps to get completed. <br>
+
+#### C. Stable-Diffusion-1.5 Inpainting - Make sure to download and convert the model during install process. 
+1. Choose a layer or Open an image of size 512x512. (Currently works best with this resolution) <br>
+2. Use "Free select tool" to select the area in your image that you wish to change. <br>
+3. Right click on you image and click on "Add layer mask". Then choose "Selection" in "Initalize layer Mask to". This should create a mask with your selection.
+4. Follow steps 2,3,4,5 from section A. Please note that you will only see "SD_1.5_Inpainting" in model options if you added a mask layer to your image. <br>
+5. Click on “Run Inference”. Wait for the total inference steps to get completed. <br>
+
+*If create gif option is selected, please note that performance will reduce. The generated gif is located in below path. You can play it in GIMP by going to Filters -> Animations -> Playback* <br>
 ```C:\Users\<user_name>\openvino-ai-plugins-gimp\gif\stable_diffusion.gif``` <br>
 
 ![](gifs/stable-diffusion.png)

choose_sd_model.py

@@ -16,28 +16,31 @@
 print("3 - Portrait (360x640 output image, 9:16) ")
 print("4 - Portrait_512x768 (512x768 output image), will take time since model is large ")
 print("5 - Landscape_768x512 (768x512 output image),  will take time since model is large ")
-print("6 - ALL the above SD-1.5 models ")
-print("7 - Only Square, Landscape, Portrait")
-print("8 - Skip All SD-1.5 Model setup ")   
+print("6 - SD-1.5 Inapinting model (512x512 output image) ")
+print("7 - ALL the above SD-1.5 models ")
+print("8 - Only Square, Landscape, Portrait")
+print("9 - Skip All SD-1.5 Model setup ")   
 
 while True:
     choice = input("Enter the Number for the model you want to download & convert: ")
 
-    if choice=="6":
-        for i in range(1,6):
+    # setup all the SD1.5 models
+    if choice=="7":
+        for i in range(1,7):
 
             subprocess.call([sd_python_path, chose_model, str(i)])
 
         break
-    elif choice=="7":
+    # setup only square, landscape and Portrait
+    elif choice=="8":
         for i in range(1,4):
 
             subprocess.call([sd_python_path, chose_model, str(i)])
         break
-    elif choice=="8":
+    elif choice=="9":
         print("Exiting SD-1.5 Model setup.........")
         break
-    elif choice in ["1","2","3","4","5"]:
+    elif choice in ["1","2","3","4","5","6"]:
 
         subprocess.call([sd_python_path, chose_model, choice])