Adding a Gradio Example

examples/gradio/gradio_example.md

@@ -0,0 +1,92 @@
+# Gradio Example
+
+- Playground to test differential diffusion SD2 pipeline with different masks.
+
+## Features
+- Generate gradient mask on runtime.
+- Import the mask image.
+- Extract the depth image from the given image with [Intel/dpt-large](https://huggingface.co/Intel/dpt-large) model.
+- Brightness, contrast and image transform functionalities for the mask images.
+- Generate output with input image, positive and negative prompts, guidance scale, strength, number of inference steps parameters.
+
+## Required Packages
+- transformers
+- accelerate
+- torch
+- diffusers
+- gradio
+
+## Usage
+1. Go to ```differential-diffusion``` directory
+2. Active the virtual environment if there is one
+3. Run ```python examples/gradio/main.py``` command
+
+## Tabs
+
+- When changing between tabs, configured depth images are updated on the ```Mask Image```.
+
+### Gradient Mask
+
+![gradient_mask_tab_ss](screenshots/gradient_mask_tab.png)
+
+|       Variable        | Definition                                          |
+| :-------------------: | :-------------------------------------------------- |
+|     Gradient Mask     | Temp gradient mask to use in calculations           |
+| Gradient Image Width  | Define the width of the gradient image              |
+| Gradient Image Height | Define the height of the gradient image             |
+|   Gradient Strength   | Set the white and black ratios                      |
+|   Image Brightness    | Set the gradient mask image's brightness            |
+|    Image Contrast     | Set the gradient mask images's contrast             |
+|    Flip Horizontal    | Flip the gradient mask image horizontally           |
+|      To Vertical      | Rotate the gradient mask image 90 degrees clockwise |
+|     Flip Vertical     | Flip the gradient mask image vertically             |
+
+- ```Mask Image``` is updated when any variable ,except ```Gradient Mask```, value is changed
+
+### Image Mask
+
+![image_mask_tab_ss](screenshots/image_mask_tab.png)
+
+|     Variable     | Definition                                                   |
+| :--------------: | :----------------------------------------------------------- |
+|    Image Mask    | The mask image can be uploaded as file or from the clipboard |
+| Image Brightness | Set the input mask image's brightness                        |
+|  Image Contrast  | Set the input mask images's contrast                         |
+| Flip Horizontal  | Flip the input mask image horizontally                       |
+|   To Vertical    | Rotate the input mask image 90 degrees clockwise             |
+|  Flip Vertical   | Flip the input mask image vertically                         |
+
+- ```Mask Image``` is updated when any variable value is changed
+
+### Extract Depth
+
+![extract_depth_tab_ss](screenshots/extract_depth_tab.png)
+
+|       Variable        | Definition                                              |
+| :-------------------: | :------------------------------------------------------ |
+|      Input Image      | Original image which is used to extract the depth image |
+| Extracted Depth Image | Showing the extracted depth image                       |
+|   Image Brightness    | Set the extracted depth image's brightness              |
+|    Image Contrast     | Set the extracted depth images's contrast               |
+|    Flip Horizontal    | Flip the extracted depth image horizontally             |
+|      To Vertical      | Rotate the extracted depth image 90 degrees clockwise   |
+|     Flip Vertical     | Flip the extracted depth image vertically               |
+
+- ```Mask Image``` is updated when any variable ,except ```Extracted Depth Image```, value is changed
+
+### Generate
+
+![generate_tab_ss](screenshots/generated_tab.png)
+
+|    Variable     | Definition                                            |
+| :-------------: | :---------------------------------------------------- |
+|      Image      | Image for the differential diffusion Img2Img pipeline |
+| Guidance Scale  | Guidance scale for the pipeline                       |
+| Inference Step  | Number of inference steps for the pipeline            |
+|    Strength     | Strength variable for the pipeline                    |
+| Positive Prompt | Positive prompt for the pipeline                      |
+| Negative Prompt | Negative prompt for the pipeline                      |
+
+- The progress is started with ```Generate``` button.
+- The generated output is showed in ```Output Image```.
+- ```Mask Image``` has to have image otherwise generation will fail.

examples/gradio/main.py

@@ -0,0 +1,116 @@
+from gradio import Blocks, Tab, Row, Column, Image
+
+from utilities.event_funcs import gradient_calculate, image_edit_change
+
+from tabs.gradient_tab import GradientTab
+from tabs.image_mask_tab import ImageMaskTab
+from tabs.generate_tab import GenerateTab
+from tabs.extract_depth_tab import ExtractDepthTab
+
+from importlib.util import find_spec
+
+
+def check_package(package_name: str) -> None:
+    if find_spec(package_name):
+        print(f"/_\ {package_name} is found")
+    else:
+        print(
+            f"/_\ {package_name} is not found. Please install the {package_name} with pip"
+        )
+        exit(1)
+
+
+print(" Differential Diffusion Gradio Example ".center(100, "-"))
+print("/_\ Checking Packages")
+
+check_package("diffusers")
+check_package("transformers")
+check_package("accelerate")
+check_package("torch")
+check_package("gradio")
+
+print("/_\ Launching example")
+
+gradient_tab = GradientTab()
+image_mask_tab = ImageMaskTab()
+extract_depth_tab = ExtractDepthTab()
+generate_tab = GenerateTab()
+
+with Blocks() as example:
+    with Row() as main_row:
+        with Tab("Gradient Mask") as tab_gradient:
+            gradient_tab.render()
+
+        with Tab("Image Mask") as tab_image_mask:
+            image_mask_tab.render()
+
+        with Tab("Extract Depth") as tab_extracted_depth:
+            extract_depth_tab.render()
+
+        with Tab("Generate") as tab_generate:
+            generate_tab.render()
+
+        with Column():
+            mask_image = Image(
+                value=gradient_calculate(512, 512, 1.0, 1.0, 1.0, False, False, False),
+                sources=None,
+                label="Mask Image",
+                width=512,
+                height=512,
+            )
+            output_image = Image(
+                sources=None, label="Output Image", width=512, height=512
+            )
+
+    tab_gradient.select(
+        gradient_calculate,
+        inputs=[
+            gradient_tab.width_slider,
+            gradient_tab.height_slider,
+            gradient_tab.strength_slider,
+            gradient_tab.image_edit.brightness_slider,
+            gradient_tab.image_edit.contrast_slider,
+            gradient_tab.image_edit.flip_horizontal_checkbox,
+            gradient_tab.image_edit.to_vertical_checkbox,
+            gradient_tab.image_edit.flip_vertical_checkbox,
+        ],
+        outputs=mask_image,
+        show_progress="hidden",
+    )
+
+    tab_image_mask.select(
+        image_edit_change,
+        inputs=[
+            image_mask_tab.mask_image,
+            image_mask_tab.image_edit.brightness_slider,
+            image_mask_tab.image_edit.contrast_slider,
+            image_mask_tab.image_edit.flip_horizontal_checkbox,
+            image_mask_tab.image_edit.to_vertical_checkbox,
+            image_mask_tab.image_edit.flip_vertical_checkbox,
+        ],
+        outputs=mask_image,
+        show_progress="hidden",
+    )
+
+    tab_extracted_depth.select(
+        image_edit_change,
+        inputs=[
+            extract_depth_tab.extracted_depth_image,
+            extract_depth_tab.image_edit.brightness_slider,
+            extract_depth_tab.image_edit.contrast_slider,
+            extract_depth_tab.image_edit.flip_horizontal_checkbox,
+            extract_depth_tab.image_edit.to_vertical_checkbox,
+            extract_depth_tab.image_edit.flip_vertical_checkbox,
+        ],
+        outputs=mask_image,
+        show_progress="hidden",
+    )
+
+    gradient_tab.attach_event(mask_image)
+    image_mask_tab.attach_event(mask_image)
+    extract_depth_tab.attach_event(mask_image)
+    generate_tab.attach_event(mask_image, output_image)
+
+
+if __name__ == "__main__":
+    example.launch()

examples/gradio/tabs/extract_depth_tab.py

@@ -0,0 +1,77 @@
+from transformers import pipeline
+from gradio import Column, Image, Row
+import numpy as np
+from PIL import Image as pil_image
+from torch.nn.functional import interpolate
+
+from .image_edit_block import ImageEditBlock
+
+
+class ExtractDepthTab:
+    def __init__(self) -> None:
+        self.main_column = Column()
+
+        self.images_row = Row()
+
+        self.wanted_image = Image(
+            sources=["upload", "clipboard"],
+            type="pil",
+            width=512,
+            height=512,
+            label="Input Image",
+        )
+
+        from os.path import join
+
+        self.extracted_depth_image = Image(
+            value=pil_image.open(join("assets", "map2.jpg")),
+            width=512,
+            height=512,
+            type="pil",
+            label="Extracted Depth Image",
+            sources=None,
+        )
+
+        self.pipe = pipeline(
+            "depth-estimation",
+            model="Intel/dpt-large",
+            framework="pt",
+            torch_dtype="auto",
+        )
+
+        self.image_edit = ImageEditBlock()
+
+    def render(self) -> None:
+        with self.images_row:
+            self.wanted_image.render()
+            self.extracted_depth_image.render()
+
+        self.images_row.render()
+        self.image_edit.render()
+
+    def attach_event(self, output_image) -> None:
+        def extract_depth(given_image):
+            outputs = self.pipe(given_image)
+            predicted_depth = outputs["predicted_depth"]
+
+            prediction = interpolate(
+                input=predicted_depth.unsqueeze(1),
+                size=given_image.size[::-1],
+                mode="bicubic",
+                align_corners=False,
+            )
+
+            output = prediction.squeeze().cpu().numpy()
+            formatted = (output * 255 / np.max(output)).astype("uint8")
+            pil_formatted_image = pil_image.fromarray(formatted)
+
+            return [pil_formatted_image, pil_formatted_image]
+
+        self.wanted_image.upload(
+            extract_depth,
+            inputs=self.wanted_image,
+            outputs=[self.extracted_depth_image, output_image],
+            show_progress="full",
+        )
+
+        self.image_edit.attach_event(self.extracted_depth_image, output_image)