|
| 1 | +--- |
| 2 | +title: Running the LCM-LoRA-SDv1-5 Model on MaixPy MaixCAM |
| 3 | +update: |
| 4 | + - date: 2025-12-03 |
| 5 | + author: lxowalle |
| 6 | + version: 1.0.0 |
| 7 | + content: Added LCM-LoRA-SDv1-5 code and documentation |
| 8 | +--- |
| 9 | + |
| 10 | +## Supported Devices |
| 11 | + |
| 12 | +| Device | Supported | |
| 13 | +| -------- | ------- | |
| 14 | +| MaixCAM2 | ✅ | |
| 15 | +| MaixCAM | ❌ | |
| 16 | + |
| 17 | + |
| 18 | +## Introduction to the LCM-LoRA-SDv1-5 Model |
| 19 | + |
| 20 | +LCM-LoRA-SDv1-5 is a model that supports text-to-image and image-to-image generation, based on the StableDiffusion 1.5 LCM project. With this model, you can generate conceptual images for artistic creation—simply provide a text description, and the model will generate an image based on it. |
| 21 | + |
| 22 | +## Running the LCM-LoRA-SDv1-5 Model on MaixPy MaixCAM |
| 23 | + |
| 24 | +### Model and Download Link |
| 25 | + |
| 26 | +If the `LCM-LoRA-SDv1-5` model is not present in the system directory `/root/models` by default, you need to download it manually. |
| 27 | + |
| 28 | + * Memory requirement: CMM memory 1 GiB. For details, refer to the [Memory Usage Documentation](../pro/memory.md) |
| 29 | + |
| 30 | + * Download link: https://huggingface.co/sipeed/lcm-lora-sdv1-5-maixcam2 |
| 31 | + |
| 32 | +For the download method, refer to the instructions in the [Qwen documentation](../pro/memory.md) |
| 33 | + |
| 34 | +### Running the Model with MaixPy |
| 35 | + |
| 36 | +> Note: MaixPy version `4.12.3` or above is required for support. |
| 37 | +
|
| 38 | +This example demonstrates how to use MaixPy to run the LCM-LoRA-SDv1-5 model for both text-to-image and image-to-image generation. |
| 39 | + |
| 40 | +```python |
| 41 | +from maix import sdv1_5 |
| 42 | + |
| 43 | +model = sdv1_5.SDV1_5("/root/models/lcm-lora-sdv1-5-maixcam2/ax620e_models") |
| 44 | +model.init(img2img=True) |
| 45 | +model.refer(prompt="A white dog.", save_path="/root/text2img.jpg") |
| 46 | +model.refer(prompt="Replace the dog with a cat.", init_image_path="/root/text2img.jpg", seed=1, save_path="/root/img2img.jpg") |
| 47 | + |
| 48 | +model.deinit() |
| 49 | +``` |
| 50 | +Output: |
| 51 | + |
| 52 | + |
| 53 | +Explanation: |
| 54 | +1. When performing text-to-image generation, you need to define a `prompt` describing the image you want. The prompt must be written in English, and the generated image will be saved to the path specified by `save_path`. |
| 55 | +2. When performing image-to-image generation, you need to define a prompt describing the content you want to generate`init_image_path` specifies the initial image,`seed` controls randomness in image generation |
| 56 | +The output image will be saved to the path specified by save_path. |
| 57 | +3. Since `sdv1_5` depends on several large Python packages, importing it may take some time. If you want the application to enter the UI quickly during development, you can use `importlib` and `threading` to load the module in the background. Example: |
| 58 | +```python |
| 59 | +from maix import time |
| 60 | +import importlib, threading |
| 61 | + |
| 62 | +class ModuleLoader(): |
| 63 | + def __load_module(self): |
| 64 | + self.module = importlib.import_module(self.module_name) |
| 65 | + self.load_ok = True |
| 66 | + pass |
| 67 | + def __init__(self, module_name: str): |
| 68 | + self.module_name = module_name |
| 69 | + self.module = None |
| 70 | + self.load_ok = False |
| 71 | + self.t = threading.Thread(target=self.__load_module) |
| 72 | + self.t.start() |
| 73 | + def try_get_module(self, block = True): |
| 74 | + if self.load_ok: |
| 75 | + return self.module |
| 76 | + if block: |
| 77 | + while not self.load_ok: |
| 78 | + time.sleep_ms(100) |
| 79 | + return self.module |
| 80 | + else: |
| 81 | + return None |
| 82 | + |
| 83 | +sdv1_5_loader = ModuleLoader("maix.sdv1_5") |
| 84 | +while True: |
| 85 | + sdv1_5 = sdv1_5_loader.try_get_module(block=False) |
| 86 | + if sdv1_5 is not None: |
| 87 | + break |
| 88 | + time.sleep_ms(1000) |
| 89 | +print(sdv1_5) |
| 90 | +``` |
| 91 | + |
| 92 | +### Running the Model with Command Line |
| 93 | + |
| 94 | +Refer to `launcher.py` in the model directory to run the model. |
| 95 | + |
| 96 | +#### Text-to-Image |
| 97 | +```shell |
| 98 | +cd lcm-lora-sdv1-5-maixcam2 |
| 99 | +python3 launcher.py --isize 256 --model_dir ax620e_models/ -o "ax620e_txt2img_axe.png" --prompt "a white dog" |
| 100 | +``` |
| 101 | + |
| 102 | +Parameter description: |
| 103 | +- `--isize`: Input image size, recommended value is 256 |
| 104 | +- `--model_dir`: Model directory |
| 105 | +- `-o`: Output image filename |
| 106 | +- `--prompt`: Description text; the model generates an image based on this description |
| 107 | + |
| 108 | +#### Image-to-Image |
| 109 | +```shell |
| 110 | +cd lcm-lora-sdv1-5-maixcam2 |
| 111 | +python3 launcher.py --init_image ax620e_models/img2img-init.png --isize 256 --model_dir ax620e_models/ --seed 1 --prompt "Change to black clothes" -o "ax620e_img2img_axe.png" |
| 112 | +``` |
| 113 | + |
| 114 | +Parameter description: |
| 115 | +- `--init_image`: Initial image; the model generates a new image based on this |
| 116 | +- `--isize`: Input image size, recommended value is 256 |
| 117 | +- `--model_dir`: Model directory |
| 118 | +- `--seed`: Random seed, controls randomness during image generation |
| 119 | +- `-o`: Output image filename |
| 120 | +- `--prompt`: Description text; the model generates an image based on this description |
| 121 | + |
| 122 | + |
0 commit comments