Important
🚧 This documentation is still under construction. The development of the upscaler is a ongoing activity. There might be small differenes in comparison of node and documentation.
Caution
Please note, that the node will run, only if an error in one Python package file is fixed. See the related section for informations on this topic.
This node uses the RealESRGAN model from xinntao [1]. This is my personal favourite upscaling model and upscaling approach.
What I am writing is valid for a computer on which as the operating system Linux is installed.
To run the upscaler one needs a NVIDIA graphic card.
Yesterday I added a bunch of things to the node. Now the node is a little bit more complicated than before. On the other hand side the node considers now everything which is possible to do with the node.
One needs a model for the upscaling. While installing the nodes some models from xinntao are downloaded and placed in the directory models so that one can start working with the upscaling.
Following models are compatible with the node [2]:
- RealESRGAN_x4plus.pth
- RealESRNet_x4plus.pth
- RealESRGAN_x2plus.pth
- ESRGAN_SRx4_DF2KOST_official-ff704c30.pth
One important setting depends on the model like the ones in the last section. The x2 and the x4 denotes the scale factor. This scale factor must fit the value of netscale. Otherwise one will get an error message.
One can set the scaling factor in steps of 0.1 in the node. For the moment there is no known limit for the scaling factor. I used for example an unrealistic scaling factor of 30.0 for upscaling of a test image. The limiting factor is in this case the size of the image file.
Figure 1 shows the node preview. Input is an image. Main output is the upscaled image. In addition to this one can output the local installation with respect to the GPUs. In case of an error the error can be printed too.
Figure 1: Main node preview
The node has following parameter
- scale_factor (FLOAT)
- tile_number (INT)
- tile_pad (INT)
- pre_pad (INT)
- fp_format (STRING)
- denoise (FLOAT)
- netscale (INT)
- gpu_id (LIST)
- model (LIST)
For an upscaling one needs in a first shot the scale factor and a model. The other parameter can be as they are. For a resolution of the image of 512 x 512 pixel this is in general valid.
Over the gpu_id the user can decide, which GPU should be used for the upscaling.
Depending on the hardware one needs the tiling. In my case I needs tiling for a resolution of 1024 upwards.
Figure 2: Model selection
The node can upscale images which have following formats
- JPG/JPEG
- PNG
- WEBP
Figure 3 shows the example workflow preview.
Figure 3: Workflow preview
Figure 4 shows the example workflow preview.
Figure 4: Workflow preview
If the tiling is not set (for larger images), one will get following error:
Figure 4a: Tiling problem preview
If the tiling is wrong set, one will get following error:
Figure 4b: Tiling problem preview
Use the ComfyUI Manager for the installation. Use my nick 'zentrocdot' or for 'ComfyUI_realESRGAN_Upscaler'
While first run I got an error message that the installation of the node failed. After first investigation I found the problem. Fix for the moment is as follows. Go into directory ComfyUI. Then search for a file.
find -name "degradations.py"
This looks like.
./venv/lib/python3.10/site-packages/basicsr/data/degradations.py
Open the file e.g.
nano ./venv/lib/python3.10/site-packages/basicsr/data/degradations.py
and change
from torchvision.transforms.functional_tensor import rgb_to_grayscale
to
from torchvision.transforms.functional import rgb_to_grayscale
After that, the node will work.
In the directory scripts there is a file called fix.bash. This can
be used to fix the bug in the Python package. The only one prerequiste for this
script is that ComfyUI is installed in a Python virtual envirnoment.
While working on the node I have also provoked errors for testing purposes. When the number of netscale is not matching the value of the model's netscale one got an error.
Figure 5: Error message preview
The error led to unexpected behaviour of the node. This was the reason to rethink the programming. Now I catch the error Over the output ERROR one can make such an error visible. To prevent misleading results the output is an empty image with text.
Limitation at the moment not known.
There are a plenty of things to to
Figure 6 and Figure 7 show two examples where I used two converted models for the upscaling. it depends on your personal taste which upscaler you like.
Figure 6: Upscaling using 4x-UltraSharp.pth
Figure 7: Upscaling using 4xNomos8kSC.pth
Special credits goes to xinntao (Xintao Wang) for his excellent work on ESRGAN and RealESRGAN models The results achieved are quite impressive.
Have fun. Be inspired!
[1] https://github.com/xinntao/Real-ESRGAN
[2] https://github.com/xinntao/Real-ESRGAN/blob/master/docs/model_zoo.md
[3] https://arxiv.org/abs/2107.10833
[4] https://github.com/zentrocdot/ESRGAN_RRBD_CONVERTER
