-
Notifications
You must be signed in to change notification settings - Fork 45
Description
At the moment I am able to use Canny Edge Detection, Pose Estimation and Semantic Segmentation (all in JavaScript thanks to OpenCV.js) in diffusers.js for the annotation (pre-processing) of images that are used as input for various ControlNets. These are the only ones that I've managed to implement so far but there are more Annotator models such as Scribble, HED, M-LSD, etc that I have not found a JavaScript implementation for.
It would be useful to have these models in ONNX format so they can be called the same way as an UNET, VAE or other model. A script that can convert the Annotator models to ONNX format or the models directly in ONNX format so they can be used in diffusers.js would be an ideal solution if possible.
I believe the original implementation of the Annotator models is controlnet_aux.