Segment-Everything-Everywhere-All-At-Once model addition

[Heinrik-20]

Model description

Hi Xenova,

I've been using transformers.js recently, and found the package to be useful and complete. For some context, I'm trying to build an application that allows myself to run segmentation tasks with texts, spatial querying, as well as image reference querying. In order to do so, I'll need to load three models into the application, that can be quite RAM consuming.

I've recently found this model by UX-Decoder, where they have trained a model that does the above all in one model. I was wondering if you could add this into the transformers.js library?

I know this model is on huggingface, but I'm not too sure if the model can be exported to ONNX with Optimum.

Cheers,
Heinrik

Prerequisites

• The model is supported in Transformers (i.e., listed here)
• The model can be exported to ONNX with Optimum (i.e., listed here)

Additional information

No response

Your contribution

I'm more than happy to help get this model into the library. Are there any docs you would recommend for me to get started?