Using background-removal pipeline produces images with 50% opacity

[LuSrodri]

Question

I have a issue using the background-removal pipeline. Some models returns the exacly same image, but 50% opacite (RGBA: [X, Y, Z, 127]). So other models, returns an error like this: Uncaught Error: Unsupported model type: null transformers:1:670067.

How can I procede?

[pikilipita]

I noticed the same behavior: returning 127 when it should be zero.
I am using this formula to convert it back to 0 --> 255

var v = (pixel - 127)*255/59;
v = Math.round(v);
if (v < 0)v = 0;
if (v> 255)v = 255;

Also I noticed that the model onnx-community/BEN2-ONNX does not work well when using webGPU.
here are results using this model, with and without webGPU:

(macOS - chrome 134 - transformers 3.4.0)

[LuSrodri]

I don't think so. All the single pixel be the exaclt same, but 50% transparent, so, the models will have not results. A model with this behavior is modnet. A model that result in an error is the RMRG-2.0, for example.

[pikilipita]

I don't think so. All the single pixel be the exaclt same, but 50% transparent, so, the models will have not results. A model with this behavior is modnet. A model that result in an error is the RMRG-2.0, for example.

try onnx-community/ormbg-ONNX

[LuSrodri]

Wow!! You are right. It works!

I don't remember about this pos-processing reading the documentations, it's weird. Any way, Thanks so much.

[pikilipita]

It's not documented but likely a bug introduced in version 3.4.0

[xenova]

Thanks for the report. After further investigation, this looks to be a bug in onnxruntime when running the sigmoid function - specifically affecting models that have a sigmoid node before the output (only in CPU/WASM, not WebGPU).

Before	After (fixed)

See microsoft/onnxruntime#23943 for more information.

This will be fixed by #1255

[xenova]

So other models, returns an error like this: Uncaught Error: Unsupported model type: null

@LuSrodri Could you provide a list of models that have this error?

[LuSrodri]

@xenova I think the biggest model that doesn't works is the RMBG-2.0 by Briaai. (https://huggingface.co/briaai/RMBG-2.0)

When I calling something like this:

import { pipeline } from 'https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.4.0';
pipeline('background-removal', 'briaai/RMBG-2.0', { device: "webgpu" })

Returns the follow error:

transformers@3.4.0:1 Uncaught (in promise) Error: Unsupported model type: null
    at qd.from_pretrained (transformers@3.4.0:1:563598)
    at async transformers@3.4.0:1:670067

I don't know if this model has yet support, but I follow this website: https://huggingface.co/models?library=transformers.js&other=background-removal&sort=trending

[pikilipita]

Thanks for fixing it, does it also solve the webGPU error I reported? ( see image posted in the 2nd comment )

[xenova]

Thanks for fixing it, does it also solve the webGPU error I reported? ( see image posted in the 2nd comment )

Possibly, but I'm unable to test since I don't have the original image. Otherwise, it might be due to WebGPU precision issues.

[xenova]

So other models, returns an error like this: Uncaught Error: Unsupported model type: null

This is due to a misconfiguration in the config.json of the model. I attempted to fix it here, but this change needed to be reverted due to compatibility with transformers (python)

[xenova]

The main issue is resolved in #1255, so I will close this issue. The secondary issue is unrelated to the Transformers.js codebase and is rather a misconfiguration in the model repo.