[Feature request] Deno Support

[omar2205]

Name of the feature
Support running in the Deno runtime.

Additional context
I tried all the tricks, esm.sh, unpkg, esm.run, and npm:@xenova/transformers, but nothing worked.

Using unpkg import I get

This browser lacks typed array (Uint8Array) support which is required 
by `buffer` v5.x. Use `buffer` v4.x if you require old browser support.

Using esm.sh

Uncaught SyntaxError: The requested module '/v114/pngjs@6.0.0/deno/pngjs.mjs' 
does not provide an export named 'PNG'

[xenova]

This seems to be an problem with onnxruntime itself: microsoft/onnxruntime#10913

The person who opened the issue did seem to find a workaround, so, maybe that will help too?

@josephrocca is quite active in the community, and since he opened up that issue, he might have some more insight.

[omar2205]

So @nestarz created a Deno onnx_runtime here https://github.com/nestarz/onnx_runtime

[xenova]

So @nestarz created a Deno onnx_runtime here https://github.com/nestarz/onnx_runtime

That's good to know - thanks! Have you tried overriding the dependency in your project?

[omar2205]

Not sure how. For example, this is my script file

  import * as t from 'npm:@xenova/transformers'
  
  const pipeline = t.default.pipeline
  
  let pipe = await pipeline('sentiment-analysis')
  
  let out = await pipe('I love transformers!')

This gets a new error, error: Uncaught Error: Not implemented: Worker from onnxruntime-web/1.14.0/dist/ort-web.node.js

[xenova]

Maybe this might help: https://www.reddit.com/r/Deno/comments/x9q9vp/how_do_you_patch_a_module_like_patchpackage_in/ ? I haven't used Deno before, so, I probably won't be able to provide much more help (other than googling or "chatgpt-ing" 🤣 )

[xenova]

Have you been able to get it working in Deno yet? If not, feel free to try with the latest version 2.0.0-alpha.0. This may fix it.

[omar2205]

Hey, I just tried it, using npm: and esm.sh and both got an error.
esm.sh:

Uncaught Error: Dynamic require of "../bin/napi-v3/linux/x64/onnxruntime_binding.node" is not supported

npm:

Uncaught Error:                                                                 
Something went wrong installing the "sharp" module                              
                                                                                
Cannot find module '../build/Release/sharp-linux-x64.node'
Require stack:

[takoyaro]

it looks like sharp doesn't support deno at the moment

[xenova]

@omar2205 Have you tried the steps outlined here: lovell/sharp#2583 (comment) ?

This is an apparent workaround until denoland/deno#16164 is made available.

[omar2205]

Thanks for the ping @xenova, I followed that and it worked, however, I'm met with Invalid URL: error.

import { pipeline } from 'npm:@xenova/transformers';
/* warning
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
napi_add_finalizer is not yet supported.
*/

let pipe = await pipeline('sentiment-analysis');

/* Error
No model specified. Using default model: "Xenova/distilbert-base-uncased-finetuned-sst-2-english".
Uncaught TypeError: Invalid URL: 'Xenova/distilbert-base-uncased-finetuned-sst-2-english/tokenizer.json'
*/

[xenova]

Okay great! That error is probably deno specific, because the check to see whether something is a valid URL is surrounded by a try-catch block (and works in other environments).

https://github.com/xenova/transformers.js/blob/0b403ce8e4cf4ab447f8f3b208920aa4e77dc499/src/utils/hub.js#L171-L179

While I look into it, you can get around it temporarily by disabling local models:

import { pipeline, env } from 'npm:@xenova/transformers';

env.allowLocalModels=false;

let pipe = await pipeline('sentiment-analysis');

[omar2205]

I tried to do that before, but it fails:

No model specified. Using default model: "Xenova/distilbert-base-uncased-finetuned-sst-2-english".
Uncaught TypeError: Invalid URL: 'Xenova/distilbert-base-uncased-finetuned-sst-2-english/tokenizer.json'

[xenova]

Hmm, it must be failing elsewhere then. Are you able to send the full stack trace?

[omar2205]

Here you go

No model specified. Using default model: "Xenova/distilbert-base-uncased-finetuned-sst-2-english".
Uncaught TypeError: Invalid URL: 'Xenova/distilbert-base-uncased-finetuned-sst-2-english/tokenizer.json'
    at getSerialization (ext:deno_url/00_url.js:88:11)
    at new URL (ext:deno_url/00_url.js:383:27)
    at new Request (ext:deno_fetch/23_request.js:297:25)
    at Cache.[[[matchAll]]] (ext:deno_cache/01_cache.js:230:11)
    at Cache.match (ext:deno_cache/01_cache.js:173:36)
    at getModelFile (file:///home/ubuntu/.cache/deno/npm/registry.npmjs.org/@xenova/transformers/2.0.0-alpha.2/src/utils/hub.js:347:32)
    at eventLoopTick (ext:core/01_core.js:166:11)
    at async getModelJSON (file:///home/ubuntu/.cache/deno/npm/registry.npmjs.org/@xenova/transformers/2.0.0-alpha.2/src/utils/hub.js:456:18)
    at async Promise.all (index 0)
    at async loadTokenizer (file:///home/ubuntu/.cache/deno/npm/registry.npmjs.org/@xenova/transformers/2.0.0-alpha.2/src/tokenizers.js:49:16)