what different with mlc-ai/web-llm

[novvoo]

https://github.com/mlc-ai/web-llm

[cnrpman]

Same question.
Also WebGPT is usually referring this paper in the field of LLM: https://arxiv.org/abs/2112.09332v3
Hope clarifying the project name decision

[0hq]

Sure.

WebLLM is using TVM and compiling from C++ to WebAssembly, which is more performant but vastly more complicated. This is built to be both a solid educational reference as well as a place to start more complex projects from, being only written in a small amount of Javascript.

Considering using Javascript here has minor performance impacts vs. the simplicity it provides (mostly all computation is done on WebGPU, meaning Javascript isn't involved), this is preferable in various circumstances.

[novvoo]

Sure.

WebLLM is using TVM and compiling from C++ to WebAssembly, which is more performant but vastly more complicated. This is built to be both a solid educational reference as well as a place to start more complex projects from, being only written in a small amount of Javascript.

Considering using Javascript here has minor performance impacts vs. the simplicity it provides (mostly all computation is done on WebGPU, meaning Javascript isn't involved), this is preferable in various circumstances.

the same as this proj? https://xenova.github.io/transformers.js

[cnrpman]

@jingslunt It's using onnxruntimeweb, mature but no webGPU backend for now.

[novvoo]

@jingslunt It's using onnxruntimeweb, mature but no webGPU backend for now.
webgpu merged
microsoft/onnxruntime#14579