Skip to content

Commit d47c2ae

Browse files
committed
Add more detailed webgpu support info
1 parent 22551da commit d47c2ae

File tree

1 file changed

+12
-3
lines changed

1 file changed

+12
-3
lines changed

src/routes/blog/post/chatbot-with-webllm-and-webgpu/+page.markdoc

Lines changed: 12 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ When you hear "LLM," you probably think of APIs, tokens, and cloud infrastructur
1414

1515
Local LLMs running inside the browser were nearly impossible just a year ago. But thanks to new technologies like WebLLM and WebGPU, you can now load a full language model into memory, run it on your device, and have a real-time conversation, all without a server.
1616

17-
In this guide, we'll build a local chatbot that runs entirely in the browser. No backend. No API keys. By the end, you should have a good understanding of WebLLM and WebGPU, and will have built an app that looks and functions like this:
17+
In this guide, we'll build a local chatbot that runs entirely in the browser. No backend. No API keys. By the end, you should have a good understanding of [WebLLM](https://webllm.mlc.ai/) and [WebGPU](https://developer.mozilla.org/en-US/docs/Web/API/WebGPU_API), and will have built an app that looks and functions like this:
1818

1919
![WebLLM and WebGPU chat app demo](/images/blog/chatbot-with-webllm-and-webgpu/webllm-webgpu-chat-app-demo.gif)
2020

@@ -46,8 +46,15 @@ In our case, WebGPU lets the browser perform the heavy math required to generate
4646
Here's what WebGPU does for us:
4747

4848
- **Performance**: Runs faster than JavaScript or even WebAssembly for these workloads
49-
- **Accessibility**: Available in major browsers like Chrome, Edge, and Firefox (with a flag)
5049
- **GPU-first**: Designed from the ground up for compute, not just rendering
50+
- **Accessibility**: Available across different browsers, though support varies by platform. As of 2025:
51+
52+
- **Chrome/Edge**: Fully supported on Windows, Mac, and ChromeOS since version 113. On Linux, it requires enabling the `chrome://flags/#enable-unsafe-webgpu` flag
53+
- **Firefox**: Available in Nightly builds by default, with stable release tentatively planned for Firefox 141
54+
- **Safari**: Available in Safari Technology Preview, with support in iOS 18 and visionOS 2 betas via Feature Flags
55+
- **Android**: Chrome 121+ supports WebGPU on Android
56+
57+
For production applications, you should include proper WebGPU feature detection and provide fallbacks for unsupported browsers.
5158

5259
Together, WebLLM and WebGPU allow us to do something powerful: load a quantized language model directly in the browser and have real-time chat without any backend server.
5360

@@ -126,7 +133,7 @@ In the HTML file, we've created a chat interface with controls for model selecti
126133

127134
### Model selection
128135

129-
Notice that in the `div` with class `controls`, we have a `select` element for model selection and a `button` for loading the model. Here are the detailed specifications for each model:
136+
Notice that in the `div` with class `controls`, we have a `select` element for model selection and a `button` for loading the model. Here are the specifications for each model:
130137

131138
| Model | Parameters | Q4 file size (MB) | VRAM needed (MB) |
132139
| ------------ | ------------ | ----------------- | ---------------- |
@@ -235,6 +242,8 @@ if (!navigator.gpu) {
235242
}
236243
```
237244

245+
This check is crucial because WebGPU availability varies significantly across browsers and platforms. The code will gracefully fail if WebGPU isn't available, allowing you to show appropriate fallback content to users.
246+
238247
Then we download and initialize the model:
239248

240249
```js

0 commit comments

Comments
 (0)