Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
189 changes: 189 additions & 0 deletions docs/src/content/docs/guides/linux-media.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
---
title: Linux Media Playback
sidebar:
order: 12
---

import {Badge} from '@astrojs/starlight/components';

Relevant Platforms: <Badge text="Linux" variant="note" />
<br/>

On Linux, Wails applications use WebKitGTK for rendering web content. WebKitGTK delegates media playback (video and audio) to GStreamer, which handles the actual decoding and rendering of media files.

## The GStreamer Protocol Issue

GStreamer operates independently from WebKit's custom URI scheme handling. While Wails registers a `wails://` protocol handler with WebKit for serving your application's assets, GStreamer doesn't have access to this handler. This means when an HTML `<video>` or `<audio>` element tries to load a media file from your bundled assets, GStreamer cannot resolve the `wails://` URL.

This is a [known upstream WebKit issue](https://bugs.webkit.org/show_bug.cgi?id=146351) that has been open since 2015.

### Symptoms

Without the workaround, you might experience:
- Video elements showing a black screen or failing to load
- Audio elements remaining silent
- Browser console errors about failed media requests
- Media elements stuck in a "loading" state

## Automatic Workaround

Wails automatically works around this limitation by injecting JavaScript that intercepts media elements and converts their sources to blob URLs. This happens transparently - you don't need to modify your frontend code.

### How It Works

1. When your application loads, Wails injects platform-specific JavaScript
2. This JavaScript monitors the DOM for `<video>` and `<audio>` elements
3. When a media element is found with a relative or `wails://` URL, the script:
- Fetches the media file using the standard `fetch()` API (which works with the Wails asset server)
- Creates a blob URL from the response
- Replaces the element's `src` with the blob URL
4. GStreamer can then play the blob URL normally

### What Gets Intercepted

The workaround intercepts:
- `<video>` elements with `src` attributes
- `<audio>` elements with `src` attributes
- `<source>` elements inside `<video>` and `<audio>` elements
- Elements added dynamically via JavaScript
- Elements with `src` set via JavaScript after page load

## Configuration Options

You can configure the GStreamer workaround behavior using `LinuxOptions`:

```go
app := application.New(application.Options{
Name: "My App",
Linux: application.LinuxOptions{
// Disable the GStreamer workaround entirely
DisableGStreamerFix: false,

// Enable caching of blob URLs for better performance
EnableGStreamerCaching: true,
},
})
```

### DisableGStreamerFix

Set to `true` to disable the automatic media interception. You might want to do this if:
- You're only using external media URLs (http/https)
- You're implementing your own media handling solution
- You're debugging media-related issues

```go
Linux: application.LinuxOptions{
DisableGStreamerFix: true,
},
```

### EnableGStreamerCaching

When enabled, blob URLs are cached so that the same media file doesn't need to be fetched and converted multiple times. This improves performance when:
- The same media is played repeatedly
- Multiple elements reference the same media file
- Media elements are recreated (e.g., in a SPA with component remounting)

```go
Linux: application.LinuxOptions{
EnableGStreamerCaching: true,
},
```

:::note[Memory Considerations]
Enabling caching keeps blob URLs in memory for the lifetime of the page. For applications with many large media files, consider whether the memory trade-off is worthwhile for your use case.
:::

## Working with Media Files

### Bundled Assets

Place media files in your frontend's public or assets directory:

```
frontend/
public/
video.mp4
audio.ogg
src/
main.js
```

Reference them with relative URLs:

```html
<video controls>
<source src="/video.mp4" type="video/mp4">
</video>

<audio controls>
<source src="/audio.ogg" type="audio/ogg">
</audio>
```

### Supported Formats

The supported media formats depend on your GStreamer installation. Common formats include:
- Video: MP4 (H.264), WebM (VP8/VP9), OGG (Theora)
- Audio: MP3, OGG (Vorbis), WAV, FLAC

To check available codecs on your system:

```bash
gst-inspect-1.0 --version
gst-inspect-1.0 | grep -i decoder
```

### Installing Codecs

On Ubuntu/Debian:
```bash
sudo apt install gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly
```

On Fedora:
```bash
sudo dnf install gstreamer1-plugins-good gstreamer1-plugins-bad-free gstreamer1-plugins-ugly-free
```

On Arch Linux:
```bash
sudo pacman -S gst-plugins-good gst-plugins-bad gst-plugins-ugly
```

## Example Application

See the [audio-video example](https://github.com/wailsapp/wails/tree/v3-alpha/v3/examples/audio-video) for a complete working example that demonstrates:
- Video playback with bundled MP4 files
- Audio playback with bundled MP3 files
- Using the `@wailsio/runtime` npm module
- Proper project structure for media assets

## Troubleshooting

### Media Not Playing

1. **Check GStreamer installation**: Ensure GStreamer and required plugins are installed
2. **Check the console**: Look for JavaScript errors or failed network requests
3. **Verify file paths**: Ensure media files are in the correct location and accessible
4. **Check codec support**: Verify GStreamer has the necessary decoder for your media format

### Performance Issues

If media playback is sluggish:
1. Enable caching with `EnableGStreamerCaching: true`
2. Consider using more efficient codecs (e.g., H.264 instead of VP9)
3. Reduce media file sizes through compression

### Debugging

To see what the media interceptor is doing, check the browser console. The interceptor logs its actions when media sources are converted.

You can also temporarily disable the fix to compare behavior:

```go
Linux: application.LinuxOptions{
DisableGStreamerFix: true,
},
```
15 changes: 15 additions & 0 deletions docs/src/content/docs/quick-start/installation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -144,18 +144,33 @@ If `wails3 doctor` passes, you're done. [Skip to First App →](/quick-start/fir
sudo apt update
sudo apt install build-essential pkg-config libgtk-3-dev libwebkit2gtk-4.0-dev
```

**For audio/video playback** (optional but recommended):
```bash
sudo apt install gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly
```
</TabItem>

<TabItem label="Fedora">
```bash
sudo dnf install gcc pkg-config gtk3-devel webkit2gtk4.0-devel
```

**For audio/video playback** (optional but recommended):
```bash
sudo dnf install gstreamer1-plugins-good gstreamer1-plugins-bad-free gstreamer1-plugins-ugly-free
```
</TabItem>

<TabItem label="Arch">
```bash
sudo pacman -S base-devel gtk3 webkit2gtk
```

**For audio/video playback** (optional but recommended):
```bash
sudo pacman -S gst-plugins-good gst-plugins-bad gst-plugins-ugly
```
</TabItem>

<TabItem label="Other">
Expand Down
3 changes: 3 additions & 0 deletions v3/UNRELEASED_CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,15 @@ After processing, the content will be moved to the main changelog and this file
-->

## Added
- Add `DisableGStreamerFix` and `EnableGStreamerCaching` options to `LinuxOptions` for controlling media playback workaround (#4412) by @leaanthony
- Add `audio-video` example demonstrating HTML5 media playback with the `@wailsio/runtime` npm module by @leaanthony
<!-- New features, capabilities, or enhancements -->

## Changed
<!-- Changes in existing functionality -->

## Fixed
- Fix video and audio playback on Linux/WebKitGTK by intercepting media elements and converting wails:// URLs to blob URLs (#4412) by @leaanthony
<!-- Bug fixes -->

## Deprecated
Expand Down
38 changes: 38 additions & 0 deletions v3/examples/audio-video/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Audio/Video Example

This example demonstrates HTML5 audio and video playback using the `@wailsio/runtime` npm module.

## Linux Notes

On Linux, WebKitGTK uses GStreamer for media playback. GStreamer doesn't have a URI handler for the `wails://` protocol, which means media files served from the bundled assets won't play directly.

Wails automatically works around this limitation by intercepting media elements and converting their sources to blob URLs. This happens transparently - you don't need to change your code.

See the [Linux-specific documentation](https://wails.io/docs/guides/linux-media) for details on:
- How the GStreamer workaround works
- How to disable it if needed (`DisableGStreamerFix`)
- How to enable caching for better performance (`EnableGStreamerCaching`)

## Building

```bash
cd frontend
npm install
npm run build
cd ..
go build
./audio-video
```

## Development

For development with hot-reload:

```bash
# Terminal 1: Run Vite dev server
cd frontend
npm run dev

# Terminal 2: Run Go app with dev server URL
FRONTEND_DEVSERVER_URL=http://localhost:5173 go run .
```
Loading
Loading