Go SDK AV1 Publishing: NewLocalSampleTrack leads to "codec does not have a payloader" error despite SDPFmtpLine #778
Description
Hi LiveKit Team,
I'm trying to publish an AV1 video track to a LiveKit room using the Go SDK (v2.11.3) by piping FFmpeg output. While the basic connection and track publishing seem to initiate correctly, I'm consistently encountering an SDP negotiation error, preventing clients from subscribing successfully.
Environment:
LiveKit Server: livekit/livekit-server:latest Docker image (running with --dev --bind 0.0.0.0 on custom ports due to other services).
Go SDK: github.com/livekit/server-sdk-go/v2 v2.11.3.
Go App OS: Linux (Ubuntu likely, based on ufw).
FFmpeg: Using libsvtav1 for CPU encoding, outputting IVF format to stdout.
Client: Next.js app using @livekit/components-react v2.9.0 and livekit-client v2.9.9.
Network: Local network setup (192.168.1.x), using a separate coturn Docker container for TURN configured on the client side via rtcConfig. Firewall ports for LiveKit and Coturn (UDP ranges, TCP ports) have been opened.
What I'm Trying to Do:
My Go application connects to a LiveKit room as a publisher. It runs an FFmpeg process that:
Decodes an H.264 RTSP sub-stream (using HWAccel if available, e.g., cuda).
Encodes the video to AV1 using libsvtav1 (CPU encoding for testing) with specific resolution/bitrate settings.
Outputs the AV1 stream in IVF format to stdout.
My Go application reads the IVF stream from FFmpeg's stdout, parses the IVF frames, and uses lksdk.NewLocalSampleTrack followed by WriteSample to feed the AV1 data into the Go SDK track, which is then published to the room using PublishTrack.
How I'm Creating the Track:
Go
// Inside StartLayer function in main.go
codecCapability := webrtc.RTPCodecCapability{
MimeType: webrtc.MimeTypeAV1,
ClockRate: 90000,
Channels: 0,
SDPFmtpLine: "profile-id=0;level-idx=0", // Tried both "profile-id=0" and this
RTCPFeedback: []webrtc.RTCPFeedback{
{Type: "nack", Parameter: ""},
{Type: "nack", Parameter: "pli"},
{Type: "ccm", Parameter: "fir"},
{Type: "goog-remb", Parameter: ""},
},
}
log.Printf("%s NewLocalSampleTrack calling. Codec: %+v", logPrefix, codecCapability) // Added log
track, err := lksdk.NewLocalSampleTrack(codecCapability)
// ... error handling ...
log.Printf("%s LocalSampleTrack (AV1) created. Track ID: %s", logPrefix, track.ID()) // Added log
publishOpts := &lksdk.TrackPublicationOptions{
Name: trackName, // e.g., "av1-kamera1-low"
Source: livekit.TrackSource_CAMERA,
VideoWidth: int(definition.Width),
VideoHeight: int(definition.Height),
}
log.Printf("%s Calling LocalParticipant.PublishTrack. Track ID: %s, Options: %+v", logPrefix, track.ID(), publishOpts) // Added log
publication, pubErr := si.room.LocalParticipant.PublishTrack(track, publishOpts)
if pubErr != nil {
log.Printf("%s ERROR: PublishTrack failed: %v.", logPrefix, pubErr) // Added log
// ... handle error ...
} else {
log.Printf("%s PublishTrack SUCCESS. Publication SID: %s", logPrefix, publication.SID()) // Added log
}
Observed Behavior:
The Go application connects successfully to the LiveKit room.
FFmpeg starts correctly, encodes AV1, and outputs IVF data.
The Go app reads the IVF header and starts reading/writing AV1 frames using WriteSample. Logs confirm the first frame is read and written successfully.
The PublishTrack call in the Go SDK returns successfully, providing a Publication SID.
However, immediately after the successful PublishTrack call, the Go application logs show:
logger.go:456: "msg"="could not set remote description" "error"="the requested codec does not have a payloader"
(This log appears originates from within the Go SDK after publishing).
The Next.js client successfully gets a token and connects its WebSocket.
The client detects the published track (av1-kamera1-low).
The client fails to establish the PeerConnection for media, showing the error ConnectionError: could not establish pc connection in the browser console.
LiveKit server logs confirm ICE negotiation failure ("state": "failed" for candidate pairs) and that the client eventually closes the connection ("closedByClient": true).
Expected Behavior:
I expect the AV1 track published via the Go SDK to be successfully negotiated (SDP) and for the client to establish a PeerConnection (potentially via TURN) and display the AV1 video stream.
What I've Tried:
Ensured the correct MimeTypeAV1 is used in NewLocalSampleTrack.
Added SDPFmtpLine: "profile-id=0" based on common AV1 SDP requirements.
Also tried SDPFmtpLine: "profile-id=0;level-idx=0" based on further research. Neither resolved the "payloader" error.
Confirmed FFmpeg is outputting valid IVF AV1 data (header read successfully, frames are being processed and sent via WriteSample).
Set up a separate coturn TURN server in Docker and configured the Next.js client's rtcConfig within connectOptions to use it. Firewall ports are open for both LiveKit and Coturn UDP/TCP traffic. The Coturn server itself starts without errors after fixing a config comment issue. However, the connection still fails at the PeerConnection stage, likely preceded/caused by the "payloader" error during SDP negotiation.
Question:
Is publishing AV1 video tracks (specifically via NewLocalSampleTrack and WriteSample with external data like FFmpeg output) fully supported and considered stable in the Go SDK v2.11.3? The AV1 tracking issue mentioned Go SDK support was "work is planned", which might explain the "payloader" error.
Is the way I'm providing the SDPFmtpLine in RTPCodecCapability the correct method for ensuring proper SDP negotiation for AV1 with the Go SDK? Are there other parameters required?
Could the "codec does not have a payloader" error be masking the underlying ICE/PeerConnection issue, or is it the primary blocker preventing successful SDP negotiation and thus stopping the ICE process from completing correctly?
Any guidance on how to resolve the "payloader" error when publishing AV1 with the Go SDK, or confirmation on the current support status, would be greatly appreciated.
Thanks!