Connection time-out when camera infrequently sends RTP/RTCP packets

[bwevan]

We use gortsplib to handle ONVIF metadata streams, to receive motion detection events and such (see section 5.2.2.4 and 5.1.2.1.1 of the ONVIF streaming spec for more info).

We stream this metadata over TCP while video is streamed over UDP, so we have a separate gortsplib.Client instance for these streams.

However, some cameras are not guaranteed to frequently send data over RTP on this metadata stream if no events occur, and they do not send RTCP sender reports (specifically some Hikvision cameras).
This means that, if no events occur for the duration of the read timeout, the client will run into a time-out here:

gortsplib/client.go

Lines 1291 to 1295 in d0b37aa

	func (c *Client) isInTCPTimeout() bool {
	now := c.timeNow()
	lft := time.Unix(atomic.LoadInt64(c.tcpLastFrameTime), 0)
	return now.Sub(lft) >= c.ReadTimeout
	}

Because tcpLastFrameTime is only set when an RTP/RTCP packet is received:

gortsplib/client_media.go

Lines 303 to 324 in d0b37aa

	func (cm *clientMedia) readPacketRTPTCPPlay(payload []byte) bool {
	atomic.AddUint64(cm.bytesReceived, uint64(len(payload)))
	now := cm.c.timeNow()
	atomic.StoreInt64(cm.c.tcpLastFrameTime, now.Unix())
	return cm.readPacketRTP(payload, now)
	}
	func (cm *clientMedia) readPacketRTCPTCPPlay(payload []byte) bool {
	atomic.AddUint64(cm.bytesReceived, uint64(len(payload)))
	now := cm.c.timeNow()
	atomic.StoreInt64(cm.c.tcpLastFrameTime, now.Unix())
	if len(payload) > udpMaxPayloadSize {
	cm.onPacketRTCPDecodeError(liberrors.ErrClientRTCPPacketTooBig{L: len(payload), Max: udpMaxPayloadSize})
	return false
	}
	return cm.readPacketRTCPPlay(payload)
	}

A work-around is to bump the read timeout. For example, these Hikvision cameras send processor usage events every minute or so, so bumping the read timeout to >60 seconds will work.
However, this timeout is also used for dialing the initial TCP connection and in other places:

gortsplib/client.go

Line 1118 in d0b37aa

dialCtx, dialCtxCancel := context.WithTimeout(c.ctx, c.ReadTimeout)

And I would preferably have a shorter timeout there, to avoid hanging too long if the camera is completely unresponsive during stream setup.

Being able to set this check timeout separately from the read timeout or being able to disable the RTP/RTCP timeout check altogether (and relying on RTSP keep-alives only for liveness) would be nice. I can make a PR for this if needed.

[bwevan]

This is a very simple reproduction with a mock 'metadata' server that infrequently sends RTP packets: https://github.com/bwevan/gortsplib-timeout-issue

The issue isn't that exciting, but it makes it easier to see what I mean :)