Support different rtp header extentions and payload for each consumer.

[penguinol]

Feature Request

According to WHEP, when subscribing stream, client send Offer first, and server replay Answer then.
So the rtp header extention ids of each consumer may be different.
But in mediasoup, we send the same rtp header extentions to every consumer.
The purpose is to support any general WHEP client, so we can't motify the local sdp before setLocalDescrption.
I think #1107 do part of this, but i'm worring about the performence.

[ibc]

Can you please split these 2 feature requests into 2 different tickets? I'd like to comment about them separately.

[ibc]

Performance of the change suggested in #1107 will be good, it's explained in that ticket. And yes, it will allow custom extension ids per Consumer. However I assume we also need custom codec payload types per Consumer, am I wrong?

[penguinol]

Yes, the payload may also different between consumers.

[paulorlyk]

I was also attempting to make WHEP work with mediasoup.

This branch allows to set payload types and header extension IDs per consumer. It also has some other unrelated things (option to enable bandwidth feedback from simple consumers to producer, time offset calculation for abs-capture-time and shifting media server time base to the future to work around bug in Chrome) that should not affect WHIP/WHEP.

The problem I was not able to fully solve was that because mediasoup is ice-lite implementation, sometimes, if client is not polite, it is impossible to know that it is gone. This makes impossible to track resources. Client always have to explicitly unsubscribe or some other mechanism, like keep-alives, must be involved.

[s074]

I'm in the same boat. Need mediasoup to work with WHEP clients so I need it to support different codec payload ids for each consumer.

[MathMan05]

Spacebar needs this for VC support

[ibc]

Spacebar needs this for VC support

I don't know what VC means.

[MathMan05]

Voice Chat

[QuizzityMC]

Voice Chat

We really need this

[ibc]

We develop based on priority (urgent bugs first), based on what we primary need for our paid work, and then based on the free time we have to work on them. We cannot change that but donations help prioritising:

https://opencollective.com/mediasoup