webrtcbin and caps on sink pads

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

webrtcbin and caps on sink pads

Trey Hutcheson
Ok, so I know that before generating an offer/answer, that we need to wait on all sink pads to get caps from upstream. But we are encountering a problem: sometimes caps are never sent.

Example:
* webrtcbin1 is receiving audio and video from its remote peer. Each incoming media stream is terminated by a tee element. 
* I create a new webrtcbin instance, and branch off the tee's hanging off webrtcbin1; each branch contains a queue and then depayloads and repayloads the rtp, and connected via new sink pads on webrtcbin2. 

Repeat that process for each new webrtc peer/webrtcbin instance that needs to participate in the media session.

Problem #1: the first time a branch is created from the source tee, caps are sent downstream. However, new branches never receive caps. To get around that, we explicitly create a caps event and send it down the branch.

Problem #2: sometimes that caps event takes several seconds (2 seconds, 5 seconds, it's indeterminate) to actually get to the new sink pad. And I have no idea why. This delay creates all kinds of problems for us internally. 

Why is the tee element not sending caps down to new elements after they are linked? Is it supposed to? If it's not, is there any built in element that will function like a tee, cache the last known caps, and send them downstream when new src pads are linked?

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: webrtcbin and caps on sink pads

Olivier Crête-3
On Fri, 2021-03-19 at 12:20 -0500, Trey Hutcheson wrote:
Ok, so I know that before generating an offer/answer, that we need to wait on all sink pads to get caps from upstream. But we are encountering a problem: sometimes caps are never sent.

You can also skip all of that by just setting the exact codec you will receive as the codec preferences on the transceiver itself.



Example:
* webrtcbin1 is receiving audio and video from its remote peer. Each incoming media stream is terminated by a tee element. 
* I create a new webrtcbin instance, and branch off the tee's hanging off webrtcbin1; each branch contains a queue and then depayloads and repayloads the rtp, and connected via new sink pads on webrtcbin2. 

Repeat that process for each new webrtc peer/webrtcbin instance that needs to participate in the media session.

Problem #1: the first time a branch is created from the source tee, caps are sent downstream. However, new branches never receive caps. To get around that, we explicitly create a caps event and send it down the branch.


Problem #2: sometimes that caps event takes several seconds (2 seconds, 5 seconds, it's indeterminate) to actually get to the new sink pad. And I have no idea why. This delay creates all kinds of problems for us internally. 

Possibly blocking on a keyframe in the parser ?


Why is the tee element not sending caps down to new elements after they are linked? Is it supposed to? If it's not, is there any built in element that will function like a tee, cache the last known caps, and send them downstream when new src pads are linked?

Tee should do exactly that already.


-- 
Olivier Crête


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel