1.17.9 - webrtcbin sdp answer missing ssrc's?

[Trey Hutcheson]

I have a media server that currently runs against gstreamer 1.16.2. When I run that same code on top of the current 1.17.9 master, I sometimes get no media to my remote peers. 

The workflow is simple. The browser provides the offer, and webrtcbin provides the answer (signaling is our own custom app). Looking at the sdp, I see that in the cases where media never starts flowing that the sdp answer does not include ssrc's. 

Example fragment from good sdp answer:

m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=ice-ufrag:fENRu4J+ohkX+c21z/3vzhvQ0a7LMqrX
a=ice-pwd:64ese6hPqiLwjB8QytOBokIXJGqf37Az
a=mid:0
a=rtcp-mux
a=setup:active
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 ccm fir
a=framerate:30
a=ssrc:1009469877 msid:user286747991@host-a7789f8b webrtctransceiver5
a=ssrc:1009469877 cname:user286747991@host-a7789f8b

Example fragment from bad sdp answer:

m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=ice-ufrag:2t2Wy+2pPcDPVPmVyixyMH2O/s63DFiX
a=ice-pwd:gPMprJzvik/ulzX3S92hcsGsdBMcYfgd
a=mid:0
a=rtcp-mux
a=setup:active
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 ccm fir
a=sendonly

Notice that the second fragment does not include a=ssrc lines (nor does it include the framerate attribute). 

This is troubling because the issue is intermittent. It happens with great frequency. If, however, I set GST_DEBUG to 4, then it happens far less often, though still occasionally. That would indicate to me it is some internal race condition?

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le jeudi 27 août 2020 à 11:50 -0500, Trey Hutcheson a écrit :

> I have a media server that currently runs against gstreamer 1.16.2.
> When I run that same code on top of the current 1.17.9 master, I
> sometimes get no media to my remote peers.
>
> The workflow is simple. The browser provides the offer, and webrtcbin
> provides the answer (signaling is our own custom app). Looking at the
> sdp, I see that in the cases where media never starts flowing that
> the sdp answer does not include ssrc's.
>
> Example fragment from good sdp answer:
> m=video 9 UDP/TLS/RTP/SAVPF 96
> c=IN IP4 0.0.0.0
> a=ice-ufrag:fENRu4J+ohkX+c21z/3vzhvQ0a7LMqrX
> a=ice-pwd:64ese6hPqiLwjB8QytOBokIXJGqf37Az
> a=mid:0
> a=rtcp-mux
> a=setup:active
> a=rtpmap:96 VP8/90000
> a=rtcp-fb:96 nack pli
> a=rtcp-fb:96 ccm fir
> a=framerate:30
> a=ssrc:1009469877 msid:user286747991@host-a7789f8b webrtctransceiver5
> a=ssrc:1009469877 cname:user286747991@host-a7789f8b
>
> Example fragment from bad sdp answer:
> m=video 9 UDP/TLS/RTP/SAVPF 96
> c=IN IP4 0.0.0.0
> a=ice-ufrag:2t2Wy+2pPcDPVPmVyixyMH2O/s63DFiX
> a=ice-pwd:gPMprJzvik/ulzX3S92hcsGsdBMcYfgd
> a=mid:0
> a=rtcp-mux
> a=setup:active
> a=rtpmap:96 VP8/90000
> a=rtcp-fb:96 nack pli
> a=rtcp-fb:96 ccm fir
> a=sendonly
>
> Notice that the second fragment does not include a=ssrc lines (nor
> does it include the framerate attribute).
>
> This is troubling because the issue is intermittent. It happens with
> great frequency. If, however, I set GST_DEBUG to 4, then it happens
> far less often, though still occasionally. That would indicate to me
> it is some internal race condition?

Consider using GST_DEBUG="webrtcbin:7,3" or similar log level. Be aware
that you need to wait for the sources to have received data before
proceeding with an offer or answer. This corner case has multiple FIXME
in the webrtcbin. That is worked reliable without special care in 1.16
was likely just luck.

Nicolas

> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Trey Hutcheson]

I'm sorry, but what do you mean that I have to wait for the sources to have received data before proceeding with the offer or answer?

Here's the workflow:

1) Pipeline is created. Initially it contains a bin that has a videotestsrc that tee's to a fakesink. 

2) A new bin is created that contains a webrtcbin element. The videotestsrc is linked to this bin, and encoded into vp8 along the way. 

3) remote peer (chrome) creates and submits an sdp offer, including ice candidates (non-trickle workflow)

4) I set the remote description of type offer, then manually extract ice candidates from the sdp add them

5) i then create an sdp answer, while also listening for locally gathered candidates. 

6) I return the sdp and ice candidates via the signalling channel and the remote peer uses it as the remote description

7) Media flows. Video is rendered. Etc etc etc.

This workflow worked 100% of the time, over hundreds and hundreds of iterations, for the past several months on 1.16. 

On Thu, Aug 27, 2020 at 3:16 PM Nicolas Dufresne <[hidden email]> wrote:

Le jeudi 27 août 2020 à 11:50 -0500, Trey Hutcheson a écrit :
> I have a media server that currently runs against gstreamer 1.16.2.
> When I run that same code on top of the current 1.17.9 master, I
> sometimes get no media to my remote peers.
>
> The workflow is simple. The browser provides the offer, and webrtcbin
> provides the answer (signaling is our own custom app). Looking at the
> sdp, I see that in the cases where media never starts flowing that
> the sdp answer does not include ssrc's.
>
> Example fragment from good sdp answer:
> m=video 9 UDP/TLS/RTP/SAVPF 96
> c=IN IP4 0.0.0.0
> a=ice-ufrag:fENRu4J+ohkX+c21z/3vzhvQ0a7LMqrX
> a=ice-pwd:64ese6hPqiLwjB8QytOBokIXJGqf37Az
> a=mid:0
> a=rtcp-mux
> a=setup:active
> a=rtpmap:96 VP8/90000
> a=rtcp-fb:96 nack pli
> a=rtcp-fb:96 ccm fir
> a=framerate:30
> a=ssrc:1009469877 msid:user286747991@host-a7789f8b webrtctransceiver5
> a=ssrc:1009469877 cname:user286747991@host-a7789f8b
>
> Example fragment from bad sdp answer:
> m=video 9 UDP/TLS/RTP/SAVPF 96
> c=IN IP4 0.0.0.0
> a=ice-ufrag:2t2Wy+2pPcDPVPmVyixyMH2O/s63DFiX
> a=ice-pwd:gPMprJzvik/ulzX3S92hcsGsdBMcYfgd
> a=mid:0
> a=rtcp-mux
> a=setup:active
> a=rtpmap:96 VP8/90000
> a=rtcp-fb:96 nack pli
> a=rtcp-fb:96 ccm fir
> a=sendonly
>
> Notice that the second fragment does not include a=ssrc lines (nor
> does it include the framerate attribute).
>
> This is troubling because the issue is intermittent. It happens with
> great frequency. If, however, I set GST_DEBUG to 4, then it happens
> far less often, though still occasionally. That would indicate to me
> it is some internal race condition?

Consider using GST_DEBUG="webrtcbin:7,3" or similar log level. Be aware
that you need to wait for the sources to have received data before
proceeding with an offer or answer. This corner case has multiple FIXME
in the webrtcbin. That is worked reliable without special care in 1.16
was likely just luck.

Nicolas

> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Matthew Waters]

On 28/8/20 6:54 am, Trey Hutcheson wrote:

I'm sorry, but what do you mean that I have to wait for the sources to have received data before proceeding with the offer or answer?

Here's the workflow:

1) Pipeline is created. Initially it contains a bin that has a videotestsrc that tee's to a fakesink. 

2) A new bin is created that contains a webrtcbin element. The videotestsrc is linked to this bin, and encoded into vp8 along the way. 

3) remote peer (chrome) creates and submits an sdp offer, including ice candidates (non-trickle workflow)

4) I set the remote description of type offer, then manually extract ice candidates from the sdp add them

5) i then create an sdp answer, while also listening for locally gathered candidates. 

6) I return the sdp and ice candidates via the signalling channel and the remote peer uses it as the remote description

7) Media flows. Video is rendered. Etc etc etc.

Depending on your code, there may be a race between 2 and 5 of the caps reaching a webrtcbin sink pad.  You can workaround by waiting for the later of notify::caps being emitted on the webrtcbin sink pad and the successful setting of the offer before you ask to create the answer.

This workflow worked 100% of the time, over hundreds and hundreds of iterations, for the past several months on 1.16. 

On Thu, Aug 27, 2020 at 3:16 PM Nicolas Dufresne <[hidden email]> wrote:

Le jeudi 27 août 2020 à 11:50 -0500, Trey Hutcheson a écrit :
> I have a media server that currently runs against gstreamer 1.16.2.
> When I run that same code on top of the current 1.17.9 master, I
> sometimes get no media to my remote peers.
>
> The workflow is simple. The browser provides the offer, and webrtcbin
> provides the answer (signaling is our own custom app). Looking at the
> sdp, I see that in the cases where media never starts flowing that
> the sdp answer does not include ssrc's.
>
> Example fragment from good sdp answer:
> m=video 9 UDP/TLS/RTP/SAVPF 96
> c=IN IP4 0.0.0.0
> a=ice-ufrag:fENRu4J+ohkX+c21z/3vzhvQ0a7LMqrX
> a=ice-pwd:64ese6hPqiLwjB8QytOBokIXJGqf37Az
> a=mid:0
> a=rtcp-mux
> a=setup:active
> a=rtpmap:96 VP8/90000
> a=rtcp-fb:96 nack pli
> a=rtcp-fb:96 ccm fir
> a=framerate:30
> a=ssrc:1009469877 msid:user286747991@host-a7789f8b webrtctransceiver5
> a=ssrc:1009469877 cname:user286747991@host-a7789f8b
>
> Example fragment from bad sdp answer:
> m=video 9 UDP/TLS/RTP/SAVPF 96
> c=IN IP4 0.0.0.0
> a=ice-ufrag:2t2Wy+2pPcDPVPmVyixyMH2O/s63DFiX
> a=ice-pwd:gPMprJzvik/ulzX3S92hcsGsdBMcYfgd
> a=mid:0
> a=rtcp-mux
> a=setup:active
> a=rtpmap:96 VP8/90000
> a=rtcp-fb:96 nack pli
> a=rtcp-fb:96 ccm fir
> a=sendonly
>
> Notice that the second fragment does not include a=ssrc lines (nor
> does it include the framerate attribute).
>
> This is troubling because the issue is intermittent. It happens with
> great frequency. If, however, I set GST_DEBUG to 4, then it happens
> far less often, though still occasionally. That would indicate to me
> it is some internal race condition?

Consider using GST_DEBUG="webrtcbin:7,3" or similar log level. Be aware
that you need to wait for the sources to have received data before
proceeding with an offer or answer. This corner case has multiple FIXME
in the webrtcbin. That is worked reliable without special care in 1.16
was likely just luck.

Nicolas

> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel