webrtcbin only receives single track from browser

[Soebirk, Thorsten Nordholm]

I am working on an application that should receive both audio and video from a browser using webrtcbin. My application is modelled on WebrtcSendRecv.java (https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/gst-java/src/main/java/WebrtcSendRecv.java), except that it is receive-only and it works in passive mode (receives a call by waiting for an offer, rather than making a call by sending an offer).

 

Things work fine, as long as the browser sends only an audio track or only a video track. But when the browser sends both audio and video, my application receives only one of the tracks. Which track the application “sees” depends on the order they are added by the browser, i.e. the order of RTCPeerConnection.addTrack(…) calls. The first track added by the browser is the track my application receives. Only one callback to onIncomingStream() and onIncomingDecodebinStream() are received.

 

I can see in the browser (<a href="chrome://webrtc-internals/">chrome://webrtc-internals/) that it is streaming both audio and video, and I have had essentially the same browser application working with a different server implementation (Python aiortc), so I am 99% certain that the browser JS is working correctly.

 

My assumption is that I am missing something in my code, but I haven’t been able to figure out what. Hope someone can offer some advice.

 

I am using GStreamer 1.16.2.

 

/ Thorsten

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Soebirk, Thorsten Nordholm]

Thanks Matthew. This is definitely a needed correction. I am currently working around this issue by suppressing any ICE candidates that are produced before the answer, which seems to work ok.

I don't think it directly affects my current issue though.

/ Thorsten

From: Matthew Waters [mailto:[hidden email]]
Sent: 30 April 2020 12:40
To: Soebirk, Thorsten Nordholm
Subject: Re: webrtcbin only receives single track from browser

Hi,

This sounds like something I just discovered and fixed in https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1226

That fix only applies in the following situation:
1. Browser is offering, GStreamer is answering
2. Browser is advertising BUNDLE (the default on all major browsers now), GStreamer is not using BUNDLE at all (bundle-policy=none)
3. More than one stream is being transported.
3. The send or receive state of each individual stream is irrelevant.

Cheers
-Matt

On 30/4/20 7:34 pm, Soebirk, Thorsten Nordholm wrote:
I am working on an application that should receive both audio and video from a browser using webrtcbin. My application is modelled on WebrtcSendRecv.java (https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/gst-java/src/main/java/WebrtcSendRecv.java), except that it is receive-only and it works in passive mode (receives a call by waiting for an offer, rather than making a call by sending an offer).
 
Things work fine, as long as the browser sends only an audio track or only a video track. But when the browser sends both audio and video, my application receives only one of the tracks. Which track the application "sees" depends on the order they are added by the browser, i.e. the order of RTCPeerConnection.addTrack(.) calls. The first track added by the browser is the track my application receives. Only one callback to onIncomingStream() and onIncomingDecodebinStream() are received.
 
I can see in the browser (chrome://webrtc-internals/) that it is streaming both audio and video, and I have had essentially the same browser application working with a different server implementation (Python aiortc), so I am 99% certain that the browser JS is working correctly.
 
My assumption is that I am missing something in my code, but I haven't been able to figure out what. Hope someone can offer some advice.
 
I am using GStreamer 1.16.2.
 
/ Thorsten


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Soebirk, Thorsten Nordholm]

Thanks Jan.

Wow! Very simple fix. It works now.

/ Thorsten

From: Jan Schmidt [mailto:[hidden email]]
Sent: 30 April 2020 12:48
To: Discussion of the development of and with GStreamer; Soebirk, Thorsten Nordholm
Subject: Re: webrtcbin only receives single track from browser

EXTERNAL MAIL:  
Hi,
On 30/4/20 7:34 pm, Soebirk, Thorsten Nordholm wrote:
I am working on an application that should receive both audio and video from a browser using webrtcbin. My application is modelled on WebrtcSendRecv.java (https://github.com/centricular/gstwebrtc-demos/blob/master/sendrecv/gst-java/src/main/java/WebrtcSendRecv.java), except that it is receive-only and it works in passive mode (receives a call by waiting for an offer, rather than making a call by sending an offer).
 
Things work fine, as long as the browser sends only an audio track or only a video track. But when the browser sends both audio and video, my application receives only one of the tracks. Which track the application "sees" depends on the order they are added by the browser, i.e. the order of RTCPeerConnection.addTrack(.) calls. The first track added by the browser is the track my application receives. Only one callback to onIncomingStream() and onIncomingDecodebinStream() are received.
Set the bundle-policy on webrtcbin - bundle-policy=max-bundle.
Cheers,
Jan.
 
I can see in the browser (chrome://webrtc-internals/) that it is streaming both audio and video, and I have had essentially the same browser application working with a different server implementation (Python aiortc), so I am 99% certain that the browser JS is working correctly.
 
My assumption is that I am missing something in my code, but I haven't been able to figure out what. Hope someone can offer some advice.
 
I am using GStreamer 1.16.2.
 
/ Thorsten


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
 
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel