GStreamer-devel

webrtcbin and ulpfec issue when using more than one track

classic Classic list List threaded Threaded
♦
♦
 2 messages Options
Faraz Khan
Reply | Threaded
Open this post in threaded view
♦
♦
|

webrtcbin and ulpfec issue when using more than one track

Faraz Khan
I'm using webrtcbin with multiple tracks and successfully streaming it to chrome. The issue arises when I start using ULPFEC which i've enabled as this:

    g_signal_emit_by_name (webrtcbin, "get-transceivers",&transceivers);
    g_assert (transceivers != NULL && transceivers->len > 0);

    for (int i = 0; i < transceivers->len; i++) {
        trans = g_array_index (transceivers, GstWebRTCRTPTransceiver *, i);
        trans->direction = GST_WEBRTC_RTP_TRANSCEIVER_DIRECTION_SENDONLY;
        g_object_set (trans, "fec-type", GST_WEBRTC_FEC_TYPE_ULP_RED,
                  "fec-percentage", 50, "do-nack", FALSE, NULL);
    }
    g_array_unref (transceivers);

I'm generating the pipeline from gst_parse_launch like this:

webrtcbin  bundle-policy=max-bundle name=webrtcbin stun-server=stun://stun.l.google.com:19302 v4l2src device=/dev/video4 ! image/jpeg,width=1920,height=1080,framerate=30/1 !
 jpegdec ! videoconvert ! queue max-size-buffers=1 ! x264enc speed-preset=1 tune=zerolatency bitrate=512 key-int-max=60 ! video/x-h264,profile=baseline ! h264parse !rtph264pay
config-interval=-1  ! application/x-rtp,media=video,encoding-name=H264,payload=96,extmap-2=http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time ! webrtcbin.
v4l2src device=/dev/video2 ! image/jpeg,width=1920,height=1080,framerate=30/1 ! jpegdec ! videoconvert ! queue max-size-buffers=1 ! x264enc speed-preset=1 tune=zerolatency
bitrate=512 key-int-max=60 ! video/x-h264,profile=baseline ! h264parse ! rtph264pay config-interval=-1  !
application/x-rtp,media=video,encoding-name=H264,payload=96,extmap-2=http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time ! webrtcbin.
audiotestsrc is-live=true wave=red-noise ! audioconvert ! audioresample ! queue ! opusenc ! rtpopuspay ! queue !
application/x-rtp,media=audio,encoding-name=OPUS,payload=97 ! webrtcbin.



Only the first track works and gets decoded on the chrome side - the other ones don't. Works fine without FEC. Am I doing something wrong? My setup includes 2 video only and 1 audio only track. 
The SDP seems to be constructed correctly:


offer: v=0
o=- 8023833080363098678 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
a=group:BUNDLE front rear audio0 application1
m=video 9 UDP/TLS/RTP/SAVPF 96 98 99
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:rRVZ5Sl1hCdN/NekWOXVDP0FtsebUAlJ
a=ice-pwd:lb+WRyHg2y4kvmNiE9Iz/bfB+8+z2qlf
a=rtcp-mux
a=rtcp-rsize
a=sendonly
a=rtpmap:96 H264/90000
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 goog-remb
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=framerate:30
a=fmtp:96 packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKNoB4AiflwFuBAQNSgAAAwACAAADAHkeMGVA,aM48gA==
a=rtpmap:98 red/90000
a=rtpmap:99 ulpfec/90000
a=ssrc:4272510362 msid:user2626643232@host-5af7e82 webrtctransceiver0
a=ssrc:4272510362 cname:user2626643232@host-5af7e82
a=mid:front
a=fingerprint:sha-256 E7:2A:42:26:02:68:3F:56:5B:FB:C7:6F:15:0E:38:8E:86:E8:C0:83:69:5B:31:64:73:8F:47:02:BC:87:74:27
m=video 0 UDP/TLS/RTP/SAVPF 96 100 101
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:rRVZ5Sl1hCdN/NekWOXVDP0FtsebUAlJ
a=ice-pwd:lb+WRyHg2y4kvmNiE9Iz/bfB+8+z2qlf
a=bundle-only
a=rtcp-mux
a=rtcp-rsize
a=sendonly
a=rtpmap:96 H264/90000
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 goog-remb
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=framerate:30
a=fmtp:96 packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKNoB4AiflwFuBAQNSgAAAwACAAADAHkeMGVA,aM48gA==
a=rtpmap:100 red/90000
a=rtpmap:101 ulpfec/90000
a=ssrc:2753800917 msid:user2626643232@host-5af7e82 webrtctransceiver1
a=ssrc:2753800917 cname:user2626643232@host-5af7e82
a=mid:rear
a=fingerprint:sha-256 E7:2A:42:26:02:68:3F:56:5B:FB:C7:6F:15:0E:38:8E:86:E8:C0:83:69:5B:31:64:73:8F:47:02:BC:87:74:27
m=audio 0 UDP/TLS/RTP/SAVPF 97 102 103
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:rRVZ5Sl1hCdN/NekWOXVDP0FtsebUAlJ
a=ice-pwd:lb+WRyHg2y4kvmNiE9Iz/bfB+8+z2qlf
a=bundle-only
a=rtcp-mux
a=rtcp-rsize
a=sendonly
a=rtpmap:97 OPUS/48000/2
a=rtcp-fb:97 nack pli
a=fmtp:97 sprop-maxcapturerate=48000;sprop-stereo=0
a=rtpmap:102 red/48000
a=rtpmap:103 ulpfec/48000
a=ssrc:3317392810 msid:user2626643232@host-5af7e82 webrtctransceiver2
a=ssrc:3317392810 cname:user2626643232@host-5af7e82
a=mid:audio0
a=fingerprint:sha-256 E7:2A:42:26:02:68:3F:56:5B:FB:C7:6F:15:0E:38:8E:86:E8:C0:83:69:5B:31:64:73:8F:47:02:BC:87:74:27
m=application 0 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:rRVZ5Sl1hCdN/NekWOXVDP0FtsebUAlJ
a=ice-pwd:lb+WRyHg2y4kvmNiE9Iz/bfB+8+z2qlf
a=bundle-only
a=mid:application1
a=sctp-port:5000
a=fingerprint:sha-256 E7:2A:42:26:02:68:3F:56:5B:FB:C7:6F:15:0E:38:8E:86:E8:C0:83:69:5B:31:64:73:8F:47:02:BC:87:74:27

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Olivier Crête-3
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: webrtcbin and ulpfec issue when using more than one track

Olivier Crête-3
Hi,

You're hitting a known bug in webrtcbin. You can't use ULPFEC with bundle enabled.

Olivier

On Mon, 2020-10-19 at 15:14 -0700, Faraz Khan wrote:
I'm using webrtcbin with multiple tracks and successfully streaming it to chrome. The issue arises when I start using ULPFEC which i've enabled as this:

    g_signal_emit_by_name (webrtcbin, "get-transceivers",&transceivers);
    g_assert (transceivers != NULL && transceivers->len > 0);

    for (int i = 0; i < transceivers->len; i++) {
        trans = g_array_index (transceivers, GstWebRTCRTPTransceiver *, i);
        trans->direction = GST_WEBRTC_RTP_TRANSCEIVER_DIRECTION_SENDONLY;
        g_object_set (trans, "fec-type", GST_WEBRTC_FEC_TYPE_ULP_RED,
                  "fec-percentage", 50, "do-nack", FALSE, NULL);
    }
    g_array_unref (transceivers);

I'm generating the pipeline from gst_parse_launch like this:

webrtcbin  bundle-policy=max-bundle name=webrtcbin stun-server=stun://stun.l.google.com:19302 v4l2src device=/dev/video4 ! image/jpeg,width=1920,height=1080,framerate=30/1 !
 jpegdec ! videoconvert ! queue max-size-buffers=1 ! x264enc speed-preset=1 tune=zerolatency bitrate=512 key-int-max=60 ! video/x-h264,profile=baseline ! h264parse !rtph264pay
config-interval=-1  ! application/x-rtp,media=video,encoding-name=H264,payload=96,extmap-2=http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time ! webrtcbin.
v4l2src device=/dev/video2 ! image/jpeg,width=1920,height=1080,framerate=30/1 ! jpegdec ! videoconvert ! queue max-size-buffers=1 ! x264enc speed-preset=1 tune=zerolatency
bitrate=512 key-int-max=60 ! video/x-h264,profile=baseline ! h264parse ! rtph264pay config-interval=-1  !
application/x-rtp,media=video,encoding-name=H264,payload=96,extmap-2=http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time ! webrtcbin.
audiotestsrc is-live=true wave=red-noise ! audioconvert ! audioresample ! queue ! opusenc ! rtpopuspay ! queue !
application/x-rtp,media=audio,encoding-name=OPUS,payload=97 ! webrtcbin.



Only the first track works and gets decoded on the chrome side - the other ones don't. Works fine without FEC. Am I doing something wrong? My setup includes 2 video only and 1 audio only track. 
The SDP seems to be constructed correctly:


offer: v=0
o=- 8023833080363098678 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
a=group:BUNDLE front rear audio0 application1
m=video 9 UDP/TLS/RTP/SAVPF 96 98 99
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:rRVZ5Sl1hCdN/NekWOXVDP0FtsebUAlJ
a=ice-pwd:lb+WRyHg2y4kvmNiE9Iz/bfB+8+z2qlf
a=rtcp-mux
a=rtcp-rsize
a=sendonly
a=rtpmap:96 H264/90000
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 goog-remb
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=framerate:30
a=fmtp:96 packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKNoB4AiflwFuBAQNSgAAAwACAAADAHkeMGVA,aM48gA==
a=rtpmap:98 red/90000
a=rtpmap:99 ulpfec/90000
a=ssrc:4272510362 msid:user2626643232@host-5af7e82 webrtctransceiver0
a=ssrc:4272510362 cname:user2626643232@host-5af7e82
a=mid:front
a=fingerprint:sha-256 E7:2A:42:26:02:68:3F:56:5B:FB:C7:6F:15:0E:38:8E:86:E8:C0:83:69:5B:31:64:73:8F:47:02:BC:87:74:27
m=video 0 UDP/TLS/RTP/SAVPF 96 100 101
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:rRVZ5Sl1hCdN/NekWOXVDP0FtsebUAlJ
a=ice-pwd:lb+WRyHg2y4kvmNiE9Iz/bfB+8+z2qlf
a=bundle-only
a=rtcp-mux
a=rtcp-rsize
a=sendonly
a=rtpmap:96 H264/90000
a=rtcp-fb:96 nack pli
a=rtcp-fb:96 goog-remb
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=framerate:30
a=fmtp:96 packetization-mode=1;profile-level-id=42c028;sprop-parameter-sets=Z0LAKNoB4AiflwFuBAQNSgAAAwACAAADAHkeMGVA,aM48gA==
a=rtpmap:100 red/90000
a=rtpmap:101 ulpfec/90000
a=ssrc:2753800917 msid:user2626643232@host-5af7e82 webrtctransceiver1
a=ssrc:2753800917 cname:user2626643232@host-5af7e82
a=mid:rear
a=fingerprint:sha-256 E7:2A:42:26:02:68:3F:56:5B:FB:C7:6F:15:0E:38:8E:86:E8:C0:83:69:5B:31:64:73:8F:47:02:BC:87:74:27
m=audio 0 UDP/TLS/RTP/SAVPF 97 102 103
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:rRVZ5Sl1hCdN/NekWOXVDP0FtsebUAlJ
a=ice-pwd:lb+WRyHg2y4kvmNiE9Iz/bfB+8+z2qlf
a=bundle-only
a=rtcp-mux
a=rtcp-rsize
a=sendonly
a=rtpmap:97 OPUS/48000/2
a=rtcp-fb:97 nack pli
a=fmtp:97 sprop-maxcapturerate=48000;sprop-stereo=0
a=rtpmap:102 red/48000
a=rtpmap:103 ulpfec/48000
a=ssrc:3317392810 msid:user2626643232@host-5af7e82 webrtctransceiver2
a=ssrc:3317392810 cname:user2626643232@host-5af7e82
a=mid:audio0
a=fingerprint:sha-256 E7:2A:42:26:02:68:3F:56:5B:FB:C7:6F:15:0E:38:8E:86:E8:C0:83:69:5B:31:64:73:8F:47:02:BC:87:74:27
m=application 0 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:rRVZ5Sl1hCdN/NekWOXVDP0FtsebUAlJ
a=ice-pwd:lb+WRyHg2y4kvmNiE9Iz/bfB+8+z2qlf
a=bundle-only
a=mid:application1
a=sctp-port:5000
a=fingerprint:sha-256 E7:2A:42:26:02:68:3F:56:5B:FB:C7:6F:15:0E:38:8E:86:E8:C0:83:69:5B:31:64:73:8F:47:02:BC:87:74:27
_______________________________________________
gstreamer-devel mailing list
[hidden email]

https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

-- 
Olivier Crête
[hidden email]

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Free forum by Nabble Edit this page