GStreamer-devel

How to restart webrtcbin plugin without stopping whole pipeline

Classic

List

Threaded

3 messages Options

Vladimir Tyutin

How to restart webrtcbin plugin without stopping whole pipeline

Hello gstreamer experts,

I need your advice on the issue below.

I have a pipeline that records mpeg ts files (2 different video resolutions and audio) and webrtc (please see pipeline below. v536videosrc is my plugin that produces h264 encoded frames).

Everything works fine for the first webrtc call. Now when the call is over I need somehow to reset webrtc to the initial state to be prepared for the new call.

I tried two approaches:

1. Block capsfilter src before webrtc and move webrtcbin to state NULL and PLAYING again and unblock capsfilter.

2. Block capsfilter src before webrtc, remove webrtcbin from pipeline recreate it and add to pipeline again, link and unblock capsfilter.

Both approaches do not work. The major issue I observe is that when webrtc moves to NULL state it tries to set NULL state to all plugins inside webrtcbin. And it hangs somewhere in rtpjittbuffer or so despite I do everything on a separate thread (not main loop thread).

Please advise what is the correct way to reset webrtcbin to initial state and get it ready for a new incoming call without stopping the whole pipeline.

Here is my pipeline example:

#define PIPELINE "webrtcbin name=webrtc " STUN_SERVER_PROP "=" STUN_1 " " STUN_SERVER_PROP "=" STUN_2 " " STUN_SERVER_PROP "=" STUN_3 " " \

STUN_SERVER_PROP "=" STUN_4 " " STUN_SERVER_PROP "=" STUN_5 " " TURN_SERVER_PROP "=" TURN_1 " " \

"mpegtsmux name=fullhdts ! hlssink max-files=100 target-duration=10 location=/mnt/ramdisk/fullhd_%09d.ts " \

"mpegtsmux name=vgats ! hlssink max-files=100 target-duration=10 location=/mnt/ramdisk/vga_%09d.ts " \

"v536videosrc sys-init=false device=1 channel=0 encoder=0 format=H264 width=1920 height=1080 ! video/x-h264, stream-format=byte-stream, alignment=au, profile=baseline ! queue ! h264parse ! fullhdts. " \

"v536videosrc sys-init=false device=1 channel=1 encoder=1 format=H264 width=640 height=480 ! video/x-h264, stream-format=byte-stream, alignment=au, profile=baseline ! tee name=tv ! queue ! h264parse ! vgats. " \

"alsasrc ! tee name=t ! queue ! avenc_aac ! aacparse ! fullhdts. " \

"t. ! queue ! avenc_aac ! aacparse ! vgats. " \

"tv. ! queue name=videoqueue leaky=downstream max-size-buffers=25 ! rtph264pay name=vrtp ! capsfilter name=vrtpcaps caps=" RTP_CAPS_H264 "96 ! webrtc. " \

"t. ! queue name=audioqueue leaky=downstream ! audioconvert name=aconvert ! opusenc ! rtpopuspay name=artp ! capsfilter name=artpcaps caps=" RTP_CAPS_OPUS "97 ! webrtc. "

Thanks,

Vladimir

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Zachary Hueras

Re: How to restart webrtcbin plugin without stopping whole pipeline

I would recommend something akin to your latter approach. When a call comes in, you would modify the pipeline to include a new webrtcbin to handle it. Insert a tee element after capsfilter and link that to your webrtcbins

When a call comes in, grab a new pad from the tee. Create a queue, rtph264pay, and a webrtcbin; add them to your pipeline, and link them together (queue -> rtph264pay -> webrtcbin). Link the pad from the tee to the queue's src pad. Do all your normal webrtc setup, and finally call gst_element_sync_state_with_parent on the queue, rtph264pay, and webrtcbin elements.

Zachary Hueras

Principal Software Engineer
AMD Global Telemedicine, Inc.
Cell : 978-660-3812
[hidden email]
www.amdtelemedicine.com | Follow us on LinkedIn

From: gstreamer-devel <[hidden email]> on behalf of Vladimir Tyutin <[hidden email]>
Sent: Monday, December 14, 2020 3:58 AM
To: [hidden email]
Subject: How to restart webrtcbin plugin without stopping whole pipeline

Hello gstreamer experts,

I need your advice on the issue below.

I have a pipeline that records mpeg ts files (2 different video resolutions and audio) and webrtc (please see pipeline below. v536videosrc is my plugin that produces h264 encoded frames).

Everything works fine for the first webrtc call. Now when the call is over I need somehow to reset webrtc to the initial state to be prepared for the new call.

I tried two approaches:

1. Block capsfilter src before webrtc and move webrtcbin to state NULL and PLAYING again and unblock capsfilter.

2. Block capsfilter src before webrtc, remove webrtcbin from pipeline recreate it and add to pipeline again, link and unblock capsfilter.

Please advise what is the correct way to reset webrtcbin to initial state and get it ready for a new incoming call without stopping the whole pipeline.

Here is my pipeline example:

#define PIPELINE "webrtcbin name=webrtc " STUN_SERVER_PROP "=" STUN_1 " " STUN_SERVER_PROP "=" STUN_2 " " STUN_SERVER_PROP "=" STUN_3 " " \

STUN_SERVER_PROP "=" STUN_4 " " STUN_SERVER_PROP "=" STUN_5 " " TURN_SERVER_PROP "=" TURN_1 " " \

"mpegtsmux name=fullhdts ! hlssink max-files=100 target-duration=10 location=/mnt/ramdisk/fullhd_%09d.ts " \

"mpegtsmux name=vgats ! hlssink max-files=100 target-duration=10 location=/mnt/ramdisk/vga_%09d.ts " \

"alsasrc ! tee name=t ! queue ! avenc_aac ! aacparse ! fullhdts. " \

"t. ! queue ! avenc_aac ! aacparse ! vgats. " \

"tv. ! queue name=videoqueue leaky=downstream max-size-buffers=25 ! rtph264pay name=vrtp ! capsfilter name=vrtpcaps caps=" RTP_CAPS_H264 "96 ! webrtc. " \

"t. ! queue name=audioqueue leaky=downstream ! audioconvert name=aconvert ! opusenc ! rtpopuspay name=artp ! capsfilter name=artpcaps caps=" RTP_CAPS_OPUS "97 ! webrtc. "

Thanks,

Vladimir

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Vladimir Tyutin

Re: How to restart webrtcbin plugin without stopping whole pipeline

Hi Zachary,

Thanks for your quick reply. Sounds like a very good idea. Let me try your approach and will get back to you with the results.

Thanks,

Vladimir

On Mon, Dec 14, 2020 at 12:46 PM Zachary Hueras <[hidden email]> wrote:

I would recommend something akin to your latter approach. When a call comes in, you would modify the pipeline to include a new webrtcbin to handle it. Insert a tee element after capsfilter and link that to your webrtcbins

When a call comes in, grab a new pad from the tee. Create a queue, rtph264pay, and a webrtcbin; add them to your pipeline, and link them together (queue -> rtph264pay -> webrtcbin). Link the pad from the tee to the queue's src pad. Do all your normal webrtc setup, and finally call gst_element_sync_state_with_parent on the queue, rtph264pay, and webrtcbin elements.

Zachary Hueras
Principal Software Engineer
AMD Global Telemedicine, Inc.
Cell : 978-660-3812
[hidden email]
www.amdtelemedicine.com | Follow us on LinkedIn

From: gstreamer-devel <[hidden email]> on behalf of Vladimir Tyutin <[hidden email]>
Sent: Monday, December 14, 2020 3:58 AM
To: [hidden email]
Subject: How to restart webrtcbin plugin without stopping whole pipeline

Hello gstreamer experts,
I need your advice on the issue below.

I have a pipeline that records mpeg ts files (2 different video resolutions and audio) and webrtc (please see pipeline below. v536videosrc is my plugin that produces h264 encoded frames).

Everything works fine for the first webrtc call. Now when the call is over I need somehow to reset webrtc to the initial state to be prepared for the new call.

I tried two approaches:

1. Block capsfilter src before webrtc and move webrtcbin to state NULL and PLAYING again and unblock capsfilter.

2. Block capsfilter src before webrtc, remove webrtcbin from pipeline recreate it and add to pipeline again, link and unblock capsfilter.

Both approaches do not work. The major issue I observe is that when webrtc moves to NULL state it tries to set NULL state to all plugins inside webrtcbin. And it hangs somewhere in rtpjittbuffer or so despite I do everything on a separate thread (not main loop thread).

Please advise what is the correct way to reset webrtcbin to initial state and get it ready for a new incoming call without stopping the whole pipeline.

Here is my pipeline example:

#define PIPELINE "webrtcbin name=webrtc " STUN_SERVER_PROP "=" STUN_1 " " STUN_SERVER_PROP "=" STUN_2 " " STUN_SERVER_PROP "=" STUN_3 " " \

STUN_SERVER_PROP "=" STUN_4 " " STUN_SERVER_PROP "=" STUN_5 " " TURN_SERVER_PROP "=" TURN_1 " " \

"mpegtsmux name=fullhdts ! hlssink max-files=100 target-duration=10 location=/mnt/ramdisk/fullhd_%09d.ts " \

"mpegtsmux name=vgats ! hlssink max-files=100 target-duration=10 location=/mnt/ramdisk/vga_%09d.ts " \

"v536videosrc sys-init=false device=1 channel=0 encoder=0 format=H264 width=1920 height=1080 ! video/x-h264, stream-format=byte-stream, alignment=au, profile=baseline ! queue ! h264parse ! fullhdts. " \

"v536videosrc sys-init=false device=1 channel=1 encoder=1 format=H264 width=640 height=480 ! video/x-h264, stream-format=byte-stream, alignment=au, profile=baseline ! tee name=tv ! queue ! h264parse ! vgats. " \

"alsasrc ! tee name=t ! queue ! avenc_aac ! aacparse ! fullhdts. " \

"t. ! queue ! avenc_aac ! aacparse ! vgats. " \

"tv. ! queue name=videoqueue leaky=downstream max-size-buffers=25 ! rtph264pay name=vrtp ! capsfilter name=vrtpcaps caps=" RTP_CAPS_H264 "96 ! webrtc. " \

"t. ! queue name=audioqueue leaky=downstream ! audioconvert name=aconvert ! opusenc ! rtpopuspay name=artp ! capsfilter name=artpcaps caps=" RTP_CAPS_OPUS "97 ! webrtc. "

Thanks,

Vladimir

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel