Video Chat using mpegtsmux
Locked
Video Chat using mpegtsmux
 
|
Hi,
I am trying to implement a video chat using GStreamer on Windows 10. I intend to use gstreamer C API, but for now I am just using the command line for basic testing. In order to keep video and audio in sync, I am muxing them using mpegtsmux and then sending them via RTP. It works fine, but the latency is pretty high (I think it's about 1 second). My sending pipeline is as follows: gst-launch-1.0 mpegtsmux name=mux ! rtpmp2tpay ! udpsink host=127.0.0.1 port=5555^ wasapisrc ! audioresample ! audioconvert ! avenc_aac ! queue ! mux.^ ksvideosrc ! videoconvert ! x264enc tune=zerolatency ! h264parse config-interval=-1 ! queue ! mux.^ My receiving pipeline is as follows: gst-launch-1.0^ udpsrc address=127.0.0.1 port=5555 caps="application/x-rtp" ! rtpmp2tdepay ! tsdemux name=demux^ demux. ! queue ! avdec_aac ! audioresample ! audioconvert ! wasapisink low-latency=true^ demux. ! queue ! h264parse ! avdec_h264 ! videoconvert ! autovideosink Questions: 1. Is this the correct approach for a video chat or should I use separate streams for audio and video? 2. If this is the correct approach, am I using the correct muxer or should I use something else such as asfmux? 3. If this is the correct approach and the correct muxer, is there something I can do in order to reduce the latency? _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Video Chat using mpegtsmux
|
|
Hey,
I would recommend using webRTC (webrtcbin in the GStreamer context) for such an application :) Best, --
Mathieu Duponchelle · https://www.centricular.com On 5/19/21 8:14 AM, Gregory AE40 via
gstreamer-devel wrote:
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Video Chat using mpegtsmux
|
|
Hi, thank you for the prompt response. I wasn't specific enough, the application will run on a LAN, so I can use multicast in order to send only one stream from each PC. With WebRTC multicast is not possible, because it requires a separate stream for each receiver. On Wed, May 19, 2021, 15:02 Mathieu Duponchelle <[hidden email]> wrote:
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Video Chat using mpegtsmux
|
|
Le mercredi 19 mai 2021 à 18:23 +0300, Gregory AE40 via gstreamer-devel a
écrit : > Hi, thank you for the prompt response. > I wasn't specific enough, the application will run on a LAN, so I can use > multicast in order to send only one stream from each PC. > With WebRTC multicast is not possible, because it requires a separate stream > for each receiver. Yet, you should still use RTP for mulicast. For your use case, just creating an RTSP server (using gst-rtp-server library) seem sufficient no ? > > On Wed, May 19, 2021, 15:02 Mathieu Duponchelle <[hidden email]> > wrote: > > Hey, > > > > I would recommend using webRTC (webrtcbin in the GStreamer context) for > > such an application :) > > > > Best, > > > > _______________________________________________ > > gstreamer-devel mailing list > > [hidden email] > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Video Chat using mpegtsmux
|
|
Yes, I understand that I should use RTP, my question is whether to mux audio and video to a single stream or to send two separate streams (with RTCP for every stream for synchronization). The first option seems more fail-safe to me in terms of audio/video synchronization. However, I tested also the second option and its latency is lower. I guess that if you suggested to use WebRTC, I should stick with the second option, because AFAIK, WebRTC sends separate streams for video and audio. On Wed, May 19, 2021, 20:54 Nicolas Dufresne <[hidden email]> wrote: Le mercredi 19 mai 2021 à 18:23 +0300, Gregory AE40 via gstreamer-devel a _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Video Chat using mpegtsmux
|
|
Le mercredi 19 mai 2021 à 21:44 +0300, Gregory AE40 a écrit :
> Yes, I understand that I should use RTP, my question is whether to mux audio > and video to a single stream or to send two separate streams (with RTCP for > every stream for synchronization). The first option seems more fail-safe to me > in terms of audio/video synchronization. However, I tested also the second > option and its latency is lower. I guess that if you suggested to use WebRTC, > I should stick with the second option, because AFAIK, WebRTC sends separate > streams for video and audio. Correct, muxing and demuxing MPEG TS requires adding some latency due to how the timing information is stored. We added a latency property to reduce it from the default 700ms (maximum specified) for those that have the luxury to control the muxing parameters, which should be your case here. Yet, you still need to introduce a small amount of latency compare to streaming audio/video individually. > > On Wed, May 19, 2021, 20:54 Nicolas Dufresne <[hidden email]> wrote: > > Le mercredi 19 mai 2021 à 18:23 +0300, Gregory AE40 via gstreamer-devel a > > écrit : > > > Hi, thank you for the prompt response. > > > I wasn't specific enough, the application will run on a LAN, so I can use > > > multicast in order to send only one stream from each PC. > > > With WebRTC multicast is not possible, because it requires a separate > > > stream > > > for each receiver. > > > > Yet, you should still use RTP for mulicast. For your use case, just creating > > an > > RTSP server (using gst-rtp-server library) seem sufficient no ? > > > > > > > > On Wed, May 19, 2021, 15:02 Mathieu Duponchelle <[hidden email]> > > > wrote: > > > > Hey, > > > > > > > > I would recommend using webRTC (webrtcbin in the GStreamer context) for > > > > such an application :) > > > > > > > > Best, > > > > > > > > _______________________________________________ > > > > gstreamer-devel mailing list > > > > [hidden email] > > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > > > _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Video Chat using mpegtsmux
|
|
In reply to this post by GStreamer-devel mailing list
I echo the suggestions given by Nicolas and Mathieu. But also wanted to add that, if you chose to continue using TS, there is a “latency” property in the tsdemux element that you can tweak. By default it is set to 700ms (to comply with the TS standard), which sounds like the 1s lag you are experiencing.
Michael
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
RE: Video Chat using mpegtsmux
|
|
Ok, thank you all for the useful information and suggestions
From: Michael Gruner <[hidden email]>
I echo the suggestions given by Nicolas and Mathieu. But also wanted to add that, if you chose to continue using TS, there is a “latency” property in the tsdemux element that you can tweak. By default it is set to 700ms (to comply with the TS standard), which sounds like the 1s lag you are experiencing.
Michael
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
«
Return to GStreamer-devel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |