synchronising multiple audio clients

[Johannes Bauer-2]

Hello!

Since I had problems getting gst-python running i wrote my programs in C for both serer and client.

I now have GStreamer 1.8.1 on my server (Win7) and also on both of my clients (raspi2 with wheezy and raspi3 with jessy).

My server starts streaming an mpa stream from the mic input to the clients via rtp and rtcp multicast. The clients receive the rtp stream and the rtcp data.

I hear the output on both of my raspberries, however raspi2 is about 0.5 seconds delayed in contrast to the raspi3.

Is it possible to synchronize them, so there is "no" delay among the clients?

I read something about the rtpbin parameters ntp-time-source, ntp-sync, latency, buffer-mode and tried to play around with them, but i can't manage to get the clients synchronised. Is it possible with the properties or do i need to do something else here?

here is a snippet of the server:

g_object_set(G_OBJECT(rtpbin), "latency", 200, NULL);

g_object_set(G_OBJECT(rtpbin), "buffer-mode", 0, NULL);

// g_object_set(G_OBJECT(rtpbin), "ntp-sync", true, NULL);

g_object_set(G_OBJECT(rtpbin), "ntp-time-source", 3, NULL);

// g_object_set(G_OBJECT(rtpbin), "use-pipeline-clock", true, NULL);

g_object_set(G_OBJECT(source), "location", call->wav_file, NULL);

g_object_set(G_OBJECT(udpsink_rtp), "host", call->ip, NULL);

g_object_set(G_OBJECT(udpsink_rtp), "port", call->in_rtp_port, NULL);

g_object_set(G_OBJECT(udpsink_rtcp), "host", call->ip, NULL);

g_object_set(G_OBJECT(udpsink_rtcp), "port", call->in_rtcp_port, NULL);

// g_object_set(G_OBJECT(udpsink_rtcp), "sync", false, NULL); 

here is a snippet of the client:

g_object_set(G_OBJECT(rtpbin), "latency", 200, NULL);

g_object_set(G_OBJECT(rtpbin), "buffer-mode", 0, NULL);

// g_object_set(G_OBJECT(rtpbin), "ntp-sync", TRUE, NULL);

g_object_set(G_OBJECT(rtpbin), "ntp-time-source", 3, NULL);

// g_object_set(G_OBJECT(rtpbin), "use-pipeline-clock", TRUE, NULL);

g_object_set(G_OBJECT(call->rtpsrc), "address", call->ip, NULL);

g_object_set(G_OBJECT(call->rtpsrc), "port", call->in_rtp_port, NULL);

g_object_set(G_OBJECT(call->rtpsrc), "auto-multicast", TRUE, NULL);

g_object_set(G_OBJECT(call->rtpsrc), "timeout", 10000000000, NULL);

g_object_set(G_OBJECT(call->rtpsrc), "caps", caps_rtp, NULL);

g_object_set(G_OBJECT(call->rtcpsrc), "address", call->ip, NULL);

g_object_set(G_OBJECT(call->rtcpsrc), "port", call->in_rtcp_port, NULL);

g_object_set(G_OBJECT(call->rtcpsrc), "auto-multicast", TRUE, NULL);

g_object_set(G_OBJECT(call->rtcpsrc), "caps", caps_rtcp, NULL);

Kind regards

Johannes Bauer  

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Fr, 2016-07-08 at 12:21 +0200, Johannes Bauer wrote:

> Hello!
>
> Since I had problems getting gst-python running i wrote my programs
> in C for both serer and client.
> I now have GStreamer 1.8.1 on my server (Win7) and also on both of my
> clients (raspi2 with wheezy and raspi3 with jessy).
>
> My server starts streaming an mpa stream from the mic input to the
> clients via rtp and rtcp multicast. The clients receive the rtp
> stream and the rtcp data.
>
> I hear the output on both of my raspberries, however raspi2 is about
> 0.5 seconds delayed in contrast to the raspi3.
>
> Is it possible to synchronize them, so there is "no" delay among the
> clients?
>
> I read something about the rtpbin parameters ntp-time-source, ntp-
> sync, latency, buffer-mode and tried to play around with them, but i
> can't manage to get the clients synchronised. Is it possible with the
> properties or do i need to do something else here?

The easiest solution would be by using gst-rtsp-server and sharing a
clock between the different clients:
https://cgit.freedesktop.org/gstreamer/gst-rtsp-server/tree/examples/test-netclock-client.c
https://cgit.freedesktop.org/gstreamer/gst-rtsp-server/tree/examples/test-netclock.c

But you can of course also implement the same manually without RTSP.

See also my talk at the GStreamer conference about this:
https://gstconf.ubicast.tv/videos/synchronised-multi-room-media-playback-and-distributed-live-media-processing-and-mixing-with-gstreamer/

--

Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Johannes Bauer-2]

Hello!

Thanks for your reply!

Your talk is really informative and i think I understood a lot of those parameters much better now.

In my understanding I need to distribute a common GstClock in order to have the clients synchronised.

As I understand, the gst_net_clock needs to be requested by every client and cannot be received via multicast.

So there is a lot of network traffic if every clients request clock info from the server.

Is there a way to distribute the clock via multicast?

Kind regards

JB

2016-07-08 12:34 GMT+02:00 Sebastian Dröge <[hidden email]>:

On Fr, 2016-07-08 at 12:21 +0200, Johannes Bauer wrote:
> Hello!
>
> Since I had problems getting gst-python running i wrote my programs
> in C for both serer and client.
> I now have GStreamer 1.8.1 on my server (Win7) and also on both of my
> clients (raspi2 with wheezy and raspi3 with jessy).
>
> My server starts streaming an mpa stream from the mic input to the
> clients via rtp and rtcp multicast. The clients receive the rtp
> stream and the rtcp data.
>
> I hear the output on both of my raspberries, however raspi2 is about
> 0.5 seconds delayed in contrast to the raspi3.
>
> Is it possible to synchronize them, so there is "no" delay among the
> clients?
>
> I read something about the rtpbin parameters ntp-time-source, ntp-
> sync, latency, buffer-mode and tried to play around with them, but i
> can't manage to get the clients synchronised. Is it possible with the
> properties or do i need to do something else here?

The easiest solution would be by using gst-rtsp-server and sharing a
clock between the different clients:
https://cgit.freedesktop.org/gstreamer/gst-rtsp-server/tree/examples/test-netclock-client.c
https://cgit.freedesktop.org/gstreamer/gst-rtsp-server/tree/examples/test-netclock.c

But you can of course also implement the same manually without RTSP.

See also my talk at the GStreamer conference about this:
https://gstconf.ubicast.tv/videos/synchronised-multi-room-media-playback-and-distributed-live-media-processing-and-mixing-with-gstreamer/

--

Sebastian Dröge, Centricular Ltd · http://www.centricular.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Fr, 2016-07-08 at 17:37 +0200, Johannes Bauer wrote:

> Hello!
>
> Thanks for your reply!
> Your talk is really informative and i think I understood a lot of
> those parameters much better now.
>
> In my understanding I need to distribute a common GstClock in order
> to have the clients synchronised.
>
> As I understand, the gst_net_clock needs to be requested by every
> client and cannot be received via multicast.
> So there is a lot of network traffic if every clients request clock
> info from the server.
> Is there a way to distribute the clock via multicast?

You could use a different protocol :) PTP for example works via
multicast by default, you can use the GStreamer PTP clock on the
clients and use something like ptpd2 on one server.

Apart from that it all would work the same.


Instead of externally distributing the information about which clock to
use, you can also use the RFC7273 mode in gst-rtsp-server and rtspsrc
(or set up the same things via rtpbin). Then this information is also
distributed via the SDP.

--

Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Johannes Bauer-2]

Hi!

Thank you for your answer!

I ended up using the gst_ntp_clock_new and it works good so far.

Sometimes you hear them drifting apart by a little (I guess about 50ms), but they are getting in sync again.

Maybe I will try to use a ptp clock source for further improvement, but im happy with that for the moment.

My problem now is, that i want to have the delay between the mic source and the remote sinks as small as possible.

So i use

  g_object_set(G_OBJECT(micsrc), "latency-time", (guint64)2000, NULL);

  g_object_set(G_OBJECT(micsrc), "buffer-time", (guint64)20000, NULL);

for my directsoundsrc at the sender and 

  g_object_set(G_OBJECT(audiosink), "latency-time", (guint64)2000, NULL);

  g_object_set(G_OBJECT(audiosink), "buffer-time", (guint64)50000, NULL);

  g_object_set(G_OBJECT(rtpbin), "latency", 200, NULL); 

for the alsasinks at the remote clients.

The latency at the beginning is ok, with about 300ms, but as long as the stream is active it increases.

After 24 hours it was about 5 seconds and after 72 hours of live streaming it was about 25 seconds.

Is there something wrong with my settings here. How can i stop the increasing latency and further reduce the overall latency of the pipeline?

My pipeline uses following elements:

  directsoundsrc ! audioconvert ! lamemp3enc ! rtpmpapay ! rtpbin ! udpsink_rtp  rtpbin ! udpsink_rtcp 

  udpsrc_rtp ! rtpbin udpsrc_rtcp ! rtpbin ! rtpmpadepay ! mpegaudioparse ! mad ! audioconvert ! alsasink

Kind regards

JB

 

2016-07-10 20:41 GMT+02:00 Sebastian Dröge <[hidden email]>:

On Fr, 2016-07-08 at 17:37 +0200, Johannes Bauer wrote:
> Hello!
>
> Thanks for your reply!
> Your talk is really informative and i think I understood a lot of
> those parameters much better now.
>
> In my understanding I need to distribute a common GstClock in order
> to have the clients synchronised.
>
> As I understand, the gst_net_clock needs to be requested by every
> client and cannot be received via multicast.
> So there is a lot of network traffic if every clients request clock
> info from the server.
> Is there a way to distribute the clock via multicast?

You could use a different protocol :) PTP for example works via
multicast by default, you can use the GStreamer PTP clock on the
clients and use something like ptpd2 on one server.

Apart from that it all would work the same.


Instead of externally distributing the information about which clock to
use, you can also use the RFC7273 mode in gst-rtsp-server and rtspsrc
(or set up the same things via rtpbin). Then this information is also
distributed via the SDP.

--

Sebastian Dröge, Centricular Ltd · http://www.centricular.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel