How to gurantee lip sync in the multicasting mode

[Nathan]

Greetings,

I am also little bit confused by the way of a/v syn of RTP/RTCP of gstreamer.
In our usecase, one tx will send a/v data to multiple clients using the multicasting mode.
I created the code following the gstreamer's rtpbin example, and  found when there is only one client, the lip sync is ok.

 However, when there is more than one client, all the clients starting to lose the lip sync.
Here is the way I use the gstreamer:
Tx side:

        g_object_set(G_OBJECT(app->video_rtpsink),  "host",contents.new_mcaddr, "port", VIDEO_RTPSINK_PORT, "sync",FALSE,"async",FALSE ,"auto-multicast",TRUE,NULL);
        g_object_set(G_OBJECT(app->audio_rtpsink),  "host",contents.new_mcaddr, "port", AUDIO_RTPSINK_PORT, "sync",FALSE,"async",FALSE ,"auto-multicast",TRUE,NULL);
       
       
        g_object_set(G_OBJECT(app->video_rtcpsink), "host",contents.new_mcaddr, "port", VIDEO_RTCPSINK_PORT,"sync",FALSE,"async",FALSE ,"auto-multicast",TRUE,NULL);
        g_object_set(G_OBJECT(app->audio_rtcpsink), "host",contents.new_mcaddr, "port", AUDIO_RTCPSINK_PORT,"sync",FALSE,"async",FALSE ,"auto-multicast",TRUE,NULL);
       
        g_object_set(G_OBJECT(app->video_rtcpsrc), "multicast-group", contents.new_mcaddr, "port", VIDEO_RTCPSRC_PORT, NULL);
        g_object_set(G_OBJECT(app->audio_rtcpsrc), "multicast-group", contents.new_mcaddr, "port", AUDIO_RTCPSRC_PORT, NULL);


Rx side:
        g_object_set(G_OBJECT(vsource), "timeout", G_GUINT64_CONSTANT(5000000), "port", params.vrtp_port, "caps", video_caps, NULL);
        g_object_set(G_OBJECT(vsource), "multicast-group", params.mcaddr, "auto-multicast", TRUE, NULL);
        g_object_set(G_OBJECT(vrtcpsrc), "port", VIDEO_RTCP_SRC_PORT, "multicast-group", params.mcaddr, "auto-multicast", TRUE, NULL);
        g_object_set(G_OBJECT(vrtcpsink), "port", VIDEO_RTCP_SINK_PORT, "host", params.mcaddr, "auto-multicast", TRUE,
                        "sync", FALSE, "async", FALSE, NULL);

        g_object_set(G_OBJECT(asource), "port", params.artp_port, "caps", audio_caps, NULL);
        g_object_set(G_OBJECT(asource), "multicast-group", params.mcaddr, "auto-multicast", TRUE, NULL);
        g_object_set(G_OBJECT(artcpsrc), "port", AUDIO_RTCP_SRC_PORT, "multicast-group", params.mcaddr, "auto-multicast", TRUE, NULL);
        g_object_set(G_OBJECT(artcpsink), "port", AUDIO_RTCP_SINK_PORT, "host", params.mcaddr, "auto-multicast", TRUE,
                        "sync", FALSE, "async", FALSE, NULL);
        g_object_set(G_OBJECT(asink), "sync", FALSE, NULL);


Did I do anything wrong here? What is the right way of seting the gstreamer for the 1 to M mode?

Thanks,

John

[Nicolas Dufresne]

Le mercredi 03 octobre 2012 à 07:23 -0700, Nathan a écrit :
> "sync",FALSE,

Make sure you have sync=TRUE on the receiver side. This option tells the
sink elements to only play the data at the time specified by the
timestamp.

regards,
Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[mohammadk]

Hello,

I run the Gstreamer pipeline in multicasting mode with one transmitter and two receivers. The audio/video synchronization fails after a few minutes. I wonder if this problem occurs because the receivers send the RTCP information to the multi-cast address. In other words, how can the transmitter pick up RTCP information from multiple receivers in multicasting mode?

Thanks a lot for your comments.

Regards,
Mohammad

[Nathan]

Hi Nicolas,

I also tried to put the "sync" of udpsrc or rtpsrc of the receiver to be ture, but that will make the received video very non-smooth.  It seems that this property has to be FALSE.
Here is our other setting related to the gsteamer, I really am in the dark. Hope someone can point me out some directions regarding the lipsyn in the multicasting mode:
Tx:
        g_object_set(G_OBJECT(app->rtpbin), "use-pipeline-clock",TRUE,"ntp-sync",TRUE,NULL);

Rx:

g_object_set(G_OBJECT(rtpbin), "latency", 200, "buffer-mode", 0, "ntp-sync", TRUE, "use-pipeline-clock", TRUE, NULL);

[Nicolas Dufresne]

Le mercredi 03 octobre 2012 à 12:30 -0700, mohammadk a écrit :
> Thanks a lot for your comments.

Please do not reply to existing message when you have new question.

thanks,
Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne]

Le mercredi 03 octobre 2012 à 13:33 -0700, Nathan a écrit :
> Hi Nicolas,
>
> I also tried to put the "sync" of udpsrc or rtpsrc of the receiver to be
> ture, but that will make the received video very non-smooth.  It seems that
> this property has to be FALSE.

SRC elements don't have this parameter. To have synchronisation, you
must have those to TRUE on the audio and video SINK elements on the
receiver side. For sure, video won't be smooth if your network (combine
with encoding parameters) cause more latency then what is being
configured. (note, I have no idea what you are referring to for rtpsrc)

> Here is our other setting related to the gsteamer, I really am in the dark.
> Hope someone can point me out some directions regarding the lipsyn in the
> multicasting mode:
> Tx:
>
> Rx:
>
> g_object_set(G_OBJECT(rtpbin), "latency", 200, "buffer-mode", 0, "ntp-sync",
> TRUE, "use-pipeline-clock", TRUE, NULL);

You can try increasing the latency parameter, the rest seems to depend
on your use case, and on how the sender side is implemented. Normally,
200ms shall be sufficient, but that depends on many factor, and the
quality you want to achieve. You should consider having a second look at
how you are encoding your video. If you are using a public encoder, you
could expose the settings you use here.

regards,
Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nathan]

Hi Nicolas,

Sorry, I was off the office for one week. Really appreciated your attention on this poster.
By RTPSrc, vsource, asource in fact they are udpsrc.
I will try to increase the latency and put the sync as TRUE and give it for another try.

Thanks,

John

[Nathan]

Dear Nicolas,

The gstreamer use RTCP to guarantee audio video sync, and when there is only one receiver, this receiver(udpsrc) will report rtcp packages, which contains the timing information to the transmitter（udpsink), then the transmitter can adjust the delay between audio and video pipeline.
However, when there is multiple receivers, (I might be wrong here) so there are multiple receivers reporting the rtcp packages to the transmitter. I was thinking this might be the reason when there is multiple receivers, the lip sync is gone.

My question is, shoud I set the rtcp related pipeline works in the multcasting? How the transmitter pipeline handles different rtcp packets from different receiver?

Do yo you have any idea about this?

Thanks very much,

Jun