Regarding pipeline latency and how to achieve a low-latency pipeline

[danny.smith]

Hi!

I am working on a networked scenario in which one sender will transmit audio to multiple receivers whom will render the audio synchronized, following the description in this presentation:

https://gstreamer.freedesktop.org/data/events/gstreamer-conference/2015/Sebastian%20Dr%C3%B6ge%20-%20Synchronised%20multi-room%20media%20playback%20and%20distributed%20live%20media%20processing%20and%20mixing%20with%20GStreamer.pdf

and have some questions:

1) Why is setting the total pipeline latency necessary if the receivers are identical?

2) Which element will handle the additional buffering required by setting the total pipeline latency higher than the minimum latency?

3) what is the explanation for min/max latency for a pipeline?

4) For the recievers; If I set a jitterbuffer latency of 100ms and alsasink buffer-time of 100ms, will this result in a total delay of 200ms?

5) I am using the gstreamer netclock to achieve synchronized playback, sometimes during periods of high load on my network I get simultaneous skew on all my recievers (etime drifts roughly the same amount in the same direction). When the network load decreases this drift disappears. Is there a explanation for this and maybe a solution?

6) What would be the best approach to achieve a <100ms latency "best effort" pipeline?

Would be grateful is someone could shed some light on the above topics :)

Regards,
Danny

[Nicolas Dufresne-5]

Le lundi 20 mars 2017 à 01:08 -0700, danny.smith a écrit :

> Hi!
>
> I am working on a networked scenario in which one sender will transmit audio
> to multiple receivers whom will render the audio synchronized, following the
> description in this presentation:
>
> https://gstreamer.freedesktop.org/data/events/gstreamer-conference/2015/Sebastian%20Dr%C3%B6ge%20-%20Synchronised%20multi-room%20media%20playback%20and%20distributed%20live%20media%20processing%20and%20mixing%20with%20GStreamer.pdf
>
> and have some questions:
>
> 1) Why is setting the total pipeline latency necessary if the receivers are
> identical?

You may need to add extra latency in case one devices takes a bit more
time to setup. Also, most of the time, the signalling is not done in
parallel, so there is a small delay.

>
> 2) Which element will handle the additional buffering required by setting
> the total pipeline latency higher than the minimum latency?

The elements that have max_latency > min_latency in general. Most audio
src/sink, queues, etc.

>
> 3) what is the explanation for min/max latency for a pipeline?

If you go lower then min latency, all frames will be late. Max latency,
is the duration that can be accumulated before data starts being
dropped.

>
> 4) For the recievers; If I set a jitterbuffer latency of 100ms and alsasink
> buffer-time of 100ms, will this result in a total delay of 200ms?

Or more. The latency is a sum of each element latency.

>
> 5) I am using the gstreamer netclock to achieve synchronized playback,
> sometimes during periods of high load on my network I get simultaneous skew
> on all my recievers (etime drifts roughly the same amount in the same
> direction). When the network load decreases this drift disappears. Is there
> a explanation for this and maybe a solution?

Reduce the network load or increase your bandwidth. The clock could
probably be improved for this case, but that would need research. I
don't myself know the details of this clock design.

>
> 6) What would be the best approach to achieve a <100ms latency "best effort"
> pipeline?

Just setup your latencies so that they are below that level. Test to
make sure it does not stutter.

>
> Would be grateful is someone could shed some light on the above topics :)
>

Feel free to ask more question, and go in depth, we'll help as much as we can.

regards,
Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[danny.smith]

Thank you Nicolas!

Do I need to manage base_time manually for this scenario (one sender, multiple synchronised receivers) or is it handled automatically? I have seen some examples were the servers sender pipeline base_time was propagated to the clients whom applied it to their receiver pipelines.

From what I read in the documentation the rtpjitterbuffer will use the timestamp in the first received buffer as the base_time. What would happen if for instance the sender rebooted and we started up the sender pipeline with a new base_time? Would probably need to restart the receiver pipelines again to get the correct base_time?

Regards,
Danny