Measure pipeline I/O latency

[Lijia (George Lee, Euler)]

I am working on ultra-low latency pipeline design for GStreamer 1.10.x, with that goal the primary step should measure the total real processing latency across each element which the overall pipeline makes up of, i.e. pipeline I/O latency. Even though there is latency compensation mechanism from source to sink in the pipeline, but it is still not what I am looking for.

 

On the other hand I am not sure whether existing GStreamer release has built-in latency metrics mechanism for respective each element or overall pipeline? Maybe we need re-invent such kind of this metrics for GStreamer.

 

 

George Lee

 

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Arun Raghavan-4]

On Wed, 21 Dec 2016, at 12:24 PM, Lijia (George Lee, Euler) wrote:

> I am working on ultra-low latency pipeline design for GStreamer 1.10.x,
> with that goal the primary step should measure the total real processing
> latency across each element which the overall pipeline makes up of, i.e.
> pipeline I/O latency. Even though there is latency compensation mechanism
> from source to sink in the pipeline, but it is still not what I am
> looking for.
>
> On the other hand I am not sure whether existing GStreamer release has
> built-in latency metrics mechanism for respective each element or overall
> pipeline? Maybe we need re-invent such kind of this metrics for
> GStreamer.

We do have the GstTracer framework which might let you break things down
to at least how much time a buffer spends within each element, but if
you're looking for sources of I/O latency, you might find something like
latencytop or ftrace more helpful (since they are able to perform more
system-wide profiling which might be relevant in the case of I/O).

-- Arun
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Lijia (George Lee, Euler)]

GstTracer framework indeed can help identify and calculate the time it takes for gst buffer traverses from source to sink in the pipeline, but that time DOES NOT reflect real latency metrics of pipeline performance. In terms of ultra-low latency design goal of media engine that consists of GStreamer sub-system and underlying video/audio driver stack such as demux/decoder/display(render) building block, it is essential to know breakdown of latency for GStreamer and media driver stack, after that we also can know where to optimize or re-work plugin architecture for attaining ultra-low latency goal at endpoint side. However GstTracer is less helpful in that regard coz the data gathered from it does not cover entire situation.

In my opinion maybe for next formal release of GStreamer, we should consider this new feature which in real-world is very stunning to developer.

-----邮件原件-----
发件人: Arun Raghavan [mailto:[hidden email]]
发送时间: 2016年12月21日 15:06
收件人: Lijia (George Lee, Euler) <[hidden email]>; [hidden email]
主题: Re: Measure pipeline I/O latency

On Wed, 21 Dec 2016, at 12:24 PM, Lijia (George Lee, Euler) wrote:

> I am working on ultra-low latency pipeline design for GStreamer
> 1.10.x, with that goal the primary step should measure the total real
> processing latency across each element which the overall pipeline makes up of, i.e.
> pipeline I/O latency. Even though there is latency compensation
> mechanism from source to sink in the pipeline, but it is still not
> what I am looking for.
>
> On the other hand I am not sure whether existing GStreamer release has
> built-in latency metrics mechanism for respective each element or
> overall pipeline? Maybe we need re-invent such kind of this metrics
> for GStreamer.

We do have the GstTracer framework which might let you break things down to at least how much time a buffer spends within each element, but if you're looking for sources of I/O latency, you might find something like latencytop or ftrace more helpful (since they are able to perform more system-wide profiling which might be relevant in the case of I/O).

-- Arun
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Stefan Sauer]

On 12/21/2016 09:36 AM, Lijia (George Lee, Euler) wrote:

GstTracer framework indeed can help identify and calculate the time it takes for gst buffer traverses from source to sink in the pipeline, but that time DOES NOT reflect real latency metrics of pipeline performance. In terms of ultra-low latency design goal of media engine that consists of GStreamer sub-system and underlying video/audio driver stack such as demux/decoder/display(render) building block, it is essential to know breakdown of latency for GStreamer and media driver stack, after that we also can know where to optimize or re-work plugin architecture for attaining ultra-low latency goal at endpoint side. However GstTracer is less helpful in that regard coz the data gathered from it does not cover entire situation. 

First step is to formulate the exact question we want to get answers for - 'getting a latency break down' is unfortunately too high level. Look at the definition of latency, e.g.
" Latency is the delay from input into a system to desired outcome".

In the case of playing a video latency is only an issue if the sum of the latencies in each element is more that what we require to play video (the frame rate and audio rate detemining the upper bounds). Now there are many thing that make this more complicated. E.g. when parsing the container format we aren't yet constarined by the frame rate, since we don't play yet. But this is also a kind of latency (the preroll-time). Once we're playing we also need to consider buffering, so if we have buffering peak latencies before the buffer are not fatal, but the overall latency increases.

In general we can get a lot of data out of a running pipleine with the tracer. So if we have concrete questions that we seek answers for we can write tracer tools that post process the logs to answer those and extend the tracers as needed.

Stefan

In my opinion maybe for next formal release of GStreamer, we should consider this new feature which in real-world is very stunning to developer.

-----邮件原件-----
发件人: Arun Raghavan [[hidden email]] 
发送时间: 2016年12月21日 15:06
收件人: Lijia (George Lee, Euler) [hidden email]; [hidden email]
主题: Re: Measure pipeline I/O latency

On Wed, 21 Dec 2016, at 12:24 PM, Lijia (George Lee, Euler) wrote:

I am working on ultra-low latency pipeline design for GStreamer 
1.10.x, with that goal the primary step should measure the total real 
processing latency across each element which the overall pipeline makes up of, i.e.
pipeline I/O latency. Even though there is latency compensation 
mechanism from source to sink in the pipeline, but it is still not 
what I am looking for.

On the other hand I am not sure whether existing GStreamer release has 
built-in latency metrics mechanism for respective each element or 
overall pipeline? Maybe we need re-invent such kind of this metrics 
for GStreamer.

We do have the GstTracer framework which might let you break things down to at least how much time a buffer spends within each element, but if you're looking for sources of I/O latency, you might find something like latencytop or ftrace more helpful (since they are able to perform more system-wide profiling which might be relevant in the case of I/O).

-- Arun
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel