GStreamer-devel

How to reduce avdec_h264 latency

Classic

List

Threaded

9 messages Options

paolo perotinus

How to reduce avdec_h264 latency

Hello,

I can play a HTTP-mpegts-H264 stream, with 640x480@25fps and zerolatency settings, with this command, which I translated into C code:

"gst-launch-1.0 -v souphttpsrc location=http://127.0.0.1:8080/stream.ts ! tsdemux name=demuxer demuxer. ! h264parse ! avdec_h264 ! autovideosink"

Then I measured that the avdec_h264 decoder adds a latency of ~70ms, which corresponds to the interval between two GstBuffers with the same timestamp: the first one taken from the h264parse's src, and the second one taken from the avdec_h264's src. It corresponds to about two frames of the stream, so it seems to me that the avdec_h264 does some buffering.

Is there a way to reduce this latency and/or reduce/avoid buffering in the decoder?

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Olivier Crête-3

Re: How to reduce avdec_h264 latency

Hi,

The only way to do that is to create a stream with no B-frames, for example using the H. 264 baseline profile.

Olivier

On July 10, 2017 6:57:16 PM EDT, paolo perotinus <[hidden email]> wrote:

>Hello,
>I can play a HTTP-mpegts-H264 stream, with 640x480@25fps and
>zerolatency settings, with this command, which I translated into C
>code:
>"gst-launch-1.0 -v souphttpsrc location=http://127.0.0.1:8080/stream.ts
>! tsdemux name=demuxer demuxer. ! h264parse ! avdec_h264 !
>autovideosink"
>Then I measured that the avdec_h264 decoder adds a latency of ~70ms,
>which corresponds to the interval between two GstBuffers with the same
>timestamp: the first one taken from the h264parse's src, and the second
>one taken from the avdec_h264's src. It corresponds to about two frames
>of the stream, so it seems to me that the avdec_h264 does some
>buffering.
>Is there a way to reduce this latency and/or reduce/avoid buffering in
>the decoder?

--
Olivier Crête
[hidden email]
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

paolo perotinus

Re: How to reduce avdec_h264 latency

Il Martedì 11 Luglio 2017 1:06, Olivier Crête <[hidden email]> ha scritto:

> Hi,

> The only way to do that is to create a stream with no B-frames, for example using the H. 264 baseline profile.

> Olivier

Hi Olivier,

The stream is already coded with no B-frames (zerolatency tune in ffmpeg) but I still have that latency in avdec_h264...

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Lijia (George Lee, Euler)

答复: How to reduce avdec_h264 latency

Pls take a bit of time to delve into system –level thread scheduling in attempt to find clue to long latency in plugin implementation. As per experience excluding bitstream encoding option, in many situations we often can find negative impact brought by threading scheduling over Linux. Even though bitstream has been encoded very well for low-latency use-case such as removal of B-frame/sub-frame processing (parallel pipeline), implicit threading scheduling defect is hard to find.

Best Regards,

George Lee

发件人: gstreamer-devel [mailto:[hidden email]] 代表 paolo perotinus
发送时间: 2017年7月11日 17:45
收件人: Discussion of the development of and with GStreamer <[hidden email]>
主题: Re: How to reduce avdec_h264 latency

Il Martedì 11 Luglio 2017 1:06, Olivier Crête <[hidden email]> ha scritto:

> Hi,

> The only way to do that is to create a stream with no B-frames, for example using the H. 264 baseline profile.

> Olivier

Hi Olivier,

The stream is already coded with no B-frames (zerolatency tune in ffmpeg) but I still have that latency in avdec_h264...

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

paolo perotinus

Re: 答复: How to reduce avdec_h264 latency

I just solved the problem (thanks to Tim-Philipp Muller) by setting is-live=1 on the receiver. The latency was added in gstavviddec.c (check the line with: "gst_query_parse_latency (query, &is_live, NULL, NULL);" ).

However I think that this option (is-live) can appear too generic and misleading, and should be removed/changed.

What do you think about?

Il Martedì 11 Luglio 2017 12:40, "Lijia (George Lee, Euler)" <[hidden email]> ha scritto:

Best Regards,

George Lee

Il Martedì 11 Luglio 2017 1:06, Olivier Crête <[hidden email]> ha scritto:

> Hi,

> The only way to do that is to create a stream with no B-frames, for example using the H. 264 baseline profile.

> Olivier

Hi Olivier,

The stream is already coded with no B-frames (zerolatency tune in ffmpeg) but I still have that latency in avdec_h264...

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Lijia (George Lee, Euler)

答复: 答复: How to reduce avdec_h264 latency

This option is-live instructs GST h264 decoder element to perform low latency mode for Live event, thus this option takes role of key configuration of pipeline behavior in response to real-time decoding. In my opinion it should not be removed but more elaborate description is also needed to avoid misleading programmer.

George

发件人: gstreamer-devel [mailto:[hidden email]] 代表 paolo perotinus
发送时间: 2017年7月11日 18:50
收件人: Discussion of the development of and with GStreamer <[hidden email]>
主题: Re: 答复: How to reduce avdec_h264 latency

However I think that this option (is-live) can appear too generic and misleading, and should be removed/changed.

What do you think about?

Il Martedì 11 Luglio 2017 12:40, "Lijia (George Lee, Euler)" <[hidden email]> ha scritto:

Best Regards,

George Lee

发件人: gstreamer-devel [[hidden email]] 代表 paolo perotinus
发送时间: 2017年7月11日 17:45
收件人: Discussion of the development of and with GStreamer <[hidden email]>
主题: Re: How to reduce avdec_h264 latency

Il Martedì 11 Luglio 2017 1:06, Olivier Crête <[hidden email]> ha scritto:

> Hi,

> The only way to do that is to create a stream with no B-frames, for example using the H. 264 baseline profile.

> Olivier

Hi Olivier,

The stream is already coded with no B-frames (zerolatency tune in ffmpeg) but I still have that latency in avdec_h264...

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Gst-Geek

Re: How to reduce avdec_h264 latency

In reply to this post by paolo perotinus

See if fine tuning max-threads in avdec_h264 reduces the latency.

Note: CPU need more than 1 thread to decode HD streams. Check how many threads required to decode stream

paolo perotinus

Re: 答复: 答复: How to reduce avdec_h264 latency

In reply to this post by Lijia (George Lee, Euler)

More precisely, in gstavviddec.c there are the following lines:

query = gst_query_new_latency ();

is_live = FALSE;

/* Check if upstream is live. If it isn't we can enable frame based

* threading, which is adding latency */

if (gst_pad_peer_query (GST_VIDEO_DECODER_SINK_PAD (ffmpegdec), query)) {

gst_query_parse_latency (query, &is_live, NULL, NULL);

}

gst_query_unref (query);

if (is_live)

ffmpegdec->context->thread_type = FF_THREAD_SLICE;

else

ffmpegdec->context->thread_type = FF_THREAD_SLICE | FF_THREAD_FRAME;

Then, latency is introduced depending on "is_live" property, checked by the query. Now, I don't understand how to set the "is_live" property in the code of my receiver. I tried with:

  GstPad* pad_decoder_sink = gst_element_get_static_pad (data.video_decoder, "sink");
  GstQuery *query;
  query = gst_query_new_latency ();
  if (gst_pad_peer_query (pad_decoder, query)) {
     gst_query_set_latency (query, TRUE,  0, 0);        
  }

gst_object_unref (pad_decoder_sink);

The gst_query_set_latency() function is called, but gst_query_parse_latency (query, &is_live, NULL, NULL) inside gstavviddec.c returns FALSE for "is_live".

Why? How can I set that property?

Il Martedì 11 Luglio 2017 12:57, "Lijia (George Lee, Euler)" <[hidden email]> ha scritto:

George

However I think that this option (is-live) can appear too generic and misleading, and should be removed/changed.

What do you think about?

Il Martedì 11 Luglio 2017 12:40, "Lijia (George Lee, Euler)" <[hidden email]> ha scritto:

Best Regards,

George Lee

Il Martedì 11 Luglio 2017 1:06, Olivier Crête <[hidden email]> ha scritto:

> Hi,

> The only way to do that is to create a stream with no B-frames, for example using the H. 264 baseline profile.

> Olivier

Hi Olivier,

The stream is already coded with no B-frames (zerolatency tune in ffmpeg) but I still have that latency in avdec_h264...

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

noscgag

Re: 答复: 答复: How to reduce avdec_h264 latency

In reply to this post by Lijia (George Lee, Euler)

Is the "is-live" setting only available programmatically? If it will improve latency, I'd like to use it on the decode (receiver) side, but I use gst_parse_launch() to create my pipeline.

I did a gst-inspect on all the GStreamer elements (especially avdec_h264) and as far as I could tell only the source elements support the "is-live" setting (i.e. videotestsrc, souphttpsrc, gltestsrc, fakesrc, audiotestsrc, appsrc). This was with 1.10.4.

Gary

From: "Lijia (George Lee, Euler)" <[hidden email]>
To: paolo perotinus <[hidden email]>; Discussion of the development of and with GStreamer <[hidden email]>
Sent: Tuesday, July 11, 2017 3:58 AM
Subject: 答复: 答复: How to reduce avdec_h264 latency

George

However I think that this option (is-live) can appear too generic and misleading, and should be removed/changed.

What do you think about?

Il Martedì 11 Luglio 2017 12:40, "Lijia (George Lee, Euler)" <[hidden email]> ha scritto:

Best Regards,

George Lee

Il Martedì 11 Luglio 2017 1:06, Olivier Crête <[hidden email]> ha scritto:

> Hi,

> The only way to do that is to create a stream with no B-frames, for example using the H. 264 baseline profile.

> Olivier

Hi Olivier,

The stream is already coded with no B-frames (zerolatency tune in ffmpeg) but I still have that latency in avdec_h264...

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel