vaapidecode vs avdec_h264 latency

[Joona Laine]

Hi folks,

I'm doing some FHD real time streaming stuff and I'm curious about the latency I'm experiencing using two different decoders. The pipeline I'm using is the following:

gst-launch-1.0 rtspsrc location=rtsp://root:root@10.128.1.88/axis-media/media.amp?videocodec=h264 latency=0 ! rtph264depay ! h264parse ! queue ! avdec_h264 ! vaapisink

That gives me a more or less solid end-to-end latency of 130 ms. However, if I change the decoder to vaapidecode I can see a drop in CPU load (which is what I need), but the unfortunate drawback is the added 70 ms of latency.

I'm just curious where that comes from and is there anything I can do about it?

I know this much:
  - vaapi needs to buffer whole frame(s) before the data is transferred to GPU memory, and this adds latency compared to sw decoding
  - in the vaapi source code there has been defined a constant latency for the pipeline (dunno if I should just change that)

My system includes
  - i7-6700
  - Intel HD 530
  - Ubuntu 16.04
  - GStreamer 1.8.1 + plugins
  - libva-intel-driver-1.7.0


Thanks,

Joona

[Jiang Pin]

I found the same issue. Any further findings or conclusions on this ?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Victor Jaquez]

On Tue, 21 Nov 2017 at 21:10, Jiang Pin wrote:
> I found the same issue. Any further findings or conclusions on this ?

In gstreamer-vaapi master, vaapih264dec has a new property: 'low-latency', for
those streams that break the H264 spec and it is needed to flush the DPB as soon
as a frame is generated.

vmjl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Jiang Pin]

Great to hear that. When was the feature being introduced? I am currently
using 1.8.3 from ubuntu 16.04 official repo. Does it mean I need to compile
a recent version from scratch?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Victor Jaquez]

On Fri, 24 Nov 2017 at 00:05, Jiang Pin wrote:
> Great to hear that. When was the feature being introduced?

In July.

> I am currently using 1.8.3 from ubuntu 16.04 official repo. Does it mean I
> need to compile a recent version from scratch?

For now, yes.

vmjl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Jiang Pin]

Hi, Jaquez

      I take a try to compile the master branch from
git://anongit.freedesktop.org/gstreamer/gstreamer-vaapi. But it looks it
cannnot be done right now, it indicate following errors:

gstvaapiencoder_h264.c:2500:60: error: 'struct <anonymous>' has no member
named 'mb_rate_control'
   GST_VAAPI_ENCODER_VA_RATE_CONTROL (encoder).rc_flags.bits.mb_rate_control
=
                                                            ^
Makefile:1718: recipe for target 'libgstvaapi_la-gstvaapiencoder_h264.lo'
failed
make[4]: *** [libgstvaapi_la-gstvaapiencoder_h264.lo] Error 1
make[4]: *** Waiting for unfinished jobs....
gstvaapiencoder_h265.c: In function 'ensure_control_rate_params':
gstvaapiencoder_h265.c:1836:60: error: 'struct <anonymous>' has no member
named 'mb_rate_control'
   GST_VAAPI_ENCODER_VA_RATE_CONTROL (encoder).rc_flags.bits.mb_rate_control
=

       So when will a workable version be available?






--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Victor Jaquez]

On Sun, 26 Nov 2017 at 18:53, Jiang Pin wrote:

> Hi, Jaquez
>
>       I take a try to compile the master branch from
> git://anongit.freedesktop.org/gstreamer/gstreamer-vaapi. But it looks it
> cannnot be done right now, it indicate following errors:
>
> gstvaapiencoder_h264.c:2500:60: error: 'struct <anonymous>' has no member
> named 'mb_rate_control'
>    GST_VAAPI_ENCODER_VA_RATE_CONTROL (encoder).rc_flags.bits.mb_rate_control
> =
>                                                             ^
> Makefile:1718: recipe for target 'libgstvaapi_la-gstvaapiencoder_h264.lo'
> failed
> make[4]: *** [libgstvaapi_la-gstvaapiencoder_h264.lo] Error 1
> make[4]: *** Waiting for unfinished jobs....
> gstvaapiencoder_h265.c: In function 'ensure_control_rate_params':
> gstvaapiencoder_h265.c:1836:60: error: 'struct <anonymous>' has no member
> named 'mb_rate_control'
>    GST_VAAPI_ENCODER_VA_RATE_CONTROL (encoder).rc_flags.bits.mb_rate_control
> =
>
>        So when will a workable version be available?

Which version of libva are you using?

vmjl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Victor Jaquez]

On Mon, 27 Nov 2017 at 11:56, Víctor Jáquez wrote:

> On Sun, 26 Nov 2017 at 18:53, Jiang Pin wrote:
> > Hi, Jaquez
> >
> >       I take a try to compile the master branch from
> > git://anongit.freedesktop.org/gstreamer/gstreamer-vaapi. But it looks it
> > cannnot be done right now, it indicate following errors:
> >
> > gstvaapiencoder_h264.c:2500:60: error: 'struct <anonymous>' has no member
> > named 'mb_rate_control'
> >    GST_VAAPI_ENCODER_VA_RATE_CONTROL (encoder).rc_flags.bits.mb_rate_control
> > =
> >                                                             ^
> > Makefile:1718: recipe for target 'libgstvaapi_la-gstvaapiencoder_h264.lo'
> > failed
> > make[4]: *** [libgstvaapi_la-gstvaapiencoder_h264.lo] Error 1
> > make[4]: *** Waiting for unfinished jobs....
> > gstvaapiencoder_h265.c: In function 'ensure_control_rate_params':
> > gstvaapiencoder_h265.c:1836:60: error: 'struct <anonymous>' has no member
> > named 'mb_rate_control'
> >    GST_VAAPI_ENCODER_VA_RATE_CONTROL (encoder).rc_flags.bits.mb_rate_control
> > =
> >
> >        So when will a workable version be available?
>
> Which version of libva are you using?

I just figured out: 1.8.3.

But we have a bug, because mb_rate_control was introduced in libva 2.0.0, thus
we need to guard those instructions.

In order to have a workable version of gstreamer-vaapi we need your valuable
testing, as you found out. Thanks.

vmjl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Victor Jaquez]

On Sun, 26 Nov 2017 at 18:53, Jiang Pin wrote:

> Hi, Jaquez
>
>       I take a try to compile the master branch from
> git://anongit.freedesktop.org/gstreamer/gstreamer-vaapi. But it looks it
> cannnot be done right now, it indicate following errors:
>
> gstvaapiencoder_h264.c:2500:60: error: 'struct <anonymous>' has no member
> named 'mb_rate_control'
>    GST_VAAPI_ENCODER_VA_RATE_CONTROL (encoder).rc_flags.bits.mb_rate_control
> =
>                                                             ^
> Makefile:1718: recipe for target 'libgstvaapi_la-gstvaapiencoder_h264.lo'
> failed
> make[4]: *** [libgstvaapi_la-gstvaapiencoder_h264.lo] Error 1
> make[4]: *** Waiting for unfinished jobs....
> gstvaapiencoder_h265.c: In function 'ensure_control_rate_params':
> gstvaapiencoder_h265.c:1836:60: error: 'struct <anonymous>' has no member
> named 'mb_rate_control'
>    GST_VAAPI_ENCODER_VA_RATE_CONTROL (encoder).rc_flags.bits.mb_rate_control
> =
>
>        So when will a workable version be available?

Please pull the latest version (there's a fix for this issue) and try it.

vmjl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Jiang Pin]

well, I have got the new master branch and gstreamer-vaapi compiled. I didn't
spend a lot time reading gstreamer vaapi code. So how to enable low-latency
properties in gstreamer pipeline?

          gst-launch-1.0 udpsrc port=5000 ! vaapidecode low-latency=1 ！
autovideosink



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel