GStreamer-devel

vaapih264dec - latency

Classic

List

Threaded

7 messages Options

Weber, Holger

vaapih264dec - latency

I try to get the vaapih264dec running for low latency streaming. I have good results if I use the avdec_h264 module but if I use the vaapih264dec the latency increases.

Pipeline (software):

gst-launch-1.0 rtspsrc drop-on-latency=true latency=0 location=$SRC ! rtpmp2tdepay ! tsdemux ! h264parse ! avdec_h264 ! glimagesink sync=false

Pipeline (hardware):

gst-launch-1.0 rtspsrc drop-on-latency=true latency=0 location=$SRC ! rtpmp2tdepay ! tsdemux ! h264parse ! vaapih264dec ! glimagesink sync=false

The encoder uses intra refresh and never sends a whole I-frame. So at the beginning vaapih264dec dropped all frames.

I modified gstvaapidecoder_h264.c:

###################################################

priv->decoder_state |= sps_pi->state;

if (!(priv->decoder_state & GST_H264_VIDEO_STATE_GOT_I_FRAME)) {

/* removed don't wait for a valid I-Frame (intra refresh problem)

if (priv->decoder_state & GST_H264_VIDEO_STATE_GOT_P_SLICE)

goto drop_frame;

###################################################

After that modification the stream started but the latency is still about 150ms higher as with avdec_h264. I guess this has something to do with frame reordering. Some decoders collect some frames and reorder them. In this case the encoder does no reordering because for low latency but the decoder seem to collect the frames.

Does someone have an idea where the behavior can be changed e.g. reduce frame buffer to 0/1?

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Victor Jaquez

Re: vaapih264dec - latency

On Fri, 01 Sep 2017 at 11:24, Weber, Holger wrote:
> I try to get the vaapih264dec running for low latency streaming. I have good
> results if I use the avdec_h264 module but if I use the vaapih264dec the
> latency increases.

Have you tried with the new vaapih264dec's 'low-latency' property in master?

>
> Pipeline (software):
> gst-launch-1.0 rtspsrc drop-on-latency=true latency=0 location=$SRC ! rtpmp2tdepay ! tsdemux ! h264parse ! avdec_h264 ! glimagesink sync=false
>
> Pipeline (hardware):
> gst-launch-1.0 rtspsrc drop-on-latency=true latency=0 location=$SRC ! rtpmp2tdepay ! tsdemux ! h264parse ! vaapih264dec ! glimagesink sync=false
>
> The encoder uses intra refresh and never sends a whole I-frame. So at the beginning vaapih264dec dropped all frames.
> I modified gstvaapidecoder_h264.c:
> ###################################################
> priv->decoder_state |= sps_pi->state;
> if (!(priv->decoder_state & GST_H264_VIDEO_STATE_GOT_I_FRAME)) {
> /* removed don't wait for a valid I-Frame (intra refresh problem)
> if (priv->decoder_state & GST_H264_VIDEO_STATE_GOT_P_SLICE)
> goto drop_frame;
> */
> ###################################################
>
> After that modification the stream started but the latency is still about 150ms higher as with avdec_h264. I guess this has something to do with frame reordering. Some decoders collect some frames and reorder them. In this case the encoder does no reordering because for low latency but the decoder seem to collect the frames.
>
> Does someone have an idea where the behavior can be changed e.g. reduce frame buffer to 0/1?

Weber, Holger

AW: vaapih264dec - latency

I upgraded to the master version which has the suggested "low-latency" flag.
But because of the massive code changes I don't know how I can get it running again.
It is waiting for the I-Frame which will never come....

Any suggestions where the frames are dropped?

Regards

On Fri, 01 Sep 2017 at 11:24, Weber, Holger wrote:
> I try to get the vaapih264dec running for low latency streaming. I
> have good results if I use the avdec_h264 module but if I use the
> vaapih264dec the latency increases.

Have you tried with the new vaapih264dec's 'low-latency' property in master?

>
> Pipeline (software):
> gst-launch-1.0 rtspsrc drop-on-latency=true latency=0 location=$SRC !
> rtpmp2tdepay ! tsdemux ! h264parse ! avdec_h264 ! glimagesink
> sync=false
>
> Pipeline (hardware):
> gst-launch-1.0 rtspsrc drop-on-latency=true latency=0 location=$SRC !
> rtpmp2tdepay ! tsdemux ! h264parse ! vaapih264dec ! glimagesink
> sync=false
>
> The encoder uses intra refresh and never sends a whole I-frame. So at the beginning vaapih264dec dropped all frames.
> I modified gstvaapidecoder_h264.c:
> ###################################################
> priv->decoder_state |= sps_pi->state;
> if (!(priv->decoder_state & GST_H264_VIDEO_STATE_GOT_I_FRAME)) {
> /* removed don't wait for a valid I-Frame (intra refresh problem)
> if (priv->decoder_state & GST_H264_VIDEO_STATE_GOT_P_SLICE)
> goto drop_frame;
> */
> ###################################################
>
> After that modification the stream started but the latency is still about 150ms higher as with avdec_h264. I guess this has something to do with frame reordering. Some decoders collect some frames and reorder them. In this case the encoder does no reordering because for low latency but the decoder seem to collect the frames.
>
> Does someone have an idea where the behavior can be changed e.g. reduce frame buffer to 0/1?

> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Weber, Holger

AW: vaapih264dec - latency

Sorry my fault. I looked into the wrong file. Everything is working as expected and low latency is now working!
But I guess the my "I-Frame" patch is not correct. Has somebody an idea how to fix this the right way?

Regards

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Weber, Holger
Gesendet: Montag, 4. September 2017 10:33
An: Discussion of the development of and with GStreamer
Betreff: AW: vaapih264dec - latency

I upgraded to the master version which has the suggested "low-latency" flag.
But because of the massive code changes I don't know how I can get it running again.
It is waiting for the I-Frame which will never come....

Any suggestions where the frames are dropped?

Regards

On Fri, 01 Sep 2017 at 11:24, Weber, Holger wrote:
> I try to get the vaapih264dec running for low latency streaming. I
> have good results if I use the avdec_h264 module but if I use the
> vaapih264dec the latency increases.

Have you tried with the new vaapih264dec's 'low-latency' property in master?

Nicolas Dufresne-5

Re: AW: vaapih264dec - latency

Le lundi 04 septembre 2017 à 10:02 +0000, Weber, Holger a écrit :
> Sorry my fault. I looked into the wrong file. Everything is working
> as expected and low latency is now working!
> But I guess the my "I-Frame" patch is not correct. Has somebody an
> idea how to fix this the right way?

The decoder should detect intra-refresh streams and do the right thing
automatically. I bet this was just never tested.

regards,
Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Weber, Holger

AW: AW: vaapih264dec - latency

Where can I raise the bug / feature request?

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Nicolas Dufresne
Gesendet: Montag, 4. September 2017 15:22
An: Discussion of the development of and with GStreamer
Betreff: Re: AW: vaapih264dec - latency

Le lundi 04 septembre 2017 à 10:02 +0000, Weber, Holger a écrit :
> Sorry my fault. I looked into the wrong file. Everything is working as
> expected and low latency is now working!
> But I guess the my "I-Frame" patch is not correct. Has somebody an
> idea how to fix this the right way?

The decoder should detect intra-refresh streams and do the right thing automatically. I bet this was just never tested.

regards,
Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Nicolas Dufresne-5

Re: AW: AW: vaapih264dec - latency

Le lundi 04 septembre 2017 à 15:04 +0000, Weber, Holger a écrit :
> Where can I raise the bug / feature request?

https://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer

Component is gstreamer-vaapi.

regards,
Nicolas

>
>
> -----Ursprüngliche Nachricht-----
> Von: gstreamer-devel [mailto:[hidden email]
> p.org] Im Auftrag von Nicolas Dufresne
> Gesendet: Montag, 4. September 2017 15:22
> An: Discussion of the development of and with GStreamer
> Betreff: Re: AW: vaapih264dec - latency
>
> Le lundi 04 septembre 2017 à 10:02 +0000, Weber, Holger a écrit :
> > Sorry my fault. I looked into the wrong file. Everything is working
> > as
> > expected and low latency is now working!
> > But I guess the my "I-Frame" patch is not correct. Has somebody an
> > idea how to fix this the right way?
>
> The decoder should detect intra-refresh streams and do the right
> thing automatically. I bet this was just never tested.
>
> regards,
> Nicolas
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel