GStreamer-devel

Streaming freezes (vaapi or v4l2src)

Classic

List

Threaded

6 messages Options

Timtchenko, Michael

Streaming freezes (vaapi or v4l2src)

Hello guys,

i’ve encountered a very strange behavior while streaming from a v4l2src to a kmssink (or glimagesink) by using vaapi-components for decoding. The problem is, that the streaming starts for 2 seconds, then freezes for 2 seconds and continues again for 2 seconds and so on. This effect is only noticeable when using vaapi-components. If I use a pipeline with simply software decoding, the stream is displayed fluently.

My sending pipeline is constructed as follows:

gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video$(($1+4)) ! video/x-raw,format=YV12,width=720,height=576 ! v4l2h264enc ! rtph264pay ! udpsink host=${IP} port=55555

Setting pipeline to PAUSED ...

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

Redistribute latency...

The erroneous reading pipeline looks like this:

gst-launch-1.0 udpsrc port=55555 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! vaapih264dec ! vaapipostproc width=720 height=576 ! glimagesink

Setting pipeline to PAUSED ...

Pipeline is live and does not need PREROLL ...

Got context from element 'vaapipostproc0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"$GstVaapiDisplayDRM$\ vaapidisplaydrm1";

Setting pipeline to PLAYING ...

New clock: GstSystemClock

Redistribute latency...

WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.

Additional debug info:

../libs/gst/base/gstbasesink.c(3005): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:

There may be a timestamping problem, or this computer is too slow.

WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.

Additional debug info:

../libs/gst/base/gstbasesink.c(3005): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:

There may be a timestamping problem, or this computer is too slow.

If I run the following pipeline on the same system, the stream is presented fluently without any problems:

gst-launch-1. 0 udpsrc port=55555 caps=\"application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96\" ! rtph264depay ! avdec_h264 ! decodebin ! videoconvert ! kmssink

I’m not sure what exactly the problem is. Maybe the timestamping mechanism in vaapi has a different behavior then in the avdec_h264 as the log message says? I’ve also changed log levels but I couldn’t find anything helpful.

Any help for further diagnostics is highly appreciated.

Kind regards

Michael

Bitte beachten / Please note!

*******************************************************************************

AGCO GmbH
Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf, Germany
Registergericht Amtsgericht Kempten HRB 10327
Geschäftsführer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin, Dr. Heribert Reiter, Ekkehart Glaeser
Vorsitzender des Aufsichtsrates: Torsten Dehner

*******************************************************************************

Diese E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet ist und kann vertrauliches
bzw. unter das Berufsgeheimnis fallendes Material enthalten. Jegliche darin enthaltene Ansicht
oder Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise die Ansicht oder
Meinung von AGCO dar. Sind Sie nicht der Empfänger, so haben Sie diese E-Mail irrtümlich
erhalten und jegliche Verwendung, Veröffentlichung, Weiterleitung, Abschrift oder jeglicher
Druck dieser E-Mail ist strengstens untersagt. Weder AGCO noch der Absender übernehmen die
Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angehängte
Dateien (sofern vorhanden) auf Viren zu überprüfen.

*******************************************************************************

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Victor Jaquez

Re: Streaming freezes (vaapi or v4l2src)

On Thu, 16 Jul 2020 at 06:53, Timtchenko, Michael wrote:

> Hello guys,
>
> i've encountered a very strange behavior while streaming from a v4l2src to a
> kmssink (or glimagesink) by using vaapi-components for decoding. The problem
> is, that the streaming starts for 2 seconds, then freezes for 2 seconds and
> continues again for 2 seconds and so on. This effect is only noticeable when
> using vaapi-components. If I use a pipeline with simply software decoding, the
> stream is displayed fluently.
>
> My sending pipeline is constructed as follows:
>
> gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video$(($1+4)) !
> video/x-raw,format=YV12,width=720,height=576 ! v4l2h264enc ! rtph264pay !
> udpsink host=${IP} port=55555

You may tune the encoder to reduce the keyframe-period. Also you might choose a
good h264 profile (main or base-constrained) for encoding.

>
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> Redistribute latency...
>
> The erroneous reading pipeline looks like this:
>
> gst-launch-1.0 udpsrc port=55555
> caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,
> encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! vaapih264dec !
> vaapipostproc width=720 height=576 ! glimagesink

Perhaps a h264parser after rtph264depay would make sense. Not sure if it helps,
but in my head makes sense.

Try to avoid width and height vaapipostproc properties, they are deprecated. Use
a caps filter after it.

2 secs is a lot, but perhaps the default encoding creates large gops with big
latency. So two options: set a short keyframe-period in encoder, and in the
decoder set low-latency as true, in order to push out frames as soon as decoded
even if they are required to be kept as reference.

>
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> Got context from element 'vaapipostproc0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"$GstVaapiDisplayDRM$\ vaapidisplaydrm1";
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> Redistribute latency...
> WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.
> Additional debug info:
> ../libs/gst/base/gstbasesink.c(3005): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:
> There may be a timestamping problem, or this computer is too slow.
> WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.
> Additional debug info:
> ../libs/gst/base/gstbasesink.c(3005): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:
> There may be a timestamping problem, or this computer is too slow.

This is a different thing: it says that the decoder is producing more frames
than kmssink can render (above you use glimagesink, don't know if it is
different).

You can set sync=false to kmssink to render without clock time synchronization.

>
> If I run the following pipeline on the same system, the stream is presented
> fluently without any problems:
>
> gst-launch-1. 0 udpsrc port=55555 caps=\"application/x-rtp,
> media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264,
> payload=(int)96\"q ! rtph264depay ! avdec_h264 ! decodebin ! videoconvert !
> kmssink
>
> I'm not sure what exactly the problem is. Maybe the timestamping mechanism in
> vaapi has a different behavior then in the avdec_h264 as the log message says?
> I've also changed log levels but I couldn't find anything helpful.
>
> Any help for further diagnostics is highly appreciated.
>
> Kind regards
> Michael
>
>
>
>
>
> Bitte beachten / Please note!
>
> *******************************************************************************
>
> AGCO GmbH
> Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf, Germany
> Registergericht Amtsgericht Kempten HRB 10327
> Gesch?ftsf?hrer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin, Dr. Heribert Reiter, Ekkehart Glaeser
> Vorsitzender des Aufsichtsrates: Torsten Dehner
>
> *******************************************************************************
>
> Diese E-Mail ist nur f?r den Empf?nger bestimmt, an den es gerichtet ist und kann vertrauliches
> bzw. unter das Berufsgeheimnis fallendes Material enthalten. Jegliche darin enthaltene Ansicht
> oder Meinungs?u?erung ist die des Autors und stellt nicht notwendigerweise die Ansicht oder
> Meinung von AGCO dar. Sind Sie nicht der Empf?nger, so haben Sie diese E-Mail irrt?mlich
> erhalten und jegliche Verwendung, Ver?ffentlichung, Weiterleitung, Abschrift oder jeglicher
> Druck dieser E-Mail ist strengstens untersagt. Weder AGCO noch der Absender ?bernehmen die
> Haftung f?r Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angeh?ngte
> Dateien (sofern vorhanden) auf Viren zu ?berpr?fen.
>
> *******************************************************************************
>
> This email is intended solely for the use of the individual to whom it is addressed and may contain
> confidential and/or privileged material. Any views or opinions presented are solely those of the
> author and do not necessarily represent those of AGCO. If you are not the intended recipient, be
> advised that you have received this email in error and that any use, dissemination, forwarding,
> printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any
> responsibility for viruses and it is your responsibility to scan and virus check the email and its
> attachment(s) (if any).
>
> *******************************************************************************

> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Nicolas Dufresne-5

Re: Streaming freezes (vaapi or v4l2src)

In reply to this post by Timtchenko, Michael

Le jeu. 16 juill. 2020 04 h 00, Timtchenko, Michael <[hidden email]> a écrit :

Hello guys,

i’ve encountered a very strange behavior while streaming from a v4l2src to a kmssink (or glimagesink) by using vaapi-components for decoding. The problem is, that the streaming starts for 2 seconds, then freezes for 2 seconds and continues again for 2 seconds and so on. This effect is only noticeable when using vaapi-components. If I use a pipeline with simply software decoding, the stream is displayed fluently.

Two seconds could be the key frame distance ?

My sending pipeline is constructed as follows:

gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video$(($1+4)) ! video/x-raw,format=YV12,width=720,height=576 ! v4l2h264enc ! rtph264pay ! udpsink host=${IP} port=55555

v4l2h264enc has this extra-controls property that let you genetically change he specific configuration, just an information.

Setting pipeline to PAUSED ...

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

Redistribute latency...

The erroneous reading pipeline looks like this:

gst-launch-1.0 udpsrc port=55555 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! vaapih264dec ! vaapipostproc width=720 height=576 ! glimagesink

One thing to note is the absence of rtpjitterbuffer element, which would allow adding some latency to remove network jitter. So that is one thing to try.

Another possibility is that the issue is cause by vaaph264dec having more latency then avdec_h264. Perhaps give the low-latency property a try ?

Let's hope one of these suggestions helps.

Setting pipeline to PAUSED ...

Pipeline is live and does not need PREROLL ...

Got context from element 'vaapipostproc0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"$GstVaapiDisplayDRM$\ vaapidisplaydrm1";

Setting pipeline to PLAYING ...

New clock: GstSystemClock

Redistribute latency...

WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.

Additional debug info:

../libs/gst/base/gstbasesink.c(3005): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:

There may be a timestamping problem, or this computer is too slow.

WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.

Additional debug info:

../libs/gst/base/gstbasesink.c(3005): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:

There may be a timestamping problem, or this computer is too slow.

If I run the following pipeline on the same system, the stream is presented fluently without any problems:

gst-launch-1. 0 udpsrc port=55555 caps=\"application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96\" ! rtph264depay ! avdec_h264 ! decodebin ! videoconvert ! kmssink

I’m not sure what exactly the problem is. Maybe the timestamping mechanism in vaapi has a different behavior then in the avdec_h264 as the log message says? I’ve also changed log levels but I couldn’t find anything helpful.

Any help for further diagnostics is highly appreciated.

Kind regards

Michael

Bitte beachten / Please note!

*******************************************************************************
AGCO GmbH
Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf, Germany
Registergericht Amtsgericht Kempten HRB 10327
Geschäftsführer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin, Dr. Heribert Reiter, Ekkehart Glaeser
Vorsitzender des Aufsichtsrates: Torsten Dehner

*******************************************************************************
Diese E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet ist und kann vertrauliches
bzw. unter das Berufsgeheimnis fallendes Material enthalten. Jegliche darin enthaltene Ansicht
oder Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise die Ansicht oder
Meinung von AGCO dar. Sind Sie nicht der Empfänger, so haben Sie diese E-Mail irrtümlich
erhalten und jegliche Verwendung, Veröffentlichung, Weiterleitung, Abschrift oder jeglicher
Druck dieser E-Mail ist strengstens untersagt. Weder AGCO noch der Absender übernehmen die
Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angehängte
Dateien (sofern vorhanden) auf Viren zu überprüfen.

*******************************************************************************
This email is intended solely for the use of the individual to whom it is addressed and may contain
confidential and/or privileged material. Any views or opinions presented are solely those of the
author and do not necessarily represent those of AGCO. If you are not the intended recipient, be
advised that you have received this email in error and that any use, dissemination, forwarding,
printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any
responsibility for viruses and it is your responsibility to scan and virus check the email and its
attachment(s) (if any).

*******************************************************************************

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Timtchenko, Michael

AW: [External] Re: Streaming freezes (vaapi or v4l2src)

Hi all,

sorry 4 my late response. I had a lot of work during the last two weeks.

I want to thank Nicolas and Victor for fast response and very good input!

I fixed the issue by using a rtpjitterbuffer in combination with the low-latency flag. Without the rtpjitterbuffer the image is getting teared. So I guess it’s necessary in this case.

The following pipeline works quite well in my case:

gst-launch-1.0 udpsrc port=55555 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtpjitterbuffer latency=10 ! rtph264depay ! vaapih264dec low-latency=true ! vaapipostproc ! kmssink

Timestamping the sink did not improve the misbehavior.

Best regards,

Michael

Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Nicolas Dufresne
Gesendet: Donnerstag, 16. Juli 2020 14:33
An: Discussion of the development of and with GStreamer <[hidden email]>
Betreff: [External] Re: Streaming freezes (vaapi or v4l2src)

Le jeu. 16 juill. 2020 04 h 00, Timtchenko, Michael <[hidden email]> a écrit :

Hello guys,

i’ve encountered a very strange behavior while streaming from a v4l2src to a kmssink (or glimagesink) by using vaapi-components for decoding. The problem is, that the streaming starts for 2 seconds, then freezes for 2 seconds and continues again for 2 seconds and so on. This effect is only noticeable when using vaapi-components. If I use a pipeline with simply software decoding, the stream is displayed fluently.

Two seconds could be the key frame distance ?

My sending pipeline is constructed as follows:

gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video$(($1+4)) ! video/x-raw,format=YV12,width=720,height=576 ! v4l2h264enc ! rtph264pay ! udpsink host=${IP} port=55555

v4l2h264enc has this extra-controls property that let you genetically change he specific configuration, just an information.

Setting pipeline to PAUSED ...

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

Redistribute latency...

The erroneous reading pipeline looks like this:

gst-launch-1.0 udpsrc port=55555 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! vaapih264dec ! vaapipostproc width=720 height=576 ! glimagesink

One thing to note is the absence of rtpjitterbuffer element, which would allow adding some latency to remove network jitter. So that is one thing to try.

Another possibility is that the issue is cause by vaaph264dec having more latency then avdec_h264. Perhaps give the low-latency property a try ?

Let's hope one of these suggestions helps.

Setting pipeline to PAUSED ...

Pipeline is live and does not need PREROLL ...

Got context from element 'vaapipostproc0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"$GstVaapiDisplayDRM$\ vaapidisplaydrm1";

Setting pipeline to PLAYING ...

New clock: GstSystemClock

Redistribute latency...

WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.

Additional debug info:

../libs/gst/base/gstbasesink.c(3005): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:

There may be a timestamping problem, or this computer is too slow.

WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.

Additional debug info:

../libs/gst/base/gstbasesink.c(3005): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:

There may be a timestamping problem, or this computer is too slow.

If I run the following pipeline on the same system, the stream is presented fluently without any problems:

gst-launch-1. 0 udpsrc port=55555 caps=\"application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96\" ! rtph264depay ! avdec_h264 ! decodebin ! videoconvert ! kmssink

I’m not sure what exactly the problem is. Maybe the timestamping mechanism in vaapi has a different behavior then in the avdec_h264 as the log message says? I’ve also changed log levels but I couldn’t find anything helpful.

Any help for further diagnostics is highly appreciated.

Kind regards

Michael

Bitte beachten / Please note!

*******************************************************************************

AGCO GmbH
Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf, Germany
Registergericht Amtsgericht Kempten HRB 10327
Geschäftsführer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin, Dr. Heribert Reiter, Ekkehart Glaeser
Vorsitzender des Aufsichtsrates: Torsten Dehner

*******************************************************************************

Diese E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet ist und kann vertrauliches
bzw. unter das Berufsgeheimnis fallendes Material enthalten. Jegliche darin enthaltene Ansicht
oder Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise die Ansicht oder
Meinung von AGCO dar. Sind Sie nicht der Empfänger, so haben Sie diese E-Mail irrtümlich
erhalten und jegliche Verwendung, Veröffentlichung, Weiterleitung, Abschrift oder jeglicher
Druck dieser E-Mail ist strengstens untersagt. Weder AGCO noch der Absender übernehmen die
Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angehängte
Dateien (sofern vorhanden) auf Viren zu überprüfen.

*******************************************************************************

This email is intended solely for the use of the individual to whom it is addressed and may contain
confidential and/or privileged material. Any views or opinions presented are solely those of the
author and do not necessarily represent those of AGCO. If you are not the intended recipient, be
advised that you have received this email in error and that any use, dissemination, forwarding,
printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any
responsibility for viruses and it is your responsibility to scan and virus check the email and its
attachment(s) (if any).

*******************************************************************************

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).
This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).

. .

, . . .

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Marc Leeman

Re: [External] Re: Streaming freezes (vaapi or v4l2src)

Also, do not send RTP data over odd ports.

You should use even ports (RFC 3550) and leave the odd port (port+1) for RTCP.

If you're using GStreamer git or the pre-release, you can simplify the pipeline to

gst-launch-1.0 rtpsrc uri=rtp://localhost:5554?encoding-name=H264 ! rtph264depay ! vaapih264dec low-latency=true ! vaapipostproc ! kmssink

it will also plug in SSRC demultiplexing (amongst others) and send RTCP to allow clock domain transfer from the sender to receiver.

On Tue, 28 Jul 2020 at 15:30, Timtchenko, Michael <[hidden email]> wrote:

Hi all,

sorry 4 my late response. I had a lot of work during the last two weeks.

I want to thank Nicolas and Victor for fast response and very good input!

I fixed the issue by using a rtpjitterbuffer in combination with the low-latency flag. Without the rtpjitterbuffer the image is getting teared. So I guess it’s necessary in this case.

The following pipeline works quite well in my case:

gst-launch-1.0 udpsrc port=55555 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtpjitterbuffer latency=10 ! rtph264depay ! vaapih264dec low-latency=true ! vaapipostproc ! kmssink

Timestamping the sink did not improve the misbehavior.

Best regards,

Michael

Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Nicolas Dufresne
Gesendet: Donnerstag, 16. Juli 2020 14:33
An: Discussion of the development of and with GStreamer <[hidden email]>
Betreff: [External] Re: Streaming freezes (vaapi or v4l2src)

Le jeu. 16 juill. 2020 04 h 00, Timtchenko, Michael <[hidden email]> a écrit :

Hello guys,

i’ve encountered a very strange behavior while streaming from a v4l2src to a kmssink (or glimagesink) by using vaapi-components for decoding. The problem is, that the streaming starts for 2 seconds, then freezes for 2 seconds and continues again for 2 seconds and so on. This effect is only noticeable when using vaapi-components. If I use a pipeline with simply software decoding, the stream is displayed fluently.

Two seconds could be the key frame distance ?

My sending pipeline is constructed as follows:

gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video$(($1+4)) ! video/x-raw,format=YV12,width=720,height=576 ! v4l2h264enc ! rtph264pay ! udpsink host=${IP} port=55555

v4l2h264enc has this extra-controls property that let you genetically change he specific configuration, just an information.

Setting pipeline to PAUSED ...

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

Redistribute latency...

The erroneous reading pipeline looks like this:

gst-launch-1.0 udpsrc port=55555 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! vaapih264dec ! vaapipostproc width=720 height=576 ! glimagesink

One thing to note is the absence of rtpjitterbuffer element, which would allow adding some latency to remove network jitter. So that is one thing to try.

Another possibility is that the issue is cause by vaaph264dec having more latency then avdec_h264. Perhaps give the low-latency property a try ?

Let's hope one of these suggestions helps.

Setting pipeline to PAUSED ...

Pipeline is live and does not need PREROLL ...

Got context from element 'vaapipostproc0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"$GstVaapiDisplayDRM$\ vaapidisplaydrm1";

Setting pipeline to PLAYING ...

New clock: GstSystemClock

Redistribute latency...

WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.

Additional debug info:

../libs/gst/base/gstbasesink.c(3005): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:

There may be a timestamping problem, or this computer is too slow.

WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.

Additional debug info:

../libs/gst/base/gstbasesink.c(3005): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:

There may be a timestamping problem, or this computer is too slow.

If I run the following pipeline on the same system, the stream is presented fluently without any problems:

gst-launch-1. 0 udpsrc port=55555 caps=\"application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96\" ! rtph264depay ! avdec_h264 ! decodebin ! videoconvert ! kmssink

I’m not sure what exactly the problem is. Maybe the timestamping mechanism in vaapi has a different behavior then in the avdec_h264 as the log message says? I’ve also changed log levels but I couldn’t find anything helpful.

Any help for further diagnostics is highly appreciated.

Kind regards

Michael

Bitte beachten / Please note!

*******************************************************************************

AGCO GmbH
Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf, Germany
Registergericht Amtsgericht Kempten HRB 10327
Geschäftsführer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin, Dr. Heribert Reiter, Ekkehart Glaeser
Vorsitzender des Aufsichtsrates: Torsten Dehner

*******************************************************************************

Diese E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet ist und kann vertrauliches
bzw. unter das Berufsgeheimnis fallendes Material enthalten. Jegliche darin enthaltene Ansicht
oder Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise die Ansicht oder
Meinung von AGCO dar. Sind Sie nicht der Empfänger, so haben Sie diese E-Mail irrtümlich
erhalten und jegliche Verwendung, Veröffentlichung, Weiterleitung, Abschrift oder jeglicher
Druck dieser E-Mail ist strengstens untersagt. Weder AGCO noch der Absender übernehmen die
Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angehängte
Dateien (sofern vorhanden) auf Viren zu überprüfen.

*******************************************************************************

This email is intended solely for the use of the individual to whom it is addressed and may contain
confidential and/or privileged material. Any views or opinions presented are solely those of the
author and do not necessarily represent those of AGCO. If you are not the intended recipient, be
advised that you have received this email in error and that any use, dissemination, forwarding,
printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any
responsibility for viruses and it is your responsibility to scan and virus check the email and its
attachment(s) (if any).

*******************************************************************************

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).
This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).

. .
, . . .

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

g. Marc

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Marc Leeman

Re: [External] Re: Streaming freezes (vaapi or v4l2src)

gst-launch-1.0 uridecodebin uri=rtp://localhost:5554?encoding-name=H264 ! vaapipostproc ! kmssink

On Tue, 28 Jul 2020 at 15:45, Marc Leeman <[hidden email]> wrote:

Also, do not send RTP data over odd ports.

You should use even ports (RFC 3550) and leave the odd port (port+1) for RTCP.

If you're using GStreamer git or the pre-release, you can simplify the pipeline to

gst-launch-1.0 rtpsrc uri=rtp://localhost:5554?encoding-name=H264 ! rtph264depay ! vaapih264dec low-latency=true ! vaapipostproc ! kmssink

it will also plug in SSRC demultiplexing (amongst others) and send RTCP to allow clock domain transfer from the sender to receiver.

On Tue, 28 Jul 2020 at 15:30, Timtchenko, Michael <[hidden email]> wrote:

Hi all,

sorry 4 my late response. I had a lot of work during the last two weeks.

I want to thank Nicolas and Victor for fast response and very good input!

I fixed the issue by using a rtpjitterbuffer in combination with the low-latency flag. Without the rtpjitterbuffer the image is getting teared. So I guess it’s necessary in this case.

The following pipeline works quite well in my case:

gst-launch-1.0 udpsrc port=55555 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtpjitterbuffer latency=10 ! rtph264depay ! vaapih264dec low-latency=true ! vaapipostproc ! kmssink

Timestamping the sink did not improve the misbehavior.

Best regards,

Michael

Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von Nicolas Dufresne
Gesendet: Donnerstag, 16. Juli 2020 14:33
An: Discussion of the development of and with GStreamer <[hidden email]>
Betreff: [External] Re: Streaming freezes (vaapi or v4l2src)

Le jeu. 16 juill. 2020 04 h 00, Timtchenko, Michael <[hidden email]> a écrit :

Hello guys,

i’ve encountered a very strange behavior while streaming from a v4l2src to a kmssink (or glimagesink) by using vaapi-components for decoding. The problem is, that the streaming starts for 2 seconds, then freezes for 2 seconds and continues again for 2 seconds and so on. This effect is only noticeable when using vaapi-components. If I use a pipeline with simply software decoding, the stream is displayed fluently.

Two seconds could be the key frame distance ?

My sending pipeline is constructed as follows:

gst-launch-1.0 v4l2src io-mode=dmabuf device=/dev/video$(($1+4)) ! video/x-raw,format=YV12,width=720,height=576 ! v4l2h264enc ! rtph264pay ! udpsink host=${IP} port=55555

v4l2h264enc has this extra-controls property that let you genetically change he specific configuration, just an information.

Setting pipeline to PAUSED ...

Pipeline is live and does not need PREROLL ...

Setting pipeline to PLAYING ...

New clock: GstSystemClock

Redistribute latency...

The erroneous reading pipeline looks like this:

gst-launch-1.0 udpsrc port=55555 caps="application/x-rtp,media=(string)video,clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! rtph264depay ! vaapih264dec ! vaapipostproc width=720 height=576 ! glimagesink

One thing to note is the absence of rtpjitterbuffer element, which would allow adding some latency to remove network jitter. So that is one thing to try.

Another possibility is that the issue is cause by vaaph264dec having more latency then avdec_h264. Perhaps give the low-latency property a try ?

Let's hope one of these suggestions helps.

Setting pipeline to PAUSED ...

Pipeline is live and does not need PREROLL ...

Got context from element 'vaapipostproc0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"$GstVaapiDisplayDRM$\ vaapidisplaydrm1";

Setting pipeline to PLAYING ...

New clock: GstSystemClock

Redistribute latency...

WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.

Additional debug info:

../libs/gst/base/gstbasesink.c(3005): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:

There may be a timestamping problem, or this computer is too slow.

WARNING: from element /GstPipeline:pipeline0/GstKMSSink:kmssink0: A lot of buffers are being dropped.

Additional debug info:

../libs/gst/base/gstbasesink.c(3005): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstKMSSink:kmssink0:

There may be a timestamping problem, or this computer is too slow.

If I run the following pipeline on the same system, the stream is presented fluently without any problems:

gst-launch-1. 0 udpsrc port=55555 caps=\"application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96\" ! rtph264depay ! avdec_h264 ! decodebin ! videoconvert ! kmssink

I’m not sure what exactly the problem is. Maybe the timestamping mechanism in vaapi has a different behavior then in the avdec_h264 as the log message says? I’ve also changed log levels but I couldn’t find anything helpful.

Any help for further diagnostics is highly appreciated.

Kind regards

Michael

Bitte beachten / Please note!

*******************************************************************************

AGCO GmbH
Sitz der AGCO GmbH: Johann-Georg-Fendt-Str.4, 87616 Marktoberdorf, Germany
Registergericht Amtsgericht Kempten HRB 10327
Geschäftsführer: Christoph Groeblinghoff, Ingrid Bussjaeger-Martin, Dr. Heribert Reiter, Ekkehart Glaeser
Vorsitzender des Aufsichtsrates: Torsten Dehner

*******************************************************************************

Diese E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet ist und kann vertrauliches
bzw. unter das Berufsgeheimnis fallendes Material enthalten. Jegliche darin enthaltene Ansicht
oder Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise die Ansicht oder
Meinung von AGCO dar. Sind Sie nicht der Empfänger, so haben Sie diese E-Mail irrtümlich
erhalten und jegliche Verwendung, Veröffentlichung, Weiterleitung, Abschrift oder jeglicher
Druck dieser E-Mail ist strengstens untersagt. Weder AGCO noch der Absender übernehmen die
Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren angehängte
Dateien (sofern vorhanden) auf Viren zu überprüfen.

*******************************************************************************

This email is intended solely for the use of the individual to whom it is addressed and may contain
confidential and/or privileged material. Any views or opinions presented are solely those of the
author and do not necessarily represent those of AGCO. If you are not the intended recipient, be
advised that you have received this email in error and that any use, dissemination, forwarding,
printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any
responsibility for viruses and it is your responsibility to scan and virus check the email and its
attachment(s) (if any).

*******************************************************************************

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).
This email is intended solely for the use of the individual to whom it is addressed and may contain confidential and/or privileged material. Any views or opinions presented are solely those of the author and do not necessarily represent those of AGCO. If you are not the intended recipient, be advised that you have received this email in error and that any use, dissemination, forwarding, printing or copying of this email is strictly prohibited. Neither AGCO nor the sender accepts any responsibility for viruses and it is your responsibility to scan and virus check the e-mail and its attachment(s) (if any).

. .
, . . .

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--
g. Marc

g. Marc

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel