GStreamer-devel

Gstreamer custom sink renderer takes too long time !

Classic

List

Threaded

5 messages Options

Duchassin Frederic

Gstreamer custom sink renderer takes too long time !

Hello All !

I try to make my own sink.

My main problem is that my renderer is quite slow (about 50000ns to get out render() function) and give poor performance with high input bitrate. (packet lost !)

For example a case which doesn’t work :

This is the interval between each time I feed my sink (without renderer):

dt=127654ns

dt=138320ns

dt=51995

dt=55661

dt=75326

dt=139319

dt=76660

dt=91991

dt=159318

dt=91991

dt=21664

dt=54995

dt=88324

dt=135986

dt=88324

dt=18665

dt=81326

dt=51995

dt=77659

dt=100990

dt=17665

dt=78659

As you can see, sometimes I feed my sink every 21 µs and as my renderer take 50µs, I can lose packet.

Can you advice me about a way to stop feeding my sink during renderer ???

Or if there is another way….

Thanks in for your help.

Frederic

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Nicolas Dufresne-5

Re: Gstreamer custom sink renderer takes too long time !

Le mercredi 05 septembre 2018 à 12:41 +0000, Duchassin Frédéric a
écrit :

> Hello All !
>
> I try to make my own sink.
> My main problem is that my renderer is quite slow (about 50000ns to
> get out render() function) and give poor performance with high input
> bitrate. (packet lost !)
>
> For example a case which doesn’t work :
> This is the interval between each time I feed my sink (without
> renderer):
> dt=127654ns
> dt=138320ns
> dt=51995
> dt=55661
> dt=75326
> dt=139319
> dt=76660
> dt=91991
> dt=159318
> dt=91991
> dt=21664
> dt=54995
> dt=88324
> dt=135986
> dt=88324
> dt=18665
> dt=81326
> dt=51995
> dt=77659
> dt=100990
> dt=17665
> dt=78659
>
> As you can see, sometimes I feed my sink every 21 µs and as my
> renderer take 50µs, I can lose packet.
>
> Can you advice me about a way to stop feeding my sink during renderer
> ???
> Or if there is another way….

What we usually do is to make the renderer asynchronous. This way, you
don't wait for the renderer to finish, and can receive the following
frame immediately. Then, whenever a frame is submitted while another is
being rendered, you can skip it (ideally sending a QoS message to tell
the app you have done so).

You should also check that max-lateness/deadline-timeout and other sink
properties are configured correctly. That QoS is enabled also, since
some element can decide a head of time to drop a buffer is it's safe to
do so and that this buffer isn't going to make it.

>
> Thanks in for your help.
>
> Frederic
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Duchassin Frederic

RE: Gstreamer custom sink renderer takes too long time !

Hello Nicolas,

My renderer is locked by mutex so it makes it synchronous. There is no way for me to make different.

And I can't lose any frame (or data).

But because I have a queue in the upstream, I probably could ask for buffering during my renderer isn'it ?

-----Message d'origine-----
De : gstreamer-devel [mailto:[hidden email]] De la part de Nicolas Dufresne
Envoyé : mercredi 5 septembre 2018 14:52
À : Discussion of the development of and with GStreamer
Objet : Re: Gstreamer custom sink renderer takes too long time !

Le mercredi 05 septembre 2018 à 12:41 +0000, Duchassin Frédéric a
écrit :

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Nicolas Dufresne-5

Re: Gstreamer custom sink renderer takes too long time !

Le mercredi 05 septembre 2018 à 15:07 +0200, Duchassin Frederic a
écrit :
> Hello Nicolas,
>
> My renderer is locked by mutex so it makes it synchronous. There is
> no way for me to make different.

Well, that's a pretty bad design, but you can workaround with a thread.

>
> And I can't lose any frame (or data).

Well, then you cannot do framerate adaptation. Your rendered must be as
fast as your video rate, otherwise it will break.

>
> But because I have a queue in the upstream, I probably could ask for
> buffering during my renderer isn'it ?

That will require an unlimited amount of RAM, also it will eliminate
the possibility to use a HW decoder (pretty much all the HW decoder we
have limits the number of output buffers).

regards,
Nicolas

>
>
> -----Message d'origine-----
> De : gstreamer-devel [mailto:
> [hidden email]] De la part de Nicolas
> Dufresne
> Envoyé : mercredi 5 septembre 2018 14:52
> À : Discussion of the development of and with GStreamer
> Objet : Re: Gstreamer custom sink renderer takes too long time !
>
> Le mercredi 05 septembre 2018 à 12:41 +0000, Duchassin Frédéric a
> écrit :
> > Hello All !
> >
> > I try to make my own sink.
> > My main problem is that my renderer is quite slow (about 50000ns to
> > get out render() function) and give poor performance with high
> > input
> > bitrate. (packet lost !)
> >
> > For example a case which doesn’t work :
> > This is the interval between each time I feed my sink (without
> > renderer):
> > dt=127654ns
> > dt=138320ns
> > dt=51995
> > dt=55661
> > dt=75326
> > dt=139319
> > dt=76660
> > dt=91991
> > dt=159318
> > dt=91991
> > dt=21664
> > dt=54995
> > dt=88324
> > dt=135986
> > dt=88324
> > dt=18665
> > dt=81326
> > dt=51995
> > dt=77659
> > dt=100990
> > dt=17665
> > dt=78659
> >
> > As you can see, sometimes I feed my sink every 21 µs and as my
> > renderer take 50µs, I can lose packet.
> >
> > Can you advice me about a way to stop feeding my sink during
> > renderer
> > ???
> > Or if there is another way….
>
> What we usually do is to make the renderer asynchronous. This way,
> you
> don't wait for the renderer to finish, and can receive the following
> frame immediately. Then, whenever a frame is submitted while another
> is
> being rendered, you can skip it (ideally sending a QoS message to
> tell
> the app you have done so).
>
> You should also check that max-lateness/deadline-timeout and other
> sink
> properties are configured correctly. That QoS is enabled also, since
> some element can decide a head of time to drop a buffer is it's safe
> to
> do so and that this buffer isn't going to make it.
>
> >
> > Thanks in for your help.
> >
> > Frederic
> >
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (201 bytes) Download Attachment

Duchassin Frederic

RE: Gstreamer custom sink renderer takes too long time !

> Well, that's a pretty bad design, but you can workaround with a thread.

Of course after your last answer, I try to implement a new thread with a queue (protected by mutex) and it nearly gives the same behavior.

> Well, then you cannot do framerate adaptation. Your rendered must be as fast as your video rate, otherwise it will break.

My renderer give me the possibility to output a stream at 90Mb/s whereas when I connect it to gstreamer, the bitrate has a limitation at about 40 Mb/s.
It's a very strange behavior because when I increase the bitrate of my souce, the sink bitrate can increase until 40Mb/s and after the bitrate decrease. (more and more packets are lost)

The pipeline used is :

gst-launch-1.0 udpsrc address=239.0.0.1 port=1234 buffer-size=30000000 ! queue ! eimsink sync=false async=true

> That will require an unlimited amount of RAM, also it will eliminate the possibility to use a HW decoder (pretty much all the HW decoder we have limits the number of output buffers).

No because my renderer bitrate could reach 90Mb/s.

-----Message d'origine-----
De : gstreamer-devel [mailto:[hidden email]] De la part de Nicolas Dufresne
Envoyé : mercredi 5 septembre 2018 16:33
À : Discussion of the development of and with GStreamer
Objet : Re: Gstreamer custom sink renderer takes too long time !

Le mercredi 05 septembre 2018 à 15:07 +0200, Duchassin Frederic a écrit :
> Hello Nicolas,
>
> My renderer is locked by mutex so it makes it synchronous. There is no
> way for me to make different.

Well, that's a pretty bad design, but you can workaround with a thread.

>
> And I can't lose any frame (or data).

Well, then you cannot do framerate adaptation. Your rendered must be as fast as your video rate, otherwise it will break.

>
> But because I have a queue in the upstream, I probably could ask for
> buffering during my renderer isn'it ?

That will require an unlimited amount of RAM, also it will eliminate the possibility to use a HW decoder (pretty much all the HW decoder we have limits the number of output buffers).

regards,
Nicolas

>
>
> -----Message d'origine-----
> De : gstreamer-devel [mailto:
> [hidden email]] De la part de Nicolas
> Dufresne Envoyé : mercredi 5 septembre 2018 14:52 À : Discussion of
> the development of and with GStreamer Objet : Re: Gstreamer custom
> sink renderer takes too long time !
>
> Le mercredi 05 septembre 2018 à 12:41 +0000, Duchassin Frédéric a
> écrit :
> > Hello All !
> >
> > I try to make my own sink.
> > My main problem is that my renderer is quite slow (about 50000ns to
> > get out render() function) and give poor performance with high input
> > bitrate. (packet lost !)
> >
> > For example a case which doesn’t work :
> > This is the interval between each time I feed my sink (without
> > renderer):
> > dt=127654ns
> > dt=138320ns
> > dt=51995
> > dt=55661
> > dt=75326
> > dt=139319
> > dt=76660
> > dt=91991
> > dt=159318
> > dt=91991
> > dt=21664
> > dt=54995
> > dt=88324
> > dt=135986
> > dt=88324
> > dt=18665
> > dt=81326
> > dt=51995
> > dt=77659
> > dt=100990
> > dt=17665
> > dt=78659
> >
> > As you can see, sometimes I feed my sink every 21 µs and as my
> > renderer take 50µs, I can lose packet.
> >
> > Can you advice me about a way to stop feeding my sink during
> > renderer ???
> > Or if there is another way….
>
> What we usually do is to make the renderer asynchronous. This way, you
> don't wait for the renderer to finish, and can receive the following
> frame immediately. Then, whenever a frame is submitted while another
> is being rendered, you can skip it (ideally sending a QoS message to
> tell the app you have done so).
>
> You should also check that max-lateness/deadline-timeout and other
> sink properties are configured correctly. That QoS is enabled also,
> since some element can decide a head of time to drop a buffer is it's
> safe to do so and that this buffer isn't going to make it.
>
> >
> > Thanks in for your help.
> >
> > Frederic
> >
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel