Unexpectedly expensive conversion from BGRA to BGR

[Tyler Compton]

Hi everyone,

I have a pipeline that, at some point, needs to convert video frames from BGRA to BGR. I was surprised to see about a ~17% increase in CPU usage just from doing this conversion. I wouldn't expect it to be that expensive, considering it's just a matter of stripping off some data for each pixel.

In my application, I have BGRA frames, but never any actual alpha information. Is there perhaps an optimization opportunity because of that?

On average I see about 17% CPU usage when running this pipeline:

videotestsrc is-live=true ! "video/x-raw,format=BGRA,width=1920,height=1080,framerate=30/1" ! fakesink

However, I see around 38% CPU usage just by adding a videoconvert to go from BGRA to BGR:

videotestsrc is-live=true ! "video/x-raw,format=BGRA,width=1920,height=1080,framerate=30/1" ! videoconvert ! "video/x-raw,format=BGR" ! fakesink

I'm running GStreamer 1.18.0 on Ubuntu 20.04.

Thanks,

Tyler

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le vendredi 16 octobre 2020 à 16:10 -0700, Tyler Compton a écrit :

> Hi everyone,
>
> I have a pipeline that, at some point, needs to convert video frames from BGRA to BGR. I was surprised to see about a ~17% increase in CPU usage just from doing this conversion. I wouldn't expect it to be that expensive, considering it's just a matter of stripping off some data for each pixel.
>
> In my application, I have BGRA frames, but never any actual alpha information. Is there perhaps an optimization opportunity because of that?
>
> On average I see about 17% CPU usage when running this pipeline:
>
> videotestsrc is-live=true ! "video/x-raw,format=BGRA,width=1920,height=1080,framerate=30/1" ! fakesink
>
> However, I see around 38% CPU usage just by adding a videoconvert to go from BGRA to BGR:
>
> videotestsrc is-live=true ! "video/x-raw,format=BGRA,width=1920,height=1080,framerate=30/1" ! videoconvert ! "video/x-raw,format=BGR" ! fakesink

This conversion will move 32bit aligned pixels to 24bits (unaligned)
pixels. Unaligned writes are 3 time slower. Unless you need the
bandwidth gain (even though yuv formats may give a better compression),
I would not recommending using 24bit packed formats in recent
applications. At least not when doing CPU access.

>
> I'm running GStreamer 1.18.0 on Ubuntu 20.04.
>
> Thanks,
> Tyler
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tyler Compton]

Interesting, thank you Nicolas. Unfortunately, the majority of machine learning models I'm using require BGR or RGB as their input format, so there isn't a lot I can do about it right now. I guess I'll have to look for optimizations elsewhere.

On Sat, Oct 17, 2020 at 7:38 AM Nicolas Dufresne <[hidden email]> wrote:

Le vendredi 16 octobre 2020 à 16:10 -0700, Tyler Compton a écrit :
> Hi everyone,
>
> I have a pipeline that, at some point, needs to convert video frames from BGRA to BGR. I was surprised to see about a ~17% increase in CPU usage just from doing this conversion. I wouldn't expect it to be that expensive, considering it's just a matter of stripping off some data for each pixel.
>
> In my application, I have BGRA frames, but never any actual alpha information. Is there perhaps an optimization opportunity because of that?
>
> On average I see about 17% CPU usage when running this pipeline:
>
> videotestsrc is-live=true ! "video/x-raw,format=BGRA,width=1920,height=1080,framerate=30/1" ! fakesink
>
> However, I see around 38% CPU usage just by adding a videoconvert to go from BGRA to BGR:
>
> videotestsrc is-live=true ! "video/x-raw,format=BGRA,width=1920,height=1080,framerate=30/1" ! videoconvert ! "video/x-raw,format=BGR" ! fakesink

This conversion will move 32bit aligned pixels to 24bits (unaligned)
pixels. Unaligned writes are 3 time slower. Unless you need the
bandwidth gain (even though yuv formats may give a better compression),
I would not recommending using 24bit packed formats in recent
applications. At least not when doing CPU access.

>
> I'm running GStreamer 1.18.0 on Ubuntu 20.04.
>
> Thanks,
> Tyler
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le lundi 19 octobre 2020 à 09:52 -0700, Tyler Compton a écrit :
> Interesting, thank you Nicolas. Unfortunately, the majority of
> machine learning models I'm using require BGR or RGB as their input
> format, so there isn't a lot I can do about it right now. I guess
> I'll have to look for optimizations elsewhere.

Note that videoconvert element have few "fast_path". Even thous it's
unaligned, you know that RGB24 will re-aligne every 4 pixels. Perhaps
there a ways to add some 4 pixels (128bits -> 96bit compression) SIMD
optimization (using orc or not).

Nicolas

>
> On Sat, Oct 17, 2020 at 7:38 AM Nicolas Dufresne <
> [hidden email]> wrote:
> > Le vendredi 16 octobre 2020 à 16:10 -0700, Tyler Compton a écrit :
> > > Hi everyone,
> > >
> > > I have a pipeline that, at some point, needs to convert video
> > frames from BGRA to BGR. I was surprised to see about a ~17%
> > increase in CPU usage just from doing this conversion. I wouldn't
> > expect it to be that expensive, considering it's just a matter of
> > stripping off some data for each pixel.
> > >
> > > In my application, I have BGRA frames, but never any actual alpha
> > information. Is there perhaps an optimization opportunity because
> > of that?
> > >
> > > On average I see about 17% CPU usage when running this pipeline:
> > >
> > > videotestsrc is-live=true ! "video/x-
> > raw,format=BGRA,width=1920,height=1080,framerate=30/1" ! fakesink
> > >
> > > However, I see around 38% CPU usage just by adding a videoconvert
> > to go from BGRA to BGR:
> > >
> > > videotestsrc is-live=true ! "video/x-
> > raw,format=BGRA,width=1920,height=1080,framerate=30/1" !
> > videoconvert ! "video/x-raw,format=BGR" ! fakesink
> >
> > This conversion will move 32bit aligned pixels to 24bits
> > (unaligned)
> > pixels. Unaligned writes are 3 time slower. Unless you need the
> > bandwidth gain (even though yuv formats may give a better
> > compression),
> > I would not recommending using 24bit packed formats in recent
> > applications. At least not when doing CPU access.
> >
> > >
> > > I'm running GStreamer 1.18.0 on Ubuntu 20.04.
> > >
> > > Thanks,
> > > Tyler
> > >
> > >
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > [hidden email]
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tony Houghton]

On Tue, 20 Oct 2020 at 14:47, Nicolas Dufresne <[hidden email]> wrote:

Note that videoconvert element have few "fast_path". Even thous it's
unaligned, you know that RGB24 will re-aligne every 4 pixels. Perhaps
there a ways to add some 4 pixels (128bits -> 96bit compression) SIMD
optimization (using orc or not).

Even without SIMD it should be possible to avoid unaligned writes by using shift operations on a CPU register as a mini output buffer. That would probably need a fast path or even a specialised element though.

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel