Orc and conversion from I420 to BGRA/RGBA

[Peter Maersk-Moller-2]

Hi.

Noticing that conversion from I420 to BGRA and RGBA still uses different amount of CPU with conversion to BGRA being the fastest (lowest CPU load).

Looking into gst-plugins-base-1.8.1/gst-libs/gst/video/video-converter.c I have identified the Orc enabled function video_orc_convert_I420_BGRA() used for converting to BGRA, but I can't seem to find the similar function for RGBA. What function is used for conversion to RGBA?

Is there a quality difference in color quality between converting I420 to BGRA respectively RGBA ? I seem to remember mentioned the "fast path conversion" for I420 to some variant of RGB/BGR. Is fast path conversion the video_orc_convert_I420_BGRA() ?

In case there is a color quality difference between fast path conversion and non fast path conversion (slow/normal path conversion?), is it the possibly to quantify this difference either specifically or loosly (like a little degradation, a lot, about 10% or these color ranges gets wrong etc) ?

Looking into the Orc code for video_orc_convert_I420_BGRA(), I see this can be modified to convert to RGBA with just copying the video_orc_convert_I420_BGRA() function and then make it save in another order for the color components. Wouldn't that be a low hanging fruit for GStreamer since the conversion, as it is now, can be done in almost half the time for BGRA compared to RGBA .... or is there a quality aspect I don't now about, making the current conversion to RGBA preferred?

best regards

Peter Maersk-Moller

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim Müller]

On Tue, 2016-06-21 at 16:17 +0200, Peter Maersk-Moller wrote:

Hi,

Noticing that conversion from I420 to BGRA and RGBA still uses different amount of CPU with conversion to BGRA being the fastest (lowest CPU load).

Looking into gst-plugins-base-1.8.1/gst-libs/gst/video/video-converter.c I have identified the Orc enabled function video_orc_convert_I420_BGRA() used for converting to BGRA, but I can't seem to find the similar function for RGBA. What function is used for conversion to RGBA?

Is there a quality difference in color quality between converting I420 to BGRA respectively RGBA ? I seem to remember mentioned the "fast path conversion" for I420 to some variant of RGB/BGR. Is fast path conversion the video_orc_convert_I420_BGRA() ?

In case there is a color quality difference between fast path conversion and non fast path conversion (slow/normal path conversion?), is it the possibly to quantify this difference either specifically or loosly (like a little degradation, a lot, about 10% or these color ranges gets wrong etc) ?

Looking into the Orc code for video_orc_convert_I420_BGRA(), I see this can be modified to convert to RGBA with just copying the video_orc_convert_I420_BGRA() function and then make it save in another order for the color components. Wouldn't that be a low hanging fruit for GStreamer since the conversion, as it is now, can be done in almost half the time for BGRA compared to RGBA .... or is there a quality aspect I don't now about, making the current conversion to RGBA preferred?

It's just low-hanging fruit. Note that some additional fastpaths have been added in git master already.

Cheers

 -Tim

-- 
Tim Müller, Centricular Ltd - http://www.centricular.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Peter Maersk-Moller-2]

Hi Tim.

On Tue, Jun 21, 2016 at 4:23 PM, Tim Müller <[hidden email]> wrote:

It's just low-hanging fruit. Note that some additional fastpaths have been added in git master already.

Sounds great. Thanks.

Slightly related topic. What is it in the Orc-code for video_orc_convert_I420_BGRA that makes the code sort of reuse the U and V values for every other/second RGB pixel it calculates?

What I mean is the RGB pixel(0) and RGB pixel(1) are calculated from the values of Y(0),U(0),V(0) and Y(1),U(0),V(0) respectively. Why is it the Orc code knows that the Y pointer is incremented for every RGB pixel it generates while the U and V pointers are only incremented for every second RGB pixel?

Best regards

Peter MM

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Peter Maersk-Moller-2]

Hi

On Tue, Jun 21, 2016 at 11:29 PM, Peter Maersk-Moller <[hidden email]> wrote:

Slightly related topic. What is it in the Orc-code for video_orc_convert_I420_BGRA that makes the code sort of reuse the U and V values for every other/second RGB pixel it calculates?

What I mean is the RGB pixel(0) and RGB pixel(1) are calculated from the values of Y(0),U(0),V(0) and Y(1),U(0),V(0) respectively. Why is it the Orc code knows that the Y pointer is incremented for every RGB pixel it generates while the U and V pointers are only incremented for every second RGB pixel?

Ahh, the command is 'loadupdb' which load from the array[i>>1], where 'i' is the iteration number (assumed). Unfortunately, according to loadupdb is not supported by neon. Does that mean that orc/neon accelerated I420 to RGB conversion is NOT available on ARM (with Neon)?

I'm looking for RGBA/BGRA to I420 (and Y42B - assuming that is YUV 4:2:2 planar) Orc enabled/accelerated conversion, but can not find it gst-plugins-base/gst-libs/gst/video. Does that mean GStreamer does not have such Orc enabled conversion or is it in another module?

Does there exist a table somewhere that can provide an overview of which conversions that are Orc enabled/accelerated (I assume that is what you call fast path conversion)? It would be nice with such a table in the documentation.

Best regards

Peter MM

Best regards

Peter MM

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Peter Maersk-Moller-2]

Hi

On Fri, Jun 24, 2016 at 9:38 AM, Peter Maersk-Moller <[hidden email]> wrote:

Ahh, the command is 'loadupdb' which load from the array[i>>1], where 'i' is the iteration number (assumed). Unfortunately, according to loadupdb is not supported by neon. Does that mean that orc/neon accelerated I420 to RGB conversion is NOT available on ARM (with Neon)?

If videoconvert module on NEON hardware (ARM) creates an array 'offset' the size of a single line in number of pixels with preloaded values 0,0,1,1,2,2,3,3... and the opcodes 'loadupdb r, u' and 'loadupdb r, v' in function video_orc_convert_I420_BGRA and similar functions are changed to 'loadoffb r u offset' and 'loadoffb r v offset' and offset is an argument to the function(s), then NEON accelerated conversion from I420 to RGB would be possible as a low hanging fruit. It would also work on x86 with only a minor or no performance penalty in case you only want one solution and not two.

Regards

Peter Maersk-Moller

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel