make video/x-bayer a first class gstreamer citizen

[Philippe De Muyter]

Hi all,

I am currently trying to extend an exisiting plugin which accepts
a big set of color formats on his sink to also accept Bayer-encoded
images produced by a sensor.  Everything is utterly complicated
because video/x-bayer is not treated equally to video/x-raw, while
it is basically the same, with constant-sized frames.

Problems are e.g,
gst_video_info_from_caps does not fill the size field of the info struct,
leading to a cryptic

(gst-launch-1.0:26159): GStreamer-CRITICAL **:
        gst_query_add_allocation_pool: assertion 'size != 0' failed

or
there is no GST_VIDEO_FORMAT_* constant for Bayer images, and
gst_video_format_to_string can thus not be used with Bayer formats

Could video/x-bayer,format=bggr be replaced by video/x-raw,format=bggr
 (or BGGR) ?

v4l2 does not do that distinction, and actually some sensors come with
the same electronic part, changing only the color filter between a GREY
sensor and a Bayer sensor

Best regards

Philippe

--
Philippe De Muyter [phone] Macq SA rue de l'Aeronef 2 B-1140 Bruxelles

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le vendredi 25 novembre 2016 à 02:17 +0100, Philippe De Muyter a
écrit :
>
> Could video/x-bayer,format=bggr be replaced by video/x-
> raw,format=bggr
>  (or BGGR) ?

It's hard to say if this can be changed now or not. Probably that the
old video/x-bayer would need to stay. Meanwhile, someone need to study
the libgstvideo library and find what may not fit, or what is needed to
make it fit. We managed to retrofit tile formats few years ago. With
enough effort, it could be possible to do the same for bayer. It's
obvious that color correction specific to your hardware won't be done
auto-magically though.

>
> v4l2 does not do that distinction, and actually some sensors come
> with
> the same electronic part, changing only the color filter between a
> GREY
> sensor and a Bayer sensor

We do have support video/x-bayer in GStreamer V4L2 plugins. It is
though not often tested. I need to check, but if vivid (the V4L2 test
driver) expose those format, then I'll be able to implement some gst-
validate test around it.

best regards,
Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Philippe De Muyter]

On Thu, Nov 24, 2016 at 08:39:18PM -0500, Nicolas Dufresne wrote:

> Le vendredi 25 novembre 2016 à 02:17 +0100, Philippe De Muyter a
> écrit :
> >
> > Could video/x-bayer,format=bggr be replaced by video/x-
> > raw,format=bggr
> >  (or BGGR) ?
>
> It's hard to say if this can be changed now or not. Probably that the
> old video/x-bayer would need to stay. Meanwhile, someone need to study
> the libgstvideo library and find what may not fit, or what is needed to
> make it fit. We managed to retrofit tile formats few years ago. With
> enough effort, it could be possible to do the same for bayer.

Well, if we had GST_VIDEO_FORMAT_ constants for video/x-bayer formats,

in gst_video_info_from_caps, we could set format to a meaningfull format
instead of GST_VIDEO_FORMAT_ENCODED,

and in fill_planes, we would not fall into

    case GST_VIDEO_FORMAT_ENCODED:
      break;

which silently keeps size set to 0

There is probably more, but that would be a good start

best regards

Philippe

--
Philippe De Muyter [phone] Macq SA rue de l'Aeronef 2 B-1140 Bruxelles
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-4]

Le vendredi 25 novembre 2016 à 03:16 +0100, Philippe De Muyter a
écrit :

> Well, if we had GST_VIDEO_FORMAT_ constants for video/x-bayer
> formats,
>
> in gst_video_info_from_caps, we could set format to a meaningfull
> format
> instead of GST_VIDEO_FORMAT_ENCODED,
>
> and in fill_planes, we would not fall into
>
>     case GST_VIDEO_FORMAT_ENCODED:
>       break;
>
> which silently keeps size set to 0
>
> There is probably more, but that would be a good start

Sure, but this is nothing we can merge upstream. You have to fill all
the fields and make sure they are fully usable. It's a lot more work.
Someone need to take ownership of this task.

Remember that using gst_video_info_parse_from_string() is not valid for
 anything non video/x-raw, which your code seems to do.

Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel