GStreamer-devel

Modifying GstVideoMeta format on a buffer in-place

Classic

List

Threaded

3 messages Options

Ryan Talbot

Modifying GstVideoMeta format on a buffer in-place

I'm modifying the OpenCV disparity plugin to get at the raw 16-bit signed disparity values, if the downstream caps request "video/x-raw,format=GRAY16_LE". After some headache, I managed to get the caps negotiation to work so that either RGB or GRAY16_LE could be obtained from the source pad. I was still running into trouble trying to use the buffers in pipelines, though, most notably when I tried to use videoconvert on them to prep them for encoding or display, I would get a message like this with every video frame:

** (gst-launch-1.0:9670): CRITICAL **: gst_video_frame_map_id: assertion 'info->finfo->format == meta->format' failed
WARNING: from element /GstPipeline:pipeline0/GstVideoConvert:videoconvert0: Internal GStreamer error: code not implemented.  Please file a bug at http://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer.
Additional debug info:
gstvideofilter.c(292): gst_video_filter_transform (): /GstPipeline:pipeline0/GstVideoConvert:videoconvert0:
invalid video buffer received

I didn't realize that just getting the caps set wasn't enough, and that there was essentially another copy of some of the caps data in meta data embedded in the buffer.

What is the correct way to modify this meta data?  As it is, I'm basically doing:

GstVideoMeta *meta = gst_buffer_get_video_meta (buffer);
meta->format = GST_VIDEO_FORMAT_GRAY16_LE;
ret = gst_pad_push (srcpad, buffer);

...that works, pipelines are happy, but it seems like a hack.  Is there a more preferred way of doing this?  It seems like maybe this plugin should derive from GST_TYPE_VIDEO_FILTER instead of GST_TYPE_ELEMENT, so that GstBaseTransform::transform_meta could be hooked into, although I'm not sure I have time to rewrite the plugin to that extent at the moment.

Thanks for any insight!

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Nicolas Dufresne-5

Re: Modifying GstVideoMeta format on a buffer in-place

Le vendredi 17 février 2017 à 10:37 +0000, Ryan Talbot a écrit :

> I'm modifying the OpenCV disparity plugin to get at the raw 16-bit
> signed disparity values, if the downstream caps request "video/x-
> raw,format=GRAY16_LE". After some headache, I managed to get the
> caps negotiation to work so that either RGB or GRAY16_LE could be
> obtained from the source pad. I was still running into trouble
> trying to use the buffers in pipelines, though, most notably when I
> tried to use videoconvert on them to prep them for encoding or
> display, I would get a message like this with every video frame:
>
> ** (gst-launch-1.0:9670): CRITICAL **: gst_video_frame_map_id:
> assertion 'info->finfo->format == meta->format' failed
> WARNING: from element
> /GstPipeline:pipeline0/GstVideoConvert:videoconvert0: Internal
> GStreamer error: code not implemented.  Please file a bug at http://b
> ugzilla.gnome.org/enter_bug.cgi?product=GStreamer.
> Additional debug info:
> gstvideofilter.c(292): gst_video_filter_transform ():
> /GstPipeline:pipeline0/GstVideoConvert:videoconvert0:
> invalid video buffer received
>
> I didn't realize that just getting the caps set wasn't enough, and
> that there was essentially another copy of some of the caps data in
> meta data embedded in the buffer.
>
> What is the correct way to modify this meta data?  As it is, I'm
> basically doing:
>
> GstVideoMeta *meta = gst_buffer_get_video_meta (buffer);
> meta->format = GST_VIDEO_FORMAT_GRAY16_LE;
> ret = gst_pad_push (srcpad, buffer);
>
> ...that works, pipelines are happy, but it seems like a hack.  Is
> there a more preferred way of doing this?  It seems like maybe this
> plugin should derive from GST_TYPE_VIDEO_FILTER instead of
> GST_TYPE_ELEMENT, so that GstBaseTransform::transform_meta could be
> hooked into, although I'm not sure I have time to rewrite the plugin
> to that extent at the moment.

This element should be ported to the BaseOpenCVFilter class (like all
the others) and should use the BaseTransform mechanism indeed. Doing
in-place color conversion isn't common though. We usually concern into
a new buffer.

Nicolas

>
> Thanks for any insight!
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (188 bytes) Download Attachment

Ryan Talbot

RE: Modifying GstVideoMeta format on a buffer in-place

Hi Nicolas,

Thanks for the response.

>This element should be ported to the BaseOpenCVFilter class (like all
>the others) and should use the BaseTransform mechanism indeed. Doing
>in-place color conversion isn't common though. We usually concern into
>a new buffer.

OK, that's good to know. I did not take a close look at the other filters. I was doing the transformation in place since the plugin already does an RGB->GRAY->RGB colorspace conversion in place, but it seems this is not the best strategy. I will see about trying to port it to the BaseOpenCVFilter class.

________________________________________
From: gstreamer-devel [[hidden email]] on behalf of Nicolas Dufresne [[hidden email]]
Sent: Sunday, February 19, 2017 2:58 PM
To: Discussion of the development of and with GStreamer
Subject: Re: Modifying GstVideoMeta format on a buffer in-place

Le vendredi 17 février 2017 à 10:37 +0000, Ryan Talbot a écrit :

> I'm modifying the OpenCV disparity plugin to get at the raw 16-bit
> signed disparity values, if the downstream caps request "video/x-
> raw,format=GRAY16_LE". After some headache, I managed to get the
> caps negotiation to work so that either RGB or GRAY16_LE could be
> obtained from the source pad. I was still running into trouble
> trying to use the buffers in pipelines, though, most notably when I
> tried to use videoconvert on them to prep them for encoding or
> display, I would get a message like this with every video frame:
>
> ** (gst-launch-1.0:9670): CRITICAL **: gst_video_frame_map_id:
> assertion 'info->finfo->format == meta->format' failed
> WARNING: from element
> /GstPipeline:pipeline0/GstVideoConvert:videoconvert0: Internal
> GStreamer error: code not implemented. Please file a bug at http://b
> ugzilla.gnome.org/enter_bug.cgi?product=GStreamer.
> Additional debug info:
> gstvideofilter.c(292): gst_video_filter_transform ():
> /GstPipeline:pipeline0/GstVideoConvert:videoconvert0:
> invalid video buffer received
>
> I didn't realize that just getting the caps set wasn't enough, and
> that there was essentially another copy of some of the caps data in
> meta data embedded in the buffer.
>
> What is the correct way to modify this meta data? As it is, I'm
> basically doing:
>
> GstVideoMeta *meta = gst_buffer_get_video_meta (buffer);
> meta->format = GST_VIDEO_FORMAT_GRAY16_LE;
> ret = gst_pad_push (srcpad, buffer);
>
> ...that works, pipelines are happy, but it seems like a hack. Is
> there a more preferred way of doing this? It seems like maybe this
> plugin should derive from GST_TYPE_VIDEO_FILTER instead of
> GST_TYPE_ELEMENT, so that GstBaseTransform::transform_meta could be
> hooked into, although I'm not sure I have time to rewrite the plugin
> to that extent at the moment.