Convert CvMat to GstBuffer

[raomayurs]

Hi,
I am a newbie to gstreamer plugin development. I am trying to create a plugin using the tool provided by gstreamer. I am following the plugin writer's guide. I am trying to process data in the _chain function. I was able to successfully extract the video frame from GstBuffer and do my processing using openCV. But now I am unable to reconvert CvMat into GstBuffer.

Here is my code:

GBuffer* process_data(GstPad *sinkPad, GstMyFilter *filter, GstBuffer *buf) {
 //Get Height and width of input frame
  gint gheight, gwidth;
  static GstClockTime timestamp = 0;
  int height, width, channels = 3;
  GstCaps *caps = gst_pad_get_current_caps(sinkPad);
  GstStructure *str = gst_caps_get_structure (caps, 0);
  if (!gst_structure_get_int (str, "width", &gwidth)) {
    width = 1280;
  } else {
    width = (int) gwidth;
  }
  if (!gst_structure_get_int (str, "height", &gheight)) {
    height = 720;
  } else {
    height = (int) gheight;
  }

// convert buffer to opencv mat
  GstMapInfo map;
  gst_buffer_map (buf, &map, GST_MAP_READ);
  CvMat frame_yuv = cvMat(height + height/2, width, CV_8UC1, map.data);
  CvMat *frame = cvCreateMat(height, width, CV_8UC3);
  cvCvtColor(&frame_yuv, frame, CV_YUV2RGB_YV12);

// Process frame  
  CvMat *r = cvCreateMat(height, width, CV_8UC1);
  CvMat *g = cvCreateMat(height, width, CV_8UC1);
  CvMat *b = cvCreateMat(height, width, CV_8UC1);

  CvMat cameraMatrix = cvMat(3, 3, CV_64F, filter->cameraMatrix);                                                                                                                                                                                      
  CvMat distortionCoefficients = cvMat(5, 1, CV_64F, filter->distortionCoefficients);
  CvMat *newcamera_matrix = cvCreateMat(3, 3, CV_64F);
  CvMat *r1 = cvCreateMat(height, width, CV_8UC1);
  CvMat *g1 = cvCreateMat(height, width, CV_8UC1);
  CvMat *b1 = cvCreateMat(height, width, CV_8UC1);
  CvMat *undistorted = cvCreateMat(height, width, CV_8UC3);
  CvSize imgSize = cvSize(width, height);
  cvGetOptimalNewCameraMatrix(&cameraMatrix, &distortionCoefficients, imgSize, 1.0, newcamera_matrix, imgSize, NULL, 0);
  cvSplit(frame, r,g,b, NULL);
  cvUndistort2(r, r1, &cameraMatrix, &distortionCoefficients, newcamera_matrix);
  cvUndistort2(g, g1, &cameraMatrix, &distortionCoefficients, newcamera_matrix);
  cvUndistort2(b, b1, &cameraMatrix, &distortionCoefficients, newcamera_matrix);
  cvMerge(1r,g1,b1, NULL, undistorted);

//reconvert processed cvMat into GstBuffer
  GstMapInfo mapinfo;
  GstBuffer *buffer;
  gint size = height * width * channels;
  buffer = gst_buffer_new_allocate (NULL, size, NULL);
  gst_buffer_map (buffer, &mapinfo, GST_MAP_WRITE);
  memcpy( (guchar *)map.data, (guchar *)&(undistorted->data),  gst_buffer_get_size( buffer ) );
  GST_BUFFER_PTS (buffer) = timestamp;
  GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1, GST_SECOND, 2);

  timestamp += GST_BUFFER_DURATION (buffer);
  return buffer;
}

The returned buffer is passed to the src pad of the filter. The above code is not creating the buffer correctly. Am I converting cvMat to GstBuffer correctly? Because the openCV part is working correctly. The undistorted matrix is as expected. But the conversion to GstBuffer is failing for some reason. Any help will be really appreciated.

Thanks

[Florian Echtler]

I'd recommend using gst_buffer_new_wrapped_full instead, that saves a buffer
copy. Here's an example I'm using with appsrc which works fine (warning:
hardcoded buffer size):


void buffer_destroy(gpointer data) {
  cv::Mat* done = (cv::Mat*)data;
  delete done;
}

GstFlowReturn prepare_buffer(GstAppSrc* appsrc, cv::Mat* frame) {

  guint size = 1280 * 720 * 4;
  GstBuffer *buffer = gst_buffer_new_wrapped_full( (GstMemoryFlags)0,
(gpointer)(frame->data), size, 0, size, frame, buffer_destroy );

  return gst_app_src_push_buffer(appsrc, buffer);
}


Best regards, Florian


On 09.06.2017 00:34, raomayurs wrote:

> Hi,
> I am a newbie to gstreamer plugin development. I am trying to create a
> plugin using the tool provided by gstreamer. I am following the plugin
> writer's guide. I am trying to process data in the _chain function. I was
> able to successfully extract the video frame from GstBuffer and do my
> processing using openCV. But now I am unable to reconvert CvMat into
> GstBuffer.
>
> Here is my code:
>
> GBuffer* process_data(GstPad *sinkPad, GstMyFilter *filter, GstBuffer *buf)
> {
>  //Get Height and width of input frame
>   gint gheight, gwidth;
>   static GstClockTime timestamp = 0;
>   int height, width, channels = 3;
>   GstCaps *caps = gst_pad_get_current_caps(sinkPad);
>   GstStructure *str = gst_caps_get_structure (caps, 0);
>   if (!gst_structure_get_int (str, "width", &gwidth)) {
>     width = 1280;
>   } else {
>     width = (int) gwidth;
>   }
>   if (!gst_structure_get_int (str, "height", &gheight)) {
>     height = 720;
>   } else {
>     height = (int) gheight;
>   }
>
> // convert buffer to opencv mat
>   GstMapInfo map;
>   gst_buffer_map (buf, &map, GST_MAP_READ);
>   CvMat frame_yuv = cvMat(height + height/2, width, CV_8UC1, map.data);
>   CvMat *frame = cvCreateMat(height, width, CV_8UC3);
>   cvCvtColor(&frame_yuv, frame, CV_YUV2RGB_YV12);
>
> // Process frame  
>   CvMat *r = cvCreateMat(height, width, CV_8UC1);
>   CvMat *g = cvCreateMat(height, width, CV_8UC1);
>   CvMat *b = cvCreateMat(height, width, CV_8UC1);
>
>   CvMat cameraMatrix = cvMat(3, 3, CV_64F, filter->cameraMatrix);                                                                                                                                                                                      
>   CvMat distortionCoefficients = cvMat(5, 1, CV_64F,
> filter->distortionCoefficients);
>   CvMat *newcamera_matrix = cvCreateMat(3, 3, CV_64F);
>   CvMat *r1 = cvCreateMat(height, width, CV_8UC1);
>   CvMat *g1 = cvCreateMat(height, width, CV_8UC1);
>   CvMat *b1 = cvCreateMat(height, width, CV_8UC1);
>   CvMat *undistorted = cvCreateMat(height, width, CV_8UC3);
>   CvSize imgSize = cvSize(width, height);
>   cvGetOptimalNewCameraMatrix(&cameraMatrix, &distortionCoefficients,
> imgSize, 1.0, newcamera_matrix, imgSize, NULL, 0);
>   cvSplit(frame, r,g,b, NULL);
>   cvUndistort2(r, r1, &cameraMatrix, &distortionCoefficients,
> newcamera_matrix);
>   cvUndistort2(g, g1, &cameraMatrix, &distortionCoefficients,
> newcamera_matrix);
>   cvUndistort2(b, b1, &cameraMatrix, &distortionCoefficients,
> newcamera_matrix);
>   cvMerge(1r,g1,b1, NULL, undistorted);
>
> //reconvert processed cvMat into GstBuffer
>   GstMapInfo mapinfo;
>   GstBuffer *buffer;
>   gint size = height * width * channels;
>   buffer = gst_buffer_new_allocate (NULL, size, NULL);
>   gst_buffer_map (buffer, &mapinfo, GST_MAP_WRITE);
>   memcpy( (guchar *)map.data, (guchar *)&(undistorted->data),
> gst_buffer_get_size( buffer ) );
>   GST_BUFFER_PTS (buffer) = timestamp;
>   GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale_int (1, GST_SECOND,
> 2);
>
>   timestamp += GST_BUFFER_DURATION (buffer);
>   return buffer;
> }
>
> The returned buffer is passed to the src pad of the filter. The above code
> is not creating the buffer correctly. Am I converting cvMat to GstBuffer
> correctly? Because the openCV part is working correctly. The undistorted
> matrix is as expected. But the conversion to GstBuffer is failing for some
> reason. Any help will be really appreciated.
>
> Thanks
>
>
>
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble.com/Convert-CvMat-to-GstBuffer-tp4683259.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>

--
SENT FROM MY DEC VT50 TERMINAL


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Thu, 2017-06-08 at 15:34 -0700, raomayurs wrote:
> Hi,
> I am a newbie to gstreamer plugin development. I am trying to create a
> plugin using the tool provided by gstreamer. I am following the plugin
> writer's guide. I am trying to process data in the _chain function. I was
> able to successfully extract the video frame from GstBuffer and do my
> processing using openCV. But now I am unable to reconvert CvMat into
> GstBuffer.

Check the opencv plugin's code in gst-plugins-bad. It also uses CvMat
in a few places and should give some ideas how it can be done.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel