Conversion of cv::Mat to GstBuffer

[Saurabh Bora]

Hi Experts!!, 

I am trying to read opencv images locally and push them into gstreamer pipeline. I am using appsrc element with "need-data" signal attached to call-back function.

Please look at the code below. 

//------------------------------------------------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width* channels;

// Copy cv::Mat to GstBuffer

buffer = gst_buffer_new_allocate(NULL, size, NULL);

gst_buffer_map(buffer, &map, GST_MAP_WRITE);

memcpy((guchar *)map.data, data1, gst_buffer_get_size(buffer));

//Convert GstBuffer back to cv::Mat and write it, to check if data was correctly copied. //This is where I get to know that data is not correct.

cv::Mat img2(cv::Size(134, 134), CV_8UC3, (char*)(buffer));

cv::imwrite("C:/Users/212730892.LOGON/Desktop/dummy1.jpg", img2);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//-------------------------------------------------------------------------------------------------------------------------------------

In cb_need_data, the part where copying I am copying to GstBuffer seems to be wrong and it does not work as expected. 

Can anyone, please help me understand, how can I copy the image data perfectly and pass it downstream to other elements without losing any data?

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Michael Gruner]

Hi Saurabh

Yo may convert from a cv::Mat to a GstBuffer without a memory copy by using gst_buffer_new_wrapped:

https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstBuffer.html#gst-buffer-new-wrapped

Something like: 
buffer = gst_buffer_new_wrapped (mat.data, mat.total()*mat.elemSize());

In order to make a cv::Mat from a GstBuffer you first need to take the data out of the buffer. 

GstMapInfo info;

gst_buffer_map (buffer, &info, GST_MAP_READ); // or write

mat = cv::Mat(height, width, CV_8C3, map.data); // change your format accordingly

...

gst_buffer_unmap (buffer, &info);

Hope it helps.

Michael

www.ridgerun.com

On Mar 13, 2019, at 7:15 AM, Saurabh Bora <[hidden email]> wrote:

Hi Experts!!, 

I am trying to read opencv images locally and push them into gstreamer pipeline. I am using appsrc element with "need-data" signal attached to call-back function.

Please look at the code below. 

//------------------------------------------------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width* channels;

// Copy cv::Mat to GstBuffer

buffer = gst_buffer_new_allocate(NULL, size, NULL);

gst_buffer_map(buffer, &map, GST_MAP_WRITE);

memcpy((guchar *)map.data, data1, gst_buffer_get_size(buffer));

//Convert GstBuffer back to cv::Mat and write it, to check if data was correctly copied. //This is where I get to know that data is not correct.

cv::Mat img2(cv::Size(134, 134), CV_8UC3, (char*)(buffer));

cv::imwrite("C:/Users/212730892.LOGON/Desktop/dummy1.jpg", img2);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//-------------------------------------------------------------------------------------------------------------------------------------

In cb_need_data, the part where copying I am copying to GstBuffer seems to be wrong and it does not work as expected. 

Can anyone, please help me understand, how can I copy the image data perfectly and pass it downstream to other elements without losing any data?

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Saurabh Bora]

Hi Michael, 

Thanks! for your input. I tried what you suggested as below:

//---------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

#include <opencv2/imgproc/imgproc.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img2;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

cv::cvtColor(img, img2, cv::COLOR_BGR2RGB);

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width * channels * sizeof(guchar);

std::cout << "cb_need_data called" << std::endl;

buffer = gst_buffer_new_wrapped(img2.data, img2.total() * img2.elemSize());

GstMapInfo info;

gst_buffer_map(buffer, &info, GST_MAP_READ);

cv::Mat mat(height, width, CV_8UC3, info.data);

cv::imshow("Output", mat);

//gst_buffer_unmap(buffer, &info);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//----------------------------------------------------------------------------------------------

It still does not work as expected. It would be a great help if you could point out where I am going wrong.

Thanks!!

On Wed, Mar 13, 2019 at 7:40 PM Michael Gruner <[hidden email]> wrote:

Hi Saurabh

Yo may convert from a cv::Mat to a GstBuffer without a memory copy by using gst_buffer_new_wrapped:

https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstBuffer.html#gst-buffer-new-wrapped

Something like: 
buffer = gst_buffer_new_wrapped (mat.data, mat.total()*mat.elemSize());

In order to make a cv::Mat from a GstBuffer you first need to take the data out of the buffer. 

GstMapInfo info;

gst_buffer_map (buffer, &info, GST_MAP_READ); // or write

mat = cv::Mat(height, width, CV_8C3, map.data); // change your format accordingly

...

gst_buffer_unmap (buffer, &info);

Hope it helps.

Michael

www.ridgerun.com

On Mar 13, 2019, at 7:15 AM, Saurabh Bora <[hidden email]> wrote:

Hi Experts!!, 

I am trying to read opencv images locally and push them into gstreamer pipeline. I am using appsrc element with "need-data" signal attached to call-back function.

Please look at the code below. 

//------------------------------------------------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width* channels;

// Copy cv::Mat to GstBuffer

buffer = gst_buffer_new_allocate(NULL, size, NULL);

gst_buffer_map(buffer, &map, GST_MAP_WRITE);

memcpy((guchar *)map.data, data1, gst_buffer_get_size(buffer));

//Convert GstBuffer back to cv::Mat and write it, to check if data was correctly copied. //This is where I get to know that data is not correct.

cv::Mat img2(cv::Size(134, 134), CV_8UC3, (char*)(buffer));

cv::imwrite("C:/Users/212730892.LOGON/Desktop/dummy1.jpg", img2);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//-------------------------------------------------------------------------------------------------------------------------------------

In cb_need_data, the part where copying I am copying to GstBuffer seems to be wrong and it does not work as expected. 

Can anyone, please help me understand, how can I copy the image data perfectly and pass it downstream to other elements without losing any data?

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Michael Gruner]

It looks like img2, being a local variable, ends  its life when “cb_need_data” ends. That leaves the pointer in “buffer” invalid. For this one you’re better off with the copy. 

Another option is to alloc img2 on the heap, and use gst_buffer_new_wrapped_full to carry img2 along with buffer and free then together.

Michael

www.ridgerun.com

On Mar 13, 2019, at 10:51 AM, Saurabh Bora <[hidden email]> wrote:

Hi Michael, 

Thanks! for your input. I tried what you suggested as below:

//---------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

#include <opencv2/imgproc/imgproc.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img2;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

cv::cvtColor(img, img2, cv::COLOR_BGR2RGB);

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width * channels * sizeof(guchar);

std::cout << "cb_need_data called" << std::endl;

buffer = gst_buffer_new_wrapped(img2.data, img2.total() * img2.elemSize());

GstMapInfo info;

gst_buffer_map(buffer, &info, GST_MAP_READ);

cv::Mat mat(height, width, CV_8UC3, info.data);

cv::imshow("Output", mat);

//gst_buffer_unmap(buffer, &info);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//----------------------------------------------------------------------------------------------

It still does not work as expected. It would be a great help if you could point out where I am going wrong.

Thanks!!

On Wed, Mar 13, 2019 at 7:40 PM Michael Gruner <[hidden email]> wrote:

Hi Saurabh

Yo may convert from a cv::Mat to a GstBuffer without a memory copy by using gst_buffer_new_wrapped:

https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstBuffer.html#gst-buffer-new-wrapped

Something like: 
buffer = gst_buffer_new_wrapped (mat.data, mat.total()*mat.elemSize());

In order to make a cv::Mat from a GstBuffer you first need to take the data out of the buffer. 

GstMapInfo info;

gst_buffer_map (buffer, &info, GST_MAP_READ); // or write

mat = cv::Mat(height, width, CV_8C3, map.data); // change your format accordingly

...

gst_buffer_unmap (buffer, &info);

Hope it helps.

Michael

www.ridgerun.com

On Mar 13, 2019, at 7:15 AM, Saurabh Bora <[hidden email]> wrote:

Hi Experts!!, 

I am trying to read opencv images locally and push them into gstreamer pipeline. I am using appsrc element with "need-data" signal attached to call-back function.

Please look at the code below. 

//------------------------------------------------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width* channels;

// Copy cv::Mat to GstBuffer

buffer = gst_buffer_new_allocate(NULL, size, NULL);

gst_buffer_map(buffer, &map, GST_MAP_WRITE);

memcpy((guchar *)map.data, data1, gst_buffer_get_size(buffer));

//Convert GstBuffer back to cv::Mat and write it, to check if data was correctly copied. //This is where I get to know that data is not correct.

cv::Mat img2(cv::Size(134, 134), CV_8UC3, (char*)(buffer));

cv::imwrite("C:/Users/212730892.LOGON/Desktop/dummy1.jpg", img2);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//-------------------------------------------------------------------------------------------------------------------------------------

In cb_need_data, the part where copying I am copying to GstBuffer seems to be wrong and it does not work as expected. 

Can anyone, please help me understand, how can I copy the image data perfectly and pass it downstream to other elements without losing any data?

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[filnet]

Note that there a quite a few existing opencv plugins in gst-plugins-bad.

Most of them extend gstopencvvideofilter which provides the logic for converting a cv::Mat to a GstBugger (and the other way around).

See https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/blob/master/gst-libs/gst/opencv/gstopencvvideofilter.cpp.

Le Mercredi 13 mars 2019 19h18, Michael Gruner <[hidden email]> a écrit :

It looks like img2, being a local variable, ends  its life when “cb_need_data” ends. That leaves the pointer in “buffer” invalid. For this one you’re better off with the copy. 

Another option is to alloc img2 on the heap, and use gst_buffer_new_wrapped_full to carry img2 along with buffer and free then together.

Michael

www.ridgerun.com

On Mar 13, 2019, at 10:51 AM, Saurabh Bora <[hidden email]> wrote:

Hi Michael, 

Thanks! for your input. I tried what you suggested as below:

//---------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

#include <opencv2/imgproc/imgproc.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img2;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

cv::cvtColor(img, img2, cv::COLOR_BGR2RGB);

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width * channels * sizeof(guchar);

std::cout << "cb_need_data called" << std::endl;

buffer = gst_buffer_new_wrapped(img2.data, img2.total() * img2.elemSize());

GstMapInfo info;

gst_buffer_map(buffer, &info, GST_MAP_READ);

cv::Mat mat(height, width, CV_8UC3, info.data);

cv::imshow("Output", mat);

//gst_buffer_unmap(buffer, &info);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//----------------------------------------------------------------------------------------------

It still does not work as expected. It would be a great help if you could point out where I am going wrong.

Thanks!!

On Wed, Mar 13, 2019 at 7:40 PM Michael Gruner <[hidden email]> wrote:

Hi Saurabh

Yo may convert from a cv::Mat to a GstBuffer without a memory copy by using gst_buffer_new_wrapped:

https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstBuffer.html#gst-buffer-new-wrapped

Something like: 
buffer = gst_buffer_new_wrapped (mat.data, mat.total()*mat.elemSize());

In order to make a cv::Mat from a GstBuffer you first need to take the data out of the buffer. 

GstMapInfo info;

gst_buffer_map (buffer, &info, GST_MAP_READ); // or write

mat = cv::Mat(height, width, CV_8C3, map.data); // change your format accordingly

...

gst_buffer_unmap (buffer, &info);

Hope it helps.

Michael

www.ridgerun.com

On Mar 13, 2019, at 7:15 AM, Saurabh Bora <[hidden email]> wrote:

Hi Experts!!, 

I am trying to read opencv images locally and push them into gstreamer pipeline. I am using appsrc element with "need-data" signal attached to call-back function.

Please look at the code below. 

//------------------------------------------------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width* channels;

// Copy cv::Mat to GstBuffer

buffer = gst_buffer_new_allocate(NULL, size, NULL);

gst_buffer_map(buffer, &map, GST_MAP_WRITE);

memcpy((guchar *)map.data, data1, gst_buffer_get_size(buffer));

//Convert GstBuffer back to cv::Mat and write it, to check if data was correctly copied. //This is where I get to know that data is not correct.

cv::Mat img2(cv::Size(134, 134), CV_8UC3, (char*)(buffer));

cv::imwrite("C:/Users/212730892.LOGON/Desktop/dummy1.jpg", img2);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//-------------------------------------------------------------------------------------------------------------------------------------

In cb_need_data, the part where copying I am copying to GstBuffer seems to be wrong and it does not work as expected. 

Can anyone, please help me understand, how can I copy the image data perfectly and pass it downstream to other elements without losing any data?

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Saurabh Bora]

Hi Experts!, 

Thanks!! Michael, I tried the approach suggested by you and it works fine. I could verify the conversion of cv::Mat to GstBuffer and vice versa. On cv::write I get exact image that I had copied to GstBuffer.

However, I am unable to get correct image on autovidesink. With current code below, I see a skewed and distorted grayscale image. Note that I had to set width and height of videocaps on appsrc element as 133, 133 respectively, even though dimesnions of my input image is 134*134. If I set width and height as 134, 134 in videocaps, I see a black videosink.

//----------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

static GMainLoop *loop;

void buffer_destroy(gpointer data)

{

cv::Mat* done = (cv::Mat*)data;

delete done;

}

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

GstFlowReturn ret;

char *data1;

GstMapInfo map;

cv::Mat* img = new cv::Mat(cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg"));

std::cout << "cb_need_data called" << std::endl;

gsize sizeInBytes = img->total() * img->elemSize();

std::cout << "Size getting copied: " << sizeInBytes << std::endl;

buffer = gst_buffer_new_wrapped_full((GstMemoryFlags)0, (gpointer)(img->data), sizeInBytes, 0, sizeInBytes, (gpointer)img, (GDestroyNotify)buffer_destroy);

gsize bufferSize = gst_buffer_get_size(buffer);

std::cout << "Buffer Size: " << bufferSize << std::endl;

//Convert GstBuffer back to cv::Mat, to check if data was correctly copied

GstMapInfo info;

gst_buffer_map(buffer, &info, GST_MAP_READ);

cv::Mat img2(cv::Size(img->size().height, img->size().width), CV_8UC3, (char*)(info.data));

cv::imwrite("C:/Users/212730892.LOGON/Desktop/dummy1.jpg", img2);

GST_BUFFER_OFFSET(buffer) = 0;

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 30);

//GST_BUFFER_DURATION(buffer) = GST_CLOCK_TIME_NONE;

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 133,

"height", G_TYPE_INT, 133,

"framerate", GST_TYPE_FRACTION, 1, 30,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//-------------------------------------------------------------------------

I am new to gstreamer framework and it would be a great help if someone could point me in right direction.

Thanks and Regards,

Saurabh Bora

On Wed, Mar 13, 2019 at 11:48 PM Michael Gruner <[hidden email]> wrote:

It looks like img2, being a local variable, ends  its life when “cb_need_data” ends. That leaves the pointer in “buffer” invalid. For this one you’re better off with the copy. 

Another option is to alloc img2 on the heap, and use gst_buffer_new_wrapped_full to carry img2 along with buffer and free then together.

Michael

www.ridgerun.com

On Mar 13, 2019, at 10:51 AM, Saurabh Bora <[hidden email]> wrote:

Hi Michael, 

Thanks! for your input. I tried what you suggested as below:

//---------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

#include <opencv2/imgproc/imgproc.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img2;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

cv::cvtColor(img, img2, cv::COLOR_BGR2RGB);

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width * channels * sizeof(guchar);

std::cout << "cb_need_data called" << std::endl;

buffer = gst_buffer_new_wrapped(img2.data, img2.total() * img2.elemSize());

GstMapInfo info;

gst_buffer_map(buffer, &info, GST_MAP_READ);

cv::Mat mat(height, width, CV_8UC3, info.data);

cv::imshow("Output", mat);

//gst_buffer_unmap(buffer, &info);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//----------------------------------------------------------------------------------------------

It still does not work as expected. It would be a great help if you could point out where I am going wrong.

Thanks!!

On Wed, Mar 13, 2019 at 7:40 PM Michael Gruner <[hidden email]> wrote:

Hi Saurabh

Yo may convert from a cv::Mat to a GstBuffer without a memory copy by using gst_buffer_new_wrapped:

https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstBuffer.html#gst-buffer-new-wrapped

Something like: 
buffer = gst_buffer_new_wrapped (mat.data, mat.total()*mat.elemSize());

In order to make a cv::Mat from a GstBuffer you first need to take the data out of the buffer. 

GstMapInfo info;

gst_buffer_map (buffer, &info, GST_MAP_READ); // or write

mat = cv::Mat(height, width, CV_8C3, map.data); // change your format accordingly

...

gst_buffer_unmap (buffer, &info);

Hope it helps.

Michael

www.ridgerun.com

On Mar 13, 2019, at 7:15 AM, Saurabh Bora <[hidden email]> wrote:

Hi Experts!!, 

I am trying to read opencv images locally and push them into gstreamer pipeline. I am using appsrc element with "need-data" signal attached to call-back function.

Please look at the code below. 

//------------------------------------------------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width* channels;

// Copy cv::Mat to GstBuffer

buffer = gst_buffer_new_allocate(NULL, size, NULL);

gst_buffer_map(buffer, &map, GST_MAP_WRITE);

memcpy((guchar *)map.data, data1, gst_buffer_get_size(buffer));

//Convert GstBuffer back to cv::Mat and write it, to check if data was correctly copied. //This is where I get to know that data is not correct.

cv::Mat img2(cv::Size(134, 134), CV_8UC3, (char*)(buffer));

cv::imwrite("C:/Users/212730892.LOGON/Desktop/dummy1.jpg", img2);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//-------------------------------------------------------------------------------------------------------------------------------------

In cb_need_data, the part where copying I am copying to GstBuffer seems to be wrong and it does not work as expected. 

Can anyone, please help me understand, how can I copy the image data perfectly and pass it downstream to other elements without losing any data?

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Michael Gruner]

Can you run your app with warning logging enabled? Any of the following should work:

./app --gst-debug=2

Or

GST_DEBUG=2 ./app

On Mar 14, 2019, at 2:17 PM, Saurabh Bora <[hidden email]> wrote:

Hi Experts!, 

Thanks!! Michael, I tried the approach suggested by you and it works fine. I could verify the conversion of cv::Mat to GstBuffer and vice versa. On cv::write I get exact image that I had copied to GstBuffer.

However, I am unable to get correct image on autovidesink. With current code below, I see a skewed and distorted grayscale image. Note that I had to set width and height of videocaps on appsrc element as 133, 133 respectively, even though dimesnions of my input image is 134*134. If I set width and height as 134, 134 in videocaps, I see a black videosink.

//----------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

static GMainLoop *loop;

void buffer_destroy(gpointer data)

{

cv::Mat* done = (cv::Mat*)data;

delete done;

}

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

GstFlowReturn ret;

char *data1;

GstMapInfo map;

cv::Mat* img = new cv::Mat(cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg"));

std::cout << "cb_need_data called" << std::endl;

gsize sizeInBytes = img->total() * img->elemSize();

std::cout << "Size getting copied: " << sizeInBytes << std::endl;

buffer = gst_buffer_new_wrapped_full((GstMemoryFlags)0, (gpointer)(img->data), sizeInBytes, 0, sizeInBytes, (gpointer)img, (GDestroyNotify)buffer_destroy);

gsize bufferSize = gst_buffer_get_size(buffer);

std::cout << "Buffer Size: " << bufferSize << std::endl;

//Convert GstBuffer back to cv::Mat, to check if data was correctly copied

GstMapInfo info;

gst_buffer_map(buffer, &info, GST_MAP_READ);

cv::Mat img2(cv::Size(img->size().height, img->size().width), CV_8UC3, (char*)(info.data));

cv::imwrite("C:/Users/212730892.LOGON/Desktop/dummy1.jpg", img2);

GST_BUFFER_OFFSET(buffer) = 0;

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 30);

//GST_BUFFER_DURATION(buffer) = GST_CLOCK_TIME_NONE;

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 133,

"height", G_TYPE_INT, 133,

"framerate", GST_TYPE_FRACTION, 1, 30,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//-------------------------------------------------------------------------

I am new to gstreamer framework and it would be a great help if someone could point me in right direction.

Thanks and Regards,

Saurabh Bora

On Wed, Mar 13, 2019 at 11:48 PM Michael Gruner <[hidden email]> wrote:

It looks like img2, being a local variable, ends  its life when “cb_need_data” ends. That leaves the pointer in “buffer” invalid. For this one you’re better off with the copy. 

Another option is to alloc img2 on the heap, and use gst_buffer_new_wrapped_full to carry img2 along with buffer and free then together.

Michael

www.ridgerun.com

On Mar 13, 2019, at 10:51 AM, Saurabh Bora <[hidden email]> wrote:

Hi Michael, 

Thanks! for your input. I tried what you suggested as below:

//---------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

#include <opencv2/imgproc/imgproc.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img2;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

cv::cvtColor(img, img2, cv::COLOR_BGR2RGB);

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width * channels * sizeof(guchar);

std::cout << "cb_need_data called" << std::endl;

buffer = gst_buffer_new_wrapped(img2.data, img2.total() * img2.elemSize());

GstMapInfo info;

gst_buffer_map(buffer, &info, GST_MAP_READ);

cv::Mat mat(height, width, CV_8UC3, info.data);

cv::imshow("Output", mat);

//gst_buffer_unmap(buffer, &info);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//----------------------------------------------------------------------------------------------

It still does not work as expected. It would be a great help if you could point out where I am going wrong.

Thanks!!

On Wed, Mar 13, 2019 at 7:40 PM Michael Gruner <[hidden email]> wrote:

Hi Saurabh

Yo may convert from a cv::Mat to a GstBuffer without a memory copy by using gst_buffer_new_wrapped:

https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstBuffer.html#gst-buffer-new-wrapped

Something like: 
buffer = gst_buffer_new_wrapped (mat.data, mat.total()*mat.elemSize());

In order to make a cv::Mat from a GstBuffer you first need to take the data out of the buffer. 

GstMapInfo info;

gst_buffer_map (buffer, &info, GST_MAP_READ); // or write

mat = cv::Mat(height, width, CV_8C3, map.data); // change your format accordingly

...

gst_buffer_unmap (buffer, &info);

Hope it helps.

Michael

www.ridgerun.com

On Mar 13, 2019, at 7:15 AM, Saurabh Bora <[hidden email]> wrote:

Hi Experts!!, 

I am trying to read opencv images locally and push them into gstreamer pipeline. I am using appsrc element with "need-data" signal attached to call-back function.

Please look at the code below. 

//------------------------------------------------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width* channels;

// Copy cv::Mat to GstBuffer

buffer = gst_buffer_new_allocate(NULL, size, NULL);

gst_buffer_map(buffer, &map, GST_MAP_WRITE);

memcpy((guchar *)map.data, data1, gst_buffer_get_size(buffer));

//Convert GstBuffer back to cv::Mat and write it, to check if data was correctly copied. //This is where I get to know that data is not correct.

cv::Mat img2(cv::Size(134, 134), CV_8UC3, (char*)(buffer));

cv::imwrite("C:/Users/212730892.LOGON/Desktop/dummy1.jpg", img2);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//-------------------------------------------------------------------------------------------------------------------------------------

In cb_need_data, the part where copying I am copying to GstBuffer seems to be wrong and it does not work as expected. 

Can anyone, please help me understand, how can I copy the image data perfectly and pass it downstream to other elements without losing any data?

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Saurabh Bora]

Thanks Michael, 

It seems there is some problem with the input image itself. I tried different image of 640*480 resolution and code worked just fine.

Regards,

Saurabh Bora

On Fri, Mar 15, 2019 at 2:52 AM Michael Gruner <[hidden email]> wrote:

Can you run your app with warning logging enabled? Any of the following should work:

./app --gst-debug=2

Or

GST_DEBUG=2 ./app

On Mar 14, 2019, at 2:17 PM, Saurabh Bora <[hidden email]> wrote:

Hi Experts!, 

Thanks!! Michael, I tried the approach suggested by you and it works fine. I could verify the conversion of cv::Mat to GstBuffer and vice versa. On cv::write I get exact image that I had copied to GstBuffer.

However, I am unable to get correct image on autovidesink. With current code below, I see a skewed and distorted grayscale image. Note that I had to set width and height of videocaps on appsrc element as 133, 133 respectively, even though dimesnions of my input image is 134*134. If I set width and height as 134, 134 in videocaps, I see a black videosink.

//----------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

static GMainLoop *loop;

void buffer_destroy(gpointer data)

{

cv::Mat* done = (cv::Mat*)data;

delete done;

}

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

GstFlowReturn ret;

char *data1;

GstMapInfo map;

cv::Mat* img = new cv::Mat(cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg"));

std::cout << "cb_need_data called" << std::endl;

gsize sizeInBytes = img->total() * img->elemSize();

std::cout << "Size getting copied: " << sizeInBytes << std::endl;

buffer = gst_buffer_new_wrapped_full((GstMemoryFlags)0, (gpointer)(img->data), sizeInBytes, 0, sizeInBytes, (gpointer)img, (GDestroyNotify)buffer_destroy);

gsize bufferSize = gst_buffer_get_size(buffer);

std::cout << "Buffer Size: " << bufferSize << std::endl;

//Convert GstBuffer back to cv::Mat, to check if data was correctly copied

GstMapInfo info;

gst_buffer_map(buffer, &info, GST_MAP_READ);

cv::Mat img2(cv::Size(img->size().height, img->size().width), CV_8UC3, (char*)(info.data));

cv::imwrite("C:/Users/212730892.LOGON/Desktop/dummy1.jpg", img2);

GST_BUFFER_OFFSET(buffer) = 0;

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 30);

//GST_BUFFER_DURATION(buffer) = GST_CLOCK_TIME_NONE;

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 133,

"height", G_TYPE_INT, 133,

"framerate", GST_TYPE_FRACTION, 1, 30,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//-------------------------------------------------------------------------

I am new to gstreamer framework and it would be a great help if someone could point me in right direction.

Thanks and Regards,

Saurabh Bora

On Wed, Mar 13, 2019 at 11:48 PM Michael Gruner <[hidden email]> wrote:

It looks like img2, being a local variable, ends  its life when “cb_need_data” ends. That leaves the pointer in “buffer” invalid. For this one you’re better off with the copy. 

Another option is to alloc img2 on the heap, and use gst_buffer_new_wrapped_full to carry img2 along with buffer and free then together.

Michael

www.ridgerun.com

On Mar 13, 2019, at 10:51 AM, Saurabh Bora <[hidden email]> wrote:

Hi Michael, 

Thanks! for your input. I tried what you suggested as below:

//---------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

#include <opencv2/imgproc/imgproc.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img2;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

cv::cvtColor(img, img2, cv::COLOR_BGR2RGB);

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width * channels * sizeof(guchar);

std::cout << "cb_need_data called" << std::endl;

buffer = gst_buffer_new_wrapped(img2.data, img2.total() * img2.elemSize());

GstMapInfo info;

gst_buffer_map(buffer, &info, GST_MAP_READ);

cv::Mat mat(height, width, CV_8UC3, info.data);

cv::imshow("Output", mat);

//gst_buffer_unmap(buffer, &info);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//----------------------------------------------------------------------------------------------

It still does not work as expected. It would be a great help if you could point out where I am going wrong.

Thanks!!

On Wed, Mar 13, 2019 at 7:40 PM Michael Gruner <[hidden email]> wrote:

Hi Saurabh

Yo may convert from a cv::Mat to a GstBuffer without a memory copy by using gst_buffer_new_wrapped:

https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstBuffer.html#gst-buffer-new-wrapped

Something like: 
buffer = gst_buffer_new_wrapped (mat.data, mat.total()*mat.elemSize());

In order to make a cv::Mat from a GstBuffer you first need to take the data out of the buffer. 

GstMapInfo info;

gst_buffer_map (buffer, &info, GST_MAP_READ); // or write

mat = cv::Mat(height, width, CV_8C3, map.data); // change your format accordingly

...

gst_buffer_unmap (buffer, &info);

Hope it helps.

Michael

www.ridgerun.com

On Mar 13, 2019, at 7:15 AM, Saurabh Bora <[hidden email]> wrote:

Hi Experts!!, 

I am trying to read opencv images locally and push them into gstreamer pipeline. I am using appsrc element with "need-data" signal attached to call-back function.

Please look at the code below. 

//------------------------------------------------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width* channels;

// Copy cv::Mat to GstBuffer

buffer = gst_buffer_new_allocate(NULL, size, NULL);

gst_buffer_map(buffer, &map, GST_MAP_WRITE);

memcpy((guchar *)map.data, data1, gst_buffer_get_size(buffer));

//Convert GstBuffer back to cv::Mat and write it, to check if data was correctly copied. //This is where I get to know that data is not correct.

cv::Mat img2(cv::Size(134, 134), CV_8UC3, (char*)(buffer));

cv::imwrite("C:/Users/212730892.LOGON/Desktop/dummy1.jpg", img2);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//-------------------------------------------------------------------------------------------------------------------------------------

In cb_need_data, the part where copying I am copying to GstBuffer seems to be wrong and it does not work as expected. 

Can anyone, please help me understand, how can I copy the image data perfectly and pass it downstream to other elements without losing any data?

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Jan Schmidt-6]

Hi,

On 25/3/19 10:36 pm, Saurabh Bora wrote:

Thanks Michael, 

It seems there is some problem with the input image itself. I tried different image of 640*480 resolution and code worked just fine.

For frame sizes that aren't a multiple of 4 bytes, you need to round up and add padding - GStreamer buffers have an implicit stride that's the nearest multiple of 4. Alternatively, you can add a GstVideoMeta to the GstBuffer describing the actual stride.

Cheers,

Jan.

Regards,

Saurabh Bora

On Fri, Mar 15, 2019 at 2:52 AM Michael Gruner <[hidden email]> wrote:

Can you run your app with warning logging enabled? Any of the following should work:

./app --gst-debug=2

Or

GST_DEBUG=2 ./app

On Mar 14, 2019, at 2:17 PM, Saurabh Bora <[hidden email]> wrote:

Hi Experts!, 

Thanks!! Michael, I tried the approach suggested by you and it works fine. I could verify the conversion of cv::Mat to GstBuffer and vice versa. On cv::write I get exact image that I had copied to GstBuffer.

However, I am unable to get correct image on autovidesink. With current code below, I see a skewed and distorted grayscale image. Note that I had to set width and height of videocaps on appsrc element as 133, 133 respectively, even though dimesnions of my input image is 134*134. If I set width and height as 134, 134 in videocaps, I see a black videosink.

//----------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

static GMainLoop *loop;

void buffer_destroy(gpointer data)

{

cv::Mat* done = (cv::Mat*)data;

delete done;

}

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

GstFlowReturn ret;

char *data1;

GstMapInfo map;

cv::Mat* img = new cv::Mat(cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg"));

std::cout << "cb_need_data called" << std::endl;

gsize sizeInBytes = img->total() * img->elemSize();

std::cout << "Size getting copied: " << sizeInBytes << std::endl;

buffer = gst_buffer_new_wrapped_full((GstMemoryFlags)0, (gpointer)(img->data), sizeInBytes, 0, sizeInBytes, (gpointer)img, (GDestroyNotify)buffer_destroy);

gsize bufferSize = gst_buffer_get_size(buffer);

std::cout << "Buffer Size: " << bufferSize << std::endl;

//Convert GstBuffer back to cv::Mat, to check if data was correctly copied

GstMapInfo info;

gst_buffer_map(buffer, &info, GST_MAP_READ);

cv::Mat img2(cv::Size(img->size().height, img->size().width), CV_8UC3, (char*)(info.data));

cv::imwrite("C:/Users/212730892.LOGON/Desktop/dummy1.jpg", img2);

GST_BUFFER_OFFSET(buffer) = 0;

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 30);

//GST_BUFFER_DURATION(buffer) = GST_CLOCK_TIME_NONE;

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 133,

"height", G_TYPE_INT, 133,

"framerate", GST_TYPE_FRACTION, 1, 30,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//-------------------------------------------------------------------------

I am new to gstreamer framework and it would be a great help if someone could point me in right direction.

Thanks and Regards,

Saurabh Bora

On Wed, Mar 13, 2019 at 11:48 PM Michael Gruner <[hidden email]> wrote:

It looks like img2, being a local variable, ends  its life when “cb_need_data” ends. That leaves the pointer in “buffer” invalid. For this one you’re better off with the copy. 

Another option is to alloc img2 on the heap, and use gst_buffer_new_wrapped_full to carry img2 along with buffer and free then together.

Michael

www.ridgerun.com

On Mar 13, 2019, at 10:51 AM, Saurabh Bora <[hidden email]> wrote:

Hi Michael, 

Thanks! for your input. I tried what you suggested as below:

//---------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

#include <opencv2/imgproc/imgproc.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img2;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

cv::cvtColor(img, img2, cv::COLOR_BGR2RGB);

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width * channels * sizeof(guchar);

std::cout << "cb_need_data called" << std::endl;

buffer = gst_buffer_new_wrapped(img2.data, img2.total() * img2.elemSize());

GstMapInfo info;

gst_buffer_map(buffer, &info, GST_MAP_READ);

cv::Mat mat(height, width, CV_8UC3, info.data);

cv::imshow("Output", mat);

//gst_buffer_unmap(buffer, &info);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//----------------------------------------------------------------------------------------------

It still does not work as expected. It would be a great help if you could point out where I am going wrong.

Thanks!!

On Wed, Mar 13, 2019 at 7:40 PM Michael Gruner <[hidden email]> wrote:

Hi Saurabh

Yo may convert from a cv::Mat to a GstBuffer without a memory copy by using gst_buffer_new_wrapped:

https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/GstBuffer.html#gst-buffer-new-wrapped

Something like: 
buffer = gst_buffer_new_wrapped (mat.data, mat.total()*mat.elemSize());

In order to make a cv::Mat from a GstBuffer you first need to take the data out of the buffer. 

GstMapInfo info;

gst_buffer_map (buffer, &info, GST_MAP_READ); // or write

mat = cv::Mat(height, width, CV_8C3, map.data); // change your format accordingly

...

gst_buffer_unmap (buffer, &info);

Hope it helps.

Michael

www.ridgerun.com

On Mar 13, 2019, at 7:15 AM, Saurabh Bora <[hidden email]> wrote:

Hi Experts!!, 

I am trying to read opencv images locally and push them into gstreamer pipeline. I am using appsrc element with "need-data" signal attached to call-back function.

Please look at the code below. 

//------------------------------------------------------------------------------------------------------

#include <gst/gst.h>

#include <opencv2/opencv.hpp>

#include <opencv2/highgui.hpp>

static GMainLoop *loop;

static void

cb_need_data(GstElement *appsrc,

guint       unused_size,

gpointer    user_data)

{

static GstClockTime timestamp = 0;

GstBuffer *buffer;

guint size, depth, height, width, step, channels;

GstFlowReturn ret;

guchar *data1;

GstMapInfo map;

cv::Mat img = cv::imread("C:/Users/212730892.LOGON/Desktop/lena.jpg");

height = img.size().height;

width = img.size().width;

//step = img.widthStep;

channels = img.channels();

//depth = img->depth;

data1 = (guchar *)img.data;

size = height * width* channels;

// Copy cv::Mat to GstBuffer

buffer = gst_buffer_new_allocate(NULL, size, NULL);

gst_buffer_map(buffer, &map, GST_MAP_WRITE);

memcpy((guchar *)map.data, data1, gst_buffer_get_size(buffer));

//Convert GstBuffer back to cv::Mat and write it, to check if data was correctly copied. //This is where I get to know that data is not correct.

cv::Mat img2(cv::Size(134, 134), CV_8UC3, (char*)(buffer));

cv::imwrite("C:/Users/212730892.LOGON/Desktop/dummy1.jpg", img2);

GST_BUFFER_PTS(buffer) = timestamp;

GST_BUFFER_DURATION(buffer) = gst_util_uint64_scale_int(1, GST_SECOND, 2);

timestamp += GST_BUFFER_DURATION(buffer);

g_signal_emit_by_name(appsrc, "push-buffer", buffer, &ret);

//gst_buffer_unref(buffer);

if (ret != GST_FLOW_OK) {

/* something wrong, stop pushing */

g_main_loop_quit(loop);

}

}

gint

main(gint   argc,

gchar *argv[])

{

GstElement *pipeline, *appsrc, *conv, *videosink;

/* init GStreamer */

gst_init(&argc, &argv);

loop = g_main_loop_new(NULL, FALSE);

/* setup pipeline */

pipeline = gst_pipeline_new("pipeline");

appsrc = gst_element_factory_make("appsrc", "source");

conv = gst_element_factory_make("videoconvert", "conv");

videosink = gst_element_factory_make("autovideosink", "videosink");

/* setup */

g_object_set(G_OBJECT(appsrc), "caps",

gst_caps_new_simple("video/x-raw",

"format", G_TYPE_STRING, "RGB",

"width", G_TYPE_INT, 134,

"height", G_TYPE_INT, 134,

"framerate", GST_TYPE_FRACTION, 1, 1,

NULL), NULL);

gst_bin_add_many(GST_BIN(pipeline), appsrc, conv, videosink, NULL);

gst_element_link_many(appsrc, conv, videosink, NULL);

//g_object_set (videosink, "device", "/dev/video0", NULL);

/* setup appsrc */

g_object_set(G_OBJECT(appsrc),

"stream-type", 0,

"format", GST_FORMAT_TIME, NULL);

g_signal_connect(appsrc, "need-data", G_CALLBACK(cb_need_data), NULL);

/* play */

gst_element_set_state(pipeline, GST_STATE_PLAYING);

g_main_loop_run(loop);

if (gst_element_set_state(pipeline, GST_STATE_NULL) == GST_STATE_CHANGE_FAILURE)

{

g_printerr("Unable to set the pipeline to the playing state.\n");

}

// Free resources

gst_object_unref(GST_OBJECT(pipeline));

g_main_loop_unref(loop);

return 0;

}

//-------------------------------------------------------------------------------------------------------------------------------------

In cb_need_data, the part where copying I am copying to GstBuffer seems to be wrong and it does not work as expected. 

Can anyone, please help me understand, how can I copy the image data perfectly and pass it downstream to other elements without losing any data?

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[jphossein]

Hi,

In order for the buffer_destroy() function to be called, you should also
unamp buffer :

gst_buffer_unmap (buffer, &info);
 



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel