GStreamer-devel

Speeding up a queue

classic Classic list List threaded Threaded
♦
♦
 5 messages Options
simo-zz
Reply | Threaded
Open this post in threaded view
♦
♦
|

Speeding up a queue

simo-zz
Hello,
I need to record a video from the USB camera and process the raw-data at the same time, so I successfully implemented this pipeline in C:

v4l2src device=/dev/webcam ! videoconvert ! video/x-raw,width=544,height=288,framerate=10/1 ! tee name=t ! queue ! v4l2h264enc ! h264parse ! mp4mux ! filesink location=video.mp4 t. ! queue ! appsink

I can access to raw-data from USB camera is read using the "new-sample" signal of appsink in a function called "dataProbe", code as follows:

GstFlowReturn dataProbe(GstElement *source)
{ 
    // buffCntr is just a counter to choose when I want to process the raw-data
    if (!(buffCntr % 10))
    {  
        GstMapInfo map;
        GstSample *sample = gst_app_sink_pull_sample(GST_APP_SINK(source));
        GstBuffer *buffer = gst_sample_get_buffer(sample);
        gst_buffer_ref(buffer);
        gst_buffer_map (buffer, &map, GST_MAP_READ);
        g_print("Map size: %d\n", map.size);

        /*int8_t fd = open("/path/to/someFile", O_CREAT | O_WRONLY);
        if (fd != -1)
        {
            if(write(fd, map.data, map.size) == map.size)
                g_print("Written whole data\n");
            else
                g_print("Cannot write whole data\n");
        }
        close(fd);*/
        gst_buffer_unmap(buffer, &map);
        gst_sample_unref(sample);
        gst_buffer_unref(buffer);

        //buffCntr = 0;
    }
}

But the execution of the function dataProbe is so critical that if I set buffCntr to zero (as shown in the last commented instruction of the function) even without writing map.data to some file, the whole program hangs, so the mp4 video is not written anymore..
But I need both raw-data and mp4 video.

What can I do to 'speed up' this queue and let my gstreamer program to record the video and write the raw data somewhere ? Pthread ? There is some helpful feature already available from gstreamer library to handle this situation ?

The device which run the application is an embedded board with a quad core CPU.

Thank you in advance.
Regards,
Simon

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Martin Vachovski
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Speeding up a queue

Martin Vachovski

Hi Simon,


Three suggestions in the order of relevance:


1) I think there is no need to explicitly call 

gst_buffer_ref(buffer)-- the gst_sample_get_buffer should do it internally


2) Are you sure that the performance is the issue here?

Try changing the condition whether you access the data

and see when it stops working, like:

if(!(buffCntr % 20)) // every 20th

then 

if(!(buffCntr % 15)) // every 15th

and see if for sufficiently small denominator it stops working


3) Also, try calling the "unref" methods in the opposite order as the "get/ref" methods: 

You have 

ref(sample)

ref(buffer)

But you release the sample first

unref(sample)

unref(buffer)​

I think it should be:

unref(buffer)

unref(sample)


Hope that helps

Martin


From: gstreamer-devel <[hidden email]> on behalf of simo zz <[hidden email]>
Sent: Thursday, September 7, 2017 5:00 PM
To: Discussion of the Development of and With GStreamer
Subject: Speeding up a queue
 
Hello,
I need to record a video from the USB camera and process the raw-data at the same time, so I successfully implemented this pipeline in C:

v4l2src device=/dev/webcam ! videoconvert ! video/x-raw,width=544,height=288,framerate=10/1 ! tee name=t ! queue ! v4l2h264enc ! h264parse ! mp4mux ! filesink location=video.mp4 t. ! queue ! appsink

I can access to raw-data from USB camera is read using the "new-sample" signal of appsink in a function called "dataProbe", code as follows:

GstFlowReturn dataProbe(GstElement *source)
{ 
    // buffCntr is just a counter to choose when I want to process the raw-data
    if (!(buffCntr % 10))
    {  
        GstMapInfo map;
        GstSample *sample = gst_app_sink_pull_sample(GST_APP_SINK(source));
        GstBuffer *buffer = gst_sample_get_buffer(sample);
        gst_buffer_ref(buffer);
        gst_buffer_map (buffer, &map, GST_MAP_READ);
        g_print("Map size: %d\n", map.size);

        /*int8_t fd = open("/path/to/someFile", O_CREAT | O_WRONLY);
        if (fd != -1)
        {
            if(write(fd, map.data, map.size) == map.size)
                g_print("Written whole data\n");
            else
                g_print("Cannot write whole data\n");
        }
        close(fd);*/
        gst_buffer_unmap(buffer, &map);
        gst_sample_unref(sample);
        gst_buffer_unref(buffer);

        //buffCntr = 0;
    }
}

But the execution of the function dataProbe is so critical that if I set buffCntr to zero (as shown in the last commented instruction of the function) even without writing map.data to some file, the whole program hangs, so the mp4 video is not written anymore..
But I need both raw-data and mp4 video.

What can I do to 'speed up' this queue and let my gstreamer program to record the video and write the raw data somewhere ? Pthread ? There is some helpful feature already available from gstreamer library to handle this situation ?

The device which run the application is an embedded board with a quad core CPU.

Thank you in advance.
Regards,
Simon

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Nicolas Dufresne-5
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Speeding up a queue

Nicolas Dufresne-5
Le vendredi 08 septembre 2017 à 09:11 +0000, Martin Vachovski a écrit :
> Hi Simon,
>
> Three suggestions in the order of relevance:
>
> 1) I think there is no need to explicitly call
> gst_buffer_ref(buffer)-- the gst_sample_get_buffer should do it
> internally

No it "transfer none".

>
> 2) Are you sure that the performance is the issue here?
> Try changing the condition whether you access the data
> and see when it stops working, like:
> if(!(buffCntr % 20)) // every 20th
> then
> if(!(buffCntr % 15)) // every 15th
> and see if for sufficiently small denominator it stops working
>
> 3) Also, try calling the "unref" methods in the opposite order as the
> "get/ref" methods:
> You have
> ref(sample)
> ref(buffer)
> But you release the sample first
> unref(sample)
> unref(buffer)
> I think it should be:
> unref(buffer)
> unref(sample)
This is usually not required with ref-counting in general. Since if
there dependent object will have a ref on the other, which guaranty the
free order for you.

>
> Hope that helps
> Martin
>
> From: gstreamer-devel <[hidden email]>
> on behalf of simo zz <[hidden email]>
> Sent: Thursday, September 7, 2017 5:00 PM
> To: Discussion of the Development of and With GStreamer
> Subject: Speeding up a queue
>  
> Hello,
> I need to record a video from the USB camera and process the raw-data
> at the same time, so I successfully implemented this pipeline in C:
>
> v4l2src device=/dev/webcam ! videoconvert ! video/x-
> raw,width=544,height=288,framerate=10/1 ! tee name=t ! queue
> ! v4l2h264enc ! h264parse ! mp4mux ! filesink location=video.mp4 t. !
> queue ! appsink
>
> I can access to raw-data from USB camera is read using the "new-
> sample" signal of appsink in a function called "dataProbe", code as
> follows:
>
> GstFlowReturn dataProbe(GstElement *source)
> {  
>     // buffCntr is just a counter to choose when I want to process
> the raw-data
>     if (!(buffCntr % 10))
>     {  
>         GstMapInfo map;
>         GstSample *sample =
> gst_app_sink_pull_sample(GST_APP_SINK(source));
>         GstBuffer *buffer = gst_sample_get_buffer(sample);
>         gst_buffer_ref(buffer);
>         gst_buffer_map (buffer, &map, GST_MAP_READ);
>         g_print("Map size: %d\n", map.size);
>
>         /*int8_t fd = open("/path/to/someFile", O_CREAT | O_WRONLY);
>         if (fd != -1)
>         {
>             if(write(fd, map.data, map.size) == map.size)
>                 g_print("Written whole data\n");
>             else
>                 g_print("Cannot write whole data\n");
>         }
>         close(fd);*/
>         gst_buffer_unmap(buffer, &map);
>         gst_sample_unref(sample);
>         gst_buffer_unref(buffer);
>
>         //buffCntr = 0;
>     }
> }
>
> But the execution of the function dataProbe is so critical that if I
> set buffCntr to zero (as shown in the last commented instruction of
> the function) even without writing map.data to some file, the whole
> program hangs, so the mp4 video is not written anymore..
> But I need both raw-data and mp4 video.
>
> What can I do to 'speed up' this queue and let my gstreamer program
> to record the video and write the raw data somewhere ? Pthread ?
> There is some helpful feature already available from gstreamer
> library to handle this situation ?
>
> The device which run the application is an embedded board with a quad
> core CPU.
>
> Thank you in advance.
> Regards,
> Simon
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (201 bytes) Download Attachment
simo-zz
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Speeding up a queue

simo-zz
In reply to this post by simo-zz
Hello Martín,
Thank you for tour hint. I will keep in mind that.

At the moment, since I want to write the content of the map.data to a pipe, I tested the fdsink and the filesink instead of the appsink since they write data without the need to do it manually.

The fdsink give me problems.. It actually overload the program and the video is not written any more after the first chunk of data.
filesink works far better.
The only problem I see with using filesink with a pipe, is that if the reader closes the stream, gstreamer stops/crashes.

However I don't discard to use the appsink again after your hints. I will test it and report the results.

Regards,
Simon


--------------------------------------
El vie, 8/9/17, Nicolas Dufresne <[hidden email]> escribió:

 Asunto: Re: Speeding up a queue
 Para: "Discussion of the development of and with GStreamer" <[hidden email]>, "simo zz" <[hidden email]>
 Fecha: viernes, 8 de septiembre, 2017 19:21
 
 Le vendredi 08 septembre 2017 à
 09:11 +0000, Martin Vachovski a écrit :
 > Hi Simon,
 >
 > Three suggestions in the order of
 relevance:
 >
 > 1) I
 think there is no need to explicitly call
 > gst_buffer_ref(buffer)-- the
 gst_sample_get_buffer should do it
 >
 internally
 
 No it
 "transfer none".
 
 >
 > 2) Are you sure that
 the performance is the issue here?
 > Try
 changing the condition whether you access the data
 > and see when it stops working, like:
 > if(!(buffCntr % 20)) // every 20th
 > then
 > if(!(buffCntr %
 15)) // every 15th
 > and see if for
 sufficiently small denominator it stops working
 >
 > 3) Also, try calling
 the "unref" methods in the opposite order as
 the
 > "get/ref" methods:
 > You have
 >
 ref(sample)
 > ref(buffer)
 > But you release the sample first
 > unref(sample)
 >
 unref(buffer)
 > I think it should be:
 > unref(buffer)
 >
 unref(sample)
 
 This is
 usually not required with ref-counting in general. Since
 if
 there dependent object will have a ref on
 the other, which guaranty the
 free order for
 you.
 
 >
 >
 Hope that helps
 > Martin
 >
 > From:
 gstreamer-devel <[hidden email]>
 > on behalf of simo zz <[hidden email]>
 > Sent: Thursday, September 7, 2017 5:00
 PM
 > To: Discussion of the Development of
 and With GStreamer
 > Subject: Speeding up
 a queue
 > 
 > Hello,
 
 > I need to record a video from the USB
 camera and process the raw-data
 > at the
 same time, so I successfully implemented this pipeline in
 C:
 >
 > v4l2src
 device=/dev/webcam ! videoconvert ! video/x-
 > raw,width=544,height=288,framerate=10/1 !
 tee name=t ! queue
 > ! v4l2h264enc !
 h264parse ! mp4mux ! filesink location=video.mp4 t. !
 > queue ! appsink
 >
 > I can access to raw-data from USB camera
 is read using the "new-
 >
 sample" signal of appsink in a function called
 "dataProbe", code as
 >
 follows:
 >
 >
 GstFlowReturn dataProbe(GstElement *source)
 > { 
 >     //
 buffCntr is just a counter to choose when I want to
 process
 > the raw-data
 >     if (!(buffCntr % 10))
 >     {   
 >     
    GstMapInfo map;
 >       
 GstSample *sample =
 >
 gst_app_sink_pull_sample(GST_APP_SINK(source));
 >         GstBuffer *buffer =
 gst_sample_get_buffer(sample);
 >     
    gst_buffer_ref(buffer);
 >       
 gst_buffer_map (buffer, &map, GST_MAP_READ);
 >         g_print("Map size:
 %d\n", map.size);
 >
 >         /*int8_t fd =
 open("/path/to/someFile", O_CREAT | O_WRONLY);
 >         if (fd != -1)
 >         {
 >     
        if(write(fd, map.data, map.size) == map.size)
 >               
 g_print("Written whole data\n");
 >             else
 >               
 g_print("Cannot write whole data\n");
 >         }
 >     
    close(fd);*/
 >       
 gst_buffer_unmap(buffer, &map);
 > 
        gst_sample_unref(sample);
 > 
        gst_buffer_unref(buffer);
 >
 >         //buffCntr = 0;
 >     }
 > }
 >
 > But the execution of
 the function dataProbe is so critical that if I
 > set buffCntr to zero (as shown in the last
 commented instruction of
 > the function)
 even without writing map.data to some file, the whole
 > program hangs, so the mp4 video is not
 written anymore..
 > But I need both
 raw-data and mp4 video.
 >
 > What can I do to 'speed up' this
 queue and let my gstreamer program
 > to
 record the video and write the raw data somewhere ? Pthread
 ?
 > There is some helpful feature already
 available from gstreamer
 > library to
 handle this situation ?
 >
 > The device which run the application is an
 embedded board with a quad
 > core CPU.
 >
 > Thank you in
 advance.
 > Regards,
 >
 Simon
 >
 _______________________________________________
 > gstreamer-devel mailing list
 > [hidden email]
 > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel_______________________________________________
 gstreamer-devel mailing list
 [hidden email]
 https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
 
 -----Adjunto en línea a continuación-----
 
 
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
simo-zz
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Speeding up a queue

simo-zz
Hello,
As promised, I followed the Nicolas's suggestion and now it's working fine.
Regards.





El Sábado 9 de septiembre de 2017 16:25, "[hidden email]" <[hidden email]> escribió:


Hello Martín,
Thank you for tour hint. I will keep in mind that.

At the moment, since I want to write the content of the map.data to a pipe, I tested the fdsink and the filesink instead of the appsink since they write data without the need to do it manually.

The fdsink give me problems.. It actually overload the program and the video is not written any more after the first chunk of data.
filesink works far better.
The only problem I see with using filesink with a pipe, is that if the reader closes the stream, gstreamer stops/crashes.

However I don't discard to use the appsink again after your hints. I will test it and report the results.

Regards,
Simon


--------------------------------------
El vie, 8/9/17, Nicolas Dufresne <[hidden email]> escribió:

Asunto: Re: Speeding up a queue
Para: "Discussion of the development of and with GStreamer" <[hidden email]>, "simo zz" <[hidden email]>
Fecha: viernes, 8 de septiembre, 2017 19:21

Le vendredi 08 septembre 2017 à
09:11 +0000, Martin Vachovski a écrit :
> Hi Simon,
>
> Three suggestions in the order of
relevance:
>
> 1) I
think there is no need to explicitly call
> gst_buffer_ref(buffer)-- the
gst_sample_get_buffer should do it
>
internally

No it
"transfer none".

>
> 2) Are you sure that
the performance is the issue here?
> Try
changing the condition whether you access the data
> and see when it stops working, like:
> if(!(buffCntr % 20)) // every 20th
> then
> if(!(buffCntr %
15)) // every 15th
> and see if for
sufficiently small denominator it stops working
>
> 3) Also, try calling
the "unref" methods in the opposite order as
the
> "get/ref" methods:
> You have
>
ref(sample)
> ref(buffer)
> But you release the sample first
> unref(sample)
>
unref(buffer)
> I think it should be:
> unref(buffer)
>
unref(sample)

This is
usually not required with ref-counting in general. Since
if
there dependent object will have a ref on
the other, which guaranty the
free order for
you.

>
>
Hope that helps
> Martin
>
> From:
gstreamer-devel <[hidden email]>
> on behalf of simo zz <[hidden email]>
> Sent: Thursday, September 7, 2017 5:00
PM
> To: Discussion of the Development of
and With GStreamer
> Subject: Speeding up
a queue
> 
> Hello,

> I need to record a video from the USB
camera and process the raw-data
> at the
same time, so I successfully implemented this pipeline in
C:
>
> v4l2src
device=/dev/webcam ! videoconvert ! video/x-
> raw,width=544,height=288,framerate=10/1 !
tee name=t ! queue
> ! v4l2h264enc !
h264parse ! mp4mux ! filesink location=video.mp4 t. !
> queue ! appsink
>
> I can access to raw-data from USB camera
is read using the "new-
>
sample" signal of appsink in a function called
"dataProbe", code as
>
follows:
>
>
GstFlowReturn dataProbe(GstElement *source)
> { 
>     //
buffCntr is just a counter to choose when I want to
process
> the raw-data
>     if (!(buffCntr % 10))
>     {  
>     
   GstMapInfo map;
>       
GstSample *sample =
>
gst_app_sink_pull_sample(GST_APP_SINK(source));
>         GstBuffer *buffer =
gst_sample_get_buffer(sample);
>     
   gst_buffer_ref(buffer);
>       
gst_buffer_map (buffer, &map, GST_MAP_READ);
>         g_print("Map size:
%d\n", map.size);
>
>         /*int8_t fd =
open("/path/to/someFile", O_CREAT | O_WRONLY);
>         if (fd != -1)
>         {
>     
       if(write(fd, map.data, map.size) == map.size)
>               
g_print("Written whole data\n");
>             else
>               
g_print("Cannot write whole data\n");
>         }
>     
   close(fd);*/
>       
gst_buffer_unmap(buffer, &map);
> 
       gst_sample_unref(sample);
> 
       gst_buffer_unref(buffer);
>
>         //buffCntr = 0;
>     }
> }
>
> But the execution of
the function dataProbe is so critical that if I
> set buffCntr to zero (as shown in the last
commented instruction of
> the function)
even without writing map.data to some file, the whole
> program hangs, so the mp4 video is not
written anymore..
> But I need both
raw-data and mp4 video.
>
> What can I do to 'speed up' this
queue and let my gstreamer program
> to
record the video and write the raw data somewhere ? Pthread
?
> There is some helpful feature already
available from gstreamer
> library to
handle this situation ?
>
> The device which run the application is an
embedded board with a quad
> core CPU.
>
> Thank you in
advance.
> Regards,
>
Simon
>
_______________________________________________
> gstreamer-devel mailing list
gstreamer-devel mailing list

-----Adjunto en línea a continuación-----




_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Free forum by Nabble Edit this page