Getting data from buffer to pass to another function/process outside of Gstreamer

[dayunbao]

Hi all,

I have a pipeline that basically looks like this:

v4l2src -> videorate -> jpegenc -> queue -> jifmux -> multifilesink

I'm capturing a frame once a second from a web cam, and saving is as a JPEG.  I've also added a probe to the source pad of the JPEG encoder, so whenever the buffer is pushed downstream I can have a function called to do something.  What I'd also like to do is copy the data from the buffer before it is encoded by the JPEG encoder so I can pass it off to another program for some separate image processing.  I think this should be relatively easy, and I've read about the identity plug in, and its dump function, as well as the GSTBuffer's gst_buffer_copy_deep () function, but I'm not sure what is the best way to do this.  Does anyone have any suggestions?

Thanks!

[Arjen Veenhuizen]

There are multiple options. You could indeed insert the identity element before the encoder and wait for the handoff signal to fire. This callback will get you a reference to the GstBuffer that is about the pass the identity element. You can use gst_buffer_extract to extract the data from the buffer and do what you want.

Alternatively, you can split the pipeline in two, use appsink and wait for its callback:

 
                                    - queue - jpegenc -> queue -> jifmux -> multifilesink 
                                   /
v4l2src - videorate - queue - tee -
                                   \
                                    - queue - appsink

[Nicolas Dufresne-5]

Le jeudi 18 mai 2017 à 12:45 -0700, dayunbao a écrit :

> Hi all,
>
> I have a pipeline that basically looks like this:
>
> v4l2src -> videorate -> jpegenc -> queue -> jifmux -> multifilesink
>
> I'm capturing a frame once a second from a web cam, and saving is as
> a JPEG. 
> I've also added a probe to the source pad of the JPEG encoder, so
> whenever
> the buffer is pushed downstream I can have a function called to do
> something.  What I'd also like to do is copy the data from the buffer
> /before it is encoded by the JPEG encoder/ so I can pass it off to
> another
> program for some separate image processing.  I think this should be
> relatively easy, and I've read about the identity plug in, and its
> dump
> function, as well as the GSTBuffer's gst_buffer_copy_deep ()
> function, but
> I'm not sure what is the best way to do this.  Does anyone have any
> suggestions?

You could also use tee and appsink:

 v42lsrc ! tee name=t
    t. ! queue ! jpegenc ! ...
    t. ! queue ! appsink

>
> Thanks!
>
>
>
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble
> .com/Getting-data-from-buffer-to-pass-to-another-function-process-
> outside-of-Gstreamer-tp4683038.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Arjen Veenhuizen]

@Nicolas: great minds think alike ;)

[dayunbao]

Thanks for input everyone.  I think I'm going to go with using an identity element, as it seems simpler.  I'm not entirely sure how to catch and manage the identity's handoff signal, though.  There seems to be a few different ways of dealing with signals/callbacks/handoffs in GStreamer.  In my code so far, I have a call to add_signal_watch() and connect() on the bus element.  I also have an add_probe() call that takes a callback function as an argument.  However, I'm not sure what to use for the identity element's handoff signal.  I'm using Python bindings for GStreamer, by the way.

Excluding the code for all the other elements in my pipeline, here's what I know to do for the identity element:

#Create it
self.identity = Gst.ElementFactory.make('identity', 'identity')

#I know it defaults to True, but explicitly setting it for documentation purposes
self.identity.set_property('signal_handoffs', True)

#add it to the pipeline
self.pipeline.add(self.identity)

#link it with other elements
self.previousElementInPipeline.link(self.identity)
self.identity.link(self.nextElementInPipeline)

def functionToBeCalledWhenHandoffMessageReceived(self, args):
   doSomething()

#How do I link the above function with the identity element's handoff message?
#Like this???
self.identity.connect('handoff', self.functionToBeCalledWhenHandoffMessageReceived)

Thanks!

[dayunbao]

Never mind, I figured it out.  I had guess correctly in my last post.  it boiled down to just calling:

self.identity.connect('handoff', self.functionToBeCalledWhenHandoffMessageReceived)

Thanks!