GStreamer-devel

VideoFilter(transformation element) implementation

Classic

List

Threaded

12 messages Options

alex-14

VideoFilter(transformation element) implementation

Hello to all.

I'm a novice in gstreamer plugins writing, so the question to "novice".
I'm using debian squeeze with gstreamer-0.10.28 from distribution.

I need to implement element which can be described as
video_coverter/video_filter.
Unfortunately there are no such example in PWG(as for now i don't read
it to the end, just finished chapter 2), there is only gsttransform.c
element and gstplugin.c . Particularly my element must have independent
input and output buffers, because of kind of processing happening.

At first I tried to realize this on base of gstplugin code with creation
of additional GstBuffer (which act as output) but this causes memory
leak, because i can't free memory before pass data to output.

2-nd try was to create a temporary buffer directly using glib memory
management function g_memdup() with copy of input GstBuffer data, then
modify something, then copy data back to GstBuffer, free allocated
memory and output modified GstBuffer to element sink pad. In this case i
haven't troubles with memory but it can be seen (i see for example
blinking(black - orig, red - after modify) square) that, as i understand
that outputted data overwritten by inputted sometimes, as element has
one buffer.

Finally, as i understand there is no pre-made class for video-filters,
so i must derive my plugin from GstBaseTransform (so use gsttransform.c
template). As all that i try to do earlier - stupid.

In gstreamer-libs documentation i found that i need to implement
"prepare_output_buffer" function where output buffer must be created:
==============================================================
Normal mode
* always_in_place flag is not set, or there is no transform_ip
function
* Element will receive an input buffer and output buffer to
operate on.
* Output buffer is allocated by calling the
prepare_output_buffer function.
Example elements
* Videoscale, ffmpegcolorspace, audioconvert when doing
scaling/conversions

Special output buffer allocations
* Elements which need to do special allocation of their output
buffers other than what gst_buffer_pad_alloc allows should implement a
prepare_output_buffer method, which calls the parent implementation and
passes the newly allocated buffer.
===============================================================

But i'm found no example on how to implement such function.
Can anyone be so kind and give implementation example or some direction
where i can learn how to write video transformation element that work in
"Normal mode"

P.S. If you need any additional info please ask. And I tried to look at
"Example elements" which pointed in docs (videoscale, ffmpegcolorspace),
but they are at least complicated and overloaded by concrete
case|implementation, and so not clear for novice like me.

I found solution with temporary buffer that works (in gstplugin.c case),
so now I'm not sure that my try with g_memdup()/g_free() was totally wrong.

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

alex-14

Re: VideoFilter(transformation element) implementation

On 04/02/2010 12:17 AM, alex wrote:
> und solution with temporary buffer that works (in gstplugin.c case),
> so now I'm not sure that my try with g_memdup()/g_free() was totally wrong.

I talk about gstedgedetect plugin from gst-opencv distribution.

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Stefan Sauer

Re: VideoFilter(transformation element) implementation

In reply to this post by alex-14

Am 02.04.2010 00:17, schrieb alex:

> Hello to all.
>
> I'm a novice in gstreamer plugins writing, so the question to "novice".
> I'm using debian squeeze with gstreamer-0.10.28 from distribution.
>
> I need to implement element which can be described as
> video_coverter/video_filter.
> Unfortunately there are no such example in PWG(as for now i don't read
> it to the end, just finished chapter 2), there is only gsttransform.c
> element and gstplugin.c . Particularly my element must have independent
> input and output buffers, because of kind of processing happening.

There are lot of examples in the atual plugins. Also
gst-plugins-base/gst-libs/gst/video has a GstVideoFilter baseclass.

For transform type of filters there are two modes: in place where you modyfy the
received bufffer and push that further and not in place. When operating not in
place, you allocate a new buffer (gst_pad_alloc_new_and_set_caps()) process
input, putting the result into the new buffer, then you copy buffer metadata
over (gst_buffer_copy_metadata()). FInally you unref the input-buffer and push
the new buffer.

Stefan

>
> At first I tried to realize this on base of gstplugin code with creation
> of additional GstBuffer (which act as output) but this causes memory
> leak, because i can't free memory before pass data to output.
>
> 2-nd try was to create a temporary buffer directly using glib memory
> management function g_memdup() with copy of input GstBuffer data, then
> modify something, then copy data back to GstBuffer, free allocated
> memory and output modified GstBuffer to element sink pad. In this case i
> haven't troubles with memory but it can be seen (i see for example
> blinking(black - orig, red - after modify) square) that, as i understand
> that outputted data overwritten by inputted sometimes, as element has
> one buffer.
>
> Finally, as i understand there is no pre-made class for video-filters,
> so i must derive my plugin from GstBaseTransform (so use gsttransform.c
> template). As all that i try to do earlier - stupid.
>
> In gstreamer-libs documentation i found that i need to implement
> "prepare_output_buffer" function where output buffer must be created:
> ==============================================================
> Normal mode
> * always_in_place flag is not set, or there is no transform_ip
> function
> * Element will receive an input buffer and output buffer to
> operate on.
> * Output buffer is allocated by calling the
> prepare_output_buffer function.
> Example elements
> * Videoscale, ffmpegcolorspace, audioconvert when doing
> scaling/conversions
>
> Special output buffer allocations
> * Elements which need to do special allocation of their output
> buffers other than what gst_buffer_pad_alloc allows should implement a
> prepare_output_buffer method, which calls the parent implementation and
> passes the newly allocated buffer.
> ===============================================================
>
> But i'm found no example on how to implement such function.
> Can anyone be so kind and give implementation example or some direction
> where i can learn how to write video transformation element that work in
> "Normal mode"
>
> P.S. If you need any additional info please ask. And I tried to look at
> "Example elements" which pointed in docs (videoscale, ffmpegcolorspace),
> but they are at least complicated and overloaded by concrete
> case|implementation, and so not clear for novice like me.
>
> I found solution with temporary buffer that works (in gstplugin.c case),
> so now I'm not sure that my try with g_memdup()/g_free() was totally wrong.
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

alex-14

Re: VideoFilter(transformation element) implementation

On 04/02/2010 10:27 AM, Stefan Kost wrote:
> Am 02.04.2010 00:17, schrieb alex:
> There are lot of examples in the atual plugins. Also

Yeh, but as for me they mostly overloaded by concrete thing
implementation, so it's hard to see basic structure.

> gst-plugins-base/gst-libs/gst/video has a GstVideoFilter baseclass.

Ok. Thanks that's hotter point. Here i see even template for plugin,
but when i try to build plugin generated from this template i got error:

0/videofilter> ./make_filter test
$Id$
1/videofilter> gcc `pkg-config --libs --cflags gstreamer-0.10`
`pkg-config --libs --cflags gstreamer-video-0.10`
-I/usr/include/gstreamer-0.10/gst/video gsttest.c --shared -o libtest.so

gsttest.c:52: error: expected specifier-qualifier-list before
‘GstVideofilter’
gsttest.c:58: error: expected specifier-qualifier-list before
‘GstVideofilterClass’
gsttest.c:85: error: expected ‘)’ before ‘*’ token
gsttest.c:87: error: expected ‘)’ before ‘*’ token
gsttest.c: In function ‘gst_test_get_type’:
gsttest.c:107: error: ‘GST_TYPE_VIDEOFILTER’ undeclared (first use in
this function)
gsttest.c:107: error: (Each undeclared identifier is reported only once
gsttest.c:107: error: for each function it appears in.)
gsttest.c: At top level:
gsttest.c:113: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’
before ‘gst_test_formats’
gsttest.c: In function ‘gst_test_base_init’:
gsttest.c:127: error: ‘GstVideofilterClass’ undeclared (first use in
this function)
gsttest.c:127: error: ‘videofilter_class’ undeclared (first use in this
function)
gsttest.c:132: error: ‘gst_test_formats’ undeclared (first use in this
function)
gsttest.c: In function ‘gst_test_class_init’:
gsttest.c:144: error: ‘GstVideofilterClass’ undeclared (first use in
this function)
gsttest.c:144: error: ‘videofilter_class’ undeclared (first use in this
function)
gsttest.c:159: error: ‘gst_test_setup’ undeclared (first use in this
function)
gsttest.c: In function ‘gst_test_init’:
gsttest.c:166: error: ‘GstVideofilter’ undeclared (first use in this
function)
gsttest.c:166: error: ‘videofilter’ undeclared (first use in this function)
gsttest.c: At top level:
gsttest.c:224: error: ‘VERSION’ undeclared here (not in a function)
gsttest.c:224: error: ‘GST_LICENSE’ undeclared here (not in a function)
gsttest.c:224: error: ‘PACKAGE’ undeclared here (not in a function)
gsttest.c:224: error: ‘GST_PACKAGE_NAME’ undeclared here (not in a function)
gsttest.c:224: error: ‘GST_PACKAGE_ORIGIN’ undeclared here (not in a
function)
gsttest.c:230: error: expected ‘)’ before ‘*’ token
gsttest.c:242: error: expected ‘)’ before ‘*’ token

> For transform type of filters there are two modes: in place where you modyfy the
> received bufffer and push that further and not in place. When operating not in

Ok, I understand this.
> place, you allocate a new buffer (gst_pad_alloc_new_and_set_caps()) process

I dont find that function and assume that you mean
gst_pad_alloc_buffer_and_set_caps()

> input, putting the result into the new buffer, then you copy buffer metadata
> over (gst_buffer_copy_metadata()). FInally you unref the input-buffer and push
> the new buffer.

The core question for me is where place buffer allocation? Is it must be
in *_transform(*_chain) method(function), or like in gstedgedetect
plugin from gst-opencv in _set_caps() function and unref it in _finalize()
or like in edgetv in -plugins-good:
alloc in _start(), free in _finalize()

At the end you say i must unref input-buffer and then push new buffer.
But I don't see in referenced examples[edgetv,edgedetect] that they
unref input buffer.

Also when i tried to make the whole thing like in gstedgedetect, i found
that blinking stayed (in 1st xvimagesink image sometimes overwriten by
the image in 2nd). Launch string was:

gst-launch --gst-plugin-path=./ videotestsrc num-buffers=1000 !
video/x-raw-yuv,width=320,height=240 ! tee name=tscreen ! queue !
autovideosink tscreen. ! queue ! decodebin ! ffmpegcolorspace !
examplefilter ! edge_detector ! ffmpegcolorspace ! autovideosink

here "edge_detector" is discussed element.
But there is no such effect in gstedgedetect.

p.s. i just try this pipeline with edgetv, and it has same trouble, so
maybe this is not my element problem.

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

David Schleef-2

Re: VideoFilter(transformation element) implementation

On Sun, Apr 04, 2010 at 02:12:37PM +0300, alex wrote:

> On 04/02/2010 10:27 AM, Stefan Kost wrote:
> > Am 02.04.2010 00:17, schrieb alex:
> > There are lot of examples in the atual plugins. Also
>
> Yeh, but as for me they mostly overloaded by concrete thing
> implementation, so it's hard to see basic structure.
>
> > gst-plugins-base/gst-libs/gst/video has a GstVideoFilter baseclass.
>
> Ok. Thanks that's hotter point. Here i see even template for plugin,
> but when i try to build plugin generated from this template i got error:
>
> 0/videofilter> ./make_filter test

This template was never updated for 0.10, so it's not a useful
starting point.

dave...

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Kulecz, Walter (JSC-SK)[WYLE INTEG. SCI. & ENG.]

Re: VideoFilter(transformation element) implementation

________________________________________

>From: David Schleef [[hidden email]]
>Sent: Sunday, April 04, 2010 2:35 PM
>To: Discussion of the development of GStreamer
>Subject: Re: [gst-devel] VideoFilter(transformation element) implementation
>
>On Sun, Apr 04, 2010 at 02:12:37PM +0300, alex wrote:
>> On 04/02/2010 10:27 AM, Stefan Kost wrote:
>> > Am 02.04.2010 00:17, schrieb alex:
>> > There are lot of examples in the atual plugins. Also
>>
>> Yeh, but as for me they mostly overloaded by concrete thing
>> implementation, so it's hard to see basic structure.
>>
>> > gst-plugins-base/gst-libs/gst/video has a GstVideoFilter baseclass.
>>
>> Ok. Thanks that's hotter point. Here i see even template for plugin,
>> but when i try to build plugin generated from this template i got error:
>>
>> 0/videofilter> ./make_filter test
>
>This template was never updated for 0.10, so it's not a useful
>starting point.

So why is it still in there? Gstreamer is hard enough to figure out as it is.
------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Edward Hervey

Re: VideoFilter(transformation element) implementation

Administrator

On Tue, 2010-04-06 at 08:44 -0500, Kulecz, Walter (JSC-SK)[WYLE INTEG.
SCI. & ENG.] wrote:

> ________________________________________
> >From: David Schleef [[hidden email]]
> >Sent: Sunday, April 04, 2010 2:35 PM
> >To: Discussion of the development of GStreamer
> >Subject: Re: [gst-devel] VideoFilter(transformation element) implementation
> >
> >On Sun, Apr 04, 2010 at 02:12:37PM +0300, alex wrote:
> >> On 04/02/2010 10:27 AM, Stefan Kost wrote:
> >> > Am 02.04.2010 00:17, schrieb alex:
> >> > There are lot of examples in the atual plugins. Also
> >>
> >> Yeh, but as for me they mostly overloaded by concrete thing
> >> implementation, so it's hard to see basic structure.
> >>
> >> > gst-plugins-base/gst-libs/gst/video has a GstVideoFilter baseclass.
> >>
> >> Ok. Thanks that's hotter point. Here i see even template for plugin,
> >> but when i try to build plugin generated from this template i got error:
> >>
> >> 0/videofilter> ./make_filter test
> >
> >This template was never updated for 0.10, so it's not a useful
> >starting point.
>
> So why is it still in there? Gstreamer is hard enough to figure out as it is.

Define 'there'. I can't see any mention of that obsolete video filter
template in git of gstreamer core, -base or -template.

Edward

P.S. Asking for it to be removed instead of complaining as you're doing
might have, you know... a better chance of it being fixed.

> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

alex-14

Re: VideoFilter(transformation element) implementation

On 07/04/2010 19:44, Edward Hervey wrote:
>> So why is it still in there? Gstreamer is hard enough to figure out as it is.
>
> Define 'there'. I can't see any mention of that obsolete video filter
> template in git of gstreamer core, -base or -template.
>
> Edward
>
> P.S. Asking for it to be removed instead of complaining as you're doing
> might have, you know... a better chance of it being fixed.
http://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/gst/videofilter/gstvideotemplate.c

I'll be very happy if it will be fixed. For now I found ~3 working
solutions (really 3 variations of 1 solution). As for now i derive my
plugin from GstElement, and variants are:
1) create temp buffer once in _start() method and free it in _finalize()
2a) create temp buffer, use it and free it in _transform() with GLib
memory management functions
2b) create output buffer (gst_buffer_copy), make transformations on it
and free input buffer (gst_buffer_unref) in _transform() with those
gst-specific functions.

I think 1-st method better, because buffer allocated and freed only once.

But i still think, that it is not the fully right way.

P.S. If in near future i will gain better understanding of this i'll try
to fix template by myself.

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Edward Hervey

Understanding and Reading GObject code Re: VideoFilter(transformation element) implementation

Administrator

On Thu, 2010-04-08 at 00:46 +0300, alex wrote:

> On 07/04/2010 19:44, Edward Hervey wrote:
> >> So why is it still in there? Gstreamer is hard enough to figure out as it is.
> >
> > Define 'there'. I can't see any mention of that obsolete video filter
> > template in git of gstreamer core, -base or -template.
> >
> > Edward
> >
> > P.S. Asking for it to be removed instead of complaining as you're doing
> > might have, you know... a better chance of it being fixed.
> http://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/gst/videofilter/gstvideotemplate.c

Indeed, that template shouldn't be used. In fact, as we'll see below,
implemeting a video filter doesn't really require a template, there
isn't much code involved once you put aside all the GObject template
required.

The tip from Stefan is still valid. Have a look at videobalance in
that directory for example, it's a *rather* clean example for an
in-place filter.

How to read such code ? Here's the step-by-step of reading and
understanding GObject-based code. We'll be going over it in roughly the
same order as the code excution.

(Anybody having issues/troubles *reading* and understanding
GStreamer/GObject code should read this too and then head to a more
extensive GObject documentation to understand the finer details [1])

* GType

The trick to understand ANY GObject classes implementation (and what
is needed/used) is to realize that everything GObject related starts
with a GType.

A GType is *the* identifier/alias for a class. C doesn't have the
notion of classes, so GObject offers a system to be able to
differentiate classes (and have inheritance, etc... like any other
object-oriented system). If inheritance and classes are unknown to you,
go have a quick read on wikipedia about object-oriented programming.

For every class, we need a way to be able to get its GType. The
GObject way of doing that is to define a macro to get to it:
gstvideobalance.h:
#define GST_TYPE_VIDEO_BALANCE (gst_video_balance_get_type())

==> THIS is the entry point to reading any gobject-code. Anything
related to a GObject class starts here !. Look at the rest of the macros
in that header, they all use GST_TYPE_VIDEO_BALANCE.
If you look at other header files in GStreamer core (like
gstelement.h, gstbin.h, ....) they all have those macros pointing to a
_get_type() function.

In the GStreamer plugin context, when you a plugin is read/loaded (see
plugin_init at the bottom of gstvideobalance.c) you also just pass along
that GType to GStreamer ("hey GStreamer, here's the GType of a
GstElement class you should offer to users").

** GObject type registration

Let's carry on reading the code, and have a look at _get_type.

* gst_video_balance_get_type
This method return the GType correponding to this class. If not
present (i.e. the very first time it's called), it will register it.
i.e. define the basics of this type.

The g_type_register_*() line gives the overview:
* We give it the parent type. It's parent GType here is
GST_TYPE_VIDEO_FILTER (it could have been GST_TYPE_BASE_TRANSFORM for
the sake of clarity, but no biggie). The most basic class will derive
from G_TYPE_OBJECT.
* It's well know name is "GstVideoBalance". This is mainly used for
debugging.
* Some information about our GType is registered (video_balance_info)
* methods to be called the very first time a GType is used (i.e.
the first time an instance of this class (or a subclass) will be needed.
* details about the difference between base_init and class_init can
be seen in GObject tutorials, suffice to say they're both called before
the very first time an instance of this type will be created. Not all
classes use the _base_init, but all classes that need to override some
class vmethods need to have a _class_init and do it there.
* we also give the method which will be called whenever an instance
of this type will be created (gst_video_balance_init).

You can ignore the parts about interfaces for the time being since
they're not essential for this, but if you understood what
g_type_register does, it's basically the same idea but for interfaces.

** GObject class initialization

This is where we will refine the behaviour of our class, i.e. override
class-global behaviour (methods, class global properties, ...)

* gst_video_balance_class_init
=> it overrides the set_property/get_property methods of GObjectClass,
those are the methods that will be called when you want to read/set one
of the properties you exposed.
=> it installs some properties that can be used from the outside
(Those are the ones that appear in gst-inspect, that you can set on
gst-launch or with g_object_set())
=> It implements a GObjectClass finalize method (this is where you
should clean up everything you allocated in your instance)
=> It overrides the GstTransformClass 'set_caps' method, this is
because it wants to know the configured width and height of the video
buffers we will receive.
=> Finally, and most important in our case, it implements the
'transform_ip' method of GstBaseTransform. We are hereby saying that we
can modify the buffers in-place (we don't need to create a new one).
Useful if your input and output buffer are the same size.

** GObject instance initialization

This method (defined in the type registration, here
gst_video_balance_init) will be called whenever a new instance of our
class is created.
If instance-specific data needs to be allocated/initialized, it should
be done here.
Don't forget to free any allocated data in the 'finalize' method of
your class.

** Carrying on with our transform element

Everything from now on is specific to the videobalance class, based on
all the initialization we did in the few steps above.

We implemented a GstBaseTransform 'set_caps' method
gst_video_balance_set_caps:
* We remember in our instance what the configured width/height are and
return TRUE (provided those properties are present in the caps)

Our class doesn't modify the caps and implements a 'transform_ip'
method, so that method will be called for every buffer coming in.

gst_video_balance_transform_ip:
* ignore the passthrough related parts for the time being, and the
checks for sizes.
What does it do ? Takes the buffer data... and calls the actual
transformation method (i.e. what's specific to your element) with the
buffer data and the configured width/height.

*** What now ?

This showed the breakdown of how/what code gets called in a GObject
class. Hopefully this should help people be able to *read* gstreamer
plugins a bit quicker, and help read/understand the GStreamer reference
manual.

If you want an example of a video filter that *does* create new
buffers, read in the same way the code to gstvideoflip in the same
directory. It's a 700 line self-contained file.

Hope this helps,

Edward

[1] : http://library.gnome.org/devel/gobject/stable/

>
> I'll be very happy if it will be fixed. For now I found ~3 working
> solutions (really 3 variations of 1 solution). As for now i derive my
> plugin from GstElement, and variants are:
> 1) create temp buffer once in _start() method and free it in _finalize()
> 2a) create temp buffer, use it and free it in _transform() with GLib
> memory management functions
> 2b) create output buffer (gst_buffer_copy), make transformations on it
> and free input buffer (gst_buffer_unref) in _transform() with those
> gst-specific functions.
>
> I think 1-st method better, because buffer allocated and freed only once.
>
> But i still think, that it is not the fully right way.
>
> P.S. If in near future i will gain better understanding of this i'll try
> to fix template by myself.
>
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Kulecz, Walter (JSC-SK)[WYLE INTEG. SCI. & ENG.]

Re: Understanding and Reading GObject code Re: VideoFilter(transformation element) implementation

Outstanding! This should have been in the top level gstreamer docs from day one, not buried in an Email list.

I'd wasted several days worth of my limited free time over several weeks to learn this (by poking around with the gamma plugin after the Video Filter transform element in the template I downloaded proved to be a dead end).

The book "Foundations of GTK+ Development" by Andrew Krause provided the key info for me.

I didn't find http://library.gnome.org/devel/gobject/stable/ all that helpful as its more oriented towards creating GObjects instead of explaing how to use them.

Turns out appsink and appsrc seem more appropriate for what I'm trying to do, but these weren't part of the gstreamer version that shipped with Ubuntu 8.04 :(

________________________________________
From: Edward Hervey [[hidden email]]
Sent: Thursday, April 08, 2010 2:43 AM
To: Discussion of the development of GStreamer
Subject: [gst-devel] Understanding and Reading GObject code Re: VideoFilter(transformation element) implementation

On Thu, 2010-04-08 at 00:46 +0300, alex wrote:

Edward Hervey

Re: Understanding and Reading GObject code Re: VideoFilter(transformation element) implementation

Administrator

On Thu, 2010-04-08 at 08:40 -0500, Kulecz, Walter (JSC-SK)[WYLE INTEG.
SCI. & ENG.] wrote:
> Outstanding! This should have been in the top level gstreamer docs
> from day one, not buried in an Email list.

I think you mean it should have been in the top-leve *GOBJECT* docs.
Most of what I explained is gobject stuff, not that much gstreamer
specific.

>
> I'd wasted several days worth of my limited free time over several
> weeks to learn this (by poking around with the gamma plugin after the
> Video Filter transform element in the template I downloaded proved to
> be a dead end).

I'm sure you (would) have learnt things along the way that I didn't
explain below.

>
> The book "Foundations of GTK+ Development" by Andrew Krause provided
> the key info for me.
>
> I didn't find http://library.gnome.org/devel/gobject/stable/ all that
> helpful as its more oriented towards creating GObjects instead of
> explaing how to use them.
>
> Turns out appsink and appsrc seem more appropriate for what I'm trying
> to do, but these weren't part of the gstreamer version that shipped
> with Ubuntu 8.04 :(

You really should consider getting a newer distribution or finding
backports of GStreamer for that distribution (which might prove clumsy
considering you'd also have to update the dependencies).

Edward

>
>
> ________________________________________
> From: Edward Hervey [[hidden email]]
> Sent: Thursday, April 08, 2010 2:43 AM
> To: Discussion of the development of GStreamer
> Subject: [gst-devel] Understanding and Reading GObject code Re: VideoFilter(transformation element) implementation
>
> On Thu, 2010-04-08 at 00:46 +0300, alex wrote:
> > On 07/04/2010 19:44, Edward Hervey wrote:
> > >> So why is it still in there? Gstreamer is hard enough to figure out as it is.
> > >
> > > Define 'there'. I can't see any mention of that obsolete video filter
> > > template in git of gstreamer core, -base or -template.
> > >
> > > Edward
> > >
> > > P.S. Asking for it to be removed instead of complaining as you're doing
> > > might have, you know... a better chance of it being fixed.
> > http://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/gst/videofilter/gstvideotemplate.c
>
> Indeed, that template shouldn't be used. In fact, as we'll see below,
> implemeting a video filter doesn't really require a template, there
> isn't much code involved once you put aside all the GObject template
> required.
>
> The tip from Stefan is still valid. Have a look at videobalance in
> that directory for example, it's a *rather* clean example for an
> in-place filter.
>
> How to read such code ? Here's the step-by-step of reading and
> understanding GObject-based code. We'll be going over it in roughly the
> same order as the code excution.
>
> (Anybody having issues/troubles *reading* and understanding
> GStreamer/GObject code should read this too and then head to a more
> extensive GObject documentation to understand the finer details [1])
>
>
>
> * GType
>
> The trick to understand ANY GObject classes implementation (and what
> is needed/used) is to realize that everything GObject related starts
> with a GType.
>
> A GType is *the* identifier/alias for a class. C doesn't have the
> notion of classes, so GObject offers a system to be able to
> differentiate classes (and have inheritance, etc... like any other
> object-oriented system). If inheritance and classes are unknown to you,
> go have a quick read on wikipedia about object-oriented programming.
>
> For every class, we need a way to be able to get its GType. The
> GObject way of doing that is to define a macro to get to it:
> gstvideobalance.h:
> #define GST_TYPE_VIDEO_BALANCE (gst_video_balance_get_type())
>
> ==> THIS is the entry point to reading any gobject-code. Anything
> related to a GObject class starts here !. Look at the rest of the macros
> in that header, they all use GST_TYPE_VIDEO_BALANCE.
> If you look at other header files in GStreamer core (like
> gstelement.h, gstbin.h, ....) they all have those macros pointing to a
> _get_type() function.
>
> In the GStreamer plugin context, when you a plugin is read/loaded (see
> plugin_init at the bottom of gstvideobalance.c) you also just pass along
> that GType to GStreamer ("hey GStreamer, here's the GType of a
> GstElement class you should offer to users").
>
>
>
> ** GObject type registration
>
> Let's carry on reading the code, and have a look at _get_type.
>
> * gst_video_balance_get_type
> This method return the GType correponding to this class. If not
> present (i.e. the very first time it's called), it will register it.
> i.e. define the basics of this type.
>
> The g_type_register_*() line gives the overview:
> * We give it the parent type. It's parent GType here is
> GST_TYPE_VIDEO_FILTER (it could have been GST_TYPE_BASE_TRANSFORM for
> the sake of clarity, but no biggie). The most basic class will derive
> from G_TYPE_OBJECT.
> * It's well know name is "GstVideoBalance". This is mainly used for
> debugging.
> * Some information about our GType is registered (video_balance_info)
> * methods to be called the very first time a GType is used (i.e.
> the first time an instance of this class (or a subclass) will be needed.
> * details about the difference between base_init and class_init can
> be seen in GObject tutorials, suffice to say they're both called before
> the very first time an instance of this type will be created. Not all
> classes use the _base_init, but all classes that need to override some
> class vmethods need to have a _class_init and do it there.
> * we also give the method which will be called whenever an instance
> of this type will be created (gst_video_balance_init).
>
> You can ignore the parts about interfaces for the time being since
> they're not essential for this, but if you understood what
> g_type_register does, it's basically the same idea but for interfaces.
>
>
>
> ** GObject class initialization
>
> This is where we will refine the behaviour of our class, i.e. override
> class-global behaviour (methods, class global properties, ...)
>
> * gst_video_balance_class_init
> => it overrides the set_property/get_property methods of GObjectClass,
> those are the methods that will be called when you want to read/set one
> of the properties you exposed.
> => it installs some properties that can be used from the outside
> (Those are the ones that appear in gst-inspect, that you can set on
> gst-launch or with g_object_set())
> => It implements a GObjectClass finalize method (this is where you
> should clean up everything you allocated in your instance)
> => It overrides the GstTransformClass 'set_caps' method, this is
> because it wants to know the configured width and height of the video
> buffers we will receive.
> => Finally, and most important in our case, it implements the
> 'transform_ip' method of GstBaseTransform. We are hereby saying that we
> can modify the buffers in-place (we don't need to create a new one).
> Useful if your input and output buffer are the same size.
>
>
> ** GObject instance initialization
>
> This method (defined in the type registration, here
> gst_video_balance_init) will be called whenever a new instance of our
> class is created.
> If instance-specific data needs to be allocated/initialized, it should
> be done here.
> Don't forget to free any allocated data in the 'finalize' method of
> your class.
>
>
> ** Carrying on with our transform element
>
> Everything from now on is specific to the videobalance class, based on
> all the initialization we did in the few steps above.
>
> We implemented a GstBaseTransform 'set_caps' method
> gst_video_balance_set_caps:
> * We remember in our instance what the configured width/height are and
> return TRUE (provided those properties are present in the caps)
>
> Our class doesn't modify the caps and implements a 'transform_ip'
> method, so that method will be called for every buffer coming in.
>
> gst_video_balance_transform_ip:
> * ignore the passthrough related parts for the time being, and the
> checks for sizes.
> What does it do ? Takes the buffer data... and calls the actual
> transformation method (i.e. what's specific to your element) with the
> buffer data and the configured width/height.
>
>
> *** What now ?
>
> This showed the breakdown of how/what code gets called in a GObject
> class. Hopefully this should help people be able to *read* gstreamer
> plugins a bit quicker, and help read/understand the GStreamer reference
> manual.
>
> If you want an example of a video filter that *does* create new
> buffers, read in the same way the code to gstvideoflip in the same
> directory. It's a 700 line self-contained file.
>
> Hope this helps,
>
> Edward
>
>
> [1] : http://library.gnome.org/devel/gobject/stable/
> ------------------------------------------------------------------------------
> Download Intel® Parallel Studio Eval
> Try the new software tools for yourself. Speed compiling, find bugs
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Kulecz, Walter (JSC-SK)[WYLE INTEG. SCI. & ENG.]

Re: Understanding and Reading GObject code Re: VideoFilter(transformation element) implementation

>From: Edward Hervey [[hidden email]]
>Sent: Thursday, April 08, 2010 9:04 AM
>To: Discussion of the development of GStreamer
>Subject: Re: [gst-devel] Understanding and Reading GObject code Re: VideoFilter(transformation element) implementation
>
>On Thu, 2010-04-08 at 08:40 -0500, Kulecz, Walter (JSC-SK)[WYLE INTEG.SCI. & ENG.] wrote:
>> Outstanding! This should have been in the top level gstreamer docs
>> from day one, not buried in an Email list.
>
> I think you mean it should have been in the top-level *GOBJECT* docs.
>Most of what I explained is gobject stuff, not that much gstreamer specific.

Call me a luddite, but I think one should be able to use gstreamer without a lot of knowledge of the underlying machinery, but currently its darn near impossible for lack of documentation and fully functional examples. That is allegedly one of the benefits of the Object Oriented abstraction, which seems to be a dismal failure in actual usage. The gst-launch command is really about the only way to know if you are even on the right track in using the pieces.

>> I'd wasted several days worth of my limited free time over several
>> weeks to learn this (by poking around with the gamma plugin after the
>> Video Filter transform element in the template I downloaded proved to
>> be a dead end).
>
> I'm sure you (would) have learnt things along the way that I didn't explain below.

Sure, but precious little of it was germane to the task at hand. I'm not being paid to learn
any object system or API, I'm being paid to produce results.

>> Turns out appsink and appsrc seem more appropriate for what I'm trying
>> to do, but these weren't part of the gstreamer version that shipped
>> with Ubuntu 8.04 :(
>
> You really should consider getting a newer distribution or finding
>backports of GStreamer for that distribution (which might prove clumsy
>considering you'd also have to update the dependencies).

That was the system I had available to evaluate gstreamer on at the time. If I choose to commit to using gstreamer for this project, it'll target Ubuntu 10.04. I'm playing with its Beta release and appsrc and appsink gstreamer-0.10.28 examples now.
------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel