GStreamer-devel

GnlComposition Signals

Classic

List

Threaded

7 messages Options

Timothy Braun

GnlComposition Signals

Hey Everyone,
So I have a gnlcomposition which, for the matter of depicting the situation, has 5 sources. 3 sources play for 30 seconds followed by the remaining 2 sources for another 30 seconds. This should produce a 1 minute video. The composition has a videomixer in a gnloperation to mix the sources for final output.

I'm running into a situation where the pipeline gets stuck during the transition from 3 sources to 2 sources. It seems the video mixer is trying to send a seek event to all it's sink pads, which fails on the pad which no-longer has a source. During my tests, the gnlcomposition will happily create new pads, ie. changing the order so it's 2 sources followed by 3 sources works fine. But the opposite doesn't seem to be true, the gnlcomposition doesn't remove unused pads to it's operations.

This brings me to the question of, is there a signal I'm not handling appropriately? If so, any pointers on which one I should be listening for?

Thanks,
Tim

------------------------------------------------------------------------------
Lotusphere 2011
Register now for Lotusphere 2011 and learn how
to connect the dots, take your collaborative environment
to the next level, and enter the era of Social Business.
http://p.sf.net/sfu/lotusphere-d2d
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Edward Hervey

Re: GnlComposition Signals

Administrator

On Mon, 2010-12-13 at 17:33 -0500, Timothy Braun wrote:

> Hey Everyone,
> So I have a gnlcomposition which, for the matter of depicting the
> situation, has 5 sources. 3 sources play for 30 seconds followed by
> the remaining 2 sources for another 30 seconds. This should produce a
> 1 minute video. The composition has a videomixer in a gnloperation to
> mix the sources for final output.
>
> I'm running into a situation where the pipeline gets stuck during
> the transition from 3 sources to 2 sources. It seems the video mixer
> is trying to send a seek event to all it's sink pads, which fails on
> the pad which no-longer has a source. During my tests, the
> gnlcomposition will happily create new pads, ie. changing the order so
> it's 2 sources followed by 3 sources works fine. But the opposite
> doesn't seem to be true, the gnlcomposition doesn't remove unused pads
> to it's operations.
>
> This brings me to the question of, is there a signal I'm not
> handling appropriately? If so, any pointers on which one I should be
> listening for?

If the element you put in a gnloperation has dynamic sink pads, the
gnloperation will call gst_element_request_pad() and
gst_element_release_request_pad() when needed.
videomixer alone should work fine in those cases. If you have put a
custom element with ghostpads, make sure you check when they are
added/removed and properly request/release the targets of those
ghostpads.

Edward

>
> Thanks,
> Tim
> ------------------------------------------------------------------------------
> Lotusphere 2011
> Register now for Lotusphere 2011 and learn how
> to connect the dots, take your collaborative environment
> to the next level, and enter the era of Social Business.
> http://p.sf.net/sfu/lotusphere-d2d
> _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------
Lotusphere 2011
Register now for Lotusphere 2011 and learn how
to connect the dots, take your collaborative environment
to the next level, and enter the era of Social Business.
http://p.sf.net/sfu/lotusphere-d2d
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Timothy Braun

Re: GnlComposition Signals

Thank you Edward. That makes sense.

I have a custom bin with ghosted pads in a gnlsource. I'm currently ghosting the pad upon creation and then adding the final internal link once decodebin2 tells me it's ready. On occasion, I run into a data flow error, but not consistently. I'm thinking it may be related to the order in which I'm adding pads. Is it better to ghost the bins pad after decodebin is connected? or does this order not matter?

Also, if I'm ghosting the pad, do I have to manually remove it once streaming of that source is complete? If so, what signal should I be listening for from gnlsource/gnlcomposition to know that its completed it's duties?

Thanks again.

Best,
Tim

On Tue, Dec 14, 2010 at 5:42 AM, Edward Hervey <[hidden email]> wrote:

On Mon, 2010-12-13 at 17:33 -0500, Timothy Braun wrote:
> Hey Everyone,
> So I have a gnlcomposition which, for the matter of depicting the
> situation, has 5 sources. 3 sources play for 30 seconds followed by
> the remaining 2 sources for another 30 seconds. This should produce a
> 1 minute video. The composition has a videomixer in a gnloperation to
> mix the sources for final output.
>
> I'm running into a situation where the pipeline gets stuck during
> the transition from 3 sources to 2 sources. It seems the video mixer
> is trying to send a seek event to all it's sink pads, which fails on
> the pad which no-longer has a source. During my tests, the
> gnlcomposition will happily create new pads, ie. changing the order so
> it's 2 sources followed by 3 sources works fine. But the opposite
> doesn't seem to be true, the gnlcomposition doesn't remove unused pads
> to it's operations.
>
> This brings me to the question of, is there a signal I'm not
> handling appropriately? If so, any pointers on which one I should be
> listening for?

If the element you put in a gnloperation has dynamic sink pads, the
gnloperation will call gst_element_request_pad() and
gst_element_release_request_pad() when needed.
videomixer alone should work fine in those cases. If you have put a
custom element with ghostpads, make sure you check when they are
added/removed and properly request/release the targets of those
ghostpads.

Edward

>
> Thanks,
> Tim
> ------------------------------------------------------------------------------
> Lotusphere 2011
> Register now for Lotusphere 2011 and learn how
> to connect the dots, take your collaborative environment
> to the next level, and enter the era of Social Business.
> http://p.sf.net/sfu/lotusphere-d2d
> _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------
Lotusphere 2011
Register now for Lotusphere 2011 and learn how
to connect the dots, take your collaborative environment
to the next level, and enter the era of Social Business.
http://p.sf.net/sfu/lotusphere-d2d
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Edward Hervey

Re: GnlComposition Signals

Administrator

On Tue, 2010-12-14 at 13:34 -0500, Timothy Braun wrote:
> Thank you Edward. That makes sense.
>
> I have a custom bin with ghosted pads in a gnlsource. I'm currently
> ghosting the pad upon creation and then adding the final internal link
> once decodebin2 tells me it's ready.

If you do that... gnloperation will think there's a static number of
sink pads.
gnloperation will dynamically request/release pads only if the element
it's controlling has a request sink pad template... meaning it'll work
with videomixer, but you'll have to do your own subclass of GstBin with
the proper pad templates to make it work (I'm assuming you want to put
some extra elements before videomixer).

> On occasion, I run into a data flow error, but not consistently.
> I'm thinking it may be related to the order in which I'm adding pads.
> Is it better to ghost the bins pad after decodebin is connected? or
> does this order not matter?

For sources it doesn't matter.

>
> Also, if I'm ghosting the pad, do I have to manually remove it once
> streaming of that source is complete? If so, what signal should I be
> listening for from gnlsource/gnlcomposition to know that its completed
> it's duties?

I'm assuming you mean for operations (doesn't matter for sources).

For operations with dynamic pads you want to make sure that:

* Your element (the one you put in gnloperation)

* Has a sink pad template of type GST_PAD_REQUEST
* you have overriden the GstElementClass::request_new_pad vmethod
In there you request a pad from videomixer, add your extra
elements, ghost the first element's sink pad, add that pad to yourself,
and return that pad
* you have overriden the GstElementClass::release_request_pad vmethod
In there you remove that pad from yourself, release the sink pad
you requested from videomixer, and remove all elements for that path

* You listen to the 'input-priority-changed' signal from gnloperation

This will notify you of changes in priority of incoming stream for
each of your ghost sink pad.
When you receive that, you change the 'associated' videomixer sinkpad
for that ghostpad of yours by setting the 'zorder' property of that
videomixer sinkpad to the priority you receive

g_object_set(thatstream->videomixersinkpad, "zorder", thenewpriority,
NULL);

This assumes you need to insert extra elements before/after your
videomixer in that operation (else you can just stick videomixer in a
gnloperation and all of that will be taken care of).

Hope this helps,

Edward

>
> Thanks again.
>
> Best,
> Tim
>
> On Tue, Dec 14, 2010 at 5:42 AM, Edward Hervey <[hidden email]>
> wrote:
>
> On Mon, 2010-12-13 at 17:33 -0500, Timothy Braun wrote:
> > Hey Everyone,
> > So I have a gnlcomposition which, for the matter of
> depicting the
> > situation, has 5 sources. 3 sources play for 30 seconds
> followed by
> > the remaining 2 sources for another 30 seconds. This should
> produce a
> > 1 minute video. The composition has a videomixer in a
> gnloperation to
> > mix the sources for final output.
> >
> > I'm running into a situation where the pipeline gets stuck
> during
> > the transition from 3 sources to 2 sources. It seems the
> video mixer
> > is trying to send a seek event to all it's sink pads, which
> fails on
> > the pad which no-longer has a source. During my tests, the
> > gnlcomposition will happily create new pads, ie. changing
> the order so
> > it's 2 sources followed by 3 sources works fine. But the
> opposite
> > doesn't seem to be true, the gnlcomposition doesn't remove
> unused pads
> > to it's operations.
> >
> > This brings me to the question of, is there a signal I'm
> not
> > handling appropriately? If so, any pointers on which one I
> should be
> > listening for?
>
>
> If the element you put in a gnloperation has dynamic sink
> pads, the
> gnloperation will call gst_element_request_pad() and
> gst_element_release_request_pad() when needed.
> videomixer alone should work fine in those cases. If you have
> put a
> custom element with ghostpads, make sure you check when they
> are
> added/removed and properly request/release the targets of
> those
> ghostpads.
>
> Edward
>
> >
> > Thanks,
> > Tim
> >
> ------------------------------------------------------------------------------
> > Lotusphere 2011
> > Register now for Lotusphere 2011 and learn how
> > to connect the dots, take your collaborative environment
> > to the next level, and enter the era of Social Business.
> > http://p.sf.net/sfu/lotusphere-d2d
> > _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>
>
>
> ------------------------------------------------------------------------------
> Lotusphere 2011
> Register now for Lotusphere 2011 and learn how
> to connect the dots, take your collaborative environment
> to the next level, and enter the era of Social Business.
> http://p.sf.net/sfu/lotusphere-d2d
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>
> ------------------------------------------------------------------------------
> Lotusphere 2011
> Register now for Lotusphere 2011 and learn how
> to connect the dots, take your collaborative environment
> to the next level, and enter the era of Social Business.
> http://p.sf.net/sfu/lotusphere-d2d
> _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Timothy Braun

Re: GnlComposition Signals

Thank You Edward. It was a great help. I will make an attempt at it in the coming days.

Much appreciated.

Tim

On Tue, Dec 14, 2010 at 4:04 PM, Edward Hervey <[hidden email]> wrote:

On Tue, 2010-12-14 at 13:34 -0500, Timothy Braun wrote:
> Thank you Edward. That makes sense.
>
> I have a custom bin with ghosted pads in a gnlsource. I'm currently
> ghosting the pad upon creation and then adding the final internal link
> once decodebin2 tells me it's ready.

If you do that... gnloperation will think there's a static number of
sink pads.
gnloperation will dynamically request/release pads only if the element
it's controlling has a request sink pad template... meaning it'll work
with videomixer, but you'll have to do your own subclass of GstBin with
the proper pad templates to make it work (I'm assuming you want to put
some extra elements before videomixer).

> On occasion, I run into a data flow error, but not consistently.
> I'm thinking it may be related to the order in which I'm adding pads.
> Is it better to ghost the bins pad after decodebin is connected? or
> does this order not matter?

For sources it doesn't matter.

>
> Also, if I'm ghosting the pad, do I have to manually remove it once
> streaming of that source is complete? If so, what signal should I be
> listening for from gnlsource/gnlcomposition to know that its completed
> it's duties?

I'm assuming you mean for operations (doesn't matter for sources).

For operations with dynamic pads you want to make sure that:

* Your element (the one you put in gnloperation)

* Has a sink pad template of type GST_PAD_REQUEST
* you have overriden the GstElementClass::request_new_pad vmethod
In there you request a pad from videomixer, add your extra
elements, ghost the first element's sink pad, add that pad to yourself,
and return that pad
* you have overriden the GstElementClass::release_request_pad vmethod
In there you remove that pad from yourself, release the sink pad
you requested from videomixer, and remove all elements for that path

* You listen to the 'input-priority-changed' signal from gnloperation

This will notify you of changes in priority of incoming stream for
each of your ghost sink pad.
When you receive that, you change the 'associated' videomixer sinkpad
for that ghostpad of yours by setting the 'zorder' property of that
videomixer sinkpad to the priority you receive

g_object_set(thatstream->videomixersinkpad, "zorder", thenewpriority,
NULL);

This assumes you need to insert extra elements before/after your
videomixer in that operation (else you can just stick videomixer in a
gnloperation and all of that will be taken care of).

Hope this helps,

Edward

>
> Thanks again.
>
> Best,
> Tim
>
> On Tue, Dec 14, 2010 at 5:42 AM, Edward Hervey <[hidden email]>
> wrote:
>
> On Mon, 2010-12-13 at 17:33 -0500, Timothy Braun wrote:
> > Hey Everyone,
> > So I have a gnlcomposition which, for the matter of
> depicting the
> > situation, has 5 sources. 3 sources play for 30 seconds
> followed by
> > the remaining 2 sources for another 30 seconds. This should
> produce a
> > 1 minute video. The composition has a videomixer in a
> gnloperation to
> > mix the sources for final output.
> >
> > I'm running into a situation where the pipeline gets stuck
> during
> > the transition from 3 sources to 2 sources. It seems the
> video mixer
> > is trying to send a seek event to all it's sink pads, which
> fails on
> > the pad which no-longer has a source. During my tests, the
> > gnlcomposition will happily create new pads, ie. changing
> the order so
> > it's 2 sources followed by 3 sources works fine. But the
> opposite
> > doesn't seem to be true, the gnlcomposition doesn't remove
> unused pads
> > to it's operations.
> >
> > This brings me to the question of, is there a signal I'm
> not
> > handling appropriately? If so, any pointers on which one I
> should be
> > listening for?
>
>
> If the element you put in a gnloperation has dynamic sink
> pads, the
> gnloperation will call gst_element_request_pad() and
> gst_element_release_request_pad() when needed.
> videomixer alone should work fine in those cases. If you have
> put a
> custom element with ghostpads, make sure you check when they
> are
> added/removed and properly request/release the targets of
> those
> ghostpads.
>
> Edward
>
> >
> > Thanks,
> > Tim
> >
> ------------------------------------------------------------------------------
> > Lotusphere 2011
> > Register now for Lotusphere 2011 and learn how
> > to connect the dots, take your collaborative environment
> > to the next level, and enter the era of Social Business.
> > http://p.sf.net/sfu/lotusphere-d2d
> > _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>
>
>
> ------------------------------------------------------------------------------
> Lotusphere 2011
> Register now for Lotusphere 2011 and learn how
> to connect the dots, take your collaborative environment
> to the next level, and enter the era of Social Business.
> http://p.sf.net/sfu/lotusphere-d2d
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>
> ------------------------------------------------------------------------------
> Lotusphere 2011
> Register now for Lotusphere 2011 and learn how
> to connect the dots, take your collaborative environment
> to the next level, and enter the era of Social Business.
> http://p.sf.net/sfu/lotusphere-d2d
> _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------
Lotusphere 2011
Register now for Lotusphere 2011 and learn how
to connect the dots, take your collaborative environment
to the next level, and enter the era of Social Business.
http://p.sf.net/sfu/lotusphere-d2d
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

amartin

Re: GnlComposition Signals

In reply to this post by Edward Hervey

Dear Edward and Tim,

I have exactly the same problem.

In my case I have a gnloperation that places each content through the xpos, ypos and zorder attributes of the sinkpad of the videomixer.

gnloperation = gst_element_factory_make ("gnloperation", "general-video-composition");
videomixer = gst_element_factory_make ("videomixer", "general-video-adder");

gst_bin_add (GST_BIN (gnloperation), videomixer);
g_object_set(G_OBJECT (gnloperation),
"expandable", TRUE,
"priority", 0,
NULL);
gst_bin_add (GST_BIN (gnlcomposition), gnloperation);

g_signal_connect(gnloperation, "input-priority-changed", G_CALLBACK (onPriorityChange), data);

In the onPriorityChange function I assign to each video/image content the parameters mentioned before.

So, as far as i understood it is supposed that the videomixer should manage properly the request_new_pad and release_request_pad calls, and no redefinition would be needed.

However, I have the same problem that Tim, depending on the number of performed contents the Seek Error appears or not:
ERROR gnlcomposition gnlcomposition.c:1724:no_more_pads_object_cb:<video-composition> Sending seek event failed!

I mean if 3 contents are played and afterwards only 1 is performed the error appears, but if i remove one by one it works:
Content A start 0 dur 10
Content B start 0 dur 10
Content C start 0 dur 10
Content D start 10 dur 10
-> Seek Error

Content A start 0 dur 10
Content B start 0 dur 9
Content C start 0 dur 8
Content D start 10 dur 10
-> Ok!

So I guess if I don't need to insert any extra element between the gnlfilesource and the videomixer how could I fix it?

I will try to implement the solution that Edward described.

Thank you in advance.

Best Regards,

Angel

amartin

Re: GnlComposition Signals

This post was updated on .

Dear all,

After trying to understand the problem from the code of videomixer, gnlcomposition and gnloperation, I think that the problem could be the "gnloperation" element.

In the code of of the function "synchronize_sinks" there is a call to:
/* Remove pad */
/* FIXME, which one do we remove ? :) */
remove_sink_pad (operation, NULL);

I think that this is the tricky point, because this function seems not be prepared to remove several sinks at once.

So a loop inside "synchronize_sinks" could fix it.
/* Find unlinked sinkpads */
GstPad * sinkpad;
while ((sinkpad = get_unlinked_sink_ghost_pad (operation)) != NULL) {
remove_sink_pad (operation, sinkpad);
}

According to the GST_DEBUG log I think that the function "synchronize_sinks" is not called for each multimedia content removed by the composition, but once, so this code could fix it.

However, I don't feel comfortable with solutions that require "standard" gstreamer plugins changes, I'd prefer a solution from the application that improves the source code's life-cycle ... Any idea?

Best,

Angel