sink element with multiple sink pads

[Gregory McGarry]

I'm working on a plugin which wraps a library.  The library requires roughly synchronised audio and/or video frames.

Since GstBaseSink only has one sink pad, I need to develop my own sink class.

What is the best way to handle this problem?  Here are the approaches I have tried:

1) custom sink element with a collection of request sink pads
- seems to deadlock (thread waits for other pads to receive data?)

- add queues to the pipeline - is the pipeline now in pull mode?

- must support push and pull mode, although there is no time sensitivity in application?


2) custom sink element that provides explicit _chain() handlers on the pads
- lots of locking problems

- still have problem blocking streams until all stream info is available to initialise the library


3) element derived from GstBin which contains audio and video sink elements
- how to create internal elements without registering with factory?

- difficulty aggregating streams info to initialise library

- overly complicated


Does a sink element exist which has multiple sink pads?  I couldn't find one.

Thanks for any advice.



      Start your day with Yahoo!7 and win a Sony Bravia TV. Enter now http://au.docs.yahoo.com/homepageset/?p1=other&p2=au&p3=tagline

------------------------------------------------------------------------------
SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
The future of the web can't happen without you.  Join us at MIX09 to help
pave the way to the Next Web now. Learn more and register at
http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Edward Hervey]

On Mon, 2008-12-08 at 22:23 -0800, Gregory McGarry wrote:
> I'm working on a plugin which wraps a library.  The library requires
>  roughly synchronised audio and/or video frames.

  Using GstBaseSink is definitely a must-have to have synchronized
rendering (across several sinks).

>
> Since GstBaseSink only has one sink pad, I need to develop my own sink class.
>
> What is the best way to handle this problem?  Here are the approaches I have tried:

> 3) element derived from GstBin which contains audio and video sink elements

  I'd say this one is definitely the fastest to implement, and you won't
have to worry about all the 'smartness' (locking, threading,
synchronization, message emission, seek handling, ...) that GstBaseSink
provides if you implement your individual-stream sinks with it.

> - how to create internal elements without registering with factory?

  Just... create them and use them, you're the bin, you control what you
put in it. Provided the objects you're adding to the pipeline are
GstElement subclasses you can use them in the pipeline. You don't need
to register elements in order to use them in a pipeline.
gst_bin_add(g_object_new(MY_TYPE_ELEMENT), ...);

>
> - difficulty aggregating streams info to initialise library

  Since your individual elements are custom, you can access whatever
methods/functions/properties those elements provide from your bin, and
vice-versa. You can then splitup the single-stream-specific library
calls in the individual sinks, and the global library calls in your bin.
  If you mean that you're not sure when there will be no more extra
streams connected... that problem will be the same whatever technique
you use.

>
> - overly complicated

  Definitely the simplest if you don't know gstreamer inside out.

>
>
> Does a sink element exist which has multiple sink pads?  I couldn't find one.

  Not to my knowledge.

>
> Thanks for any advice.
>
>
>
>       Start your day with Yahoo!7 and win a Sony Bravia TV. Enter now http://au.docs.yahoo.com/homepageset/?p1=other&p2=au&p3=tagline
>
> ------------------------------------------------------------------------------
> SF.Net email is Sponsored by MIX09, March 18-20, 2009 in Las Vegas, Nevada.
> The future of the web can't happen without you.  Join us at MIX09 to help
> pave the way to the Next Web now. Learn more and register at
> http://ad.doubleclick.net/clk;208669438;13503038;i?http://2009.visitmix.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Edward Hervey]

On Mon, 2009-02-09 at 23:41 -0800, Gregory McGarry wrote:

> Hi Edward.
>
> Edward Hervey wrote:
>
>
> > On Mon, 2008-12-08 at 22:23 -0800, Gregory McGarry wrote:
> > > I'm working on a plugin which wraps a library.  The library requires
> > >  roughly synchronised audio and/or video frames.
> >
> >   Using GstBaseSink is definitely a must-have to have synchronized
> > rendering (across several sinks).
> >
> > > 3) element derived from GstBin which contains audio and video sink elements
> >
> >   I'd say this one is definitely the fastest to implement, and you won't
> > have to worry about all the 'smartness' (locking, threading,
> > synchronization, message emission, seek handling, ...) that GstBaseSink
> > provides if you implement your individual-stream sinks with it.
>
> Thanks for your recommendations.  Indeed it was quite simple to get audio/video working with GstBin.
>
> I created an audio sink element and a video sink element and ghosted the pins to GstBin.  It works fine when both the audio and video pads are linked, but it doesn't work if only one of the pads is linked.  In this case, the pipeline wedges.  I think GstBin may be waiting for messages from the unlinked sink element.
>
> I then tried to change the pads in the bin to request pads and create instances of the video and audio elements when a request arrives.  This doesn't work reliably:
>
> (bin:10041): GStreamer-WARNING **: adding flushing pad 'video' to running element 'mybin0'
> (bin:9997): GStreamer-WARNING **: adding flushing pad 'audio' to running element 'mybin0'
>
> Do you have any recommendations on when to add the elements to the bin and how to ghost the pads correctly?

  Using request pads (and only creating/adding the elements at that
time) is the correct way to do it.
  Since you're adding the pads in PAUSED/PLAYING, you will have to
activate them before adding them to yourself.
  gst_pad_set_active(newpad, TRUE);

  Finally, to make sure your elements are in the right state, once
you've added them and just before returning the requested pad, you need
to call 'gst_element_sync_state_with_parent(newelement)' on each of the
newly added elements. This will ensure they're in the correct state.

    Edward

>
> Thanks for you advice.
>
>
>
>       Make Yahoo!7 your homepage and win a trip to the Quiksilver Pro. Find out more


------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Gregory McGarry]

 Edward Hervey wrote:

> > I created an audio sink element and a video sink element and ghosted the pins
> to GstBin.  It works fine when both the audio and video pads are linked, but it
> doesn't work if only one of the pads is linked.  In this case, the pipeline
> wedges.  I think GstBin may be waiting for messages from the unlinked sink
> element.
> >
> > I then tried to change the pads in the bin to request pads and create
> instances of the video and audio elements when a request arrives.  This doesn't
> work reliably:
> >
> > (bin:10041): GStreamer-WARNING **: adding flushing pad 'video' to running
> element 'mybin0'
> > (bin:9997): GStreamer-WARNING **: adding flushing pad 'audio' to running
> element 'mybin0'
> >
> > Do you have any recommendations on when to add the elements to the bin and how
> to ghost the pads correctly?
>
>   Using request pads (and only creating/adding the elements at that
> time) is the correct way to do it.
>   Since you're adding the pads in PAUSED/PLAYING, you will have to
> activate them before adding them to yourself.
>   gst_pad_set_active(newpad, TRUE);
>
>   Finally, to make sure your elements are in the right state, once
> you've added them and just before returning the requested pad, you need
> to call 'gst_element_sync_state_with_parent(newelement)' on each of the
> newly added elements. This will ensure they're in the correct state.

Thanks for tip.

My bin now contains two sink elements; one for video and one for audio.  I funnel all data from the video and audio sinks into my library.

When I open my library I need to know if both audio and video will be available.  I defer opening the library until the first sink render() is invoked.  Hopefully set_caps() has been invoked on each sink, so I know if the library should be opened for audio and video.

Well, this isn't always the case.  It seems that flvdemux and avidemux will start decoding one of the streams well before the other has invoked request_new_pad().  The behaviour isn't deterministic, since it sometimes works, other times it doesn't.

When is there an opportunity to synchronise all sinks inside a bin?

Thanks.



      Stay connected to the people that matter most with a smarter inbox. Take a look http://au.docs.yahoo.com/mail/smarterinbox

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel