2 questions: threading and faad

[franchan]

Hello,

I'm using a playbin base pipeline.

Initially I thought gstreamer was running in its own thread context.
But after setting the state to GST_STATE_PLAYING from the main thread context,
I get the impression that I only return from the call,
when the typefinding procedure has finished.

My source takes it content from the internet (kind of progressive download).
And I inform this source plug-in each time more data is available from
the main thread.

The source plug-in presents itself as a pull source because
I support seeking in the downloaded part.
And the qtdemux requires its source to operate pull based.
(don't know if this pull is relevant for my question here).

When the source is asked for data but the plug-in sees there's not enough,
it blocks in a sleep (or returns an error and
certain type finding that is not relevant fails e.g. apetag).

Because this all happens in the main thread content,
I cannot inform the source plug-in that new data did arrive.
And so type finding halts waiting for data (that is there actually).

Is my conclusion right?
If so, why does the typefinding phase not use
the normal playbin buffering mechanism
and informs when to pause/play the pipeline for cause of buffering.

I likely have to make sure the PLAY_STATE is only set
after enough data is downloaded for doing a successful type finding?
Or do you know another work around?

My second question is regarding faad.
I tweak the configure script of the bad plug-in to accept the faad-library
installed on my platform and it seems to work well.

Yet I get a feeling faad is not welcomed in gstreamer.
What is the reason for this?
Some people prefer to make a new AAC decoder and a plug-in?
Is it impossible to fix faad?

Thanks,
franchan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[michael smith-6-3]

On Wed, Sep 10, 2008 at 12:50 AM, franchan <[hidden email]> wrote:
> Hello,
>
> I'm using a playbin base pipeline.
>
> Initially I thought gstreamer was running in its own thread context.
> But after setting the state to GST_STATE_PLAYING from the main thread context,
> I get the impression that I only return from the call,
> when the typefinding procedure has finished.

This is incorrect, typefinding is never done from the main thread.
It's possible, of course, for buggy plugins to make setting the
pipeline state block - some of the http sources have bugs like that.
Since you're writing your own source, I'd guess the problem is there.


>
> My source takes it content from the internet (kind of progressive download).
> And I inform this source plug-in each time more data is available from
> the main thread.
>
> The source plug-in presents itself as a pull source because
> I support seeking in the downloaded part.
> And the qtdemux requires its source to operate pull based.
> (don't know if this pull is relevant for my question here).

That's probably not a good idea. qtdemux works fine in push mode, and
you can support seeking in push mode if you want.

>
> When the source is asked for data but the plug-in sees there's not enough,
> it blocks in a sleep (or returns an error and
> certain type finding that is not relevant fails e.g. apetag).
>
> Because this all happens in the main thread content,
> I cannot inform the source plug-in that new data did arrive.
> And so type finding halts waiting for data (that is there actually).

This doesn't happen in the main thread, so you've diagnosed the
problem incorrectly.

>
> Is my conclusion right?
> If so, why does the typefinding phase not use
> the normal playbin buffering mechanism
> and informs when to pause/play the pipeline for cause of buffering.
>
> I likely have to make sure the PLAY_STATE is only set
> after enough data is downloaded for doing a successful type finding?
> Or do you know another work around?
>
> My second question is regarding faad.
> I tweak the configure script of the bad plug-in to accept the faad-library
> installed on my platform and it seems to work well.
>
> Yet I get a feeling faad is not welcomed in gstreamer.
> What is the reason for this?
> Some people prefer to make a new AAC decoder and a plug-in?
> Is it impossible to fix faad?
>

The FAAD library is mostly unmaintained, and is generally not very
nice to use. If someone were to step up and maintain it properly, and
the gstreamer plugin, and add appropriate tests and documentation, we
would be happy to move it to gst-plugins-ugly - all plugins have the
same criteria for being moved out of -bad.

Mike

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Wim Taymans]

On Wed, 2008-09-10 at 08:28 -0700, Michael Smith wrote:

> On Wed, Sep 10, 2008 at 12:50 AM, franchan <[hidden email]> wrote:
> > Hello,
> >
> > I'm using a playbin base pipeline.
> >
> > Initially I thought gstreamer was running in its own thread context.
> > But after setting the state to GST_STATE_PLAYING from the main thread context,
> > I get the impression that I only return from the call,
> > when the typefinding procedure has finished.
>
> This is incorrect, typefinding is never done from the main thread.
> It's possible, of course, for buggy plugins to make setting the
> pipeline state block - some of the http sources have bugs like that.
> Since you're writing your own source, I'd guess the problem is there.
>

The typefind element does typefinding in the state-change to PAUSED.
This is arguably the wrong thing to do but it's currently done like that
because we otherwise would need a scheduling mode switch from the
streaming thread. I'm not quite sure how that would work...

Wim

>
> >
> > My source takes it content from the internet (kind of progressive download).
> > And I inform this source plug-in each time more data is available from
> > the main thread.
> >
> > The source plug-in presents itself as a pull source because
> > I support seeking in the downloaded part.
> > And the qtdemux requires its source to operate pull based.
> > (don't know if this pull is relevant for my question here).
>
> That's probably not a good idea. qtdemux works fine in push mode, and
> you can support seeking in push mode if you want.
>
> >
> > When the source is asked for data but the plug-in sees there's not enough,
> > it blocks in a sleep (or returns an error and
> > certain type finding that is not relevant fails e.g. apetag).
> >
> > Because this all happens in the main thread content,
> > I cannot inform the source plug-in that new data did arrive.
> > And so type finding halts waiting for data (that is there actually).
>
> This doesn't happen in the main thread, so you've diagnosed the
> problem incorrectly.
>
> >
> > Is my conclusion right?
> > If so, why does the typefinding phase not use
> > the normal playbin buffering mechanism
> > and informs when to pause/play the pipeline for cause of buffering.
> >
> > I likely have to make sure the PLAY_STATE is only set
> > after enough data is downloaded for doing a successful type finding?
> > Or do you know another work around?
> >
> > My second question is regarding faad.
> > I tweak the configure script of the bad plug-in to accept the faad-library
> > installed on my platform and it seems to work well.
> >
> > Yet I get a feeling faad is not welcomed in gstreamer.
> > What is the reason for this?
> > Some people prefer to make a new AAC decoder and a plug-in?
> > Is it impossible to fix faad?
> >
>
> The FAAD library is mostly unmaintained, and is generally not very
> nice to use. If someone were to step up and maintain it properly, and
> the gstreamer plugin, and add appropriate tests and documentation, we
> would be happy to move it to gst-plugins-ugly - all plugins have the
> same criteria for being moved out of -bad.
>
> Mike
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[michael smith-6-3]

On Wed, Sep 10, 2008 at 8:39 AM, Wim Taymans <[hidden email]> wrote:

> On Wed, 2008-09-10 at 08:28 -0700, Michael Smith wrote:
>> On Wed, Sep 10, 2008 at 12:50 AM, franchan <[hidden email]> wrote:
>> > Hello,
>> >
>> > I'm using a playbin base pipeline.
>> >
>> > Initially I thought gstreamer was running in its own thread context.
>> > But after setting the state to GST_STATE_PLAYING from the main thread context,
>> > I get the impression that I only return from the call,
>> > when the typefinding procedure has finished.
>>
>> This is incorrect, typefinding is never done from the main thread.
>> It's possible, of course, for buggy plugins to make setting the
>> pipeline state block - some of the http sources have bugs like that.
>> Since you're writing your own source, I'd guess the problem is there.
>>
>
> The typefind element does typefinding in the state-change to PAUSED.
> This is arguably the wrong thing to do but it's currently done like that
> because we otherwise would need a scheduling mode switch from the
> streaming thread. I'm not quite sure how that would work...
>

Ah, you're right. Sorry about that, franchan. Wim: we should think
about a fix for that, though probably for 0.11?

So: yes, apparently we do typefinding from the main thread, if we're
in pull mode. That's unfortunate. However, it doesn't work this way in
push mode. Since I recommended using push mode _anyway_, that's not
too bad - you'll just need to change your plugin to work in push mode.

There are a number of other network sources that operate in push mode
and support streaming (e.g. the http sources), those might be useful
examples for you.

Mike

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[franchan]

Hello,

Thanks for sharing your insights!
>> The typefind element does typefinding in the state-change to PAUSED.
>> This is arguably the wrong thing to do but it's currently done like that
>> because we otherwise would need a scheduling mode switch from the
>> streaming thread. I'm not quite sure how that would work...

Well, I was inaccurate about that, the problem popped up in the paused
state indeed.
But by then the email was already send already.

I'm not sure why it would be wrong doing it in the PAUSE state.
Although I don't work long with gstreamer,
it simple state model makes sense to me:
READY state: don't make contact with server's yet but have allocated
as much resources as necessary.
PAUSE: ready for playing as soon as state changes to PLAYING.
So by going from READY to the PAUSE state,
it should start connecting to the network, get initial content to do
typefinding.

It would be nice if also in PAUSE transition,
the playbin buffering mechanism would be in place already,
so that the whole gmain_loop does not block in a "read()".

> Ah, you're right. Sorry about that, franchan. Wim: we should think
> about a fix for that, though probably for 0.11?
>
> So: yes, apparently we do typefinding from the main thread, if we're
> in pull mode. That's unfortunate. However, it doesn't work this way in
> push mode. Since I recommended using push mode _anyway_, that's not
> too bad - you'll just need to change your plugin to work in push mode.

Perhaps I should use another demuxer
(now this was auto selected by playbin for the h264/AAC content)
or use some API I did not find yet?
At first I started with a progressive download plug-in for push operation.

Then I found qtdemux refused to seek with this source plug-in.
I made these conclusions back then when reading and tracing qtdemux.c
(I write just what I remember from last weeks so I might miss)

qtdemux_sink_activate_pull() sets its data member demux->pullbased=true.
gst_qtdemux_handle_src_event() checks this data member
in case of a GST_EVENT_SEEK event.
And it concludes with GST_DEBUG_OBJECT (qtdemux, "cannot seek in
streaming mode")
if pullbased is false.

qtdemux_sink_activate_pull() resets the pullbased flag when
the qtdemux_sink_activate() concludes through the gst_pad_check_pull_range()
that the source plug-in was push only because it missed a
_check_range_get() function,
typical for pull plug-ins ... so I thought ...

So I decided to rewrite my plug-in at that time to not use a push model.
I was surprised to see this demux code because gstreamer's documentation,
seems to favor push operation, as you also tell here ...

> There are a number of other network sources that operate in push mode
> and support streaming (e.g. the http sources), those might be useful
> examples for you.
Well, I looked into the soup/neon based http source plug-ins,
but these are complicated :-)
Now the pull based plug-in is very simple
(note that the HTTP downloading work is outside this plug-in
and based on legacy code).

For now the easiest for me is to have a delayed "PAUSE/PLAY".
Once the legacy code tells me that it has a fair amount of buffered data,
I put the pipeline into the PAUSE/PLAY state and
typefinding phase can likely finish correctly.
(I'm just afraid, how much data do I need exactly.
But that's a completely different problem)
For now about 300K seems to be enough to get the pipeline rolling ...
(moov bag in front of the mp4 file of course)

Thanks,
franchan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel