blocked glib idle/timeout

[Dan Taylor-2]

My simplest test pipeline has dvbsrc, mpegtsdemux (tried flutsdemux,
too), and
two filesinks (location is /dev/null).  There's a new pad callback to
connect
the demuxer to the filesinks, which works when the specified program
number
is in the stream.

When I use a program number that exists in the stream, the functions
called with g_idle_add_full() and g_timeout_add_full(), and the shorter
forms, are called.  When the program number doesn't exist in the stream,
the timeout is never called, and the idle function stops within a
second.
The application is only consuming 3% of a CPU, though, so it isn't busy-
waiting in the traditionally bad sense.

dvbsrc does not seem to care to where it sources data, and the only
difference
for the demuxer(s) is the the program number exists/does not exist.

Anyone got an idea where the demuxers could be blocking glib idle and
timeout?

I'm working down at the sink end of the stream, so I don't know the
demuxers
very well, but I will look deeper into them if someone knows where to
start.


------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Edward Hervey]

On Tue, 2009-03-24 at 17:18 -0700, Dan Taylor wrote:

> My simplest test pipeline has dvbsrc, mpegtsdemux (tried flutsdemux,
> too), and
> two filesinks (location is /dev/null).  There's a new pad callback to
> connect
> the demuxer to the filesinks, which works when the specified program
> number
> is in the stream.
>
> When I use a program number that exists in the stream, the functions
> called with g_idle_add_full() and g_timeout_add_full(), and the shorter
> forms, are called.  When the program number doesn't exist in the stream,
> the timeout is never called, and the idle function stops within a
> second.
> The application is only consuming 3% of a CPU, though, so it isn't busy-
> waiting in the traditionally bad sense.
>
> dvbsrc does not seem to care to where it sources data, and the only
> difference
> for the demuxer(s) is the the program number exists/does not exist.
>
> Anyone got an idea where the demuxers could be blocking glib idle and
> timeout?
>
> I'm working down at the sink end of the stream, so I don't know the
> demuxers
> very well, but I will look deeper into them if someone knows where to
> start.

  The first question is ... do you have a GMainLoop running ?
g_idle_add* and g_timeout_add* will only work if you have a main loop
running.
  The other question is ... why do you need to do that from the main
thread ? Can't you connect your pads directly from the 'pad_added'
callback ?

   Edward

>
>
> ------------------------------------------------------------------------------
> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> easily build your RIAs with Flex Builder, the Eclipse(TM)based development
> software that enables intelligent coding and step-through debugging.
> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Dan Taylor-6]

Same guy, different account ...

Edward Hervey wrote:

> On Tue, 2009-03-24 at 17:18 -0700, Dan Taylor wrote:
>  
>> My simplest test pipeline has dvbsrc, mpegtsdemux (tried flutsdemux,
>> too), and
>> two filesinks (location is /dev/null).  There's a new pad callback to
>> connect
>> the demuxer to the filesinks, which works when the specified program
>> number
>> is in the stream.
>>
>> When I use a program number that exists in the stream, the functions
>> called with g_idle_add_full() and g_timeout_add_full(), and the shorter
>> forms, are called.  When the program number doesn't exist in the stream,
>> the timeout is never called, and the idle function stops within a
>> second.
>> The application is only consuming 3% of a CPU, though, so it isn't busy-
>> waiting in the traditionally bad sense.
>>
>> dvbsrc does not seem to care to where it sources data, and the only
>> difference
>> for the demuxer(s) is the the program number exists/does not exist.
>>
>> Anyone got an idea where the demuxers could be blocking glib idle and
>> timeout?
>>
>> I'm working down at the sink end of the stream, so I don't know the
>> demuxers
>> very well, but I will look deeper into them if someone knows where to
>> start.
>>    
>
>   The first question is ... do you have a GMainLoop running ?
> g_idle_add* and g_timeout_add* will only work if you have a main loop
> running.
>  

Yes, there is a GMainLoop running.  This is a very cut-down piece of a
much larger application,
trying to isolate the symptom.  As I said, when the program stream
exists within the transport
stream, both callbacks ARE run.  The ONLY difference in behavior is
whether, or not, the program
stream is present in the transport stream.

>   The other question is ... why do you need to do that from the main
> thread ? Can't you connect your pads directly from the 'pad_added'
> callback ?
>  
I do, in fact use the pad_added to connect the sinks downstream of the
demultiplexer.  Those
callbacks only get called IFF the demultiplexer finds the the program
stream number specified,
otherwise it cannot construct the "video_%04x" and "audio_%04x"
"sometimes" pads, AFAIK.

In the "full" application, there are pad_added and pad_removed for the
demultiplexer (first paragraph
of the question) as well as dynamic connections for the respective video
and audio streams connections
through decodebin through some additional processing (scaling and
volume) to the fbdev and alsa
sinks most downstream.  The behavior is the same for both the cut-down
and full applications.

Overall, the application runs as a daemon in an embedded system.  When
commanded, it creates a
"TV" for digital (ATSC or DVB-T) or analog TV, or displays composite
video from an external player.
If  a "TV" is already playing, it changes  "stations", which is trivial
for analog TV, but requires the
dynamic behavior for digital, since the program and elementary stream  
numbers usually change, and,
sometimes, the encoding of the video and audio (hence, the decodebin in
both downstream links from
the demultiplexer), and the application handles the transition between
analog and digital pipelines.

It is not, of course, acceptable for the daemon to "hang", so I put in,
initially, a timeout_* to handle the
case where (as on our local cable) the station lineup is occasionally
shuffled, or where we do not have
an appropriate decoder for the content.  I discovered that the timeout
was called reliably when the
stream completed, but not when the program stream is not present in the
transport stream.  I cut back the
application until I isolated it to the few pieces I have now (dvbsrc,
although filesrc works the same,
either of the two demultiplexers, and filesink).  I added the idle_*, to
see if it had any different behavior.
If the timeout_* worked, the idle_* would not be there.  The idle_*
function is called repeatedly (100s
of times in the first second, then stops if the program stream is not
present, otherwise it runs until a
known elapsed time and exits FALSE to stop itself, essentially trying to
emulate a timeout_*.

As can be seen from my other questions, I'm trying to fill out the
functionality of fbdevsink (caps
negotiation, a bounding box and more-flexible centering/cropping
properties) to make it usable in
this embedded system, but I also need robustness in the daemon, and
oddities such as timeout_*
not even running is happening in parts of the gstreamer system that I
haven't looked into.

Thanks,

Dan

>    Edward
>
>  
>> ------------------------------------------------------------------------------
>> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
>> powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
>> easily build your RIAs with Flex Builder, the Eclipse(TM)based development
>> software that enables intelligent coding and step-through debugging.
>> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
>> _______________________________________________
>> gstreamer-devel mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>>    
>
>
> ------------------------------------------------------------------------------
> Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
> powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
> easily build your RIAs with Flex Builder, the Eclipse(TM)based development
> software that enables intelligent coding and step-through debugging.
> Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>
>  

------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Arnout Vandecappelle]

On Wednesday 25 March 2009 01:18:10 Dan Taylor wrote:

> My simplest test pipeline has dvbsrc, mpegtsdemux (tried flutsdemux,
> too), and
> two filesinks (location is /dev/null).  There's a new pad callback to
> connect
> the demuxer to the filesinks, which works when the specified program
> number
> is in the stream.
>
> When I use a program number that exists in the stream, the functions
> called with g_idle_add_full() and g_timeout_add_full(), and the shorter
> forms, are called.  When the program number doesn't exist in the stream,
> the timeout is never called, and the idle function stops within a
> second.

 This may be a very stupid remark, but do you use queues in your pipeline?  If
not, the two sinks will deadlock the main thread...

 Regards,
 Arnout

--
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 [phone]
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  D206 D44B 5155 DF98 550D  3F2A 2213 88AA A1C7 C933

------------------------------------------------------------------------------
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Dan Taylor-6]

Arnout Vandecappelle wrote:

> On Wednesday 25 March 2009 01:18:10 Dan Taylor wrote:
>  
>> My simplest test pipeline has dvbsrc, mpegtsdemux (tried flutsdemux,
>> too), and
>> two filesinks (location is /dev/null).  There's a new pad callback to
>> connect
>> the demuxer to the filesinks, which works when the specified program
>> number
>> is in the stream.
>>
>> When I use a program number that exists in the stream, the functions
>> called with g_idle_add_full() and g_timeout_add_full(), and the shorter
>> forms, are called.  When the program number doesn't exist in the stream,
>> the timeout is never called, and the idle function stops within a
>> second.
>>    
>
>  This may be a very stupid remark, but do you use queues in your pipeline?  If
> not, the two sinks will deadlock the main thread...
>
>  Regards,
>  Arnout
>
>  

Not  a stupid question.

I didn't mention them, since they don't look like the source of the
problem, but there are queues between the source (dvbsrc or filesrc) and
the demultiplexer and between the sink pads for the demuxed audio and
video elementary streams and the filesinks.  In the real application,
the queues from the sink pads feed decodebin in each downstream branch
and there's another queue between each of those and the final processing
steps and sinks.  I have tried fewer and more queues, but it doesn't
change the symptom.  In the queues, "max-size-buffers" and
"max-size-bytes" are both set to 0.

I may be overstressing this point, but the ONLY difference between a
trial run that works, including the callbacks being run, and one in
which the timeout is NEVER called (and the idle stops after a second) is
that the program number given to the demultiplexer  exists in the stream
or it doesn't.  It is even the same transport stream in the case of filesrc.

------------------------------------------------------------------------------
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Arnout Vandecappelle]

On Thursday 26 March 2009 09:53:59 Dan Taylor wrote:
> I may be overstressing this point, but the ONLY difference between a
> trial run that works, including the callbacks being run, and one in
> which the timeout is NEVER called (and the idle stops after a second) is
> that the program number given to the demultiplexer  exists in the stream
> or it doesn't.  It is even the same transport stream in the case of
> filesrc.

 When g_idle_... isn't called, this can only mean that the mainloop is blocked
somewhere.  Maybe you can check with a debugger where it is blocked.  You
could also check if any output produces, but I guess that when the program
number doesn't exist there is no output anyway, right?

 Regards,
 Arnout

--
Arnout Vandecappelle                               arnout at mind be
Senior Embedded Software Architect                 [phone]
Essensium/Mind                                     http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium                BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  D206 D44B 5155 DF98 550D  3F2A 2213 88AA A1C7 C933

------------------------------------------------------------------------------
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel