Gnonlin as a constant source of audio

[Mike MacHenry]

I am writing a program using Gnonlin, which I feel might be a using
the plugin in a way that it was not quite designed for. I am getting
non-deterministic errors which are very difficult to pin down and I
would like to just explain my design to the mailing list now. I am
hoping to get some feedback on whether this design is too far from
what gnonlin is supposed to do and if it's a wonder that it works. Or
if maybe it's perfectly normal and the errors are well known.

The design:
Gnonlin seems to be designed to have several gnlfilesrc objects added
to a gnlcomposition along with gnloperations and such and then played.
There is a distinct separation between composition and playing. What I
am doing is creating a composition with one file source in it and
listening to the bus for a message telling me that that file has
started playing. When it has, I add another file source to the
composition after it. Once I get the signal that that next file is
playing I queue up a third file and remove the first file source from
the composition and stop it so that it can be cleaned up in memory.
This goes on infinitely (in theory).

The Problem:
Occasionally, and eventually, it seems, for all streams I create, the
composition gets confused. As soon as I queue up a file source for
file number N with in the call back for file number N-1, I immediately
receive a message that file N is playing, which causes my code to
queue up a file source for N+1 and then N+1's message is also
immediately sent and so forth until a ton of filesources are added. I
immediately get an overload of file sources in the program and it
crashes. It also seems to increase with load on the system. When 10
clients play at once, they each run away at a faster rate.

I can't figure out how the composition could get so out of sink. I
suspect it has to do with its internal notion of the time line.
Possibly I am confusing it with the way I add things. I welcome any
ideas at all. Including questioning whether or not using gnonlin is
right for this. Is there some other way of playing scheduled audio
infinitely and dynamically? I have given the plugins a really good
look over and I'm pretty sure there is no clear alternative but
perhaps there is a clever use of other plugins I've not thought of.

Thanks for you help. I apologize for not having a reduced code example
in this email but I tried very for a while to get a reduction that
exhibits the issue but it's just so hard to reproduce consistantly. If
you would like to read through the code that does this it is on source
forge. Here is a link to the specific file of interest.

http://roundware.svn.sourceforge.net/viewvc/roundware/server/roundwared/stream.py?revision=42&view=markup

There is actually a function in there called has_runaway which is a
hack to make the effects of this bug in my program more tolerable. You
can find the rest of the code from their if you're interested. In case
you are wondering what the big picture is, this code is a museum audio
installation open now. Here is a link:

http://aldrichart.org/exhibitions/burgund.php

The link has an audio demo that may be illuminating as to what I'm
doing with this code.

-mike

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Edward Hervey-2]

Hi,

On Mon, 2008-06-09 at 22:46 -0400, Mike MacHenry wrote:

> I am writing a program using Gnonlin, which I feel might be a using
> the plugin in a way that it was not quite designed for. I am getting
> non-deterministic errors which are very difficult to pin down and I
> would like to just explain my design to the mailing list now. I am
> hoping to get some feedback on whether this design is too far from
> what gnonlin is supposed to do and if it's a wonder that it works. Or
> if maybe it's perfectly normal and the errors are well known.
>
> The design:
> Gnonlin seems to be designed to have several gnlfilesrc objects added
> to a gnlcomposition along with gnloperations and such and then played.
> There is a distinct separation between composition and playing. What I
> am doing is creating a composition with one file source in it and
> listening to the bus for a message telling me that that file has
> started playing. When it has, I add another file source to the
> composition after it. Once I get the signal that that next file is
> playing I queue up a third file and remove the first file source from
> the composition and stop it so that it can be cleaned up in memory.
> This goes on infinitely (in theory).
>
> The Problem:
> Occasionally, and eventually, it seems, for all streams I create, the
> composition gets confused. As soon as I queue up a file source for
> file number N with in the call back for file number N-1, I immediately
> receive a message that file N is playing, which causes my code to
> queue up a file source for N+1 and then N+1's message is also
> immediately sent and so forth until a ton of filesources are added. I
> immediately get an overload of file sources in the program and it
> crashes. It also seems to increase with load on the system. When 10
> clients play at once, they each run away at a faster rate.

  How exactly do you detect that a 'file is playing' ? GnlComposition
will preroll sources added to it, in order to have fast switching (from
one source to another). You might want to track the newsegment events
coming out of the composition (there will be one everytime the
composition switches sources).

>
> I can't figure out how the composition could get so out of sink. I
> suspect it has to do with its internal notion of the time line.
> Possibly I am confusing it with the way I add things. I welcome any
> ideas at all. Including questioning whether or not using gnonlin is
> right for this. Is there some other way of playing scheduled audio
> infinitely and dynamically? I have given the plugins a really good
> look over and I'm pretty sure there is no clear alternative but
> perhaps there is a clever use of other plugins I've not thought of.

  Try fixing the issue of when you add/remove the sources. Maybe the
rate issues is only related to the cpu overload. The composition doesn't
do any synchronization, it only modifies the values of the segment
events to provide proper synchronization downstream (in the sink
elements).

>
> Thanks for you help. I apologize for not having a reduced code example
> in this email but I tried very for a while to get a reduction that
> exhibits the issue but it's just so hard to reproduce consistantly. If
> you would like to read through the code that does this it is on source
> forge. Here is a link to the specific file of interest.
>
> http://roundware.svn.sourceforge.net/viewvc/roundware/server/roundwared/stream.py?revision=42&view=markup
>
> There is actually a function in there called has_runaway which is a
> hack to make the effects of this bug in my program more tolerable. You
> can find the rest of the code from their if you're interested. In case
> you are wondering what the big picture is, this code is a museum audio
> installation open now. Here is a link:
>
> http://aldrichart.org/exhibitions/burgund.php
>
> The link has an audio demo that may be illuminating as to what I'm
> doing with this code.
>
> -mike

  Edward

>
> -------------------------------------------------------------------------
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

--
Edward Hervey <[hidden email]>
Collabora Multimedia


-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Mike MacHenry]

On Tue, Jun 10, 2008 at 9:43 AM, Edward Hervey
<[hidden email]> wrote:
>  How exactly do you detect that a 'file is playing' ? GnlComposition
> will preroll sources added to it, in order to have fast switching (from
> one source to another). You might want to track the newsegment events
> coming out of the composition (there will be one everytime the
> composition switches sources).

Whenever a gnlfilesource begins to play, the bus of the pipeline is
sent a state change message. If you look in get_message in the code,
you can see that I accept the state change messages and if the new
state is gst.STATE_PLAYING I check to see which composition the source
object that is changing state belongs to by calling the is_finishing
method of the composition. That function just compares the source
object to that composition's currently playing object, in effect
saying "Is this you that just started playing?" On of them returns
true and that composition is instructed to queue up another.

>  Try fixing the issue of when you add/remove the sources. Maybe the
> rate issues is only related to the cpu overload. The composition doesn't
> do any synchronization, it only modifies the values of the segment
> events to provide proper synchronization downstream (in the sink
> elements).

I'm afraid I don't quite understand this. What does it mean to modify
the values of the segment? Is there a place where I can read more
about this. I'm pretty sure I've read all there is out there about
gnonlin unfortunately. Here's my understanding. Maybe you can explain
why it's wrong.

The time line is represented by a gnlcomposition. Time line in the
video editor's software sense. The bar at the bottom. One creates
gnlsources to put in the compostion. The start and end represent where
on the time line the source starts and end. The media-start and
media-end represent where in the file that source will draw from. I
know that there must always be something playing in the composition or
it will smoosh them together to fill the gap. If I put down a blank
audiotestsrc in the composition for one hour, and then a filesrc at 1
second to 2 second and another from 5 seconds to 7 seconds. I should
here 1 second of silence, 1 second of a file, 3 seconds of silence and
then 2 seconds of a file. This to me means that the composition must
have some notion of time in it that understands when to start playing
those two files. Am I way off base here?

Thanks for you help,
-mike

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://sourceforge.net/services/buy/index.php
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel