Time delay in setting property & audio mixer element

[Pietro Battiston]

Hello,

if you

git clone git://pietrobattiston.it/fingerpicking -b minimal
cd fingerpicking
python finger.py


and then try to move the mouse over the empty area, you will hear that
the pitch changes along with the horizontal position of the cursor. And
that's precisely what I desired.
What is unintented is that it does so with approximately 30 ms. of
delay. And I'm not able to clearly understand the cause.

The wave is provided by an "audiotestsrc" element. Every time the
pointer moves, its "freq" property is changed. So:
1) is the audiotestsrc element intrinsecally too slow in changing
frequency? or,
2) is the audiotestsrc element asked to change frequency only when the
next main loop is ran? or,
3) is there an audio buffer somewhere "after" gstreamer?
4) am I missing something else (in the end, it's been few days since I
first programmed with gstreamer)?

I would tend to exclude the third, because I notice no delay in the act
of pressing - releasing the mouse.
The second looks reasonable to me: if that's the problem, is there any
solution?
Or more in general: do you suggest that gstreamer can do what I want or
am I using the wrong tool?



While I'm at it, another question: I searched for some element doing the
inverse of "tee" for audio. I found a discussion on the argument:
http://gstreamer-devel.966125.n4.nabble.com/aggregator-element-td973247.html
which however was dropped. Dan:
http://gstreamer-devel.966125.n4.nabble.com/aggregator-element-td973247.html#a973254
proposed several "interpretations" of that request, and what I need is
precisely the last: "For audio-only mix the inputs to one output, like a
multi-track studio mixer? ". Is there such an element?

Thanks to all in advance

Pietro

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Marco Ballesio]

Hi,

another late reply from my famous series, bte find my comment(s) below..

On Fri, May 20, 2011 at 7:21 PM, Pietro Battiston <[hidden email]> wrote:
..snip..

4) am I missing something else (in the end, it's been few days since I
first programmed with gstreamer)?

latencies can come fro various places... how are you measuring it? (e.g. you have an oscilloscope connected to the speakers). As a note, you should check the overall latency in the whole pipeline, but most probably, if you're using pulseaudio, it's in pulsesink that you should focus (try tweaking buffer-time and latency-time, defaulting to about 10ms multiples).

Another influence factor is the size of the buffers wrt the sample rate you're using in the source element: obviously, the smaller buffers the lower latencies. audiotestsrc has default blocksize set to 4096, default sample rate set to 44100Hz, that is about 22 ms of latency. Combined with the default pulsesink latency of above, rounded up to the 10ms multiples, we've pretty much the 30ms you've been measuring.

Btw I doubt you'll ever be able to get latencies lower than a few multiples of 1/HZ (http://www.unix.com/linux/48694-kernel-how-modify-read-tick-rate-hz.html), especially if dealing with the CFS..

Or more in general: do you suggest that gstreamer can do what I want or
am I using the wrong tool?

it depends on your target latency (see above), for sub-millisecond values one might even argue about the opportunity to use any non-realtime approaches ;)
 

While I'm at it, another question: I searched for some element doing the
inverse of "tee" for audio. I found a discussion on the argument:
http://gstreamer-devel.966125.n4.nabble.com/aggregator-element-td973247.html
which however was dropped. Dan:
http://gstreamer-devel.966125.n4.nabble.com/aggregator-element-td973247.html#a973254
proposed several "interpretations" of that request, and what I need is
precisely the last: "For audio-only mix the inputs to one output, like a
multi-track studio mixer? ". Is there such an element?

here you go:

http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-plugins/html/gst-plugins-base-plugins-adder.html

Regards

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Pietro Battiston]

Il giorno sab, 28/05/2011 alle 20.07 +0300, Marco Ballesio ha scritto:
> Hi,
>
> another late reply from my famous series, bte find my comment(s)
> below..


Being arrived when I had a bit lost my hope, it's even more
appreciated...

>
> On Fri, May 20, 2011 at 7:21 PM, Pietro Battiston
> <[hidden email]> wrote:
> ..snip..
>
>         4) am I missing something else (in the end, it's been few days
>         since I
>         first programmed with gstreamer)?
>        
>
> latencies can come fro various places... how are you measuring it?
> (e.g. you have an oscilloscope connected to the speakers).

First off, sorry, I did a clear error in the last mail: the latency I
observe is around 150 (not 30) ms.

To approximately measure it, that's what I do: I run my simple script on
a touchpad, start moving the finger up and down regularly until I'm able
to be perfectly out of sync (the highest pitch being reached when my
finger is lowest, and vice-versa), and then verify that I was moving up
& down ~ 3 times per second ==> the lag is ~1/6 of second.


That said, I realize not everyone has a touchscreen at hand, so if you
want to reproduce, I modified


git clone git://pietrobattiston.it/fingerpicking -b minimal
cd fingerpicking
python finger.py

so that now just clicking/releasing the mouse on the window changes
note. On my system, the delay is clearly noticeable.

> As a note, you should check the overall latency in the whole pipeline,
> but most probably, if you're using pulseaudio, it's in pulsesink that
> you should focus (try tweaking buffer-time and latency-time,
> defaulting to about 10ms multiples).

I'm using alsa directly...

>
> Another influence factor is the size of the buffers wrt the sample
> rate you're using in the source element: obviously, the smaller
> buffers the lower latencies. audiotestsrc has default blocksize set to
> 4096, default sample rate set to 44100Hz, that is about 22 ms of
> latency. Combined with the default pulsesink latency of above, rounded
> up to the 10ms multiples, we've pretty much the 30ms you've been
> measuring.
>

I tried to call set_property( 'blocksize', 1024 ) and lower values, but
apparently nothing changed... (do I misunderstand your point?)

> Btw I doubt you'll ever be able to get latencies lower than a few
> multiples of 1/HZ
> (http://www.unix.com/linux/48694-kernel-how-modify-read-tick-rate-hz.html), especially if dealing with the CFS..
>
>
>         Or more in general: do you suggest that gstreamer can do what
>         I want or
>         am I using the wrong tool?
>
> it depends on your target latency (see above), for sub-millisecond
> values one might even argue about the opportunity to use any
> non-realtime approaches ;)

I would just like something that a user would generally not notice... I
guess that something like 30 ms. in fact would be perfectly fine.

>  
>        
>         While I'm at it, another question: I searched for some element
>         doing the
>         inverse of "tee" for audio. I found a discussion on the
>         argument:
>         http://gstreamer-devel.966125.n4.nabble.com/aggregator-element-td973247.html
>         which however was dropped. Dan:
>         http://gstreamer-devel.966125.n4.nabble.com/aggregator-element-td973247.html#a973254
>         proposed several "interpretations" of that request, and what I
>         need is
>         precisely the last: "For audio-only mix the inputs to one
>         output, like a
>         multi-track studio mixer? ". Is there such an element?
>        
>
> here you go:
>
> http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-plugins/html/gst-plugins-base-plugins-adder.html
>

thanks a lot

Pietro


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Marco Ballesio]

Hi,

On Sat, May 28, 2011 at 10:16 PM, Pietro Battiston
<[hidden email]> wrote:
..snip..

> That said, I realize not everyone has a touchscreen at hand, so if you
> want to reproduce, I modified
>
>
> git clone git://pietrobattiston.it/fingerpicking -b minimal
> cd fingerpicking
> python finger.py
>
> so that now just clicking/releasing the mouse on the window changes
> note. On my system, the delay is clearly noticeable.
>

ok, tested on my ancient netbook and it's really bad ;)

..snip..
>
> I'm using alsa directly...
>

actually, you're using "autoaudiosink", which does not imply the usage
of alsasink. Besides, if pulseaudio is running on your system, you're
very likely using it even when you're accessing the audio subsystem
through the alsa API (this is one of the mighty powers of pulseaudio
;) ).

I've replaced autoaudiosink with pulsesink just to get a finer control
over the sink latencies (see the attached file). Modifying the block
size, buffer-time and latency-time to low values improved the
situation a lot on my system.

..snip..
> I tried to call set_property( 'blocksize', 1024 ) and lower values, but
> apparently nothing changed... (do I misunderstand your point?)
>

as I wrote earlier, you should focus on the sink (as well), as it's
there that eventually all the latency is accrued. See above for some
more hints.

Regards

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Pietro Battiston]

Il giorno dom, 29/05/2011 alle 20.38 +0300, Marco Ballesio ha scritto:

> Hi,
>
> On Sat, May 28, 2011 at 10:16 PM, Pietro Battiston
> <[hidden email]> wrote:
> ..snip..
> > That said, I realize not everyone has a touchscreen at hand, so if you
> > want to reproduce, I modified
> >
> >
> > git clone git://pietrobattiston.it/fingerpicking -b minimal
> > cd fingerpicking
> > python finger.py
> >
> > so that now just clicking/releasing the mouse on the window changes
> > note. On my system, the delay is clearly noticeable.
> >
>
> ok, tested on my ancient netbook and it's really bad ;)
>
> ..snip..
> >
> > I'm using alsa directly...
> >
>
> actually, you're using "autoaudiosink", which does not imply the usage
> of alsasink. Besides, if pulseaudio is running on your system, you're
> very likely using it even when you're accessing the audio subsystem
> through the alsa API (this is one of the mighty powers of pulseaudio
> ;) ).
>
> I've replaced autoaudiosink with pulsesink just to get a finer control
> over the sink latencies (see the attached file). Modifying the block
> size, buffer-time and latency-time to low values improved the
> situation a lot on my system.

Well, I should have been clearer: I'm using alsa directly in the sense
that pulseaudio is _not_ installed on my (Debian) system.

I tried to replace the autoaudiosink with an alsasink or alsamixer, but
got a "gst.LinkError: failed to link filter with sink" when trying to
link the previous component to them.

Tomorrow I'll try to understand this issue, for the moment thanks.

Pietro


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Marco Ballesio]

Hi,

On Mon, May 30, 2011 at 1:26 AM, Pietro Battiston <[hidden email]> wrote:
>>
> Well, I should have been clearer: I'm using alsa directly in the sense
> that pulseaudio is _not_ installed on my (Debian) system.
>

ok now that's clear. When using plain ALSA, consider that the
GStreamer interface is not as optimized as the pulseaudio one. One
example for all: mmapping is not possible with ALSA so to say that,
unless somebody fixes it ;) , latencies will always be significantly
higher than with pulseaudio.

Configuring both buffer-time and latency-time with alsasink could
anyway give some improvements.

> I tried to replace the autoaudiosink with an alsasink or alsamixer, but
> got a "gst.LinkError: failed to link filter with sink" when trying to
> link the previous component to them.

you should probably add an "audioconvert" element bw the effects and the sink.

Regards
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Pietro Battiston]

Il giorno lun, 30/05/2011 alle 09.29 +0300, Marco Ballesio ha scritto:

> Hi,
>
> On Mon, May 30, 2011 at 1:26 AM, Pietro Battiston <[hidden email]> wrote:
> >>
> > Well, I should have been clearer: I'm using alsa directly in the sense
> > that pulseaudio is _not_ installed on my (Debian) system.
> >
>
> ok now that's clear. When using plain ALSA, consider that the
> GStreamer interface is not as optimized as the pulseaudio one. One
> example for all: mmapping is not possible with ALSA so to say that,
> unless somebody fixes it ;) , latencies will always be significantly
> higher than with pulseaudio.
>
> Configuring both buffer-time and latency-time with alsasink could
> anyway give some improvements.

It really does.
That said, I will install and try pulseaudio too.

>
> > I tried to replace the autoaudiosink with an alsasink or alsamixer, but
> > got a "gst.LinkError: failed to link filter with sink" when trying to
> > link the previous component to them.
>
> you should probably add an "audioconvert" element bw the effects and the sink.
>

I had to, yes, thanks again.

Pietro

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel