Spectrum Behaviour

[Cefn Hoile]

I've recently been working with the Spectrum element, (thanks for your
help on this) but trying to get it to work with relatively low-power
hardware. I'm wondering what I can do to minimise the processing load
of the spectrum element for a given FFT specification (with the same
interval and bands).

My pipeline currently looks something like this...
pulsesrc ! spectrum interval=40000000 bands=8 threshold=-80 ! fakesink

Whilst the pipeline configured in this way has practically
instantaneous updates on a quad core i7, it's taking up huge amounts
of CPU and running behind real time until it fails on a 900MHz machine
,with some kind of latency destroying the relationship between the
audio and the 'graphic equaliser' even before it fails. It's possible
to reduce the frequency of updates, which helps, but that impacts on
the visual effectiveness of the graphic equaliser.

I'm wondering for example if it's feasible to efficiently (and
lossily) downsample the audio and save effort in the FFT? Something
else I can do to make it more responsive?

Cefn
http:/cefn.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Stefan Sauer]

On 06/21/2012 03:42 AM, Cefn Hoile wrote:

> I've recently been working with the Spectrum element, (thanks for your
> help on this) but trying to get it to work with relatively low-power
> hardware. I'm wondering what I can do to minimise the processing load
> of the spectrum element for a given FFT specification (with the same
> interval and bands).
>
> My pipeline currently looks something like this...
> pulsesrc ! spectrum interval=40000000 bands=8 threshold=-80 ! fakesink
>
> Whilst the pipeline configured in this way has practically
> instantaneous updates on a quad core i7, it's taking up huge amounts
> of CPU and running behind real time until it fails on a 900MHz machine
> ,with some kind of latency destroying the relationship between the
> audio and the 'graphic equaliser' even before it fails. It's possible
> to reduce the frequency of updates, which helps, but that impacts on
> the visual effectiveness of the graphic equaliser.
>
> I'm wondering for example if it's feasible to efficiently (and
> lossily) downsample the audio and save effort in the FFT? Something
> else I can do to make it more responsive?

The fft is running in floating point precision and is not using any
hardware acceleration (like SS2). Allowing it to optionally use integer
fft routines (which we do have) could be worth trying. Alternatively
trying to apply ORC optimizations to the butterfly operation of the FFT
library would speed it up.

Both are not trivial things to do ...

Stefan

>
> Cefn
> http:/cefn.com
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Cefn Hoile]

The fft is running in floating point precision and is not using any

hardware acceleration (like SS2). Allowing it to optionally use integer
fft routines (which we do have) could be worth trying. Alternatively
trying to apply ORC optimizations to the butterfly operation of the FFT
library would speed it up.

Both are not trivial things to do ...

Stefan

Thanks for the suggestions, Stefan.

For interest, here's the result. I was just about able to run the Gstreamer spectrum element on this eleven year-old Compaq Presario 700 as part of a shrimping project, now immortalised in video.

https://vimeo.com/44535515

Unfortunately when I try to launch anything substantial on the machine, like open a browser, the CPU time this takes away from gstreamer somehow causes pulseaudio to halt altogether until I kill the python script which launched the pipeline. Basically once there's an underrun it seems to be terminal.

As I think you guessed, I was looking for a low-effort way to reduce load, so I didn't get into rewriting the core logic of the spectrum element. I was surprised that the CPU load of the FFT doesn't depend on some other factors like the sample rate, which I might be able to reduce cheaply. For now, it works, and that's good enough!

Cefn

http://cefn.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel