Audio visualizations

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Audio visualizations

Matteo Landi
Hi guys,
these days I was wondering how to use gstreamer in order to create
some audio visualizations like an oscilloscope or a spectrum analyzer.

The application consist of a window with a drawing area used to
display the visualizations.
Regarding the pipeline, I thought about something like

filesrc !
decodebin !
audioconvert !
audio/x-raw-int, channels=1, rate=44100, width=16, depth=16, endianness=1234 !
tee name=t !
  queue !
    fakesink name=fakesink signal-handoffs=true t. !
  queue !
    autoaudiosink

with the handoff callback linked to a method which spawn a new thread,
process the signal, and queue a redraw of the window.

I got few problems with the implementation:
1 - I often get core dumped, and I cannot understand why: all the
drawing actions are done inside the main thread, so it should be ok
...
2 - Synchronization between audio and video: the two sources are not
synchronized.. maybe is it something related to the queues elements?

Have you got any advices regarding the chosen implementation, and how
to solve the listed problems?

Thanks in advance for the attention.


--
Matteo Landi
http://www.matteolandi.net/

------------------------------------------------------------------------------
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://ad.doubleclick.net/clk;226879339;13503038;l?
http://clk.atdmt.com/CRS/go/247765532/direct/01/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Audio visualizations

Matteo Landi
I forgot to post the source of my script: http://pastebin.com/jqZYmV7X

On Tue, Jul 27, 2010 at 5:53 PM, Matteo Landi <[hidden email]> wrote:

> Hi guys,
> these days I was wondering how to use gstreamer in order to create
> some audio visualizations like an oscilloscope or a spectrum analyzer.
>
> The application consist of a window with a drawing area used to
> display the visualizations.
> Regarding the pipeline, I thought about something like
>
> filesrc !
> decodebin !
> audioconvert !
> audio/x-raw-int, channels=1, rate=44100, width=16, depth=16, endianness=1234 !
> tee name=t !
>  queue !
>    fakesink name=fakesink signal-handoffs=true t. !
>  queue !
>    autoaudiosink
>
> with the handoff callback linked to a method which spawn a new thread,
> process the signal, and queue a redraw of the window.
>
> I got few problems with the implementation:
> 1 - I often get core dumped, and I cannot understand why: all the
> drawing actions are done inside the main thread, so it should be ok
> ...
> 2 - Synchronization between audio and video: the two sources are not
> synchronized.. maybe is it something related to the queues elements?
>
> Have you got any advices regarding the chosen implementation, and how
> to solve the listed problems?
>
> Thanks in advance for the attention.
>
>
> --
> Matteo Landi
> http://www.matteolandi.net/
>



--
Matteo Landi
http://www.matteolandi.net/

------------------------------------------------------------------------------
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://ad.doubleclick.net/clk;226879339;13503038;l?
http://clk.atdmt.com/CRS/go/247765532/direct/01/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Audio visualizations

Matteo Landi
In reply to this post by Matteo Landi
Just a few news about the topic.

On Tue, Jul 27, 2010 at 5:53 PM, Matteo Landi <[hidden email]> wrote:

> Hi guys,
> these days I was wondering how to use gstreamer in order to create
> some audio visualizations like an oscilloscope or a spectrum analyzer.
>
> The application consist of a window with a drawing area used to
> display the visualizations.
> Regarding the pipeline, I thought about something like
>
> filesrc !
> decodebin !
> audioconvert !
> audio/x-raw-int, channels=1, rate=44100, width=16, depth=16, endianness=1234 !
> tee name=t !
>  queue !
>    fakesink name=fakesink signal-handoffs=true t. !
>  queue !
>    autoaudiosink
>
> with the handoff callback linked to a method which spawn a new thread,
> process the signal, and queue a redraw of the window.
>
> I got few problems with the implementation:
> 1 - I often get core dumped, and I cannot understand why: all the
> drawing actions are done inside the main thread, so it should be ok

Could this be due to the allocation of gtk/cairo resources inside side
threads different from the mainloop one? Maybe should I have to
protect them with threads_enter/leave?

> ...
> 2 - Synchronization between audio and video: the two sources are not
> synchronized.. maybe is it something related to the queues elements?

I noticed that the synchronization problem was due to the fakesink
element which try to run as fast as possible. I solve that by setting
the sync property to true (why is it not present inside the fakesink
documentation? is it a general property of sink objects?)

> Have you got any advices regarding the chosen implementation, and how
> to solve the listed problems?
>
> Thanks in advance for the attention.
>
>
> --
> Matteo Landi
> http://www.matteolandi.net/
>



--
Matteo Landi
http://www.matteolandi.net/

------------------------------------------------------------------------------
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://p.sf.net/sfu/dev2dev-palm
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Audio visualizations

Stefan Sauer
In reply to this post by Matteo Landi
Am 27.07.2010 18:53, schrieb Matteo Landi:

> Hi guys,
> these days I was wondering how to use gstreamer in order to create
> some audio visualizations like an oscilloscope or a spectrum analyzer.
>
> The application consist of a window with a drawing area used to
> display the visualizations.
> Regarding the pipeline, I thought about something like
>
> filesrc !
> decodebin !
> audioconvert !
> audio/x-raw-int, channels=1, rate=44100, width=16, depth=16, endianness=1234 !
> tee name=t !
>   queue !
>     fakesink name=fakesink signal-handoffs=true t. !
>   queue !
>     autoaudiosink
>
> with the handoff callback linked to a method which spawn a new thread,
> process the signal, and queue a redraw of the window.

Have a look at gst-plugins-good/test/examples/spectrum/

Anyway, don't draw to gtk-windows from a custom thread. Rather do processing in
a gst element and post the data via GstBus to the main thread and draw there.

Stefan

>
> I got few problems with the implementation:
> 1 - I often get core dumped, and I cannot understand why: all the
> drawing actions are done inside the main thread, so it should be ok
> ...
> 2 - Synchronization between audio and video: the two sources are not
> synchronized.. maybe is it something related to the queues elements?
>
> Have you got any advices regarding the chosen implementation, and how
> to solve the listed problems?
>
> Thanks in advance for the attention.
>
>


------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Audio visualizations

Matteo Landi
On Thu, Aug 5, 2010 at 11:29 PM, Stefan Kost <[hidden email]> wrote:

> Am 27.07.2010 18:53, schrieb Matteo Landi:
>> Hi guys,
>> these days I was wondering how to use gstreamer in order to create
>> some audio visualizations like an oscilloscope or a spectrum analyzer.
>>
>> The application consist of a window with a drawing area used to
>> display the visualizations.
>> Regarding the pipeline, I thought about something like
>>
>> filesrc !
>> decodebin !
>> audioconvert !
>> audio/x-raw-int, channels=1, rate=44100, width=16, depth=16, endianness=1234 !
>> tee name=t !
>>   queue !
>>     fakesink name=fakesink signal-handoffs=true t. !
>>   queue !
>>     autoaudiosink
>>
>> with the handoff callback linked to a method which spawn a new thread,
>> process the signal, and queue a redraw of the window.
>
> Have a look at gst-plugins-good/test/examples/spectrum/

Thank you, at first I haven't noticed the demo used to display bars
values read from the spectrum built in plugin. I was trying to
re-implement my own just for the curiosity-sake, nothing else :P

>
> Anyway, don't draw to gtk-windows from a custom thread. Rather do processing in
> a gst element and post the data via GstBus to the main thread and draw there.

Indeed, I suppose I was not been too clear while explaining my approach:

1 the fakesink queue a callback linked to the event (this is equal to
listening for messages on the bus): the only difference is that I need
more time for audio processing (fft) inside that callback
2 invoke a redraw method using gobject.idle_add with the fft data
processed at the previous step
3 draw the content on a backup cairo surface and invalidate the window
4 automatically redraw the window (paint the surface) using the
generated expose event.

The main difference I see with the code you linked [1], is that I
don't care at all about all those clocks operations (are they needed?)

I talked about threads, because I was thinking about spawning a new
thread for the audio process:
1 fakesink callback which spawns a new thread
2 threaded audio processing
3 gobject.idleadd ..
4 ...

There last one obscure thing: why the demo example does not initialize
threads for gobject? I thought it was necessary while working
simultaneously with gtk and gst..

>
> Stefan
>
>>
>> I got few problems with the implementation:
>> 1 - I often get core dumped, and I cannot understand why: all the
>> drawing actions are done inside the main thread, so it should be ok
>> ...
>> 2 - Synchronization between audio and video: the two sources are not
>> synchronized.. maybe is it something related to the queues elements?
>>
>> Have you got any advices regarding the chosen implementation, and how
>> to solve the listed problems?
>>
>> Thanks in advance for the attention.
>>
>>
>
>

Regards,


[1] http://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/tests/examples/spectrum/demo-audiotest.c


--
Matteo Landi
http://www.matteolandi.net/

------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Audio visualizations

Stefan Sauer
Am 06.08.2010 14:48, schrieb Matteo Landi:

> On Thu, Aug 5, 2010 at 11:29 PM, Stefan Kost <[hidden email]> wrote:
>> Am 27.07.2010 18:53, schrieb Matteo Landi:
>>> Hi guys,
>>> these days I was wondering how to use gstreamer in order to create
>>> some audio visualizations like an oscilloscope or a spectrum analyzer.
>>>
>>> The application consist of a window with a drawing area used to
>>> display the visualizations.
>>> Regarding the pipeline, I thought about something like
>>>
>>> filesrc !
>>> decodebin !
>>> audioconvert !
>>> audio/x-raw-int, channels=1, rate=44100, width=16, depth=16, endianness=1234 !
>>> tee name=t !
>>>   queue !
>>>     fakesink name=fakesink signal-handoffs=true t. !
>>>   queue !
>>>     autoaudiosink
>>>
>>> with the handoff callback linked to a method which spawn a new thread,
>>> process the signal, and queue a redraw of the window.
>>
>> Have a look at gst-plugins-good/test/examples/spectrum/
>
> Thank you, at first I haven't noticed the demo used to display bars
> values read from the spectrum built in plugin. I was trying to
> re-implement my own just for the curiosity-sake, nothing else :P
>
>>
>> Anyway, don't draw to gtk-windows from a custom thread. Rather do processing in
>> a gst element and post the data via GstBus to the main thread and draw there.
>
> Indeed, I suppose I was not been too clear while explaining my approach:
>
> 1 the fakesink queue a callback linked to the event (this is equal to
> listening for messages on the bus): the only difference is that I need
> more time for audio processing (fft) inside that callback
> 2 invoke a redraw method using gobject.idle_add with the fft data
> processed at the previous step
> 3 draw the content on a backup cairo surface and invalidate the window
> 4 automatically redraw the window (paint the surface) using the
> generated expose event.
>
> The main difference I see with the code you linked [1], is that I
> don't care at all about all those clocks operations (are they needed?)

The clock stuff just sync the drawing with the playback. Just imagine lots of
slow elements after the spectrum element.

>
> I talked about threads, because I was thinking about spawning a new
> thread for the audio process:
> 1 fakesink callback which spawns a new thread
> 2 threaded audio processing
> 3 gobject.idleadd ..
> 4 ...
>

So you want to do the fft yourself, instead of using gstreamers? Have a look at
http://versia.com/category/spek/ until 0.5 it used gstreamer like I described above.

> There last one obscure thing: why the demo example does not initialize
> threads for gobject? I thought it was necessary while working
> simultaneously with gtk and gst..
>
It should, but gst_init does that also.

Stefan

>>
>> Stefan
>>
>>>
>>> I got few problems with the implementation:
>>> 1 - I often get core dumped, and I cannot understand why: all the
>>> drawing actions are done inside the main thread, so it should be ok
>>> ...
>>> 2 - Synchronization between audio and video: the two sources are not
>>> synchronized.. maybe is it something related to the queues elements?
>>>
>>> Have you got any advices regarding the chosen implementation, and how
>>> to solve the listed problems?
>>>
>>> Thanks in advance for the attention.
>>>
>>>
>>
>>
>
> Regards,
>
>
> [1] http://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/tests/examples/spectrum/demo-audiotest.c
>
>


------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Audio visualizations

Matteo Landi
On Fri, Aug 6, 2010 at 9:01 PM, Stefan Kost <[hidden email]> wrote:

> Am 06.08.2010 14:48, schrieb Matteo Landi:
>> On Thu, Aug 5, 2010 at 11:29 PM, Stefan Kost <[hidden email]> wrote:
>>> Am 27.07.2010 18:53, schrieb Matteo Landi:
>>>> Hi guys,
>>>> these days I was wondering how to use gstreamer in order to create
>>>> some audio visualizations like an oscilloscope or a spectrum analyzer.
>>>>
>>>> The application consist of a window with a drawing area used to
>>>> display the visualizations.
>>>> Regarding the pipeline, I thought about something like
>>>>
>>>> filesrc !
>>>> decodebin !
>>>> audioconvert !
>>>> audio/x-raw-int, channels=1, rate=44100, width=16, depth=16, endianness=1234 !
>>>> tee name=t !
>>>>   queue !
>>>>     fakesink name=fakesink signal-handoffs=true t. !
>>>>   queue !
>>>>     autoaudiosink
>>>>
>>>> with the handoff callback linked to a method which spawn a new thread,
>>>> process the signal, and queue a redraw of the window.
>>>
>>> Have a look at gst-plugins-good/test/examples/spectrum/
>>
>> Thank you, at first I haven't noticed the demo used to display bars
>> values read from the spectrum built in plugin. I was trying to
>> re-implement my own just for the curiosity-sake, nothing else :P
>>
>>>
>>> Anyway, don't draw to gtk-windows from a custom thread. Rather do processing in
>>> a gst element and post the data via GstBus to the main thread and draw there.
>>
>> Indeed, I suppose I was not been too clear while explaining my approach:
>>
>> 1 the fakesink queue a callback linked to the event (this is equal to
>> listening for messages on the bus): the only difference is that I need
>> more time for audio processing (fft) inside that callback
>> 2 invoke a redraw method using gobject.idle_add with the fft data
>> processed at the previous step
>> 3 draw the content on a backup cairo surface and invalidate the window
>> 4 automatically redraw the window (paint the surface) using the
>> generated expose event.
>>
>> The main difference I see with the code you linked [1], is that I
>> don't care at all about all those clocks operations (are they needed?)
>
> The clock stuff just sync the drawing with the playback. Just imagine lots of
> slow elements after the spectrum element.

I imagined it. I thought the sync=true property was enough.

>>
>> I talked about threads, because I was thinking about spawning a new
>> thread for the audio process:
>> 1 fakesink callback which spawns a new thread
>> 2 threaded audio processing
>> 3 gobject.idleadd ..
>> 4 ...
>>
>
> So you want to do the fft yourself, instead of using gstreamers? Have a look at
> http://versia.com/category/spek/ until 0.5 it used gstreamer like I described above.

Thanks, I'll give it a look asap.

>
>> There last one obscure thing: why the demo example does not initialize
>> threads for gobject? I thought it was necessary while working
>> simultaneously with gtk and gst..
>>
> It should, but gst_init does that also.

Reading the docs of gst_init ([1]), it comes out that g_threads_init()
should be invoked explicitly.. that is in contrast with the example
inside the documentation or not?


Regards,


[1] http://www.gstreamer.net/data/doc/gstreamer/head/gstreamer/html/gstreamer-Gst.html#gst-init

>
> Stefan
>
>>>
>>> Stefan
>>>
>>>>
>>>> I got few problems with the implementation:
>>>> 1 - I often get core dumped, and I cannot understand why: all the
>>>> drawing actions are done inside the main thread, so it should be ok
>>>> ...
>>>> 2 - Synchronization between audio and video: the two sources are not
>>>> synchronized.. maybe is it something related to the queues elements?
>>>>
>>>> Have you got any advices regarding the chosen implementation, and how
>>>> to solve the listed problems?
>>>>
>>>> Thanks in advance for the attention.
>>>>
>>>>
>>>
>>>
>>
>> Regards,
>>
>>
>> [1] http://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/tests/examples/spectrum/demo-audiotest.c
>>
>>
>
>


--
Matteo Landi
http://www.matteolandi.net/

------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Audio visualizations

Stefan Sauer
Am 07.08.2010 13:47, schrieb Matteo Landi:

> On Fri, Aug 6, 2010 at 9:01 PM, Stefan Kost <[hidden email]> wrote:
>> Am 06.08.2010 14:48, schrieb Matteo Landi:
>>> On Thu, Aug 5, 2010 at 11:29 PM, Stefan Kost <[hidden email]> wrote:
>>>> Am 27.07.2010 18:53, schrieb Matteo Landi:
>>>>> Hi guys,
>>>>> these days I was wondering how to use gstreamer in order to create
>>>>> some audio visualizations like an oscilloscope or a spectrum analyzer.
>>>>>
>>>>> The application consist of a window with a drawing area used to
>>>>> display the visualizations.
>>>>> Regarding the pipeline, I thought about something like
>>>>>
>>>>> filesrc !
>>>>> decodebin !
>>>>> audioconvert !
>>>>> audio/x-raw-int, channels=1, rate=44100, width=16, depth=16, endianness=1234 !
>>>>> tee name=t !
>>>>>   queue !
>>>>>     fakesink name=fakesink signal-handoffs=true t. !
>>>>>   queue !
>>>>>     autoaudiosink
>>>>>
>>>>> with the handoff callback linked to a method which spawn a new thread,
>>>>> process the signal, and queue a redraw of the window.
>>>>
>>>> Have a look at gst-plugins-good/test/examples/spectrum/
>>>
>>> Thank you, at first I haven't noticed the demo used to display bars
>>> values read from the spectrum built in plugin. I was trying to
>>> re-implement my own just for the curiosity-sake, nothing else :P
>>>
>>>>
>>>> Anyway, don't draw to gtk-windows from a custom thread. Rather do processing in
>>>> a gst element and post the data via GstBus to the main thread and draw there.
>>>
>>> Indeed, I suppose I was not been too clear while explaining my approach:
>>>
>>> 1 the fakesink queue a callback linked to the event (this is equal to
>>> listening for messages on the bus): the only difference is that I need
>>> more time for audio processing (fft) inside that callback
>>> 2 invoke a redraw method using gobject.idle_add with the fft data
>>> processed at the previous step
>>> 3 draw the content on a backup cairo surface and invalidate the window
>>> 4 automatically redraw the window (paint the surface) using the
>>> generated expose event.
>>>
>>> The main difference I see with the code you linked [1], is that I
>>> don't care at all about all those clocks operations (are they needed?)
>>
>> The clock stuff just sync the drawing with the playback. Just imagine lots of
>> slow elements after the spectrum element.
>
> I imagined it. I thought the sync=true property was enough.

Setting sync=true on the sink is making the playback synced. In the example we
sync drawing to our custom graphs the screen to the playback time.

>
>>>
>>> I talked about threads, because I was thinking about spawning a new
>>> thread for the audio process:
>>> 1 fakesink callback which spawns a new thread
>>> 2 threaded audio processing
>>> 3 gobject.idleadd ..
>>> 4 ...
>>>
>>
>> So you want to do the fft yourself, instead of using gstreamers? Have a look at
>> http://versia.com/category/spek/ until 0.5 it used gstreamer like I described above.
>
> Thanks, I'll give it a look asap.
>
>>
>>> There last one obscure thing: why the demo example does not initialize
>>> threads for gobject? I thought it was necessary while working
>>> simultaneously with gtk and gst..
>>>
>> It should, but gst_init does that also.
>
> Reading the docs of gst_init ([1]), it comes out that g_threads_init()
> should be invoked explicitly.. that is in contrast with the example
> inside the documentation or not?

As I said, the examples should do that. I will improve the spectrum example soon
(already did that in buzztard).

Stefan

>
>
> Regards,
>
>
> [1] http://www.gstreamer.net/data/doc/gstreamer/head/gstreamer/html/gstreamer-Gst.html#gst-init
>
>>
>> Stefan
>>
>>>>
>>>> Stefan
>>>>
>>>>>
>>>>> I got few problems with the implementation:
>>>>> 1 - I often get core dumped, and I cannot understand why: all the
>>>>> drawing actions are done inside the main thread, so it should be ok
>>>>> ...
>>>>> 2 - Synchronization between audio and video: the two sources are not
>>>>> synchronized.. maybe is it something related to the queues elements?
>>>>>
>>>>> Have you got any advices regarding the chosen implementation, and how
>>>>> to solve the listed problems?
>>>>>
>>>>> Thanks in advance for the attention.
>>>>>
>>>>>
>>>>
>>>>
>>>
>>> Regards,
>>>
>>>
>>> [1] http://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/tests/examples/spectrum/demo-audiotest.c
>>>
>>>
>>
>>
>
>


------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Audio visualizations

Matteo Landi
On Sat, Aug 7, 2010 at 1:26 PM, Stefan Kost <[hidden email]> wrote:

> Am 07.08.2010 13:47, schrieb Matteo Landi:
>> On Fri, Aug 6, 2010 at 9:01 PM, Stefan Kost <[hidden email]> wrote:
>>> Am 06.08.2010 14:48, schrieb Matteo Landi:
>>>> On Thu, Aug 5, 2010 at 11:29 PM, Stefan Kost <[hidden email]> wrote:
>>>>> Am 27.07.2010 18:53, schrieb Matteo Landi:
>>>>>> Hi guys,
>>>>>> these days I was wondering how to use gstreamer in order to create
>>>>>> some audio visualizations like an oscilloscope or a spectrum analyzer.
>>>>>>
>>>>>> The application consist of a window with a drawing area used to
>>>>>> display the visualizations.
>>>>>> Regarding the pipeline, I thought about something like
>>>>>>
>>>>>> filesrc !
>>>>>> decodebin !
>>>>>> audioconvert !
>>>>>> audio/x-raw-int, channels=1, rate=44100, width=16, depth=16, endianness=1234 !
>>>>>> tee name=t !
>>>>>>   queue !
>>>>>>     fakesink name=fakesink signal-handoffs=true t. !
>>>>>>   queue !
>>>>>>     autoaudiosink
>>>>>>
>>>>>> with the handoff callback linked to a method which spawn a new thread,
>>>>>> process the signal, and queue a redraw of the window.
>>>>>
>>>>> Have a look at gst-plugins-good/test/examples/spectrum/
>>>>
>>>> Thank you, at first I haven't noticed the demo used to display bars
>>>> values read from the spectrum built in plugin. I was trying to
>>>> re-implement my own just for the curiosity-sake, nothing else :P
>>>>
>>>>>
>>>>> Anyway, don't draw to gtk-windows from a custom thread. Rather do processing in
>>>>> a gst element and post the data via GstBus to the main thread and draw there.
>>>>
>>>> Indeed, I suppose I was not been too clear while explaining my approach:
>>>>
>>>> 1 the fakesink queue a callback linked to the event (this is equal to
>>>> listening for messages on the bus): the only difference is that I need
>>>> more time for audio processing (fft) inside that callback
>>>> 2 invoke a redraw method using gobject.idle_add with the fft data
>>>> processed at the previous step
>>>> 3 draw the content on a backup cairo surface and invalidate the window
>>>> 4 automatically redraw the window (paint the surface) using the
>>>> generated expose event.
>>>>
>>>> The main difference I see with the code you linked [1], is that I
>>>> don't care at all about all those clocks operations (are they needed?)
>>>
>>> The clock stuff just sync the drawing with the playback. Just imagine lots of
>>> slow elements after the spectrum element.
>>
>> I imagined it. I thought the sync=true property was enough.
>
> Setting sync=true on the sink is making the playback synced. In the example we
> sync drawing to our custom graphs the screen to the playback time.

Ok sorry. The achieved synchronization is in term of actually
displaying the fft data of the current audio buffer. This should
prevent from analyzing and drawing spectrum data from old audio
buffered.

>>
>>>>
>>>> I talked about threads, because I was thinking about spawning a new
>>>> thread for the audio process:
>>>> 1 fakesink callback which spawns a new thread
>>>> 2 threaded audio processing
>>>> 3 gobject.idleadd ..
>>>> 4 ...
>>>>
>>>
>>> So you want to do the fft yourself, instead of using gstreamers? Have a look at
>>> http://versia.com/category/spek/ until 0.5 it used gstreamer like I described above.
>>
>> Thanks, I'll give it a look asap.
>>
>>>
>>>> There last one obscure thing: why the demo example does not initialize
>>>> threads for gobject? I thought it was necessary while working
>>>> simultaneously with gtk and gst..
>>>>
>>> It should, but gst_init does that also.
>>
>> Reading the docs of gst_init ([1]), it comes out that g_threads_init()
>> should be invoked explicitly.. that is in contrast with the example
>> inside the documentation or not?
>
> As I said, the examples should do that. I will improve the spectrum example soon
> (already did that in buzztard).

Sorry again, I misunderstood your statement.

Thanks again,

>
> Stefan
>
>>
>>
>> Regards,
>>
>>
>> [1] http://www.gstreamer.net/data/doc/gstreamer/head/gstreamer/html/gstreamer-Gst.html#gst-init
>>
>>>
>>> Stefan
>>>
>>>>>
>>>>> Stefan
>>>>>
>>>>>>
>>>>>> I got few problems with the implementation:
>>>>>> 1 - I often get core dumped, and I cannot understand why: all the
>>>>>> drawing actions are done inside the main thread, so it should be ok
>>>>>> ...
>>>>>> 2 - Synchronization between audio and video: the two sources are not
>>>>>> synchronized.. maybe is it something related to the queues elements?
>>>>>>
>>>>>> Have you got any advices regarding the chosen implementation, and how
>>>>>> to solve the listed problems?
>>>>>>
>>>>>> Thanks in advance for the attention.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>> Regards,
>>>>
>>>>
>>>> [1] http://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/tests/examples/spectrum/demo-audiotest.c
>>>>
>>>>
>>>
>>>
>>
>>
>
>



--
Matteo Landi
http://www.matteolandi.net/

------------------------------------------------------------------------------
This SF.net email is sponsored by

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel