GSoC project: Waveform generation element

[Peteris Krisjanis-2]

Hi everyone!

My this summer project idea is to create waveform generation element
in C. I have few rudimentary ideas how this could happen and if anyone
has anything to add to this, please do so.

Currently when it's done in Jokosher it's just playing back file to
fakesink and then listening for level element messages. Also Pitivi
has it's own method of reading and storing waveforms. And buzztard as
I understands decode the audio in array using fdsink and mmap.

My idea is to create one unified Gstreamer element who would read
levels in C and who could be used by all three (and potentially other)
applications. It could have several optional features, like changeable
frequency of level reads and caching. One crazy idea is do the drawing
in C Cairo too, but that's probably overkill (and would require nice
set of features for developers to modify it according to their need).

Currently I keep going trough materials provided by Jeff Fortin. Also
I will try to understand what could be beneficial for Jokosher in this
case.

Suggestions, cheers, criticism and pure denials welcome!

Respectfully,
Peteris.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Stefan Sauer]

On 03/27/2012 03:07 PM, [hidden email] wrote:

> Hi everyone!
>
> My this summer project idea is to create waveform generation element
> in C. I have few rudimentary ideas how this could happen and if anyone
> has anything to add to this, please do so.
>
> Currently when it's done in Jokosher it's just playing back file to
> fakesink and then listening for level element messages. Also Pitivi
> has it's own method of reading and storing waveforms. And buzztard as
> I understands decode the audio in array using fdsink and mmap.

Gnome audio recorder could also use this.
> My idea is to create one unified Gstreamer element who would read
> levels in C and who could be used by all three (and potentially other)
> applications. It could have several optional features, like changeable
> frequency of level reads and caching.
The level element already has a changeable frequency. Of course if you
want to get the waveform at a higher precission, you need to re-run the
pipeline right now.
>  One crazy idea is do the drawing
> in C Cairo too, but that's probably overkill (and would require nice
> set of features for developers to modify it according to their need).
To me this sounds more like we want a wave-form gtk widget, which could
have a model-view split. One model could use gst with the level element
to build the waveform data, another one could peek at a waveform in
memory, yet another one could build a sophisticated multi-zoom level
structure (which hardly pays off these days anymore).

The view would do the drawing using cairo and could have a couple of
options to customize the appearance. Be careful as here it gets
complicated. Applications will most likely want to overlay this with:
- selections
- loop markers (section markers)
- controller curves
- ...

> Currently I keep going trough materials provided by Jeff Fortin.
Do you have a link?
> Also I will try to understand what could be beneficial for Jokosher in this
> case.
>
> Suggestions, cheers, criticism and pure denials welcome!
>
> Respectfully,
> Peteris.

Stefan
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Martijn Grendelman]

Hi all,

>> My this summer project idea is to create waveform generation element
>> in C. I have few rudimentary ideas how this could happen and if anyone
>> has anything to add to this, please do so.
>>
>> Currently when it's done in Jokosher it's just playing back file to
>> fakesink and then listening for level element messages. Also Pitivi
>> has it's own method of reading and storing waveforms. And buzztard as
>> I understands decode the audio in array using fdsink and mmap.
> Gnome audio recorder could also use this.
>> My idea is to create one unified Gstreamer element who would read
>> levels in C and who could be used by all three (and potentially other)
>> applications. It could have several optional features, like changeable
>> frequency of level reads and caching.
> The level element already has a changeable frequency. Of course if you
> want to get the waveform at a higher precission, you need to re-run the
> pipeline right now.

Perhaps this is a little off-topic, but I have recently been playing with
this method and I ran into some trouble.

I created a small Python application that stored values from a level
element in an array for later processing.

The problem I had with it, was that the number of messages seemed to max
out at something like 150/s; I was not able to collect more detailed stats
than that. Lower values for level's 'interval' property didn't increase
the number of messages.

I consider 150 values per second too low for creating a generally usable
dataset for waveform-drawing, so I gave up. Ideally, I would like to be
able to measure every sample, so 44100 or 48000 values per second.

Any idea what I might have been doing wrong? Is it just that Python is too
slow, or is there something else?

Best regards,
Martijn Grendelman
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Peteris Krisjanis-2]

T , 2012-03-28 12:48 +0200, Stefan Sauer wrote:
> >  One crazy idea is do the drawing
> > in C Cairo too, but that's probably overkill (and would require nice
> > set of features for developers to modify it according to their need).
> To me this sounds more like we want a wave-form gtk widget, which could
> have a model-view split. One model could use gst with the level element
> to build the waveform data, another one could peek at a waveform in
> memory, yet another one could build a sophisticated multi-zoom level
> structure (which hardly pays off these days anymore).

That's what I have been thinking about this proposal - gstreamer
waveform data part and drawing part in gtk.

> The view would do the drawing using cairo and could have a couple of
> options to customize the appearance. Be careful as here it gets
> complicated. Applications will most likely want to overlay this with:
> - selections
> - loop markers (section markers)
> - controller curves
> - ...

Yep, I will need this as Jokosher draws cursor and fade markers on
waveforms. I will have to expand my knowledge about Gtk, but it seems
doable.

Also I will have to think how to handle audio with multiple channels.

> > Currently I keep going trough materials provided by Jeff Fortin.
> Do you have a link?

These are your IRC #pitivi and email discussion logs in about waveforms
in general and in Pitivi. Jeff gave me this so I can read more what's
already been discussed (mostly about caching and .sfk type of files
usage).

In this definition now it seems reasonable GSoC project to propose,
thank you for comments.

Respectfully,
Peteris Krisjanis

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Robin Gareus]

On 03/28/2012 01:43 PM, Peteris Krisjanis wrote:
> T , 2012-03-28 12:48 +0200, Stefan Sauer wrote:
>>>  One crazy idea is do the drawing
>>> in C Cairo too, but that's probably overkill (and would require nice
>>> set of features for developers to modify it according to their need).

https://github.com/erikd/sndfile-tools/blob/master/src/waveform.c

best,
robin
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel