Re: [gst-cvs] slomo gst-plugins-bad: gst-plugins-bad/ gst-plugins-bad/ext/soundtouch/

[Stefan Sauer]

hi,

[hidden email] schrieb:

> CVS Root:       /cvs/gstreamer
> Module:         gst-plugins-bad
> Changes by:     slomo
> Date:           Sun Jan 27 2008  05:56:18 UTC
>
> Log message:
> * ext/soundtouch/Makefile.am:
> * ext/soundtouch/gstbpmdetect.cc:
> * ext/soundtouch/gstbpmdetect.hh:
> * ext/soundtouch/plugin.c: (plugin_init):
> Add BPM detection plugin based on SoundTouch's libBPM.
> * ext/soundtouch/gstpitch.cc:
> Allow sample rates until MAX instead of only 48kHz and remove the
> buffer-frames field from that caps.
> Clear the remaining samples completely when necessary to get into
> a clean state again.

cool plugin! some comments for gst_bpm_detect_transform_ip

  if (filter->format.channels == 0 || filter->format.rate == 0) {
    GST_ERROR_OBJECT (bpm_detect, "No channels or rate set yet");
    return GST_FLOW_ERROR;
  }

  nsamples = GST_BUFFER_SIZE (in) / (4 * filter->format.channels);

  if (!bpm_detect->priv->detect)
    bpm_detect->priv->detect =
        new BPMDetect (filter->format.channels, filter->format.rate);

I would change that to

  if (G_UNLIKELY(!bpm_detect->priv->detect)) {
    if (filter->format.channels == 0 || filter->format.rate == 0) {
      GST_ERROR_OBJECT (bpm_detect, "No channels or rate set yet");
      return GST_FLOW_ERROR;
    }
    bpm_detect->priv->detect =
        new BPMDetect (filter->format.channels, filter->format.rate);
  }

  nsamples = GST_BUFFER_SIZE (in) / (4 * filter->format.channels);


Another one:

  data = (gfloat *) g_memdup (GST_BUFFER_DATA (in), GST_BUFFER_SIZE (in));
  bpm_detect->priv->detect->inputSamples (data, nsamples);
  g_free (data);

why do you need to copy?


Stefan

>
> Modified files:
>     .               : ChangeLog
>     ext/soundtouch  : Makefile.am gstpitch.cc
> Added files:
>     ext/soundtouch  : gstbpmdetect.cc gstbpmdetect.hh plugin.c
>
> Links:
> http://freedesktop.org/cgi-bin/viewcvs.cgi/gstreamer/gst-plugins-bad/ChangeLog.diff?r1=1.3063&r2=1.3064
> http://freedesktop.org/cgi-bin/viewcvs.cgi/gstreamer/gst-plugins-bad/ext/soundtouch/Makefile.am.diff?r1=1.3&r2=1.4
> http://freedesktop.org/cgi-bin/viewcvs.cgi/gstreamer/gst-plugins-bad/ext/soundtouch/gstbpmdetect.cc?rev=1.1&content-type=text/vnd.viewcvs-markup
> http://freedesktop.org/cgi-bin/viewcvs.cgi/gstreamer/gst-plugins-bad/ext/soundtouch/gstbpmdetect.hh?rev=1.1&content-type=text/vnd.viewcvs-markup
> http://freedesktop.org/cgi-bin/viewcvs.cgi/gstreamer/gst-plugins-bad/ext/soundtouch/gstpitch.cc.diff?r1=1.8&r2=1.9
> http://freedesktop.org/cgi-bin/viewcvs.cgi/gstreamer/gst-plugins-bad/ext/soundtouch/plugin.c?rev=1.1&content-type=text/vnd.viewcvs-markup
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> gstreamer-cvs mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-cvs

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Sebastian Dröge]

Am Sonntag, den 27.01.2008, 20:15 +0200 schrieb Stefan Kost:

> hi,
>
> [hidden email] schrieb:
> > CVS Root:       /cvs/gstreamer
> > Module:         gst-plugins-bad
> > Changes by:     slomo
> > Date:           Sun Jan 27 2008  05:56:18 UTC
> >
> > Log message:
> > * ext/soundtouch/Makefile.am:
> > * ext/soundtouch/gstbpmdetect.cc:
> > * ext/soundtouch/gstbpmdetect.hh:
> > * ext/soundtouch/plugin.c: (plugin_init):
> > Add BPM detection plugin based on SoundTouch's libBPM.
> > * ext/soundtouch/gstpitch.cc:
> > Allow sample rates until MAX instead of only 48kHz and remove the
> > buffer-frames field from that caps.
> > Clear the remaining samples completely when necessary to get into
> > a clean state again.
>
> cool plugin! some comments for gst_bpm_detect_transform_ip
>
>   if (filter->format.channels == 0 || filter->format.rate == 0) {
>     GST_ERROR_OBJECT (bpm_detect, "No channels or rate set yet");
>     return GST_FLOW_ERROR;
>   }
>
>   nsamples = GST_BUFFER_SIZE (in) / (4 * filter->format.channels);
>
>   if (!bpm_detect->priv->detect)
>     bpm_detect->priv->detect =
>         new BPMDetect (filter->format.channels, filter->format.rate);
>
> I would change that to
>
>   if (G_UNLIKELY(!bpm_detect->priv->detect)) {
>     if (filter->format.channels == 0 || filter->format.rate == 0) {
>       GST_ERROR_OBJECT (bpm_detect, "No channels or rate set yet");
>       return GST_FLOW_ERROR;
>     }
>     bpm_detect->priv->detect =
>         new BPMDetect (filter->format.channels, filter->format.rate);
>   }
>
>   nsamples = GST_BUFFER_SIZE (in) / (4 * filter->format.channels);

Why? :) I'll change it later because it looks a bit cleaner but is there
another reason? ;)

> Another one:
>
>   data = (gfloat *) g_memdup (GST_BUFFER_DATA (in), GST_BUFFER_SIZE (in));
>   bpm_detect->priv->detect->inputSamples (data, nsamples);
>   g_free (data);
>
> why do you need to copy?

Because BPMDetect downmixes from stereo to mono when getting stereo. And
the result is placed in the first half of the input buffer.

For mono it's unnecessary though. I'll change that later...

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Florent THIERY-2]

Just... a big THANK YOU :)

Do you think it could be extended to sending "beat detected" and
"bpm_detected" gobject messages on the fly (not only upon file
playback completion/stop), additionnaly to writing tags ? I'm not
aware of libsoundtouch's on-the-fly features...

Thanks

FLo

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Sebastian Dröge]

Am Montag, den 28.01.2008, 17:38 +0100 schrieb Florent:
> Just... a big THANK YOU :)
>
> Do you think it could be extended to sending "beat detected" and
> "bpm_detected" gobject messages on the fly (not only upon file
> playback completion/stop), additionnaly to writing tags ? I'm not
> aware of libsoundtouch's on-the-fly features...

Hi,
IMHO there's no need for a special message for this. We have a BPM tag
and tags are send as a message too... so... :)

Also the tag is sent every time the BPM changes by more than 1 and there
actually is a BPM value available.

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Florent THIERY-2]

> IMHO there's no need for a special message for this. We have a BPM tag
> and tags are send as a message too... so... :)

Ok

> Also the tag is sent every time the BPM changes by more than 1 and there
> actually is a BPM value available.

Well, what if you want to fire a graphical event (ex: light?) for
every beat ? I'd like to sync things based on beats + bpm rate.

* Do you think the initial bpm message + bpm measurement would be
precise enough to sync at once (beat(t) = initial_time + time*1/bpm) ?
* When the bpm is changing for more than 1, would the new message be
right after a beat ?

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Stefan Sauer]

Hi florent,

Florent schrieb:

>> IMHO there's no need for a special message for this. We have a BPM tag
>> and tags are send as a message too... so... :)
>
> Ok
>
>> Also the tag is sent every time the BPM changes by more than 1 and there
>> actually is a BPM value available.
>
> Well, what if you want to fire a graphical event (ex: light?) for
> every beat ? I'd like to sync things based on beats + bpm rate.
>
> * Do you think the initial bpm message + bpm measurement would be
> precise enough to sync at once (beat(t) = initial_time + time*1/bpm) ?
> * When the bpm is changing for more than 1, would the new message be
> right after a beat ?
>

the problem is that the algorithm used only gives you bpm estimates, it does not
tell you when the beat occured. Furthermore the plugins would need to know about
the latency to the sink then, or somehow marks the beats as payload on the
buffer it analyzed, so that the audiosink could send the message in sync with
the audio rendering. Not easy, but doable.

Stefan

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Florent THIERY-2]

> Hi florent,

Hi

> the problem is that the algorithm used only gives you bpm estimates, it does not
> tell you when the beat occured. Furthermore the plugins would need to know about
> the latency to the sink then, or somehow marks the beats as payload on the
> buffer it analyzed, so that the audiosink could send the message in sync with
> the audio rendering. Not easy, but doable.

Another option appears to me, please correct me if i'm wrong:
extracting frequency information of the detected beats from the
plugin, putting it into the audio pipeline as a fork (tee), applying a
low pass filter based on the fore-cited frequency, and using then the
level plugin (with a threshold) as beat signal...

What do you guys think ?

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Paul Brossier-5]

Hi all,

well, shameless and late plug here, but how about using aubio [0] for
beat detection? it should be fairly straightforward to extract beat
locations with it.

We talked already of doing a gst plugin for this, and I will as soon as
I find time. If anyone wants to have a look at it before then, i'd be
happy to give a hand (although i'm off for the week end from now).

cheers, Paul

[0] http://aubio.org

On Fri, 2008-02-01 at 17:43 +0100, Florent wrote:

> > Hi florent,
>
> Hi
>
> > the problem is that the algorithm used only gives you bpm estimates, it does not
> > tell you when the beat occured. Furthermore the plugins would need to know about
> > the latency to the sink then, or somehow marks the beats as payload on the
> > buffer it analyzed, so that the audiosink could send the message in sync with
> > the audio rendering. Not easy, but doable.
>
> Another option appears to me, please correct me if i'm wrong:
> extracting frequency information of the detected beats from the
> plugin, putting it into the audio pipeline as a fork (tee), applying a
> low pass filter based on the fore-cited frequency, and using then the
> level plugin (with a threshold) as beat signal...
>
> What do you guys think ?
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Sebastian Dröge]

Am Freitag, den 01.02.2008, 17:56 +0100 schrieb Paul Brossier:

> Hi all,
>
> well, shameless and late plug here, but how about using aubio [0] for
> beat detection? it should be fairly straightforward to extract beat
> locations with it.
>
> We talked already of doing a gst plugin for this, and I will as soon as
> I find time. If anyone wants to have a look at it before then, i'd be
> happy to give a hand (although i'm off for the week end from now).
>
> cheers, Paul
>
> [0] http://aubio.org

aubio is unfortunately GPL so it has to be in ugly :/ Other than that
looks great, I'll add it to my TODO list :) Thanks

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Stefan Sauer]

hi,

Sebastian Dröge schrieb:

> Am Freitag, den 01.02.2008, 17:56 +0100 schrieb Paul Brossier:
>> Hi all,
>>
>> well, shameless and late plug here, but how about using aubio [0] for
>> beat detection? it should be fairly straightforward to extract beat
>> locations with it.
>>
>> We talked already of doing a gst plugin for this, and I will as soon as
>> I find time. If anyone wants to have a look at it before then, i'd be
>> happy to give a hand (although i'm off for the week end from now).
>>
>> cheers, Paul
>>
>> [0] http://aubio.org
>
> aubio is unfortunately GPL so it has to be in ugly :/ Other than that
> looks great, I'll add it to my TODO list :) Thanks

GPL is because of libfftw :/ Still would be nice to see how it performs.

Stefan

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel