gstreamr-bad-plugin: remove silence adding "pre-length"

[Sagar]

Hi,

I am currently working on "removesilence" bad plugin, I have applied path provided by Nicola https://bugzilla.gnome.org/show_bug.cgi?id=671954

using Nicola's patch we get the audio buffer as message callbacks, but I have noticed that for some small non-silent buffers to we are getting bus callback.

I was thinking of adding one more property called "pre-length" inspired bu cutter element.

I want to know what all scenarios should be considered and how can I make sure to properly add "pre-length" property

[Mailing List SVR]

Il 24/03/2016 08:03, Sagar ha scritto:

> Hi,
>
> I am currently working on "removesilence" bad plugin, I have applied path
> provided by Nicola  https://bugzilla.gnome.org/show_bug.cgi?id=671954
> <https://bugzilla.gnome.org/show_bug.cgi?id=671954>
>
> using Nicola's patch we get the audio buffer as message callbacks, but I
> have noticed that for some small non-silent buffers to we are getting bus
> callback.
>
> I was thinking of adding one more property called *"pre-length"* inspired bu
> *cutter* element.

not sure if I have correctly understand what you mean, after applying my
patch you can specify:

- minimum-silence-buffers or minimum-silence-time before detect silence
- get notified on the bus when silence is detected/not detected
- retimestamp when silence is detected/removed

additionally you can specify the detection hysteresis too, this value is
applyed before minimum-silence-buffers/minimum-silence-time

is the plugin detecting silence for non silence buffers? Can you please
share an audio sample and indicate the pipeline you used, what you
expect and what you get?

thanks
Nicola

>
> I want to know what all scenarios should be considered and how can I make
> sure to properly add "pre-length" property
>
>
>
>
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble.com/gstreamr-bad-plugin-remove-silence-adding-pre-length-tp4676522.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sagar]

Mailing List SVR wrote

Il 24/03/2016 08:03, Sagar ha scritto:
> Hi,
>
> I am currently working on "removesilence" bad plugin, I have applied path
> provided by Nicola  https://bugzilla.gnome.org/show_bug.cgi?id=671954
> <https://bugzilla.gnome.org/show_bug.cgi?id=671954>
>
> using Nicola's patch we get the audio buffer as message callbacks, but I
> have noticed that for some small non-silent buffers to we are getting bus
> callback.
>
> I was thinking of adding one more property called *"pre-length"* inspired bu
> *cutter* element.

not sure if I have correctly understand what you mean, after applying my
patch you can specify:

- My concern is that the non silent buffer which we receive are sometimes noise, I am using pulsesrc as an input and if there is background noise even for fraction of second I get silence_finished message


- minimum-silence-buffers or minimum-silence-time before detect silence
- get notified on the bus when silence is detected/not detected
- retimestamp when silence is detected/removed

additionally you can specify the detection hysteresis too, this value is
applyed before minimum-silence-buffers/minimum-silence-time

- can you please explain what detection hysteresis mean in simple words, I didnt understand what it is.

is the plugin detecting silence for non silence buffers? Can you please
share an audio sample and indicate the pipeline you used, what you
expect and what you get?

plugin is not detecting silence for non silence buffer but when there is a small amount of noise I am getting silence_finished message which is correct BUT I want to ignore this non-silent buffer as it is noise.

thanks
Nicola

>
> I want to know what all scenarios should be considered and how can I make
> sure to properly add "pre-length" property
>
>
>
>
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble.com/gstreamr-bad-plugin-remove-silence-adding-pre-length-tp4676522.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Mailing List SVR]

Il 24/03/2016 09:59, Sagar ha scritto:

> Mailing List SVR wrote
>> Il 24/03/2016 08:03, Sagar ha scritto:
>>> Hi,
>>>
>>> I am currently working on "removesilence" bad plugin, I have applied path
>>> provided by Nicola  https://bugzilla.gnome.org/show_bug.cgi?id=671954
>>> &lt;https://bugzilla.gnome.org/show_bug.cgi?id=671954&gt;
>>>
>>> using Nicola's patch we get the audio buffer as message callbacks, but I
>>> have noticed that for some small non-silent buffers to we are getting bus
>>> callback.
>>>
>>> I was thinking of adding one more property called *"pre-length"* inspired
>>> bu
>>> *cutter* element.
>> not sure if I have correctly understand what you mean, after applying my
>> patch you can specify:
>>
>> - My concern is that the non silent buffer which we receive are sometimes
>> noise, I am using pulsesrc as an input and if there is background noise
>> even for fraction of second I get
> /
>> silence_finished
> /
>>   message
>>
>>
>> - minimum-silence-buffers or minimum-silence-time before detect silence
>> - get notified on the bus when silence is detected/not detected
>> - retimestamp when silence is detected/removed
>>
>> additionally you can specify the detection hysteresis too, this value is
>> applyed before minimum-silence-buffers/minimum-silence-time
>>
>> - can you please explain what
> *
>> detection hysteresis
> *
>>   mean in simple words, I didnt understand what it is.

seems what you need, try to increase hysteresis,

when a non silence sample is detected (based on the dB threshold) if
consecutive non silence samples exceed the hysteresis then silence is
considered finished,

Nicola

>>
>> is the plugin detecting silence for non silence buffers? Can you please
>> share an audio sample and indicate the pipeline you used, what you
>> expect and what you get?
>>
>> plugin is not detecting silence for non silence buffer but when there is a
>> small amount of noise I am getting silence_finished message which is
>> correct BUT I want to ignore this non-silent buffer as it is noise.
>>
>> thanks
>> Nicola
>>
>>> I want to know what all scenarios should be considered and how can I make
>>> sure to properly add "pre-length" property
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://gstreamer-devel.966125.n4.nabble.com/gstreamr-bad-plugin-remove-silence-adding-pre-length-tp4676522.html
>>> Sent from the GStreamer-devel mailing list archive at Nabble.com.
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>>
>> gstreamer-devel@.freedesktop
>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel@.freedesktop
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
>
>
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble.com/gstreamr-bad-plugin-remove-silence-adding-pre-length-tp4676522p4676525.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sagar]

Mailing List SVR wrote

Il 24/03/2016 09:59, Sagar ha scritto:
> Mailing List SVR wrote
>> Il 24/03/2016 08:03, Sagar ha scritto:
>>> Hi,
>>>
>>> I am currently working on "removesilence" bad plugin, I have applied path
>>> provided by Nicola  https://bugzilla.gnome.org/show_bug.cgi?id=671954
>>> <https://bugzilla.gnome.org/show_bug.cgi?id=671954>
>>>
>>> using Nicola's patch we get the audio buffer as message callbacks, but I
>>> have noticed that for some small non-silent buffers to we are getting bus
>>> callback.
>>>
>>> I was thinking of adding one more property called *"pre-length"* inspired
>>> bu
>>> *cutter* element.
>> not sure if I have correctly understand what you mean, after applying my
>> patch you can specify:
>>
>> - My concern is that the non silent buffer which we receive are sometimes
>> noise, I am using pulsesrc as an input and if there is background noise
>> even for fraction of second I get
> /
>> silence_finished
> /
>>   message
>>
>>
>> - minimum-silence-buffers or minimum-silence-time before detect silence
>> - get notified on the bus when silence is detected/not detected
>> - retimestamp when silence is detected/removed
>>
>> additionally you can specify the detection hysteresis too, this value is
>> applyed before minimum-silence-buffers/minimum-silence-time
>>
>> - can you please explain what
> *
>> detection hysteresis
> *
>>   mean in simple words, I didnt understand what it is.


seems what you need, try to increase hysteresis,

when a non silence sample is detected (based on the dB threshold) if
consecutive non silence samples exceed the hysteresis then silence is
considered finished,

Nicola


- Thanks,  I increased hysteresis it worked like a charm on prerecored videos and audios but if I pass source as pulse src there is a lot on noise in the output... I will try using it inside kurento server where i intend to use it.

>>
>> is the plugin detecting silence for non silence buffers? Can you please
>> share an audio sample and indicate the pipeline you used, what you
>> expect and what you get?
>>
>> plugin is not detecting silence for non silence buffer but when there is a
>> small amount of noise I am getting silence_finished message which is
>> correct BUT I want to ignore this non-silent buffer as it is noise.
>>
>> thanks
>> Nicola
>>
>>> I want to know what all scenarios should be considered and how can I make
>>> sure to properly add "pre-length" property
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://gstreamer-devel.966125.n4.nabble.com/gstreamr-bad-plugin-remove-silence-adding-pre-length-tp4676522.html
>>> Sent from the GStreamer-devel mailing list archive at Nabble.com.
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>>
>> gstreamer-devel@.freedesktop
>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel@.freedesktop
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
>
>
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble.com/gstreamr-bad-plugin-remove-silence-adding-pre-length-tp4676522p4676525.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Mailing List SVR]

Il 24/03/2016 10:51, Sagar ha scritto:

> Mailing List SVR wrote
>> Il 24/03/2016 09:59, Sagar ha scritto:
>>> Mailing List SVR wrote
>>>> Il 24/03/2016 08:03, Sagar ha scritto:
>>>>> Hi,
>>>>>
>>>>> I am currently working on "removesilence" bad plugin, I have applied
>>>>> path
>>>>> provided by Nicola  https://bugzilla.gnome.org/show_bug.cgi?id=671954
>>>>> &lt;https://bugzilla.gnome.org/show_bug.cgi?id=671954&gt;
>>>>>
>>>>> using Nicola's patch we get the audio buffer as message callbacks, but
>>>>> I
>>>>> have noticed that for some small non-silent buffers to we are getting
>>>>> bus
>>>>> callback.
>>>>>
>>>>> I was thinking of adding one more property called *"pre-length"*
>>>>> inspired
>>>>> bu
>>>>> *cutter* element.
>>>> not sure if I have correctly understand what you mean, after applying my
>>>> patch you can specify:
>>>>
>>>> - My concern is that the non silent buffer which we receive are
>>>> sometimes
>>>> noise, I am using pulsesrc as an input and if there is background noise
>>>> even for fraction of second I get
>>> /
>>>> silence_finished
>>> /
>>>>    message
>>>>
>>>>
>>>> - minimum-silence-buffers or minimum-silence-time before detect silence
>>>> - get notified on the bus when silence is detected/not detected
>>>> - retimestamp when silence is detected/removed
>>>>
>>>> additionally you can specify the detection hysteresis too, this value is
>>>> applyed before minimum-silence-buffers/minimum-silence-time
>>>>
>>>> - can you please explain what
>>> *
>>>> detection hysteresis
>>> *
>>>>    mean in simple words, I didnt understand what it is.
>>
>> seems what you need, try to increase hysteresis,
>>
>> when a non silence sample is detected (based on the dB threshold) if
>> consecutive non silence samples exceed the hysteresis then silence is
>> considered finished,
>>
>> Nicola
>>
>>
>> - Thanks,
>>   I increased hysteresis it worked like a charm on prerecored videos and
>> audios

ok well!

>>   but if I pass source as pulse src there is a lot on noise in the
>> output...

you can experiment changhing the decibel threshold, now you know how to
covert from decibel to hex ;-)

as said in another mail if useful I can make a property for this too,

Nicola

>>   I will try using it inside kurento server where i intend to use
>> it.
>>
>>>> is the plugin detecting silence for non silence buffers? Can you please
>>>> share an audio sample and indicate the pipeline you used, what you
>>>> expect and what you get?
>>>>
>>>> plugin is not detecting silence for non silence buffer but when there is
>>>> a
>>>> small amount of noise I am getting silence_finished message which is
>>>> correct BUT I want to ignore this non-silent buffer as it is noise.
>>>>
>>>> thanks
>>>> Nicola
>>>>
>>>>> I want to know what all scenarios should be considered and how can I
>>>>> make
>>>>> sure to properly add "pre-length" property
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://gstreamer-devel.966125.n4.nabble.com/gstreamr-bad-plugin-remove-silence-adding-pre-length-tp4676522.html
>>>>> Sent from the GStreamer-devel mailing list archive at Nabble.com.
>>>>> _______________________________________________
>>>>> gstreamer-devel mailing list
>>>>>
>>>> gstreamer-devel@.freedesktop
>>>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>> _______________________________________________
>>>> gstreamer-devel mailing list
>>>> gstreamer-devel@.freedesktop
>>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://gstreamer-devel.966125.n4.nabble.com/gstreamr-bad-plugin-remove-silence-adding-pre-length-tp4676522p4676525.html
>>> Sent from the GStreamer-devel mailing list archive at Nabble.com.
>>> _______________________________________________
>>> gstreamer-devel mailing list
>>>
>> gstreamer-devel@.freedesktop
>>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>> _______________________________________________
>> gstreamer-devel mailing list
>> gstreamer-devel@.freedesktop
>> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
>
>
>
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble.com/gstreamr-bad-plugin-remove-silence-adding-pre-length-tp4676522p4676527.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sagar]

Yes, if we could configure the internal VAD too then it will be very good, -60dB is very low I have been using -40dB in cutter element but still there is noise in the audio, I figure we need something to remove noise like audiocheblimit does but it also affects human voice too...

Thanks, I will get back to you if I have any queries or input..

[Sagar]

Hi Nicola,

I was wondering if I could output/signal voice buffer when silence is detected (i.e voice before silence) then I will have to keep on adding map.data  into an array and when silence is detected then this data_array will be having raw audio data ? Is the correct ?

[Mailing List SVR]

Il 24/03/2016 13:42, Sagar ha scritto:
> Hi Nicola,
>
> I was wondering if I could output/signal voice buffer when silence is
> detected (i.e voice before silence)

this seems something too specific to be implemented inside removesilence
plugin

>   then I will have to keep on adding
> *map.data*  into an array and when silence is detected then this
> *data_array* will be having raw audio data ? Is the correct ?

I think you can use something like this:

... ! removesilence ! appsink

get buffer/sample from appsink and keep your custom queue and then push
to another pipeline like this:

appsrc ! ....

if you want to push your buffer when silence is detected I think you
should at least retimestamp them,

Nicola

>
>
>
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble.com/gstreamr-bad-plugin-remove-silence-adding-pre-length-tp4676522p4676533.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sagar]

Hi,

I am trying to create voice buffer when user is talking and I intend to send this voice buffer raw audio to a web service, so when ever there is a voice activity I append the buffers and when there is silence I intend to send to a webservice.

This is my code snippet:
...
GstBuffer *voice_buf;
...
if(voicedetected)
{
if (incoming_buf) {
      voice_buf = gst_buffer_append (voice_buf, incoming_buf);
    } else {
      voice_buf = incoming_buf;
}

}
else{
...
voice_buf=NULL;
...
}
...


but I am getting assertion failed message in kurento server error logs file.
GST_IS_BUFFER failed.


is there anything wrong which you think I am doing ?