Timestamp based extraction of audio data from GstAdapter

[dingoegret]

Is there a plugin that would allow me to pull 100ms pts of audio every 100ms of real time? I was looking at audiorate but nothing I've tried accomplished what I wanted. My program is simple filesrc to appsink where I have set 'new-sample' signal' with the following callback

static void new_sample(GstElement *sink) {
    GstSample *sample;
    g_signal_emit_by_name(sink, "pull-sample", &sample);

    if(sample) {
        GstBuffer* buffer = gst_sample_get_buffer(sample);
        gst_adapter_push(adapter, buffer);
    }
}

I have another thread that reads off the adapter. Don't worry I've left out thread syncing mutex code for the example.

[Sebastian Dröge-3]

On Thu, 2017-04-13 at 02:39 -0700, dingoegret wrote:
> Is there a plugin that would allow me to pull 100ms pts of audio
> every 100ms of real time? I was looking at audiorate but nothing I've
> tried accomplished what I wanted.

You probably want to look at the new audiobuffersplit element:
https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst/audiobuffersplit

It does exactly that. You configure an output buffer size and it
outputs exactly that many samples, while making sure that timestamps
are correctly handled.

>  My program is simple filesrc to appsink where I have set 'new-
> sample' signal' with the following callback 
>
> static void new_sample(GstElement *sink) { 
>     GstSample *sample; 
>     g_signal_emit_by_name(sink, "pull-sample", &sample); 
>
>     if(sample) { 
>         GstBuffer* buffer = gst_sample_get_buffer(sample); 
>         gst_adapter_push(adapter, buffer); 
>     } 
> } 

You can't pass the buffer from the sample to the adapter like that as
you don't own the reference to the buffer. You have to ref() first.

For getting timestamps, GstAdapter has the gst_adapter_prev_pts() API,
which also gives you the distance to where this timestamp was observed.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[dingoegret]

How can I install that plugin? I tried installing gst-plugins-bad via homebrew but it doesnt seem to include it. '/usr/local/Cellar/gstreamer/1.10.4/bin/gst-inspect-1.0 audiobuffersplit' No such element or plugin 'audiobuffersplit'

Once installed, I assume the property to set is 'alignment_threshold'?

Can you link me to an explanation for ref? Or lightly touch on it? It 'seems' to work in that I can gst_adapter_take from another thread and save to a file and the audio is there.

[Sebastian Dröge-3]

On Thu, 2017-04-13 at 07:48 -0700, dingoegret wrote:
> How can I install that plugin? I tried installing gst-plugins-bad via
> homebrew but it doesnt seem to include it.

It's a new element that is in 1.11.1 and above. You can build it
yourself with older versions though, there is no dependency on other
parts of GStreamer newer than 1.10 or even 1.8.

> Once installed, I assume the
> property to set is 'alignment_threshold'?

You want to set output-buffer-duration to the value you want. It's in
fractional seconds.

> Can you link me to an explanation for ref? Or lightly touch on it?

See gst_buffer_ref() in the docs, and read up in general about
reference counting.

> It
> 'seems' to work in that I can gst_adapter_take from another thread and save
> to a file and the audio is there.

GstAdapter is not thread-safe

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[dingoegret]

Also just to be clear, I'm setting audiobuffersplit like this? In seconds and can I use milliseconds?

GValue f = { 0 };
g_value_init(&f, GST_TYPE_FRACTION);
gst_value_set_fraction(&f, 1, 50)
g_object_set(audiobuffersplit, "output-buffer-duration", f, NULL);

I get a

(gstreamer:67137): GLib-GObject-WARNING **: g_object_set_valist: passed '0' as denominator for `GstFraction'

[Sebastian Dröge-3]

On Sun, 2017-04-16 at 17:48 -0700, dingoegret wrote:
> I'm confused and completely lost and I would just rather copy and
> paste the audiobuffersplit code and compile that myself then use it
> in my homebrew gstreamer application. Any advise on how to do that?

You could just copy it and then compile it like any other C code that
uses GStreamer. There's nothing special about it really.

You probably want to remove the GST_PLUGIN_DEFINE bits and call
gst_element_register() yourself from your application then.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Sun, 2017-04-16 at 20:15 -0700, dingoegret wrote:
> Also just to be clear, I'm setting audiobuffersplit like this? in
> nanoseconds?
>
> g_object_set(audiobuffersplit, "output-buffer-duration", 200000000,
> NULL);

No, it's a fraction. So you provide the value as two integers to
g_object_set().

A fraction allows to make the output buffer duration exactly the same
as a video frame duration (e.g. in cases like 30000/1001 frames per
second) by compensation for rounding errors in the long run.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[dingoegret]

Thanks for your  help. I got it working on my osx build. I'm installing gstreamer through cerberos on ubuntu 16 lts right now and will eventually try it on there. Thank you for the work you did on audiobuffersplit. Its super useful.