Adaptive resampling in gstaudiobasesink.c

[Pavel Hofman]

Hi,

Please do I understand correctly that the slave clock alignment methods
in gstaudiobasesink.c never touch the actual samples and handle only
various ways of adjusting pointers into the ringbuffer? Including the
GST_AUDIO_BASE_SINK_SLAVE_CUSTOM method.

Calculation of the master/slave clock difference is already available in
gstaudiobasesink.c. Adaptive resampling (like used in PA, alsaloop,
etc.) would require having two buffers with input and resampled samples:
in -> resampling -> out. If the out buffer were available, calling e.g.
src_process of libsamplerate would not be so difficult, IMO.

Please how complicated would be to add the out buffer and let the
implementations for specific audio outputs (e.g. GstAlsaSink) use it
instead of the buffer with incoming data? I am afraid it is not trivial
due to the "inheritance" within the code

struct _GstAlsaSink {
   GstAudioSink    sink;
...

} .

Perhaps the ring buffer would become the "out" buffer and the resampling
code had to read from another buffer written by the "input" side of the
plugin.

Thanks a lot for any suggestions and comments.

Best regards,

Pavel.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Charlie Laub]

CONTENTS DELETED

The author has deleted this message.

[Nicolas Dufresne-5]

Le sam. 17 oct. 2020 17 h 30, <[hidden email]> a écrit :

Nicolas, this is the message/thread on adaptive resampling that I mentioned.
-Charlie

I see, so he was proposing a new version of the callback, that would let app implement their own. To preserve API, this would simply imply adding a second callback. Certainly straight forward, but I think the approach does not cover your (and my) concern about having something to Open Source directly available.

I'd rather suggest looking into replacing the resampling code in the sink, using GStreamer resampler or an external one, that is up to the author really. Such contribution will be very welcome, it would with a decent cpu allow easy and professional synchronized playback by anyone.

-----Original Message-----
From: gstreamer-devel <[hidden email]> On
Behalf Of Pavel Hofman
Sent: Wednesday, October 14, 2020 4:12 AM
To: [hidden email]
Subject: Adaptive resampling in gstaudiobasesink.c

Hi,

Please do I understand correctly that the slave clock alignment methods in
gstaudiobasesink.c never touch the actual samples and handle only various
ways of adjusting pointers into the ringbuffer? Including the
GST_AUDIO_BASE_SINK_SLAVE_CUSTOM method.

Calculation of the master/slave clock difference is already available in
gstaudiobasesink.c. Adaptive resampling (like used in PA, alsaloop,
etc.) would require having two buffers with input and resampled samples:
in -> resampling -> out. If the out buffer were available, calling e.g.
src_process of libsamplerate would not be so difficult, IMO.

Please how complicated would be to add the out buffer and let the
implementations for specific audio outputs (e.g. GstAlsaSink) use it instead
of the buffer with incoming data? I am afraid it is not trivial due to the
"inheritance" within the code

struct _GstAlsaSink {
   GstAudioSink    sink;
..

} .

Perhaps the ring buffer would become the "out" buffer and the resampling
code had to read from another buffer written by the "input" side of the
plugin.

Thanks a lot for any suggestions and comments.

Best regards,

Pavel.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Pavel Hofman]

Dne 18. 10. 20 v 15:23 Nicolas Dufresne napsal(a):
>
>
> Le sam. 17 oct. 2020 17 h 30, <[hidden email]
> <mailto:[hidden email]>> a écrit :
>
> I see, so he was proposing a new version of the callback, that would let
> app implement their own. To preserve API, this would simply imply adding
> a second callback. Certainly straight forward, but I think the approach
> does not cover your (and my) concern about having something to Open > Source directly available.

Thanks for your follow up. IIUC all the existing code in alsasink
handles only playback positions in the playback ringbuffer, or offers a
way to control playback speed in the custom callback. But adaptive
resampling generates new samples, into a new buffer which is passed to
the sound device. IMO adding another callback would not help, if there
is no buffer to store the resampled stream.

Perhaps placing another audioresample element into the pipeline BEFORE
the audio sink  and controlling its momentary resampling rate by some
messages sent upstream by a GST_AUDIO_BASE_SINK_SLAVE_CUSTOM callback,
which knows the momentary timing situation and requirements, could be a
way with minimum changes. That would implement the adaptive resampling
satisfactorily, IMO.

>
> I'd rather suggest looking into replacing the resampling code in the
> sink, using GStreamer resampler or an external one, that is up to the
> author really. Such contribution will be very welcome, it would with a
> decent cpu allow easy and professional synchronized playback by anyone.
>

But IIUC the existing resampling code in the sink does not touch the
actual sample values and adding the second buffer would mean rewriting
the audio sink completely, IMO. Way above my understanding of gstreamer
internals.


Best regards,

Pavel.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel