How to properly switch between multiple audio sources while playing

[OPENSOURCE Manuel Huber]

Hello,

I would like to switch between playing sine tones of specific duration
and playback files - both with low latency. There seem to be different
ways possible to potentially achieve this, and I already tried most of
them. However, I'm unsure which approach is cleanest and should be
followed in that case and would kindly ask for guidance...

I would of course be happy to report back how the final solution
looked like, which might be helpful for others.


There are multiple things important here:

- I would like to prevent that gstreamer has to spawns threads
  whenever a tone or file is played. I would rather like to keep the
  threads idle, which should be achieved if kept in PAUSED state as I
  understand it.

- Fast switching between playing files and tones. It doesn't have to
  be gap-less, but I think it would be good to avoid having to
  reassemble the pipeline every time. I'd rather have one pipeline and
  just replace / switch the current source.

To simplify the use-case, I tried to only switch between two sine
tones (just to get a basic understanding).

1. Using input-selector and valve

    audiotestsrc1 -> valve1 -+
                             +-> input-selector -> alsasink
    audiotestsrc2 -> valve2 -+

  I heard some click's and pop's when switching...

2. Dynamically changing the pipeline using probes:

    <source> -> [queue ->] alsasink

  The idea would be to replace the source using blocking probes similar to this:
 
  - https://gstreamer.freedesktop.org/documentation/application-development/advanced/pipeline-manipulation.html#changing-elements-in-a-pipeline

  I'm not sure whether it's possible to apply the same idea when
  replacing the source. Would this work:

  - block on source.src pad, install a new probe and send EOS
  - wait for EOS on alsasink.sink (or on the queue.src?)
  - in the callback for EOS remove the source, add a new source, set
    the state, ...

3. Using playbin directly

  I'm unsure whether it's possible to replace the source while playing
  or there is some other way to start playing sine tones mixed with
  actual files. Maybe also a combination with input-selector is
  possible?


A second problem is how to play the sine tone only for the requested
time. I see two approaches:

- audiotestsrc seems to be quite capable on the second look, as
  adjusting the parameters (lower buffer size, smaller blocks,
  live-source, etc) can lower the latency during starting and also
  allows to set the playback time (more or less) exactly to the
  requested duration.  The only problem I had with that approach is
  how to "rewind" and re-configure the element (or do I have to
  recreate the element and replace it (for example using option 2)
  every time a new tone will be played?

- appsrc gives me (more or less) full control over the playback, seems
  to work fine for me. However, if not necessary, I would of course
  prefer to stick with a standard source like audiotestsrc.


If anybody has suggestions regarding the topic, I would be happy to
hear about it ;)


Best regards,

Manuel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nirbheek Chauhan]

On Thu, Apr 4, 2019 at 6:55 PM OPENSOURCE Manuel Huber
<[hidden email]> wrote:

> To simplify the use-case, I tried to only switch between two sine
> tones (just to get a basic understanding).
>
> 1. Using input-selector and valve
>
>     audiotestsrc1 -> valve1 -+
>                              +-> input-selector -> alsasink
>     audiotestsrc2 -> valve2 -+
>
>   I heard some click's and pop's when switching...
>

Have you tried audiomixer? You can use it as a 'seamless switcher' by
muting one sinkpad and unmuting the other.

If you get 'glitching' (due to silence in between) which can happen if
you get unlucky w.r.t. scheduling between muting one and unmuting the
other, you can use the gst controller API for scheduling both to
happen at the same time. See crossfade_app_set_pad_volume_fade() in
the example attached here:
https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/208

You can also use a linear fade to do a 'cross-fade' instead of a hard switch.

Cheers,
Nirbheek
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[OPENSOURCE Manuel Huber]

On Thu, Apr 4, 2019 at 4:53 PM Nirbheek Chauhan wrote:

> On Thu, Apr 4, 2019 at 6:55 PM OPENSOURCE Manuel Huber
> <[hidden email]> wrote:
>> To simplify the use-case, I tried to only switch between two sine
>> tones (just to get a basic understanding).
>>
>> 1. Using input-selector and valve
>>
>>     audiotestsrc1 -> valve1 -+
>>                              +-> input-selector -> alsasink
>>     audiotestsrc2 -> valve2 -+
>>
>>   I heard some click's and pop's when switching...
>>
>
> Have you tried audiomixer? You can use it as a 'seamless switcher' by
> muting one sinkpad and unmuting the other.

The question is, how can I handle the actual case when I use filesrc +
decodebin or playbin instead of audiotestsrc2. Would putting a valve
there allow me to have the file playback PAUSED or even in NULL state
while playing a tone?

The pipeline should be "hidden" behind a simple API that allows to
play a file (till the end) or play a tone for a specific duration (not
at the same time).

>
> If you get 'glitching' (due to silence in between) which can happen if
> you get unlucky w.r.t. scheduling between muting one and unmuting the
> other, you can use the gst controller API for scheduling both to
> happen at the same time. See crossfade_app_set_pad_volume_fade() in
> the example attached here:
> https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/208
>
> You can also use a linear fade to do a 'cross-fade' instead of a hard
> switch.
>

It sounds like a good idea for "stopping" the playback of the sine
tone when the specified duration is over without introducing an ugly
click / pop noise. Thanks for pointing me already to a good example ;)


Thanks for the quick response.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nirbheek Chauhan]

On Thu, Apr 4, 2019 at 9:55 PM OPENSOURCE Manuel Huber
<[hidden email]> wrote:

> The question is, how can I handle the actual case when I use filesrc +
> decodebin or playbin instead of audiotestsrc2. Would putting a valve
> there allow me to have the file playback PAUSED or even in NULL state
> while playing a tone?
>
> The pipeline should be "hidden" behind a simple API that allows to
> play a file (till the end) or play a tone for a specific duration (not
> at the same time).
>
> It sounds like a good idea for "stopping" the playback of the sine
> tone when the specified duration is over without introducing an ugly
> click / pop noise. Thanks for pointing me already to a good example ;)
>
>

Incidentally, that example handles the filesrc + decodebin case, which
is why it's so complicated. There's a lot of corner-cases in doing
that correctly.

Cheers,
Nirbheej
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel