appsink ignoring playback rate set with seek event

[Alejandro González]

Hello.

I'm trying to add extra sound speed changing functionality to my Rust
that processes audio, which currently reads sound files, processes
them a bit and them encodes them to OGG, using a pipeline like this:

filesrc -> decodebin -> audioconvert -> capsfilter -> audioresample ->
capsfilter -> vorbisenc -> oggmux -> appsink (appsink has "sync" set
to false)

I think I should mention that I'm using the latest version of
GStreamer Rust crates. Also, I use the appsink because I need to
process the raw result file bytes for packaging in a ZIP file. I could
use a filesink with a temporary file, but that sounds much less
performant. Most of the GStreamer related code of the application
follows the structure of the appsink example included with GStreamer
sources.

I've read that, in order to change the audio speed, I should send a
seek event to a paused pipeline, which I'm doing currently like this:

gstreamer_pipeline.seek(
    2.0, gstreamer::SeekFlags::FLUSH,
    gstreamer::SeekType::None, gstreamer::CLOCK_TIME_NONE,
    gstreamer::SeekType::None, gstreamer::CLOCK_TIME_NONE
)?;

However, this does not work, and if I copy the bytes received in the
appsink to a file, the result file plays at normal speed, which is not
what I want. I've tried the following without success:

- Setting "sync" to false: same result
- Using frame stepping: does not work (if I remember right it throws
errors; I've tried it last time some long hours ago)
- Seeking to a start time too: works, but playback rate is unaffected,
so it does not solve the issue
- Combining all the above and several other tricks I don't remember: nope

I've checked the debug logs, and as far as I can view, the seek event
propagates the rate successfully, and both PTS and DTS are calculated
fine in several elements. However, near the appsink sample pulling
calls, I see only mentions to the PTS.

But, if I swap the appsink for a filesink or autoaudiosink, things
work okay, and the audio is audibly faster. Can anyone tell me any
pointers to get the appsink to behave like I want? I believe that this
may be related to an older, unresolved email thread in this list
titled "gst_element_seek() when the pipeline has an appsink - bug? ",
but I'm not sure.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim Müller]

Hi Alejandro,

Try adding a 'scaletempo' element to the pipeline.

Cheers
 Tim

--
Tim Müller, Centricular Ltd - http://www.centricular.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Alejandro González]

Thank you very much, that was a step in the right direction!

The sink now receives the audio at the right speed, but sadly it maintains
the pitch, which is something that I do not want. I want both the tempo and
pitch to be changed; in other words, I want to change the speed while
producing the chipmunk effect, like the speed change filter of Audacity
does. I've played around with the "overlap", "search" and "stride"
properties of scaletempo, but that doesn't stop the pitch from being
maintained.

I want to note that, when I use a filesink without the scaletempo filter,
the speed is changed producing the chipmunk effect, which is exactly what I
want, but using a filesink is not a satisfactory solution for my
application.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Alejandro González]

Oh, and sorry for the quick followup, but I was wrong when I said that the
filesink worked. Only the autoaudiosink plays the audio as intended; the
filesink has the same behavior as the appsink. Maybe this correction is
useful for further troubleshooting, but for now it means that I have no
workarounds left that I can implement with a single pipeline or without
reading back the audio data that is sent somewhere outside the application.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Alejandro González]

Whew, another message, but I have good news: I found an acceptable way to
make this work!

After reading again the documentation for the "speed" filter at
https://gstreamer.freedesktop.org/documentation/speed/index.html?gi-language=c#speed,
which by its description should do what I want but has a scary warning
reading "do not use this element", I decided to give the "pitch" element,
documented here, a try:
https://gstreamer.freedesktop.org/documentation/soundtouch/pitch.html?gi-language=c#pitch.
I tried using it yesterday, but I was so dazed that I didn't realize that it
is a part of the bad GStreamer plugins package, which I didn't have
installed, so it didn't work. After installing the needed package for my
distro, plugging the pitch element to my pipeline, removing the seek event
and configuring the pitch element with equal "pitch" and "tempo" properties,
the appsink generates an audio stream with the desired speed, tempo and
pitch.

Maybe the appsink/muxer/encoder should support the seek rate properly, but
for now I'm satisfied with this. Thanks!



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel