GStreamer-devel

Position info when doing time stretch

Classic

List

Threaded

7 messages Options

Gabriel M. Beddingfield

Position info when doing time stretch

Hello,

I'm doing time stretching app, and want the UI to show
position/length info in terms of the original media.
However, gst_element_query_position() and _duration() always
gives me playback times relative to the wall clock. This
time is affected by the playback speed.

Does anyone know a way to handle this?[1]

Here's the details................

Application: StretchPlayer[2]
Implementation: C++
Current API's: sndfile, rubberband, JACK

My GStreamer port is setting up this pipeline:

playbin --> audioconvert --> pitch
--> volume --> autoaudiosink

Pitch the soundtouch 'pitch' plugin that does time
stretching and pitch shifting. The playbin is only given
local files.

Whenever I query playbin for position and duration, it
always reports time in terms of the wall clock. That is, if
I've been listening for 30 secs, it tells me 30 secs.
However, if I'm playing at 1/2 speed, I want it to tell me
"15 secs".

At the moment, the only solution I see is to break up my
pipeline like this:

playbin --> appsink --> memory

memory --> appsrc --> pitch --> volume --> autoaudiosink

This would allow me to precisely track the position of the
audio, since I'm metering it out from memory. However, when
I tried to set up appsrc/appsink in a Qt application it gave
me fits last week.[3]

Does anyone have any suggestions??

Thanks in advance,
Gabriel

[1] I did search the archives, though I had to use
something like Nabble. For some reason, I couldn't
make SF's search work. The only thing I found was
the app playitslowly.
[2] http://www.teuton.org/~gabriel/stretchplayer/
[3] I started off trying to do appsrc because it would
allow me to wrap gstreamer into a form that looked
like JACK or ALSA.

------------------------------------------------------------------------------
This SF.net Dev2Dev email is sponsored by:

Show off your parallel programming skills.
Enter the Intel(R) Threading Challenge 2010.
http://p.sf.net/sfu/intel-thread-sfd
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Gabriel M. Beddingfield

Re: Position info when doing time stretch

Hello,

On Sunday, September 05, 2010 08:59:47 pm Gabriel M. Beddingfield wrote:
>
> I'm doing time stretching app, and want the UI to show
> position/length info in terms of the original media.
> However, gst_element_query_position() and _duration() always
> gives me playback times relative to the wall clock. This
> time is affected by the playback speed.
>
> Does anyone know a way to handle this?[1]

Anyone? :-)

Just curious.... is my question hard to understand? Or is it a hard question to answer? Or... other?

I don't really want to give up on gstreamer... but this looks like a dead end.

Thanks,
Gabriel

------------------------------------------------------------------------------
This SF.net Dev2Dev email is sponsored by:

Show off your parallel programming skills.
Enter the Intel(R) Threading Challenge 2010.
http://p.sf.net/sfu/intel-thread-sfd
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

michael smith-6-3

Re: Position info when doing time stretch

This works the way you want it to if you speed up/slow down playback
using the seek API.

You're instead using the pitch element, which doesn't do that.

Since you're controlling the playback speed, you could track the
corrections yourself.

Mike

On Tue, Sep 7, 2010 at 7:50 PM, Gabriel M. Beddingfield
<[hidden email]> wrote:

>
> Hello,
>
> On Sunday, September 05, 2010 08:59:47 pm Gabriel M. Beddingfield wrote:
>>
>> I'm doing time stretching app, and want the UI to show
>> position/length info in terms of the original media.
>> However, gst_element_query_position() and _duration() always
>> gives me playback times relative to the wall clock. This
>> time is affected by the playback speed.
>>
>> Does anyone know a way to handle this?[1]
>
> Anyone? :-)
>
> Just curious.... is my question hard to understand? Or is it a hard question to answer? Or... other?
>
> I don't really want to give up on gstreamer... but this looks like a dead end.
>
> Thanks,
> Gabriel
>
> ------------------------------------------------------------------------------
> This SF.net Dev2Dev email is sponsored by:
>
> Show off your parallel programming skills.
> Enter the Intel(R) Threading Challenge 2010.
> http://p.sf.net/sfu/intel-thread-sfd
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>

------------------------------------------------------------------------------
This SF.net Dev2Dev email is sponsored by:

Show off your parallel programming skills.
Enter the Intel(R) Threading Challenge 2010.
http://p.sf.net/sfu/intel-thread-sfd
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Gabriel M. Beddingfield

Re: Position info when doing time stretch

Hi Michael,

Thank you!

On Tuesday, September 07, 2010 09:56:36 pm Michael Smith wrote:
> This works the way you want it to if you speed up/slow down playback
> using the seek API.
>
> You're instead using the pitch element, which doesn't do that.

So, if I were to replace the pitch element with something similar... would I be able to gain control of that?

> Since you're controlling the playback speed, you could track the
> corrections yourself.

...except that the audio stream is buffered and passed through a vocoder, so I don't know at what time my change will take effect. Which is an OK estimation... until I try to add A/B looping.

-gabriel

------------------------------------------------------------------------------
This SF.net Dev2Dev email is sponsored by:

Show off your parallel programming skills.
Enter the Intel(R) Threading Challenge 2010.
http://p.sf.net/sfu/intel-thread-sfd
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Wim Taymans

Re: Position info when doing time stretch

In reply to this post by Gabriel M. Beddingfield

On Tue, 2010-09-07 at 21:50 -0500, Gabriel M. Beddingfield wrote:

> Hello,
>
> On Sunday, September 05, 2010 08:59:47 pm Gabriel M. Beddingfield wrote:
> >
> > I'm doing time stretching app, and want the UI to show
> > position/length info in terms of the original media.
> > However, gst_element_query_position() and _duration() always
> > gives me playback times relative to the wall clock. This
> > time is affected by the playback speed.
> >
> > Does anyone know a way to handle this?[1]
>
> Anyone? :-)
>
> Just curious.... is my question hard to understand? Or is it a hard question to answer? Or... other?
>
> I don't really want to give up on gstreamer... but this looks like a dead end.

The element probably doesn't adjust the applied_rate in the segment
events. This can be fixed if you file a bug.

Wim

>
> Thanks,
> Gabriel
>
> ------------------------------------------------------------------------------
> This SF.net Dev2Dev email is sponsored by:
>
> Show off your parallel programming skills.
> Enter the Intel(R) Threading Challenge 2010.
> http://p.sf.net/sfu/intel-thread-sfd
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Gabriel M. Beddingfield

Re: Position info when doing time stretch

Hi Wim,

On Wed, 8 Sep 2010, Wim Taymans wrote:

>>> I'm doing time stretching app, and want the UI to show
>>> position/length info in terms of the original media.
>>> However, gst_element_query_position() and _duration() always
>>> gives me playback times relative to the wall clock. This
>>> time is affected by the playback speed.
[snip]
>
> The element probably doesn't adjust the applied_rate in the segment
> events. This can be fixed if you file a bug.

Thank you, that makes sense. I will file a bug.

-gabriel

------------------------------------------------------------------------------
This SF.net Dev2Dev email is sponsored by:

Show off your parallel programming skills.
Enter the Intel(R) Threading Challenge 2010.
http://p.sf.net/sfu/intel-thread-sfd
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Gabriel M. Beddingfield

Re: Position info when doing time stretch

In reply to this post by Wim Taymans