GStreamer-devel

handling seeking in audio decoder plugin

Classic

List

Threaded

6 messages Options

Sameer Naik

handling seeking in audio decoder plugin

Hi,
I have developed an audio decoder plugin for gstreamer that uses TI's
dsp decoders that decode the audio frames on the dsp.

An overview of the plugin architecture:
In the plugin i am parsing the audio stream (if the stream is not
packetized) so as to feed entire audio frames to the dsp decoder. Due
to the parsing code i store some of the input data to parse with the
next input buffer. in the stream once i have sync'd to an audioframe,
i calculate the encoded framelength and start copying the data into an
internal buffer that is designated as the dsp decoders input buffer.
Only after the entire frame has been copying, the DSP algorithm is
fired up to decode the encoded audio frame.

Regarding seeking i have a few questions.
Firstly, on an seek event i will need to clear the data in the
decoders input buffer as well as the data that i store for parsing
with the next input buffer. I want to know whether
1] the plugin could receive a buffer (_chain process) and an event
(_event_handler) at the same time. What i mean to say is if the
_chain function is called, could the pad _event handler be called
while the _chain function is still processing the input buffer.
2] Or does this happen in an orderly fashion, where if the _chain
function is currently processing data, then the pad _event handler
will not be called until the _chain has finished its processing.
Currently i have specified locks on the dsp decoders input buffers,
which is required if case 1 true, while it is absolutely unnecessary
if case 2 is true.

Secondly, as mentioned above i clear the decoders input buffer and the
other buffers on a seek event. Should i do this buffer cleanup on a
seek event or should i handle this in a new segment event or both.

Currently seeking is working most of the time. however it does fail
sometimes. Your response will help me in narrowing things down.

Please comment
~Sameer

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Tiago Katcipis

Re: handling seeking in audio decoder plugin

On Mon, Nov 30, 2009 at 8:50 AM, Sameer Naik <[hidden email]> wrote:

Hi,
I have developed an audio decoder plugin for gstreamer that uses TI's
dsp decoders that decode the audio frames on the dsp.

An overview of the plugin architecture:
In the plugin i am parsing the audio stream (if the stream is not
packetized) so as to feed entire audio frames to the dsp decoder. Due
to the parsing code i store some of the input data to parse with the
next input buffer. in the stream once i have sync'd to an audioframe,
i calculate the encoded framelength and start copying the data into an
internal buffer that is designated as the dsp decoders input buffer.
Only after the entire frame has been copying, the DSP algorithm is
fired up to decode the encoded audio frame.

Regarding seeking i have a few questions.
Firstly, on an seek event i will need to clear the data in the
decoders input buffer as well as the data that i store for parsing
with the next input buffer. I want to know whether
1] the plugin could receive a buffer (_chain process) and an event
(_event_handler) at the same time. What i mean to say is if the
_chain function is called, could the pad _event handler be called
while the _chain function is still processing the input buffer.
2] Or does this happen in an orderly fashion, where if the _chain
function is currently processing data, then the pad _event handler
will not be called until the _chain has finished its processing.
Currently i have specified locks on the dsp decoders input buffers,
which is required if case 1 true, while it is absolutely unnecessary
if case 2 is true.

This might help you Sameer:

"The actual seek is performed in the application thread so that success
or failure can be reported as a return value of the seek event. It is
therefore important that before executing the seek, the element acquires

the STREAM_LOCK so that the streaming thread and the seek get serialized."

http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-events.txt

Secondly, as mentioned above i clear the decoders input buffer and the
other buffers on a seek event. Should i do this buffer cleanup on a
seek event or should i handle this in a new segment event or both.

The documentation of the events should help you with the newsegment too.

Currently seeking is working most of the time. however it does fail
sometimes. Your response will help me in narrowing things down.

Please comment
~Sameer

best regards,
Katcipis

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Sameer Naik

Re: handling seeking in audio decoder plugin

Thanks for the info, it was quite helpful.
>From what i understand from
http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-events.txt
is that the gstreamer event mechanism will lock the stream while
GST_EVENT_FLUSH_STOP. Following which the calls will serialize.
Through a little debugging i found that this is the sequence of events
that take place (in my plugin).

GST_EVENT_SEEK on srcpad
GST_EVENT_FLUSH_START on sinkpad
GST_EVENT_FLUSH_STOP on sinkpad
GST_EVENT_NEWSEGMENT on sinkpad

According to events design doc, the plugin is supposed to discard any
input data it receives after the GST_EVENT_FLUSH_START event. This
call is not serialized. Which means the chain function would have to
be sprinkled with the is_flushing check, which is not very desirable.
Looking at decoders from the gstreamer packages, it seems none of them
handle GST_EVENT_FLUSH_START. However in my case i request a buffer
from the peer element (_pad_alloc()), which will fail when the
pipeline is flushing, so in my case it is the best place to check
whether the pipeline is flushing.

the GST_EVENT_FLUSH_STOP call is synchronized with STREAM_LOCK, so
this is the best place to clear all internal buffers/data. Since
GST_EVENT_FLUSH_STOP event and the _chain( ) function are the only
places where my internal buffers could be manipulated and since
GST_EVENT_FLUSH_STOP is already serialized, there is really no reason
to acquire locks on my internal buffers.

Thanks and Regards
~Sameer

On Mon, Nov 30, 2009 at 4:32 PM, Tiago Katcipis <[hidden email]> wrote:

>
>
> On Mon, Nov 30, 2009 at 8:50 AM, Sameer Naik
> <[hidden email]> wrote:
>>
>> Hi,
>> I have developed an audio decoder plugin for gstreamer that uses TI's
>> dsp decoders that decode the audio frames on the dsp.
>>
>> An overview of the plugin architecture:
>> In the plugin i am parsing the audio stream (if the stream is not
>> packetized) so as to feed entire audio frames to the dsp decoder. Due
>> to the parsing code i store some of the input data to parse with the
>> next input buffer. in the stream once i have sync'd to an audioframe,
>> i calculate the encoded framelength and start copying the data into an
>> internal buffer that is designated as the dsp decoders input buffer.
>> Only after the entire frame has been copying, the DSP algorithm is
>> fired up to decode the encoded audio frame.
>>
>> Regarding seeking i have a few questions.
>> Firstly, on an seek event i will need to clear the data in the
>> decoders input buffer as well as the data that i store for parsing
>> with the next input buffer. I want to know whether
>> 1] the plugin could receive a buffer (_chain process) and an event
>> (_event_handler) at the same time. What i mean to say is if the
>> _chain function is called, could the pad _event handler be called
>> while the _chain function is still processing the input buffer.
>> 2] Or does this happen in an orderly fashion, where if the _chain
>> function is currently processing data, then the pad _event handler
>> will not be called until the _chain has finished its processing.
>> Currently i have specified locks on the dsp decoders input buffers,
>> which is required if case 1 true, while it is absolutely unnecessary
>> if case 2 is true.
>>
>
> This might help you Sameer:
>
> "The actual seek is performed in the application thread so that success
> or failure can be reported as a return value of the seek event. It is
> therefore important that before executing the seek, the element acquires
>
> the STREAM_LOCK so that the streaming thread and the seek get serialized."
>
> http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-events.txt
>
>>
>> Secondly, as mentioned above i clear the decoders input buffer and the
>> other buffers on a seek event. Should i do this buffer cleanup on a
>> seek event or should i handle this in a new segment event or both.
>>
>
> The documentation of the events should help you with the newsegment too.
>
>>
>> Currently seeking is working most of the time. however it does fail
>> sometimes. Your response will help me in narrowing things down.
>>
>> Please comment
>> ~Sameer
>>
>
> best regards,
> Katcipis
>
>
>>
>>
>> ------------------------------------------------------------------------------
>> Let Crystal Reports handle the reporting - Free Crystal Reports 2008
>> 30-Day
>> trial. Simplify your report design, integration and deployment - and focus
>> on
>> what you do best, core application coding. Discover what's new with
>> Crystal Reports now. http://p.sf.net/sfu/bobj-july
>> _______________________________________________
>> gstreamer-devel mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>
>
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus
> on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>
>

Stefan Sauer

Re: handling seeking in audio decoder plugin

In reply to this post by Sameer Naik

Am 30.11.2009 12:50, schrieb Sameer Naik:
> Hi,
> I have developed an audio decoder plugin for gstreamer that uses TI's
> dsp decoders that decode the audio frames on the dsp.

for which codec is that?
>
> An overview of the plugin architecture:
> In the plugin i am parsing the audio stream (if the stream is not
> packetized) so as to feed entire audio frames to the dsp decoder. Due
> to the parsing code i store some of the input data to parse with the
> next input buffer.

I would recommend you to make use of separate parsers (mp3parse, aacparse, ...).
No need for you to write the parsing code and less to do when seeking.

Stefan

> in the stream once i have sync'd to an audioframe,
> i calculate the encoded framelength and start copying the data into an
> internal buffer that is designated as the dsp decoders input buffer.
> Only after the entire frame has been copying, the DSP algorithm is
> fired up to decode the encoded audio frame.
>
> Regarding seeking i have a few questions.
> Firstly, on an seek event i will need to clear the data in the
> decoders input buffer as well as the data that i store for parsing
> with the next input buffer. I want to know whether
> 1] the plugin could receive a buffer (_chain process) and an event
> (_event_handler) at the same time. What i mean to say is if the
> _chain function is called, could the pad _event handler be called
> while the _chain function is still processing the input buffer.
> 2] Or does this happen in an orderly fashion, where if the _chain
> function is currently processing data, then the pad _event handler
> will not be called until the _chain has finished its processing.
> Currently i have specified locks on the dsp decoders input buffers,
> which is required if case 1 true, while it is absolutely unnecessary
> if case 2 is true.
>
> Secondly, as mentioned above i clear the decoders input buffer and the
> other buffers on a seek event. Should i do this buffer cleanup on a
> seek event or should i handle this in a new segment event or both.
>
> Currently seeking is working most of the time. however it does fail
> sometimes. Your response will help me in narrowing things down.
>
> Please comment
> ~Sameer
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Tim-Philipp Müller-2

Re: handling seeking in audio decoder plugin

In reply to this post by Sameer Naik

On Mon, 2009-11-30 at 22:32 +0530, Sameer Naik wrote:

> According to events design doc, the plugin is supposed to discard any
> input data it receives after the GST_EVENT_FLUSH_START event. This
> call is not serialized. Which means the chain function would have to
> be sprinkled with the is_flushing check, which is not very desirable.

That's not how it works. The flush-start would be pushed out of band
from a thread other than the streaming threads (and your sink event
handler would/should pass it downstream). Pads will set themselves
automatically to flushing when they receive flush-start. In your chain
function you will then receive a wrong-state flow return when you do
gst_pad_push() or gst_pad_alloc_buffer(), at which point you'd stop or
skip the processing/decoding and bail out by returning wrong-state
upstream. You won't receive data after the flush-start automatically,
you just have to make sure you bail out quickly if needed.

Cheers
-Tim

------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing.
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Sameer Naik

Re: handling seeking in audio decoder plugin

got it. thanks for the info.

Regards
~Sameer

On Fri, Dec 4, 2009 at 4:21 AM, Tim-Philipp Müller <[hidden email]> wrote:

> On Mon, 2009-11-30 at 22:32 +0530, Sameer Naik wrote:
>
>> According to events design doc, the plugin is supposed to discard any
>> input data it receives after the GST_EVENT_FLUSH_START event. This
>> call is not serialized. Which means the chain function would have to
>> be sprinkled with the is_flushing check, which is not very desirable.
>
> That's not how it works. The flush-start would be pushed out of band
> from a thread other than the streaming threads (and your sink event
> handler would/should pass it downstream). Pads will set themselves
> automatically to flushing when they receive flush-start. In your chain
> function you will then receive a wrong-state flow return when you do
> gst_pad_push() or gst_pad_alloc_buffer(), at which point you'd stop or
> skip the processing/decoding and bail out by returning wrong-state
> upstream. You won't receive data after the flush-start automatically,
> you just have to make sure you bail out quickly if needed.
>
> Cheers
> -Tim
>
>
>
> ------------------------------------------------------------------------------
> Join us December 9, 2009 for the Red Hat Virtual Experience,
> a free event focused on virtualization and cloud computing.
> Attend in-depth sessions from your desk. Your couch. Anywhere.
> http://p.sf.net/sfu/redhat-sfdev2dev
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>