Gst-omx buffer renegotiation problem

[Mikko Hurskainen]

Hi all,

I ran into a buffer renegotiation problem when trying to use gst-omx.
Since I didn't found any other mailing list for gst-omx and the problem
might be relevant for other gstreamer components as well I am posting my
question here.

So the problem is that when using the gst-omx video decoding the buffers
are allocated automatically. The gst-omx queries from the component that
how many and which size buffers it should allocate. The omx component
replies to allocate very few and very small buffers. With those buffer
sizes the decoding cannot proceed (input 2 buffers of size 256 and
output 4 buffers of size 256).

When gstreamer does the preroll and the component starts decoding it
posts settings changed event that contains the correct sizes of the
buffers for input & output. The settings changed event is handled in the
base_video_dec class, but only for resolution & color format.

So far I've identified two solutions:
 - Always at the beginning allocate a big enough buffer, however, this
solution has a drawback when dealing with HD resolutions
 - Implement buffer reneallocation into the base_video_dec class.

The buffer reallocation seems to be the way to go, but before jumping
into this I would like to ask if anybody else has encountered the same
problem with some non-omx codec and how it was handled ? For example
which state the gstreamer pipeline needs to be and so on ? Normally of
course media file container provides these values, but sometimes that
might be broken and/or unavailable.

 -- Mikko Hurskainen


------------------------------------------------------------------------------
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://p.sf.net/sfu/dev2dev-palm
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Mikko Hurskainen]

Hi all,

> Since I didn't found any other mailing list for gst-omx and the problem
> might be relevant for other gstreamer components as well I am posting my
> question here.

Some short-circuit in brain. Of course therese gstreamer-openmax@ but I
meant I did not find help from the archives of that list.

 -- Mikko Hurskainen


------------------------------------------------------------------------------
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://p.sf.net/sfu/dev2dev-palm
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Clark, Rob]

On Aug 1, 2010, at 11:59 AM, Mikko Hurskainen wrote:

> Hi all,
>
> I ran into a buffer renegotiation problem when trying to use gst-omx.
> Since I didn't found any other mailing list for gst-omx and the problem
> might be relevant for other gstreamer components as well I am posting my
> question here.
>
> So the problem is that when using the gst-omx video decoding the buffers
> are allocated automatically. The gst-omx queries from the component that
> how many and which size buffers it should allocate. The omx component
> replies to allocate very few and very small buffers. With those buffer
> sizes the decoding cannot proceed (input 2 buffers of size 256 and
> output 4 buffers of size 256).

I think normally the input port params should be set already, before buffers are allocated..  so the OMX component should be able to return sensible buffer sizes before gst-openmax transitions into executing state (and allocates buffers as part of that transition).

So I'm tempted to call this a bug in your OMX component.  But including a log with "*omx*:5" traces would be helpful to see exactly what is going on.


BR,
-R

>
> When gstreamer does the preroll and the component starts decoding it
> posts settings changed event that contains the correct sizes of the
> buffers for input & output. The settings changed event is handled in the
> base_video_dec class, but only for resolution & color format.
>
> So far I've identified two solutions:
> - Always at the beginning allocate a big enough buffer, however, this
> solution has a drawback when dealing with HD resolutions
> - Implement buffer reneallocation into the base_video_dec class.
>
> The buffer reallocation seems to be the way to go, but before jumping
> into this I would like to ask if anybody else has encountered the same
> problem with some non-omx codec and how it was handled ? For example
> which state the gstreamer pipeline needs to be and so on ? Normally of
> course media file container provides these values, but sometimes that
> might be broken and/or unavailable.
>
> -- Mikko Hurskainen
>
>
> ------------------------------------------------------------------------------
> The Palm PDK Hot Apps Program offers developers who use the
> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
> of $1 Million in cash or HP Products. Visit us here for more details:
> http://p.sf.net/sfu/dev2dev-palm
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://p.sf.net/sfu/dev2dev-palm
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Felipe Contreras]

Hello Mikko,

On Sun, Aug 1, 2010 at 7:59 PM, Mikko Hurskainen
<[hidden email]> wrote:

> I ran into a buffer renegotiation problem when trying to use gst-omx.
> Since I didn't found any other mailing list for gst-omx and the problem
> might be relevant for other gstreamer components as well I am posting my
> question here.
>
> So the problem is that when using the gst-omx video decoding the buffers
> are allocated automatically. The gst-omx queries from the component that
> how many and which size buffers it should allocate. The omx component
> replies to allocate very few and very small buffers. With those buffer
> sizes the decoding cannot proceed (input 2 buffers of size 256 and
> output 4 buffers of size 256).
>
> When gstreamer does the preroll and the component starts decoding it
> posts settings changed event that contains the correct sizes of the
> buffers for input & output. The settings changed event is handled in the
> base_video_dec class, but only for resolution & color format.
>
> So far I've identified two solutions:
>  - Always at the beginning allocate a big enough buffer, however, this
> solution has a drawback when dealing with HD resolutions
>  - Implement buffer reneallocation into the base_video_dec class.
>
> The buffer reallocation seems to be the way to go, but before jumping
> into this I would like to ask if anybody else has encountered the same
> problem with some non-omx codec and how it was handled ? For example
> which state the gstreamer pipeline needs to be and so on ? Normally of
> course media file container provides these values, but sometimes that
> might be broken and/or unavailable.

Well there's a third solution:
 * Don't allocate buffers in the first place if you don't know what
the final size would be

So, in gst-dsp that means we have to delay the DSP algorithm
initialization until we know the width and height of the clip. In
gst-openmax this would mean to delay the port buffer allocation, and
therefore the switch to Idle state.

IIRC in the past the switch to Idle state was synchronized with
GStreamer states, but not any more. So if your component is updating
nBufferSize when the framesize is updated, then everything should work
fine.

--
Felipe Contreras

------------------------------------------------------------------------------
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://p.sf.net/sfu/dev2dev-palm
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Mikko Hurskainen]

Hi Rob, Felipe,

> Well there's a third solution:
>  * Don't allocate buffers in the first place if you don't know what
> the final size would be
>
> So, in gst-dsp that means we have to delay the DSP algorithm
> initialization until we know the width and height of the clip. In
> gst-openmax this would mean to delay the port buffer allocation, and
> therefore the switch to Idle state.

That approach has a problem that I do not know the buffer size before
decoding first frame of the stream. The OMX component on the other hand
does not start decoding before it has buffers allocated.

So, how do you do it in gst-dsp if you do not know the width & height of
the clip ? Can it decode without having the buffers set ?

 -- Mikko


------------------------------------------------------------------------------
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://p.sf.net/sfu/dev2dev-palm
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Clark, Rob]

On Aug 4, 2010, at 1:20 AM, Mikko Hurskainen wrote:

> Hi Rob, Felipe,
>
>> Well there's a third solution:
>> * Don't allocate buffers in the first place if you don't know what
>> the final size would be
>>
>> So, in gst-dsp that means we have to delay the DSP algorithm
>> initialization until we know the width and height of the clip. In
>> gst-openmax this would mean to delay the port buffer allocation, and
>> therefore the switch to Idle state.
>
> That approach has a problem that I do not know the buffer size before
> decoding first frame of the stream. The OMX component on the other hand
> does not start decoding before it has buffers allocated.
>

how does that work from the point of OpenMAX API.. because the buffers must be allocated before going to executing state?


> So, how do you do it in gst-dsp if you do not know the width & height of
> the clip ? Can it decode without having the buffers set ?


well, gst-dsp is not burdened with the OpenMAX API ;-)


BR,
-R


------------------------------------------------------------------------------
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://p.sf.net/sfu/dev2dev-palm
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel