omx - in error state at end of video (before EOS)

[Stuart Axon]

I've been playing a video, just before it gets to the end it freezes and I get the output below.

For now I haven't managed to make some minimal code to reproduce it yet, so just wondering it is known - similar to the OMX bug mentioned earier ?

https://lists.freedesktop.org/archives/gstreamer-devel/2016-December/061931.html

0:01:32.395974718  7472  0x2364200 ERROR                    omx gstomx.c:1955:gst_omx_port_wait_buffers_released_unlocked:<omxh264dec-omxh264dec0> Timeout waiting for egl_render port 221 to release all buffers

0:01:32.396736066  7472  0x2364200 WARN             omxvideodec gstomxvideodec.c:1620:gst_omx_video_dec_loop:<omxh264dec-omxh264dec0> error: Unable to reconfigure output port
0:01:32.398866620  7472  0x224f480 ERROR                    omx gstomx.c:836:gst_omx_component_set_state:<omxh264dec-omxh264dec0> Last operation returned an error. Setting last_error manually.
0:01:32.398946516  7472  0x224f480 ERROR                    omx gstomx.c:845:gst_omx_component_set_state:<omxh264dec-omxh264dec0> Error setting egl_render state from 4 to 3: Insufficient resources (0x80001000)
0:01:32.399041984  7472  0x224f480 ERROR                    omx gstomx.c:872:gst_omx_component_get_state:<omxh264dec-omxh264dec0> Component egl_render in error state: Insufficient resources (0x80001000)
0:01:32.399663280  7472  0x224f480 ERROR                    omx gstomx.c:1467:gst_omx_port_set_flushing:<omxh264dec-omxh264dec0> Component egl_render is in error state: Insufficient resources (0x80001000)
0:01:32.400010725  7472  0x224f480 ERROR                    omx gstomx.c:1467:gst_omx_port_set_flushing:<omxh264dec-omxh264dec0> Component egl_render is in error state: Insufficient resources (0x80001000)
0:01:32.400604053  7472  0x224f480 WARN             omxvideodec gstomxvideodec.c:2146:gst_omx_video_dec_flush:<omxh264dec-omxh264dec0> Failed to populate output port: Incorrect state operation (0x80001018)
0:01:32.402515808  7472  0x2364200 ERROR                    omx gstomx.c:1257:gst_omx_port_acquire_buffer:<omxh264dec-omxh264dec0> Component egl_render is in error state: Insufficient resources
0:01:32.403512778  7472  0x2364200 WARN             omxvideodec gstomxvideodec.c:1527:gst_omx_video_dec_loop:<omxh264dec-omxh264dec0> error: OpenMAX component in error state None (0x00000000)

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Enrique Ocaña González]

El jueves, 8 de diciembre de 2016 23:14:49 (CET) Stuart Axon escribió:

> I've been playing a video, just before it gets to the end it freezes and I
> get the output below. For now I haven't managed to make some minimal code
> to reproduce it yet, so just wondering it is known - similar to the OMX bug
> mentioned earier ?

We had similar "Insufficient resources" errors in our project in the past
(gst-omx, Raspberry Pi 2, omx decoder, egl buffers). Knowing how we solved it
might be useful for you.

In our particular case, the problem was that we had a custom video sink which
provided the last sample (the last egl_buffer) to the application. Our
application was holding that egl_buffer even after the pipeline wanted to
flush (eg: during seeks). Eventually, this exhausted the egl buffer pool and
led to the same error you're having. The summary: double check that that your
sink honors the flush-start event and that it doesn't hold or leak any egl
buffer after flush-starts.

Cheers.

--
Enrique Ocaña González
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Stuart Axon]

Interesting...  I'm using a pi3 with python with this pipeline:

     filesrc->decodebin->glimagesink.

I use set_window_handle from python and am also seeking within my videos.

the only event I handle is "prepare_window_handle", maybe I need to handle some more to clean things up ?

Alternately maybe the double free is in glimagesink ?

 

On Friday, December 9, 2016 10:45 AM, Enrique Ocaña González <[hidden email]> wrote:

El jueves, 8 de diciembre de 2016 23:14:49 (CET) Stuart Axon escribió:

> I've been playing a video, just before it gets to the end it freezes and I
> get the output below. For now I haven't managed to make some minimal code
> to reproduce it yet, so just wondering it is known - similar to the OMX bug
> mentioned earier ?

We had similar "Insufficient resources" errors in our project in the past
(gst-omx, Raspberry Pi 2, omx decoder, egl buffers). Knowing how we solved it
might be useful for you.

In our particular case, the problem was that we had a custom video sink which
provided the last sample (the last egl_buffer) to the application. Our
application was holding that egl_buffer even after the pipeline wanted to
flush (eg: during seeks). Eventually, this exhausted the egl buffer pool and
led to the same error you're having. The summary: double check that that your
sink honors the flush-start event and that it doesn't hold or leak any egl
buffer after flush-starts.

Cheers.

--
Enrique Ocaña González
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Enrique Ocaña González]

El viernes, 9 de diciembre de 2016 11:41:00 (CET) Stuart Axon escribió:

> > [...] The summary: double check that your sink honors the flush-start
> > event and that it doesn't hold or leak any egl buffer after flush-starts.
>
> Interesting...  I'm using a pi3 with python with this pipeline:
>      filesrc->decodebin->glimagesink.
>
> I use set_window_handle from python and am also seeking within my videos.
> the only event I handle is "prepare_window_handle", maybe I need to handle
> some more to clean things up ? Alternately maybe the double free is in
> glimagesink ?

Which version of glimagesink are you using, the one from gst-plugins-gl or the
one from gst-plugins-bad?

There's a redisplay_texture field in glimagesink which seems to hold the last
texture received by the sink. In the gst-plugins-gl version, it's only cleared
[1] on PAUSED --> READY state changes, but not on PAUSED --> PAUSED changes.
Also there's no code to explicitly handle FLUSH_START events in that gst-
plugins-gl version, so I guess that's done in some base class and nobody
releases the texture in those cases.

On the other hand, in the gst-plugins-bad version, there's additional support
for the DRAIN query [2] but still no FLUSH_START management. I'm not fully
acquainted with the code, but at first sight looks to me that the egl buffer
which the omxdecoder wants to release so badly is held in redisplay_texture by
gstglimakesink without permission during flushes, breaking the OMX buffer
release behaviour.

Cheers.

[1] https://cgit.freedesktop.org/gstreamer/attic/gst-plugins-gl/tree/gst/gl/
gstglimagesink.c#n567
[2] https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/
gstglimagesink.c?id=1.10.2#n1123

--
Enrique Ocaña González
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Stuart Axon]

I'm pretty sure I'm using the one from gst-plugins-bad (I'm using gstreamer master, last built about 3 days ago).

But not 100% sure, is there a way to tell ?

So maybe I could mitigate this by manually calling drain() if in the error state or hook into the FLUSH_START ?   -  this seems a bit vague, as I haven't looked at these parts of Gstreamer yet.

I haven't seen the error in dispmanx-gst-play, however, my player is a bit more complex - I update the play state (seek, or try and change file) in gobject timer, from data I get on the network.

 

On Friday, December 9, 2016 12:26 PM, Enrique Ocaña González <[hidden email]> wrote:

El viernes, 9 de diciembre de 2016 11:41:00 (CET) Stuart Axon escribió:

> > [...] The summary: double check that your sink honors the flush-start
> > event and that it doesn't hold or leak any egl buffer after flush-starts.
>
> Interesting...  I'm using a pi3 with python with this pipeline:
>      filesrc->decodebin->glimagesink.
>
> I use set_window_handle from python and am also seeking within my videos.
> the only event I handle is "prepare_window_handle", maybe I need to handle
> some more to clean things up ? Alternately maybe the double free is in
> glimagesink ?

Which version of glimagesink are you using, the one from gst-plugins-gl or the
one from gst-plugins-bad?

There's a redisplay_texture field in glimagesink which seems to hold the last
texture received by the sink. In the gst-plugins-gl version, it's only cleared
[1] on PAUSED --> READY state changes, but not on PAUSED --> PAUSED changes.
Also there's no code to explicitly handle FLUSH_START events in that gst-
plugins-gl version, so I guess that's done in some base class and nobody
releases the texture in those cases.

On the other hand, in the gst-plugins-bad version, there's additional support
for the DRAIN query [2] but still no FLUSH_START management. I'm not fully
acquainted with the code, but at first sight looks to me that the egl buffer
which the omxdecoder wants to release so badly is held in redisplay_texture by
gstglimakesink without permission during flushes, breaking the OMX buffer
release behaviour.

Cheers.

[1] https://cgit.freedesktop.org/gstreamer/attic/gst-plugins-gl/tree/gst/gl/
gstglimagesink.c#n567
[2] https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/
gstglimagesink.c?id=1.10.2#n1123

--
Enrique Ocaña González
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Enrique Ocaña González]

El viernes, 9 de diciembre de 2016 13:41:18 (CET) Stuart Axon escribió:

> I'm pretty sure I'm using the one from gst-plugins-bad (I'm using gstreamer
> master, last built about 3 days ago). But not 100% sure, is there a way to
> tell ?

If you're using master, then you're using gst-plugins-bad almost for sure.
Anyway, it doesn't really mind, as none of the two versions seem to listen to
the FLUSH_START event.

> So maybe I could mitigate this by manually calling drain() if in the error
> state or hook into the FLUSH_START?

My suggestion would be to focus on FLUSH_START instead of reacting in the
error state. In my experience, when "Insufficient resources" happens, it's too
late to revert to a working state again.

So, what I would do would be to touch gst_glimage_sink_event()[1], adding a
case for FLUSH_START. There you could do the same as the DRAIN query does[2]
(copy-paste or refactor into a common function called from both places). Make
sure that you don't return inside the case, so the code flow continues and
ends up calling to the ancestor class method to process the event[3]. That
last part is important for the right operation of the element.

> I haven't seen the error in dispmanx-gst-play, however, my player is a bit
> more complex - I update the play state (seek, or try and change file) in
> gobject timer, from data I get on the network. S++

I haven't looked in deep how dispmanx-gst-play works, so I can't give an
opinion about why things work there. Maybe a different sink is used (one which
doesn't hold EGL buffers for itself on flush), I don't know.

Another think I forgot: check that your Raspberry has enough graphics memory
configured. Assuming you're using raspbian: sudo raspi-config, advanced
options, memory split. Use some higher value such as 128.

In any case, I hope the suggestions help you to fix the issue. :-)

Cheers.


[1] https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/
gstglimagesink.c?id=1.10.2#n1023

[2] https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/
gstglimagesink.c?id=1.10.2#n1118

[3] https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/
gstglimagesink.c?id=1.10.2#n1069



--
Enrique Ocaña González
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Stuart Axon]

Cheers, some really good info there + directions, I'll definitely give this a go, the code seems pretty understandable and I don't really touch C.

There should be enough gfx memory, it's set to 256mb - and I can play the videos through to the end when I hit this without incident.

 

On Friday, December 9, 2016 3:35 PM, Enrique Ocaña González <[hidden email]> wrote:

El viernes, 9 de diciembre de 2016 13:41:18 (CET) Stuart Axon escribió:

> I'm pretty sure I'm using the one from gst-plugins-bad (I'm using gstreamer
> master, last built about 3 days ago). But not 100% sure, is there a way to
> tell ?

If you're using master, then you're using gst-plugins-bad almost for sure.
Anyway, it doesn't really mind, as none of the two versions seem to listen to
the FLUSH_START event.

> So maybe I could mitigate this by manually calling drain() if in the error
> state or hook into the FLUSH_START?

My suggestion would be to focus on FLUSH_START instead of reacting in the
error state. In my experience, when "Insufficient resources" happens, it's too
late to revert to a working state again.

So, what I would do would be to touch gst_glimage_sink_event()[1], adding a
case for FLUSH_START. There you could do the same as the DRAIN query does[2]
(copy-paste or refactor into a common function called from both places). Make
sure that you don't return inside the case, so the code flow continues and
ends up calling to the ancestor class method to process the event[3]. That
last part is important for the right operation of the element.

> I haven't seen the error in dispmanx-gst-play, however, my player is a bit
> more complex - I update the play state (seek, or try and change file) in
> gobject timer, from data I get on the network. S++

I haven't looked in deep how dispmanx-gst-play works, so I can't give an
opinion about why things work there. Maybe a different sink is used (one which
doesn't hold EGL buffers for itself on flush), I don't know.

Another think I forgot: check that your Raspberry has enough graphics memory
configured. Assuming you're using raspbian: sudo raspi-config, advanced
options, memory split. Use some higher value such as 128.

In any case, I hope the suggestions help you to fix the issue. :-)

Cheers.


[1] https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/
gstglimagesink.c?id=1.10.2#n1023

[2] https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/
gstglimagesink.c?id=1.10.2#n1118

[3] https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/
gstglimagesink.c?id=1.10.2#n1069

--
Enrique Ocaña González
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Stuart Axon]

OK, patch made and conversation moved here -

https://bugzilla.gnome.org/show_bug.cgi?id=776091

 

On Friday, December 9, 2016 7:09 PM, Stuart Axon <[hidden email]> wrote:

Cheers, some really good info there + directions, I'll definitely give this a go, the code seems pretty understandable and I don't really touch C.

There should be enough gfx memory, it's set to 256mb - and I can play the videos through to the end when I hit this without incident.

 

On Friday, December 9, 2016 3:35 PM, Enrique Ocaña González <[hidden email]> wrote:

El viernes, 9 de diciembre de 2016 13:41:18 (CET) Stuart Axon escribió:

> I'm pretty sure I'm using the one from gst-plugins-bad (I'm using gstreamer
> master, last built about 3 days ago). But not 100% sure, is there a way to
> tell ?

If you're using master, then you're using gst-plugins-bad almost for sure.
Anyway, it doesn't really mind, as none of the two versions seem to listen to
the FLUSH_START event.

> So maybe I could mitigate this by manually calling drain() if in the error
> state or hook into the FLUSH_START?

My suggestion would be to focus on FLUSH_START instead of reacting in the
error state. In my experience, when "Insufficient resources" happens, it's too
late to revert to a working state again.

So, what I would do would be to touch gst_glimage_sink_event()[1], adding a
case for FLUSH_START. There you could do the same as the DRAIN query does[2]
(copy-paste or refactor into a common function called from both places). Make
sure that you don't return inside the case, so the code flow continues and
ends up calling to the ancestor class method to process the event[3]. That
last part is important for the right operation of the element.

> I haven't seen the error in dispmanx-gst-play, however, my player is a bit
> more complex - I update the play state (seek, or try and change file) in
> gobject timer, from data I get on the network. S++

I haven't looked in deep how dispmanx-gst-play works, so I can't give an
opinion about why things work there. Maybe a different sink is used (one which
doesn't hold EGL buffers for itself on flush), I don't know.

Another think I forgot: check that your Raspberry has enough graphics memory
configured. Assuming you're using raspbian: sudo raspi-config, advanced
options, memory split. Use some higher value such as 128.

In any case, I hope the suggestions help you to fix the issue. :-)

Cheers.


[1] https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/
gstglimagesink.c?id=1.10.2#n1023

[2] https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/
gstglimagesink.c?id=1.10.2#n1118

[3] https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/
gstglimagesink.c?id=1.10.2#n1069

--
Enrique Ocaña González
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel