Fbdevsink Low Framerate on Raspberry Pi

[Dave]

Hi,

I am trying to display HD video (MJPEG or Quicktime Animation Codec) on a Raspberry Pi2 but experiencing very low framerates.  Is it possible to achieve reasonable framerates (25+fps) on the Pi perhaps with hardware acceleration?  I am not using an X server, I need to use the framebuffer only.  I have looked at glimagesink but this seems to require X.  Is it possible to use glimagesink directly on the framebuffer as GLES works at a good speed directly on the FB?

Regards,

Dave.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Samuel Hurst]

Hi Dave,

On 13/06/16 13:59, Dave Johnstone wrote:
> I am trying to display HD video (MJPEG or Quicktime Animation Codec) on
> a Raspberry Pi2 but experiencing very low framerates.  Is it possible to
> achieve reasonable framerates (25+fps) on the Pi perhaps with hardware
> acceleration?  I am not using an X server, I need to use the framebuffer
> only.  I have looked at glimagesink but this seems to require X.  Is it
> possible to use glimagesink directly on the framebuffer as GLES works at
> a good speed directly on the FB?

I'm currently using a Raspberry Pi 2 with only a framebuffer and can
reliably get 50fps H.264 video decode using hardware acceleration using
gst-omx for hardware decoding and glimagesink. In my case, glimagesink
is using EGL. I can't give you any information as to how well it'll
tackle MJPEG, but gst-omx does seem to have elements for it so I'd
assume it works fine.

https://cgit.freedesktop.org/gstreamer/gst-omx

I'd recommend using very recent versions of GStreamer as there's been a
fair few patches recently which have really helped boost performance
specifically on the Pi.

-Sam


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Dave]

Hi,

I'm having a hard time compiling on the Pi2 - lots of make errors, so I tried the Raspbian Jessie packages but these are quite out of date.  Is there some repo for up-to-date Raspbian packages or do I need to persevere with the build process?

Regards,

Dave.

On 13/06/2016 10:40 PM, Samuel Hurst wrote:

Hi Dave,

On 13/06/16 13:59, Dave Johnstone wrote:

I am trying to display HD video (MJPEG or Quicktime Animation Codec) on
a Raspberry Pi2 but experiencing very low framerates.  Is it possible to
achieve reasonable framerates (25+fps) on the Pi perhaps with hardware
acceleration?  I am not using an X server, I need to use the framebuffer
only.  I have looked at glimagesink but this seems to require X.  Is it
possible to use glimagesink directly on the framebuffer as GLES works at
a good speed directly on the FB?

I'm currently using a Raspberry Pi 2 with only a framebuffer and can
reliably get 50fps H.264 video decode using hardware acceleration using
gst-omx for hardware decoding and glimagesink. In my case, glimagesink
is using EGL. I can't give you any information as to how well it'll
tackle MJPEG, but gst-omx does seem to have elements for it so I'd
assume it works fine.

https://cgit.freedesktop.org/gstreamer/gst-omx

I'd recommend using very recent versions of GStreamer as there's been a
fair few patches recently which have really helped boost performance
specifically on the Pi.

-Sam

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Dave]

I switched to h264 files and have managed to compile both -plugins-bad and omx (git head) using the /opt/vc libraries but the framerate is still slow. Could you tell me which distro you using and which gstreamer version?

BTW the reference "hello_video" performs beautifully with the same files.

Dave.


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-4]

Le samedi 25 juin 2016 à 18:21 +0930, Dave Johnstone a écrit :
> I switched to h264 files and have managed to compile both -plugins-
> bad and omx (git head) using the /opt/vc libraries but the framerate
> is still slow. Could you tell me which distro you using and which
> gstreamer version?
>
> BTW the reference "hello_video" performs beautifully with the same
> files.

fbdevsink is accessing the deprecated raw RGB framebuffer. This means,
you video get colorspace converted on the CPU. While in hello video, it
uses proprietary broadcom display extensions. You'll get the best with
GStreamer by using glimagesink. It renders using GL on a dispmanx
plane, slightly less efficient then hello_video, but much more
flexible.

Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Dave]

I tried glimagesink but this does not link - unable to negotiate RGBA for EGLImagesink.

Dave.

> On 26 Jun 2016, at 12:46 AM, Nicolas Dufresne <[hidden email]> wrote:
>
>> Le samedi 25 juin 2016 à 18:21 +0930, Dave Johnstone a écrit :
>> I switched to h264 files and have managed to compile both -plugins-
>> bad and omx (git head) using the /opt/vc libraries but the framerate
>> is still slow. Could you tell me which distro you using and which
>> gstreamer version?
>>
>> BTW the reference "hello_video" performs beautifully with the same
>> files.
>
> fbdevsink is accessing the deprecated raw RGB framebuffer. This means,
> you video get colorspace converted on the CPU. While in hello video, it
> uses proprietary broadcom display extensions. You'll get the best with
> GStreamer by using glimagesink. It renders using GL on a dispmanx
> plane, slightly less efficient then hello_video, but much more
> flexible.
>
> Nicolas
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le dimanche 26 juin 2016 à 05:07 +0930, Dave Johnstone a écrit :
> I tried glimagesink but this does not link - unable to negotiate RGBA
> for EGLImagesink.

If you provide slightly more information, we might be able to help you
fix your mistakes. Please mention GStreamer version and/or Distribution
being used.

Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Dave]

The glimagesink is from -plugins-bad 1.9.0.1
The omxh264dec is from the latest (27/6/16) git master which reports 1.2.0

My pipeline:

gst-launch-1.0 playbin uri=file:///home/digits/footy/testGFX/test.h264
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayEGL\)\ gldisplayegl0";
^Chandling interrupt.
Interrupt: Stopping pipeline ...
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
0:00:03.367729807 12671 0x747914f0 ERROR            omxvideodec gstomxvideodec.c:974:gst_omx_video_dec_reconfigure_output_port:<omxh264dec-omxh264dec0> Failed to negotiate RGBA for EGLImage
0:00:03.369718477 12671 0x747914f0 ERROR            omxvideodec gstomxvideodec.c:1144:gst_omx_video_dec_reconfigure_output_port:<omxh264dec-omxh264dec0> Failed to negotiate
Freeing pipeline ...

"test.h264" is the reference file from "hello_pi" on Raspbian Jessie Lite, Raspberry Pi 2.

Regards,

Dave.

On 26/06/2016 8:47 PM, Nicolas Dufresne wrote:

If you provide slightly more information, we might be able to help you
fix your mistakes. Please mention GStreamer version and/or Distribution
being used.

Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-4]

Le lundi 27 juin 2016 à 09:12 +0930, Dave Johnstone a écrit :
> "test.h264" is the reference file from "hello_pi" on Raspbian Jessie
> Lite, Raspberry Pi 2.

This file has no container, hence require special treatment. You should
be able to play it, but since there is no timestamp information, the
behaviour will be slightly undefined.

gst-launch-1.0 filesrc location=test.h264 ! h264parse ! decodebin !
glimagsink

Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Dave]

Ok, I tried a new file in an mp4 container (created in prores, converted with avprobe libx264 -pix_fmt yuv420p) and this will play.  When starting playback there is a flash on screen, the file then plays OK until the end when one of the last few frames stays on screen for around 3 seconds before it stops (my file starts and ends in black).  Gst-launch-1.0 then exits with an error:


gst-launch-1.0 playbin uri=file:///home/digits/footy/testGFX/noise.mp4
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayEGL\)\ gldisplayegl0";
Got context from element 'playsink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayEGL\)\ gldisplayegl0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:04.559570252 13167 0x73d224f0 ERROR               glmemory gstglmemoryegl.c:201:_gl_mem_copy: GstGLMemoryEGL does not support copy
Got EOS from element "playbin0".
Execution ended after 0:00:03.813062393
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Regards,

Dave.

On 27/06/2016 9:38 AM, Nicolas Dufresne wrote:

This file has no container, hence require special treatment. You should
be able to play it, but since there is no timestamp information, the
behaviour will be slightly undefined.

gst-launch-1.0 filesrc location=test.h264 ! h264parse ! decodebin !
glimagsink

Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Dave]

I meant avconv, of course.

I also tried a mjpeg file in a quicktime container which plays but drops frames.  It also exhibits the brief flash at the start and does not exit when finished.

Dave.

On 27/06/2016 10:46 AM, Dave Johnstone wrote:

Ok, I tried a new file in an mp4 container (created in prores, converted with avprobe libx264 -pix_fmt yuv420p) and this will play.  When starting playback there is a flash on screen, the file then plays OK until the end when one of the last few frames stays on screen for around 3 seconds before it stops (my file starts and ends in black).  Gst-launch-1.0 then exits with an error:


gst-launch-1.0 playbin uri=file:///home/digits/footy/testGFX/noise.mp4
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayEGL\)\ gldisplayegl0";
Got context from element 'playsink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayEGL\)\ gldisplayegl0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:04.559570252 13167 0x73d224f0 ERROR               glmemory gstglmemoryegl.c:201:_gl_mem_copy: GstGLMemoryEGL does not support copy
Got EOS from element "playbin0".
Execution ended after 0:00:03.813062393
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Regards,

Dave.

On 27/06/2016 9:38 AM, Nicolas Dufresne wrote:

This file has no container, hence require special treatment. You should
be able to play it, but since there is no timestamp information, the
behaviour will be slightly undefined.

gst-launch-1.0 filesrc location=test.h264 ! h264parse ! decodebin !
glimagsink

Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-4]

Le lundi 27 juin 2016 à 10:46 +0930, Dave Johnstone a écrit :
> Ok, I tried a new file in an mp4 container (created in prores,
> converted with avprobe libx264 -pix_fmt yuv420p) and this will play. 
> When starting playback there is a flash on screen, the file then
> plays OK until the end when one of the last few frames stays on
> screen for around 3 seconds before it stops (my file starts and ends
> in black).  Gst-launch-1.0 then exits with an error:

This though smell like bugs (in fact regression, as it worked before).
Please file a bug to bugs.gnome.org, produce GStreamer.

cheers,
Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Dave]

Hi Sam,

I am experiencing a flash on screen when playback starts, this is just a single frame - I'm now using omxh264dec and glimagesink.  I reported it as a bug but, after investigation, the devs tell me it is a Broadcom problem and so it's unable to be fixed from this end.  Do you experience this flash in your use case?  Do you have a work-around? 

The Gnome Bugzilla reference is 768116.

Best regards,

Dave.

On 13/06/2016 10:40 PM, Samuel Hurst wrote:

Hi Dave,

On 13/06/16 13:59, Dave Johnstone wrote:

I am trying to display HD video (MJPEG or Quicktime Animation Codec) on
a Raspberry Pi2 but experiencing very low framerates.  Is it possible to
achieve reasonable framerates (25+fps) on the Pi perhaps with hardware
acceleration?  I am not using an X server, I need to use the framebuffer
only.  I have looked at glimagesink but this seems to require X.  Is it
possible to use glimagesink directly on the framebuffer as GLES works at
a good speed directly on the FB?

I'm currently using a Raspberry Pi 2 with only a framebuffer and can
reliably get 50fps H.264 video decode using hardware acceleration using
gst-omx for hardware decoding and glimagesink. In my case, glimagesink
is using EGL. I can't give you any information as to how well it'll
tackle MJPEG, but gst-omx does seem to have elements for it so I'd
assume it works fine.

https://cgit.freedesktop.org/gstreamer/gst-omx

I'd recommend using very recent versions of GStreamer as there's been a
fair few patches recently which have really helped boost performance
specifically on the Pi.

-Sam

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel