gstreamer windowsless opengl rendering (wayland)

[Jake Zhang]

I have been struggling to get gstreamer opengl running on my remote target (without monitor so no X server). My target is a NVIDIA TK1 board. 

I have no issue of running opengl gstreamer pipeline on my another target which can connect to a hdmi output. 

It seems that wayland/EGL is the right direction to go but I have issue of creating a wayland display. wl_display_connect() will just failed. 

My question is does wayland work without a physical display? 

If it does, should I run a wayland server and how?

Any advice about how to get windowless opengl rendering with gstreamer running in general? 

Here is my pipeline and error message:

Insider function gst_gl_display_new() gst_gl_display_wayland_new will just failed and a egl display will be created. gst_gl_window_new() will fail as wayland window is only able to connect a wayland display. 

-------------------------------------------------------------

 gst-launch-1.0 videotestsrc num-buffers=600 is-live=true do-timestamp=true ! 'video/x-raw, format=YUY2, width=1920, height=1080, framerate=30/1' ! glupload ! glcolorconvert ! gldownload ! 'video/x-raw, format=BGRx, width=1920, height=1080, framerate=30/1' ! fakesink

Setting pipeline to PAUSED ...

error: XDG_RUNTIME_DIR not set in the environment.

0:00:00.186603249  1000    0xeff60 ERROR              gldisplay gstgldisplay_wayland.c:120:gst_gl_display_wayland_new: Failed to open wayland display connection with name, '(null)'

0:00:00.205157999  1000    0xeff60 WARN               gldisplay gstgldisplay_egl.c:91:gst_gl_display_egl_new: Succesully open EGL display connection

Pipeline is live and does not need PREROLL ...

0:00:00.208342749  1000   0x14be00 FIXME                default gstutils.c:3764:gst_pad_create_stream_id_internal:<videotestsrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id

Got context from element 'gldownloadelement0': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayEGL\)\ gldisplayegl0";

Setting pipeline to PLAYING ...

New clock: GstSystemClock

0:00:00.216310583  1000   0x14be00 WARN                glwindow gstglwindow.c:298:gst_gl_window_new: =======GST_GL_HAVE_WINDOW_WAYLAND==============(null)

0:00:00.216447666  1000   0x14be00 WARN                glwindow gstglwindow.c:325:gst_gl_window_new: Could not create window. user specified (null), creating dummy window

0:00:00.218048499  1000 0xb5b02630 WARN               glcontext gstglcontext.c:1161:gst_gl_context_create_thread:<glcontextegl0> Failed to create context

0:00:00.218346166  1000   0x14be00 WARN            glbasefilter gstglbasefilter.c:445:gst_gl_base_filter_decide_allocation:<gldownloadelement0> error: Failed to create a OpenGL context: EGL_BAD_CONFIG

0:00:00.218836583  1000   0x14be00 WARN           basetransform gstbasetransform.c:1042:gst_base_transform_do_bufferpool:<gldownloadelement0> Subclass failed to decide allocation

0:00:00.219770666  1000   0x14be00 WARN                glwindow gstglwindow.c:298:gst_gl_window_new: =======GST_GL_HAVE_WINDOW_WAYLAND==============(null)

0:00:00.219852166  1000   0x14be00 WARN                glwindow gstglwindow.c:325:gst_gl_window_new: Could not create window. user specified (null), creating dummy window

ERROR: from element /GstPipeline:pipeline0/GstGLDownloadElement:gldownloadelement0: Failed to create a OpenGL context: EGL_BAD_CONFIG

Additional debug info:

gstglbasefilter.c(445): gst_gl_base_filter_decide_allocation (): /GstPipeline:pipeline0/GstGLDownloadElement:gldownloadelement0

Execution ended after 0:00:00.007189084

Setting pipeline to PAUSED ...

Setting pipeline to READY ...

0:00:00.221281416  1000 0xb5b023b0 WARN               glcontext gstglcontext.c:1161:gst_gl_context_create_thread:<glcontextegl1> Failed to create context

0:00:00.221553916  1000   0x14be00 WARN            glbasefilter gstglbasefilter.c:445:gst_gl_base_filter_decide_allocation:<gldownloadelement0> error: Failed to create a OpenGL context: EGL_BAD_CONFIG

0:00:00.221845749  1000   0x14be00 WARN           basetransform gstbasetransform.c:1042:gst_base_transform_do_bufferpool:<gldownloadelement0> Subclass failed to decide allocation

0:00:00.222047499  1000   0x14be00 WARN                GST_PADS gstpad.c:4080:gst_pad_peer_query:<glcolorconvertelement0:src> could not send sticky events

0:00:00.222311749  1000   0x14be00 WARN                glwindow gstglwindow.c:298:gst_gl_window_new: =======GST_GL_HAVE_WINDOW_WAYLAND==============(null)

0:00:00.222398499  1000   0x14be00 WARN                glwindow gstglwindow.c:325:gst_gl_window_new: Could not create window. user specified (null), creating dummy window

0:00:00.223035666  1000 0xb5b022c0 WARN               glcontext gstglcontext.c:1161:gst_gl_context_create_thread:<glcontextegl2> Failed to create context

0:00:00.223251166  1000   0x14be00 WARN            glbasefilter gstglbasefilter.c:445:gst_gl_base_filter_decide_allocation:<glcolorconvertelement0> error: Failed to create a OpenGL context: EGL_BAD_CONFIG

0:00:00.223419166  1000   0x14be00 WARN           basetransform gstbasetransform.c:1042:gst_base_transform_do_bufferpool:<glcolorconvertelement0> Subclass failed to decide allocation

0:00:00.223497083  1000   0x14be00 WARN                GST_PADS gstpad.c:4080:gst_pad_peer_query:<gluploadelement0:src> could not send sticky events

0:00:00.223724583  1000   0x14be00 WARN                glwindow gstglwindow.c:298:gst_gl_window_new: =======GST_GL_HAVE_WINDOW_WAYLAND==============(null)

0:00:00.223806499  1000   0x14be00 WARN                glwindow gstglwindow.c:325:gst_gl_window_new: Could not create window. user specified (null), creating dummy window

0:00:00.224453916  1000 0xb5b02180 WARN               glcontext gstglcontext.c:1161:gst_gl_context_create_thread:<glcontextegl3> Failed to create context

0:00:00.224652999  1000   0x14be00 WARN            glbasefilter gstglbasefilter.c:445:gst_gl_base_filter_decide_allocation:<gluploadelement0> error: Failed to create a OpenGL context: EGL_BAD_CONFIG

0:00:00.224845083  1000   0x14be00 WARN           basetransform gstbasetransform.c:1042:gst_base_transform_do_bufferpool:<gluploadelement0> Subclass failed to decide allocation

0:00:00.224960833  1000   0x14be00 WARN                GST_PADS gstpad.c:4080:gst_pad_peer_query:<capsfilter0:src> could not send sticky events

----------------------------------------------------------------------------------------

Here is the log if I run ldd on my libgstopengl.so.

[root@target ~]# ldd /usr/lib/gstreamer-1.0/libgstopengl.so

libgstgl-1.0.so.0 => /usr/lib/libgstgl-1.0.so.0 (0xb6ee0000)

libgstallocators-1.0.so.0 => /usr/lib/libgstallocators-1.0.so.0 (0xb6ed6000)

libgstbadbase-1.0.so.0 => /usr/lib/libgstbadbase-1.0.so.0 (0xb6ec1000)

libgstbadvideo-1.0.so.0 => /usr/lib/libgstbadvideo-1.0.so.0 (0xb6eaf000)

libgstpbutils-1.0.so.0 => /usr/lib/libgstpbutils-1.0.so.0 (0xb6e81000)

libgstvideo-1.0.so.0 => /usr/lib/libgstvideo-1.0.so.0 (0xb6e1f000)

libgstaudio-1.0.so.0 => /usr/lib/libgstaudio-1.0.so.0 (0xb6dd6000)

liborc-0.4.so.0 => /usr/lib/liborc-0.4.so.0 (0xb6d77000)

libgsttag-1.0.so.0 => /usr/lib/libgsttag-1.0.so.0 (0xb6d45000)

libgstbase-1.0.so.0 => /usr/lib/libgstbase-1.0.so.0 (0xb6cf4000)

libgstreamer-1.0.so.0 => /usr/lib/libgstreamer-1.0.so.0 (0xb6c18000)

libgmodule-2.0.so.0 => /usr/lib/libgmodule-2.0.so.0 (0xb6c0e000)

libgobject-2.0.so.0 => /usr/lib/libgobject-2.0.so.0 (0xb6bce000)

libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0xb6afb000)

libpcre.so.1 => /usr/lib/libpcre.so.1 (0xb6ac0000)

libGLESv2.so.2 => /usr/lib/tegra-egl/libGLESv2.so.2 (0xb6aa9000)

libglapi.so.0 => /usr/lib/libglapi.so.0 (0xb6a61000)

libwayland-cursor.so.0 => /usr/lib/libwayland-cursor.so.0 (0xb6a53000)

libwayland-client.so.0 => /usr/lib/libwayland-client.so.0 (0xb6a44000)

libwayland-egl.so.1 => /usr/lib/libwayland-egl.so.1 (0xb6a3b000)

libpng16.so.16 => /usr/lib/libpng16.so.16 (0xb6a12000)

libz.so.1 => /usr/lib/libz.so.1 (0xb69fb000)

libjpeg.so.8 => /usr/lib/libjpeg.so.8 (0xb69cd000)

libEGL.so.1 => /usr/lib/tegra-egl/libEGL.so.1 (0xb6942000)

libpthread.so.0 => /lib/libpthread.so.0 (0xb6926000)

libX11-xcb.so.1 => /usr/lib/libX11-xcb.so.1 (0xb691d000)

libX11.so.6 => /usr/lib/libX11.so.6 (0xb682e000)

libxcb-dri2.so.0 => /usr/lib/libxcb-dri2.so.0 (0xb6824000)

libxcb-xfixes.so.0 => /usr/lib/libxcb-xfixes.so.0 (0xb6818000)

libxcb-render.so.0 => /usr/lib/libxcb-render.so.0 (0xb680a000)

libxcb-shape.so.0 => /usr/lib/libxcb-shape.so.0 (0xb6800000)

libxcb.so.1 => /usr/lib/libxcb.so.1 (0xb67e4000)

libXau.so.6 => /usr/lib/libXau.so.6 (0xb67da000)

libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0xb67ce000)

libgbm.so.1 => /usr/lib/libgbm.so.1 (0xb67bd000)

libwayland-server.so.0 => /usr/lib/libwayland-server.so.0 (0xb67ac000)

libffi.so.6 => /usr/lib/libffi.so.6 (0xb679f000)

librt.so.1 => /lib/librt.so.1 (0xb6790000)

libexpat.so.1 => /usr/lib/libexpat.so.1 (0xb6771000)

libdl.so.2 => /lib/libdl.so.2 (0xb6766000)

libdrm.so.2 => /usr/lib/libdrm.so.2 (0xb675d000)

libm.so.6 => /lib/libm.so.6 (0xb66f0000)

libc.so.6 => /lib/libc.so.6 (0xb6607000)

libnvidia-glsi.so.21.5 => /usr/lib/tegra/libnvidia-glsi.so.21.5 (0xb659e000)

libnvrm.so => /usr/lib/tegra/libnvrm.so (0xb6584000)

libnvrm_graphics.so => /usr/lib/tegra/libnvrm_graphics.so (0xb656f000)

libnvos.so => /usr/lib/tegra/libnvos.so (0xb6555000)

/lib/ld-linux-armhf.so.3 (0xb6f59000)

libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb6535000)

libnvidia-rmapi-tegra.so.21.5 => /usr/lib/tegra/libnvidia-rmapi-tegra.so.21.5 (0xb6523000)

Thanks in advance. 

Jake

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Matthew Waters]

No public NVidia driver currently supports wayland so OpenGL usage on the TK1 must go through X11.

You also don't need a physical display to run a X server and OpenGL does work in things like Xephyr (at least with the open drivers).

Wayland also requires a display server/compositor to connect to and can also work without a physical display e.g. there's a RDP backend to weston (the reference wayland compositor) which doesn't require a physical display.

Cheer
-Matt

On 26/08/16 22:33, Jake Zhang wrote:

I have been struggling to get gstreamer opengl running on my remote target (without monitor so no X server). My target is a NVIDIA TK1 board. 

I have no issue of running opengl gstreamer pipeline on my another target which can connect to a hdmi output. 

It seems that wayland/EGL is the right direction to go but I have issue of creating a wayland display. wl_display_connect() will just failed. 

My question is does wayland work without a physical display? 

If it does, should I run a wayland server and how?

Any advice about how to get windowless opengl rendering with gstreamer running in general? 

Here is my pipeline and error message:

Insider function gst_gl_display_new() gst_gl_display_wayland_new will just failed and a egl display will be created. gst_gl_window_new() will fail as wayland window is only able to connect a wayland display. 

-------------------------------------------------------------

 gst-launch-1.0 videotestsrc num-buffers=600 is-live=true do-timestamp=true ! 'video/x-raw, format=YUY2, width=1920, height=1080, framerate=30/1' ! glupload ! glcolorconvert ! gldownload ! 'video/x-raw, format=BGRx, width=1920, height=1080, framerate=30/1' ! fakesink

Setting pipeline to PAUSED ...

error: XDG_RUNTIME_DIR not set in the environment.

0:00:00.186603249  1000    0xeff60 ERROR              gldisplay gstgldisplay_wayland.c:120:gst_gl_display_wayland_new: Failed to open wayland display connection with name, '(null)'

0:00:00.205157999  1000    0xeff60 WARN               gldisplay gstgldisplay_egl.c:91:gst_gl_display_egl_new: Succesully open EGL display connection

Pipeline is live and does not need PREROLL ...

0:00:00.208342749  1000   0x14be00 FIXME                default gstutils.c:3764:gst_pad_create_stream_id_internal:<videotestsrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id

Got context from element 'gldownloadelement0': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayEGL\)\ gldisplayegl0";

Setting pipeline to PLAYING ...

New clock: GstSystemClock

0:00:00.216310583  1000   0x14be00 WARN                glwindow gstglwindow.c:298:gst_gl_window_new: =======GST_GL_HAVE_WINDOW_WAYLAND==============(null)

0:00:00.216447666  1000   0x14be00 WARN                glwindow gstglwindow.c:325:gst_gl_window_new: Could not create window. user specified (null), creating dummy window

0:00:00.218048499  1000 0xb5b02630 WARN               glcontext gstglcontext.c:1161:gst_gl_context_create_thread:<glcontextegl0> Failed to create context

0:00:00.218346166  1000   0x14be00 WARN            glbasefilter gstglbasefilter.c:445:gst_gl_base_filter_decide_allocation:<gldownloadelement0> error: Failed to create a OpenGL context: EGL_BAD_CONFIG

0:00:00.218836583  1000   0x14be00 WARN           basetransform gstbasetransform.c:1042:gst_base_transform_do_bufferpool:<gldownloadelement0> Subclass failed to decide allocation

0:00:00.219770666  1000   0x14be00 WARN                glwindow gstglwindow.c:298:gst_gl_window_new: =======GST_GL_HAVE_WINDOW_WAYLAND==============(null)

0:00:00.219852166  1000   0x14be00 WARN                glwindow gstglwindow.c:325:gst_gl_window_new: Could not create window. user specified (null), creating dummy window

ERROR: from element /GstPipeline:pipeline0/GstGLDownloadElement:gldownloadelement0: Failed to create a OpenGL context: EGL_BAD_CONFIG

Additional debug info:

gstglbasefilter.c(445): gst_gl_base_filter_decide_allocation (): /GstPipeline:pipeline0/GstGLDownloadElement:gldownloadelement0

Execution ended after 0:00:00.007189084

Setting pipeline to PAUSED ...

Setting pipeline to READY ...

0:00:00.221281416  1000 0xb5b023b0 WARN               glcontext gstglcontext.c:1161:gst_gl_context_create_thread:<glcontextegl1> Failed to create context

0:00:00.221553916  1000   0x14be00 WARN            glbasefilter gstglbasefilter.c:445:gst_gl_base_filter_decide_allocation:<gldownloadelement0> error: Failed to create a OpenGL context: EGL_BAD_CONFIG

0:00:00.221845749  1000   0x14be00 WARN           basetransform gstbasetransform.c:1042:gst_base_transform_do_bufferpool:<gldownloadelement0> Subclass failed to decide allocation

0:00:00.222047499  1000   0x14be00 WARN                GST_PADS gstpad.c:4080:gst_pad_peer_query:<glcolorconvertelement0:src> could not send sticky events

0:00:00.222311749  1000   0x14be00 WARN                glwindow gstglwindow.c:298:gst_gl_window_new: =======GST_GL_HAVE_WINDOW_WAYLAND==============(null)

0:00:00.222398499  1000   0x14be00 WARN                glwindow gstglwindow.c:325:gst_gl_window_new: Could not create window. user specified (null), creating dummy window

0:00:00.223035666  1000 0xb5b022c0 WARN               glcontext gstglcontext.c:1161:gst_gl_context_create_thread:<glcontextegl2> Failed to create context

0:00:00.223251166  1000   0x14be00 WARN            glbasefilter gstglbasefilter.c:445:gst_gl_base_filter_decide_allocation:<glcolorconvertelement0> error: Failed to create a OpenGL context: EGL_BAD_CONFIG

0:00:00.223419166  1000   0x14be00 WARN           basetransform gstbasetransform.c:1042:gst_base_transform_do_bufferpool:<glcolorconvertelement0> Subclass failed to decide allocation

0:00:00.223497083  1000   0x14be00 WARN                GST_PADS gstpad.c:4080:gst_pad_peer_query:<gluploadelement0:src> could not send sticky events

0:00:00.223724583  1000   0x14be00 WARN                glwindow gstglwindow.c:298:gst_gl_window_new: =======GST_GL_HAVE_WINDOW_WAYLAND==============(null)

0:00:00.223806499  1000   0x14be00 WARN                glwindow gstglwindow.c:325:gst_gl_window_new: Could not create window. user specified (null), creating dummy window

0:00:00.224453916  1000 0xb5b02180 WARN               glcontext gstglcontext.c:1161:gst_gl_context_create_thread:<glcontextegl3> Failed to create context

0:00:00.224652999  1000   0x14be00 WARN            glbasefilter gstglbasefilter.c:445:gst_gl_base_filter_decide_allocation:<gluploadelement0> error: Failed to create a OpenGL context: EGL_BAD_CONFIG

0:00:00.224845083  1000   0x14be00 WARN           basetransform gstbasetransform.c:1042:gst_base_transform_do_bufferpool:<gluploadelement0> Subclass failed to decide allocation

0:00:00.224960833  1000   0x14be00 WARN                GST_PADS gstpad.c:4080:gst_pad_peer_query:<capsfilter0:src> could not send sticky events

----------------------------------------------------------------------------------------

Here is the log if I run ldd on my libgstopengl.so.

[root@target ~]# ldd /usr/lib/gstreamer-1.0/libgstopengl.so

libgstgl-1.0.so.0 => /usr/lib/libgstgl-1.0.so.0 (0xb6ee0000)

libgstallocators-1.0.so.0 => /usr/lib/libgstallocators-1.0.so.0 (0xb6ed6000)

libgstbadbase-1.0.so.0 => /usr/lib/libgstbadbase-1.0.so.0 (0xb6ec1000)

libgstbadvideo-1.0.so.0 => /usr/lib/libgstbadvideo-1.0.so.0 (0xb6eaf000)

libgstpbutils-1.0.so.0 => /usr/lib/libgstpbutils-1.0.so.0 (0xb6e81000)

libgstvideo-1.0.so.0 => /usr/lib/libgstvideo-1.0.so.0 (0xb6e1f000)

libgstaudio-1.0.so.0 => /usr/lib/libgstaudio-1.0.so.0 (0xb6dd6000)

liborc-0.4.so.0 => /usr/lib/liborc-0.4.so.0 (0xb6d77000)

libgsttag-1.0.so.0 => /usr/lib/libgsttag-1.0.so.0 (0xb6d45000)

libgstbase-1.0.so.0 => /usr/lib/libgstbase-1.0.so.0 (0xb6cf4000)

libgstreamer-1.0.so.0 => /usr/lib/libgstreamer-1.0.so.0 (0xb6c18000)

libgmodule-2.0.so.0 => /usr/lib/libgmodule-2.0.so.0 (0xb6c0e000)

libgobject-2.0.so.0 => /usr/lib/libgobject-2.0.so.0 (0xb6bce000)

libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0xb6afb000)

libpcre.so.1 => /usr/lib/libpcre.so.1 (0xb6ac0000)

libGLESv2.so.2 => /usr/lib/tegra-egl/libGLESv2.so.2 (0xb6aa9000)

libglapi.so.0 => /usr/lib/libglapi.so.0 (0xb6a61000)

libwayland-cursor.so.0 => /usr/lib/libwayland-cursor.so.0 (0xb6a53000)

libwayland-client.so.0 => /usr/lib/libwayland-client.so.0 (0xb6a44000)

libwayland-egl.so.1 => /usr/lib/libwayland-egl.so.1 (0xb6a3b000)

libpng16.so.16 => /usr/lib/libpng16.so.16 (0xb6a12000)

libz.so.1 => /usr/lib/libz.so.1 (0xb69fb000)

libjpeg.so.8 => /usr/lib/libjpeg.so.8 (0xb69cd000)

libEGL.so.1 => /usr/lib/tegra-egl/libEGL.so.1 (0xb6942000)

libpthread.so.0 => /lib/libpthread.so.0 (0xb6926000)

libX11-xcb.so.1 => /usr/lib/libX11-xcb.so.1 (0xb691d000)

libX11.so.6 => /usr/lib/libX11.so.6 (0xb682e000)

libxcb-dri2.so.0 => /usr/lib/libxcb-dri2.so.0 (0xb6824000)

libxcb-xfixes.so.0 => /usr/lib/libxcb-xfixes.so.0 (0xb6818000)

libxcb-render.so.0 => /usr/lib/libxcb-render.so.0 (0xb680a000)

libxcb-shape.so.0 => /usr/lib/libxcb-shape.so.0 (0xb6800000)

libxcb.so.1 => /usr/lib/libxcb.so.1 (0xb67e4000)

libXau.so.6 => /usr/lib/libXau.so.6 (0xb67da000)

libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0xb67ce000)

libgbm.so.1 => /usr/lib/libgbm.so.1 (0xb67bd000)

libwayland-server.so.0 => /usr/lib/libwayland-server.so.0 (0xb67ac000)

libffi.so.6 => /usr/lib/libffi.so.6 (0xb679f000)

librt.so.1 => /lib/librt.so.1 (0xb6790000)

libexpat.so.1 => /usr/lib/libexpat.so.1 (0xb6771000)

libdl.so.2 => /lib/libdl.so.2 (0xb6766000)

libdrm.so.2 => /usr/lib/libdrm.so.2 (0xb675d000)

libm.so.6 => /lib/libm.so.6 (0xb66f0000)

libc.so.6 => /lib/libc.so.6 (0xb6607000)

libnvidia-glsi.so.21.5 => /usr/lib/tegra/libnvidia-glsi.so.21.5 (0xb659e000)

libnvrm.so => /usr/lib/tegra/libnvrm.so (0xb6584000)

libnvrm_graphics.so => /usr/lib/tegra/libnvrm_graphics.so (0xb656f000)

libnvos.so => /usr/lib/tegra/libnvos.so (0xb6555000)

/lib/ld-linux-armhf.so.3 (0xb6f59000)

libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb6535000)

libnvidia-rmapi-tegra.so.21.5 => /usr/lib/tegra/libnvidia-rmapi-tegra.so.21.5 (0xb6523000)

Thanks in advance. 

Jake

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel