render to egl texture

[Joel Winarske-2]

Is https://github.com/freedesktop/gstreamer-gst-plugins-gl/blob/master/tests/examples/clutter still the recommended pattern for rendering to an EGL texture?

Thanks,

Joel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Matthew Waters]

No.

clutter has not been recommended for many years.  gst-plugins-gl neither for many more.  gst-plugins-gl has been migrated into gst-plugins-bad as can be seen from the latest commit on that repo: https://github.com/freedesktop/gstreamer-gst-plugins-gl/commit/bedade404ec82432742a901c663f18dfaa24356f) and then promoted to gst-plugins-base and is available as the libgstgl-1.0 library.

Cheers
-Matt

On 9/3/21 8:59 am, Joel Winarske wrote:

Is https://github.com/freedesktop/gstreamer-gst-plugins-gl/blob/master/tests/examples/clutter still the recommended pattern for rendering to an EGL texture?

Thanks,

Joel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Joel Winarske-2]

Thank you for that.

What is the current recommended pattern for rendering to a GL texture which gets consumed by a shared context?  The shared context handles the rendering.

Cheers,

Joel

On Mon, Mar 8, 2021 at 4:32 PM Matthew Waters <[hidden email]> wrote:

No.

clutter has not been recommended for many years.  gst-plugins-gl neither for many more.  gst-plugins-gl has been migrated into gst-plugins-bad as can be seen from the latest commit on that repo: https://github.com/freedesktop/gstreamer-gst-plugins-gl/commit/bedade404ec82432742a901c663f18dfaa24356f) and then promoted to gst-plugins-base and is available as the libgstgl-1.0 library.

Cheers
-Matt

On 9/3/21 8:59 am, Joel Winarske wrote:

Is https://github.com/freedesktop/gstreamer-gst-plugins-gl/blob/master/tests/examples/clutter still the recommended pattern for rendering to an EGL texture?

Thanks,

Joel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Joel Winarske-2]

I'm figuring a pipeline like this:

uridecodebin uri=file:///usr/local/share/assets/video.mp4 ! video/x-raw(memory:GLMemory),format=RGBA,texture-target=2D ! glimagesink

To get the texture id I see a pattern in the cube example of attaching callback to "client-draw" of glimagesink, then mapping the video buffer which provides access to the texture id.  Is this the only way to access the texture id?

Thanks,

Joel

On Mon, Mar 8, 2021 at 4:59 PM Joel Winarske <[hidden email]> wrote:

Thank you for that.

What is the current recommended pattern for rendering to a GL texture which gets consumed by a shared context?  The shared context handles the rendering.

Cheers,

Joel

On Mon, Mar 8, 2021 at 4:32 PM Matthew Waters <[hidden email]> wrote:

No.

clutter has not been recommended for many years.  gst-plugins-gl neither for many more.  gst-plugins-gl has been migrated into gst-plugins-bad as can be seen from the latest commit on that repo: https://github.com/freedesktop/gstreamer-gst-plugins-gl/commit/bedade404ec82432742a901c663f18dfaa24356f) and then promoted to gst-plugins-base and is available as the libgstgl-1.0 library.

Cheers
-Matt

On 9/3/21 8:59 am, Joel Winarske wrote:

Is https://github.com/freedesktop/gstreamer-gst-plugins-gl/blob/master/tests/examples/clutter still the recommended pattern for rendering to an EGL texture?

Thanks,

Joel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Matthew Waters]

That is one option if you're looking to use glimagesink's rendering.  If you're rendering the texture yourself, something like https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/tests/examples/gl/sdl/sdlshare.c is more appropriate.

Cheers
-Matt

On 9/3/21 1:37 pm, Joel Winarske wrote:

I'm figuring a pipeline like this:

uridecodebin uri=file:///usr/local/share/assets/video.mp4 ! video/x-raw(memory:GLMemory),format=RGBA,texture-target=2D ! glimagesink

To get the texture id I see a pattern in the cube example of attaching callback to "client-draw" of glimagesink, then mapping the video buffer which provides access to the texture id.  Is this the only way to access the texture id?

Thanks,

Joel

On Mon, Mar 8, 2021 at 4:59 PM Joel Winarske <[hidden email]> wrote:

Thank you for that.

What is the current recommended pattern for rendering to a GL texture which gets consumed by a shared context?  The shared context handles the rendering.

Cheers,

Joel

On Mon, Mar 8, 2021 at 4:32 PM Matthew Waters <[hidden email]> wrote:

No.

clutter has not been recommended for many years.  gst-plugins-gl neither for many more.  gst-plugins-gl has been migrated into gst-plugins-bad as can be seen from the latest commit on that repo: https://github.com/freedesktop/gstreamer-gst-plugins-gl/commit/bedade404ec82432742a901c663f18dfaa24356f) and then promoted to gst-plugins-base and is available as the libgstgl-1.0 library.

Cheers
-Matt

On 9/3/21 8:59 am, Joel Winarske wrote:

Is https://github.com/freedesktop/gstreamer-gst-plugins-gl/blob/master/tests/examples/clutter still the recommended pattern for rendering to an EGL texture?

Thanks,

Joel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Joel Winarske-2]

In my use case (video player) I just need to initialize the pipeline and return a texture id.

Is there a way to determine the texture id without loading a frame?

Is the texture id constant over the lifecycle of the pipeline?

On Mon, Mar 8, 2021 at 9:57 PM Matthew Waters <[hidden email]> wrote:

That is one option if you're looking to use glimagesink's rendering.  If you're rendering the texture yourself, something like https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/tests/examples/gl/sdl/sdlshare.c is more appropriate.

Cheers
-Matt

On 9/3/21 1:37 pm, Joel Winarske wrote:

I'm figuring a pipeline like this:

uridecodebin uri=file:///usr/local/share/assets/video.mp4 ! video/x-raw(memory:GLMemory),format=RGBA,texture-target=2D ! glimagesink

To get the texture id I see a pattern in the cube example of attaching callback to "client-draw" of glimagesink, then mapping the video buffer which provides access to the texture id.  Is this the only way to access the texture id?

Thanks,

Joel

On Mon, Mar 8, 2021 at 4:59 PM Joel Winarske <[hidden email]> wrote:

Thank you for that.

What is the current recommended pattern for rendering to a GL texture which gets consumed by a shared context?  The shared context handles the rendering.

Cheers,

Joel

On Mon, Mar 8, 2021 at 4:32 PM Matthew Waters <[hidden email]> wrote:

No.

clutter has not been recommended for many years.  gst-plugins-gl neither for many more.  gst-plugins-gl has been migrated into gst-plugins-bad as can be seen from the latest commit on that repo: https://github.com/freedesktop/gstreamer-gst-plugins-gl/commit/bedade404ec82432742a901c663f18dfaa24356f) and then promoted to gst-plugins-base and is available as the libgstgl-1.0 library.

Cheers
-Matt

On 9/3/21 8:59 am, Joel Winarske wrote:

Is https://github.com/freedesktop/gstreamer-gst-plugins-gl/blob/master/tests/examples/clutter still the recommended pattern for rendering to an EGL texture?

Thanks,

Joel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Matthew Waters]

There is no way to know the texture ID without uploading the frame.

The texture ID is almost never a constant value.  At least, there will probably be two textures that will be flipped between. At most, each texture id will be unique.

Cheers
-Matt

On 10/3/21 5:34 am, Joel Winarske wrote:

In my use case (video player) I just need to initialize the pipeline and return a texture id.

Is there a way to determine the texture id without loading a frame?

Is the texture id constant over the lifecycle of the pipeline?

On Mon, Mar 8, 2021 at 9:57 PM Matthew Waters <[hidden email]> wrote:

That is one option if you're looking to use glimagesink's rendering.  If you're rendering the texture yourself, something like https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/tests/examples/gl/sdl/sdlshare.c is more appropriate.

Cheers
-Matt

On 9/3/21 1:37 pm, Joel Winarske wrote:

I'm figuring a pipeline like this:

uridecodebin uri=file:///usr/local/share/assets/video.mp4 ! video/x-raw(memory:GLMemory),format=RGBA,texture-target=2D ! glimagesink

To get the texture id I see a pattern in the cube example of attaching callback to "client-draw" of glimagesink, then mapping the video buffer which provides access to the texture id.  Is this the only way to access the texture id?

Thanks,

Joel

On Mon, Mar 8, 2021 at 4:59 PM Joel Winarske <[hidden email]> wrote:

Thank you for that.

What is the current recommended pattern for rendering to a GL texture which gets consumed by a shared context?  The shared context handles the rendering.

Cheers,

Joel

On Mon, Mar 8, 2021 at 4:32 PM Matthew Waters <[hidden email]> wrote:

No.

clutter has not been recommended for many years.  gst-plugins-gl neither for many more.  gst-plugins-gl has been migrated into gst-plugins-bad as can be seen from the latest commit on that repo: https://github.com/freedesktop/gstreamer-gst-plugins-gl/commit/bedade404ec82432742a901c663f18dfaa24356f) and then promoted to gst-plugins-base and is available as the libgstgl-1.0 library.

Cheers
-Matt

On 9/3/21 8:59 am, Joel Winarske wrote:

Is https://github.com/freedesktop/gstreamer-gst-plugins-gl/blob/master/tests/examples/clutter still the recommended pattern for rendering to an EGL texture?

Thanks,

Joel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Joel Winarske-2]

I'm taking a look at the sdlshare example, and porting to Fedora 33 - wayland/egl.

After pausing the pipeline I get an eglCreateContext EGL Error of EGL_BAD_CONTEXT.  The context being passed in is shared, without a surface.

Code: https://gist.github.com/jwinarske/a518d16f18a4e0345d91027984098ec9

Log: https://gist.github.com/jwinarske/2d19e39590415fb8331af2edbeb1b984

How do I avoid the dummy window altogether?

My end goal is to simply update a texture on each frame.  Something else renders the texture.

On Tue, Mar 9, 2021 at 4:42 PM Matthew Waters <[hidden email]> wrote:

There is no way to know the texture ID without uploading the frame.

The texture ID is almost never a constant value.  At least, there will probably be two textures that will be flipped between. At most, each texture id will be unique.

Cheers
-Matt

On 10/3/21 5:34 am, Joel Winarske wrote:

In my use case (video player) I just need to initialize the pipeline and return a texture id.

Is there a way to determine the texture id without loading a frame?

Is the texture id constant over the lifecycle of the pipeline?

On Mon, Mar 8, 2021 at 9:57 PM Matthew Waters <[hidden email]> wrote:

That is one option if you're looking to use glimagesink's rendering.  If you're rendering the texture yourself, something like https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/tests/examples/gl/sdl/sdlshare.c is more appropriate.

Cheers
-Matt

On 9/3/21 1:37 pm, Joel Winarske wrote:

I'm figuring a pipeline like this:

uridecodebin uri=file:///usr/local/share/assets/video.mp4 ! video/x-raw(memory:GLMemory),format=RGBA,texture-target=2D ! glimagesink

To get the texture id I see a pattern in the cube example of attaching callback to "client-draw" of glimagesink, then mapping the video buffer which provides access to the texture id.  Is this the only way to access the texture id?

Thanks,

Joel

On Mon, Mar 8, 2021 at 4:59 PM Joel Winarske <[hidden email]> wrote:

Thank you for that.

What is the current recommended pattern for rendering to a GL texture which gets consumed by a shared context?  The shared context handles the rendering.

Cheers,

Joel

On Mon, Mar 8, 2021 at 4:32 PM Matthew Waters <[hidden email]> wrote:

No.

clutter has not been recommended for many years.  gst-plugins-gl neither for many more.  gst-plugins-gl has been migrated into gst-plugins-bad as can be seen from the latest commit on that repo: https://github.com/freedesktop/gstreamer-gst-plugins-gl/commit/bedade404ec82432742a901c663f18dfaa24356f) and then promoted to gst-plugins-base and is available as the libgstgl-1.0 library.

Cheers
-Matt

On 9/3/21 8:59 am, Joel Winarske wrote:

Is https://github.com/freedesktop/gstreamer-gst-plugins-gl/blob/master/tests/examples/clutter still the recommended pattern for rendering to an EGL texture?

Thanks,

Joel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Joel Winarske-2]

Using the below combo of variables seems to be invalid, for both Ubuntu 18.04 LTS (Wayland on Ubuntu) and Fedora 33 (Wayland is active by default).  Canonical (Ubuntu) is stating they will default with Wayland (again) in an upcoming release.

GST_GL_WINDOW=wayland

GST_GL_PLATFORM=egl

GST_GL_API=gles2

What gl test cases are expected to work via wayland/egl/gles2?  Or any test case for that matter.

Thanks,

Joel

On Thu, Mar 11, 2021 at 4:02 PM Joel Winarske <[hidden email]> wrote:

I'm taking a look at the sdlshare example, and porting to Fedora 33 - wayland/egl.

After pausing the pipeline I get an eglCreateContext EGL Error of EGL_BAD_CONTEXT.  The context being passed in is shared, without a surface.

Code: https://gist.github.com/jwinarske/a518d16f18a4e0345d91027984098ec9

Log: https://gist.github.com/jwinarske/2d19e39590415fb8331af2edbeb1b984

How do I avoid the dummy window altogether?

My end goal is to simply update a texture on each frame.  Something else renders the texture.

On Tue, Mar 9, 2021 at 4:42 PM Matthew Waters <[hidden email]> wrote:

There is no way to know the texture ID without uploading the frame.

The texture ID is almost never a constant value.  At least, there will probably be two textures that will be flipped between. At most, each texture id will be unique.

Cheers
-Matt

On 10/3/21 5:34 am, Joel Winarske wrote:

In my use case (video player) I just need to initialize the pipeline and return a texture id.

Is there a way to determine the texture id without loading a frame?

Is the texture id constant over the lifecycle of the pipeline?

On Mon, Mar 8, 2021 at 9:57 PM Matthew Waters <[hidden email]> wrote:

That is one option if you're looking to use glimagesink's rendering.  If you're rendering the texture yourself, something like https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/tests/examples/gl/sdl/sdlshare.c is more appropriate.

Cheers
-Matt

On 9/3/21 1:37 pm, Joel Winarske wrote:

I'm figuring a pipeline like this:

uridecodebin uri=file:///usr/local/share/assets/video.mp4 ! video/x-raw(memory:GLMemory),format=RGBA,texture-target=2D ! glimagesink

To get the texture id I see a pattern in the cube example of attaching callback to "client-draw" of glimagesink, then mapping the video buffer which provides access to the texture id.  Is this the only way to access the texture id?

Thanks,

Joel

On Mon, Mar 8, 2021 at 4:59 PM Joel Winarske <[hidden email]> wrote:

Thank you for that.

What is the current recommended pattern for rendering to a GL texture which gets consumed by a shared context?  The shared context handles the rendering.

Cheers,

Joel

On Mon, Mar 8, 2021 at 4:32 PM Matthew Waters <[hidden email]> wrote:

No.

clutter has not been recommended for many years.  gst-plugins-gl neither for many more.  gst-plugins-gl has been migrated into gst-plugins-bad as can be seen from the latest commit on that repo: https://github.com/freedesktop/gstreamer-gst-plugins-gl/commit/bedade404ec82432742a901c663f18dfaa24356f) and then promoted to gst-plugins-base and is available as the libgstgl-1.0 library.

Cheers
-Matt

On 9/3/21 8:59 am, Joel Winarske wrote:

Is https://github.com/freedesktop/gstreamer-gst-plugins-gl/blob/master/tests/examples/clutter still the recommended pattern for rendering to an EGL texture?

Thanks,

Joel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Matthew Waters]

This combination is fine if libgstgl is built for those platforms and your log would indicate that this is the case.

Almost all gl elements in gst-plugins-base support that combination.  glupload, gleffects, glimagesink, gltestsrc are non-exhaustive examples that support that just fine.  There are very few gl elements nowadays that struggle with the (gles2) requirement.  wayland+egl is fine and is supported by all OpenGL elements.

Further comments inline.

On 15/3/21 8:13 am, Joel Winarske wrote:

Using the below combo of variables seems to be invalid, for both Ubuntu 18.04 LTS (Wayland on Ubuntu) and Fedora 33 (Wayland is active by default).  Canonical (Ubuntu) is stating they will default with Wayland (again) in an upcoming release.

GST_GL_WINDOW=wayland

GST_GL_PLATFORM=egl

GST_GL_API=gles2

These variables will only have any effect when GStreamer is creating the necessary resources.  If you are passing in some shared resources (display or OpenGL context), GStreamer will use the values from there instead.

What gl test cases are expected to work via wayland/egl/gles2?  Or any test case for that matter.

The gtk examples support wayland, otherwise the gtkglsink or qmlglsink elements contain the necessary code for retrieving the wl_display from those respective toolkits and then passing the required GstGLDisplay to GStreamer.

Thanks,

Joel

On Thu, Mar 11, 2021 at 4:02 PM Joel Winarske <[hidden email]> wrote:

I'm taking a look at the sdlshare example, and porting to Fedora 33 - wayland/egl.

After pausing the pipeline I get an eglCreateContext EGL Error of EGL_BAD_CONTEXT.  The context being passed in is shared, without a surface.

Code: https://gist.github.com/jwinarske/a518d16f18a4e0345d91027984098ec9

Log: https://gist.github.com/jwinarske/2d19e39590415fb8331af2edbeb1b984

How do I avoid the dummy window altogether?

My end goal is to simply update a texture on each frame.  Something else renders the texture.

For wayland, you need to share the wl_display between SDL and GStreamer.  Once you retrieve the wl_display from SDL, create the necessary GstGLDisplay with gst_gl_display_wayland_new_with_display().

Cheers
-Matt

On Tue, Mar 9, 2021 at 4:42 PM Matthew Waters <[hidden email]> wrote:

There is no way to know the texture ID without uploading the frame.

The texture ID is almost never a constant value.  At least, there will probably be two textures that will be flipped between. At most, each texture id will be unique.

Cheers
-Matt

On 10/3/21 5:34 am, Joel Winarske wrote:

In my use case (video player) I just need to initialize the pipeline and return a texture id.

Is there a way to determine the texture id without loading a frame?

Is the texture id constant over the lifecycle of the pipeline?

On Mon, Mar 8, 2021 at 9:57 PM Matthew Waters <[hidden email]> wrote:

That is one option if you're looking to use glimagesink's rendering.  If you're rendering the texture yourself, something like https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/blob/master/tests/examples/gl/sdl/sdlshare.c is more appropriate.

Cheers
-Matt

On 9/3/21 1:37 pm, Joel Winarske wrote:

I'm figuring a pipeline like this:

uridecodebin uri=file:///usr/local/share/assets/video.mp4 ! video/x-raw(memory:GLMemory),format=RGBA,texture-target=2D ! glimagesink

To get the texture id I see a pattern in the cube example of attaching callback to "client-draw" of glimagesink, then mapping the video buffer which provides access to the texture id.  Is this the only way to access the texture id?

Thanks,

Joel

On Mon, Mar 8, 2021 at 4:59 PM Joel Winarske <[hidden email]> wrote:

Thank you for that.

What is the current recommended pattern for rendering to a GL texture which gets consumed by a shared context?  The shared context handles the rendering.

Cheers,

Joel

On Mon, Mar 8, 2021 at 4:32 PM Matthew Waters <[hidden email]> wrote:

No.

clutter has not been recommended for many years.  gst-plugins-gl neither for many more.  gst-plugins-gl has been migrated into gst-plugins-bad as can be seen from the latest commit on that repo: https://github.com/freedesktop/gstreamer-gst-plugins-gl/commit/bedade404ec82432742a901c663f18dfaa24356f) and then promoted to gst-plugins-base and is available as the libgstgl-1.0 library.

Cheers
-Matt

On 9/3/21 8:59 am, Joel Winarske wrote:

Is https://github.com/freedesktop/gstreamer-gst-plugins-gl/blob/master/tests/examples/clutter still the recommended pattern for rendering to an EGL texture?

Thanks,

Joel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Joel Winarske-2]

For wayland, you need to share the wl_display between SDL and GStreamer.  Once you retrieve the wl_display from SDL, create the necessary GstGLDisplay with gst_gl_display_wayland_new_with_display().

Replacing gst_gl_display_new() with gst_gl_display_wayland_new_with_display() did the trick.

Thank you

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel