gst-vaapi and opengl

[leo]

Hi,

I'm trying to use gst-vaapi to render a video in an OpenGl texture but i don't understand how it works.

I can play a video with this pipeline : filesrc -> qtdemux -> vaapidecode -> vaapisink

and I can upload/display a texture in an openGl context with a  GstVaapiDisplay/GstVaapiSurface/...

but how can i link these two parts?
I didn't find an example, can someone give me one?

Thank you,

Léo

[Nicolas Dufresne]

Le lundi 09 juillet 2012 à 06:03 -0700, leo yvin a écrit :

> Hi,
>
> I'm trying to use gst-vaapi to render a video in an OpenGl texture but i
> don't understand how it works.
>
> I can play a video with this pipeline : filesrc -> qtdemux -> vaapidecode ->
> vaapisink
>
> and I can upload/display a texture in an openGl context with a
> GstVaapiDisplay/GstVaapiSurface/...
>
> but how can i link these two parts?
> I didn't find an example, can someone give me one?

Hi Léo,

the vaapidecode element will produce buffers witch implement
GstSurfaceBuffer. This interface (part of gst-plugins-bad) let you
create a converter that will upload to the texture you want. To do so,
you may use the GstAppSink or implement your own video sink. When you
receive the first buffer, you will create a converter of type
"opengl" (the only supported type at the moment) and the ID of your
destination texture. Note that whenever the capabilities changes, you
need to renew the converter. Finally, call the upload method to upload
the new buffer to your texture.

Here is an exemple extracted from clutter-gst project:
        static void
        clutter_gst_hw_upload (ClutterGstVideoSink *sink,
                               GstBuffer           *buffer)
        {
          ClutterGstVideoSinkPrivate *priv = sink->priv;
          GstSurfaceBuffer *surface;
       
          g_return_if_fail (GST_IS_SURFACE_BUFFER (buffer));
          surface = GST_SURFACE_BUFFER (buffer);
       
          if (G_UNLIKELY (priv->converter == NULL)) {
            CoglHandle tex;
            GLuint gl_texture;
            GLenum gl_target;
            GValue value = {0};
       
            tex = clutter_texture_get_cogl_texture (priv->texture);
            cogl_texture_get_gl_texture (tex, &gl_texture, &gl_target);
            g_return_if_fail (gl_target == GL_TEXTURE_2D);
       
            g_value_init (&value, G_TYPE_UINT);
            g_value_set_uint (&value, gl_texture);
       
            priv->converter = gst_surface_buffer_create_converter (surface, "opengl", &value);
            g_return_if_fail (priv->converter);
          }
       
          gst_surface_converter_upload (priv->converter, surface);
       
          /* The texture is dirty, schedule a redraw */
          clutter_actor_queue_redraw (CLUTTER_ACTOR (priv->texture));
        }

cheers,
Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[leo]

Hi Nicolas and thanks for your help!

I'm trying to follow this example and this is encouraging (The buffer is a GST_IS_SURFACE_BUFFER and the converter seems to be created) but gst_surface_converter_upload return false...

I imagine that there is an error if GST_BUFFER_SIZE return 0?

Léo

[Nicolas Dufresne]

Le mardi 10 juillet 2012 à 07:36 -0700, leo yvin a écrit :

Hi Nicolas and thanks for your help!

I'm trying to follow this example and this is encouraging (The buffer is a
GST_IS_SURFACE_BUFFER and the converter seems to be created) but
gst_surface_converter_upload return false...

I imagine that there is an error if GST_BUFFER_SIZE return 0? 

You can check the debug logs for more information (GST_DEBUG=*vaapi*:5 if I remember). Also, you could install debug symbols for gstreamer-vaapi and libva, and produce a backtrace of when it returns false. This will help you, and will help us helping you.

cheers,
Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[leo]

Thanks again for your help!

It works!

I created an useless GstVaapiTexture with gst_vaapi_texture_new and i don't understand why but it produced an error with vaCopySurfaceGLX (resource allocation failed).

Léo

[Nicolas Dufresne]

Le mercredi 11 juillet 2012 à 01:52 -0700, leo yvin a écrit :
> I created an useless /GstVaapiTexture/ with /gst_vaapi_texture_new/
> and i
> don't understand why but it produced an error with /vaCopySurfaceGLX/
> (resource allocation failed).

I usually don't recommend using this API, but for your interest you may
be using the wrong VADisplay. At least with Intel driver, it's mandatory
that the display is unique over all your pipeline. You might be
interested adding support for GstVideoContext too. For this, in your
application, you should watch for prepare-video-context message on
synchronous bus, use gst_video_context_message_parse_prepare() to check
if it's such a message. You can then set a display, gstreamer-vaapi
support "vaapi-display" for the basic VADisplay, "gst-vaapi-display" for
the one seen in the wrapper library and the standard one if running on
X11 (x11-display and x11-display-name). See videocontext.c in
gst-plugins-bad for more information. If on X11, calling XInitThreads()
is strongly recommended, unless you know what you are doing.

cheers,
Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel