Hello everyone,
I am trying to decode and play a video using vaapidecode plugin. I follow this pipeline : gst-launch-1.0 filesrc location=/home/perso/video.mp4 ! qtdemux ! vaapidecode ! appsink When I use autovideosink, the video is correctly played, and the CPU usage is very low (about 15%). The video is H264 encoded, and has a UHD definition (the screen is also UHD, that's why CPU usage is important). Now, in my code, I don't know why I can't paint anything. Actually, according to `gst-inspect-1.0 vaapidecode`, it has several kind of pad caps, but I can't figure out which one is used. I think that if I knew which one is used, I could set correctly the buffer type on SDL side. Do you know how to get this value or how to change it ? I'd like to be able to print "format used is : YV12", or "RGBA" You will find attached to this post my code that tries to run a Window in which the video should be displayed. Also, can anyone explain me what is exactly a GstSample ? I've seen it used as the buffer, but there's also the GstBuffer type. I'm a little bit confused Thanks! |
Your code was probably scrubbed from the mailinglist.
You can run gst-launch-1.0 <b>-vm</b> filesrc location=/home/perso/video.mp4 ! qtdemux ! vaapidecode ! appsinkto get an idea of the caps in use. In code, you could use the notify::caps signal on a GstPad to figure out which caps are in use. W.r.t. GstSample vs GstBuffer, you should find everything you need in the linked docs. |
Yes, it's also what I think. Could I post it here ? Thanks, I now have an idea about the format used whith autovideosink. Do you know how I can set the caps ? Whenever I try to create a caps : GstCaps * caps = gst_caps_new_simple("video/x-raw", "format, G_TYPE_STRING, "RGBA", "width", G_TYPE_INT, 1920, "height", G_TYPE_INT, 1080, NULL); I get a Segfault. I wanted to set this either by linking vaapidecode to appsink using this filter, or by setting the caps on the vaapidecode src pad. I'm still confused. Let me explain. I understand that when appsink has a new frame, it sends "new-sample" signal. In the callback used for this signal, I'm able to retrieve this sample, but then I don't know what to do with it. As i said, some uses the sample as the data to paint, but I'm not sure this is the right to do. I know that I can get the buffer from this sample, but I can't find where I can then get the pointer to the data, as GstBuffer is a container for GStreamer. |
I have added glupload between vaapidecode and appsink, but the CPU usage is increasing up to 60%, so it sounds like it converts YV12 format to RGB on CPU. Isn't there anyway to use the RGB format provided by vaapidecode as GstVideoGLTextureUploadMeta directly in order to stay on the GPU and never copy any frame to CPU ? And if I follow the idea of using gstvideometa, can someone provide me a little example on how to set the current frame as a texture for OpenGL ? I have to say that the documentation about it introduces a lot of new types, and I'm a bit confused |
Consider adding a capsfilter element after your decoder and setting its "caps" property to the caps you like instead of trying to force the caps on the vaapidecodebin src pad.
Unfortunately I do not know that much about VAAPI apart from that you have to very concise to get it to do what you want. Perhaps @Victor Jaquez can help you out here. Also, this thread might be of help to you(?). |
In reply to this post by Zelnes
Hi,
Probably you're using old version of gstreamer-vaapi, I guess it's 1.8 or under it. I recommend you use higher version of gstreamer-vaapi, 1.10 at least, 1.12 highly recommended, if you want to manipulate GL buffer from vaapi decoder. Because there was important bug-fix and optimization for GLX/EGL in gstreamer-vaapi between 1.8 and 1.10. https://bugzilla.gnome.org/show_bug.cgi?id=769293 https://bugzilla.gnome.org/show_bug.cgi?id=766206 Then, I believe you can use this pipeline with GstVideoGLTextureUploadMeta negotiated without capsfilter. gst-launch-1.0 -v filesrc location=/home/perso/video.mp4 ! qtdemux ! vaapidecodebin ! glupload ! appsink (Note that this uses vaapidecodebin since vaapidecode is gone in 1.10) In addition, probably you may want to choose GLX or EGL by env variable GST_GL_PLATFORM=glx or GST_GL_PLATFORM=egl. |
This post was updated on .
That's what I intend to do, but as I said previously, I can't even create any caps using `gst_caps_new_simple` because it SEGFAULT all the time. It's funny that you found my version and told me to use 1.12, because I've already opened a thread about that Let us suppose that the output is GstVideoGLTextureUploadMeta (the type given by gst-inspect-1.0 for vaapidecode), how am I suppose to deal with that ? I'm probably going to try again for the 1.12 installation --- EDIT --- As I said earlier, I already tried glupload element, but the CPU usage grows up to 60%, what is not acceptable in the context I need it --- EDIT 2 --- I've just solved my problem with Gstreamer 1.12, I am now going to see it's possibilities ! --- EDIT 3 --- Finally, it's not working anymore with Gstreamer 1.12 (you can follow my post about that) |
In reply to this post by Zelnes
On 06/12/17 at 04:32am, Zelnes wrote:
> Arjen Veenhuizen wrote > > Your code was probably scrubbed from the mailinglist. > > Yes, it's also what I think. Could I post it here ? > > > Arjen Veenhuizen wrote > > You can run > > > to get an idea of the caps in use. In code, you could use the notify::caps > > signal on a GstPad to figure out which caps are in use. > > Thanks, I now have an idea about the format used whith autovideosink. Do you > know how I can set the caps ? Whenever I try to create a caps : > GstCaps * caps = gst_caps_new_simple("video/x-raw", "format, G_TYPE_STRING, > "RGBA", "width", G_TYPE_INT, 1920, "height", G_TYPE_INT, 1080, NULL); > I get a Segfault. I wanted to set this either by linking vaapidecode to > appsink using this filter, or by setting the caps on the vaapidecode src > pad. If you are getting a segmentation fault when creating a GstCaps instances, that means that you have severe problems in you development environment. Probably you're mixing libraries in run time. > > > Arjen Veenhuizen wrote > > W.r.t. > > GstSample > > <https://developer.gnome.org/gstreamer/stable/gstreamer-GstSample.html> > > vs > > GstBuffer > > <https://developer.gnome.org/gstreamer/stable/gstreamer-GstBuffer.html> > > , you should find everything you need in the linked docs. > > I'm still confused. Let me explain. > I understand that when appsink has a new frame, it sends "new-sample" > signal. In the callback used for this signal, I'm able to retrieve this > sample, but then I don't know what to do with it. As i said, some uses the > sample as the data to paint, but I'm not sure this is the right to do. I > know that I can get the buffer from this sample, but I can't find where I > can then get the pointer to the data, as GstBuffer is a container for > GStreamer. > > > > -- > View this message in context: http://gstreamer-devel.966125.n4.nabble.com/Vaapidecode-Format-and-OpenGL-tp4683302p4683305.html > Sent from the GStreamer-devel mailing list archive at Nabble.com. > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Now I am able to create GstCaps, it's okay, but I can't set it between vaapidecodebin and appsink, or I could not link them. Do you have a piece of code showing how to draw a frame on GPU with OpenGL, with a frame coming directly from GPU (with vaapidecodebin) and which did not go to the CPU ? I have found so many different ways, none of them worked, may be because they are out of date. I just want an access to the GPU data, in order to process it with OpenGL and the draw it on screen with SDL and OpenGL (textures). I know it is possible, but I can't figure out why my attempts all fails.
|
Is there anyone that knows if it is possible please ? |
In reply to this post by Zelnes
On 06/19/17 at 01:23am, Zelnes wrote:
> Now I am able to create GstCaps, it's okay, but I can't set it between > vaapidecodebin and appsink, or I could not link them. > > Do you have a piece of code showing how to draw a frame on GPU with OpenGL, > with a frame coming directly from GPU (with vaapidecodebin) and which did > not go to the CPU ? > I have found so many different ways, none of them worked, may be because > they are out of date. I just want an access to the GPU data, in order to > process it with OpenGL and the draw it on screen with SDL and OpenGL > (textures). > > I know it is possible, but I can't figure out why my attempts all fails. These examples might help you https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/gl/sdl vmjl _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Hi, Thanks for your examples, I had already seen them but this time I tried myself step by step. I have a working window, in which I displayed a red texture created with OpenGL. I built my pipeline step by step, to be sure that it works, and I think it is correct. To make it work, I create an empty new pipeline, to which I add my real pipeline, held in a 'GstElement* content' (otherwise it didn't work). Now, I encounter two issues : - If I unref content, I got "Trying to dispose object 'pipeline0', but it still has a parent 'gstreamer_pipeline'" and the program runs but does nothing - If I do not unref content, the program segfaults. What I've seen is that in both cases, my sync_bus_callback() function is called for STATE_CHANGED messages, but it asks for context only in the second situation (3 times : gst.vaapi.Display, then gst.gl.GLDisplay and then gst.gl.app_context). Is that normal that it calls three times for the context ? In the first case, it runs but nothing happens. My new_sample_callback() is never called. You can find the code source here : https://codeshare.io/2WeLNE |
You should not unref it there absolutely. It's your pipeline that you want to get working. Because you unref it and it's working as it says "the program runs but does nothing" Actually I tried your code with a bit modification to make it built. And I see it's working fine here(each new sample msg is printed). I tried on the master. |
That's also what I thought, that I shouldn't unref it. Could you please tell me what modifications did you do to make it work ? I still can't get it to work... If you modified the code here https://codeshare.io/2WeLNE, I did not see any change (I did a 'diff myfile.cpp codeshare.cpp' but saw nothing important). It's not because of my Intel hardware ? I saw that there were issues on this kind of hardware |
I just modified to build it just like file path and undeclared variables(argc_gstreamer_options..etc)... I can't get to know your problem but I suspect GL stuff on your machine. I think you should try to debug with gdb. |
Hm... I'll come back on this problem later, I have other things to do more important before. I didn't think it could come from my machine... Anyway, thank you for having taken the time to take a look at my code and pointing me out where the problem could come from |
This post was updated on .
In reply to this post by Hyunjun Ko
I'm back, and I have tested GL debug. Unfortunately, I don't see any problem with my installations. I've installed Gstreamer on my other machine, and tried to run my program in gst-master sub-shell, but I'm encountering the same issue. I already use GL for other purpose, and it works fine, but with Gstreamer I can't get anything. Here are some informations about my computers : The first one with Intel GPU (Mint 16.04):
If you can help me, I would be grateful. And if you need extra informations, just ask... -- EDIT -- I forgot to send you valgrind's output : ==5252== Thread 4 gstglcontext: ==5252== Invalid read of size 8 ==5252== at 0x68AA854: ??? (in /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1.2.0) ==5252== by 0x687C4E1: ??? (in /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1.2.0) ==5252== by 0x6B12631: _create_context_with_flags (gstglcontext_glx.c:180) ==5252== by 0x6B12631: gst_gl_context_glx_create_context (gstglcontext_glx.c:261) ==5252== by 0x6AED6EF: gst_gl_context_create_thread (gstglcontext.c:1207) ==5252== by 0x77A5BB4: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.4800.2) ==5252== by 0x7FE56B9: start_thread (pthread_create.c:333) ==5252== by 0x83023DC: clone (clone.S:109) ==5252== Address 0x614058 is not stack'd, malloc'd or (recently) free'd ==5252== ==5252== ==5252== Process terminating with default action of signal 11 (SIGSEGV) ==5252== Access not within mapped region at address 0x614058 ==5252== at 0x68AA854: ??? (in /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1.2.0) ==5252== by 0x687C4E1: ??? (in /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1.2.0) ==5252== by 0x6B12631: _create_context_with_flags (gstglcontext_glx.c:180) ==5252== by 0x6B12631: gst_gl_context_glx_create_context (gstglcontext_glx.c:261) ==5252== by 0x6AED6EF: gst_gl_context_create_thread (gstglcontext.c:1207) ==5252== by 0x77A5BB4: ??? (in /lib/x86_64-linux-gnu/libglib-2.0.so.0.4800.2) ==5252== by 0x7FE56B9: start_thread (pthread_create.c:333) ==5252== by 0x83023DC: clone (clone.S:109) |
Free forum by Nabble | Edit this page |