I try to play audio from the video and audio file, but have strange problem with the pads linking. Here is my code:
G_BEGIN_DECLS GST_PLUGIN_STATIC_DECLARE(coreelements); GST_PLUGIN_STATIC_DECLARE(soup); GST_PLUGIN_STATIC_DECLARE(opensles); GST_PLUGIN_STATIC_DECLARE(playback); GST_PLUGIN_STATIC_DECLARE(ogg); GST_PLUGIN_STATIC_DECLARE(vorbis); GST_PLUGIN_STATIC_DECLARE(audioconvert); GST_PLUGIN_STATIC_DECLARE(theora); GST_PLUGIN_STATIC_DECLARE(videoconvert); GST_PLUGIN_STATIC_DECLARE(videotestsrc); GST_PLUGIN_STATIC_DECLARE(opengl); G_END_DECLS void MainWindow::play(){ GST_PLUGIN_STATIC_REGISTER(coreelements); GST_PLUGIN_STATIC_REGISTER(soup); GST_PLUGIN_STATIC_REGISTER(opensles); GST_PLUGIN_STATIC_REGISTER(playback); GST_PLUGIN_STATIC_REGISTER(ogg); GST_PLUGIN_STATIC_REGISTER(vorbis); GST_PLUGIN_STATIC_REGISTER(audioconvert); GST_PLUGIN_STATIC_REGISTER(theora); GST_PLUGIN_STATIC_REGISTER(videoconvert); GST_PLUGIN_STATIC_REGISTER(videotestsrc); GST_PLUGIN_STATIC_REGISTER(opengl); gst_init(NULL, NULL); play_vorbis_audio(); } GstElement *pipeline; void on_pad_added(GstElement *element, GstPad *pad, gpointer data) { GstElement* audioelem = ((GstElement **)data)[0]; GstCaps *caps = gst_pad_get_current_caps(pad); GstStructure* str = gst_caps_get_structure(caps, 0); if (g_strrstr(gst_structure_get_name(str), "audio")) { gst_caps_unref(caps); GstPad* sinkpad = gst_element_get_static_pad(audioelem, "sink"); if (gst_pad_link(pad, sinkpad) != 0) { qDebug("link audio error: \n"); } gst_object_unref(sinkpad); gst_element_set_state(pipeline, GST_STATE_PLAYING); } } void play_vorbis_audio() { GstElement *httpsrc, *demux, *dec, *audioconvert, *audiosink; GMainLoop *loop = g_main_loop_new(NULL, FALSE); pipeline = gst_pipeline_new("pipeline"); httpsrc = gst_element_factory_make("souphttpsrc", "httpsrc"); demux = gst_element_factory_make("oggdemux", "demux"); dec = gst_element_factory_make("vorbisdec", "dec"); audioconvert = gst_element_factory_make("audioconvert", "audioconvert"); audiosink = gst_element_factory_make("openslessink", "audiosink"); g_object_set(G_OBJECT(httpsrc), "location", "http://docs.gstreamer.com/media/sintel_trailer-368p.ogv", NULL); GstCaps *caps1 = gst_caps_new_simple("application/ogg", NULL); gst_bin_add_many(GST_BIN(pipeline), httpsrc, demux, dec, audioconvert, audiosink, NULL); gst_element_link_filtered(httpsrc, demux, caps1); gst_element_link_many(dec, audioconvert, audiosink, NULL); GstElement *convert_array[2]; convert_array[0] = dec; g_signal_connect(demux, "pad-added", G_CALLBACK(on_pad_added), convert_array); gst_element_set_state(pipeline, GST_STATE_PLAYING); g_main_loop_run(loop); } This pipeline works fine on Ubuntu, problem occurs on Android. If I use uri with ogv video(Theora) and audio(Vorbis) than I get this GST_DEBUG without audio playback(of course now I need only audio, without Theora): //... W libOpenSLES: Leaving Object::GetInterface (SL_RESULT_FEATURE_UNSUPPORTED) //... creating caps event application/ogg Starting with first granule 0 Starting with first granule 0 creating segment event time segment start=0:00:00.000000000, offset=0:00:00.000000000, stop=99:99:99.999999999, rate=1.000000, applied_rate=1.000000, flags=0x00, time=0:00:00.000000000, base=0:00:00.000000000, position 0:00:00.000000000, duration 99:99:99.999999999 found pad demux:sink creating caps event video/x-theora, width=(int)854, height=(int)366, framerate=(fraction)24/1, pixel-aspect-ratio=(fraction)1/1, streamheader=(buffer)< 807468656f72610302010036001700035600016e0002000000180000000100000100000100000000c0c0, 817468656f72612b000000586970682e4f7267206c69627468656f726120312e312032303039303832322028546875736e656c64612900000000, 827468656f7261becd28f7b9cd6b18b5a9494a10739ce6318c5294a42108318c62108421084000000000000000000011f4e1642e5549b47612f570b4986b95a2a9409648a1d047d399b8d66430174b2552912c94462210078391c0d06431168b05628130944621100783a1c0c8602c1509848220f0683014fb9917d69555541412d2d19190504f0f0dcdcc8c8b4b4b4a0a0a08c8c8c787878786464646450505050503c3c3c3c3c3c3c28282828282828281414141414141402100b0a101828333d0c0c0e131a3a3c370e0d1018283945380e11161d3357503e1216253a446d674d182337405168715c31404e adding pad 'src_38a00248' found pad demux:sink creating caps event audio/x-vorbis, rate=(int)48000, channels=(int)2, streamheader=(buffer)< 01766f72626973000000000280bb0000000000008038010000000000b801, 03766f726269731d000000586970682e4f7267206c6962566f726269732049203230303930373039010000001a000000454e434f4445523d66666d706567327468656f72612d302e323401, 05766f7262697321424356010000010018635429469952d24a8919739431469962924a89a5841642489d731453a939d79c6bacb9b52084101a5350290599528e526919639029059952104b492574123a279d63105b49c1d6986b8b41b61c840d9a524c29c494528a420819538c29c594524a420725740e3ae61c538e4a2841b89c73abb59696638ba974924ae724644c42482985924a07a5534e42483596d652291d7352526a41e820841042b620840d82d0905500000100c040101ab20a00500000108aa1188a028486ac020032000004a0288ee2288e233992634916101ab20a00000200100000c07014499114c9b1244bd22c4bd34451557dd5365555f6755dd7755dd7 adding pad 'src_0a053d9b' found pad dec:sink trying to link demux:src_0a053d9b and dec:sink Expected field 'channel-mask' in structure: audio/x-raw, rate=(int){ 8000, 11025, 12000, 16000, 22050, 24000, 32000, 44100 }, channels=(int)[ 1, 2 ], layout=(string)interleaved; caps are incompatible link between demux:src_0a053d9b and dec:sink failed: no common format D libandroid_test.so: ../android_test/player.cpp:16 (void on_pad_added(GstElement*, GstPad*, gpointer)): link audio error: 04-25 04:38:47.594 6992 7090 D libandroid_test.so: //... But if I use uri with only audio(Vorbis) all works fine. So, what is the problem and how can I solve it? |
On So, 2016-04-24 at 18:19 -0700, doon wrote:
> I try to play audio from the video and audio file, but have strange > problem with the pads linking Please get a full debug log, and also read what happens there right before linking fails. This is apparently a problem with caps incompatibilities and the log should point at the exact problem. You might be missing an audioresample element between the audioconvert and sink for example. -- Sebastian Dröge, Centricular Ltd · http://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel signature.asc (968 bytes) Download Attachment |
Thanks, pipeline works with audioresample! Now I try to display video on Android like on other platforms: GstElement *pipeline; GError *error = NULL; pipeline = gst_parse_launch("videotestsrc ! glimagesink", &error); if (!pipeline) { ui->label->setText("error"); return; } if(error != NULL){ qDebug("GST error: "); qDebug(error->message); } else{ qDebug("GST without errors"); } gst_element_set_state(pipeline, GST_STATE_READY); GstElement *video_sink = gst_bin_get_by_interface(GST_BIN(pipeline), GST_TYPE_VIDEO_OVERLAY); if (!video_sink) { qDebug ("Could not retrieve video sink"); return; } gst_video_overlay_set_window_handle(GST_VIDEO_OVERLAY(video_sink), this->ui->playback_widget->winId()); gst_element_set_state(pipeline, GST_STATE_PLAYING); But this code doesn't works, after gst_video_overlay_set_window_handle() function I get: F libc : Fatal signal 11 (SIGSEGV), code 1, fault addr 0x5e in tid 3154 (gstglcontext) So, I think I need to use <gst/gl/android/gstglwindow_android_egl.h>(is it right?), but how can I do this? And why I can display video by default way on other platforms(e.g. iOS) but not on Android? P.S. Full output: registered static plugin "coreelements" added static plugin "coreelements", result: 1 registered static plugin "app" added static plugin "app", result: 1 registered static plugin "gio" added static plugin "gio", result: 1 registered static plugin "videotestsrc" added static plugin "videotestsrc", result: 1 intializing child proxy interface registered static plugin "opengl" added static plugin "opengl", result: 1 parsing pipeline description 'videotestsrc ! glimagesink' creating element "videotestsrc" adding pad 'src' creating element "glimagesink" creating element "glupload" adding pad 'sink' adding pad 'src' creating element "glcolorconvert" adding pad 'sink' adding pad 'src' creating element "glcolorbalance" adding pad 'sink' adding pad 'src' trying to link element gluploadelement0:src to element glcolorconvertelement0:sink found pad gluploadelement0:src found pad glcolorconvertelement0:sink gluploadelement0 and glcolorconvertelement0 in same bin, no need for ghost pads trying to link gluploadelement0:src and glcolorconvertelement0:sink pad has no peer pad has no peer linked gluploadelement0:src and glcolorconvertelement0:sink, successful creating reconfigure event Received event on flushing pad. Discarding trying to link element glcolorconvertelement0:src to element glcolorbalance0:sink found pad glcolorconvertelement0:src found pad glcolorbalance0:sink glcolorconvertelement0 and glcolorbalance0 in same bin, no need for ghost pads trying to link glcolorconvertelement0:src and glcolorbalance0:sink pad has no peer pad has no peer linked glcolorconvertelement0:src and glcolorbalance0:sink, successful creating reconfigure event Received event on flushing pad. Discarding found pad gluploadelement0:sink trying to link sink:proxypad0 and gluploadelement0:sink linked sink:proxypad0 and gluploadelement0:sink, successful creating reconfigure event adding pad 'sink' trying to put property 'contrast' under control trying to put property 'brightness' under control trying to put property 'hue' under control trying to put property 'saturation' under control adding pad 'sink' trying to link element glcolorbalance0:src to element sink:sink found pad glcolorbalance0:src found pad sink:sink glcolorbalance0 and sink in same bin, no need for ghost pads trying to link glcolorbalance0:src and sink:sink pad has no peer linked glcolorbalance0:src and sink:sink, successful creating reconfigure event Received event on flushing pad. Discarding creating element "pipeline" linking some pad of GstVideoTestSrc named videotestsrc0 to some pad of GstGLImageSinkBin named glimagesinkbin0 (0/0) with caps "(NULL)" trying to link element videotestsrc0:(any) to element glimagesinkbin0:(any) trying to link videotestsrc0:src and glimagesinkbin0:sink videotestsrc0 and glimagesinkbin0 in same bin, no need for ghost pads trying to link videotestsrc0:src and glimagesinkbin0:sink linked videotestsrc0:src and glimagesinkbin0:sink, successful creating reconfigure event Received event on flushing pad. Discarding D libandroid_test.so: ../android_test/mainwindow.cpp:302 (void MainWindow::play()): GST without errors current NULL pending VOID_PENDING, desired next READY current NULL pending VOID_PENDING, desired next READY pad peer query failed posting need context message creating a display, user choice:(NULL) (platform: (NULL)) posting have context (0xa31b1a30) message with display (0xa3259678) pad peer query failed posting need context message creating a context for display <gldisplayegl0>, user choice:(null) creating a window, user choice:(null) Attempting to create opengl context. user chosen api(s) (any), compiled api support (gles2) display api (opengl opengl3 gles2) egl initialized, version: 1.4 Bound OpenGL|ES D : HostConnection::get() New Host Connection established 0xa31b7130, tid 3154 config set: 2, 1 gl context created: 2737534912 surface created created context available GL APIs: gles2 GL_VERSION: OpenGL ES 2.0 (3.0 Mesa 11.0.2) GL_SHADING_LANGUAGE_VERSION: OpenGL ES GLSL ES 1.0.17 GL_VENDOR: Google (Intel Open Source Technology Center) GL_RENDERER: Android Emulator OpenGL ES Translator (Mesa DRI Intel(R) HD Graphics 5500 (Broadwell GT2) ) No debugging support available gl thread running gl thread created completed state change to READY notifying about state-changed NULL to READY (VOID_PENDING pending) child 'sink' changed state to 2(READY) successfully current NULL pending VOID_PENDING, desired next READY found context (0xa31b1980) in downstream query pad peer query failed pad peer query failed posting need context message completed state change to READY notifying about state-changed NULL to READY (VOID_PENDING pending) child 'glcolorbalance0' changed state to 2(READY) successfully current NULL pending VOID_PENDING, desired next READY found context (0xa318d2c0) in downstream query pad peer query failed pad peer query failed posting need context message completed state change to READY notifying about state-changed NULL to READY (VOID_PENDING pending) child 'glcolorconvertelement0' changed state to 2(READY) successfully current NULL pending VOID_PENDING, desired next READY found context (0xa31b18f0) in downstream query pad peer query failed pad peer query failed posting need context message completed state change to READY notifying about state-changed NULL to READY (VOID_PENDING pending) child 'gluploadelement0' changed state to 2(READY) successfully completed state change to READY notifying about state-changed NULL to READY (VOID_PENDING pending) child 'glimagesinkbin0' changed state to 2(READY) successfully current NULL pending VOID_PENDING, desired next READY completed state change to READY notifying about state-changed NULL to READY (VOID_PENDING pending) child 'videotestsrc0' changed state to 2(READY) successfully completed state change to READY notifying about state-changed NULL to READY (VOID_PENDING pending) current READY pending VOID_PENDING, desired next PAUSED current READY pending VOID_PENDING, desired next PAUSED child 'sink' is changing state asynchronously to PAUSED current READY pending VOID_PENDING, desired next PAUSED completed state change to PAUSED notifying about state-changed READY to PAUSED (VOID_PENDING pending) child 'glcolorbalance0' changed state to 3(PAUSED) successfully current READY pending VOID_PENDING, desired next PAUSED completed state change to PAUSED notifying about state-changed READY to PAUSED (VOID_PENDING pending) child 'glcolorconvertelement0' changed state to 3(PAUSED) successfully current READY pending VOID_PENDING, desired next PAUSED completed state change to PAUSED notifying about state-changed READY to PAUSED (VOID_PENDING pending) child 'gluploadelement0' changed state to 3(PAUSED) successfully child 'glimagesinkbin0' is changing state asynchronously to PAUSED current READY pending VOID_PENDING, desired next PAUSED seeking: time segment start=0:00:00.000000000, offset=0:00:00.000000000, stop=99:99:99.999999999, rate=1.000000, applied_rate=1.000000, flags=0x00, time=0:00:00.000000000, base=0:00:00.000000000, position 0:00:00.000000000, duration 99:99:99.999999999 setting stream lock 0xa319a08c on task 0xa1d57028 created task 0xa1d57028 completed state change to PAUSED notifying about state-changed READY to PAUSED (VOID_PENDING pending) child 'videotestsrc0' changed state to 3(PAUSED) successfully no such pad 'sink' in element "videotestsrc0" Creating random stream-id, consider implementing a deterministic way of creating a stream-id creating caps event video/x-raw, format=(string)RGBA, width=(int)320, height=(int)240, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive creating caps event video/x-raw(memory:GLMemory), width=(int)320, height=(int)240, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)RGBA, texture-target=(string)2D reuse caps creating caps event video/x-raw(memory:GLMemory), width=(int)320, height=(int)240, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)RGBA, texture-target=(string)2D reuse caps creating caps event video/x-raw(memory:GLMemory), width=(int)320, height=(int)240, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, format=(string)RGBA, texture-target=(string)2D No value transform to serialize field 'params' of type 'GstAllocationParams' Expected field 'gl-allocation-params' in structure: GstBufferPoolConfig, caps=(GstCaps)"video/x-raw\(memory:GLMemory\)\,\ width\=\(int\)320\,\ height\=\(int\)240\,\ framerate\=\(fraction\)30/1\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ format\=\(string\)RGBA\,\ texture-target\=\(string\)2D", size=(uint)307200, min-buffers=(uint)0, max-buffers=(uint)0, allocator=(GstAllocator)"NULL", params=(GstAllocationParams)NULL, options=(string)< GstBufferPoolOptionGLSyncMeta >; No value transform to serialize field 'params' of type 'GstAllocationParams' Expected field 'gl-allocation-params' in structure: GstBufferPoolConfig, caps=(GstCaps)"video/x-raw\(memory:GLMemory\)\,\ width\=\(int\)320\,\ height\=\(int\)240\,\ framerate\=\(fraction\)30/1\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ format\=\(string\)RGBA\,\ texture-target\=\(string\)2D", size=(uint)307200, min-buffers=(uint)2, max-buffers=(uint)0, allocator=(GstAllocator)"NULL", params=(GstAllocationParams)NULL, options=(string)< GstBufferPoolOptionGLSyncMeta >; No value transform to serialize field 'params' of type 'GstAllocationParams' Expected field 'gl-allocation-params' in structure: GstBufferPoolConfig, caps=(GstCaps)"video/x-raw\,\ format\=\(string\)RGBA\,\ width\=\(int\)320\,\ height\=\(int\)240\,\ framerate\=\(fraction\)30/1\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive", size=(uint)307200, min-buffers=(uint)0, max-buffers=(uint)0, allocator=(GstAllocator)"NULL", params=(GstAllocationParams)NULL, options=(string)< GstBufferPoolOptionGLSyncMeta, GstBufferPoolOptionGLTextureTarget2D >; No value transform to serialize field 'params' of type 'GstAllocationParams' Expected field 'gl-allocation-params' in structure: GstBufferPoolConfig, caps=(GstCaps)"video/x-raw\,\ format\=\(string\)RGBA\,\ width\=\(int\)320\,\ height\=\(int\)240\,\ framerate\=\(fraction\)30/1\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive", size=(uint)307200, min-buffers=(uint)1, max-buffers=(uint)0, allocator=(GstAllocator)"NULL", params=(GstAllocationParams)NULL, options=(string)< GstBufferPoolOptionGLSyncMeta, GstBufferPoolOptionGLTextureTarget2D, GstBufferPoolOptionVideoMeta >; creating segment event time segment start=0:00:00.000000000, offset=0:00:00.000000000, stop=99:99:99.999999999, rate=1.000000, applied_rate=1.000000, flags=0x00, time=0:00:00.000000000, base=0:00:00.000000000, position 0:00:00.000000000, duration 99:99:99.999999999 marking pending DISCONT |
On Do, 2016-04-28 at 18:42 -0700, doon wrote:
> Thanks, pipeline works with audioresample! > Now I try to display video on Android like on other platforms: > [...] > But this code doesn't works, after gst_video_overlay_set_window_handle() > function I get: > F libc : Fatal signal 11 (SIGSEGV), code 1, fault addr 0x5e in tid 3154 > (gstglcontext) > * > So, I think I need to use (is it > right?), but how can I do this? And why I can display video by default way > on other platforms(e.g. iOS) but not on Android?* you get the window handle. How do you get the window handle? See this example code: https://cgit.freedesktop.org/~slomo/gst-sdk-tutorials/tree/gst-sdk/tutorials/android-tutorial-3/jni/tutorial-3.c#n270 https://cgit.freedesktop.org/~slomo/gst-sdk-tutorials/tree/gst-sdk/tutorials/android-tutorial-3/src/com/gst_sdk_tutorials/tutorial_3/Tutorial3.java#n127 -- Sebastian Dröge, Centricular Ltd · http://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel signature.asc (968 bytes) Download Attachment |
This post was updated on .
This string describes how I get a window handle: gst_video_overlay_set_window_handle(GST_VIDEO_OVERLAY(video_sink), this->ui->playback_widget->winId()); //playback_widget - QOpenGLWidget, I also tried some other output widgets And same code works on all other platforms, but not on Android. I know, I need to get the pointer to ANativeWindow, but how can I get this with Qt on Android? Any help or tip will be very useful. |
On So, 2016-05-01 at 18:34 -0700, doon wrote:
> This string describes how I get a window handle: > gst_video_overlay_set_window_handle(GST_VIDEO_OVERLAY(video_sink), > this->ui->playback_widget->winId()); > //playback_widget - QOpenGLWidget, I also tried some other output widgets > > And same code works on all other platforms, but not on Android. > I know, I need to get the pointer to ANativeWindow, but how can I get this > with Qt on Android? Any help or tip will be very usefull. This does not describe anything really and your question is more a Qt question than a GStreamer one. You need an ANativeWindow in any case. You'll have to understand what winId exactly returns on Android, it might for example be a Java Surface object, from which you can then get the ANativeWindow via NDK API. -- Sebastian Dröge, Centricular Ltd · http://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel signature.asc (968 bytes) Download Attachment |
Moderator on Qt forum told me that I need to "build Qt's GStreamer backend for Android"(you can check the dialog https://forum.qt.io/topic/66624/display-gstreamer-video-on-android/) but I even don't know what is it and how I can build it and connect with GStreamer. Can you help with this? |
On Di, 2016-05-03 at 19:15 -0700, doon wrote:
> Moderator on Qt forum told me that I need to "build Qt's GStreamer backend > for Android"(you can check the dialog > https://forum.qt.io/topic/66624/display-gstreamer-video-on-android/) but I > even don't know what is it and how I can build it and connect with > GStreamer. Can you help with this? I don't know what Qt GStreamer backend he's talking about, and QtGStreamer most likely does not work on Android yet without further changes. Let us know if anything he says there leads you to a working application. The real question however is what QWidget::winid() is actually returning on Android. In any case, the QPlatformScreen / QAndroidPlatformScreen seems to be that thing that holds the ANativeWindow in Qt on Android. That might be helpful or not. Most likely it will be easier btw to just use glimagesink or in general libgstgl to render video into Qt via OpenGL ES, instead of trying to directly render to an ANativeWindow. -- Sebastian Dröge, Centricular Ltd · http://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel signature.asc (968 bytes) Download Attachment |
Free forum by Nabble | Edit this page |