Hi guys,
I am working on an application that integrates Gstreamer pipeline into Oculus Mobile SDK (android), with the intention of rendering a video stream in form of a texture into a VR OpenGL scene. The test pipeline is simple: "videotestsrc ! video/x-raw,format=RGBA,width=800,height=600,framerate=1/1 ! glupload ! fakesink sync=1" Firstly I worked on a application that didn't use the oculus sdk, but a simple GLSurfaceView. I used context-sharing as in the sdlshare.c gstreamer example, to be able to pass texture id from appsink/fakesink to the rendering thread. I go it working, with help of Mattew Watters [1]. The problem is, that when I ported my simple code to the Oculus Mobile SDK sample (I was using their CubeWorld_SurfaceView example [2]), for some reason I cannot get it working right. The pipeline seems to be working ok, it passes me the texture IDs, but these IDs are not recognised by the main rendering thread (OVR), as if the context sharing never happened... To exemplify what I mean: I define a few textures in the OVR renderer thread and their IDs are sth like 2, 6, 7, 8 etc. Gstreamer passes me textures with ID from 1 until 20 (I buffer up to 20 frames), so actaully the ID from different threads overlap. During scene rendering if I bind gstreamer ID it will give me GL_INVALID_OPERATION (so the texture is not present in the current context) and obviously the texture renders black. If the texture number (given by the fakesink) by chance hits on of my OVR textures, the OVR texture is rendered instead of the otherwise black space. It seems clear to me that the context have separate spaces for gstreamer and the main OVR thread, even though the context sharing seems to be occurring correctly (no error/warning etc. in the catlog). Just to be sure that context sharing works whatsoever, I've created a separate dummy thread (from the main OVR thread) in which I defined a new context, sharing the main context. Textures generated in that thread are indeed visible to the OVR thread... From this I suspect that there is sth wrong on the gstreamer context-sharing side. Note: I manage the frame buffer myself trough the fakesink queue, releasing old frames when a new one is available. I have attached a context sharing function to the glupload element, called on the bus callback. Here I put a few relevant pieces from the catlog, debug level *:6 (the full log is available here: [3]): ==> Context related calls when gstreamer starts: https://hastebin.com/adunugemad.swift ==> This happend during starting of the glupload initalzation phase: https://hastebin.com/ugowasijag.rb ==> Here the bus need-context callback is handled, and my context is passed to the gluploadelement0 https://hastebin.com/halaqabayo.rb ==> This is how the the rendering (OVR thread) creates the context. Here I also create my second test thread, that would do what gstreamer does: create a new, shared context and generate some textures, that I try to access from my OVR thread. It works like a charm. https://hastebin.com/piyelexiji.php ==> this is the part in which I manage and retrieve frames from fakesink and pass to the rendering thread https://hastebin.com/dovuhowawe.cpp ================= In this log I don't see any errors or complaint, so I assume the context sharing was successful. Would I get any error/warning here if share context was rejected by the element? When I generate my own textures in a main/side OVR thread I can see: 01-21 13:19:40.442 2214-2292/com.oculus.vrcubeworldsv I/VrCubeWorld: new texture created: 7 01-21 13:19:40.494 2214-2292/com.oculus.vrcubeworldsv I/VrCubeWorld: new texture created: 8 01-21 13:19:40.509 2214-2292/com.oculus.vrcubeworldsv I/VrCubeWorld: new texture created: 9 01-21 13:19:40.511 2214-2292/com.oculus.vrcubeworldsv I/VrCubeWorld: new texture created: 10 01-21 13:19:40.513 2214-2292/com.oculus.vrcubeworldsv I/VrCubeWorld: new texture created: 11 But the gstreamer will allocate the seme IDs anyhow like here: 01-21 13:19:48.463 2214-2303/com.oculus.vrcubeworldsv V/GStreamer+glmemory: 0:00:08.980197608 0x7a8ef838f0 gstglmemorypbo.c:149:_upload_pbo_memory upload for texture id:9, with pbo 9 800x600 01-21 13:19:48.463 2214-2303/com.oculus.vrcubeworldsv V/GStreamer+glbasetexture: 0:00:08.980291801 0x7a8ef838f0 gstglmemory.c:542:gst_gl_memory_texsubimage upload for texture id:9, 800x600 01-21 13:19:48.464 2214-2303/com.oculus.vrcubeworldsv E/GStreamer+gldebug: 0:00:08.980534224 0x7a8ef838f0 gstgldebug.c:303:_gst_gl_debug_callback:<glcontextegl0> high: GL error from API id:1, Error:glBeginQueryEXT::failed to allocate CPU memory 01-21 13:19:48.464 2214-2303/com.oculus.vrcubeworldsv E/GStreamer+gldebug: 0:00:08.980761532 0x7a8ef838f0 gstgldebug.c:303:_gst_gl_debug_callback:<glcontextegl0> high: GL error from API id:148, Error:glEndQueryEXT::query name is 0 I can see a GL API error being thrown, but this was also happening in the working, non-oculus version of the app. So I don't think this is the source of the problem. The context sharing function looks like this: case GST_MESSAGE_NEED_CONTEXT: { LOG_INFO("from OpenGL: 0x%08X 0x%08X", *gl_context, *gl_display); // we got gl_display and gl_context and starting of the main loop GstElement *element = GST_ELEMENT (message->src); const gchar *name = gst_element_get_name(element); //element const gchar *context_type; gst_message_parse_context_type(message, &context_type); LOG_INFO(" Need context callback from '%s' received ",name); LOG_INFO(" Context type: %s", context_type); // GStreamer vars GstGLDisplayEGL *gst_gl_display; GstGLContext *gst_gl_context; // GStreamer context GstContext *context; GstStructure *s; // get GStreamer display from the native OpenGL EGL vars gst_gl_display = gst_gl_display_egl_new_with_egl_display(gl_display); gst_gl_context = gst_gl_context_new_wrapped(gst_gl_display, *gl_context, GST_GL_PLATFORM_EGL, GST_GL_API_GLES2); // set the context context = gst_context_new("gst.gl.app_context", TRUE); s = gst_context_writable_structure(context); gst_structure_set(s, "context", GST_GL_TYPE_CONTEXT, gst_gl_context, NULL); gst_element_set_context(GST_ELEMENT (message->src), context); if (context) gst_context_unref(context); } break; case GST_MESSAGE_HAVE_CONTEXT: { LOG_INFO("Received 'got context' message"); } I would be grateful for any help on pinpoint and solving my problem on why the gstreamer textures are not seen my the rendering thread. The whole issue is that I do not receive any particular error that I could see in the catlog or anywhere else, which would help me understand the problem. If more info/cod sharing is required I'd be more than happy to provide it. Thanks, JW PS. I use gstreamer prebuild libraries for android, v1.12.0. My target device is a 64bit arm-v8 Refs _______________________ [1] http://gstreamer-devel.966125.n4.nabble.com/Memory-allocation-error-when-trying-to-map-a-video-frame-td4683716.html [2] https://developer.oculus.com/downloads/package/oculus-mobile-sdk/1.7.0/\ [3] https://uploadfiles.io/mvc2u -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Does anybody have any suggestions on this?
Even there is not quick solution, i would be grateful for some hits on how to debug and understand the problem first? Is there a way I could confirm that a gstreamer has actually successfully created a context using provided shared context? Best, Jeremi -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
I have used GAPID tool for OpenGL debugging (which actually is quite an
awesome tool) and I have confirmed that indeed gstreamer creates a new GL context with eglCreateContext but with share_context as NULL. This means that for some reasons the context I provide, using methods shown in sdlshare and sdlshare2 examples, is ignored. <http://gstreamer-devel.966125.n4.nabble.com/file/t377800/gapid.png> This slightly narrows down the problem, but I still don't get what is the reason. Looking into the gstglcontext_egl.c I don't see a clear point in which the "other_context" could be stripped: static gboolean gst_gl_context_egl_create_context (GstGLContext * context, GstGLAPI gl_api, GstGLContext * other_context, GError ** error) { GstGLContextEGL *egl; GstGLWindow *window = NULL; EGLNativeWindowType window_handle = (EGLNativeWindowType) 0; EGLint majorVersion; EGLint minorVersion; gboolean need_surface = TRUE; guintptr external_gl_context = 0; GstGLDisplay *display; egl = GST_GL_CONTEXT_EGL (context); window = gst_gl_context_get_window (context); GST_DEBUG_OBJECT (context, "Creating EGL context"); if (other_context) { // ======= I don’t know if other_context is != NULL at this point, but it is quite possible if (gst_gl_context_get_gl_platform (other_context) != GST_GL_PLATFORM_EGL) { g_set_error (error, GST_GL_CONTEXT_ERROR, GST_GL_CONTEXT_ERROR_WRONG_CONFIG, "Cannot share context with non-EGL context"); // ======== I don’t reach this point goto failure; } external_gl_context = gst_gl_context_get_gl_context (other_context); } if ((gl_api & (GST_GL_API_OPENGL | GST_GL_API_OPENGL3 | GST_GL_API_GLES2)) == GST_GL_API_NONE) { ... // =========== OPENGL case – not relevant ... } else if (gl_api & GST_GL_API_GLES2) { gint i; try_gles2: if (!eglBindAPI (EGL_OPENGL_ES_API)) { g_set_error (error, GST_GL_CONTEXT_ERROR, GST_GL_CONTEXT_ERROR_FAILED, "Failed to bind OpenGL|ES API: %s", gst_gl_context_egl_get_error_string (eglGetError ())); goto failure; } // ============ I get this message GST_INFO ("Bound OpenGL|ES"); for (i = 0; i < G_N_ELEMENTS (gles2_versions); i++) { gint profileMask = 0; gint contextFlags = 0; if (!gst_gl_context_egl_choose_config (egl, GST_GL_API_GLES2, gles2_versions[i].major, error)) { continue; } #if defined(EGL_KHR_create_context) /* try a debug context */ contextFlags |= EGL_CONTEXT_OPENGL_DEBUG_BIT_KHR; egl->egl_context = _create_context_with_flags (egl, (EGLContext) external_gl_context, GST_GL_API_GLES2, gles2_versions[i].major, gles2_versions[i].minor, contextFlags, profileMask); if (egl->egl_context) break; /* try without a debug context */ contextFlags &= ~EGL_CONTEXT_OPENGL_DEBUG_BIT_KHR; #endif egl->egl_context = _create_context_with_flags (egl, (EGLContext) external_gl_context, GST_GL_API_GLES2, gles2_versions[i].major, gles2_versions[i].minor, contextFlags, profileMask); if (egl->egl_context) break; } egl->gl_api = GST_GL_API_GLES2; } if (egl->egl_context != EGL_NO_CONTEXT) { // ====== I get this message GST_INFO ("gl context created: %" G_GUINTPTR_FORMAT, (guintptr) egl->egl_context); } else { g_set_error (error, GST_GL_CONTEXT_ERROR, GST_GL_CONTEXT_ERROR_CREATE_CONTEXT, "Failed to create a OpenGL context: %s", gst_gl_context_egl_get_error_string (eglGetError ())); goto failure; } Just for reference, I share the current version of context setting (in the bus need-context callback): case GST_MESSAGE_NEED_CONTEXT: { // we got gl_display and gl_context and starting of the main loop GstElement *element = GST_ELEMENT (message->src); const gchar *name = gst_element_get_name(element); LOG_INFO("Element %s asks for context. We give => 0x%08x 0x%08x", name, data->egl_context, data->egl_display); const gchar *context_type; gst_message_parse_context_type (message, &context_type); g_print ("got need context %s\n", context_type); if (g_strcmp0 (context_type, GST_GL_DISPLAY_CONTEXT_TYPE) == 0) { GstContext *display_context = gst_context_new (GST_GL_DISPLAY_CONTEXT_TYPE, TRUE); gst_context_set_gl_display (display_context, data->gst_display); gst_element_set_context (GST_ELEMENT (message->src), display_context); return TRUE; } else if (g_strcmp0 (context_type, "gst.gl.app_context") == 0) { GstContext *app_context = gst_context_new ("gst.gl.app_context", TRUE); GstStructure *s = gst_context_writable_structure (app_context); gst_structure_set (s, "context", GST_GL_TYPE_CONTEXT, data->gst_context, NULL); gst_element_set_context (GST_ELEMENT (message->src), app_context); return TRUE; } break; } Any remarks would be much appreciated. Best, Jeremi -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
On 20/02/18 09:52, jeremi.wojcicki wrote:
> I have used GAPID tool for OpenGL debugging (which actually is quite an > awesome tool) and I have confirmed that indeed gstreamer creates a new GL > context with eglCreateContext but with share_context as NULL. This means > that for some reasons the context I provide, using methods shown in sdlshare > and sdlshare2 examples, is ignored. > > <http://gstreamer-devel.966125.n4.nabble.com/file/t377800/gapid.png> > > This slightly narrows down the problem, but I still don't get what is the > reason. Looking into the gstglcontext_egl.c I don't see a clear point in > which the "other_context" could be stripped: > > static gboolean > gst_gl_context_egl_create_context (GstGLContext * context, > GstGLAPI gl_api, GstGLContext * other_context, GError ** error) > { > GstGLContextEGL *egl; > GstGLWindow *window = NULL; > EGLNativeWindowType window_handle = (EGLNativeWindowType) 0; > EGLint majorVersion; > EGLint minorVersion; > gboolean need_surface = TRUE; > guintptr external_gl_context = 0; > GstGLDisplay *display; > > egl = GST_GL_CONTEXT_EGL (context); > window = gst_gl_context_get_window (context); > > GST_DEBUG_OBJECT (context, "Creating EGL context"); > > if (other_context) { > // ======= I don’t know if other_context is != NULL at this point, but it is > quite possible > > if (gst_gl_context_get_gl_platform (other_context) != > GST_GL_PLATFORM_EGL) { > g_set_error (error, GST_GL_CONTEXT_ERROR, > GST_GL_CONTEXT_ERROR_WRONG_CONFIG, > "Cannot share context with non-EGL context"); > // ======== I don’t reach this point > goto failure; > } > external_gl_context = gst_gl_context_get_gl_context (other_context); > } > > if ((gl_api & (GST_GL_API_OPENGL | GST_GL_API_OPENGL3 | GST_GL_API_GLES2)) > == > GST_GL_API_NONE) { > > ... > // =========== OPENGL case – not relevant > ... > > > } else if (gl_api & GST_GL_API_GLES2) { > gint i; > > try_gles2: > if (!eglBindAPI (EGL_OPENGL_ES_API)) { > g_set_error (error, GST_GL_CONTEXT_ERROR, GST_GL_CONTEXT_ERROR_FAILED, > "Failed to bind OpenGL|ES API: %s", > gst_gl_context_egl_get_error_string (eglGetError ())); > goto failure; > } > > // ============ I get this message > > GST_INFO ("Bound OpenGL|ES"); > > > for (i = 0; i < G_N_ELEMENTS (gles2_versions); i++) { > gint profileMask = 0; > gint contextFlags = 0; > > if (!gst_gl_context_egl_choose_config (egl, GST_GL_API_GLES2, > gles2_versions[i].major, error)) { > continue; > } > #if defined(EGL_KHR_create_context) > /* try a debug context */ > contextFlags |= EGL_CONTEXT_OPENGL_DEBUG_BIT_KHR; > > egl->egl_context = > _create_context_with_flags (egl, (EGLContext) external_gl_context, > GST_GL_API_GLES2, gles2_versions[i].major, > gles2_versions[i].minor, contextFlags, profileMask); > > if (egl->egl_context) > break; > > /* try without a debug context */ > contextFlags &= ~EGL_CONTEXT_OPENGL_DEBUG_BIT_KHR; > #endif > > egl->egl_context = > _create_context_with_flags (egl, (EGLContext) external_gl_context, > GST_GL_API_GLES2, gles2_versions[i].major, > gles2_versions[i].minor, contextFlags, profileMask); > > if (egl->egl_context) > break; > } > egl->gl_api = GST_GL_API_GLES2; > } > > if (egl->egl_context != EGL_NO_CONTEXT) { > // ====== I get this message > GST_INFO ("gl context created: %" G_GUINTPTR_FORMAT, > (guintptr) egl->egl_context); > } else { > g_set_error (error, GST_GL_CONTEXT_ERROR, > GST_GL_CONTEXT_ERROR_CREATE_CONTEXT, > "Failed to create a OpenGL context: %s", > gst_gl_context_egl_get_error_string (eglGetError ())); > goto failure; > } > > Just for reference, I share the current version of context setting (in the > bus need-context callback): > > case GST_MESSAGE_NEED_CONTEXT: > { > > // we got gl_display and gl_context and starting of the main > loop > > GstElement *element = GST_ELEMENT (message->src); > const gchar *name = gst_element_get_name(element); > > LOG_INFO("Element %s asks for context. We give => 0x%08x > 0x%08x", name, data->egl_context, data->egl_display); > > const gchar *context_type; > > gst_message_parse_context_type (message, &context_type); > g_print ("got need context %s\n", context_type); > > if (g_strcmp0 (context_type, GST_GL_DISPLAY_CONTEXT_TYPE) == 0) > { > GstContext *display_context = > gst_context_new (GST_GL_DISPLAY_CONTEXT_TYPE, TRUE); > gst_context_set_gl_display (display_context, > data->gst_display); > gst_element_set_context (GST_ELEMENT (message->src), > display_context); > return TRUE; > } else if (g_strcmp0 (context_type, "gst.gl.app_context") == 0) > { > GstContext *app_context = gst_context_new > ("gst.gl.app_context", TRUE); > GstStructure *s = gst_context_writable_structure > (app_context); > gst_structure_set (s, "context", GST_GL_TYPE_CONTEXT, > data->gst_context, NULL); > gst_element_set_context (GST_ELEMENT (message->src), > app_context); > return TRUE; > } > break; > > } handler and not the async bus handler and that data->gst_context holds a valid GstGLContext. The other thing to check is the GStreamer debug logs, specifically with GST_DEBUG=gl*:7,GST_CONTEXT:7 to ensure that your provided GL context is being used correctly. Cheers -Matt > Any remarks would be much appreciated. > > Best, > Jeremi _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel signature.asc (527 bytes) Download Attachment |
Matthew Waters wrote
> This code looks good assuming that this callback is from the sync bus > handler and not the async bus handler and that data->gst_context holds a > valid GstGLContext. It is quite likely that the problem was the async bus call that I used, which was defined in this way: // setting bus callbacks guint bus_watch_id; bus = gst_pipeline_get_bus (GST_PIPELINE (data->pipeline)); bus_watch_id = gst_bus_add_watch (bus, bus_callback, data); gst_object_unref (bus); This callback was called twice for my glupload element, asking “gst.gl.app_context” and "gst.gl.GLDisplay". After your suggestion I deleted the previous implementation and substituted it with the sync call (taken “as is” from sdlshare2.c): /* Set the pipeline to READY, so it can already accept a window handle, if we have one */ gst_element_set_state(data->pipeline, GST_STATE_READY); bus = gst_pipeline_get_bus (GST_PIPELINE (data->pipeline)); gst_bus_add_signal_watch (bus); gst_bus_enable_sync_message_emission (bus); g_signal_connect (bus, "message::error", G_CALLBACK (end_stream_cb), data->main_loop); g_signal_connect (bus, "message::warning", G_CALLBACK (end_stream_cb), data->main_loop); g_signal_connect (bus, "message::eos", G_CALLBACK (end_stream_cb), data->main_loop); g_signal_connect (bus, "sync-message", G_CALLBACK (bus_callback), data); Now the problem is that the sync message is called several times with “state-changed” message, but never with “need-context” message, so I do not actaully have the chance to share my context… 02-20 20:02:07.931 9331-9353/com.oculus.vrcubeworldsv E/GST_log: Recieved context[0x5D081680] and display[0x00000001] from renderer 02-20 20:02:07.966 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Starting gstreamer pipeline 02-20 20:02:07.966 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Creating pipeline in CustomData at 0x735d082420 02-20 20:02:07.966 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Creating new main loop 02-20 20:02:07.966 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Building pipeline 02-20 20:02:07.978 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Checking errors 02-20 20:02:07.980 9331-9376/com.oculus.vrcubeworldsv I/GST_log: glupload display ok 02-20 20:02:07.980 9331-9376/com.oculus.vrcubeworldsv I/GST_log: glupload wrapped context ok 02-20 20:02:07.983 9331-9376/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:07.983 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Got state-changed message 02-20 20:02:07.983 9331-9376/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:07.983 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Got state-changed message 02-20 20:02:07.984 9331-9376/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:07.984 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Got stream-status message 02-20 20:02:07.985 9331-9376/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:07.985 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Got state-changed message 02-20 20:02:07.985 9331-9385/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:07.985 9331-9385/com.oculus.vrcubeworldsv I/GST_log: Got stream-status message 02-20 20:02:07.985 9331-9385/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:07.985 9331-9385/com.oculus.vrcubeworldsv I/GST_log: Got stream-start message 02-20 20:02:08.023 9331-9385/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:08.023 9331-9385/com.oculus.vrcubeworldsv I/GST_log: Got state-changed message 02-20 20:02:08.024 9331-9385/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:08.024 9331-9385/com.oculus.vrcubeworldsv I/GST_log: Got state-changed message 02-20 20:02:08.024 9331-9385/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:08.024 9331-9385/com.oculus.vrcubeworldsv I/GST_log: Got async-done message 02-20 20:02:08.024 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Entering main loop... (CustomData:0x735d082420) 02-20 20:02:08.024 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Semaphore post 02-20 20:02:08.024 9331-9376/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:08.024 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Got new-clock message 02-20 20:02:08.025 9331-9385/com.oculus.vrcubeworldsv I/GST_log: Got a buffer at fakesink 02-20 20:02:08.025 9331-9376/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:08.025 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Got state-changed message 02-20 20:02:08.025 9331-9376/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:08.025 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Got state-changed message 02-20 20:02:08.026 9331-9376/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:08.026 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Got state-changed message 02-20 20:02:08.026 9331-9376/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:08.026 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Got state-changed message 02-20 20:02:08.027 9331-9376/com.oculus.vrcubeworldsv E/GST_log: ==========> BUS SYNC !!!! 02-20 20:02:08.027 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Got state-changed message 02-20 20:02:08.027 9331-9376/com.oculus.vrcubeworldsv I/GST_log: Playing... 02-20 20:02:09.025 9331-9385/com.oculus.vrcubeworldsv I/GST_log: Got a buffer at fakesink 02-20 20:02:10.026 9331-9385/com.oculus.vrcubeworldsv I/GST_log: Got a buffer at fakesink 02-20 20:02:11.025 9331-9385/com.oculus.vrcubeworldsv I/GST_log: Got a buffer at fakesink Speaking of the validity of the GstGlContext, data->egl_display and data->egl_context are EGLContext and EGLDisplay, respectively, which are passed from the renderer. I wrap them in this way, so I think I’m ok here: // get GStreamer display from the native OpenGL EGL vars data->gst_display = gst_gl_display_egl_new_with_egl_display(data->egl_display); if(data->gst_display == NULL){ LOG_ERROR("glupload display fail"); } else { LOG_INFO("glupload display ok"); } data->gst_context = gst_gl_context_new_wrapped( data->gst_display, (guintptr)data->egl_context, GST_GL_PLATFORM_EGL, GST_GL_API_GLES2); if(data->gst_context == NULL){ LOG_ERROR("glupload wrapped context fail"); } else { LOG_INFO("glupload wrapped context ok"); } To sum up, it seems that the lack of need-context message in bus sync callback is the problem now. Any reason for that? Thanks, Jeremi PS. Soon I will add the full debug logs -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Le mardi 20 février 2018 à 12:16 -0700, jeremi.wojcicki a écrit :
> To sum up, it seems that the lack of need-context message in bus sync > callback is the problem now. Any reason for that? Have you compared your code against one of the tutorial ? https://gstreamer.freedesktop.org/documentation/tutorials/android/index.html Nicolas _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Nicolas Dufresne-5 wrote
> Le mardi 20 février 2018 à 12:16 -0700, jeremi.wojcicki a écrit : >> To sum up, it seems that the lack of need-context message in bus sync >> callback is the problem now. Any reason for that? > > Have you compared your code against one of the tutorial ? Hmm, I derived originally from tutorial-3 that I used some concepts from sdlshare.c and sdlshare2.c for the fakesink and context sharing. Anyhow, here is the current version of the code: https://hastebin.com/onoyirarof.php and as you can see the need-context message appears in the async handler (I hope I am naming them right, but that's what I understood fro the GstBus documentation). 2-21 10:00:26.546 19263-19331/com.oculus.vrcubeworldsv E/GST_log: Recieved context[0x718BE080] and display[0x00000001] from renderer 02-21 10:00:26.565 19263-19377/com.oculus.vrcubeworldsv I/GST_log: Starting gstreamer pipeline 02-21 10:00:26.565 19263-19377/com.oculus.vrcubeworldsv I/GST_log: Creating pipeline in CustomData at 0x735d084420 02-21 10:00:26.565 19263-19377/com.oculus.vrcubeworldsv I/GST_log: Creating new main loop 02-21 10:00:26.565 19263-19377/com.oculus.vrcubeworldsv I/GST_log: Building pipeline 02-21 10:00:26.569 19263-19377/com.oculus.vrcubeworldsv I/GST_log: glupload display ok 02-21 10:00:26.569 19263-19377/com.oculus.vrcubeworldsv I/GST_log: glupload wrapped context ok 02-21 10:00:26.574 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.574 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:26.574 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.574 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:26.574 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.574 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got stream-status message 02-21 10:00:26.574 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.574 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:26.574 19263-19385/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.574 19263-19385/com.oculus.vrcubeworldsv I/GST_log: > Got stream-status message 02-21 10:00:26.575 19263-19385/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.575 19263-19385/com.oculus.vrcubeworldsv I/GST_log: > Got stream-start message 02-21 10:00:26.630 19263-19385/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.630 19263-19385/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:26.630 19263-19385/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.630 19263-19385/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:26.630 19263-19385/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.630 19263-19385/com.oculus.vrcubeworldsv I/GST_log: > Got async-done message 02-21 10:00:26.630 19263-19377/com.oculus.vrcubeworldsv I/GST_log: Entering main loop... (CustomData:0x735d084420) 02-21 10:00:26.630 19263-19377/com.oculus.vrcubeworldsv I/GST_log: Semaphore post 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got new-clock message 02-21 10:00:26.631 19263-19385/com.oculus.vrcubeworldsv I/GST_log: Got a buffer at fakesink 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > sync handler called 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: Playing... 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > async handler called 02-21 10:00:26.631 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > async handler called 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got need-context message 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: Element gluploadelement0 asks for context. We give => 0x718be080 0x00000001 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: got need context gst.gl.GLDisplay 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > async handler called 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got have-context message 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: Received 'got context' message 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > async handler called 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got need-context message 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: Element gluploadelement0 asks for context. We give => 0x718be080 0x00000001 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: got need context gst.gl.app_context 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > async handler called 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > async handler called 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > async handler called 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > async handler called 02-21 10:00:26.632 19263-19377/com.oculus.vrcubeworldsv I/GST_log: > Got state-changed message 02-21 10:00:27.631 19263-19385/com.oculus.vrcubeworldsv I/GST_log: Got a buffer at fakesink 02-21 10:00:28.632 19263-19385/com.oculus.vrcubeworldsv I/GST_log: Got a buffer at fakesink 02-21 10:00:29.631 19263-19385/com.oculus.vrcubeworldsv I/GST_log: Got a buffer at fakesink 02-21 10:00:30.631 19263-19385/com.oculus.vrcubeworldsv I/GST_log: Got a buffer at fakesink Maybe the problem is not really in it. It is possible to set the context just upon element creation, right? Maybe it would be better to focus on this solution. Now I have this kind of code that sets it in the callback, depending on the message: case GST_MESSAGE_NEED_CONTEXT: { // we got gl_display and gl_context and starting of the main loop GstElement *element = GST_ELEMENT (message->src); const gchar *name = gst_element_get_name(element); LOG_INFO("Element %s asks for context. We give => 0x%08x 0x%08x", name, data->egl_context, data->egl_display); const gchar *context_type; gst_message_parse_context_type (message, &context_type); LOG_INFO("got need context %s\n", context_type); if (g_strcmp0 (context_type, GST_GL_DISPLAY_CONTEXT_TYPE) == 0) { GstContext *display_context = gst_context_new (GST_GL_DISPLAY_CONTEXT_TYPE, TRUE); gst_context_set_gl_display (display_context, data->gst_display); gst_element_set_context (GST_ELEMENT (message->src), display_context); return TRUE; } else if (g_strcmp0 (context_type, "gst.gl.app_context") == 0) { GstContext *app_context = gst_context_new ("gst.gl.app_context", TRUE); GstStructure *s = gst_context_writable_structure (app_context); gst_structure_set (s, "context", GST_GL_TYPE_CONTEXT, data->gst_context, NULL); gst_element_set_context (GST_ELEMENT (message->src), app_context); return TRUE; } break; } Is it good enough to set the context right in the beginning (in the app_function)? -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
>> I derived originally from tutorial-3 that I used some concepts from >> sdlshare.c and sdlshare2.c for the fakesink and context sharing. Hi, all these 3 examples are working for you ? What is the diff with your sample ? On 21 February 2018 at 09:09, jeremi.wojcicki <[hidden email]> wrote: Nicolas Dufresne-5 wrote _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Success!
>> Maybe the problem is not really in it. It is possible to set the context >> just upon element creation, right? Maybe it would be better to focus on >> this >> solution. > > This solution is effectively, create the GstContext containing the > GstGLDisplay/GstGLContext and then call gst_element_set_context() on the > pipeline before you start the pipeline. Following this suggestion I put: GstContext *app_context = gst_context_new ("gst.gl.app_context", TRUE); GstStructure *s = gst_context_writable_structure (app_context); gst_structure_set (s, "context", GST_GL_TYPE_CONTEXT, data->gst_context, NULL); gst_element_set_context (GST_ELEMENT (data->upload), app_context); GstContext *display_context = gst_context_new (GST_GL_DISPLAY_CONTEXT_TYPE, TRUE); gst_context_set_gl_display (display_context, data->gst_display); gst_element_set_context (GST_ELEMENT (data->upload), display_context); before starting pipeline and context is correctly shared! thanks a lot for all the guidance. Looking back at the problem I notice the following: - Originally issue was caused by the fact that I had an asynchronous bus handler (which I guess was not suitable). It somehow worked in the simple tutorial-3 case but not in the OVR code => possibly some race condition occurs with async, right? - I substituted async with sync handler ONLY, but interestingly no "need-context" message appears there, only some state-change messages arrive. Having the messy implementation with both handlers showed that need context appears only in async. This problem remains unfixed and I do not know what is the reason. - Using the alternative of applying context before pipeline starts works flawlessly and is the best solution for me, given circumstances. -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Free forum by Nabble | Edit this page |