OpenGL renderer window..

[hhony]

I have a question about launching a simple video pipeline using 'glimagesink' with the python3 gi.repository.Gst.ElementFactory..

On Mac OSX I can launch my pipeline with custom plugin:
gst-launch-1.0 filesrc <avi> ! decodebin ! queue ! my_custom_plugin <props> ! autovideosink

The output a default window titled 'OpenGL renderer'.. which I assume is gstglwindow_cocoa.m from gst-plugins-bad.. ?

Likewise, 'glimagesink' will also open the 'OpenGL renderer' window when used in a bash gst-launch-1.0 command in the same way as 'autovideosink'. Also I notice when I launch the pipeline under bash, I can see:

    Setting pipeline to PAUSED ...

    Pipeline is PREROLLING ...

    Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayCocoa\)\ gldisplaycocoa0";

However, when I launch the pipeline for gi.repository.Gst.ElementFactory (in python3) I have issues:

1. It fails horribly for ElementFactory.make('autovideosink') with Illegal instruction: 4 when NULL to READY is propagated.. so to fix that I just didn't use 'autovideosink'..
   
   
2. It will propagate PLAY for ElementFactory.make('glimagesink').. but nothing actually plays.. probably because the chain isn't correct
   
   
3. I cannot figure out how to hook the 'sync_message' to an OSX window object with glimagesink from python?!? I see information in gst-inspect-1.0 glimagesink.. but I can't find any examples to do this specifically.
   
   
4. Secondarily, I see that I can use gi.repository.GstGL.GLDisplay().new() to create:
       <__gi__.GstGLDisplayCocoa object at 0x10e05eb88 (GstGLDisplayCocoa at 0x7fc7ba9439c0)>
   .. which is seemingly what I want!
   
   
5. How to do hook this object up to 'glimagesink' ??
   
   
   
6. Finally, do I need to add:
     pipeline.get_bus().connect('sync-message::element', on_sync_message) with a function on_sync_message(bus, msg) ?

From the Gtk examples I found there was a function which would update the GUI.. I assume that I would need to do this as well.. but the syntax is not clear immediately.
More specifically:
def on_sync_message(bus, msg): if msg.get_structure().get_name() == 'prepare-window-handle':

sink = msg.src

sink.set_window_handle(<window_handle>)

sink.set_render_rectangle(0, 0, <window.width>, <window.height>)


Overall, I want to know:

1. How is it that gst-launch-1.0 works so well?
   
2. How can I use that functionality in my python3 gi code?
   
3. How can I add another element to trigger that force-aspect-ratio window magic?

$ brew list --versions | grep gst

gst-plugins-bad 1.12.3

gst-plugins-base 1.12.3

gst-plugins-good 1.12.3

gst-plugins-ugly 1.12.3

gst-python 1.12.3

gst-rtsp-server 1.12.3

gstreamer 1.12.3

Thanks,

Hans

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Matthew Waters]

Mac OS X is special in that a CFRunLoop is required to be running on the main thread in order for glimagesink (and other UI operations) to perform correctly.

That either means you need to run one yourself, use a toolkit that runs one or run a GMainLoop.

Running your gst-launch-1.0 command with GST_DEBUG=gl*:7 would give you more information about what GStreamer is attempting to do, specifically around https://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/gst-libs/gst/gl/cocoa/gstgldisplay_cocoa.m#n105.

On 08/02/18 06:49, Hans Hony wrote:

I have a question about launching a simple video pipeline using 'glimagesink' with the python3 gi.repository.Gst.ElementFactory..

On Mac OSX I can launch my pipeline with custom plugin:
gst-launch-1.0 filesrc <avi> ! decodebin ! queue ! my_custom_plugin <props> ! autovideosink

The output a default window titled 'OpenGL renderer'.. which I assume is gstglwindow_cocoa.m from gst-plugins-bad.. ?

Likewise, 'glimagesink' will also open the 'OpenGL renderer' window when used in a bash gst-launch-1.0 command in the same way as 'autovideosink'. Also I notice when I launch the pipeline under bash, I can see:

    Setting pipeline to PAUSED ...

    Pipeline is PREROLLING ...

    Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayCocoa\)\ gldisplaycocoa0";

However, when I launch the pipeline for gi.repository.Gst.ElementFactory (in python3) I have issues:

1. It fails horribly for ElementFactory.make('autovideosink') with Illegal instruction: 4 when NULL to READY is propagated.. so to fix that I just didn't use 'autovideosink'..
   
   
2. It will propagate PLAY for ElementFactory.make('glimagesink').. but nothing actually plays.. probably because the chain isn't correct
   
   
3. I cannot figure out how to hook the 'sync_message' to an OSX window object with glimagesink from python?!? I see information in gst-inspect-1.0 glimagesink.. but I can't find any examples to do this specifically.
   
   
4. Secondarily, I see that I can use gi.repository.GstGL.GLDisplay().new() to create:
       <__gi__.GstGLDisplayCocoa object at 0x10e05eb88 (GstGLDisplayCocoa at 0x7fc7ba9439c0)>
   .. which is seemingly what I want!
   
   
5. How to do hook this object up to 'glimagesink' ??
   

You don't need to hook this up to GStreamer, it will create that automatically.

1. Finally, do I need to add:
     pipeline.get_bus().connect('sync-message::element', on_sync_message) with a function on_sync_message(bus, msg) ?

This shouldn't be needed.

From the Gtk examples I found there was a function which would update the GUI.. I assume that I would need to do this as well.. but the syntax is not clear immediately.
More specifically:
def on_sync_message(bus, msg): if msg.get_structure().get_name() == 'prepare-window-handle':

sink = msg.src

sink.set_window_handle(<window_handle>)

sink.set_render_rectangle(0, 0, <window.width>, <window.height>)

This is for embedding glimagesink's output inside another application.

Overall, I want to know:

1. How is it that gst-launch-1.0 works so well?
   

Because, glimagesink is generally created and set to READY on the main thread. Otherwise, happy accident.

1. How can I use that functionality in my python3 gi code?
   

Run a CFRunLoop/GMainLoop on the main thread.

1. How can I add another element to trigger that force-aspect-ratio window magic?

I'm not sure what you mean by this.

Cheers
-Matt

$ brew list --versions | grep gst

gst-plugins-bad 1.12.3

gst-plugins-base 1.12.3

gst-plugins-good 1.12.3

gst-plugins-ugly 1.12.3

gst-python 1.12.3

gst-rtsp-server 1.12.3

gstreamer 1.12.3

Thanks,

Hans

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[hhony]

Matt,
Thanks for the response.

I'm not sure I follow.. I already have an instance of Glib.MainLoop for the pipeline to run already.

This is constructed on the main thread just after Gst.init(None) is called.
But.. I am running GLib.MainLoop.run in it's own thread - is this okay?
That Thread is started from the main application, just before the pipeline is started.

What I have tried so far:

import gi
gi.require_version('GstGL', '1.0')
from gi.repository import GstGL

self._context = Gst.Context.new(GstGL.GL_DISPLAY_CONTEXT_TYPE, True)
self._gldisplay = GstGL.GLDisplay()
self._gldisplay.new()
self._glwindow = self._gldisplay.create_window()
self._glwindow.show()
self._glcontext = GstGL.GLContext.new(self._gldisplay)
GstGL.context_set_gl_display(self._context, self._gldisplay)
outputsink = gst_elem.make('glimagesink')
outputsink.set_context(self._context)
print('----- context: %s' % str(outputsink.get_property('context')))

What I notice is that the constructed Element 'glimagesink' has a context property which returns 'None' after construction.

I can see that the python rocket bouncing.. so it's attempting to kick the display.. but no window.

Do I need to override something with the GST_MESSAGE_NEED_CONTEXT, perhaps?

I have attempted to connect this on the bus 'message::need-context', but get no signal..

On Thu, Feb 8, 2018 at 1:21 AM, Matthew Waters <[hidden email]> wrote:

Mac OS X is special in that a CFRunLoop is required to be running on the main thread in order for glimagesink (and other UI operations) to perform correctly.

That either means you need to run one yourself, use a toolkit that runs one or run a GMainLoop.

Running your gst-launch-1.0 command with GST_DEBUG=gl*:7 would give you more information about what GStreamer is attempting to do, specifically around https://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/gst-libs/gst/gl/cocoa/gstgldisplay_cocoa.m#n105.

On 08/02/18 06:49, Hans Hony wrote:

I have a question about launching a simple video pipeline using 'glimagesink' with the python3 gi.repository.Gst.ElementFactory..

On Mac OSX I can launch my pipeline with custom plugin:
gst-launch-1.0 filesrc <avi> ! decodebin ! queue ! my_custom_plugin <props> ! autovideosink

The output a default window titled 'OpenGL renderer'.. which I assume is gstglwindow_cocoa.m from gst-plugins-bad.. ?

Likewise, 'glimagesink' will also open the 'OpenGL renderer' window when used in a bash gst-launch-1.0 command in the same way as 'autovideosink'. Also I notice when I launch the pipeline under bash, I can see:

    Setting pipeline to PAUSED ...

    Pipeline is PREROLLING ...

    Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayCocoa\)\ gldisplaycocoa0";

However, when I launch the pipeline for gi.repository.Gst.ElementFactory (in python3) I have issues:

1. It fails horribly for ElementFactory.make('autovideosink') with Illegal instruction: 4 when NULL to READY is propagated.. so to fix that I just didn't use 'autovideosink'..
   
   
2. It will propagate PLAY for ElementFactory.make('glimagesink').. but nothing actually plays.. probably because the chain isn't correct
   
   
3. I cannot figure out how to hook the 'sync_message' to an OSX window object with glimagesink from python?!? I see information in gst-inspect-1.0 glimagesink.. but I can't find any examples to do this specifically.
   
   
4. Secondarily, I see that I can use gi.repository.GstGL.GLDisplay().new() to create:
       <__gi__.GstGLDisplayCocoa object at 0x10e05eb88 (GstGLDisplayCocoa at 0x7fc7ba9439c0)>
   .. which is seemingly what I want!
   
   
5. How to do hook this object up to 'glimagesink' ??
   

You don't need to hook this up to GStreamer, it will create that automatically.

1. Finally, do I need to add:
     pipeline.get_bus().connect('sync-message::element', on_sync_message) with a function on_sync_message(bus, msg) ?

This shouldn't be needed.

From the Gtk examples I found there was a function which would update the GUI.. I assume that I would need to do this as well.. but the syntax is not clear immediately.
More specifically:
def on_sync_message(bus, msg): if msg.get_structure().get_name() == 'prepare-window-handle':

sink = msg.src

sink.set_window_handle(<window_handle>)

sink.set_render_rectangle(0, 0, <window.width>, <window.height>)

This is for embedding glimagesink's output inside another application.

Overall, I want to know:

1. How is it that gst-launch-1.0 works so well?
   

Because, glimagesink is generally created and set to READY on the main thread. Otherwise, happy accident.

1. How can I use that functionality in my python3 gi code?
   

Run a CFRunLoop/GMainLoop on the main thread.

1. How can I add another element to trigger that force-aspect-ratio window magic?

I'm not sure what you mean by this.

Cheers
-Matt

$ brew list --versions | grep gst

gst-plugins-bad 1.12.3

gst-plugins-base 1.12.3

gst-plugins-good 1.12.3

gst-plugins-ugly 1.12.3

gst-python 1.12.3

gst-rtsp-server 1.12.3

gstreamer 1.12.3

Thanks,

Hans

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Matthew Waters]

On 10/02/18 10:29, Hans Hony wrote:

Matt,
Thanks for the response.

I'm not sure I follow.. I already have an instance of Glib.MainLoop for the pipeline to run already.

This is constructed on the main thread just after Gst.init(None) is called.
But.. I am running GLib.MainLoop.run in it's own thread - is this okay?
That Thread is started from the main application, just before the pipeline is started.

Unfortunately, no.  The CFRunLoop must be running on the main thread, otherwise this will not work.

Cheers
-Matt

What I have tried so far:

import gi
gi.require_version('GstGL', '1.0')
from gi.repository import GstGL

self._context = Gst.Context.new(GstGL.GL_DISPLAY_CONTEXT_TYPE, True)
self._gldisplay = GstGL.GLDisplay()
self._gldisplay.new()
self._glwindow = self._gldisplay.create_window()
self._glwindow.show()
self._glcontext = GstGL.GLContext.new(self._gldisplay)
GstGL.context_set_gl_display(self._context, self._gldisplay)
outputsink = gst_elem.make('glimagesink')
outputsink.set_context(self._context)
print('----- context: %s' % str(outputsink.get_property('context')))

What I notice is that the constructed Element 'glimagesink' has a context property which returns 'None' after construction.

I can see that the python rocket bouncing.. so it's attempting to kick the display.. but no window.

Do I need to override something with the GST_MESSAGE_NEED_CONTEXT, perhaps?

I have attempted to connect this on the bus 'message::need-context', but get no signal..

On Thu, Feb 8, 2018 at 1:21 AM, Matthew Waters <[hidden email]> wrote:

Mac OS X is special in that a CFRunLoop is required to be running on the main thread in order for glimagesink (and other UI operations) to perform correctly.

That either means you need to run one yourself, use a toolkit that runs one or run a GMainLoop.

Running your gst-launch-1.0 command with GST_DEBUG=gl*:7 would give you more information about what GStreamer is attempting to do, specifically around https://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/gst-libs/gst/gl/cocoa/gstgldisplay_cocoa.m#n105.

On 08/02/18 06:49, Hans Hony wrote:

I have a question about launching a simple video pipeline using 'glimagesink' with the python3 gi.repository.Gst.ElementFactory..

On Mac OSX I can launch my pipeline with custom plugin:
gst-launch-1.0 filesrc <avi> ! decodebin ! queue ! my_custom_plugin <props> ! autovideosink

The output a default window titled 'OpenGL renderer'.. which I assume is gstglwindow_cocoa.m from gst-plugins-bad.. ?

Likewise, 'glimagesink' will also open the 'OpenGL renderer' window when used in a bash gst-launch-1.0 command in the same way as 'autovideosink'. Also I notice when I launch the pipeline under bash, I can see:

    Setting pipeline to PAUSED ...

    Pipeline is PREROLLING ...

    Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayCocoa\)\ gldisplaycocoa0";

However, when I launch the pipeline for gi.repository.Gst.ElementFactory (in python3) I have issues:

1. It fails horribly for ElementFactory.make('autovideosink') with Illegal instruction: 4 when NULL to READY is propagated.. so to fix that I just didn't use 'autovideosink'..
   
   
2. It will propagate PLAY for ElementFactory.make('glimagesink').. but nothing actually plays.. probably because the chain isn't correct
   
   
3. I cannot figure out how to hook the 'sync_message' to an OSX window object with glimagesink from python?!? I see information in gst-inspect-1.0 glimagesink.. but I can't find any examples to do this specifically.
   
   
4. Secondarily, I see that I can use gi.repository.GstGL.GLDisplay().new() to create:
       <__gi__.GstGLDisplayCocoa object at 0x10e05eb88 (GstGLDisplayCocoa at 0x7fc7ba9439c0)>
   .. which is seemingly what I want!
   
   
5. How to do hook this object up to 'glimagesink' ??
   

You don't need to hook this up to GStreamer, it will create that automatically.

1. Finally, do I need to add:
     pipeline.get_bus().connect('sync-message::element', on_sync_message) with a function on_sync_message(bus, msg) ?

This shouldn't be needed.

From the Gtk examples I found there was a function which would update the GUI.. I assume that I would need to do this as well.. but the syntax is not clear immediately.
More specifically:
def on_sync_message(bus, msg): if msg.get_structure().get_name() == 'prepare-window-handle':

sink = msg.src

sink.set_window_handle(<window_handle>)

sink.set_render_rectangle(0, 0, <window.width>, <window.height>)

This is for embedding glimagesink's output inside another application.

Overall, I want to know:

1. How is it that gst-launch-1.0 works so well?
   

Because, glimagesink is generally created and set to READY on the main thread. Otherwise, happy accident.

1. How can I use that functionality in my python3 gi code?
   

Run a CFRunLoop/GMainLoop on the main thread.

1. How can I add another element to trigger that force-aspect-ratio window magic?

I'm not sure what you mean by this.

Cheers
-Matt

$ brew list --versions | grep gst

gst-plugins-bad 1.12.3

gst-plugins-base 1.12.3

gst-plugins-good 1.12.3

gst-plugins-ugly 1.12.3

gst-python 1.12.3

gst-rtsp-server 1.12.3

gstreamer 1.12.3

Thanks,

Hans

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[hhony]

I'm not sure I understand all the semantic details of this conversation - by which, I mean, I'm not quite sure how to do that specifically.

So, CFRunLoop - is imported from CoreFoundation.. which I seem to be able to obtain via ctypes or PyObjC..

Now how I use that in my python application is another story.. (In retrospect this might be more approachable in C++)

I'm assuming that CFRunLoopRun is a blocking call, and would need to be executed at the end of my application code's run loop.

I think I have the perfect place for that to happen in mind.. but the import details look a little hairy.. I also have to assume that CFRunLoop would react to SIGINT or SIGTERM..

Lastly, I understand that this problem is slightly outside the scope of gstreamer. So, if you can respond with meaningful tips, I would be greatful.. but I do understand that the end of scope is quickly approaching.

On Fri, Feb 9, 2018 at 8:01 PM, Matthew Waters <[hidden email]> wrote:

On 10/02/18 10:29, Hans Hony wrote:

Matt,
Thanks for the response.

I'm not sure I follow.. I already have an instance of Glib.MainLoop for the pipeline to run already.

This is constructed on the main thread just after Gst.init(None) is called.
But.. I am running GLib.MainLoop.run in it's own thread - is this okay?
That Thread is started from the main application, just before the pipeline is started.

Unfortunately, no.  The CFRunLoop must be running on the main thread, otherwise this will not work.

Cheers
-Matt

What I have tried so far:

import gi
gi.require_version('GstGL', '1.0')
from gi.repository import GstGL

self._context = Gst.Context.new(GstGL.GL_DISPLAY_CONTEXT_TYPE, True)
self._gldisplay = GstGL.GLDisplay()
self._gldisplay.new()
self._glwindow = self._gldisplay.create_window()
self._glwindow.show()
self._glcontext = GstGL.GLContext.new(self._gldisplay)
GstGL.context_set_gl_display(self._context, self._gldisplay)
outputsink = gst_elem.make('glimagesink')
outputsink.set_context(self._context)
print('----- context: %s' % str(outputsink.get_property('context')))

What I notice is that the constructed Element 'glimagesink' has a context property which returns 'None' after construction.

I can see that the python rocket bouncing.. so it's attempting to kick the display.. but no window.

Do I need to override something with the GST_MESSAGE_NEED_CONTEXT, perhaps?

I have attempted to connect this on the bus 'message::need-context', but get no signal..

On Thu, Feb 8, 2018 at 1:21 AM, Matthew Waters <[hidden email]> wrote:

Mac OS X is special in that a CFRunLoop is required to be running on the main thread in order for glimagesink (and other UI operations) to perform correctly.

That either means you need to run one yourself, use a toolkit that runs one or run a GMainLoop.

Running your gst-launch-1.0 command with GST_DEBUG=gl*:7 would give you more information about what GStreamer is attempting to do, specifically around https://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/gst-libs/gst/gl/cocoa/gstgldisplay_cocoa.m#n105.

On 08/02/18 06:49, Hans Hony wrote:

I have a question about launching a simple video pipeline using 'glimagesink' with the python3 gi.repository.Gst.ElementFactory..

On Mac OSX I can launch my pipeline with custom plugin:
gst-launch-1.0 filesrc <avi> ! decodebin ! queue ! my_custom_plugin <props> ! autovideosink

The output a default window titled 'OpenGL renderer'.. which I assume is gstglwindow_cocoa.m from gst-plugins-bad.. ?

Likewise, 'glimagesink' will also open the 'OpenGL renderer' window when used in a bash gst-launch-1.0 command in the same way as 'autovideosink'. Also I notice when I launch the pipeline under bash, I can see:

    Setting pipeline to PAUSED ...

    Pipeline is PREROLLING ...

    Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayCocoa\)\ gldisplaycocoa0";

However, when I launch the pipeline for gi.repository.Gst.ElementFactory (in python3) I have issues:

1. It fails horribly for ElementFactory.make('autovideosink') with Illegal instruction: 4 when NULL to READY is propagated.. so to fix that I just didn't use 'autovideosink'..
   
   
2. It will propagate PLAY for ElementFactory.make('glimagesink').. but nothing actually plays.. probably because the chain isn't correct
   
   
3. I cannot figure out how to hook the 'sync_message' to an OSX window object with glimagesink from python?!? I see information in gst-inspect-1.0 glimagesink.. but I can't find any examples to do this specifically.
   
   
4. Secondarily, I see that I can use gi.repository.GstGL.GLDisplay().new() to create:
       <__gi__.GstGLDisplayCocoa object at 0x10e05eb88 (GstGLDisplayCocoa at 0x7fc7ba9439c0)>
   .. which is seemingly what I want!
   
   
5. How to do hook this object up to 'glimagesink' ??
   

You don't need to hook this up to GStreamer, it will create that automatically.

1. Finally, do I need to add:
     pipeline.get_bus().connect('sync-message::element', on_sync_message) with a function on_sync_message(bus, msg) ?

This shouldn't be needed.

From the Gtk examples I found there was a function which would update the GUI.. I assume that I would need to do this as well.. but the syntax is not clear immediately.
More specifically:
def on_sync_message(bus, msg): if msg.get_structure().get_name() == 'prepare-window-handle':

sink = msg.src

sink.set_window_handle(<window_handle>)

sink.set_render_rectangle(0, 0, <window.width>, <window.height>)

This is for embedding glimagesink's output inside another application.

Overall, I want to know:

1. How is it that gst-launch-1.0 works so well?
   

Because, glimagesink is generally created and set to READY on the main thread. Otherwise, happy accident.

1. How can I use that functionality in my python3 gi code?
   

Run a CFRunLoop/GMainLoop on the main thread.

1. How can I add another element to trigger that force-aspect-ratio window magic?

I'm not sure what you mean by this.

Cheers
-Matt

$ brew list --versions | grep gst

gst-plugins-bad 1.12.3

gst-plugins-base 1.12.3

gst-plugins-good 1.12.3

gst-plugins-ugly 1.12.3

gst-python 1.12.3

gst-rtsp-server 1.12.3

gstreamer 1.12.3

Thanks,

Hans

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[hhony]

Scratch that last request.

install order matters:
http://pythonhosted.org/pyobjc/install.html#manual-installation

pip3 install pyobjc-core

pip3 install pyobjc-framework-Cocoa
pip3 install pyobjc-framework-Quartz

from CoreFoundation import CFRunLoopRun
CFRunLoopRun() # in the main thread

Unfortunately the blocking call breaks my signal handlers.. but that's another issue..

OpenGL display pops up now.

Thanks!
Hans

On Sat, Feb 10, 2018 at 2:49 PM, Hans Hony <[hidden email]> wrote:

I'm not sure I understand all the semantic details of this conversation - by which, I mean, I'm not quite sure how to do that specifically.

So, CFRunLoop - is imported from CoreFoundation.. which I seem to be able to obtain via ctypes or PyObjC..

Now how I use that in my python application is another story.. (In retrospect this might be more approachable in C++)

I'm assuming that CFRunLoopRun is a blocking call, and would need to be executed at the end of my application code's run loop.

I think I have the perfect place for that to happen in mind.. but the import details look a little hairy.. I also have to assume that CFRunLoop would react to SIGINT or SIGTERM..

Lastly, I understand that this problem is slightly outside the scope of gstreamer. So, if you can respond with meaningful tips, I would be greatful.. but I do understand that the end of scope is quickly approaching.

On Fri, Feb 9, 2018 at 8:01 PM, Matthew Waters <[hidden email]> wrote:

On 10/02/18 10:29, Hans Hony wrote:

Matt,
Thanks for the response.

I'm not sure I follow.. I already have an instance of Glib.MainLoop for the pipeline to run already.

This is constructed on the main thread just after Gst.init(None) is called.
But.. I am running GLib.MainLoop.run in it's own thread - is this okay?
That Thread is started from the main application, just before the pipeline is started.

Unfortunately, no.  The CFRunLoop must be running on the main thread, otherwise this will not work.

Cheers
-Matt

What I have tried so far:

import gi
gi.require_version('GstGL', '1.0')
from gi.repository import GstGL

self._context = Gst.Context.new(GstGL.GL_DISPLAY_CONTEXT_TYPE, True)
self._gldisplay = GstGL.GLDisplay()
self._gldisplay.new()
self._glwindow = self._gldisplay.create_window()
self._glwindow.show()
self._glcontext = GstGL.GLContext.new(self._gldisplay)
GstGL.context_set_gl_display(self._context, self._gldisplay)
outputsink = gst_elem.make('glimagesink')
outputsink.set_context(self._context)
print('----- context: %s' % str(outputsink.get_property('context')))

What I notice is that the constructed Element 'glimagesink' has a context property which returns 'None' after construction.

I can see that the python rocket bouncing.. so it's attempting to kick the display.. but no window.

Do I need to override something with the GST_MESSAGE_NEED_CONTEXT, perhaps?

I have attempted to connect this on the bus 'message::need-context', but get no signal..

On Thu, Feb 8, 2018 at 1:21 AM, Matthew Waters <[hidden email]> wrote:

Mac OS X is special in that a CFRunLoop is required to be running on the main thread in order for glimagesink (and other UI operations) to perform correctly.

That either means you need to run one yourself, use a toolkit that runs one or run a GMainLoop.

Running your gst-launch-1.0 command with GST_DEBUG=gl*:7 would give you more information about what GStreamer is attempting to do, specifically around https://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/gst-libs/gst/gl/cocoa/gstgldisplay_cocoa.m#n105.

On 08/02/18 06:49, Hans Hony wrote:

I have a question about launching a simple video pipeline using 'glimagesink' with the python3 gi.repository.Gst.ElementFactory..

On Mac OSX I can launch my pipeline with custom plugin:
gst-launch-1.0 filesrc <avi> ! decodebin ! queue ! my_custom_plugin <props> ! autovideosink

The output a default window titled 'OpenGL renderer'.. which I assume is gstglwindow_cocoa.m from gst-plugins-bad.. ?

Likewise, 'glimagesink' will also open the 'OpenGL renderer' window when used in a bash gst-launch-1.0 command in the same way as 'autovideosink'. Also I notice when I launch the pipeline under bash, I can see:

    Setting pipeline to PAUSED ...

    Pipeline is PREROLLING ...

    Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayCocoa\)\ gldisplaycocoa0";

However, when I launch the pipeline for gi.repository.Gst.ElementFactory (in python3) I have issues:

1. It fails horribly for ElementFactory.make('autovideosink') with Illegal instruction: 4 when NULL to READY is propagated.. so to fix that I just didn't use 'autovideosink'..
   
   
2. It will propagate PLAY for ElementFactory.make('glimagesink').. but nothing actually plays.. probably because the chain isn't correct
   
   
3. I cannot figure out how to hook the 'sync_message' to an OSX window object with glimagesink from python?!? I see information in gst-inspect-1.0 glimagesink.. but I can't find any examples to do this specifically.
   
   
4. Secondarily, I see that I can use gi.repository.GstGL.GLDisplay().new() to create:
       <__gi__.GstGLDisplayCocoa object at 0x10e05eb88 (GstGLDisplayCocoa at 0x7fc7ba9439c0)>
   .. which is seemingly what I want!
   
   
5. How to do hook this object up to 'glimagesink' ??
   

You don't need to hook this up to GStreamer, it will create that automatically.

1. Finally, do I need to add:
     pipeline.get_bus().connect('sync-message::element', on_sync_message) with a function on_sync_message(bus, msg) ?

This shouldn't be needed.

From the Gtk examples I found there was a function which would update the GUI.. I assume that I would need to do this as well.. but the syntax is not clear immediately.
More specifically:
def on_sync_message(bus, msg): if msg.get_structure().get_name() == 'prepare-window-handle':

sink = msg.src

sink.set_window_handle(<window_handle>)

sink.set_render_rectangle(0, 0, <window.width>, <window.height>)

This is for embedding glimagesink's output inside another application.

Overall, I want to know:

1. How is it that gst-launch-1.0 works so well?
   

Because, glimagesink is generally created and set to READY on the main thread. Otherwise, happy accident.

1. How can I use that functionality in my python3 gi code?
   

Run a CFRunLoop/GMainLoop on the main thread.

1. How can I add another element to trigger that force-aspect-ratio window magic?

I'm not sure what you mean by this.

Cheers
-Matt

$ brew list --versions | grep gst

gst-plugins-bad 1.12.3

gst-plugins-base 1.12.3

gst-plugins-good 1.12.3

gst-plugins-ugly 1.12.3

gst-python 1.12.3

gst-rtsp-server 1.12.3

gstreamer 1.12.3

Thanks,

Hans

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[hhony]

Matt,
I figured out the signal stuff - I had to import `AppHelper.initMachInterrupts()` and also `MachSignals.signal(SIGINT.value, self.on_mach_signal)`.. so I can actually kill the application without using `kill -9 <pid>` in another terminal now. ;)

So everything works great.. except for the resize. Which for some reason is acting strange. I get a window that looks about like 320x240 -ish.. but the stream is 640x360. Also it looks like the display size is incorrect.

Clearly, I'm not initializing something correctly.. If I do not create a GLDisplay object it does not work.. But if I do create a GLDisplay object and Context object.. then the pipeline creates a the GLWindow and uses 640x480 to initialize the screen size.

Note below, both are using <glcontextcocoa0> and <glwindowcocoa0>.. I did not use GLDisplay.create_window() or GLDisplay.create_context() on the GLDisplay object because then gstreamer would create a second GLContext and GLWindow and use that instead..

Constructing a GLDisplay seems to be enough to let gstreamer construct the GLWindow and GLContext.. however, how do I command a resize of that window? Or how to I initialize the GLDisplay / GLWindow with the correct dimensions?

I cannot resize or close the NSWindow that pops up.. However, my SIGINT signal will close the application just fine.

In the Pipeline class I have:

import gi
gi.require_version('GstGL', '1.0')
from gi.repository import GstGL
self._gldisplay_context = Context.new(GstGL.GL_DISPLAY_CONTEXT_TYPE, True)
self._gldisplay = GstGL.GLDisplay()
self._gldisplay.new()
GstGL.context_set_gl_display(self._gldisplay_context, self._gldisplay)
outputsink = ElementFactory.make('glimagesink')

When launched from bash, I see:

0:00:00.193636000 59279 0x7fdeb35a26a0 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glwrappedcontext0> activate:0

0:00:00.194289000 59279 0x7fdeb35a26a0 LOG          glcaopengllayer gstglcaopengllayer.m:200:-[GstGLCAOpenGLLayer drawInCGLContext:pixelFormat:forLayerTime:displayTime:]: CAOpenGLLayer drawing with cgl context 0x7fdeb613e800

0:00:00.194304000 59279 0x7fdeb35a26a0 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glwrappedcontext0> activate:1

0:00:00.194329000 59279 0x7fdeb35a26a0 DEBUG               glwindow gstglwindow_cocoa.m:396:gst_gl_cocoa_resize_cb:<glwindowcocoa0> Window resized: bounds 0.000000 0.000000 1280.000000 720.000000 visibleRect 0.000000 0.000000 1280.000000 720.000000

0:00:00.194362000 59279 0x7fdeb362be30 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:00.194375000 59279 0x7fdeb362be30 DEBUG            glimagesink gstglimagesink.c:2083:gst_glimage_sink_on_resize:<sink> GL Window resized to 640x360

0:00:00.194397000 59279 0x7fdeb362be30 DEBUG            glimagesink gstglimagesink.c:2148:gst_glimage_sink_on_resize:<sink> GL output area now 0,0 640x360

0:00:00.194420000 59279 0x7fdeb35a26a0 TRACE            glimagesink gstglimagesink.c:2186:gst_glimage_sink_on_draw: redrawing texture:3

0:00:00.194430000 59279 0x7fdeb35a26a0 LOG               glsyncmeta gstglsyncmeta.c:215:_wait: waiting 0x7fdeb5829048

0:00:00.195457000 59279 0x7fdeb35a26a0 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glwrappedcontext0> activate:0

0:00:00.254419000 59279 0x7fdeb40cea30 TRACE            glimagesink gstglimagesink.c:1730:gst_glimage_sink_show_frame: rendering buffer:0x7fdeb4118040

0:00:00.254476000 59279 0x7fdeb40cea30 TRACE            glimagesink gstglimagesink.c:1738:gst_glimage_sink_show_frame: redisplay texture:2 of size:640x360, window size:640x360

0:00:00.254501000 59279 0x7fdeb40cea30 DEBUG            glimagesink gstglimagesink.c:2335:gst_glimage_sink_redisplay: Recreating output after mode/size change

0:00:00.254513000 59279 0x7fdeb40cea30 TRACE            glimagesink gstglimagesink.c:1349:configure_display_from_info: PAR: 1/1 DAR:1/1

0:00:00.254526000 59279 0x7fdeb40cea30 DEBUG            glimagesink gstglimagesink.c:1352:configure_display_from_info: keeping video height

0:00:00.254535000 59279 0x7fdeb40cea30 DEBUG            glimagesink gstglimagesink.c:1370:configure_display_from_info: scaling to 640x360

0:00:00.254644000 59279 0x7fdeb3742b80 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:00.254700000 59279 0x7fdeb3742b80 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110b1db37 data:0x70000ec30c18

0:00:00.254714000 59279 0x7fdeb3742b80 LOG             glbasememory gstglbasememory.c:246:_map_data_gl: mapping mem 0x7fdeb35ca470 flags 20001

0:00:00.254730000 59279 0x7fdeb3742b80 LOG             glbasememory gstglbasememory.c:246:_map_data_gl: mapping mem 0x7fdeb35ca720 flags 20001

0:00:00.254746000 59279 0x7fdeb3742b80 LOG             glbasememory gstglbasememory.c:325:_unmap_data_gl: unmapping mem 0x7fdeb35ca720 flags 20001

0:00:00.254799000 59279 0x7fdeb3742b80 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:00.254812000 59279 0x7fdeb3742b80 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110b1dd80 data:0x70000ec30d50

0:00:00.254824000 59279 0x7fdeb3742b80 LOG             glbasememory gstglbasememory.c:325:_unmap_data_gl: unmapping mem 0x7fdeb35ca470 flags 20001

0:00:00.255138000 59279 0x7fdeb3742b80 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:00.255160000 59279 0x7fdeb3742b80 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110b368a3 data:0x7fdeb5829048

0:00:00.255175000 59279 0x7fdeb3742b80 LOG               glsyncmeta gstglsyncmeta.c:314:_free_gl_sync_meta: free sync meta 0x7fdeb5829048

0:00:00.255227000 59279 0x7fdeb3742b80 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:00.255253000 59279 0x7fdeb3742b80 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110b368a3 data:0x7fdeb583e4c8

0:00:00.255269000 59279 0x7fdeb3742b80 LOG               glsyncmeta gstglsyncmeta.c:314:_free_gl_sync_meta: free sync meta 0x7fdeb583e4c8

0:00:00.255443000 59279 0x7fdeb3742b80 LOG               glsyncmeta gstglsyncmeta.c:119:_default_free_gl: deleting sync object 0x3

0:00:00.256312000 59279 0x7fdeb40cea30 LOG          glcaopengllayer gstglcaopengllayer.m:200:-[GstGLCAOpenGLLayer drawInCGLContext:pixelFormat:forLayerTime:displayTime:]: CAOpenGLLayer drawing with cgl context 0x7fdeb613e800

0:00:00.256344000 59279 0x7fdeb40cea30 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glwrappedcontext0> activate:1

0:00:00.256371000 59279 0x7fdeb40cea30 DEBUG               glwindow gstglwindow_cocoa.m:396:gst_gl_cocoa_resize_cb:<glwindowcocoa0> Window resized: bounds 0.000000 0.000000 1280.000000 720.000000 visibleRect 0.000000 0.000000 1280.000000 720.000000

0:00:00.256423000 59279 0x7fdeb3742b80 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:00.256438000 59279 0x7fdeb3742b80 DEBUG            glimagesink gstglimagesink.c:2083:gst_glimage_sink_on_resize:<sink> GL Window resized to 640x360

0:00:00.256474000 59279 0x7fdeb3742b80 DEBUG            glimagesink gstglimagesink.c:2148:gst_glimage_sink_on_resize:<sink> GL output area now 0,0 640x360

0:00:00.256517000 59279 0x7fdeb40cea30 TRACE            glimagesink gstglimagesink.c:2186:gst_glimage_sink_on_draw: redrawing texture:2

0:00:00.256529000 59279 0x7fdeb40cea30 LOG               glsyncmeta gstglsyncmeta.c:215:_wait: waiting 0x7fdeb501f3a8

0:00:00.256652000 59279 0x7fdeb40cea30 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glwrappedcontext0> activate:0

0:00:00.257303000 59279 0x7fdeb40cea30 TRACE            glimagesink gstglimagesink.c:1744:gst_glimage_sink_show_frame: post redisplay

0:00:00.257486000 59279 0x7fdeb3742b80 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:00.257520000 59279 0x7fdeb3742b80 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110b1db37 data:0x70000ec313f8

0:00:00.257535000 59279 0x7fdeb3742b80 LOG             glbasememory gstglbasememory.c:246:_map_data_gl: mapping mem 0x7fdeb361d9b0 flags 10002

0:00:00.257551000 59279 0x7fdeb3742b80 LOG             glbasememory gstglbasememory.c:246:_map_data_gl: mapping mem 0x7fdeb361e0f0 flags 10002

0:00:00.257562000 59279 0x7fdeb3742b80 LOG                 glbuffer gstglbuffer.c:131:gst_gl_buffer_cpu_access: mapping id 1 size 230400

0:00:00.260711000 59279 0x7fdeb482dc60 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

AND When I create the GLDisplay.. I see:

0:00:02.170664000 67279 0x7fb1a6ea8670 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glwrappedcontext0> activate:0

0:00:02.172145000 67279 0x7fb1a6ea8670 LOG          glcaopengllayer gstglcaopengllayer.m:200:-[GstGLCAOpenGLLayer drawInCGLContext:pixelFormat:forLayerTime:displayTime:]: CAOpenGLLayer drawing with cgl context 0x7fb1a53be200

0:00:02.172201000 67279 0x7fb1a6ea8670 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glwrappedcontext0> activate:1

0:00:02.172245000 67279 0x7fb1a6ea8670 DEBUG               glwindow gstglwindow_cocoa.m:396:gst_gl_cocoa_resize_cb:<glwindowcocoa0> Window resized: bounds 0.000000 0.000000 640.000000 480.000000 visibleRect 0.000000 0.000000 640.000000 480.000000

0:00:02.172314000 67279 0x7fb1a932e590 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:02.172324000 67279 0x7fb1a932e590 DEBUG            glimagesink gstglimagesink.c:2083:gst_glimage_sink_on_resize:<sink> GL Window resized to 320x240

0:00:02.172346000 67279 0x7fb1a932e590 DEBUG            glimagesink gstglimagesink.c:2148:gst_glimage_sink_on_resize:<sink> GL output area now 0,30 320x180

0:00:02.172382000 67279 0x7fb1a6ea8670 TRACE            glimagesink gstglimagesink.c:2186:gst_glimage_sink_on_draw: redrawing texture:2

0:00:02.172396000 67279 0x7fb1a6ea8670 LOG               glsyncmeta gstglsyncmeta.c:215:_wait: waiting 0x7fb1a5358e38

0:00:02.172405000 67279 0x7fb1a6ea8670 LOG               glsyncmeta gstglsyncmeta.c:79:_default_wait_gl: waiting on sync object 0x1

0:00:02.173526000 67279 0x7fb1a6ea8670 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glwrappedcontext0> activate:0

0:00:02.177483000 67279 0x7fb1a6ea8670 DEBUG               glwindow gstglwindow_cocoa.m:275:_show_window:<glwindowcocoa0> make the window available

[INFO] stream is playing..

                   

0:00:02.225168000 67279 0x7fb1a494c050 TRACE            glimagesink gstglimagesink.c:1730:gst_glimage_sink_show_frame: rendering buffer:0x7fb1a8809a60

0:00:02.225193000 67279 0x7fb1a494c050 TRACE            glimagesink gstglimagesink.c:1738:gst_glimage_sink_show_frame: redisplay texture:3 of size:640x360, window size:640x360

0:00:02.225254000 67279 0x7fb1a494c050 DEBUG            glimagesink gstglimagesink.c:2335:gst_glimage_sink_redisplay: Recreating output after mode/size change

0:00:02.225262000 67279 0x7fb1a494c050 TRACE            glimagesink gstglimagesink.c:1349:configure_display_from_info: PAR: 1/1 DAR:1/1

0:00:02.225271000 67279 0x7fb1a494c050 DEBUG            glimagesink gstglimagesink.c:1352:configure_display_from_info: keeping video height

0:00:02.225277000 67279 0x7fb1a494c050 DEBUG            glimagesink gstglimagesink.c:1370:configure_display_from_info: scaling to 640x360

0:00:02.225335000 67279 0x7fb1a6b0a670 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:02.225372000 67279 0x7fb1a6b0a670 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110e3ab37 data:0x70000dbd2de8

0:00:02.225387000 67279 0x7fb1a6b0a670 LOG             glbasememory gstglbasememory.c:246:_map_data_gl: mapping mem 0x7fb1a6b06b30 flags 20001

0:00:02.225397000 67279 0x7fb1a6b0a670 LOG             glbasememory gstglbasememory.c:246:_map_data_gl: mapping mem 0x7fb1a6b06dc0 flags 20001

0:00:02.225406000 67279 0x7fb1a6b0a670 LOG             glbasememory gstglbasememory.c:325:_unmap_data_gl: unmapping mem 0x7fb1a6b06dc0 flags 20001

0:00:02.225449000 67279 0x7fb1a6b0a670 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:02.225467000 67279 0x7fb1a6b0a670 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110e3ad80 data:0x70000dbd2f20

0:00:02.225553000 67279 0x7fb1a6b0a670 LOG             glbasememory gstglbasememory.c:325:_unmap_data_gl: unmapping mem 0x7fb1a6b06b30 flags 20001

0:00:02.225598000 67279 0x7fb1a6b0a670 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:02.225608000 67279 0x7fb1a6b0a670 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110e538a3 data:0x7fb1a5358e38

0:00:02.225616000 67279 0x7fb1a6b0a670 LOG               glsyncmeta gstglsyncmeta.c:314:_free_gl_sync_meta: free sync meta 0x7fb1a5358e38

0:00:02.225622000 67279 0x7fb1a6b0a670 LOG               glsyncmeta gstglsyncmeta.c:119:_default_free_gl: deleting sync object 0x1

0:00:02.226015000 67279 0x7fb1a494c050 LOG          glcaopengllayer gstglcaopengllayer.m:200:-[GstGLCAOpenGLLayer drawInCGLContext:pixelFormat:forLayerTime:displayTime:]: CAOpenGLLayer drawing with cgl context 0x7fb1a53be200

0:00:02.226034000 67279 0x7fb1a494c050 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glwrappedcontext0> activate:1

0:00:02.226049000 67279 0x7fb1a494c050 DEBUG               glwindow gstglwindow_cocoa.m:396:gst_gl_cocoa_resize_cb:<glwindowcocoa0> Window resized: bounds 0.000000 0.000000 640.000000 480.000000 visibleRect 0.000000 0.000000 640.000000 480.000000

0:00:02.226077000 67279 0x7fb1a6b0a670 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:02.226087000 67279 0x7fb1a6b0a670 DEBUG            glimagesink gstglimagesink.c:2083:gst_glimage_sink_on_resize:<sink> GL Window resized to 320x240

0:00:02.226108000 67279 0x7fb1a6b0a670 DEBUG            glimagesink gstglimagesink.c:2148:gst_glimage_sink_on_resize:<sink> GL output area now 0,30 320x180

0:00:02.226146000 67279 0x7fb1a494c050 TRACE            glimagesink gstglimagesink.c:2186:gst_glimage_sink_on_draw: redrawing texture:3

0:00:02.226159000 67279 0x7fb1a494c050 LOG               glsyncmeta gstglsyncmeta.c:215:_wait: waiting 0x7fb1a53e2bb8

0:00:02.226166000 67279 0x7fb1a494c050 LOG               glsyncmeta gstglsyncmeta.c:79:_default_wait_gl: waiting on sync object 0x2

0:00:02.226240000 67279 0x7fb1a494c050 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glwrappedcontext0> activate:0

0:00:02.226546000 67279 0x7fb1a494c050 TRACE            glimagesink gstglimagesink.c:1744:gst_glimage_sink_show_frame: post redisplay

0:00:02.226677000 67279 0x7fb1a6b0a670 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:02.226717000 67279 0x7fb1a6b0a670 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110e39fae data:0x70000dbd37a8

0:00:02.226744000 67279 0x7fb1a6b0a670 TRACE           glbasememory gstglbasememory.c:82:_mem_create_gl: Create memory 0x7fb1a9128d30

0:00:02.226795000 67279 0x7fb1a6b0a670 TRACE          glbasetexture gstglmemory.c:265:_gl_tex_create: Generating texture id:5 format:6403 type:5121 dimensions:640x360

0:00:02.226815000 67279 0x7fb1a6b0a670 TRACE           glbasememory gstglbasememory.c:82:_mem_create_gl: Create memory 0x7fb1a925d560

0:00:02.226829000 67279 0x7fb1a6b0a670 DEBUG           glbasememory gstglbasememory.c:175:gst_gl_base_memory_init: new GL buffer memory:0x7fb1a925d560 size:230400

0:00:02.226837000 67279 0x7fb1a6b0a670 DEBUG               glbuffer gstglbuffer.c:104:_gl_buffer_init: new GL buffer memory:0x7fb1a925d560 size:230400

0:00:02.226846000 67279 0x7fb1a6b0a670 LOG                 glmemory gstglmemorypbo.c:212:_gl_mem_create: generated pbo 7

0:00:02.226873000 67279 0x7fb1a494c050 DEBUG           glbasememory gstglbasememory.c:175:gst_gl_base_memory_init: new GL buffer memory:0x7fb1a9128d30 size:230400

0:00:02.226890000 67279 0x7fb1a494c050 DEBUG          glbasetexture gstglmemory.c:340:gst_gl_memory_init: new GL texture context:<glcontextcocoa0> memory:0x7fb1a9128d30 target:2D format:6403 dimensions:640x360 stride:640 size:230400

0:00:02.226929000 67279 0x7fb1a6b0a670 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:02.226956000 67279 0x7fb1a6b0a670 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110e4a661 data:0x7fb1a6096110

0:00:02.226974000 67279 0x7fb1a6b0a670 LOG             glbasememory gstglbasememory.c:246:_map_data_gl: mapping mem 0x7fb1a9128d30 flags 20001

0:00:02.226983000 67279 0x7fb1a6b0a670 LOG             glbasememory gstglbasememory.c:246:_map_data_gl: mapping mem 0x7fb1a925d560 flags 20001

0:00:02.227023000 67279 0x7fb1a6b0a670 LOG                 glmemory gstglmemorypbo.c:149:_upload_pbo_memory: upload for texture id:5, with pbo 7 640x360

0:00:02.227034000 67279 0x7fb1a6b0a670 LOG            glbasetexture gstglmemory.c:542:gst_gl_memory_texsubimage: upload for texture id:5, 640x360

0:00:02.227043000 67279 0x7fb1a6b0a670 TRACE                glquery gstglquery.c:251:gst_gl_query_start: 0x7fb1a6c05af0 start query type 'time elapsed' id 1

0:00:02.227123000 67279 0x7fb1a6b0a670 TRACE                glquery gstglquery.c:278:gst_gl_query_end: 0x7fb1a6c05af0 end query type 'time elapsed' id 1

0:00:02.227137000 67279 0x7fb1a6b0a670 LOG             glbasememory gstglbasememory.c:325:_unmap_data_gl: unmapping mem 0x7fb1a925d560 flags 20001

0:00:02.227146000 67279 0x7fb1a6b0a670 LOG             glbasememory gstglbasememory.c:246:_map_data_gl: mapping mem 0x7fb1a6b064b0 flags 20002

0:00:02.227154000 67279 0x7fb1a6b0a670 LOG                glconvert gstglcolorconvert.c:2232:_do_convert_one_view:<glcolorconvert0> converting to textures:0x7fb1a6b064b0,0x0,0x0,0x0 dimensions:640x360, from textures:0x7fb1a9128d30,0x0,0x0,0x0 dimensions:640x360

0:00:02.227204000 67279 0x7fb1a6b0a670 LOG             glbasememory gstglbasememory.c:325:_unmap_data_gl: unmapping mem 0x7fb1a6b064b0 flags 20002

0:00:02.227215000 67279 0x7fb1a6b0a670 LOG             glbasememory gstglbasememory.c:325:_unmap_data_gl: unmapping mem 0x7fb1a9128d30 flags 20001

0:00:02.227223000 67279 0x7fb1a6b0a670 LOG               glsyncmeta gstglsyncmeta.c:186:_set_sync_point: setting sync point 0x7fb1a73d3018

0:00:02.227231000 67279 0x7fb1a6b0a670 LOG               glsyncmeta gstglsyncmeta.c:66:_default_set_sync_gl: setting sync object 0x1

0:00:02.227307000 67279 0x7fb1a494c050 TRACE           glbasememory gstglbasememory.c:450:_mem_free: freeing buffer memory:0x7fb1a9128d30

0:00:02.227331000 67279 0x7fb1a6b0a670 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:02.227440000 67279 0x7fb1a6b0a670 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110e3aa1c data:0x7fb1a9128d30

0:00:02.227454000 67279 0x7fb1a6b0a670 TRACE           glbasememory gstglbasememory.c:450:_mem_free: freeing buffer memory:0x7fb1a925d560

0:00:02.227465000 67279 0x7fb1a6b0a670 TRACE                glquery gstglquery.c:177:gst_gl_query_unset: 0x7fb1a925d650 unsetting query 2

0:00:02.227500000 67279 0x7fb1a6b0a670 TRACE                glquery gstglquery.c:177:gst_gl_query_unset: 0x7fb1a6c05af0 unsetting query 1

0:00:02.227511000 67279 0x7fb1a6b0a670 TRACE                glquery gstglquery.c:345:gst_gl_query_result: 0x7fb1a6c05af0 get result 0 type 'time elapsed' id 1

0:00:02.227522000 67279 0x7fb1a6b0a670 LOG            glbasetexture gstglmemory.c:553:gst_gl_memory_texsubimage: glTexSubImage took 0:00:00.000000000

0:00:02.227551000 67279 0x7fb1a494c050 DEBUG         glcolorbalance gstglcolorbalance.c:231:gst_gl_color_balance_before_transform:<glcolorbalance0> sync to 0:00:00.133333333

0:00:02.227571000 67279 0x7fb1a494c050 TRACE            glimagesink gstglimagesink.c:1660:gst_glimage_sink_prepare: preparing buffer:0x7fb1a8809950

0:00:02.227579000 67279 0x7fb1a494c050 TRACE            glimagesink gstglimagesink.c:916:_ensure_gl_setup:<sink> Ensuring setup

0:00:02.227662000 67279 0x7fb1a494c050 TRACE            glimagesink gstglimagesink.c:999:_ensure_gl_setup:<sink> Already have a context

0:00:02.227688000 67279 0x7fb1a6b0a670 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:02.227698000 67279 0x7fb1a6b0a670 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110e53544 data:0x7fb1a73d3018

0:00:02.227706000 67279 0x7fb1a6b0a670 LOG               glsyncmeta gstglsyncmeta.c:215:_wait: waiting 0x7fb1a73d3018

0:00:02.227712000 67279 0x7fb1a6b0a670 LOG               glsyncmeta gstglsyncmeta.c:79:_default_wait_gl: waiting on sync object 0x1

0:00:02.227745000 67279 0x7fb1a6b0a670 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:02.227753000 67279 0x7fb1a6b0a670 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110e3ab37 data:0x70000dbd2e38

0:00:02.227760000 67279 0x7fb1a6b0a670 LOG             glbasememory gstglbasememory.c:246:_map_data_gl: mapping mem 0x7fb1a6b064b0 flags 20001

0:00:02.227768000 67279 0x7fb1a6b0a670 LOG             glbasememory gstglbasememory.c:246:_map_data_gl: mapping mem 0x7fb1a6b066c0 flags 20001

0:00:02.227785000 67279 0x7fb1a6b0a670 LOG             glbasememory gstglbasememory.c:325:_unmap_data_gl: unmapping mem 0x7fb1a6b066c0 flags 20001

0:00:02.227825000 67279 0x7fb1a6b0a670 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:02.227907000 67279 0x7fb1a6b0a670 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110e3ad80 data:0x70000dbd2f70

0:00:02.227919000 67279 0x7fb1a6b0a670 LOG             glbasememory gstglbasememory.c:325:_unmap_data_gl: unmapping mem 0x7fb1a6b064b0 flags 20001

0:00:02.291015000 67279 0x7fb1a494c050 TRACE            glimagesink gstglimagesink.c:1730:gst_glimage_sink_show_frame: rendering buffer:0x7fb1a8809950

0:00:02.292056000 67279 0x7fb1a494c050 TRACE            glimagesink gstglimagesink.c:1738:gst_glimage_sink_show_frame: redisplay texture:2 of size:640x360, window size:640x360

0:00:02.292121000 67279 0x7fb1a6b0a670 DEBUG              glcontext gstglcontext.c:749:gst_gl_context_activate:<glcontextcocoa0> activate:1

0:00:02.292162000 67279 0x7fb1a6b0a670 TRACE              glcontext gstglcontext.c:1533:_gst_gl_context_thread_run_generic:<glcontextcocoa0> running function:0x110e538a3 data:0x7fb1a53e2bb8

0:00:02.292174000 67279 0x7fb1a6b0a670 LOG               glsyncmeta gstglsyncmeta.c:314:_free_gl_sync_meta: free sync meta 0x7fb1a53e2bb8

0:00:02.292181000 67279 0x7fb1a6b0a670 LOG               glsyncmeta gstglsyncmeta.c:119:_default_free_gl: deleting sync object 0x2

Thanks,

Hans

On Sat, Feb 10, 2018 at 3:33 PM, Hans Hony <[hidden email]> wrote:

Scratch that last request.

install order matters:
http://pythonhosted.org/pyobjc/install.html#manual-installation

pip3 install pyobjc-core

pip3 install pyobjc-framework-Cocoa
pip3 install pyobjc-framework-Quartz

from CoreFoundation import CFRunLoopRun
CFRunLoopRun() # in the main thread

Unfortunately the blocking call breaks my signal handlers.. but that's another issue..

OpenGL display pops up now.

Thanks!
Hans

On Sat, Feb 10, 2018 at 2:49 PM, Hans Hony <[hidden email]> wrote:

I'm not sure I understand all the semantic details of this conversation - by which, I mean, I'm not quite sure how to do that specifically.

So, CFRunLoop - is imported from CoreFoundation.. which I seem to be able to obtain via ctypes or PyObjC..

Now how I use that in my python application is another story.. (In retrospect this might be more approachable in C++)

I'm assuming that CFRunLoopRun is a blocking call, and would need to be executed at the end of my application code's run loop.

I think I have the perfect place for that to happen in mind.. but the import details look a little hairy.. I also have to assume that CFRunLoop would react to SIGINT or SIGTERM..

Lastly, I understand that this problem is slightly outside the scope of gstreamer. So, if you can respond with meaningful tips, I would be greatful.. but I do understand that the end of scope is quickly approaching.

On Fri, Feb 9, 2018 at 8:01 PM, Matthew Waters <[hidden email]> wrote:

On 10/02/18 10:29, Hans Hony wrote:

Matt,
Thanks for the response.

I'm not sure I follow.. I already have an instance of Glib.MainLoop for the pipeline to run already.

This is constructed on the main thread just after Gst.init(None) is called.
But.. I am running GLib.MainLoop.run in it's own thread - is this okay?
That Thread is started from the main application, just before the pipeline is started.

Unfortunately, no.  The CFRunLoop must be running on the main thread, otherwise this will not work.

Cheers
-Matt

What I have tried so far:

import gi
gi.require_version('GstGL', '1.0')
from gi.repository import GstGL

self._context = Gst.Context.new(GstGL.GL_DISPLAY_CONTEXT_TYPE, True)
self._gldisplay = GstGL.GLDisplay()
self._gldisplay.new()
self._glwindow = self._gldisplay.create_window()
self._glwindow.show()
self._glcontext = GstGL.GLContext.new(self._gldisplay)
GstGL.context_set_gl_display(self._context, self._gldisplay)
outputsink = gst_elem.make('glimagesink')
outputsink.set_context(self._context)
print('----- context: %s' % str(outputsink.get_property('context')))

What I notice is that the constructed Element 'glimagesink' has a context property which returns 'None' after construction.

I can see that the python rocket bouncing.. so it's attempting to kick the display.. but no window.

Do I need to override something with the GST_MESSAGE_NEED_CONTEXT, perhaps?

I have attempted to connect this on the bus 'message::need-context', but get no signal..

On Thu, Feb 8, 2018 at 1:21 AM, Matthew Waters <[hidden email]> wrote:

Mac OS X is special in that a CFRunLoop is required to be running on the main thread in order for glimagesink (and other UI operations) to perform correctly.

That either means you need to run one yourself, use a toolkit that runs one or run a GMainLoop.

Running your gst-launch-1.0 command with GST_DEBUG=gl*:7 would give you more information about what GStreamer is attempting to do, specifically around https://cgit.freedesktop.org/gstreamer/gst-plugins-base/tree/gst-libs/gst/gl/cocoa/gstgldisplay_cocoa.m#n105.

On 08/02/18 06:49, Hans Hony wrote:

I have a question about launching a simple video pipeline using 'glimagesink' with the python3 gi.repository.Gst.ElementFactory..

On Mac OSX I can launch my pipeline with custom plugin:
gst-launch-1.0 filesrc <avi> ! decodebin ! queue ! my_custom_plugin <props> ! autovideosink

The output a default window titled 'OpenGL renderer'.. which I assume is gstglwindow_cocoa.m from gst-plugins-bad.. ?

Likewise, 'glimagesink' will also open the 'OpenGL renderer' window when used in a bash gst-launch-1.0 command in the same way as 'autovideosink'. Also I notice when I launch the pipeline under bash, I can see:

    Setting pipeline to PAUSED ...

    Pipeline is PREROLLING ...

    Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayCocoa\)\ gldisplaycocoa0";

However, when I launch the pipeline for gi.repository.Gst.ElementFactory (in python3) I have issues:

1. It fails horribly for ElementFactory.make('autovideosink') with Illegal instruction: 4 when NULL to READY is propagated.. so to fix that I just didn't use 'autovideosink'..
   
   
2. It will propagate PLAY for ElementFactory.make('glimagesink').. but nothing actually plays.. probably because the chain isn't correct
   
   
3. I cannot figure out how to hook the 'sync_message' to an OSX window object with glimagesink from python?!? I see information in gst-inspect-1.0 glimagesink.. but I can't find any examples to do this specifically.
   
   
4. Secondarily, I see that I can use gi.repository.GstGL.GLDisplay().new() to create:
       <__gi__.GstGLDisplayCocoa object at 0x10e05eb88 (GstGLDisplayCocoa at 0x7fc7ba9439c0)>
   .. which is seemingly what I want!
   
   
5. How to do hook this object up to 'glimagesink' ??
   

You don't need to hook this up to GStreamer, it will create that automatically.

1. Finally, do I need to add:
     pipeline.get_bus().connect('sync-message::element', on_sync_message) with a function on_sync_message(bus, msg) ?

This shouldn't be needed.

From the Gtk examples I found there was a function which would update the GUI.. I assume that I would need to do this as well.. but the syntax is not clear immediately.
More specifically:
def on_sync_message(bus, msg): if msg.get_structure().get_name() == 'prepare-window-handle':

sink = msg.src

sink.set_window_handle(<window_handle>)

sink.set_render_rectangle(0, 0, <window.width>, <window.height>)

This is for embedding glimagesink's output inside another application.

Overall, I want to know:

1. How is it that gst-launch-1.0 works so well?
   

Because, glimagesink is generally created and set to READY on the main thread. Otherwise, happy accident.

1. How can I use that functionality in my python3 gi code?
   

Run a CFRunLoop/GMainLoop on the main thread.

1. How can I add another element to trigger that force-aspect-ratio window magic?

I'm not sure what you mean by this.

Cheers
-Matt

$ brew list --versions | grep gst

gst-plugins-bad 1.12.3

gst-plugins-base 1.12.3

gst-plugins-good 1.12.3

gst-plugins-ugly 1.12.3

gst-python 1.12.3

gst-rtsp-server 1.12.3

gstreamer 1.12.3

Thanks,

Hans

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel