Influencing playbin2/decodebin2 auto-plugging

[Tvrtko Ursulin]

Hi all,

Is there a way to dynamically influence auto-plugging in playbin2/decodebin2,
for example by upranking or deranking elements?

Use case would be, with vaapi elements installed, to allow distributing
playback pipelines between the GPU and CPU without having to manually build
them (the pipelines).

Tvrtko
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Stefan Sauer]

On 08/02/2012 04:39 PM, Tvrtko Ursulin wrote:

Hi all,

Is there a way to dynamically influence auto-plugging in playbin2/decodebin2, 
for example by upranking or deranking elements?

No. https://bugzilla.gnome.org/show_bug.cgi?id=649542

Stefan

Use case would be, with vaapi elements installed, to allow distributing 
playback pipelines between the GPU and CPU without having to manually build 
them (the pipelines).

Tvrtko
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim-Philipp Müller-2]

On Thu, 2012-08-02 at 19:03 +0200, Stefan Sauer wrote:
> On 08/02/2012 04:39 PM, Tvrtko Ursulin wrote:
>
> >
> > Is there a way to dynamically influence auto-plugging in playbin2/decodebin2,
> > for example by upranking or deranking elements?
>
> No. https://bugzilla.gnome.org/show_bug.cgi?id=649542

This is just convenience API, of course you can do such things
dynamically.

You can manually set element ranks at runtime in your application by
looking up the GstElementFactory/GstPluginFeature in the registry after
gst_init() and setting the plugin rank via
gst_plugin_feature_set_rank().

Then there are the "autoplug-{factories,sort,select}" signals on
decodebin2/uridecodebin, which you can use to tweak the autoplugging.

You can use those in playbin2 as well, but you need to "somehow" find
the uridecodebin element inside playbin2.

As for the vaapi stuff, I thought this should just work automagically
these days (with latest releases) if the video-sink set supports it
(like vaapi*sink or cluttersink). Perhaps it's only in git, don't
remember. I'm sure someone will correct me if needed :)

What problem are you trying to solve exactly?

 Cheers
  -Tim

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Gwenole Beauchesne]

Hi,

2012/8/2 Tim-Philipp Müller <[hidden email]>:

> As for the vaapi stuff, I thought this should just work automagically
> these days (with latest releases) if the video-sink set supports it
> (like vaapi*sink or cluttersink). Perhaps it's only in git, don't
> remember. I'm sure someone will correct me if needed :)

Yes, everything is auto-plugged for nearly one year now. :)
e.g. if you have both ffdec_h264 and vaapidecode, playbin2 will pick
the latter up first. I have not tried myself recently, but I was told
it still works no later than yesterday on Wayland with plain playbin2.

Regards,
Gwenole.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Stefan Sauer]

On 08/02/2012 07:36 PM, Tim-Philipp Müller wrote:

> On Thu, 2012-08-02 at 19:03 +0200, Stefan Sauer wrote:
>> On 08/02/2012 04:39 PM, Tvrtko Ursulin wrote:
>>
>>> Is there a way to dynamically influence auto-plugging in playbin2/decodebin2,
>>> for example by upranking or deranking elements?
>> No. https://bugzilla.gnome.org/show_bug.cgi?id=649542
> This is just convenience API, of course you can do such things
> dynamically.
>
> You can manually set element ranks at runtime in your application by
> looking up the GstElementFactory/GstPluginFeature in the registry after
> gst_init() and setting the plugin rank via
> gst_plugin_feature_set_rank().
>
> Then there are the "autoplug-{factories,sort,select}" signals on
> decodebin2/uridecodebin, which you can use to tweak the autoplugging.
>
> You can use those in playbin2 as well, but you need to "somehow" find
> the uridecodebin element inside playbin2.
>
> As for the vaapi stuff, I thought this should just work automagically
> these days (with latest releases) if the video-sink set supports it
> (like vaapi*sink or cluttersink). Perhaps it's only in git, don't
> remember. I'm sure someone will correct me if needed :)

The question sounded like he want to adjust how much of the work is done
on cpu/gpu.

>
> What problem are you trying to solve exactly?
>
>  Cheers
>   -Tim
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tvrtko Ursulin]

On Thursday 02 August 2012 21:28:31 Stefan Sauer wrote:

> On 08/02/2012 07:36 PM, Tim-Philipp Müller wrote:
> > On Thu, 2012-08-02 at 19:03 +0200, Stefan Sauer wrote:
> >> On 08/02/2012 04:39 PM, Tvrtko Ursulin wrote:
> >>> Is there a way to dynamically influence auto-plugging in
> >>> playbin2/decodebin2, for example by upranking or deranking elements?
> >>
> >> No. https://bugzilla.gnome.org/show_bug.cgi?id=649542
> >
> > This is just convenience API, of course you can do such things
> > dynamically.
> >
> > You can manually set element ranks at runtime in your application by
> > looking up the GstElementFactory/GstPluginFeature in the registry after
> > gst_init() and setting the plugin rank via
> > gst_plugin_feature_set_rank().
> >
> > Then there are the "autoplug-{factories,sort,select}" signals on
> > decodebin2/uridecodebin, which you can use to tweak the autoplugging.
> >
> > You can use those in playbin2 as well, but you need to "somehow" find
> > the uridecodebin element inside playbin2.
> >
> > As for the vaapi stuff, I thought this should just work automagically
> > these days (with latest releases) if the video-sink set supports it
> > (like vaapi*sink or cluttersink). Perhaps it's only in git, don't
> > remember. I'm sure someone will correct me if needed :)
>
> The question sounded like he want to adjust how much of the work is done
> on cpu/gpu.

Exactly, play n videos on the GPU, and when resources run out, play some more
videos on the CPU.

Tvrtko
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tvrtko Ursulin]

On Thursday 02 August 2012 18:36:05 Tim-Philipp Müller wrote:

> On Thu, 2012-08-02 at 19:03 +0200, Stefan Sauer wrote:
> > On 08/02/2012 04:39 PM, Tvrtko Ursulin wrote:
> > > Is there a way to dynamically influence auto-plugging in
> > > playbin2/decodebin2, for example by upranking or deranking elements?
> >
> > No. https://bugzilla.gnome.org/show_bug.cgi?id=649542
>
> This is just convenience API, of course you can do such things
> dynamically.
>
> You can manually set element ranks at runtime in your application by
> looking up the GstElementFactory/GstPluginFeature in the registry after
> gst_init() and setting the plugin rank via
> gst_plugin_feature_set_rank().
>
> Then there are the "autoplug-{factories,sort,select}" signals on
> decodebin2/uridecodebin, which you can use to tweak the autoplugging.
>
> You can use those in playbin2 as well, but you need to "somehow" find
> the uridecodebin element inside playbin2.

This sounds promising, if application can reject an element in the autoplug-
select signal this is exactly what we need.

I'll try this approach and report back - hopefully it is possible from Python.
And I _think_ I already had a solution to find decodebin in playbin2.

> As for the vaapi stuff, I thought this should just work automagically
> these days (with latest releases) if the video-sink set supports it
> (like vaapi*sink or cluttersink). Perhaps it's only in git, don't
> remember. I'm sure someone will correct me if needed :)
>
> What problem are you trying to solve exactly?

Let some videos be played on the GPU and some on the CPU - because together
they can play more than only one on it's own.

Tvrtko

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tvrtko Ursulin]

On Thursday 02 August 2012 18:36:05 Tim-Philipp Müller wrote:

> On Thu, 2012-08-02 at 19:03 +0200, Stefan Sauer wrote:
> > On 08/02/2012 04:39 PM, Tvrtko Ursulin wrote:
> > > Is there a way to dynamically influence auto-plugging in
> > > playbin2/decodebin2, for example by upranking or deranking elements?
> >
> > No. https://bugzilla.gnome.org/show_bug.cgi?id=649542
>
> This is just convenience API, of course you can do such things
> dynamically.
>
> You can manually set element ranks at runtime in your application by
> looking up the GstElementFactory/GstPluginFeature in the registry after
> gst_init() and setting the plugin rank via
> gst_plugin_feature_set_rank().
>
> Then there are the "autoplug-{factories,sort,select}" signals on
> decodebin2/uridecodebin, which you can use to tweak the autoplugging.
>
> You can use those in playbin2 as well, but you need to "somehow" find
> the uridecodebin element inside playbin2.

Hm.. I can get uridecodebin element via the element-added signal, but when I
connect my handler to autoplug-select it is not getting called. Code snippet
is roughly this:

    def __gst_element_added(self, parent, element):
        element.connect( 'autoplug-select', self.__gst_autoplug_select )

Reading the docs, this stands out:

"""
Note
Only the signal handler that is connected first will ever by invoked. Don't
connect signal handlers with the G_CONNECT_AFTER flag to this signal, they
will never be invoked!
"""

Could that be the problem? I don't know if my handler is first, I mean, I am
not connecting anything else to it, but maybe something internally is?

Tvrtko

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tvrtko Ursulin]

On Friday 03 August 2012 10:44:39 Tvrtko Ursulin wrote:

> On Thursday 02 August 2012 18:36:05 Tim-Philipp Müller wrote:
> > On Thu, 2012-08-02 at 19:03 +0200, Stefan Sauer wrote:
> > > On 08/02/2012 04:39 PM, Tvrtko Ursulin wrote:
> > > > Is there a way to dynamically influence auto-plugging in
> > > > playbin2/decodebin2, for example by upranking or deranking elements?
> > >
> > > No. https://bugzilla.gnome.org/show_bug.cgi?id=649542
> >
> > This is just convenience API, of course you can do such things
> > dynamically.
> >
> > You can manually set element ranks at runtime in your application by
> > looking up the GstElementFactory/GstPluginFeature in the registry after
> > gst_init() and setting the plugin rank via
> > gst_plugin_feature_set_rank().
> >
> > Then there are the "autoplug-{factories,sort,select}" signals on
> > decodebin2/uridecodebin, which you can use to tweak the autoplugging.
> >
> > You can use those in playbin2 as well, but you need to "somehow" find
> > the uridecodebin element inside playbin2.
>
> Hm.. I can get uridecodebin element via the element-added signal, but when I
> connect my handler to autoplug-select it is not getting called. Code
> snippet is roughly this:

It is called but I got the number of parameters wrong.

Just need to figure out the return values now.

Tvrtko

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tvrtko Ursulin]

On Friday 03 August 2012 12:03:20 Tvrtko Ursulin wrote:

> On Friday 03 August 2012 10:44:39 Tvrtko Ursulin wrote:
> > On Thursday 02 August 2012 18:36:05 Tim-Philipp Müller wrote:
> > > On Thu, 2012-08-02 at 19:03 +0200, Stefan Sauer wrote:
> > > > On 08/02/2012 04:39 PM, Tvrtko Ursulin wrote:
> > > > > Is there a way to dynamically influence auto-plugging in
> > > > > playbin2/decodebin2, for example by upranking or deranking elements?
> > > >
> > > > No. https://bugzilla.gnome.org/show_bug.cgi?id=649542
> > >
> > > This is just convenience API, of course you can do such things
> > > dynamically.
> > >
> > > You can manually set element ranks at runtime in your application by
> > > looking up the GstElementFactory/GstPluginFeature in the registry after
> > > gst_init() and setting the plugin rank via
> > > gst_plugin_feature_set_rank().
> > >
> > > Then there are the "autoplug-{factories,sort,select}" signals on
> > > decodebin2/uridecodebin, which you can use to tweak the autoplugging.
> > >
> > > You can use those in playbin2 as well, but you need to "somehow" find
> > > the uridecodebin element inside playbin2.
> >
> > Hm.. I can get uridecodebin element via the element-added signal, but when
> > I connect my handler to autoplug-select it is not getting called. Code
> > snippet is roughly this:
> It is called but I got the number of parameters wrong.
>
> Just need to figure out the return values now.

I am seeing a weird interaction between using autoplug-select and vaapi. The
attached reproducer does not transition into playing state for me unless I
remove the line where I connect my message handler to autoplug-select.
Interesting thing is that my handler in this configuration does not deviate
from the default behaviour ie. it always returns GST_AUTOPLUG_SELECT_TRY.

In the no VA-API setup, with use_vaapi set to False at the top of the program,
playback works fine even with the autoplug-select handler installed.

Seems weird, any ideas?

Thanks,

Tvrtko

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim-Philipp Müller-2]

Hi,

> I am seeing a weird interaction between using autoplug-select and vaapi. The
> attached reproducer does not transition into playing state for me unless I
> remove the line where I connect my message handler to autoplug-select.
> Interesting thing is that my handler in this configuration does not deviate
> from the default behaviour ie. it always returns GST_AUTOPLUG_SELECT_TRY.
>
> In the no VA-API setup, with use_vaapi set to False at the top of the program,
> playback works fine even with the autoplug-select handler installed.
>
> Seems weird, any ideas?

I think playbin2 hooks up some of these signals as well
("autoplug-factories", "autoplug-select", "autoplug-continue"), so the
default for playbin2 would be whatever playbin2 does there.

Cheers
 -Tim


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tvrtko Ursulin]

On Monday 06 August 2012 16:12:23 Tim-Philipp Müller wrote:

> Hi,
>
> > I am seeing a weird interaction between using autoplug-select and vaapi.
> > The attached reproducer does not transition into playing state for me
> > unless I remove the line where I connect my message handler to
> > autoplug-select. Interesting thing is that my handler in this
> > configuration does not deviate from the default behaviour ie. it always
> > returns GST_AUTOPLUG_SELECT_TRY.
> >
> > In the no VA-API setup, with use_vaapi set to False at the top of the
> > program, playback works fine even with the autoplug-select handler
> > installed.
> >
> > Seems weird, any ideas?
>
> I think playbin2 hooks up some of these signals as well
> ("autoplug-factories", "autoplug-select", "autoplug-continue"), so the
> default for playbin2 would be whatever playbin2 does there.

What? :) Sorry but I don't understand what you wrote. What do you think is
broken/interacting badly and why?

Thanks,

Tvrtko

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tvrtko Ursulin]

On Monday 06 August 2012 16:18:19 Tvrtko Ursulin wrote:

> On Monday 06 August 2012 16:12:23 Tim-Philipp Müller wrote:
> > Hi,
> >
> > > I am seeing a weird interaction between using autoplug-select and vaapi.
> > > The attached reproducer does not transition into playing state for me
> > > unless I remove the line where I connect my message handler to
> > > autoplug-select. Interesting thing is that my handler in this
> > > configuration does not deviate from the default behaviour ie. it always
> > > returns GST_AUTOPLUG_SELECT_TRY.
> > >
> > > In the no VA-API setup, with use_vaapi set to False at the top of the
> > > program, playback works fine even with the autoplug-select handler
> > > installed.
> > >
> > > Seems weird, any ideas?
> >
> > I think playbin2 hooks up some of these signals as well
> > ("autoplug-factories", "autoplug-select", "autoplug-continue"), so the
> > default for playbin2 would be whatever playbin2 does there.
>
> What? :) Sorry but I don't understand what you wrote. What do you think is
> broken/interacting badly and why?

I see now that when my autoplug-select handler is installed, playbin2's one is
not called at all. And it actually looks like a non-trivial code there.

Solvable? I really liked this approach.. if this is a dead end I will have to
attempt your other suggestion of iterating over the plugin registry and modify
ranks. Hopefully it is doable from Python.

Do you perhaps have any pointers (examples) to start me up there?

Thanks,

Tvrtko

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim-Philipp Müller-2]

On Mon, 2012-08-06 at 17:01 +0100, Tvrtko Ursulin wrote:

> I see now that when my autoplug-select handler is installed, playbin2's one is
> not called at all. And it actually looks like a non-trivial code there.
>
> Solvable? I really liked this approach.. if this is a dead end I will have to
> attempt your other suggestion of iterating over the plugin registry and modify
> ranks. Hopefully it is doable from Python.
>
> Do you perhaps have any pointers (examples) to start me up there?

Should be doable from python.

Or perhaps the "autoplug-sort" signal would be enough for your purposes?
I don't think playbin2 uses that (yet).

Cheers
 -Tim


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tvrtko Ursulin]

On Monday 06 August 2012 17:07:36 Tim-Philipp Müller wrote:

> On Mon, 2012-08-06 at 17:01 +0100, Tvrtko Ursulin wrote:
> > I see now that when my autoplug-select handler is installed, playbin2's
> > one is not called at all. And it actually looks like a non-trivial code
> > there.
> >
> > Solvable? I really liked this approach.. if this is a dead end I will have
> > to attempt your other suggestion of iterating over the plugin registry
> > and modify ranks. Hopefully it is doable from Python.
> >
> > Do you perhaps have any pointers (examples) to start me up there?
>
> Should be doable from python.
>
> Or perhaps the "autoplug-sort" signal would be enough for your purposes?
> I don't think playbin2 uses that (yet).

It works!

Attached sample code in case anyone else needs it one day.

Thanks Tim!

Regards,

Tvrtko

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel