Gstreamer-VAAPI and Totem

[Alessandro Crismani]

Dear all,

I am trying to get hardware accelerated playback using Totem together
with gstreamer-vaaapi and an Intel HD3000 SNB card (the CPU is a i5 2520).

The gstreamer-vaapi is compiled from its master git branch as of today,
other gstreamer libraries are listed at the end of the mail.

I am trying to watch a mkv file, and if I open it using
-> GST_DEBUG=vaapi:5,vaapidecode:5 gst-launch-0.10 -v playbin2
uri=file:///data/Video/file.mkv
I see a lot of VAAPI debug output and the CPU usage remains quite low.
(In fact, if I remove the gstreamer-vaapi package, the debug output goes
away and the CPU usage increases, which means it probably works as
expected when gstreamer-vaapi is installed)

However, when I open it with Totem, the CPU rockets to sky levels :(
As far as I understood, Totem should pick the gstreamer playbin2 and
automagically use VAAPI, however it seems to fail on me. I have tried
getting debug using:
-> totem --gst-debug-help
but I do not see anything VAAPI related (only VDPAU there). How can I
see if Totem is using any VAAPI based decoding?

I am sorry if I am missing information or if this mail contains stupid
questions, but I am very new to VAAPI things and I do not have a clue on
how to report this. If you point me in the right direction, I'll follow
up with as many information as requested :)

I'd love to thank you for any feedback.

Cheers,
Alessandro

-> Anything that has to do with Gstreamer on my systems
local/clutter-gst 1.5.6-1
     GStreamer bindings for clutter
local/gnome-video-effects 0.4.0-1
     A collection of GStreamer effects
local/gstreamer-vaapi-git 20120605-1
     collection of VA-API based plugins for GStreamer and helper libraries
local/gstreamer0.10 0.10.36-1
     GStreamer Multimedia Framework
local/gstreamer0.10-bad 0.10.23-1 (gstreamer0.10-plugins)
     GStreamer Multimedia Framework Bad Plugin libraries (gst-plugins-bad)
local/gstreamer0.10-bad-plugins 0.10.23-1 (gstreamer0.10-plugins)
     GStreamer Multimedia Framework Bad Plugins (gst-plugins-bad)
local/gstreamer0.10-base 0.10.36-1
     GStreamer Multimedia Framework Base plugin libraries
local/gstreamer0.10-base-plugins 0.10.36-1 (gstreamer0.10-plugins)
     GStreamer Multimedia Framework Base Plugins (gst-plugins-base)
local/gstreamer0.10-ffmpeg 0.10.13-1 (gstreamer0.10-plugins)
     Gstreamer FFMpeg Plugin
local/gstreamer0.10-good 0.10.31-1
     GStreamer Multimedia Framework Good plugin libraries
local/gstreamer0.10-good-plugins 0.10.31-1 (gstreamer0.10-plugins)
     GStreamer Multimedia Framework Good Plugins (gst-plugins-good)
local/gstreamer0.10-ugly 0.10.19-1
     GStreamer Multimedia Framework Ugly plugin libraries
local/gstreamer0.10-ugly-plugins 0.10.19-1 (gstreamer0.10-plugins)
     GStreamer Multimedia Framework Ugly Plugins (gst-plugins-ugly)
local/totem 3.4.2-2 (gnome-extra)
     A GNOME3 integrated movie player based on Gstreamer.


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne]

Le mardi 05 juin 2012 à 15:29 +0200, Alessandro Crismani a écrit :
> However, when I open it with Totem, the CPU rockets to sky levels :(
> As far as I understood, Totem should pick the gstreamer playbin2 and
> automagically use VAAPI, however it seems to fail on me. I have tried
> getting debug using:
> -> totem --gst-debug-help
> but I do not see anything VAAPI related (only VDPAU there). How can I
> see if Totem is using any VAAPI based decoding?

The problem is that Totem is using the videobalance element, which is a
pure software element. Using software element prevent using hardware
acceleration like libva or VDPAU.

I suggested a while ago to add color balance interface to clutter-gst,
and change totem code to use that instead, which end up with the
conclusion that clutter should provide effects to implement this
feature. I haven't look at the status of it since.

cheers,
Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Javier Jardón]

On 5 June 2012 17:18, Nicolas Dufresne <[hidden email]> wrote:

>
> Le mardi 05 juin 2012 à 15:29 +0200, Alessandro Crismani a écrit :
> > However, when I open it with Totem, the CPU rockets to sky levels :(
> > As far as I understood, Totem should pick the gstreamer playbin2 and
> > automagically use VAAPI, however it seems to fail on me. I have tried
> > getting debug using:
> > -> totem --gst-debug-help
> > but I do not see anything VAAPI related (only VDPAU there). How can I
> > see if Totem is using any VAAPI based decoding?
>
> The problem is that Totem is using the videobalance element, which is a
> pure software element. Using software element prevent using hardware
> acceleration like libva or VDPAU.

Seems that current totem master doesnt use videobalance element
anymore. Take a look here [1]
Could you test latest totem and see if the issue is already fixed?

Cheers

[1] http://git.gnome.org/browse/totem/commit/?id=bc6f874373f64a0036711bbfe845d76264ec097a
--
Javier Jardón Cabezas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Bastien Nocera-2]

On Wed, 2012-06-06 at 15:46 +0100, Javier Jardón wrote:

> On 5 June 2012 17:18, Nicolas Dufresne <[hidden email]> wrote:
> >
> > Le mardi 05 juin 2012 à 15:29 +0200, Alessandro Crismani a écrit :
> > > However, when I open it with Totem, the CPU rockets to sky levels :(
> > > As far as I understood, Totem should pick the gstreamer playbin2 and
> > > automagically use VAAPI, however it seems to fail on me. I have tried
> > > getting debug using:
> > > -> totem --gst-debug-help
> > > but I do not see anything VAAPI related (only VDPAU there). How can I
> > > see if Totem is using any VAAPI based decoding?
> >
> > The problem is that Totem is using the videobalance element, which is a
> > pure software element. Using software element prevent using hardware
> > acceleration like libva or VDPAU.
>
> Seems that current totem master doesnt use videobalance element
> anymore. Take a look here [1]
> Could you test latest totem and see if the issue is already fixed?

Totem master works fine with the Fluendo VA-API plugin, but explodes
when used with the branch of gst-vaapi. I reported it to Gwenole, but
didn't hear back.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Alessandro Crismani]

Il 06/06/2012 17:28, Bastien Nocera ha scritto:
>> Seems that current totem master doesnt use videobalance element
>> >anymore. Take a look here [1]
>>
> Could you test latest totem and see if the issue is already fixed?
> Totem master works fine with the Fluendo VA-API plugin, but explodes
> when used with the branch of gst-vaapi. I reported it to Gwenole, but
> didn't hear back.
>

Ok, that saves me from a bit of compiling. Totem master requires GTK
3.5.2, which means diving into jhbuild or going the gnome git way.

Bastien, any input where I can follow the issue? Even better, any chance
of seeing Totem working with VA-API backported to 3.4 so that I don't
have to get all the 3.5.* libraries? :)

Cheers,
Alessandro
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Farkas Levente]

On Wed, Jun 6, 2012 at 5:33 PM, Alessandro Crismani
<[hidden email]> wrote:

> Il 06/06/2012 17:28, Bastien Nocera ha scritto:
>
>>> Seems that current totem master doesnt use videobalance element
>>> >anymore. Take a look here [1]
>>>
>> Could you test latest totem and see if the issue is already fixed?
>> Totem master works fine with the Fluendo VA-API plugin, but explodes
>> when used with the branch of gst-vaapi. I reported it to Gwenole, but
>> didn't hear back.
>>
>
> Ok, that saves me from a bit of compiling. Totem master requires GTK 3.5.2,
> which means diving into jhbuild or going the gnome git way.
>
> Bastien, any input where I can follow the issue? Even better, any chance of
> seeing Totem working with VA-API backported to 3.4 so that I don't have to
> get all the 3.5.* libraries? :)

is there any good reason of 3.5? or it's just bumped why not:-(
i don't see any good reason why gtk and glib requires always so new version.
in this case eg. most enterprise linux is excluded for about 5 years from now:-(



--
  Levente                               "Si vis pacem para bellum!"
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Bastien Nocera-2]

On Wed, 2012-06-06 at 17:33 +0200, Alessandro Crismani wrote:

> Il 06/06/2012 17:28, Bastien Nocera ha scritto:
> >> Seems that current totem master doesnt use videobalance element
> >> >anymore. Take a look here [1]
> >>
> > Could you test latest totem and see if the issue is already fixed?
> > Totem master works fine with the Fluendo VA-API plugin, but explodes
> > when used with the branch of gst-vaapi. I reported it to Gwenole, but
> > didn't hear back.
> >
>
> Ok, that saves me from a bit of compiling. Totem master requires GTK
> 3.5.2, which means diving into jhbuild or going the gnome git way.
>
> Bastien, any input where I can follow the issue? Even better, any chance
> of seeing Totem working with VA-API backported to 3.4 so that I don't
> have to get all the 3.5.* libraries? :)

Totem 3.5 makes use of autocluttersink, which rely on bug fixes in
clutter-gst. If you update those two, it might be enough.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Bastien Nocera-2]

On Wed, 2012-06-06 at 19:27 +0200, Farkas Levente wrote:

> On Wed, Jun 6, 2012 at 5:33 PM, Alessandro Crismani
> <[hidden email]> wrote:
> > Il 06/06/2012 17:28, Bastien Nocera ha scritto:
> >
> >>> Seems that current totem master doesnt use videobalance element
> >>> >anymore. Take a look here [1]
> >>>
> >> Could you test latest totem and see if the issue is already fixed?
> >> Totem master works fine with the Fluendo VA-API plugin, but explodes
> >> when used with the branch of gst-vaapi. I reported it to Gwenole, but
> >> didn't hear back.
> >>
> >
> > Ok, that saves me from a bit of compiling. Totem master requires GTK 3.5.2,
> > which means diving into jhbuild or going the gnome git way.
> >
> > Bastien, any input where I can follow the issue? Even better, any chance of
> > seeing Totem working with VA-API backported to 3.4 so that I don't have to
> > get all the 3.5.* libraries? :)
>
> is there any good reason of 3.5?

3.5 of what? Of Totem? It's where we use autocluttersink. Why is it not
in 3.4? Because autocluttersink was broken.

>  or it's just bumped why not:-(
> i don't see any good reason why gtk and glib requires always so new version.

Because they require new features or bug fixes that weren't in older
versions?

> in this case eg. most enterprise linux is excluded for about 5 years from now:-(

If Totem wasn't a core GNOME app, you might have a point.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne]

Le mercredi 06 juin 2012 à 18:30 +0100, Bastien Nocera a écrit :
> > Bastien, any input where I can follow the issue? Even better, any chance
> > of seeing Totem working with VA-API backported to 3.4 so that I don't
> > have to get all the 3.5.* libraries? :)
>
> Totem 3.5 makes use of autocluttersink, which rely on bug fixes in
> clutter-gst. If you update those two, it might be enough.

autocluttersink has never been tested along with gstreamer-vaapi (this
arrived after my work). Instead of wrapping an X windows into a some DRI
pixbuf, cluttersink (not autocluttersink) uses an new API in gstreamer
that let it use the GstBuffer as textures that are directly rendered
into the clutter scene.

I do think this approach is more efficient then the autocluttersink one,
and that fluendo could have implemented the GstVideoBuffer interface for
the VA buffer rather then using this. It's a bit of a design war, my
apology if this creates flames.

cheers,
Nicolas


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Farkas Levente]

On 06/06/2012 07:33 PM, Bastien Nocera wrote:

>>>> Could you test latest totem and see if the issue is already fixed?
>>>> Totem master works fine with the Fluendo VA-API plugin, but explodes
>>>> when used with the branch of gst-vaapi. I reported it to Gwenole, but
>>>> didn't hear back.
>>>>
>>>
>>> Ok, that saves me from a bit of compiling. Totem master requires GTK 3.5.2,
>>> which means diving into jhbuild or going the gnome git way.
>>>
>>> Bastien, any input where I can follow the issue? Even better, any chance of
>>> seeing Totem working with VA-API backported to 3.4 so that I don't have to
>>> get all the 3.5.* libraries? :)
>>
>> is there any good reason of 3.5?
>
> 3.5 of what? Of Totem? It's where we use autocluttersink. Why is it not
> in 3.4? Because autocluttersink was broken.

no i mean why gst-vvapi requires gtk and glib 3.5 or do i misunderstood it?
rhel/centos-6.2 still has gtk2-2.18.9 and glib2-2.22.5-6. if gst-vvapi
requires much newer version the it needs to rebuild to whole OS to make
it work. and if it's not possible to only update part of the whole os
thne no one can use it until rehl/centos-7 which is about 5 years from
now:-(

--
   Levente                               "Si vis pacem para bellum!"
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Julien Moutte]

Hi,

autocluttersink's role is just to provide a way for different platform specific sink plugins to be automatically discovered and used to render video buffers into clutter.

Now that being said, every clutter sink element just has to respect a common api to draw in the proper actor/texture. The way you handle this drawing and the fact that it's optimal or suboptimal is just a detail here.

To reply to your question about Fluendo's design choice here, you have to take in account that we always try to support old versions of GStreamer because of our customers.

We can't just make our products depend on a very recent API added for gst-vaapi and tell all our customers using old GStreamer versions to go to hell.

So I believe this is a good enough design to give freedom to everyone. There's several clutter sink elements out there and if you can make the gst-vaapi based on respect the common requirement to be usable with autocluttersink then it should just work no ?

Best regards,

Julien MOUTTE
C.T.O.

FLUENDO Influencing the Multimedia World

San Francisco, USA & Barcelona, SPAIN

Contact phone:
Spain:  [phone]
United States: [phone]
www.fluendo.com

Please consider the environment before printing this e-mail.

On Wed, Jun 6, 2012 at 7:41 PM, Nicolas Dufresne <[hidden email]> wrote:

Le mercredi 06 juin 2012 à 18:30 +0100, Bastien Nocera a écrit :
> > Bastien, any input where I can follow the issue? Even better, any chance
> > of seeing Totem working with VA-API backported to 3.4 so that I don't
> > have to get all the 3.5.* libraries? :)
>
> Totem 3.5 makes use of autocluttersink, which rely on bug fixes in
> clutter-gst. If you update those two, it might be enough.

autocluttersink has never been tested along with gstreamer-vaapi (this
arrived after my work). Instead of wrapping an X windows into a some DRI
pixbuf, cluttersink (not autocluttersink) uses an new API in gstreamer
that let it use the GstBuffer as textures that are directly rendered
into the clutter scene.

I do think this approach is more efficient then the autocluttersink one,
and that fluendo could have implemented the GstVideoBuffer interface for
the VA buffer rather then using this. It's a bit of a design war, my
apology if this creates flames.

cheers,
Nicolas


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Bastien Nocera-2]

On Wed, 2012-06-06 at 19:52 +0200, Julien Moutte wrote:

> Hi,
>
>
> autocluttersink's role is just to provide a way for different platform
> specific sink plugins to be automatically discovered and used to
> render video buffers into clutter.
>
>
> Now that being said, every clutter sink element just has to respect a
> common api to draw in the proper actor/texture. The way you handle
> this drawing and the fact that it's optimal or suboptimal is just a
> detail here.
>
>
> To reply to your question about Fluendo's design choice here, you have
> to take in account that we always try to support old versions of
> GStreamer because of our customers.
>
>
> We can't just make our products depend on a very recent API added for
> gst-vaapi and tell all our customers using old GStreamer versions to
> go to hell.

Given that you need a new version of clutter-gst for this to not
deadlock, this is a bit of a weird requirement.


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Bastien Nocera-2]

On Wed, 2012-06-06 at 19:46 +0200, Farkas Levente wrote:

> On 06/06/2012 07:33 PM, Bastien Nocera wrote:
> >>>> Could you test latest totem and see if the issue is already fixed?
> >>>> Totem master works fine with the Fluendo VA-API plugin, but explodes
> >>>> when used with the branch of gst-vaapi. I reported it to Gwenole, but
> >>>> didn't hear back.
> >>>>
> >>>
> >>> Ok, that saves me from a bit of compiling. Totem master requires GTK 3.5.2,
> >>> which means diving into jhbuild or going the gnome git way.
> >>>
> >>> Bastien, any input where I can follow the issue? Even better, any chance of
> >>> seeing Totem working with VA-API backported to 3.4 so that I don't have to
> >>> get all the 3.5.* libraries? :)
> >>
> >> is there any good reason of 3.5?
> >
> > 3.5 of what? Of Totem? It's where we use autocluttersink. Why is it not
> > in 3.4? Because autocluttersink was broken.
>
> no i mean why gst-vvapi requires gtk and glib 3.5 or do i misunderstood it?

You misunderstood. I don't know how you can have misinterpreted that,
unless you read it sideways.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Julien Moutte]

I am talking about the Video Accelerated decoder which is supposed to work with other video sinks than just the Clutter one.

Adopting GstVideoBuffer means using it both in the decoder and in the sink, meaning that our decoder would only work with recent GStreamer versions breaking our support agreeements.

So we have to stick to our own decoder and sink sharing hardware accelerated buffers with a common library which we provide ourself. We also provide a clutter video sink that uses that same library and that will be able to render VA buffers in clutter as long as clutter is recent enough to contain the fixes we provided.

Hope this makes it clearer. But in the end those are 2 different topics. Having an autocluttersink makes a lot of sense IMHO as we were using the exact same design before with autovideosink and the prepare-xid/window-handle logic. I can imagine many different hardware vendors that will want to support Clutter and hardware accelerated decoding and that will want to do it their own way by providing a sink element. (and I think we already had that discussion in a bug somewhere :) )

Salut ! :)

Julien MOUTTE
C.T.O.

FLUENDO Influencing the Multimedia World

San Francisco, USA & Barcelona, SPAIN

Contact phone:
Spain:  [phone]
United States: [phone]
www.fluendo.com

Please consider the environment before printing this e-mail.

On Wed, Jun 6, 2012 at 8:03 PM, Bastien Nocera <[hidden email]> wrote:

On Wed, 2012-06-06 at 19:52 +0200, Julien Moutte wrote:
> Hi,
>
>
> autocluttersink's role is just to provide a way for different platform
> specific sink plugins to be automatically discovered and used to
> render video buffers into clutter.
>
>
> Now that being said, every clutter sink element just has to respect a
> common api to draw in the proper actor/texture. The way you handle
> this drawing and the fact that it's optimal or suboptimal is just a
> detail here.
>
>
> To reply to your question about Fluendo's design choice here, you have
> to take in account that we always try to support old versions of
> GStreamer because of our customers.
>
>
> We can't just make our products depend on a very recent API added for
> gst-vaapi and tell all our customers using old GStreamer versions to
> go to hell.

Given that you need a new version of clutter-gst for this to not
deadlock, this is a bit of a weird requirement.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne]

Le mercredi 06 juin 2012 à 22:00 +0200, Julien Moutte a écrit :
> Hope this makes it clearer. But in the end those are 2 different
> topics. Having an autocluttersink makes a lot of sense IMHO as we were
> using the exact same design before with autovideosink and the
> prepare-xid/window-handle logic. I can imagine many different hardware
> vendors that will want to support Clutter and hardware accelerated
> decoding and that will want to do it their own way by providing a sink
> element. (and I think we already had that discussion in a bug
> somewhere :) )

The main issue is that it impose on video player dev the choice. I think
it's an error to have two cluttersink. We should have one, with the
automatic sink capabilities. In a way that when it's possible, it's not
going to use an external sink element.

This way, if it's raw, or using video/x-surface (the format behind
GstSurfaceBuffer format), it would simply render to the texture like the
original cluttersink. I don't understand in the first place why it was
decided that two elements was the way.

I agree that vaapisink is in a terrible state and should be fixed. But I
don't agree that it shall be used with Clutter, or know about Clutter. I
also have a strong opinion on choosing design toward Open Source
solution first, and then offering ways for maybe close source
alternative later.

cheers,
Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Julien Moutte]

On Wed, Jun 6, 2012 at 10:20 PM, Nicolas Dufresne <[hidden email]> wrote:

The main issue is that it impose on video player dev the choice. I think
it's an error to have two cluttersink. We should have one, with the
automatic sink capabilities. In a way that when it's possible, it's not
going to use an external sink element.

What you are describing is what autocluttersink is about iirc. The fact that it is nesting the native clutter-gst video sink in the normal scenario is just to keep things clean. It's a bin that plugs the proper element on the fly and the child element can be used independently as well.

This design is present in many other elements in GStreamer. Here the application developer has the choice to use the clutter-gst video sink directly if they really want to (like they could force ximagesink in the past), but it's recommended to use the autoplugger to support as much video formats as possible.

This also gives flexibility to deploy new features on an existing software park which was not possible at all when Totem was creating a clutter sink element by calling a static function shortcircuiting the GStreamer registry completely.

 

This way, if it's raw, or using video/x-surface (the format behind
GstSurfaceBuffer format), it would simply render to the texture like the
original cluttersink. I don't understand in the first place why it was
decided that two elements was the way.

Simply because there are other formats than raw RGB/YUV and video/x-surface out there... Sure you're probably gonna say that you don't care and that everybody should use an unstable API that is defined in gstreamer-plugins-bad since november 2011. That's quite radical and I don't see the point really, evenmore now that we know that this is gonna change *again* in GStreamer 1.0.

Note that I am not criticizing your design on the videocontext stuff. We proposed the exact same thing a long while ago and there was just not enough consensus around it to make it happen (cairo was regarded as the magic solution that was going to fix all our issues at that time).

 

I agree that vaapisink is in a terrible state and should be fixed. But I
don't agree that it shall be used with Clutter, or know about Clutter. I
also have a strong opinion on choosing design toward Open Source
solution first, and then offering ways for maybe close source
alternative later.

Personally I care about the user experience. Users have been able to play video files with video acceleration on Totem for quite a while because the design was flexible (playbin2 using discovered video sinks, and autovideosink allowing us to plug a different sink without touching the application) and we could automagically integrate our Fluendo VA decoder and Fluendo VA sink element. We had to overcome many regressions and behavior changes to continue supporting our user community across Ubuntu and Fedora releases but we managed it.

If you take that flexibility away due to personal convictions and force others to go by your design or stay out of the trip I think you are doing more damage to the OpenSource community than helping it. But that is just my personal opinion.

By the way I think the opposite than you. I think that Clutter should not know about GStreamer at all, it should provide a proper API to upload video surfaces from any media framework and eventually have some helper widgets as a side project leveraging GStreamer. It has always been GStreamer's responsibility to provide plugins to integrate with various rendering subsystems.

Julien

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Bastien Nocera-2]

On Wed, 2012-06-06 at 13:41 -0400, Nicolas Dufresne wrote:

> Le mercredi 06 juin 2012 à 18:30 +0100, Bastien Nocera a écrit :
> > > Bastien, any input where I can follow the issue? Even better, any chance
> > > of seeing Totem working with VA-API backported to 3.4 so that I don't
> > > have to get all the 3.5.* libraries? :)
> >
> > Totem 3.5 makes use of autocluttersink, which rely on bug fixes in
> > clutter-gst. If you update those two, it might be enough.
>
> autocluttersink has never been tested along with gstreamer-vaapi (this
> arrived after my work). Instead of wrapping an X windows into a some DRI
> pixbuf, cluttersink (not autocluttersink) uses an new API in gstreamer
> that let it use the GstBuffer as textures that are directly rendered
> into the clutter scene.
>
> I do think this approach is more efficient then the autocluttersink one,
> and that fluendo could have implemented the GstVideoBuffer interface for
> the VA buffer rather then using this. It's a bit of a design war, my
> apology if this creates flames.

Thinking about it some more. Is there any reason why gstreamer-vaapi
wouldn't work with Totem 3.4 or 3.6?

In 3.4, we just use cluttersink. Does gstreamer-vaapi work with
cluttersink?

In 3.6, we use autocluttersink. Surely, if Fluendo's VAAPI
flucluttersink isn't available, it should just fallback to the same
situation as in 3.4, right?

In my testing, Totem just exploded when gstreamer-vaapi was installed. I
can certainly file a bug, but I'd like to know what's expected of it
beforehand.

Cheers

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne]

Le mardi 12 juin 2012 à 19:37 +0100, Bastien Nocera a écrit :

> On Wed, 2012-06-06 at 13:41 -0400, Nicolas Dufresne wrote:
> > Le mercredi 06 juin 2012 à 18:30 +0100, Bastien Nocera a écrit :
> > > > Bastien, any input where I can follow the issue? Even better, any chance
> > > > of seeing Totem working with VA-API backported to 3.4 so that I don't
> > > > have to get all the 3.5.* libraries? :)
> > >
> > > Totem 3.5 makes use of autocluttersink, which rely on bug fixes in
> > > clutter-gst. If you update those two, it might be enough.
> >
> > autocluttersink has never been tested along with gstreamer-vaapi (this
> > arrived after my work). Instead of wrapping an X windows into a some DRI
> > pixbuf, cluttersink (not autocluttersink) uses an new API in gstreamer
> > that let it use the GstBuffer as textures that are directly rendered
> > into the clutter scene.
> >
> > I do think this approach is more efficient then the autocluttersink one,
> > and that fluendo could have implemented the GstVideoBuffer interface for
> > the VA buffer rather then using this. It's a bit of a design war, my
> > apology if this creates flames.
>
> Thinking about it some more. Is there any reason why gstreamer-vaapi
> wouldn't work with Totem 3.4 or 3.6?
>
> In 3.4, we just use cluttersink. Does gstreamer-vaapi work with
> cluttersink?

gstreamer-vaapi works fine with cluttersink, but 3.4 uses videobalance,
a software element which cause va decoder to be discarded.

>
> In 3.6, we use autocluttersink. Surely, if Fluendo's VAAPI
> flucluttersink isn't available, it should just fallback to the same
> situation as in 3.4, right?

Normally, and it's bugged, it should fallback to using cluttersink, as
long as no other software only element is forced into the pipeline, it
should work. This need investigation for sure. Note that I don't think
cluttersink implement video balance interface yet, so that would mean no
hue/brightness/saturation in that setup.

>
> In my testing, Totem just exploded when gstreamer-vaapi was installed. I
> can certainly file a bug, but I'd like to know what's expected of it
> beforehand.

Could you be more precise. You could describe the explosion maybe ?

cheers,
Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne]

Le mardi 12 juin 2012 à 19:37 +0100, Bastien Nocera a écrit :
> In 3.4, we just use cluttersink. Does gstreamer-vaapi work with
> cluttersink?

I dig a bit this one myself. Using stock vaapi driver on Fedora 17
crashed, just as a note. I compiled them from sources.

Then I found that the basic playbin2 does not work anymore, while
manually building the pipeline do. Digging more, I found that some code
in playbin2 have changed, it select the decoder, which output
video/x-surface, and try to find a decoder for that and error-out before
there is none. The next element being a sink. Looks like there is a
regression.

cheers,
Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne]

Le mardi 12 juin 2012 à 19:37 +0100, Bastien Nocera a écrit :
> In 3.6, we use autocluttersink. Surely, if Fluendo's VAAPI
> flucluttersink isn't available, it should just fallback to the same
> situation as in 3.4, right?

Sorry, I should have found this before. Here I attached a sample patch
(not tested) that show how to share the X11 display object with a
GStreamer pipeline. This is required with Intel drivers, as GL objects
cannot be shared between display connection, even if it's the same
display. You'll also find this code in the vide-player.c example in
clutter-gst repository.

cheers,
Nicolas

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel