decodebin2 + VA-API integration

[Gwenole Beauchesne-3]

Hi,

I am trying to integrate VA-APIi support to the standard decodebin2
element. Currently, the VA-API plugins (vaapiconvert, vaapidecode,
vaapisink) have GST_RANK_NONE rank.

So, the following won't work. That's normal:
$ gst-launch filesrc ! decodebin2 ! vaapisink

Then, I changed GST_RANK_NONE to GST_RANK_PRIMARY, decodebin2 will now see
and use vaapidecode. However, it will fail to link vaapidecode to
vaapisink. Rather, it tries to link vaapidecode0:src to some
decodepad2:proxypad3 element.

Would someone know why vaapisink0:sink is not tried instead? Or what
should be done in vaapisink so that decodebin2 actually links it to the
vaapidecode element it found previously?

Thanks,
Gwenole.

------------------------------------------------------------------------------

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Julien Moutte-2]

It's playbin2 that can notice that the autoplugging pulls in a sink element and decide to push that sink to playsink.

If you have defined caps so that autopluging terminates with a sink, it should work with playbin2 automatically. That's what we do with Fluendo VA decoders and sinks.

Cheers,

Julien Moutte,
FLUENDO S.A.

On Mon, May 10, 2010 at 4:27 PM, Gwenole Beauchesne <[hidden email]> wrote:

Hi,

I am trying to integrate VA-APIi support to the standard decodebin2
element. Currently, the VA-API plugins (vaapiconvert, vaapidecode,
vaapisink) have GST_RANK_NONE rank.

So, the following won't work. That's normal:
$ gst-launch filesrc ! decodebin2 ! vaapisink

Then, I changed GST_RANK_NONE to GST_RANK_PRIMARY, decodebin2 will now see
and use vaapidecode. However, it will fail to link vaapidecode to
vaapisink. Rather, it tries to link vaapidecode0:src to some
decodepad2:proxypad3 element.

Would someone know why vaapisink0:sink is not tried instead? Or what
should be done in vaapisink so that decodebin2 actually links it to the
vaapidecode element it found previously?

Thanks,
Gwenole.

------------------------------------------------------------------------------

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[michael smith-6-3]

On Mon, May 10, 2010 at 7:27 AM, Gwenole Beauchesne
<[hidden email]> wrote:

> Hi,
>
> I am trying to integrate VA-APIi support to the standard decodebin2
> element. Currently, the VA-API plugins (vaapiconvert, vaapidecode,
> vaapisink) have GST_RANK_NONE rank.
>
> So, the following won't work. That's normal:
> $ gst-launch filesrc ! decodebin2 ! vaapisink
>
> Then, I changed GST_RANK_NONE to GST_RANK_PRIMARY, decodebin2 will now see
> and use vaapidecode. However, it will fail to link vaapidecode to
> vaapisink. Rather, it tries to link vaapidecode0:src to some
> decodepad2:proxypad3 element.

The proxy pad is an internal detail of how ghost pads work. The
capabilities and so on of the ghost pad should be the same as the real
pad, so this should work fine. If it's not... you need to figure out
why it's not linking - the actual pad seems correct here.

Mike

------------------------------------------------------------------------------

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Gwenole Beauchesne-3]

Hi,

On Mon, 10 May 2010, Michael Smith wrote:

>> $ gst-launch filesrc ! decodebin2 ! vaapisink
>>
>> Then, I changed GST_RANK_NONE to GST_RANK_PRIMARY, decodebin2 will now see
>> and use vaapidecode. However, it will fail to link vaapidecode to
>> vaapisink. Rather, it tries to link vaapidecode0:src to some
>> decodepad2:proxypad3 element.
>
> The proxy pad is an internal detail of how ghost pads work. The
> capabilities and so on of the ghost pad should be the same as the real
> pad, so this should work fine. If it's not... you need to figure out
> why it's not linking - the actual pad seems correct here.

OK, the reason why decodebin2 doesn't link vaapidecode to vaapisink is
because it considers video/x-vaapi-surface as non final caps so it's still
trying to find a better candidate for the next element. i.e. it tries to
find another element to place between vaapidecode and vaapisink with
video/x-vaapi-surface caps.

The problem is decode_bin->caps are hardcoded to some defaults that don't
include video/x-vaapi-surface. So are_raw_caps() returns FALSE and we
never go to the expose_pad case.

This means, gst-launch filesrc ! decodebin2 ! vaapisink can never work
unless either (i) decode_bin->caps include video/x-vaapi-surface or (ii)
we replace video/x-vaapi-surface with some known caps, possibly
specialised with a specific format to mean a VA surface.

What would be the proper fix to this problem? Could we consider an
"official" video/x-hw-surface caps with format = (string)
VAAPI|VDPAU|Cairo, etc.?

BTW, I now can get a working playbin2 pipeline. The trick is to make sure
any VA-API sink (vaapisink, cluttergstsink, etc.) has a custom
buffer_alloc function and communicates the VA display through it.

Thanks,
Gwenole.

------------------------------------------------------------------------------

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel