Problem with etnaviv and gstreamer 1.0

[Lukasz Majewski]

Dear All,

I would like to ask if somebody had similar problem:

- I'm using weston 2.0.0 with having gstreamer on top of it
- Linux v4.14 @ imx6q
- Gstreamer 1.12.2
- How to trigger:
        export XDG_RUNTIME_DIR=/run/user/0
        gst-launch-1.0  -v gltestsrc pattern=smpte ! glimagesink

        I do see a black rectangle
        and following error pops up:

        high: GL error from API id:2, GL_INVALID_FRAMEBUFFER_OPERATION
        in glDrawElements(incomplete framebuffer)

- I've applied following patches:
https://bugzilla.gnome.org/show_bug.cgi?id=783521

But this didn't help.

The problem is not present when I do not enable etnaviv in Linux kernel
(CONFIG_DRM_ETNAVIV=n) - it works painfully slow with sw rendering.



I would like to ask if somebody is already working on this issue?

Any hints on how to (properly) tackle this problem are more than
welcome.



Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: [phone] Fax: [phone] Email: [hidden email]

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Lukasz Majewski]

Hi Luís,

Thanks for your reply.

> Hi Lukasz,
>
> I'm not part of the mesa/etnaviv development team, but I've tried
> somethings with gstreamer, maybe I can provide some useful
> information...
>
> Can you try xvimagesink instead of glimagesink?

It would be hard since I do support Weston/Wayland - not X11.

>
> Etnaviv provides the driver for the 3D GPU, but I believe the video is
> normally output out through the 2D unit, the GC320 in i.MX6 via
> xf86-armada drm driver.
>
> I think the best approach for decoding and outputting video is to
> implement the VAAPI and/or VDPAU support in mesa, right on etnaviv

I've read that etnaviv has some issues with YUV->RGB conversion and
with some shaders (as they are required by Gstreamer) [1], [2].

> making the CODA VPU unit available to all the applications that are
> already VAAPI and VDPAU aware, like vlc, kodi and others. That way
> standard desktop applications can be used without much changes or one
> can develop for both platforms with less effort. I am currently
> starting such project to bridge the gap with a prototype driver that
> could be integrated in etanviv and mesa. Mesa already implements
> VAAPI and VDPAU, I believe one just needs to implement an internal
> mesa API named VL.

Please correct me if I'm wrong - is the above proposal is for only
playing movies?

Now, I do have a problem to get very basic test working
(black rectangle is shown):
gst-launch-1.0  -v gltestsrc pattern=smpte ! glimagesink

>
> Meanwhile, maybe you can try the following which also makes use of
> the CODA VPU unit with gstreamer:
>
> wget
> http://linode.boundarydevices.com/videos/trailer_1080p_h264_mp3.avi
>
> gst-launch-1.0 filesrc
> location=/home/ubuntu/trailer_1080p_h264_mp3.avi ! \ avidemux !
> h264parse ! v4l2video1dec capture-io-mode=dmabuf ! \ xvimagesink
> device=1 sync=0

Ok (despite the xvimagesink part).

>
> The above works both on my i.MX6Q and on my i.MX6QP under X, I haven't
> tried with wayland, but should work just the same.
>
> I based this test on the following post by Fabio Estevam:
> https://imxdev.gitlab.io/tutorial/Decoding_video_with_a_mainline_kernel_on_i.MX6/

I've followed this tutorial - I'm not able to get the movie displayed:

The following line gives some output to file.

export XDG_RUNTIME_DIR=/run/user/0;gst-launch-1.0 filesrc
location=trailer_1080p_h264_mp3.avi ! avidemux ! h264parse !
v4l2video1dec capture-io-mode=dmabuf ! filesink location=/tmp/out
sync=false

And with the below one:

gst-launch-1.0 filesrc location=trailer_1080p_h264_mp3.avi !
 avidemux ! h264parse ! v4l2video1dec capture-io-mode=dmabuf !
 video/x-raw,format=YUY2 ! glimagesink

Error is:
---------
etna_rs_gen_clear_surface:77: etna_rs_gen_clear_surface: Unhandled
clear fmt PIPE_FORMAT_R8G8_UNORM

I'm just wondering -> x-raw,format=YUY2 explicitly asks for YUY2 format
(which shall be supported by etnaviv).... why then in the middle there
is PIPE_FORMAT_R8G8_UNORM ?


And with this one:
gst-launch-1.0 filesrc location=trailer_1080p_h264_mp3.avi ! avidemux !
h264parse ! v4l2video1dec capture-io-mode=dmabuf ! videoconvert !
fbdevsink sync=false

No errors are present, but also I do not see any output.



>

[1] - https://bugzilla.gnome.org/show_bug.cgi?id=783521
[2] -
https://people.freedesktop.org/~cbrill/dri-log/index.php?channel=etnaviv&highlight_names=&date=2017-03-29

>
>  Regards,
> Luís Mendes
> Hardware and Software engineer



Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: [phone] Fax: [phone] Email: [hidden email]

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-3]

Le mardi 05 décembre 2017 à 15:33 +0100, Lukasz Majewski a écrit :

> And with the below one:
>
> gst-launch-1.0 filesrc location=trailer_1080p_h264_mp3.avi !
>  avidemux ! h264parse ! v4l2video1dec capture-io-mode=dmabuf !
>  video/x-raw,format=YUY2 ! glimagesink
>
> Error is:
> ---------
> etna_rs_gen_clear_surface:77: etna_rs_gen_clear_surface: Unhandled
> clear fmt PIPE_FORMAT_R8G8_UNORM
>
> I'm just wondering -> x-raw,format=YUY2 explicitly asks for YUY2 format
> (which shall be supported by etnaviv).... why then in the middle there
> is PIPE_FORMAT_R8G8_UNORM ?

That one requires the direct dmabuf upload patch, otherwise it will use
RG textures with two EGLImage.

Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Luís Mendes]

Replies in between.

Regards,
Luís Mendes

On Tue, Dec 5, 2017 at 2:33 PM, Lukasz Majewski <[hidden email]> wrote:

> Hi Luís,
>
> Thanks for your reply.
>
>> Hi Lukasz,
>>
>> I'm not part of the mesa/etnaviv development team, but I've tried
>> somethings with gstreamer, maybe I can provide some useful
>> information...
>>
>> Can you try xvimagesink instead of glimagesink?
>
> It would be hard since I do support Weston/Wayland - not X11.

Ok.

>
>>
>> Etnaviv provides the driver for the 3D GPU, but I believe the video is
>> normally output out through the 2D unit, the GC320 in i.MX6 via
>> xf86-armada drm driver.
>>
>> I think the best approach for decoding and outputting video is to
>> implement the VAAPI and/or VDPAU support in mesa, right on etnaviv
>
> I've read that etnaviv has some issues with YUV->RGB conversion and
> with some shaders (as they are required by Gstreamer) [1], [2].
>
>> making the CODA VPU unit available to all the applications that are
>> already VAAPI and VDPAU aware, like vlc, kodi and others. That way
>> standard desktop applications can be used without much changes or one
>> can develop for both platforms with less effort. I am currently
>> starting such project to bridge the gap with a prototype driver that
>> could be integrated in etanviv and mesa. Mesa already implements
>> VAAPI and VDPAU, I believe one just needs to implement an internal
>> mesa API named VL.
>
> Please correct me if I'm wrong - is the above proposal is for only
> playing movies?

The above proposal is mainly for playing videos, that is,
decoding/encoding video sources, could be a stream from a camera,
through a kind of "standard" API that is also used for desktop
applications, but also provides an image output path to the screen.
Kodi seems to use this to output the video but also to overlay text
and icons.

>
> Now, I do have a problem to get very basic test working
> (black rectangle is shown):
> gst-launch-1.0  -v gltestsrc pattern=smpte ! glimagesink

The above is working on my setup under Xorg. Under weston also works,
but image is distorted. What kernel/mesa version are you using?

>
>>
>> Meanwhile, maybe you can try the following which also makes use of
>> the CODA VPU unit with gstreamer:
>>
>> wget
>> http://linode.boundarydevices.com/videos/trailer_1080p_h264_mp3.avi
>>
>> gst-launch-1.0 filesrc
>> location=/home/ubuntu/trailer_1080p_h264_mp3.avi ! \ avidemux !
>> h264parse ! v4l2video1dec capture-io-mode=dmabuf ! \ xvimagesink
>> device=1 sync=0
>
> Ok (despite the xvimagesink part).
>
>>
>> The above works both on my i.MX6Q and on my i.MX6QP under X, I haven't
>> tried with wayland, but should work just the same.
>>
>> I based this test on the following post by Fabio Estevam:
>> https://imxdev.gitlab.io/tutorial/Decoding_video_with_a_mainline_kernel_on_i.MX6/
>
> I've followed this tutorial - I'm not able to get the movie displayed:

That's odd, again it should work. It works on  my i.MX6Q and i.MX6QP
as long as I haven't any graphical display running. Have you installed
the CODA VPU firmware in /lib/firmware/vpu?

>
> The following line gives some output to file.
>
> export XDG_RUNTIME_DIR=/run/user/0;gst-launch-1.0 filesrc
> location=trailer_1080p_h264_mp3.avi ! avidemux ! h264parse !
> v4l2video1dec capture-io-mode=dmabuf ! filesink location=/tmp/out
> sync=false
>
> And with the below one:
>
> gst-launch-1.0 filesrc location=trailer_1080p_h264_mp3.avi !
>  avidemux ! h264parse ! v4l2video1dec capture-io-mode=dmabuf !
>  video/x-raw,format=YUY2 ! glimagesink

I get no such error. Although the video is not playable in this mode,
it keeps repeating the first bunch of frames in a loop until the video
duration elapses in a distorted reddish colored window.
You are probably using an old mesa version. Despite what you say
regarding the color space unexpected conversion, if I'm not mistaken
R8G8_UNORM support was only implemented relatively recently, so you
are likely using an old mesa version. I would recommend you to test
with mesa-17.3.0 or the release candidate 6 that is already available,
Etnaviv improved a lot lately and now supports openGL 2.1, maybe you
should try mesa-17.3.0.

>
> Error is:
> ---------
> etna_rs_gen_clear_surface:77: etna_rs_gen_clear_surface: Unhandled
> clear fmt PIPE_FORMAT_R8G8_UNORM
>
> I'm just wondering -> x-raw,format=YUY2 explicitly asks for YUY2 format
> (which shall be supported by etnaviv).... why then in the middle there
> is PIPE_FORMAT_R8G8_UNORM ?
>
>
> And with this one:
> gst-launch-1.0 filesrc location=trailer_1080p_h264_mp3.avi ! avidemux !
> h264parse ! v4l2video1dec capture-io-mode=dmabuf ! videoconvert !
> fbdevsink sync=false
>
> No errors are present, but also I do not see any output.

I also see no image under Wayland (using Ubuntu 17.10 desktop, or
Weston on i.MX6QP), but it works (slowly) when I don't have any
graphical environment running.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-3]

Le mardi 05 décembre 2017 à 16:07 +0000, Luís Mendes a écrit :
> > > making the CODA VPU unit available to all the applications that are
> > > already VAAPI and VDPAU aware, like vlc, kodi and others. That way

FFMPEG, Khodi and other applicaitons (Chromium, GStreamer) already
support this type of decoder. It's a state full decoder, while VAAPI is
designed for state less decoders. It also make not sense to wrap this
inside a GPU driver, because it's not a GPU.

Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-3]

Le mardi 05 décembre 2017 à 16:07 +0000, Luís Mendes a écrit :

> > > The above works both on my i.MX6Q and on my i.MX6QP under X, I
> > > haven't
> > > tried with wayland, but should work just the same.
> > >
> > > I based this test on the following post by Fabio Estevam:
> > > https://imxdev.gitlab.io/tutorial/Decoding_video_with_a_mainline_
> > > kernel_on_i.MX6/
> >
> > I've followed this tutorial - I'm not able to get the movie
> > displayed:
>
> That's odd, again it should work. It works on  my i.MX6Q and i.MX6QP
> as long as I haven't any graphical display running. Have you
> installed
> the CODA VPU firmware in /lib/firmware/vpu?

You are confusing the mainlined CODA driver with the one provided by
the vendor.

Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Lukasz Majewski]

Hi Luis,

> Replies in between.
>
> Regards,
> Luís Mendes
>
> On Tue, Dec 5, 2017 at 2:33 PM, Lukasz Majewski <[hidden email]> wrote:
> > Hi Luís,
> >
> > Thanks for your reply.
> >  
> >> Hi Lukasz,
> >>
> >> I'm not part of the mesa/etnaviv development team, but I've tried
> >> somethings with gstreamer, maybe I can provide some useful
> >> information...
> >>
> >> Can you try xvimagesink instead of glimagesink?  
> >
> > It would be hard since I do support Weston/Wayland - not X11.  
>
> Ok.
>
> >  
> >>
> >> Etnaviv provides the driver for the 3D GPU, but I believe the
> >> video is normally output out through the 2D unit, the GC320 in
> >> i.MX6 via xf86-armada drm driver.
> >>
> >> I think the best approach for decoding and outputting video is to
> >> implement the VAAPI and/or VDPAU support in mesa, right on
> >> etnaviv  
> >
> > I've read that etnaviv has some issues with YUV->RGB conversion and
> > with some shaders (as they are required by Gstreamer) [1], [2].
> >  
> >> making the CODA VPU unit available to all the applications that are
> >> already VAAPI and VDPAU aware, like vlc, kodi and others. That way
> >> standard desktop applications can be used without much changes or
> >> one can develop for both platforms with less effort. I am currently
> >> starting such project to bridge the gap with a prototype driver
> >> that could be integrated in etanviv and mesa. Mesa already
> >> implements VAAPI and VDPAU, I believe one just needs to implement
> >> an internal mesa API named VL.  
> >
> > Please correct me if I'm wrong - is the above proposal is for only
> > playing movies?  
>
> The above proposal is mainly for playing videos, that is,
> decoding/encoding video sources, could be a stream from a camera,
> through a kind of "standard" API that is also used for desktop
> applications, but also provides an image output path to the screen.
> Kodi seems to use this to output the video but also to overlay text
> and icons.
>
> >
> > Now, I do have a problem to get very basic test working
> > (black rectangle is shown):
> > gst-launch-1.0  -v gltestsrc pattern=smpte ! glimagesink  
>
> The above is working on my setup under Xorg. Under weston also works,
> but image is distorted. What kernel/mesa version are you using?

Kernel is v4.14, mesa 17.0.6. Now I will test with 17.1.7.

GST 1.12.2

>
> >  
> >>
> >> Meanwhile, maybe you can try the following which also makes use of
> >> the CODA VPU unit with gstreamer:
> >>
> >> wget
> >> http://linode.boundarydevices.com/videos/trailer_1080p_h264_mp3.avi
> >>
> >> gst-launch-1.0 filesrc
> >> location=/home/ubuntu/trailer_1080p_h264_mp3.avi ! \ avidemux !
> >> h264parse ! v4l2video1dec capture-io-mode=dmabuf ! \ xvimagesink
> >> device=1 sync=0  
> >
> > Ok (despite the xvimagesink part).
> >  
> >>
> >> The above works both on my i.MX6Q and on my i.MX6QP under X, I
> >> haven't tried with wayland, but should work just the same.
> >>
> >> I based this test on the following post by Fabio Estevam:
> >> https://imxdev.gitlab.io/tutorial/Decoding_video_with_a_mainline_kernel_on_i.MX6/ 
> >
> > I've followed this tutorial - I'm not able to get the movie
> > displayed:  
>
> That's odd, again it should work. It works on  my i.MX6Q and i.MX6QP
> as long as I haven't any graphical display running. Have you installed
> the CODA VPU firmware in /lib/firmware/vpu?

I've enabled CODA in kernel with adding /lib/firmware/vpu for it.

It seems like it is properly configured.

>
> >
> > The following line gives some output to file.
> >
> > export XDG_RUNTIME_DIR=/run/user/0;gst-launch-1.0 filesrc
> > location=trailer_1080p_h264_mp3.avi ! avidemux ! h264parse !
> > v4l2video1dec capture-io-mode=dmabuf ! filesink location=/tmp/out
> > sync=false
> >
> > And with the below one:
> >
> > gst-launch-1.0 filesrc location=trailer_1080p_h264_mp3.avi !
> >  avidemux ! h264parse ! v4l2video1dec capture-io-mode=dmabuf !
> >  video/x-raw,format=YUY2 ! glimagesink  
>
> I get no such error. Although the video is not playable in this mode,
> it keeps repeating the first bunch of frames in a loop until the video
> duration elapses in a distorted reddish colored window.
> You are probably using an old mesa version. Despite what you say
> regarding the color space unexpected conversion, if I'm not mistaken
> R8G8_UNORM support was only implemented relatively recently, so you
> are likely using an old mesa version.

As noted above - 17.0.6 / 17.1.7

So the R8G8_UNORM is now supported in the newest mesa?

> I would recommend you to test
> with mesa-17.3.0 or the release candidate 6 that is already available,
> Etnaviv improved a lot lately and now supports openGL 2.1, maybe you
> should try mesa-17.3.0.

Ok. I will try mesa-17.3.0

>
> >
> > Error is:
> > ---------
> > etna_rs_gen_clear_surface:77: etna_rs_gen_clear_surface: Unhandled
> > clear fmt PIPE_FORMAT_R8G8_UNORM
> >
> > I'm just wondering -> x-raw,format=YUY2 explicitly asks for YUY2
> > format (which shall be supported by etnaviv).... why then in the
> > middle there is PIPE_FORMAT_R8G8_UNORM ?
> >
> >
> > And with this one:
> > gst-launch-1.0 filesrc location=trailer_1080p_h264_mp3.avi !
> > avidemux ! h264parse ! v4l2video1dec capture-io-mode=dmabuf !
> > videoconvert ! fbdevsink sync=false
> >
> > No errors are present, but also I do not see any output.  
>
> I also see no image under Wayland (using Ubuntu 17.10 desktop, or
> Weston on i.MX6QP), but it works (slowly) when I don't have any
> graphical environment running.

Ok. I will test it with weston disabled.

Thanks for tip.

> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel



Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: [phone] Fax: [phone] Email: [hidden email]

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Lukasz Majewski]

On Tue, 05 Dec 2017 10:03:33 -0500
Nicolas Dufresne <[hidden email]> wrote:

> Le mardi 05 décembre 2017 à 15:33 +0100, Lukasz Majewski a écrit :
> > And with the below one:
> >
> > gst-launch-1.0 filesrc location=trailer_1080p_h264_mp3.avi !
> >  avidemux ! h264parse ! v4l2video1dec capture-io-mode=dmabuf !
> >  video/x-raw,format=YUY2 ! glimagesink
> >
> > Error is:
> > ---------
> > etna_rs_gen_clear_surface:77: etna_rs_gen_clear_surface: Unhandled
> > clear fmt PIPE_FORMAT_R8G8_UNORM
> >
> > I'm just wondering -> x-raw,format=YUY2 explicitly asks for YUY2
> > format (which shall be supported by etnaviv).... why then in the
> > middle there is PIPE_FORMAT_R8G8_UNORM ?  
>
> That one requires the direct dmabuf upload patch, otherwise it will
> use RG textures with two EGLImage.

This patch is needed for mesa or for gst?

>
> Nicolas



Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: [phone] Fax: [phone] Email: [hidden email]

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-3]

Le mardi 05 décembre 2017 à 17:34 +0100, Lukasz Majewski a écrit :

> > Le mardi 05 décembre 2017 à 15:33 +0100, Lukasz Majewski a écrit :
> > > And with the below one:
> > >
> > > gst-launch-1.0 filesrc location=trailer_1080p_h264_mp3.avi !
> > >  avidemux ! h264parse ! v4l2video1dec capture-io-mode=dmabuf !
> > >  video/x-raw,format=YUY2 ! glimagesink
> > >
> > > Error is:
> > > ---------
> > > etna_rs_gen_clear_surface:77: etna_rs_gen_clear_surface:
> Unhandled
> > > clear fmt PIPE_FORMAT_R8G8_UNORM
> > >
> > > I'm just wondering -> x-raw,format=YUY2 explicitly asks for YUY2
> > > format (which shall be supported by etnaviv).... why then in the
> > > middle there is PIPE_FORMAT_R8G8_UNORM ?  
> >
> > That one requires the direct dmabuf upload patch, otherwise it will
> > use RG textures with two EGLImage.
>
> This patch is needed for mesa or for gst?

It's a GStreamer patch to use this Etnaviv specific YUY2 feature.

Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Lukasz Majewski]

Hi Nicolas,

> Le mardi 05 décembre 2017 à 17:34 +0100, Lukasz Majewski a écrit :
> > > Le mardi 05 décembre 2017 à 15:33 +0100, Lukasz Majewski a
> > > écrit :  
> > > > And with the below one:
> > > >
> > > > gst-launch-1.0 filesrc location=trailer_1080p_h264_mp3.avi !
> > > >  avidemux ! h264parse ! v4l2video1dec capture-io-mode=dmabuf !
> > > >  video/x-raw,format=YUY2 ! glimagesink
> > > >
> > > > Error is:
> > > > ---------
> > > > etna_rs_gen_clear_surface:77: etna_rs_gen_clear_surface:  
> > Unhandled  
> > > > clear fmt PIPE_FORMAT_R8G8_UNORM
> > > >
> > > > I'm just wondering -> x-raw,format=YUY2 explicitly asks for YUY2
> > > > format (which shall be supported by etnaviv).... why then in the
> > > > middle there is PIPE_FORMAT_R8G8_UNORM ?    
> > >
> > > That one requires the direct dmabuf upload patch, otherwise it
> > > will use RG textures with two EGLImage.  
> >
> > This patch is needed for mesa or for gst?  
>
> It's a GStreamer patch to use this Etnaviv specific YUY2 feature.

Ok, I will look for it. Thanks for tip.

>
> Nicolas



Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: [phone] Fax: [phone] Email: [hidden email]

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Lukasz Majewski]

On Tue, 5 Dec 2017 17:49:48 +0100
Lukasz Majewski <[hidden email]> wrote:

> Hi Nicolas,
>
> > Le mardi 05 décembre 2017 à 17:34 +0100, Lukasz Majewski a écrit :  
> > > > Le mardi 05 décembre 2017 à 15:33 +0100, Lukasz Majewski a
> > > > écrit :    
> > > > > And with the below one:
> > > > >
> > > > > gst-launch-1.0 filesrc location=trailer_1080p_h264_mp3.avi !
> > > > >  avidemux ! h264parse ! v4l2video1dec capture-io-mode=dmabuf !
> > > > >  video/x-raw,format=YUY2 ! glimagesink
> > > > >
> > > > > Error is:
> > > > > ---------
> > > > > etna_rs_gen_clear_surface:77: etna_rs_gen_clear_surface:    
> > > Unhandled    
> > > > > clear fmt PIPE_FORMAT_R8G8_UNORM
> > > > >
> > > > > I'm just wondering -> x-raw,format=YUY2 explicitly asks for
> > > > > YUY2 format (which shall be supported by etnaviv).... why
> > > > > then in the middle there is PIPE_FORMAT_R8G8_UNORM ?      
> > > >
> > > > That one requires the direct dmabuf upload patch, otherwise it
> > > > will use RG textures with two EGLImage.    
> > >
> > > This patch is needed for mesa or for gst?    
> >
> > It's a GStreamer patch to use this Etnaviv specific YUY2 feature.  
>
> Ok, I will look for it. Thanks for tip.

But I do have this patch (and the whole series):
https://bug783521.bugzilla-attachments.gnome.org/attachment.cgi?id=353352

already applied.

Do I need something more?

>
> >
> > Nicolas  
>
>
>
> Best regards,
>
> Lukasz Majewski
>
> --
>
> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: [phone] Fax: [phone] Email: [hidden email]

Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: [phone] Fax: [phone] Email: [hidden email]

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-3]

Le mardi 05 décembre 2017 à 17:51 +0100, Lukasz Majewski a écrit :

> > > It's a GStreamer patch to use this Etnaviv specific YUY2
> feature.  
> >
> > Ok, I will look for it. Thanks for tip.
>
> But I do have this patch (and the whole series):
> https://bug783521.bugzilla-attachments.gnome.org/attachment.cgi?id=35
> 3352
>
> already applied.
>
> Do I need something more?

Some debugging to figure-out why it failed ? If the YUY2 dmabuf
importation failed, we fallback to RG88 technique (which should fail
too, it's a bug in Etnaviv).

Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Lukasz Majewski]

Hi Nicolas,

> Le mardi 05 décembre 2017 à 17:51 +0100, Lukasz Majewski a écrit :
> > > > It's a GStreamer patch to use this Etnaviv specific YUY2  
> > feature.    
> > >
> > > Ok, I will look for it. Thanks for tip.  
> >
> > But I do have this patch (and the whole series):
> > https://bug783521.bugzilla-attachments.gnome.org/attachment.cgi?id=35
> > 3352
> >
> > already applied.
> >
> > Do I need something more?  
>
> Some debugging to figure-out why it failed ? If the YUY2 dmabuf
> importation failed, we fallback to RG88 technique (which should fail
> too, it's a bug in Etnaviv).

Ach. Ok.

I thought that I do need some other patches as well.

Now it is clear.

>
> Nicolas



Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: [phone] Fax: [phone] Email: [hidden email]

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Luís Mendes]

Hi Nicolas,

Replies in between.

Luís Mendes

On Tue, Dec 5, 2017 at 4:17 PM, Nicolas Dufresne
<[hidden email]> wrote:
> Le mardi 05 décembre 2017 à 16:07 +0000, Luís Mendes a écrit :
>> > > making the CODA VPU unit available to all the applications that are
>> > > already VAAPI and VDPAU aware, like vlc, kodi and others. That way
>
> FFMPEG, Khodi and other applicaitons (Chromium, GStreamer) already
> support this type of decoder. It's a state full decoder, while VAAPI is
> designed for state less decoders. It also make not sense to wrap this
> inside a GPU driver, because it's not a GPU.

If you look into mesa you will find that it implements the VAAPI and
VDPAU APIs and that i965, amdgpu and nouveau drivers implement the
bridge through the VL internal mesa API. In ubuntu the resulting
dynamic libraries from mesa compilation will be packaged into
mesa-va-drivers and mesa-vdpau-drivers. So, yes the VPU
implementations are side-by-side with the GPU support.

>
> Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Luís Mendes]

>
> Kernel is v4.14, mesa 17.0.6. Now I will test with 17.1.7.

The kernel version is Ok, but the mesa version is old, many
improvements have been introduced by the developers and important
changes are contained in mesa-17.3.0.

>
> GST 1.12.2
>
...

>> I get no such error. Although the video is not playable in this mode,
>> it keeps repeating the first bunch of frames in a loop until the video
>> duration elapses in a distorted reddish colored window.
>> You are probably using an old mesa version. Despite what you say
>> regarding the color space unexpected conversion, if I'm not mistaken
>> R8G8_UNORM support was only implemented relatively recently, so you
>> are likely using an old mesa version.
>
> As noted above - 17.0.6 / 17.1.7
>
> So the R8G8_UNORM is now supported in the newest mesa?

Yes, it is supported.

>
>> I would recommend you to test
>> with mesa-17.3.0 or the release candidate 6 that is already available,
>> Etnaviv improved a lot lately and now supports openGL 2.1, maybe you
>> should try mesa-17.3.0.
>
> Ok. I will try mesa-17.3.0
>
>>
>> >
>> > Error is:
>> > ---------
>> > etna_rs_gen_clear_surface:77: etna_rs_gen_clear_surface: Unhandled
>> > clear fmt PIPE_FORMAT_R8G8_UNORM
>> >
>> > I'm just wondering -> x-raw,format=YUY2 explicitly asks for YUY2
>> > format (which shall be supported by etnaviv).... why then in the
>> > middle there is PIPE_FORMAT_R8G8_UNORM ?
>> >
>> >
>> > And with this one:
>> > gst-launch-1.0 filesrc location=trailer_1080p_h264_mp3.avi !
>> > avidemux ! h264parse ! v4l2video1dec capture-io-mode=dmabuf !
>> > videoconvert ! fbdevsink sync=false
>> >
>> > No errors are present, but also I do not see any output.
>>
>> I also see no image under Wayland (using Ubuntu 17.10 desktop, or
>> Weston on i.MX6QP), but it works (slowly) when I don't have any
>> graphical environment running.
>
> Ok. I will test it with weston disabled.
>
> Thanks for tip.
>

Glad I could help.

Regards,
Luís Mendes
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-3]

Le mardi 05 décembre 2017 à 17:06 +0000, Luís Mendes a écrit :

> On Tue, Dec 5, 2017 at 4:17 PM, Nicolas Dufresne
> <[hidden email]> wrote:
> > Le mardi 05 décembre 2017 à 16:07 +0000, Luís Mendes a écrit :
> > > > > making the CODA VPU unit available to all the applications
> > > > > that are
> > > > > already VAAPI and VDPAU aware, like vlc, kodi and others.
> > > > > That way
> >
> > FFMPEG, Khodi and other applicaitons (Chromium, GStreamer) already
> > support this type of decoder. It's a state full decoder, while
> > VAAPI is
> > designed for state less decoders. It also make not sense to wrap
> > this
> > inside a GPU driver, because it's not a GPU.
>
> If you look into mesa you will find that it implements the VAAPI and
> VDPAU APIs and that i965, amdgpu and nouveau drivers implement the
> bridge through the VL internal mesa API. In ubuntu the resulting
> dynamic libraries from mesa compilation will be packaged into
> mesa-va-drivers and mesa-vdpau-drivers. So, yes the VPU
> implementations are side-by-side with the GPU support.

I know this very well. It make sense for Intel, AMD and NVidia driver,
because the CODEC (which are just accelerators) are tightly coupled
with the GPU. These are also state less CODEC.

On the other hand, the CODA IP is not coupled with the GPU and is state
full. The VAAPI (and the deprecated VDPAU interfaces) are not designed
to support state full decoder, and the extra indirections does not
provide much benefit.

Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Luís Mendes]

Hi Nicolas,

On Tue, Dec 5, 2017 at 7:10 PM, Nicolas Dufresne
<[hidden email]> wrote:

> Le mardi 05 décembre 2017 à 17:06 +0000, Luís Mendes a écrit :
>> On Tue, Dec 5, 2017 at 4:17 PM, Nicolas Dufresne
>> <[hidden email]> wrote:
>> > Le mardi 05 décembre 2017 à 16:07 +0000, Luís Mendes a écrit :
>> > > > > making the CODA VPU unit available to all the applications
>> > > > > that are
>> > > > > already VAAPI and VDPAU aware, like vlc, kodi and others.
>> > > > > That way
>> >
>> > FFMPEG, Khodi and other applicaitons (Chromium, GStreamer) already
>> > support this type of decoder. It's a state full decoder, while
>> > VAAPI is
>> > designed for state less decoders. It also make not sense to wrap
>> > this
>> > inside a GPU driver, because it's not a GPU.
>>
>> If you look into mesa you will find that it implements the VAAPI and
>> VDPAU APIs and that i965, amdgpu and nouveau drivers implement the
>> bridge through the VL internal mesa API. In ubuntu the resulting
>> dynamic libraries from mesa compilation will be packaged into
>> mesa-va-drivers and mesa-vdpau-drivers. So, yes the VPU
>> implementations are side-by-side with the GPU support.
>
> I know this very well. It make sense for Intel, AMD and NVidia driver,
> because the CODEC (which are just accelerators) are tightly coupled
> with the GPU. These are also state less CODEC.
>
> On the other hand, the CODA IP is not coupled with the GPU and is state
> full. The VAAPI (and the deprecated VDPAU interfaces) are not designed
> to support state full decoder, and the extra indirections does not
> provide much benefit.

I understand, but there is still some gain, because there are several
applications that are only VDPAU/VAAPI aware. It would require a great
effort to modify all those applications to make use of the specific
CODA unit, since AFAIK there is no other standard API available for
Linux that is widely used by applications. It is true that Kodi has
support for the i.MX6 CODA unit, but only through the proprietary
drivers and APIs, it currently cannot work with the open-source
drivers. Looking into chromium, is just the same, it is possible to
patch and recompile chromium to support CODA, but again only for the
proprietary drivers.
If the same approach is followed for each specific VPU unit, we will
have proliferation of code and effort replication to support each
specific VPU unit, instead of using a standard API.
I am not very acquainted with gstreamer... but I do know gstreamer is
complex, is versatile and can perform complex manipulation on streams,
but AFAIK is not yet a "de-facto standard" and is not exactly a
replacement of VAAPI or VAAPI.
If you happen to have suggestions for alternatives, or if I'm wrong,
please provide your opinion.

>
> Nicolas

Regards,
Luis Mendes
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-3]

Le mardi 05 décembre 2017 à 19:39 +0000, Luís Mendes a écrit :

> I understand, but there is still some gain, because there are several
> applications that are only VDPAU/VAAPI aware. It would require a
> great
> effort to modify all those applications to make use of the specific
> CODA unit, since AFAIK there is no other standard API available for
> Linux that is widely used by applications. It is true that Kodi has
> support for the i.MX6 CODA unit, but only through the proprietary
> drivers and APIs, it currently cannot work with the open-source
> drivers. Looking into chromium, is just the same, it is possible to
> patch and recompile chromium to support CODA, but again only for the
> proprietary drivers.

This is incorrect. The state full CODEC interface for the kernel can be
used generically. It is supported by Chromium, in fact Chromium
supports both V4L2 state full and upcoming V4L2 state less kernel
interface. In recent Chromium you can build VAAPI, V4L2 and V4L2 Frame
decoder support all together and it will be detected properly. As a
matter of fact,the V4L2 state full kernel interface is widely
implemented by userspace these days, specially that FFMPEG gain support
for it (that enabled notably mplayer and VLC, but should enable Firefox
too).

> If the same approach is followed for each specific VPU unit, we will
> have proliferation of code and effort replication to support each
> specific VPU unit, instead of using a standard API.

This is also wrong, since V4L2 is a standard and generic kernel API,
much more generic then GPU kernel interfaces.

> I am not very acquainted with gstreamer... but I do know gstreamer is
> complex, is versatile and can perform complex manipulation on
> streams,
> but AFAIK is not yet a "de-facto standard" and is not exactly a
> replacement of VAAPI or VAAPI.
> If you happen to have suggestions for alternatives, or if I'm wrong,
> please provide your opinion.

VAAPI is not a multimedia framework, it's like comparing orange with
Apples. It simply provide an framework to enable state less hardware
codecs. VAAPI is, like mesa, an abstraction layer for CODEC that
requires a lot of HW specific code. Just try to write a simple media
player with it, and you'll quickly understand. On the other hand,
multimedia framework, like FFMPEG and GStreamer will help you out a
lot.

regards,
Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Fabio Estevam]

Hi Lukasz,

On Tue, Dec 5, 2017 at 2:34 PM, Lukasz Majewski <[hidden email]> wrote:

> Ok. I will try mesa-17.3.0

Make sure to try Weston 3.0 too.

Regards,

Fabio Estevam
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-3]

Le mardi 05 décembre 2017 à 19:39 +0000, Luís Mendes a écrit :

> If the same approach is followed for each specific VPU unit, we will
> have proliferation of code and effort replication to support each
> specific VPU unit, instead of using a standard API.
> I am not very acquainted with gstreamer... but I do know gstreamer is
> complex, is versatile and can perform complex manipulation on
> streams,
> but AFAIK is not yet a "de-facto standard" and is not exactly a
> replacement of VAAPI or VAAPI.
> If you happen to have suggestions for alternatives, or if I'm wrong,
> please provide your opinion.

Just in case, I want this conversation to be useful after all. The
driver I'm referring to is the mainline one, maintain by Philip Zabel
from Pengutronix:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/media/platform/coda

There exist staging drivers for the scaler and blitters, but I don't
know were they are maintained. There is also a staging driver for
the video capture driver and ISP. All this is implementing standard
V4L2 APIs which connect well together. V4L2 framework profile DMABuf
support, which should cover the bridging with the GPU side / Mesa.

regards,
Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel