GStreamer-devel

v4l2 allocated dma memory

classic Classic list List threaded Threaded
♦
♦
 6 messages Options
Bing Song-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

v4l2 allocated dma memory

Bing Song-2

Hi,

 

Is v4l2 video decoder output dma memory cacheable or non-cacheable? I found videoconvert performance is bad with v4l2 video dma buffer output buffer. Is it possible to use cacheable v4l2 dma memory?

 

Regards,

Bing


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Nicolas Dufresne-5
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: v4l2 allocated dma memory

Nicolas Dufresne-5
Le jeudi 01 avril 2021 à 08:17 +0000, Bing Song a écrit :
> Hi,
>  
> Is v4l2 video decoder output dma memory cacheable or non-cacheable? I found
> videoconvert performance is bad with v4l2 video dma buffer output buffer. Is
> it possible to use cacheable v4l2 dma memory?

It's driver decision and the kernel does not provide this information. In
general, drivers without an IOMMU (like most of NXP drivers) will use non-cache-
able memory allocation which is near unusable by videoconvert (often slower
software decode). Most SoC have some converter capability, which can be exposed
through an m2m driver and uses with v4l2convert element.

>  
> Regards,
> Bing
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Bing Song-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

RE: [EXT] Re: v4l2 allocated dma memory

Bing Song-2
But dmabuf map/unmap will handle cache flush/invalidate. So v4l2 without an IOMMU can use cacheable buffer in gstreamer. Right?

Regards,
Bing

-----Original Message-----
From: gstreamer-devel <[hidden email]> On Behalf Of Nicolas Dufresne
Sent: 2021年4月2日 2:16
To: Discussion of the development of and with GStreamer <[hidden email]>
Subject: [EXT] Re: v4l2 allocated dma memory

Caution: EXT Email

Le jeudi 01 avril 2021 à 08:17 +0000, Bing Song a écrit :
> Hi,
>
> Is v4l2 video decoder output dma memory cacheable or non-cacheable? I
> found videoconvert performance is bad with v4l2 video dma buffer
> output buffer. Is it possible to use cacheable v4l2 dma memory?

It's driver decision and the kernel does not provide this information. In general, drivers without an IOMMU (like most of NXP drivers) will use non-cache- able memory allocation which is near unusable by videoconvert (often slower software decode). Most SoC have some converter capability, which can be exposed through an m2m driver and uses with v4l2convert element.

>
> Regards,
> Bing
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist
> s.freedesktop.org%2Fmailman%2Flistinfo%2Fgstreamer-devel&amp;data=04%7
> C01%7Cbing.song%40nxp.com%7C5ff6c3d4fb7640cbb09a08d8f53a3b07%7C686ea1d
> 3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637528977733606983%7CUnknown%7CTWF
> pbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6M
> n0%3D%7C1000&amp;sdata=V6a2V6x1XfGr6wEosiXoBC8eA85YjpwvsE%2Fs04fD79k%3
> D&amp;reserved=0


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fgstreamer-devel&amp;data=04%7C01%7Cbing.song%40nxp.com%7C5ff6c3d4fb7640cbb09a08d8f53a3b07%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637528977733616934%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=3pt3X7CdOBKE8Q1UsJwaNrxlX0PK5UtbXyb9tRJmbb8%3D&amp;reserved=0
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Nicolas Dufresne-5
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [EXT] Re: v4l2 allocated dma memory

Nicolas Dufresne-5
Le vendredi 02 avril 2021 à 01:41 +0000, Bing Song a écrit :
> But dmabuf map/unmap will handle cache flush/invalidate. So v4l2 without an IOMMU can use cacheable buffer in gstreamer. Right?

Cache invalidation is much more complex then this, and this is very partially
implemented in V4L2. So without a system wide coherent memory (without all HW
component supporting CPU and device caches on the fly), using un-cacheable
memory is the only way to go in this short time. This indeed can be improved and
eventually fixed, but discussing this within GStreamer project won't help with
that.

regards,
Nicolas

p.s. Use linux-media mailing list or #v4l2 Freenode channel.


>
> Regards,
> Bing
>
> -----Original Message-----
> From: gstreamer-devel <[hidden email]> On Behalf Of Nicolas Dufresne
> Sent: 2021年4月2日 2:16
> To: Discussion of the development of and with GStreamer <[hidden email]>
> Subject: [EXT] Re: v4l2 allocated dma memory
>
> Caution: EXT Email
>
> Le jeudi 01 avril 2021 à 08:17 +0000, Bing Song a écrit :
> > Hi,
> >
> > Is v4l2 video decoder output dma memory cacheable or non-cacheable? I
> > found videoconvert performance is bad with v4l2 video dma buffer
> > output buffer. Is it possible to use cacheable v4l2 dma memory?
>
> It's driver decision and the kernel does not provide this information. In general, drivers without an IOMMU (like most of NXP drivers) will use non-cache- able memory allocation which is near unusable by videoconvert (often slower software decode). Most SoC have some converter capability, which can be exposed through an m2m driver and uses with v4l2convert element.
>
> >
> > Regards,
> > Bing
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist
> > s.freedesktop.org%2Fmailman%2Flistinfo%2Fgstreamer-devel&amp;data=04%7
> > C01%7Cbing.song%40nxp.com%7C5ff6c3d4fb7640cbb09a08d8f53a3b07%7C686ea1d
> > 3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637528977733606983%7CUnknown%7CTWF
> > pbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6M
> > n0%3D%7C1000&amp;sdata=V6a2V6x1XfGr6wEosiXoBC8eA85YjpwvsE%2Fs04fD79k%3
> > D&amp;reserved=0
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fgstreamer-devel&amp;data=04%7C01%7Cbing.song%40nxp.com%7C5ff6c3d4fb7640cbb09a08d8f53a3b07%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637528977733616934%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=3pt3X7CdOBKE8Q1UsJwaNrxlX0PK5UtbXyb9tRJmbb8%3D&amp;reserved=0
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Bing Song-2
Reply | Threaded
Open this post in threaded view
♦
♦
|

RE: [EXT] Re: v4l2 allocated dma memory

Bing Song-2
I got below back trace from v4l2 m2m v4l2convert. Seems v4l2 core can handle cache clean. Suppose should skip below cache clean. And v4l2 should allocate cacheable buffer and manage cache flush/invalidate when gstbuffer map function. Is it right?


          --0.52%--gst_v4l2_buffer_pool_qbuf
                                     __GI___ioctl
                                     el0_sync
                                     el0_sync_handler
                                     el0_svc
                                     do_el0_svc
                                     el0_svc_common.constprop.0
                                     __arm64_sys_ioctl
                                     v4l2_ioctl
                                     video_ioctl2
                                     video_usercopy
                                     __video_do_ioctl
                                     v4l_qbuf
                                     v4l2_m2m_ioctl_qbuf
                                     v4l2_m2m_qbuf
                                     vb2_qbuf
                                     vb2_core_qbuf
                                     __buf_prepare
                                     __prepare_dmabuf
                                     vb2_dc_map_dmabuf
                                     dma_buf_map_attachment
                                     dma_heap_map_dma_buf
                                     dma_map_sg_attrs
                                     dma_direct_map_sg
                                     __pi___clean_dcache_area_poc

Regards,
Bing

-----Original Message-----
From: gstreamer-devel <[hidden email]> On Behalf Of Nicolas Dufresne
Sent: 2021年4月2日 20:56
To: Discussion of the development of and with GStreamer <[hidden email]>
Subject: Re: [EXT] Re: v4l2 allocated dma memory

Caution: EXT Email

Le vendredi 02 avril 2021 à 01:41 +0000, Bing Song a écrit :
> But dmabuf map/unmap will handle cache flush/invalidate. So v4l2 without an IOMMU can use cacheable buffer in gstreamer. Right?

Cache invalidation is much more complex then this, and this is very partially implemented in V4L2. So without a system wide coherent memory (without all HW component supporting CPU and device caches on the fly), using un-cacheable memory is the only way to go in this short time. This indeed can be improved and eventually fixed, but discussing this within GStreamer project won't help with that.

regards,
Nicolas

p.s. Use linux-media mailing list or #v4l2 Freenode channel.


>
> Regards,
> Bing
>
> -----Original Message-----
> From: gstreamer-devel <[hidden email]>
> On Behalf Of Nicolas Dufresne
> Sent: 2021年4月2日 2:16
> To: Discussion of the development of and with GStreamer
> <[hidden email]>
> Subject: [EXT] Re: v4l2 allocated dma memory
>
> Caution: EXT Email
>
> Le jeudi 01 avril 2021 à 08:17 +0000, Bing Song a écrit :
> > Hi,
> >
> > Is v4l2 video decoder output dma memory cacheable or non-cacheable?
> > I found videoconvert performance is bad with v4l2 video dma buffer
> > output buffer. Is it possible to use cacheable v4l2 dma memory?
>
> It's driver decision and the kernel does not provide this information. In general, drivers without an IOMMU (like most of NXP drivers) will use non-cache- able memory allocation which is near unusable by videoconvert (often slower software decode). Most SoC have some converter capability, which can be exposed through an m2m driver and uses with v4l2convert element.
>
> >
> > Regards,
> > Bing
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fli
> > st
> > s.freedesktop.org%2Fmailman%2Flistinfo%2Fgstreamer-devel&amp;data=04
> > %7
> > C01%7Cbing.song%40nxp.com%7C5ff6c3d4fb7640cbb09a08d8f53a3b07%7C686ea
> > 1d
> > 3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637528977733606983%7CUnknown%7CT
> > WF
> > pbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI
> > 6M
> > n0%3D%7C1000&amp;sdata=V6a2V6x1XfGr6wEosiXoBC8eA85YjpwvsE%2Fs04fD79k
> > %3
> > D&amp;reserved=0
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist
> s.freedesktop.org%2Fmailman%2Flistinfo%2Fgstreamer-devel&amp;data=04%7
> C01%7Cbing.song%40nxp.com%7C3fbc38a316e04f6c5e0308d8f5d6b2ba%7C686ea1d
> 3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637529649756904640%7CUnknown%7CTWF
> pbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6M
> n0%3D%7C1000&amp;sdata=tedEcL98YrPp6kwmPRqTZF6CrMAHdlGRS4fEluPqjH0%3D&
> amp;reserved=0 _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist
> s.freedesktop.org%2Fmailman%2Flistinfo%2Fgstreamer-devel&amp;data=04%7
> C01%7Cbing.song%40nxp.com%7C3fbc38a316e04f6c5e0308d8f5d6b2ba%7C686ea1d
> 3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637529649756904640%7CUnknown%7CTWF
> pbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6M
> n0%3D%7C1000&amp;sdata=tedEcL98YrPp6kwmPRqTZF6CrMAHdlGRS4fEluPqjH0%3D&
> amp;reserved=0


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fgstreamer-devel&amp;data=04%7C01%7Cbing.song%40nxp.com%7C3fbc38a316e04f6c5e0308d8f5d6b2ba%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637529649756914597%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=%2FVaFGk8d0%2BC0SIWNHqmE%2B2M1jyDkRgigLAr01XhkJeM%3D&amp;reserved=0
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Nicolas Dufresne-5
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: [EXT] Re: v4l2 allocated dma memory

Nicolas Dufresne-5
Le lundi 12 avril 2021 à 07:32 +0000, Bing Song a écrit :

> I got below back trace from v4l2 m2m v4l2convert. Seems v4l2 core can handle
> cache clean. Suppose should skip below cache clean. And v4l2 should allocate
> cacheable buffer and manage cache flush/invalidate when gstbuffer map
> function. Is it right?
>
>           --0.52%--gst_v4l2_buffer_pool_qbuf
>                                      __GI___ioctl
>                                      el0_sync
>                                      el0_sync_handler
>                                      el0_svc
>                                      do_el0_svc
>                                      el0_svc_common.constprop.0
>                                      __arm64_sys_ioctl
>                                      v4l2_ioctl
>                                      video_ioctl2
>                                      video_usercopy
>                                      __video_do_ioctl
>                                      v4l_qbuf
>                                      v4l2_m2m_ioctl_qbuf
>                                      v4l2_m2m_qbuf
>                                      vb2_qbuf
>                                      vb2_core_qbuf
>                                      __buf_prepare
>                                      __prepare_dmabuf
>                                      vb2_dc_map_dmabuf
>                                      dma_buf_map_attachment
>                                      dma_heap_map_dma_buf
>                                      dma_map_sg_attrs
>                                      dma_direct_map_sg
>                                      __pi___clean_dcache_area_poc

I thought you were using the contiguous allocator in VB2, that looks like the
SQG allocator, which is meant to be used with HW that has limited scatter list
capability (no mmu, but can deal with scattered pages). Now, if you see cache
operation happening, you likely have cache-able memory.

I'm not an expert, but I believe that VB2 have always been doing cache
invalidation on QBUF(OUTPUT). With proper information, it could indeed skip this
invalidation when the cache-able buffer was passed from another HW without being
written by the CPU.

Note you didn't share your pipeline, as zero-copy is not automatically enabled,
you may be in the copy path, in which case gst will memcopy the frame, and the
cache invalidation becomes strictly needed.

>
> Regards,
> Bing
>
> -----Original Message-----
> From: gstreamer-devel <[hidden email]> On Behalf Of Nicolas Dufresne
> Sent: 2021年4月2日 20:56
> To: Discussion of the development of and with GStreamer <[hidden email]>
> Subject: Re: [EXT] Re: v4l2 allocated dma memory
>
> Caution: EXT Email
>
> Le vendredi 02 avril 2021 à 01:41 +0000, Bing Song a écrit :
> > But dmabuf map/unmap will handle cache flush/invalidate. So v4l2 without an IOMMU can use cacheable buffer in gstreamer. Right?
>
> Cache invalidation is much more complex then this, and this is very partially implemented in V4L2. So without a system wide coherent memory (without all HW component supporting CPU and device caches on the fly), using un-cacheable memory is the only way to go in this short time. This indeed can be improved and eventually fixed, but discussing this within GStreamer project won't help with that.
>
> regards,
> Nicolas
>
> p.s. Use linux-media mailing list or #v4l2 Freenode channel.
>
>
> >
> > Regards,
> > Bing
> >
> > -----Original Message-----
> > From: gstreamer-devel <[hidden email]>
> > On Behalf Of Nicolas Dufresne
> > Sent: 2021年4月2日 2:16
> > To: Discussion of the development of and with GStreamer
> > <[hidden email]>
> > Subject: [EXT] Re: v4l2 allocated dma memory
> >
> > Caution: EXT Email
> >
> > Le jeudi 01 avril 2021 à 08:17 +0000, Bing Song a écrit :
> > > Hi,
> > >
> > > Is v4l2 video decoder output dma memory cacheable or non-cacheable?
> > > I found videoconvert performance is bad with v4l2 video dma buffer
> > > output buffer. Is it possible to use cacheable v4l2 dma memory?
> >
> > It's driver decision and the kernel does not provide this information. In general, drivers without an IOMMU (like most of NXP drivers) will use non-cache- able memory allocation which is near unusable by videoconvert (often slower software decode). Most SoC have some converter capability, which can be exposed through an m2m driver and uses with v4l2convert element.
> >
> > >
> > > Regards,
> > > Bing
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > [hidden email]
> > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fli
> > > st
> > > s.freedesktop.org%2Fmailman%2Flistinfo%2Fgstreamer-devel&amp;data=04
> > > %7
> > > C01%7Cbing.song%40nxp.com%7C5ff6c3d4fb7640cbb09a08d8f53a3b07%7C686ea
> > > 1d
> > > 3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637528977733606983%7CUnknown%7CT
> > > WF
> > > pbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI
> > > 6M
> > > n0%3D%7C1000&amp;sdata=V6a2V6x1XfGr6wEosiXoBC8eA85YjpwvsE%2Fs04fD79k
> > > %3
> > > D&amp;reserved=0
> >
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist
> > s.freedesktop.org%2Fmailman%2Flistinfo%2Fgstreamer-devel&amp;data=04%7
> > C01%7Cbing.song%40nxp.com%7C3fbc38a316e04f6c5e0308d8f5d6b2ba%7C686ea1d
> > 3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637529649756904640%7CUnknown%7CTWF
> > pbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6M
> > n0%3D%7C1000&amp;sdata=tedEcL98YrPp6kwmPRqTZF6CrMAHdlGRS4fEluPqjH0%3D&
> > amp;reserved=0 _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist
> > s.freedesktop.org%2Fmailman%2Flistinfo%2Fgstreamer-devel&amp;data=04%7
> > C01%7Cbing.song%40nxp.com%7C3fbc38a316e04f6c5e0308d8f5d6b2ba%7C686ea1d
> > 3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637529649756904640%7CUnknown%7CTWF
> > pbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6M
> > n0%3D%7C1000&amp;sdata=tedEcL98YrPp6kwmPRqTZF6CrMAHdlGRS4fEluPqjH0%3D&
> > amp;reserved=0
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fgstreamer-devel&amp;data=04%7C01%7Cbing.song%40nxp.com%7C3fbc38a316e04f6c5e0308d8f5d6b2ba%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637529649756914597%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=%2FVaFGk8d0%2BC0SIWNHqmE%2B2M1jyDkRgigLAr01XhkJeM%3D&amp;reserved=0
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Free forum by Nabble Edit this page