Is there a way to crop in VAAPI?
Locked
 
|
I need to decode H.264 video via VAAPI and then extract the decoded frames to
main memory. Since I intend to crop the frames anyway, I'd like to crop them while they're still in graphics memory, to reduce memory bandwidth consumption. So conceptually, I want to do this: gst-launch-1.0 rtspsrc location=rtsp_uri ! rtph264depay ! vaapih264dec ! videocrop left=x right=x top=x bottom=x ! filesink location=video.yuv But I think this will end up copying the whole frames to main memory before cropping it there. It would be nice if vaapipostproc supports cropping? -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Is there a way to crop in VAAPI?
|
|
Le vendredi 02 août 2019 à 20:45 -0500, kmliu a écrit :
> I need to decode H.264 video via VAAPI and then extract the decoded frames to > main memory. Since I intend to crop the frames anyway, I'd like to crop them > while they're still in graphics memory, to reduce memory bandwidth > consumption. So conceptually, I want to do this: > > gst-launch-1.0 rtspsrc location=rtsp_uri ! rtph264depay ! vaapih264dec ! > videocrop left=x right=x top=x bottom=x ! filesink location=video.yuv > > But I think this will end up copying the whole frames to main memory before > cropping it there. It would be nice if vaapipostproc supports cropping? You can use the glvideomixelement for that. Only limitationis that it won't let you break aspect ratio (from my testing). ... ! vaapih264dec ! vaapipostproc ! glupload ! glcolorconvert ! glvideomixerelement sink_0::xpos=-160 sink_0::width=640 sink_0::ypos=-120 sink_0::height=480 ! video/x-raw\(memory:GLMemory\),width=320,height=240 ! glcolorconvert ! gldownload ! gtksink > > > > -- > Sent from: http://gstreamer-devel.966125.n4.nabble.com/ > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Is there a way to crop in VAAPI?
|
|
In reply to this post by kmliu
Le vendredi 02 août 2019 à 20:45 -0500, kmliu a écrit :
> I need to decode H.264 video via VAAPI and then extract the decoded frames to > main memory. Since I intend to crop the frames anyway, I'd like to crop them > while they're still in graphics memory, to reduce memory bandwidth > consumption. So conceptually, I want to do this: > > gst-launch-1.0 rtspsrc location=rtsp_uri ! rtph264depay ! vaapih264dec ! > videocrop left=x right=x top=x bottom=x ! filesink location=video.yuv > > But I think this will end up copying the whole frames to main memory before > cropping it there. It would be nice if vaapipostproc supports cropping? Another (simplier) option is to use gltransformation, but apparently not all distro ship this plugin. > > > > -- > Sent from: http://gstreamer-devel.966125.n4.nabble.com/ > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
On 8/3/2019 9:25 PM, Nicolas Dufresne-5
[via GStreamer-devel] wrote:
Le vendredi 02 août 2019 à 20:45 -0500, kmliu a écrit : May I add the crop option for vaapih264dec? It may be more direct. Thanks. Yan Wang
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Is there a way to crop in VAAPI?
|
|
On Mon, Aug 05, 2019 at 02:54:16PM +0800, Yan Wang wrote:
> > On 8/3/2019 9:25 PM, Nicolas Dufresne-5 [via GStreamer-devel] wrote: > > Le vendredi 02 août 2019 à 20:45 -0500, kmliu a écrit : > > > > > I need to decode H.264 video via VAAPI and then extract the decoded > > frames to > > > main memory. Since I intend to crop the frames anyway, I'd like to > > crop them > > > while they're still in graphics memory, to reduce memory bandwidth > > > consumption. So conceptually, I want to do this: > > > > > > gst-launch-1.0 rtspsrc location=rtsp_uri ! rtph264depay ! vaapih264dec > > ! > > > videocrop left=x right=x top=x bottom=x ! filesink location=video.yuv > > > > > > But I think this will end up copying the whole frames to main memory > > before > > > cropping it there. It would be nice if vaapipostproc supports cropping? > > > > Another (simplier) option is to use gltransformation, but apparently > > not all distro ship this plugin. > > May I add the crop option for vaapih264dec? > > It may be more direct. If I read the code correctly, then vaapipostproc already suppports GstVideoCropMeta. So I think you could do ... ! vaapih264dec ! vaapipostproc ! ... and add a pad probe that adds the crop meta to each buffer. Michael -- Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 | _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Is there a way to crop in VAAPI?
|
|
In reply to this post by Yan Wang
Le lun. 5 août 2019 02 h 55, Yan Wang <[hidden email]> a écrit :
Sounds wrong to me, and I doubt you'll find an API to support that. If there is one element in VAAPI that may have an API for this it would be vaapipostproc.
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
In reply to this post by Michael Olbrich
> If I read the code correctly, then vaapipostproc already suppports
> GstVideoCropMeta. So I think you could do ... ! vaapih264dec !
> vaapipostproc ! ... and add a pad probe that adds the crop meta to each
> buffer.
> Michael
Isn't GstVideoCropMeta just a tag that "tells" the downstream elements that the frame should be cropped? I'm not sure if it would reduce the amount of data copied from graphics memory to main memory.
Sent from the GStreamer-devel mailing list archive at Nabble.com. _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
In reply to this post by Nicolas Dufresne-5
On 8/5/2019 9:29 PM, Nicolas Dufresne-5
[via GStreamer-devel] wrote:
Yes. I know vaapipostproc can do this. IMHO, this could be done in decode context too if VAConfigAttribDecProcessing can support it. https://github.com/intel/libva/blob/master/va/va.h#L520 Currently It can do scaling (1/8 ~ 8x at most, CSC). Cropping can
be the special case of scaling. If vaapih264dec provides the interface, it can reduce the overhead and power because it is fixed function. And no need vaapipostproc context. But it may make vaapih264dec more complex and cannot support
multiply pipelines of decoding. So it is just one possible option. Thanks. Yan Wang
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Is there a way to crop in VAAPI?
|
|
In reply to this post by kmliu
On Mon, Aug 05, 2019 at 12:29:33PM -0500, kmliu wrote:
> > If I read the code correctly, then vaapipostproc already suppports> > GstVideoCropMeta. So I think you could do ... ! vaapih264dec !> > vaapipostproc ! ... and add a pad probe that adds the crop meta to each> > buffer.> MichaelIsn't GstVideoCropMeta just a tag that "tells" the > downstream elements that the frame should be cropped? I'm not sure if it > would reduce the amount of data copied from graphics memory to main memory. No, it contains a rectangle that describes the core area[1]. From what I can see in the vaapipostproc code, this cropping is applied during the transformation. Michael [1] https://gstreamer.freedesktop.org/documentation/video/gstvideometa.html?gi-language=c#GstVideoCropMeta -- Pengutronix e.K. | | Industrial Linux Solutions | http://www.pengutronix.de/ | Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 | _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Is there a way to crop in VAAPI?
|
|
In reply to this post by Yan Wang
Le mardi 06 août 2019 à 14:31 +0800, Yan Wang a écrit :
> > On 8/5/2019 9:29 PM, Nicolas Dufresne-5 [via GStreamer-devel] wrote: > > > > Le lun. 5 août 2019 02 h 55, Yan Wang <[hidden email]> a écrit : > > > > > > On 8/3/2019 9:25 PM, Nicolas Dufresne-5 [via GStreamer-devel] wrote: > > > > Le vendredi 02 août 2019 à 20:45 -0500, kmliu a écrit : > > > > > > > > > I need to decode H.264 video via VAAPI and then extract the decoded frames to > > > > > main memory. Since I intend to crop the frames anyway, I'd like to crop them > > > > > while they're still in graphics memory, to reduce memory bandwidth > > > > > consumption. So conceptually, I want to do this: > > > > > > > > > > gst-launch-1.0 rtspsrc location=rtsp_uri ! rtph264depay ! vaapih264dec ! > > > > > videocrop left=x right=x top=x bottom=x ! filesink location=video.yuv > > > > > > > > > > But I think this will end up copying the whole frames to main memory before > > > > > cropping it there. It would be nice if vaapipostproc supports cropping? > > > > > > > > Another (simplier) option is to use gltransformation, but apparently > > > > not all distro ship this plugin. > > > > > > May I add the crop option for vaapih264dec? > > > > > > It may be more direct. > > > > > > > > > > Sounds wrong to me, and I doubt you'll find an API to support that. If there is one element in VAAPI that may have an API for this it would be vaapipostproc. > > Yes. I know vaapipostproc can do this. > > IMHO, this could be done in decode context too if VAConfigAttribDecProcessing can support it. > > https://github.com/intel/libva/blob/master/va/va.h#L520 > > Currently It can do scaling (1/8 ~ 8x at most, CSC). Cropping can be the special case of scaling. > > If vaapih264dec provides the interface, it can reduce the overhead and power because it is fixed function. And no need vaapipostproc context. > > But it may make vaapih264dec more complex and cannot support multiply pipelines of decoding. So it is just one possible option. If you know how it works, just give it a try. You can find similar code in -good/sys/gstv4l2videodec.c, but only for the CSC bit. Scaling is more complex though, because of the pixel aspect ratio handling. It's this a proper scaler or just multiple of 2 ? > > Thanks. > > Yan Wang > > > > > > Thanks. > > > > > > Yan Wang > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Sent from: http://gstreamer-devel.966125.n4.nabble.com/ > > > > > _______________________________________________ > > > > > gstreamer-devel mailing list > > > > > [hidden email] > > > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > > > > > > > _______________________________________________ > > > > gstreamer-devel mailing list > > > > [hidden email] > > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > > > > > > > If you reply to this email, your message will be added to the discussion below: > > > > http://gstreamer-devel.966125.n4.nabble.com/Is-there-a-way-to-crop-in-VAAPI-tp4691566p4691569.html > > > > To start a new topic under GStreamer-devel, email [hidden email] > > > > To unsubscribe from GStreamer-devel, click here. > > > > NAML > > > > > > _______________________________________________ > > > gstreamer-devel mailing list > > > [hidden email] > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > > > _______________________________________________ > > gstreamer-devel mailing list > > [hidden email] > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > > > If you reply to this email, your message will be added to the discussion below: > > http://gstreamer-devel.966125.n4.nabble.com/Is-there-a-way-to-crop-in-VAAPI-tp4691566p4691583.html > > To start a new topic under GStreamer-devel, email [hidden email] > > To unsubscribe from GStreamer-devel, click here. > > NAML > > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
On 8/7/2019 9:40 AM, Nicolas Dufresne-5
[via GStreamer-devel] wrote:
Le mardi 06 août 2019 à 14:31 +0800, Yan Wang a écrit : I will check -good/sys/gstv4l2videodec.c. This scaling/CSC has some limitation. It could be attached only one decoding pipeline. So it is different with normal videopostproc pipeline but videopostproc pipeline could select it too. The logic of the selection should be in media driver currently. Of course, it can also be selected in gstreamer-vaapi directly. Thanks for your comments. Yan Wang
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Is there a way to crop in VAAPI?
|
|
Le mer. 7 août 2019 02 h 55, Yan Wang <[hidden email]> a écrit :
Yes, I'm familiar with those CSC, Coda IP from chip&media and Hantro IP (used in rockchip and imx8) have similar features. I guess when enabled, there will be two set of output surfaces ?
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
On 8/7/2019 8:26 PM, Nicolas Dufresne-5
[via GStreamer-devel] wrote:
Yes. If enabled, there could be 2 output surfaces at the same time which could have the different format and size. I will try it. Thanks. Yan Wang
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Is there a way to crop in VAAPI?
|
|
In reply to this post by Yan Wang
On Tue, 06 Aug 2019 at 14:31, Yan Wang wrote:
> > On 8/5/2019 9:29 PM, Nicolas Dufresne-5 [via GStreamer-devel] wrote: > > > > > > Le lun. 5 août 2019 02 h 55, Yan Wang <[hidden email] > > </user/SendEmail.jtp?type=node&node=4691583&i=0>> a écrit : > > > > > > On 8/3/2019 9:25 PM, Nicolas Dufresne-5 [via GStreamer-devel] wrote: > > > Le vendredi 02 août 2019 à 20:45 -0500, kmliu a écrit : > > > > > > > I need to decode H.264 video via VAAPI and then extract the > > > decoded frames to > > > > main memory. Since I intend to crop the frames anyway, I'd like > > > to crop them > > > > while they're still in graphics memory, to reduce memory bandwidth > > > > consumption. So conceptually, I want to do this: > > > > > > > > gst-launch-1.0 rtspsrc location=rtsp_uri ! rtph264depay ! > > > vaapih264dec ! > > > > videocrop left=x right=x top=x bottom=x ! filesink > > > location=video.yuv There's a work-in-progress for this: https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/merge_requests/121 > > > > > > > > But I think this will end up copying the whole frames to main > > > memory before > > > > cropping it there. It would be nice if vaapipostproc supports > > > cropping? > > > > > > Another (simplier) option is to use gltransformation, but apparently > > > not all distro ship this plugin. > > > > May I add the crop option for vaapih264dec? > > > > It may be more direct. > > > > Sounds wrong to me, and I doubt you'll find an API to support that. If > > there is one element in VAAPI that may have an API for this it would be > > vaapipostproc. > > Yes. I know vaapipostproc can do this. > > IMHO, this could be done in decode context too if > VAConfigAttribDecProcessing can support it. > > https://github.com/intel/libva/blob/master/va/va.h#L520 > > Currently It can do scaling (1/8 ~ 8x at most, CSC). Cropping can be the > special case of scaling. > > If vaapih264dec provides the interface, it can reduce the overhead and power > because it is fixed function. And no need vaapipostproc context. > > But it may make vaapih264dec more complex and cannot support multiply > pipelines of decoding. So it is just one possible option. > > Thanks. > > Yan Wang > > > > > > > Thanks. > > > > Yan Wang > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Sent from: http://gstreamer-devel.966125.n4.nabble.com/ > > > > _______________________________________________ > > > > gstreamer-devel mailing list > > > > [hidden email] > > > <http:///user/SendEmail.jtp?type=node&node=4691569&i=0> > > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > > > > > _______________________________________________ > > > gstreamer-devel mailing list > > > [hidden email] > > > <http:///user/SendEmail.jtp?type=node&node=4691569&i=1> > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > > > > > ------------------------------------------------------------------------ > > > If you reply to this email, your message will be added to the > > > discussion below: > > > http://gstreamer-devel.966125.n4.nabble.com/Is-there-a-way-to-crop-in-VAAPI-tp4691566p4691569.html > > > > > > To start a new topic under GStreamer-devel, email [hidden email] > > > </user/SendEmail.jtp?type=node&node=4691583&i=1> > > > To unsubscribe from GStreamer-devel, click here. > > > NAML > > > <http://gstreamer-devel.966125.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > > > _______________________________________________ > > gstreamer-devel mailing list > > [hidden email] </user/SendEmail.jtp?type=node&node=4691583&i=2> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > > > > > _______________________________________________ > > gstreamer-devel mailing list > > [hidden email] </user/SendEmail.jtp?type=node&node=4691583&i=3> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > > > ------------------------------------------------------------------------ > > If you reply to this email, your message will be added to the discussion > > below: > > http://gstreamer-devel.966125.n4.nabble.com/Is-there-a-way-to-crop-in-VAAPI-tp4691566p4691583.html > > > > To start a new topic under GStreamer-devel, email > > [hidden email] > > To unsubscribe from GStreamer-devel, click here < > > NAML <http://gstreamer-devel.966125.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
«
Return to GStreamer-devel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |