Coda: imx53 plays video with incorrect colors
Locked
Coda: imx53 plays video with incorrect colors
 
|
Hi,
I am testing video playback on an imx53-qsb running 5.10.6, but I am not getting the correct colors in the TVE output. This is the result: https://www.dropbox.com/s/a4ifivpoi663dkd/mx53.mp4?dl=0 The original video is this one: http://cdn.clipcanvas.com/sample/clipcanvas_14348_offline.mp4 I resized the TVE output to 1024x576 so that scaling is not needed for now. The Gstreamer pipeline I am using is: # gst-launch-1.0 filesrc location=/media/clip.mp4 ! qtdemux ! h264parse ! v4l2h264dec ! kmssink Gstreamer version is 1.18.2. Any ideas on how to get the video playback with the correct colors? Thanks, Fabio Estevam _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Coda: imx53 plays video with incorrect colors
|
|
On Thu, Jan 14, 2021 at 9:20 AM Fabio Estevam <[hidden email]> wrote:
> > Hi, > > I am testing video playback on an imx53-qsb running 5.10.6, but I am > not getting the correct colors in the TVE output. This is the result: > > https://www.dropbox.com/s/a4ifivpoi663dkd/mx53.mp4?dl=0 > > The original video is this one: > http://cdn.clipcanvas.com/sample/clipcanvas_14348_offline.mp4 > > I resized the TVE output to 1024x576 so that scaling is not needed for now. > > The Gstreamer pipeline I am using is: > > # gst-launch-1.0 filesrc location=/media/clip.mp4 ! qtdemux ! > h264parse ! v4l2h264dec ! kmssink > > Gstreamer version is 1.18.2. > > Any ideas on how to get the video playback with the correct colors? I forgot to mention that this incorrect color behavior is seen with other video files too. Actually, I was not able to see video playback with correct colors using mainline kernel on i.MX53 yet. Thanks _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Coda: imx53 plays video with incorrect colors
|
|
Le jeudi 14 janvier 2021 à 12:36 -0300, Fabio Estevam a écrit :
> On Thu, Jan 14, 2021 at 9:20 AM Fabio Estevam <[hidden email]> wrote: > > > > Hi, > > > > I am testing video playback on an imx53-qsb running 5.10.6, but I am > > not getting the correct colors in the TVE output. This is the result: > > > > https://www.dropbox.com/s/a4ifivpoi663dkd/mx53.mp4?dl=0 > > > > The original video is this one: > > http://cdn.clipcanvas.com/sample/clipcanvas_14348_offline.mp4 > > > > I resized the TVE output to 1024x576 so that scaling is not needed for now. > > > > The Gstreamer pipeline I am using is: > > > > # gst-launch-1.0 filesrc location=/media/clip.mp4 ! qtdemux ! > > h264parse ! v4l2h264dec ! kmssink > > > > Gstreamer version is 1.18.2. > > > > Any ideas on how to get the video playback with the correct colors? > > I forgot to mention that this incorrect color behavior is seen with > other video files too. > > Actually, I was not able to see video playback with correct colors > using mainline kernel on i.MX53 yet. Perhaps you would explain in detail what isn't color correct ? To debug this, you probably want to inspect the caps and the colorimetry negotiated between each element (use -v in gst-launch-1.0). It's quite possible that the decoder is ignoring upstream colors and get badly hinted by the driver, or that kmssink is pnot passing colorimetry to the driver. > > Thanks _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Coda: imx53 plays video with incorrect colors
|
|
Hi Nicolas,
On Sat, Jan 16, 2021 at 11:46 PM Nicolas Dufresne <[hidden email]> wrote: > Perhaps you would explain in detail what isn't color correct ? To debug this, Here is the movie showing the incorrect colors: https://www.dropbox.com/s/a4ifivpoi663dkd/mx53.mp4?dl=0 The original video is this one: http://cdn.clipcanvas.com/sample/clipcanvas_14348_offline.mp4 Please note that the color mismatch is not related to this specific video sample. It happens with all videos I have tried. > you probably want to inspect the caps and the colorimetry negotiated between > each element (use -v in gst-launch-1.0). It's quite possible that the decoder is > ignoring upstream colors and get badly hinted by the driver, or that kmssink is > pnot passing colorimetry to the driver. Here is the -v output: # gst-launch-1.0 -v filesrc location=/media/clip.mp4 ! qtdemux ! h264parse ! v4l2h264dec ! kmssink Setting pipeline to PAUSED ... [ 15.113196] msm msm: [drm:adreno_request_fw] loaded qcom/yamato_pm4.fw from new location [ 15.124377] msm msm: [drm:adreno_request_fw] loaded qcom/yamato_pfp.fw from new location Pipeline is PREROLLING ... /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 1024 /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 576 /GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3.1, profile=(string)main, codec_data=(buffer)014d401fffe10 023674d401f967200800936028100000e100002bf203460016e40016e45ef7c1e1108a24001000468de3c80, width=(int)1024, height=(int)576, pixel-aspect-ratio=(fraction)1/1 /GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)3.1, profile=(string)main, width=(int)1024, height=( int)576, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)25/1, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true /GstPipeline:pipeline0/v4l2h264dec:v4l2h264dec0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)3.1, profile=(string)main, width=(int)1024, height=(int)576, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)25/1, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true /GstPipeline:pipeline0/v4l2h264dec:v4l2h264dec0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)1024, height=(int)576, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jp eg, colorimetry=(string)bt601, framerate=(fraction)25/1 /GstPipeline:pipeline0/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)1024, height=(int)576, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg,colorimetry=(string)bt601, framerate=(fraction)25/1 Pipeline is PREROLLED ...0 %) Setting pipeline to PLAYING ... New clock: GstSystemClock Any adjustment I need to do in the Gstreamer pipeline to negotiate the correct colors to the display? Thanks _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Coda: imx53 plays video with incorrect colors
|
|
Le dim. 17 janv. 2021 08 h 47, Fabio Estevam <[hidden email]> a écrit : Hi Nicolas, Ah, didn't realise this was a link to you filming a screen. This looks like a UV inversion.
At first sight there is nothing wrong from GStreamer happening. It picks i420, pass it to KMS and it comes out wrong, first suspect would be the display driver. Understand that yuv formats are often unused and untested as most display server / compositer uses rgb type of formats converting with GPU. New clock: GstSystemClock
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Coda: imx53 plays video with incorrect colors
|
|
Hi Philipp,
On Sun, Jan 17, 2021 at 10:18 PM Nicolas Dufresne <[hidden email]> wrote: > At first sight there is nothing wrong from GStreamer happening. It picks i420, pass it to KMS and it comes out wrong, first suspect would be the display driver. Understand that yuv formats are often unused and untested as most display server / compositer uses rgb type of formats converting with GPU. > On i.MX53 I see this 'wrong color' behavior when playing videos into TVE as well as parallel display. Are you able to playback video successfully on i.MX53? If so, could you please share your Gstreamer pipeline? Thanks _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Coda: imx53 plays video with incorrect colors
|
|
Hi Fabio,
On Mon, 2021-01-18 at 07:57 -0300, Fabio Estevam wrote: > Hi Philipp, > > On Sun, Jan 17, 2021 at 10:18 PM Nicolas Dufresne <[hidden email]> wrote: > > > At first sight there is nothing wrong from GStreamer happening. It > > picks i420, pass it to KMS and it comes out wrong, first suspect > > would be the display driver. Understand that yuv formats are often > > unused and untested as most display server / compositer uses rgb > > type of formats converting with GPU. This is both a bug and a missing feature in the imx-drm driver. The driver doesn't support color space conversion on the VGA output and doesn't report that correctly. > On i.MX53 I see this 'wrong color' behavior when playing videos into > TVE as well as parallel display. > > Are you able to playback video successfully on i.MX53? If so, could > you please share your Gstreamer pipeline? I can reproduce this with gst-launch-1.0 videotestsrc ! video/x-raw,format=I420 ! kmssink and the same for NV12 or YUY2. We are missing color space conversion on the VGA output. The IPU only supports color space conversion on one output via the DP. This path is hard-coded to DI0 in the driver. The direct DC path to DI1, which TVE/VGA is connected to, does not support CSC. The driver could be modified to switch the DP->DI0/DC->DI1 mapping around to DP->DI1/DC->DI0 when required. As a simple test, you can switch statically with: ----------8<---------- diff --git a/drivers/gpu/ipu-v3/ipu-common.c b/drivers/gpu/ipu-v3/ipu-common.c index d166ee262ce4..f7c86ef0bed7 100644 --- a/drivers/gpu/ipu-v3/ipu-common.c +++ b/drivers/gpu/ipu-v3/ipu-common.c @@ -1118,18 +1118,18 @@ static struct ipu_platform_reg client_reg[] = { }, { .pdata = { .di = 0, - .dc = 5, - .dp = IPU_DP_FLOW_SYNC_BG, - .dma[0] = IPUV3_CHANNEL_MEM_BG_SYNC, + .dc = 1, + .dp = -EINVAL, + .dma[0] = IPUV3_CHANNEL_MEM_DC_SYNC, .dma[1] = IPUV3_CHANNEL_MEM_FG_SYNC, }, .name = "imx-ipuv3-crtc", }, { .pdata = { .di = 1, - .dc = 1, - .dp = -EINVAL, - .dma[0] = IPUV3_CHANNEL_MEM_DC_SYNC, + .dc = 5, + .dp = IPU_DP_FLOW_SYNC_BG, + .dma[0] = IPUV3_CHANNEL_MEM_BG_SYNC, .dma[1] = -EINVAL, }, .name = "imx-ipuv3-crtc", diff --git a/drivers/gpu/ipu-v3/ipu-dc.c b/drivers/gpu/ipu-v3/ipu-dc.c index 34b4075a6a8e..0a7f48e4aa37 100644 --- a/drivers/gpu/ipu-v3/ipu-dc.c +++ b/drivers/gpu/ipu-v3/ipu-dc.c @@ -364,9 +364,9 @@ int ipu_dc_init(struct ipu_soc *ipu, struct device *dev, writel(DC_WR_CH_CONF_WORD_SIZE_24 | DC_WR_CH_CONF_DISP_ID_PARALLEL(1) | DC_WR_CH_CONF_PROG_DI_ID, - priv->channels[1].base + DC_WR_CH_CONF); - writel(DC_WR_CH_CONF_WORD_SIZE_24 | DC_WR_CH_CONF_DISP_ID_PARALLEL(0), priv->channels[5].base + DC_WR_CH_CONF); + writel(DC_WR_CH_CONF_WORD_SIZE_24 | DC_WR_CH_CONF_DISP_ID_PARALLEL(0), + priv->channels[1].base + DC_WR_CH_CONF); writel(DC_GEN_SYNC_1_6_SYNC | DC_GEN_SYNC_PRIORITY_1, priv->dc_reg + DC_GEN); ---------->8---------- That should enable CSC (and overlay plane) on DI1, and lose it on DI0. Or, as a workaround, add a v4l2convert element and use the IC to convert to BGRx between decoder and kmssink. regards Philipp _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Coda: imx53 plays video with incorrect colors
|
|
Hi Philipp,
Thanks for your reply. On Mon, Jan 18, 2021 at 9:40 AM Philipp Zabel <[hidden email]> wrote: > The driver could be modified to switch the DP->DI0/DC->DI1 mapping > around to DP->DI1/DC->DI0 when required. As a simple test, you can > switch statically with: It does change the colors but still does not play the video with the correct colors. Looks like it plays in black-and-white now. > Or, as a workaround, add a v4l2convert element and use the IC to convert > to BGRx between decoder and kmssink. Yes, I have tried to do this, but it says that v4l2convert does not support bt601 colorimetry, and then a segfault occurs: # gst-launch-1.0 filesrc location=/media/clip.mp4 ! qtdemux ! h264parse ! v4l2h264dec ! v4l2convert ! video/x-raw,format=BGRx ! kmssink Setting pipeline to PAUSED ... Pipeline is PREROLLING ... ERROR: from element /GstPipeline:pipeline0/v4l2convert:v4l2convert0: Device '/dev/video4' does not support bt601 colorimetry Additional debug info: ../sys/v4l2/gstv4l2object.c(4032): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline0/v4l2convert:v4l2convert0: Device wants 2:4:5:4 colorimetry ERROR: pipeline doesn't want to preroll. Setting pipeline to NULL ... Caught SIGSEGV exec gdb failed: No such file or directory Spinning. Please run 'gdb gst-launch-1.0 217' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core. Is the Gstreamer pipeline above correct? Thanks _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Coda: imx53 plays video with incorrect colors
|
|
Hi Fabio,
On Mon, 2021-01-18 at 10:28 -0300, Fabio Estevam wrote: > Hi Philipp, > > Thanks for your reply. > > On Mon, Jan 18, 2021 at 9:40 AM Philipp Zabel <[hidden email]> wrote: > > > The driver could be modified to switch the DP->DI0/DC->DI1 mapping > > around to DP->DI1/DC->DI0 when required. As a simple test, you can > > switch statically with: > > It does change the colors but still does not play the video with the > correct colors. Looks like it plays in black-and-white now. Please try forcing decoder output to NV12 instead of I420. > > Or, as a workaround, add a v4l2convert element and use the IC to convert > > to BGRx between decoder and kmssink. > > Yes, I have tried to do this, but it says that v4l2convert does not > support bt601 colorimetry, and then a segfault occurs: > > # gst-launch-1.0 filesrc location=/media/clip.mp4 ! qtdemux ! > h264parse ! v4l2h264dec ! v4l2convert ! video/x-raw,format=BGRx ! > kmssink > Setting pipeline to PAUSED ... > Pipeline is PREROLLING ... > ERROR: from element /GstPipeline:pipeline0/v4l2convert:v4l2convert0: > Device '/dev/video4' does not support bt601 colorimetry > Additional debug info: > ../sys/v4l2/gstv4l2object.c(4032): gst_v4l2_object_set_format_full (): > /GstPipeline:pipeline0/v4l2convert:v4l2convert0: > Device wants 2:4:5:4 colorimetry > ERROR: pipeline doesn't want to preroll. > Setting pipeline to NULL ... > Caught SIGSEGV > exec gdb failed: No such file or directory > Spinning. Please run 'gdb gst-launch-1.0 217' to continue debugging, > Ctrl-C to quit, or Ctrl-\ to dump core. > > Is the Gstreamer pipeline above correct? Yes. Please try if the following patch makes it work: ----------8<---------- From c45afcaf6fbef56a86dce19200c06df78718db60 Mon Sep 17 00:00:00 2001 From: Philipp Zabel <[hidden email]> Date: Mon, 18 Jan 2021 15:54:43 +0100 Subject: [PATCH] v4l2object: handle GST_VIDEO_TRANSFER_BT601 V4L2 makes no difference between the BT.601 and BT.709 transfer functions [1], but GStreamer does since 1.18 [2]. Adapt gst_v4l2_object_get_colorspace() and gst_v4l2_object_set_format_full(). [1] https://linuxtv.org/downloads/v4l-dvb-apis-new/userspace-api/v4l/colorspaces-details.html#colorspace-smpte-170m-v4l2-colorspace-smpte170m [2] https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/724 --- sys/v4l2/gstv4l2object.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/sys/v4l2/gstv4l2object.c b/sys/v4l2/gstv4l2object.c index ea4363e17303..b13216d75836 100644 --- a/sys/v4l2/gstv4l2object.c +++ b/sys/v4l2/gstv4l2object.c @@ -2334,7 +2334,7 @@ gst_v4l2_object_get_colorspace (struct v4l2_format *fmt, case V4L2_COLORSPACE_SMPTE170M: cinfo->range = GST_VIDEO_COLOR_RANGE_16_235; cinfo->matrix = GST_VIDEO_COLOR_MATRIX_BT601; - cinfo->transfer = GST_VIDEO_TRANSFER_BT709; + cinfo->transfer = GST_VIDEO_TRANSFER_BT601; cinfo->primaries = GST_VIDEO_COLOR_PRIMARIES_SMPTE170M; break; case V4L2_COLORSPACE_REC709: @@ -2463,6 +2463,8 @@ gst_v4l2_object_get_colorspace (struct v4l2_format *fmt, case V4L2_XFER_FUNC_709: if (colorspace == V4L2_COLORSPACE_BT2020 && fmt->fmt.pix.height >= 2160) cinfo->transfer = GST_VIDEO_TRANSFER_BT2020_12; + else if (colorspace == V4L2_COLORSPACE_SMPTE170M) + cinfo->transfer = GST_VIDEO_TRANSFER_BT601; else cinfo->transfer = GST_VIDEO_TRANSFER_BT709; break; @@ -3855,6 +3857,7 @@ gst_v4l2_object_set_format_full (GstV4l2Object * v4l2object, GstCaps * caps, case GST_VIDEO_TRANSFER_GAMMA10: transfer = V4L2_XFER_FUNC_NONE; break; + case GST_VIDEO_TRANSFER_BT601: case GST_VIDEO_TRANSFER_BT2020_12: case GST_VIDEO_TRANSFER_BT709: transfer = V4L2_XFER_FUNC_709; -- 2.20.1 ---------->8---------- This may not be the correct solution. GStreamer could keep choosing BT709 as told by V4L2, and use the new gst_video_color_transfer_is_equivalent() function to test for equivalence instead. regards Philipp _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Coda: imx53 plays video with incorrect colors
|
|
Hi Philipp,
On Mon, Jan 18, 2021 at 12:29 PM Philipp Zabel <[hidden email]> wrote: > Please try forcing decoder output to NV12 instead of I420. Your suggestion works, thank you. After applying your IPU patch and using this Gstreamer pipeline: gst-launch-1.0 filesrc location=/media/clip.mp4 ! qtdemux ! h264parse ! v4l2h264dec ! video/x-raw,format=NV12 ! kmssink The video is played with correct colors. Can we make the NV12 format be used automatically so that a simple 'gst-play-1.0 /media/clip.mp4' works? > Yes. Please try if the following patch makes it work: With this Gstreamer patch, I don't see the segfault anymore, but it plays in black-and-white. Thanks _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: Coda: imx53 plays video with incorrect colors
|
|
Hi Fabio,
On Mon, 2021-01-18 at 15:36 -0300, Fabio Estevam wrote: > Hi Philipp, > > On Mon, Jan 18, 2021 at 12:29 PM Philipp Zabel <[hidden email]> wrote: > > > Please try forcing decoder output to NV12 instead of I420. > > Your suggestion works, thank you. After applying your IPU patch and > using this Gstreamer pipeline: > > gst-launch-1.0 filesrc location=/media/clip.mp4 ! qtdemux ! h264parse > ! v4l2h264dec ! video/x-raw,format=NV12 ! kmssink > > The video is played with correct colors. Ok, thank you. It looks to me like the I420 issue is either a bug in coda-dev, or an interaction issue between GStreamer and coda-dev, where coda-dev doesn't let GStreamer change the format at a point where it expects to be able to. The b/w image (with slight discolorations in a regular pattern) could very well be actual NV12 interpreted as I420 by GStreamer (and thus kmssink). I'll have closer look. > Can we make the NV12 format be used automatically so that a simple > 'gst-play-1.0 /media/clip.mp4' works? That would be a matter of increasing the rank of the NV12 format in gst_v4l2_object_format_get_rank() above that of I420 and YV12. > > Yes. Please try if the following patch makes it work: > > With this Gstreamer patch, I don't see the segfault anymore, but it > plays in black-and-white. Thanks for testing. I've opened https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/merge_requests/856 to track this. regards Philipp _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
«
Return to GStreamer-devel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |