Issue with colorimetry field values while using qtmux/qtdemux for H264 encode/decode

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Issue with colorimetry field values while using qtmux/qtdemux for H264 encode/decode

Amit Pandya
Hi all,

Following pipeline used for 4K H264 encode using qtmux pipeline,

*a)  gst-launch-1.0 videotestsrc num-buffers=100 ! 'video/x-raw,
width=(int)3840, height=(int)2160, format=(string)I420' ! x264enc !
h264parse ! qtmux ! filesink
location=test_x264enc_3840x2160_I420_100frames.mp4 -v -e*
....
/GstPipeline:pipeline0/GstX264Enc:x264enc0.GstPad:src: caps = video/x-h264,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
stream-format=(string)avc, alignment=(string)au, level=(string)5.1,
profile=(string)high, width=(int)3840, height=(int)2160,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1,
interlace-mode=(string)progressive, *colorimetry=(string)bt2020*,
chroma-site=(string)mpeg2, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps =
video/x-h264,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
stream-format=(string)avc, alignment=(string)au, level=(string)5.1,
profile=(string)high, width=(int)3840, height=(int)2160,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1,
interlace-mode=(string)progressive, *colorimetry=(string)bt2020*,
chroma-site=(string)mpeg2, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
chroma-format=(string)4:2:0, bit-depth-luma=(uint)8,
bit-depth-chroma=(uint)8, parsed=(boolean)true
/GstPipeline:pipeline0/GstQTMux:qtmux0.GstQTMuxPad:video_0: caps =
video/x-h264,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
stream-format=(string)avc, alignment=(string)au, level=(string)5.1,
profile=(string)high, width=(int)3840, height=(int)2160,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1,
interlace-mode=(string)progressive, *colorimetry=(string)bt2020*,
chroma-site=(string)mpeg2, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
chroma-format=(string)4:2:0, bit-depth-luma=(uint)8,
bit-depth-chroma=(uint)8, parsed=(boolean)true
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps =
video/x-h264,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
stream-format=(string)avc, alignment=(string)au, level=(string)5.1,
profile=(string)high, width=(int)3840, height=(int)2160,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1,
interlace-mode=(string)progressive, *colorimetry=(string)bt2020*,
chroma-site=(string)mpeg2, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono

With above x264enc 4k encode pipelines colorimetry bt2020 is getting
negotiated & same is used by the qtmux element.

*b) gst-launch-1.0 filesrc
location=test_x264enc_3840x2160_I420_100frames.mp4 ! qtdemux ! h264parse !
'video/x-h264, stream-format=(string)byte-stream, alignment=(string)au' !
fakesink -v -e*
....
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps =
video/x-h264, stream-format=(string)avc, alignment=(string)au,
level=(string)5.1, profile=(string)high,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
width=(int)3840, height=(int)2160, framerate=(fraction)30/1,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
*colorimetry=(string)2:6:0:7*
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps =
video/x-h264, stream-format=(string)byte-stream, alignment=(string)au,
level=(string)5.1, profile=(string)high, width=(int)3840, height=(int)2160,
framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, *colorimetry=(string)2:6:0:7*,
chroma-format=(string)4:2:0, bit-depth-luma=(uint)8,
bit-depth-chroma=(uint)8, parsed=(boolean)true

When use above pipelines using same encoded mp4 file from a) then qtdemux
provides colorimetry=2:6:0:7 with output caps.
Now, this colorimetry values are different then the format derived
colorimetry values for bt2020=2:6:11:7.

Issue seems to be arising from the way qtmux generates ext_atom for colr_ext
for colorimetry.
while building colr_extension atom "colorimetry->transfer" enum for
GST_VIDEO_TRANSFER_BT2020_12 is not handled.
(Refer gst-plugins-good-1.14.3: gst/isomp4/gstqtmux.c+5594 &
gst/isomp4/atoms.c+4152 )

With qtdemux implementation also while parsing the track for colr_extension
atom respective case is not handled yielding "colorimetry->transfer" enum
value for GST_VIDEO_TRANSFER_UNKNOWN(0).
(Refer gst-plugins-good-1.14.3: gst/isomp4/qtdemux.c+10168)

Above creates problem with the decode pipeline where decoder have
colorimetry equality check for format derived colorimetry
(bt2020-"2:6:11:7") with caps received colorimetry values "2:6:0:7" in
_set_format()

It appears to be missing functionality with qtmux/qtdemux implementation &
enhancement should be done for handling  bt2020 colorimetry value of
GstVideoTransferFunction.

Let me know in case any relevant inputs.

-Thanks.






--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Issue with colorimetry field values while using qtmux/qtdemux for H264 encode/decode

Nicolas Dufresne-5


Le jeu. 20 sept. 2018 07 h 57, Amit Pandya <[hidden email]> a écrit :
Hi all,

Following pipeline used for 4K H264 encode using qtmux pipeline,



*a)  gst-launch-1.0 videotestsrc num-buffers=100 ! 'video/x-raw,
width=(int)3840, height=(int)2160, format=(string)I420' ! x264enc !
h264parse ! qtmux ! filesink
location=test_x264enc_3840x2160_I420_100frames.mp4 -v -e*
....
/GstPipeline:pipeline0/GstX264Enc:x264enc0.GstPad:src: caps = video/x-h264,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
stream-format=(string)avc, alignment=(string)au, level=(string)5.1,
profile=(string)high, width=(int)3840, height=(int)2160,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1,
interlace-mode=(string)progressive, *colorimetry=(string)bt2020*,
chroma-site=(string)mpeg2, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps =
video/x-h264,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
stream-format=(string)avc, alignment=(string)au, level=(string)5.1,
profile=(string)high, width=(int)3840, height=(int)2160,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1,
interlace-mode=(string)progressive, *colorimetry=(string)bt2020*,
chroma-site=(string)mpeg2, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
chroma-format=(string)4:2:0, bit-depth-luma=(uint)8,
bit-depth-chroma=(uint)8, parsed=(boolean)true
/GstPipeline:pipeline0/GstQTMux:qtmux0.GstQTMuxPad:video_0: caps =
video/x-h264,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
stream-format=(string)avc, alignment=(string)au, level=(string)5.1,
profile=(string)high, width=(int)3840, height=(int)2160,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1,
interlace-mode=(string)progressive, *colorimetry=(string)bt2020*,
chroma-site=(string)mpeg2, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
chroma-format=(string)4:2:0, bit-depth-luma=(uint)8,
bit-depth-chroma=(uint)8, parsed=(boolean)true
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps =
video/x-h264,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
stream-format=(string)avc, alignment=(string)au, level=(string)5.1,
profile=(string)high, width=(int)3840, height=(int)2160,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1,
interlace-mode=(string)progressive, *colorimetry=(string)bt2020*,
chroma-site=(string)mpeg2, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono

With above x264enc 4k encode pipelines colorimetry bt2020 is getting
negotiated & same is used by the qtmux element.

*b) gst-launch-1.0 filesrc
location=test_x264enc_3840x2160_I420_100frames.mp4 ! qtdemux ! h264parse !
'video/x-h264, stream-format=(string)byte-stream, alignment=(string)au' !
fakesink -v -e*
....
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps =
video/x-h264, stream-format=(string)avc, alignment=(string)au,
level=(string)5.1, profile=(string)high,
codec_data=(buffer)01640033ffe1001e67640033acd9403c0043ec05a848104a000003000200000300791e30632c01000568ebecb22c,
width=(int)3840, height=(int)2160, framerate=(fraction)30/1,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive,
*colorimetry=(string)2:6:0:7*
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps =
video/x-h264, stream-format=(string)byte-stream, alignment=(string)au,
level=(string)5.1, profile=(string)high, width=(int)3840, height=(int)2160,
framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1,
interlace-mode=(string)progressive, *colorimetry=(string)2:6:0:7*,
chroma-format=(string)4:2:0, bit-depth-luma=(uint)8,
bit-depth-chroma=(uint)8, parsed=(boolean)true

When use above pipelines using same encoded mp4 file from a) then qtdemux
provides colorimetry=2:6:0:7 with output caps.
Now, this colorimetry values are different then the format derived
colorimetry values for bt2020=2:6:11:7.

Issue seems to be arising from the way qtmux generates ext_atom for colr_ext
for colorimetry.
while building colr_extension atom "colorimetry->transfer" enum for
GST_VIDEO_TRANSFER_BT2020_12 is not handled.
(Refer gst-plugins-good-1.14.3: gst/isomp4/gstqtmux.c+5594 &
gst/isomp4/atoms.c+4152 )

With qtdemux implementation also while parsing the track for colr_extension
atom respective case is not handled yielding "colorimetry->transfer" enum
value for GST_VIDEO_TRANSFER_UNKNOWN(0).
(Refer gst-plugins-good-1.14.3: gst/isomp4/qtdemux.c+10168)

Above creates problem with the decode pipeline where decoder have
colorimetry equality check for format derived colorimetry
(bt2020-"2:6:11:7") with caps received colorimetry values "2:6:0:7" in
_set_format()

It appears to be missing functionality with qtmux/qtdemux implementation &
enhancement should be done for handling  bt2020 colorimetry value of
GstVideoTransferFunction.

Let me know in case any relevant inputs.

-Thanks.






--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Issue with colorimetry field values while using qtmux/qtdemux for H264 encode/decode

Amit Pandya
Nicolas,

Yes, My observed issue seems to be related to
https://bugzilla.gnome.org/show_bug.cgi?id=796761

I believe fixing Bug-796761/Bug-796753 should help to resolve this issue
too.

-Thanks



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Issue with colorimetry field values while using qtmux/qtdemux for H264 encode/decode

Nicolas Dufresne-5
Le jeudi 20 septembre 2018 à 09:19 -0500, Amit Pandya a écrit :
> Nicolas,
>
> Yes, My observed issue seems to be related to
> https://bugzilla.gnome.org/show_bug.cgi?id=796761
>
> I believe fixing Bug-796761/Bug-796753 should help to resolve this
> issue
> too.

Only issue is that I'm still seeking for documentation on the newest
colorimetry values for the ISOMP4 format. There is another interesting
issue which can be referred here:

https://bugzilla.gnome.org/show_bug.cgi?id=796753

This one is about the defaults. Whenever it's 4K, we pick BT2020 in
GStreamer. In the field, it seams that the rules is fancier then that.
In general, BT2020 is picked over BT709 only when the depth is 10bit or
more.

>
> -Thanks
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (201 bytes) Download Attachment