How to efficiently replace vaapih265enc with omxh265enc?

[jeyp4]

I am working on a h265 encode-decode pipeline.

gst-launch-1.0 -v v4l2src device=/dev/video0 ! 'video/x-raw,format=(string)YV12,width=1280,height=720,framerate=(fraction)30/1' ! vaapih265enc max-bframes=0 keyframe-period=30 ! appsink // 1st app, encoding

appsrc ! vaapidecode ! xvimagesink // 2nd app, decoding

Combination of above 2 pipelines works well. Needless to say that, above 2
pipelines are in separate applications connected through ROS.

Problem: I want to use Jetson for the 1st(encoder) pipeline. So I replace

vaapih265enc max-bframes=0 keyframe-period=30

 with

omxh265enc iframeinterval=30 quant-b-frames=0

After this replacement I had to add
'h265parse' in 2nd decoder pipeline inorder to make pipeline works. I don't want to add 'h265parse' element because it results an extra latency of ~40ms in decoding.

How can I efficiently replace 'vaapih265enc max-bframes=0 keyframe-period=30' with 'omxh265enc'?

Additional info: I am forcing 'omxh265enc' to output byte-stream instead of
hvc1, because vaapih265enc outputs byte-stream by-default. So appsink caps
is following:

caps = gst_caps_new_simple("video/x-h265",
                                                    "stream-format", G_TYPE_STRING, "byte-stream",
                                                    "alignment", G_TYPE_STRING, "au",
                                                     NULL);

--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[e-mail]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[junyan.he]

vaapih265dec need a frame aligned input, so need a h265parse to
identify the frame. It seems can not parse the byte stream.
The output of vaapih265enc is frame aligned and directly connect to
decoder can make it work, but if you store the stream into file and
load it again, a parser is needed.



On 一, 2019-11-18 at 05:11 -0600, jeyp4 wrote:

> I am working on a h265 encode-decode pipeline.
> gst-launch-1.0 -v v4l2src device=/dev/video0 !
> 'video/x-raw,format=(string)YV12,width=1280,height=720,
> framerate=(fraction)30/1' ! vaapih265enc max-bframes=0 keyframe-
> period=30 !
> appsink // 1st app, encoding
>
>
> appsrc ! vaapidecode ! xvimagesink // 2nd app decoding
> Combination of above 2 pipelines works well. Needless to say that,
> above 2
> pipelines are in separate applications connected through ROS.
>
> Problem: I want to use Jetson for the 1st(encoder) pipeline. So I
> replace
> 'vaapih265enc max-bframes=0 keyframe-period=30' with 'omxh265enc
> iframeinterval=30 quant-b-frames=0'. After this replacement I had to
> add
> 'h265parse' in 2nd decoder pipeline inorder to make application
> works. *I
> don't want to add 'h265parse' element because it results an extra
> latency of
> ~40ms in decoding.*
> How can I efficiently replace 'vaapih265enc max-bframes=0
> keyframe-period=30' with 'omxh265enc'?
>
>
> Additional info: I am forcing 'omxh265enc' to output byte-stream
> instead of
> hvc1, because vaapih265enc outputs byte-stream by-default. So appsink
> caps
> is following:
> [code]caps = gst_caps_new_simple("video/x-h265",
>                            "stream-format", G_TYPE_STRING, "byte-
> stream",
>                            "alignment", G_TYPE_STRING, "au",
>                            NULL);[/code]
>
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[jeyp4]

Thanks for replying.

<http://gstreamer-devel.966125.n4.nabble.com/file/t379085/vaapih265enc.jpg> 

Actually, above pipeline works fine, even without 'h265parse'. I am sending
buffer data in ROS msg. What I am doing at receiver side is, modifying pts,
dts with current receiver pipeline running time. And also I have to start
receiver before transmitter. In this way I observe almost no delay in
decoding ~ 2ms. I am happy with this pipeline.

Just for info: If I add h265parse in above receiver pipeline, it works then
also. But now decoding time is ~40ms. Decoding time: Difference between when
receiver received buffer and when vaapidecode dropped its output. I
confirmed this latency presence by measuring glass to glass latency also.

As I said, I am happy with above pipeline which is working fine even without 'h265parse'.
I am trying to replace vaapih265enc with omxh265enc !



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[e-mail]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[junyan.he]

Yeah, I am saying that.
If I understand right, your problem is that if you replace the
vaapih265enc in the encoder side with `omxh265enc`, you must add a
parser beforce the vaapih265dec at decoder size, or, the decoder will
report error.

I suspect that the frame alignment cause this problem. vaapih265enc
output just one frame in one gstbuffer, and so, one gstbuffer is
already frame aligned(appsrc/appsink seems direct forward data in unit
of gstbuffer?). I am not famaliar with omxh265enc's hehavior, it may
mix data from several frames together, with AU as delimiter. So one
gstbuffer may not be frame aligned. At decoder side, if you need to
detect the frame edge, a parser is always needed before vaapih265dec.



On 一, 2019-11-18 at 06:20 -0600, jeyp4 wrote:

> Thanks for replying.
>
> <http://gstreamer-devel.966125.n4.nabble.com/file/t379085/vaapih265en
> c.jpg> 
>
> Actually, above pipeline works fine, even without 'h265parse'. I am
> sending
> buffer data in ROS msg. What I am doing at receiver side is,
> modifying pts,
> dts with current receiver pipeline running time. And also I have to
> start
> receiver before transmitter. In this way I observe almost no delay in
> decoding ~ 2ms. I am happy with this pipeline.
>
> Just for info: If I add h265parse in above receiver pipeline, it
> works then
> also. But now decoding time is ~40ms. Decoding time: Difference
> between when
> receiver received buffer and when vaapidecode dropped its output. I
> confirmed this latency presence by measuring glass to glass latency
> also.
>
> As I said, I am happy with above pipeline without h265parse because
> it is
> working fine.
>
> I am trying to replace vaapih265enc with omxh265enc !
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[jeyp4]

Thanks.
I kind of understand. What you wanted to say.
So, I am trying to decipher properties of 'omxh265enc' to force its output
frame aligned.

Meanwhile I check the caps of outputs of both 'vaapih265enc' and
'omxh265enc'.
If you can look into them, it would be an important help for me.

GstVaapiEncodeH265:vaapiencodeh265-0.GstPad:src: caps = "video/x-h265\,\
stream-format\=\(string\)byte-stream\,\ alignment\=\(string\)au\,\
width\=\(int\)1280\,\ height\=\(int\)720\,\
pixel-aspect-ratio\=\(fraction\)1/1\,\ framerate\=\(fraction\)30/1"

GstOMXH265Enc-omxh265enc0.GstPad:sink: caps: video/x-h265,
alignment=(string)au, profile=(string)main, level=(string)main4,
stream-format=(string)byte-stream, width=(int)1280, height=(int)720,
pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1,
interlace-mode=(string)progressive, colorimetry=(string)1:4:0:0,
chroma-site=(string)mpeg2, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono

If anything clicks you, please let me know

--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[e-mail]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le lundi 18 novembre 2019 à 06:20 -0600, jeyp4 a écrit :

> Thanks for replying.
>
> <http://gstreamer-devel.966125.n4.nabble.com/file/t379085/vaapih265enc.jpg>
>
> Actually, above pipeline works fine, even without 'h265parse'. I am sending
> buffer data in ROS msg. What I am doing at receiver side is, modifying pts,
> dts with current receiver pipeline running time. And also I have to start
> receiver before transmitter. In this way I observe almost no delay in
> decoding ~ 2ms. I am happy with this pipeline.
>
> Just for info: If I add h265parse in above receiver pipeline, it works then
> also. But now decoding time is ~40ms. Decoding time: Difference between when
> receiver received buffer and when vaapidecode dropped its output. I
> confirmed this latency presence by measuring glass to glass latency also.

Just a note that we are trying to get some patches that address this in
h264/h265parse. Aligned input should not yield latency in the parsers,
but right now it does.

>
> As I said, I am happy with above pipeline without h265parse because it is
> working fine.
>
> I am trying to replace vaapih265enc with omxh265enc !

All you need is to find which field in the caps is needed omxh265dec,
and try and figure-out what to add in omxh265enc, you have to modify
the code. Careful, NVidia have added quite an important layer of hacks
on top of these elements, so you have to start from their code.

>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[jeyp4]

Just to clarify I am wishing to replace just ' vaapih265enc ' with ' omxh265enc'.
I want to keep  vaapih265dec as it is, as my decoding station is Intel PC.

I thought to say this because you mentioned

> All you need is to find which field in the caps is needed omxh265dec, and try and figure-out what to add in omxh265enc

Secondly, I started looking at Nvidia gstomx1_src's  gstomxh265enc.c  file. I am trying to understand this file.

If by any chance you find anything in above editable C file, which can result in frame inline output, Please let me know.

Thanks



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[e-mail]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[jeyp4]

I tried to compare Raw output after h265 encoding and h265parse output.

Raw:     Frame0: 0, 0, 0, 1, 2, 1, 208, 167, 60, 234, 176, 165, 160, 246, 179, 158, 241, 169, 251, 200, 174, 111, 45, 24, 8, 114,...
            Frame1: 0, 0, 0, 1, 2, 1, 208, 175, 47, 230, 70, 171, 70, 183, 153, 248, 232, 122, 41, 185, 52, 31, 11, 74, 238, 187,...

Parsed: Frame0: 0, 0, 0, 1, 64, 1, 12, 1, 255, 255, 1, 64, 0, 0, 3, 0, 0, 3, 0, 0, 3, 0, 0, 3, 0, 120,...
            Frame1: 0, 0, 0, 1, 2, 1, 208, 15, 36, 192, 248, 45, 21, 14, 229, 216, 1, 134, 182, 168, 178, 250, 1, 26, 78, 247,...

I am still working on how to make omxh265enc output frame aligned...



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[e-mail]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel