Unexpected latency increase on video encoder with 3-camera setup

[bdjukic]

Hi there,

We are running 3 camera GStreamer setup
(https://www.e-consystems.com/three-synchronized-4k-cameras-for-nvidia-jetson-tx2.asp)
on our NVIDIA Jetson TX2 with the following pipeline:

*gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,
format=(string)UYVY, width=(int)1280, height=(int)720" ! nvvidconv
output-buffers=50 ! omxh264enc control-rate=2 bitrate=600000 !
'video/x-h264, width=1280, height=720, stream-format=(string)byte-stream' !
queue ! h264parse ! queue ! rtph264pay mtu=800 ! udpsink host=192.168.1.3
port=5000 sync=false
*
The pipeline runs at *100fps*.

After triggering all three pipelines at the same time, we can observe
noticeable latency increase after 10 seconds period. We have managed to
capture this strange behavior using GST Shark:
https://1drv.ms/u/s!As2zxTr-3RwhhLFsLYWGM3dme-pdRQ . We can see that
*omxh264enc* encoder cannot keep up with the incoming frames after 10
seconds.

We're confident that hardware can handle this flow, but it's rather the
GStreamer pipeline which is not optimized.

Any ideas on how we can increase the efficiency of the hardware encoder in
our pipeline?

Thanks!

B.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le mardi 11 septembre 2018 à 12:13 -0500, bdjukic a écrit :

> Hi there,
>
> We are running 3 camera GStreamer setup
> (https://www.e-consystems.com/three-synchronized-4k-cameras-for-nvidia-jetson-tx2.asp)
> on our NVIDIA Jetson TX2 with the following pipeline:
>
> *gst-launch-1.0 v4l2src device=/dev/video0 ! "video/x-raw,
> format=(string)UYVY, width=(int)1280, height=(int)720" ! nvvidconv
> output-buffers=50 ! omxh264enc control-rate=2 bitrate=600000 !
> 'video/x-h264, width=1280, height=720, stream-format=(string)byte-stream' !
> queue ! h264parse ! queue ! rtph264pay mtu=800 ! udpsink host=192.168.1.3
> port=5000 sync=false

With sync=false, you'll reduce the latency, but the output will be
jittery/bursty. I would experiment the effect of going sync combined
with max-bitrate udpsink property.

> *
> The pipeline runs at *100fps*.
>
> After triggering all three pipelines at the same time, we can observe
> noticeable latency increase after 10 seconds period. We have managed to
> capture this strange behavior using GST Shark:
> https://1drv.ms/u/s!As2zxTr-3RwhhLFsLYWGM3dme-pdRQ . We can see that
> *omxh264enc* encoder cannot keep up with the incoming frames after 10
> seconds.
>
> We're confident that hardware can handle this flow, but it's rather the
> GStreamer pipeline which is not optimized.

I've pushed this week fixes to gst-plugins-base and gst-omx to improve
omx video decoder/encoder throughput. I don't know how feasible it will
be for you to backport this into NVidia fork. This could help.

https://bugzilla.gnome.org/show_bug.cgi?id=715192

There is tones more improvement in gst-omx, but I have little fate it
will be doable to backport.

>
> Any ideas on how we can increase the efficiency of the hardware encoder in
> our pipeline?
>
> Thanks!
>
> B.
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel