Low latency video stream from jetson TX2 with basler pylon camera using gstreamer

[Leevip]

Hey!

We've been trying to create a solution for a low latency video stream from
pylon camera to a remote using gstreamer and the gst-pylonsrc plugin for it
(https://github.com/zingmars/gst-pylonsrc).
We are using nvidia jetson TX2 with ubuntu 16.04 LTS with ARMv8 processor.

So far we've been able to get a video stream to the remote with the
following pipeline:
/"( pylonsrc imageformat=ycbcr422_8 ! videoconvert ! videoscale !
video/x-raw,width=1024,height=768 ! x264enc bitrate=500
speed-preset=superfast tune=zerolatency ! rtph264pay name=pay0 pt=96 )"/
using the test-launch program compiled from the gst-rtsp-server source. But
the delay of this solution is around 500ms which is way too much for us.

On the receiving end we are reading the stream with Unity3d client using
ffmpeg C# wrapper. The Unity3d solution can work both as a listener to a
rtsp stream or a client to a rtsp server. We have achieved around 170ms
latency with another rtsp stream solution so the delay is probably not
caused by the Unity client.

We'd like to achieve a direct udp connection, but so far have had no luck
with the udpsink so far.

We've gotten the testvideosrc to play locally on the jetson using udpsink
with
/gst-launch-1.0 videotestsrc ! x264enc pass=qual quantizer=20
tune=zerolatency ! rtph264pay name=pay0 pt=96 ! udpsink host=127.0.0.1
port=8554/
as the sender and receive it with:
/gst-launch-1.0 udpsrc port=8554 ! "application/x-rtp, payload=127" !
rtph264depay ! avdec_h264 ! xvimagesink/

The receiver works with the aforementioned pipelines but udpsrc receiver
fails with the following pylonsrc pipeline:
/gst-launch-1.0 pylonsrc imageformat=ycbcr422_8 ! videoconvert ! videoscale
! video/x-raw,width=1024,height=768 ! x264enc pass=qual quantizer=20
tune=zerolatency ! rtph264pay name=pay0 pt=96 ! udpsink host=127.0.0.1
port=8554/
We get a: "ERROR: from element /GstPipeline:pipeline0/GstUDPSrc:udpsrc0:
Internal data flow error."

We'd like to achieve below 100ms delay video stream with the integrated
graphics capabilities of the jetson TX2 using gstreamer, but we are now
currently out of ideas.

Thanks for your help in advance!



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[sk_gst]

I notice that you are using element`rtph264pay`, but there is no use of
element `rtpbin`. Try using the element rtpbin with latency=0 to see if it
improves the behavior. Achieving something below 100ms will also depend on
the medium of communication between the client and server. It is not clear
if you are using a wireless or a wired communication.

Then again, you can try and find out which element of your pipeline is
inducing more latency w.r.t processing of the buffer. Use the GStShark
profiler tool that can provide you insights about each element:
https://developer.ridgerun.com/wiki/index.php?title=GstShark .

The individual processing time can be obtained using Processing Time
measurements :
https://developer.ridgerun.com/wiki/index.php?title=GstShark_-_Processing_Time_tracer

Hope this helps!



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[surprised]

Hi,Leevip.

I saw a post you send not long ago,and I also use jetson tx2 for gstreamer
development.And I want to get some help from you.

Recently,I set timestamp in appsrc like this: GST_BUFFER_PTS(buffer)=100;
and I get timestamp in appsink like this: timestamp=GST_BUFFER_PTS(buffer);
but I got a value which didn't equal to 100. I want to konw why.

Sincerely.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel