Hi all,
I'm facing a latency problem using mpegtsmux and tsdemux elements (windows7, gstreamer 1.15.1). I have to stream some h264 video content but my problem occurs also without actually stream through UDP. If I launch the following pipeline I can see the captured video without latency problems (few ms): gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=100 height=100 ! videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast byte-stream=true ! queue ! h264parse ! avdec_h264 ! videoconvert ! autovideosink But if I add mpegtsmux and tsdemux elements the video latency will be around 1sec: gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=100 height=100 ! videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast byte-stream=true ! mpegtsmux ! queue ! tsdemux ! h264parse ! avdec_h264 ! videoconvert ! autovideosink Any idea about this latency increment? Thank you -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Le mer. 31 juill. 2019 08 h 25, TheGiamig <[hidden email]> a écrit : Hi all, Unrelated, but quick reminder that 1.15 is a development branch and should be considered unstable. You should use 1.16 stable releases.
tsdemux has 700ms latency, as required by the MPEG TS specification. I am considering adding a property to lower that, for those who know what they are doing. Lowering it too much would lead to erroneous timestamp to be produced. On top of that, h264parse adds 1 frame latency, your display element likely add another one, and the capture element very often have up to one frame latency, there is finally a 20ms processing delay allowed iirc. So that's pretty close by 1s.
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Nicolas Dufresne-5 wrote
> Le mer. 31 juill. 2019 08 h 25, TheGiamig < > thegiamig@ > > a écrit : > > tsdemux has 700ms latency, as required by the MPEG TS specification. I am > considering adding a property to lower that, for those who know what they > are doing. Lowering it too much would lead to erroneous timestamp to be > produced. > > _______________________________________________ > gstreamer-devel mailing list > gstreamer-devel@.freedesktop > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel If I do about the same with ffmpeg the latency go away. I think 700ms is the maximum latency between PTS or someting, not the minimum latency. Anyway, what is the best approach for a very low-latency h264 video streaming? RTP? Thanks -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
In reply to this post by Nicolas Dufresne-5
Additionally, if I recall correctly, Transport Stream has a theoretical 700ms latency as per the spec and, hence, its hard coded in the element. Said in other words, this latency is induced artificially by GStreamer to comply with TS spec.
You can do a quick test by hacking the element to remove that latency, or using sync=false, or using gst_pipeline_set_latency. The later will override the overall latency reported by the elements. Just recall that this latency is there for a reason and it’s not always a good idea to remove it entirely. Michael
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
In reply to this post by TheGiamig
Le mercredi 31 juillet 2019 à 09:45 -0500, TheGiamig a écrit :
> Nicolas Dufresne-5 wrote > > Le mer. 31 juill. 2019 08 h 25, TheGiamig < > > thegiamig@ > > > a écrit : > > > > tsdemux has 700ms latency, as required by the MPEG TS specification. I am > > considering adding a property to lower that, for those who know what they > > are doing. Lowering it too much would lead to erroneous timestamp to be > > produced. > > > > _______________________________________________ > > gstreamer-devel mailing list > > gstreamer-devel@.freedesktop > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > If I do about the same with ffmpeg the latency go away. > I think 700ms is the maximum latency between PTS or someting, not the > minimum latency. > > Anyway, what is the best approach for a very low-latency h264 video > streaming? RTP? gst_pipeline_set_latency(), or disable the sync as Micheal suggest in the following email. But or out-of-the-box low latency, an RTP base pipeline will work better for now. > > Thanks > > > > > > -- > Sent from: http://gstreamer-devel.966125.n4.nabble.com/ > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel signature.asc (201 bytes) Download Attachment |
In reply to this post by Michael Gruner
Le mercredi 31 juillet 2019 à 08:47 -0600, Michael Gruner a écrit :
> Additionally, if I recall correctly, Transport Stream has a theoretical 700ms latency as per the spec and, hence, its hard coded in the element. Said in other words, this latency is induced artificially by GStreamer to comply with TS spec. > > You can do a quick test by hacking the element to remove that latency, or using sync=false, or using gst_pipeline_set_latency. > > The later will override the overall latency reported by the elements. Just recall that this latency is there for a reason and it’s not always a good idea to remove it entirely. Right now, one has the option to set the pipeline latency to a lower value in order to crop the reported latency. There will be a small warning, but it works. You can check your current latency cost, as tsdemux will report 700ms, but will only wait for what's needed to get proper timestamp. So using the latency tracer will let you know what seems to be appropriate for you stream. The stream configuration (tsmux configuration) has a large influence on this. > > Michael > www.ridgerun.com > > On Jul 31, 2019, at 7:13 AM, Nicolas Dufresne <[hidden email]> wrote: > > > > > Le mer. 31 juill. 2019 08 h 25, TheGiamig <[hidden email]> a écrit : > > > Hi all, > > > I'm facing a latency problem using mpegtsmux and tsdemux elements (windows7, > > > gstreamer 1.15.1). > > > > Unrelated, but quick reminder that 1.15 is a development branch and should be considered unstable. You should use 1.16 stable releases. > > > > > I have to stream some h264 video content but my problem occurs also without > > > actually stream through UDP. > > > > > > If I launch the following pipeline I can see the captured video without > > > latency problems (few ms): > > > > > > gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=100 height=100 ! > > > videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast > > > byte-stream=true ! queue ! h264parse ! avdec_h264 ! videoconvert ! > > > autovideosink > > > > > > But if I add mpegtsmux and tsdemux elements the video latency will be around > > > 1sec: > > > > > > gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=100 height=100 ! > > > videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast > > > byte-stream=true ! mpegtsmux ! queue ! tsdemux ! h264parse ! avdec_h264 ! > > > videoconvert ! autovideosink > > > > > > Any idea about this latency increment? > > > > tsdemux has 700ms latency, as required by the MPEG TS specification. I am considering adding a property to lower that, for those who know what they are doing. Lowering it too much would lead to erroneous timestamp to be produced. > > > > On top of that, h264parse adds 1 frame latency, your display element likely add another one, and the capture element very often have up to one frame latency, there is finally a 20ms processing delay allowed iirc. So that's pretty close by 1s. > > > > > Thank you > > > > > > > > > > > > > > > -- > > > Sent from: http://gstreamer-devel.966125.n4.nabble.com/ > > > _______________________________________________ > > > gstreamer-devel mailing list > > > [hidden email] > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > _______________________________________________ > > gstreamer-devel mailing list > > [hidden email] > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel signature.asc (201 bytes) Download Attachment |
Thanks for your responses, I removed the unwanted latency putting
autovideosink sync=false. At the moment is a good workaround for me, I'm just trying to understand how all the thing works.. So now I can capture a portion of desktop, encode in h264 and put out a mpegts stream. Next step is the streaming over network. I just pushed the stream in a udpsink and on the other side got it by udpsrc: sender: gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=800 height=600 ! videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast byte-stream=true ! mpegtsmux alignment=7 ! queue ! udpsink host=127.0.0.1 port=5000 receiver: gst-launch-1.0 udpsrc address=127.0.0.1 port=5000 ! tsdemux ! queue ! decodebin ! videoconvert ! autovideosink sync=false This way the video is corrupted, when the image changes fastly I got strange pixels, green or grey areas and so on. It seems some data will be lost. It can be normal on internet, but on 127.0.0.1.. I got the same result using RTP instead of mpegts. So I replaced udpink and udpsrc with tcpclientsink and tcpserversrc and the problem goes away. What do you think about? Thanks -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
You are likely losing your H264 reference frame and the encoder is not generating another one. Try setting the encoders “key-int-max=30” (or some prudential value) to force periodical generation.
Michael www.ridgerun.com > On Aug 1, 2019, at 1:27 AM, TheGiamig <[hidden email]> wrote: > > Thanks for your responses, I removed the unwanted latency putting > autovideosink sync=false. > At the moment is a good workaround for me, I'm just trying to understand how > all the thing works.. > > So now I can capture a portion of desktop, encode in h264 and put out a > mpegts stream. > Next step is the streaming over network. > I just pushed the stream in a udpsink and on the other side got it by > udpsrc: > > sender: > gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=800 height=600 ! > videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast > byte-stream=true ! mpegtsmux alignment=7 ! queue ! udpsink host=127.0.0.1 > port=5000 > > receiver: > gst-launch-1.0 udpsrc address=127.0.0.1 port=5000 ! tsdemux ! queue ! > decodebin ! videoconvert ! autovideosink sync=false > > This way the video is corrupted, when the image changes fastly I got strange > pixels, green or grey areas and so on. It seems some data will be lost. It > can be normal on internet, but on 127.0.0.1.. > > I got the same result using RTP instead of mpegts. > > So I replaced udpink and udpsrc with tcpclientsink and tcpserversrc and the > problem goes away. > > What do you think about? > > Thanks > > > > > > > -- > Sent from: http://gstreamer-devel.966125.n4.nabble.com/ > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Michael Gruner wrote
> You are likely losing your H264 reference frame and the encoder is not > generating another one. Try setting the encoders “key-int-max=30” (or some > prudential value) to force periodical generation. > > Michael > www.ridgerun.com > _______________________________________________ > gstreamer-devel mailing list > gstreamer-devel@.freedesktop > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel I tried key-int-max parameter but no luck. The problem seems to be some packet loss but I can't understand why, because I'm using UDP on localhost. May be there is a way to print statistics about send and receive packets on udpsink/src elements. Thanks -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
In reply to this post by TheGiamig
Le mer. 31 juill. 2019 08 h 25, TheGiamig <[hidden email]> a écrit : Hi all, Please,don't use odd versions of GStreamer, these are delopment version. Use the release version instead, this way you are less likely to hit unknown bugs that only existed for a short period of time during development.
tsdemux contributes 700ms of latency to the pipeline. We have added recently a latency property element to configure this. The latency depends on the PCR distance as configured in mpegtsmux. You need 2 PCR to start streaming, so the minimum possible latency is twice the period. Note that the configuration of mpegtsmux is based on a 9 kHz clock, so the value is N / 90000 second. Default is 3600, which means 40ms (so 80ms minimum latency).
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Thanks Nicolas, you are right and I understand the origin of latency .
I faced also a problem of packets loss with udpsink/src, the problem goes away usinf tcpsink/src. So I replied to Michael about this topic: Michael Gruner wrote > You are likely losing your H264 reference frame and the encoder is not > generating another one. Try setting the encoders “key-int-max=30” (or some > prudential value) to force periodical generation. > > Michael > www.ridgerun.com > _______________________________________________ > gstreamer-devel mailing list > gstreamer-devel@.freedesktop > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel *I tried key-int-max parameter but no luck. The problem seems to be some packet loss but I can't understand why, because I'm using UDP on localhost. May be there is a way to print statistics about send and receive packets on udpsink/src elements. Thanks * -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Finally I solved the packet loss problem enlarging the socket buffer size of
udpsink/src (buffer-size=65535 is good for me). Maybe the streaming was filling the buffers causing the packet loss. -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Le lundi 16 septembre 2019 à 02:35 -0500, TheGiamig a écrit :
> Finally I solved the packet loss problem enlarging the socket buffer size of > udpsink/src (buffer-size=65535 is good for me). > > Maybe the streaming was filling the buffers causing the packet loss. The sender is likely "bursty", as your process has relatively low priority, in such situation it might not be scheduled often enough to avoid lost. In some cases, adding a queue element after udpsrc can solve this (as it allow udpsrc to avoid most of the push pack effect in GStreamer), but increasing the buffer-size property is often more efficient (less context switches). > > > > > > -- > Sent from: http://gstreamer-devel.966125.n4.nabble.com/ > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Free forum by Nabble | Edit this page |