mpegts latency

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

mpegts latency

TheGiamig
Hi all,
I'm facing a latency problem using mpegtsmux and tsdemux elements (windows7,
gstreamer 1.15.1).

I have to stream some h264 video content but my problem occurs also without
actually stream through UDP.

If I launch the following pipeline I can see the captured video without
latency problems (few ms):

gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=100 height=100 !
videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast
byte-stream=true ! queue ! h264parse ! avdec_h264 ! videoconvert !
autovideosink

But if I add mpegtsmux and tsdemux elements the video latency will be around
1sec:

gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=100 height=100 !
videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast
byte-stream=true ! mpegtsmux ! queue ! tsdemux ! h264parse ! avdec_h264 !
videoconvert ! autovideosink

Any idea about this latency increment?

Thank you




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: mpegts latency

Nicolas Dufresne-5


Le mer. 31 juill. 2019 08 h 25, TheGiamig <[hidden email]> a écrit :
Hi all,
I'm facing a latency problem using mpegtsmux and tsdemux elements (windows7,
gstreamer 1.15.1).

Unrelated, but quick reminder that 1.15 is a development branch and should be considered unstable. You should use 1.16 stable releases.


I have to stream some h264 video content but my problem occurs also without
actually stream through UDP.

If I launch the following pipeline I can see the captured video without
latency problems (few ms):

gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=100 height=100 !
videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast
byte-stream=true ! queue ! h264parse ! avdec_h264 ! videoconvert !
autovideosink

But if I add mpegtsmux and tsdemux elements the video latency will be around
1sec:

gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=100 height=100 !
videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast
byte-stream=true ! mpegtsmux ! queue ! tsdemux ! h264parse ! avdec_h264 !
videoconvert ! autovideosink

Any idea about this latency increment?

tsdemux has 700ms latency, as required by the MPEG TS specification. I am considering adding a property to lower that, for those who know what they are doing. Lowering it too much would lead to erroneous timestamp to be produced.

On top of that, h264parse adds 1 frame latency, your display element likely add another one, and the capture element very often have up to one frame latency, there is finally a 20ms processing delay allowed iirc. So that's pretty close by 1s.


Thank you




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: mpegts latency

TheGiamig
Nicolas Dufresne-5 wrote
> Le mer. 31 juill. 2019 08 h 25, TheGiamig &lt;

> thegiamig@

> &gt; a écrit :
>
> tsdemux has 700ms latency, as required by the MPEG TS specification. I am
> considering adding a property to lower that, for those who know what they
> are doing. Lowering it too much would lead to erroneous timestamp to be
> produced.
>
> _______________________________________________
> gstreamer-devel mailing list

> gstreamer-devel@.freedesktop

> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

If I do about the same with ffmpeg the latency go away.
I think 700ms is the maximum latency between PTS or someting, not the
minimum latency.

Anyway, what is the best approach for a very low-latency h264 video
streaming? RTP?

Thanks

 



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: mpegts latency

Michael Gruner
In reply to this post by Nicolas Dufresne-5
Additionally, if I recall correctly, Transport Stream has a theoretical 700ms latency as per the spec and, hence, its hard coded in the element. Said in other words, this latency is induced artificially by GStreamer to comply with TS spec.

You can do a quick test by hacking the element to remove that latency, or using sync=false, or using gst_pipeline_set_latency.

The later will override the overall latency reported by the elements. Just recall that this latency is there for a reason and it’s not always a good idea to remove it entirely.

Michael

On Jul 31, 2019, at 7:13 AM, Nicolas Dufresne <[hidden email]> wrote:



Le mer. 31 juill. 2019 08 h 25, TheGiamig <[hidden email]> a écrit :
Hi all,
I'm facing a latency problem using mpegtsmux and tsdemux elements (windows7,
gstreamer 1.15.1).

Unrelated, but quick reminder that 1.15 is a development branch and should be considered unstable. You should use 1.16 stable releases.


I have to stream some h264 video content but my problem occurs also without
actually stream through UDP.

If I launch the following pipeline I can see the captured video without
latency problems (few ms):

gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=100 height=100 !
videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast
byte-stream=true ! queue ! h264parse ! avdec_h264 ! videoconvert !
autovideosink

But if I add mpegtsmux and tsdemux elements the video latency will be around
1sec:

gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=100 height=100 !
videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast
byte-stream=true ! mpegtsmux ! queue ! tsdemux ! h264parse ! avdec_h264 !
videoconvert ! autovideosink

Any idea about this latency increment?

tsdemux has 700ms latency, as required by the MPEG TS specification. I am considering adding a property to lower that, for those who know what they are doing. Lowering it too much would lead to erroneous timestamp to be produced.

On top of that, h264parse adds 1 frame latency, your display element likely add another one, and the capture element very often have up to one frame latency, there is finally a 20ms processing delay allowed iirc. So that's pretty close by 1s.


Thank you




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: mpegts latency

Nicolas Dufresne-5
In reply to this post by TheGiamig
Le mercredi 31 juillet 2019 à 09:45 -0500, TheGiamig a écrit :

> Nicolas Dufresne-5 wrote
> > Le mer. 31 juill. 2019 08 h 25, TheGiamig &lt;
> > thegiamig@
> > &gt; a écrit :
> >
> > tsdemux has 700ms latency, as required by the MPEG TS specification. I am
> > considering adding a property to lower that, for those who know what they
> > are doing. Lowering it too much would lead to erroneous timestamp to be
> > produced.
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > gstreamer-devel@.freedesktop
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> If I do about the same with ffmpeg the latency go away.
> I think 700ms is the maximum latency between PTS or someting, not the
> minimum latency.
>
> Anyway, what is the best approach for a very low-latency h264 video
> streaming? RTP?
If you want to stick with MPEG TS, you can hack the latency through
gst_pipeline_set_latency(), or disable the sync as Micheal suggest in
the following email. But or out-of-the-box low latency, an RTP base
pipeline will work better for now.

>
> Thanks
>
>  
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: mpegts latency

Nicolas Dufresne-5
In reply to this post by Michael Gruner
Le mercredi 31 juillet 2019 à 08:47 -0600, Michael Gruner a écrit :
> Additionally, if I recall correctly, Transport Stream has a theoretical 700ms latency as per the spec and, hence, its hard coded in the element. Said in other words, this latency is induced artificially by GStreamer to comply with TS spec.
>
> You can do a quick test by hacking the element to remove that latency, or using sync=false, or using gst_pipeline_set_latency.
>
> The later will override the overall latency reported by the elements. Just recall that this latency is there for a reason and it’s not always a good idea to remove it entirely.

Right now, one has the option to set the pipeline latency to a lower
value in order to crop the reported latency. There will be a small
warning, but it works.

You can check your current latency cost, as tsdemux will report 700ms,
but will only wait for what's needed to get proper timestamp. So using
the latency tracer will let you know what seems to be appropriate for
you stream. The stream configuration (tsmux configuration) has a large
influence on this.

>
> Michael
> www.ridgerun.com
>
> On Jul 31, 2019, at 7:13 AM, Nicolas Dufresne <[hidden email]> wrote:
>
> >
> > Le mer. 31 juill. 2019 08 h 25, TheGiamig <[hidden email]> a écrit :
> > > Hi all,
> > > I'm facing a latency problem using mpegtsmux and tsdemux elements (windows7,
> > > gstreamer 1.15.1).
> >
> > Unrelated, but quick reminder that 1.15 is a development branch and should be considered unstable. You should use 1.16 stable releases.
> >
> > > I have to stream some h264 video content but my problem occurs also without
> > > actually stream through UDP.
> > >
> > > If I launch the following pipeline I can see the captured video without
> > > latency problems (few ms):
> > >
> > > gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=100 height=100 !
> > > videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast
> > > byte-stream=true ! queue ! h264parse ! avdec_h264 ! videoconvert !
> > > autovideosink
> > >
> > > But if I add mpegtsmux and tsdemux elements the video latency will be around
> > > 1sec:
> > >
> > > gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=100 height=100 !
> > > videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast
> > > byte-stream=true ! mpegtsmux ! queue ! tsdemux ! h264parse ! avdec_h264 !
> > > videoconvert ! autovideosink
> > >
> > > Any idea about this latency increment?
> >
> > tsdemux has 700ms latency, as required by the MPEG TS specification. I am considering adding a property to lower that, for those who know what they are doing. Lowering it too much would lead to erroneous timestamp to be produced.
> >
> > On top of that, h264parse adds 1 frame latency, your display element likely add another one, and the capture element very often have up to one frame latency, there is finally a 20ms processing delay allowed iirc. So that's pretty close by 1s.
> >
> > > Thank you
> > >
> > >
> > >
> > >
> > > --
> > > Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > [hidden email]
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: mpegts latency

TheGiamig
Thanks for your responses, I removed the unwanted latency putting
autovideosink sync=false.
At the moment is a good workaround for me, I'm just trying to understand how
all the thing works..

So now I can capture a portion of desktop, encode in h264 and put out a
mpegts stream.
Next step is the streaming over network.
I just pushed the stream in a udpsink and on the other side got it by
udpsrc:

sender:
gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=800 height=600 !
videoconvert  ! queue ! x264enc tune=zerolatency speed-preset=ultrafast
byte-stream=true ! mpegtsmux alignment=7 ! queue ! udpsink host=127.0.0.1
port=5000

receiver:
gst-launch-1.0 udpsrc address=127.0.0.1 port=5000 ! tsdemux ! queue !
decodebin ! videoconvert ! autovideosink sync=false

This way the video is corrupted, when the image changes fastly I got strange
pixels, green or grey areas and so on. It seems some data will be lost. It
can be normal on internet, but on 127.0.0.1..

I got the same result using RTP instead of mpegts.

So I replaced udpink and udpsrc with tcpclientsink and tcpserversrc and the
problem goes away.

What do you think about?

Thanks






--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: mpegts latency

Michael Gruner
You are likely losing your H264 reference frame and the encoder is not generating another one. Try setting the encoders “key-int-max=30” (or some prudential value) to force periodical generation.

Michael
www.ridgerun.com

> On Aug 1, 2019, at 1:27 AM, TheGiamig <[hidden email]> wrote:
>
> Thanks for your responses, I removed the unwanted latency putting
> autovideosink sync=false.
> At the moment is a good workaround for me, I'm just trying to understand how
> all the thing works..
>
> So now I can capture a portion of desktop, encode in h264 and put out a
> mpegts stream.
> Next step is the streaming over network.
> I just pushed the stream in a udpsink and on the other side got it by
> udpsrc:
>
> sender:
> gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=800 height=600 !
> videoconvert  ! queue ! x264enc tune=zerolatency speed-preset=ultrafast
> byte-stream=true ! mpegtsmux alignment=7 ! queue ! udpsink host=127.0.0.1
> port=5000
>
> receiver:
> gst-launch-1.0 udpsrc address=127.0.0.1 port=5000 ! tsdemux ! queue !
> decodebin ! videoconvert ! autovideosink sync=false
>
> This way the video is corrupted, when the image changes fastly I got strange
> pixels, green or grey areas and so on. It seems some data will be lost. It
> can be normal on internet, but on 127.0.0.1..
>
> I got the same result using RTP instead of mpegts.
>
> So I replaced udpink and udpsrc with tcpclientsink and tcpserversrc and the
> problem goes away.
>
> What do you think about?
>
> Thanks
>
>
>
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: mpegts latency

TheGiamig
Michael Gruner wrote
> You are likely losing your H264 reference frame and the encoder is not
> generating another one. Try setting the encoders “key-int-max=30” (or some
> prudential value) to force periodical generation.
>
> Michael
> www.ridgerun.com
> _______________________________________________
> gstreamer-devel mailing list

> gstreamer-devel@.freedesktop

> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

I tried key-int-max parameter but no luck.
The problem seems to be some packet loss but I can't understand why, because
I'm using UDP on localhost.
May be there is a way to print statistics about send and receive packets on
udpsink/src elements.

Thanks




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: mpegts latency

Nicolas Dufresne-5
In reply to this post by TheGiamig


Le mer. 31 juill. 2019 08 h 25, TheGiamig <[hidden email]> a écrit :
Hi all,
I'm facing a latency problem using mpegtsmux and tsdemux elements (windows7,
gstreamer 1.15.1).

Please,don't use odd versions of GStreamer, these are delopment version. Use the release version instead, this way you are less likely to hit unknown bugs that only existed for a short period of time during development. 


I have to stream some h264 video content but my problem occurs also without
actually stream through UDP.

If I launch the following pipeline I can see the captured video without
latency problems (few ms):

gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=100 height=100 !
videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast
byte-stream=true ! queue ! h264parse ! avdec_h264 ! videoconvert !
autovideosink

But if I add mpegtsmux and tsdemux elements the video latency will be around
1sec:

tsdemux contributes 700ms of latency to the pipeline. We have added recently a latency property element to configure this. The latency depends on the PCR distance as configured in mpegtsmux. You need 2 PCR to start streaming, so the minimum possible latency is twice the period. Note that the configuration of mpegtsmux is based on a 9 kHz clock, so the value is N / 90000 second. Default is 3600, which means 40ms (so 80ms minimum latency).



gst-launch-1.0.exe dx9screencapsrc x=0 y=0 width=100 height=100 !
videoconvert ! queue ! x264enc tune=zerolatency speed-preset=ultrafast
byte-stream=true ! mpegtsmux ! queue ! tsdemux ! h264parse ! avdec_h264 !
videoconvert ! autovideosink

Any idea about this latency increment?

Thank you




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: mpegts latency

TheGiamig
Thanks Nicolas, you are right and I understand the origin of latency .

I faced also a problem of packets loss with udpsink/src, the problem goes
away usinf tcpsink/src.

So I replied to Michael about this topic:

Michael Gruner wrote
> You are likely losing your H264 reference frame and the encoder is not
> generating another one. Try setting the encoders “key-int-max=30” (or some
> prudential value) to force periodical generation.
>
> Michael
> www.ridgerun.com
> _______________________________________________
> gstreamer-devel mailing list

> gstreamer-devel@.freedesktop

> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


*I tried key-int-max parameter but no luck.
The problem seems to be some packet loss but I can't understand why, because
I'm using UDP on localhost.
May be there is a way to print statistics about send and receive packets on
udpsink/src elements.

Thanks *



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: mpegts latency

TheGiamig
Finally I solved the packet loss problem enlarging the socket buffer size of
udpsink/src (buffer-size=65535 is good for me).

Maybe the streaming was filling the buffers causing the packet loss.





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: mpegts latency

Nicolas Dufresne-5
Le lundi 16 septembre 2019 à 02:35 -0500, TheGiamig a écrit :
> Finally I solved the packet loss problem enlarging the socket buffer size of
> udpsink/src (buffer-size=65535 is good for me).
>
> Maybe the streaming was filling the buffers causing the packet loss.

The sender is likely "bursty", as your process has relatively low
priority, in such situation it might not be scheduled often enough to
avoid lost.

In some cases, adding a queue element after udpsrc can solve this (as
it allow udpsrc to avoid most of the push pack effect in GStreamer),
but increasing the buffer-size property is often more efficient (less
context switches).

>
>
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel