Frame drop when playing RTP stream using SDP file

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Frame drop when playing RTP stream using SDP file

Anuj Pahuja
Hi,

In my server-side application, I'm generating a raw video stream using appsrc and streaming it out on a udpsink which is a multicast address. So the pipeline conceptually looks like this:

gst-launch-1.0 appsrc ! videoconvert ! video/x-raw, height=576, width=720, format=UYVY ! rtpvrawpay ! udpsink host=224.1.1.1 port=5004

On the receiver's side, I can play this stream properly using this pipeline:

gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)640, height=(string)480, colorimetry=(string)BT601-5, payload=(int)96" ! rtpvrawdepay ! videoconvert ! autovideosink


I then converted my media information to an SDP file so that I can play the stream without specifying the caps every time. Here's the resultant SDP file:

v=0
o=- 21821257167 21821257167 IN IP4 192.168.1.4
s=Raw-stream
c=IN IP4 224.1.1.1
t=0 0
m=video 5004 RTP/AVP 96
a=rtpmap:96 RAW/90000
a=fmtp:96 media=video; sampling=YCbCr-4:2:2; depth=8; width=720; height=576; colorimetry=BT601-5

and I play this file using

gst-launch-1.0 -v filesrc location=stream.sdp ! sdpdemux ! rtpvrawdepay ! videoconvert ! autovideosink

The video stream plays but I'm getting a significant frame drop in the stream. Am I doing anything wrong? I tried adding queue elements in the above pipeline but there wasn't much change. Any suggestions would be much appreciated. Thanks.


Cheers,
Anuj

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Anuj Pahuja
On the receiver's side, I can play this stream properly using this pipeline:

gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)640, height=(string)480, colorimetry=(string)BT601-5, payload=(int)96" ! rtpvrawdepay ! videoconvert ! autovideosink

Minor correction:
gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)720, height=(string)576, colorimetry=(string)BT601-5, payload=(int)96" ! rtpvrawdepay ! videoconvert ! autovideosink



_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Sebastian Dröge-3
On Fr, 2016-05-06 at 11:16 +0530, Anuj Pahuja wrote:

> > On the receiver's side, I can play this stream properly using this
> > pipeline:
> >
> > gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-
> > rtp, media=(string)video, clock-rate=(int)90000, encoding-
> > name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8,
> > width=(string)640, height=(string)480, colorimetry=(string)BT601-5,
> > payload=(int)96" ! rtpvrawdepay ! videoconvert ! autovideosink
> >
> Minor correction:
> gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-
> rtp, media=(string)video, clock-rate=(int)90000, encoding-
> name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8,
> width=(string)720, height=(string)576, colorimetry=(string)BT601-5,
> payload=(int)96" ! rtpvrawdepay ! videoconvert ! autovideosink
How does it not work when using sdpdemux? Are packets getting lost, or
what exactly is the reason for the frame drops?

Also note that sdpdemux is using an rtpbin, which especially includes
an rtpjitterbuffer element. Does your manual pipeline fail in similar
ways if you add an rtpjitterbuffer before the depayloader?

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (968 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Anuj Pahuja

Also note that sdpdemux is using an rtpbin, which especially includes
an rtpjitterbuffer element. Does your manual pipeline fail in similar
ways if you add an rtpjitterbuffer before the depayloader?

I think that's the reason. Manual pipeline with rtpjitterbuffer gives the same results as playing using sdpdemux. Is there any way around it?


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Sebastian Dröge-3
On Fr, 2016-05-06 at 12:13 +0530, Anuj Pahuja wrote:
>
> > Also note that sdpdemux is using an rtpbin, which especially
> > includes
> > an rtpjitterbuffer element. Does your manual pipeline fail in
> > similar
> > ways if you add an rtpjitterbuffer before the depayloader?
> I think that's the reason. Manual pipeline with rtpjitterbuffer gives
> the same results as playing using sdpdemux. Is there any way around
> it? 

If your stream does not work with the rtpjitterbuffer, then something's
clearly wrong and that should be fixed. Especially for raw video
streams it is important to have an rtpjitterbuffer there.

So what exactly does not work? Do you get packet loss after the
jitterbuffer, or even before? You will get a lot of useful debug output
from rtpjitterbuffer and rtpbasedepayload here for finding the cause.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (968 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Anuj Pahuja
So what exactly does not work? Do you get packet loss after the
jitterbuffer, or even before? You will get a lot of useful debug output
from rtpjitterbuffer and rtpbasedepayload here for finding the cause.

I'm not sure what's not working, Could you tell me how can I enable additional debug output for rtpjitterbuffer and rtpvrawdepay? Is it using --gst-debug=rtpjitterbuffer in the pipeline?

Also, I get a better result when I use sync=false with rtpjitterbuffer. Then I get a result similar to the pipeline without rtpjitterbuffer.


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Sebastian Dröge-3
On Fr, 2016-05-06 at 15:31 +0530, Anuj Pahuja wrote:
> > So what exactly does not work? Do you get packet loss after the
> > jitterbuffer, or even before? You will get a lot of useful debug
> > output
> > from rtpjitterbuffer and rtpbasedepayload here for finding the
> > cause.
> I'm not sure what's not working, Could you tell me how can I enable
> additional debug output for rtpjitterbuffer and rtpvrawdepay? Is it
> using --gst-debug=rtpjitterbuffer in the pipeline? 

Setting GST_DEBUG=3,rtpjitterbuffer:6,rtpbasedepayload:6 in the
environment should give you something.

> Also, I get a better result when I use sync=false with
> rtpjitterbuffer. Then I get a result similar to the pipeline without
> rtpjitterbuffer.

That points at rtpjitterbuffer having problems producing correct
timestamps for your stream.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (968 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Anuj Pahuja
I tried using a V4L2 stream instead of a custom appsrc. I'm still facing the same issue (frame drop/video tearing) while using sdpdemux or rtpjitterbuffer with my receiver's pipeline. No issue when rtpjitterbuffer is not there. Here are the pipelines I'm using:

Sender side:
gst-launch-1.0 v4l2src device=/dev/video0 ! videoscale ! videoconvert ! video/x-raw, width=720, height=576, format=UYVY ! rtpvrawpay ! udpsink host=239.192.1.6 port=5004 -v

Receiver's side:
gst-launch-1.0 udpsrc address=239.192.1.6 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)720, height=(string)576, colorimetry=(string)BT601-5, payload=(int)96" ! rtpjitterbuffer ! rtpvrawdepay ! videoconvert ! autovideosink

As you said, there must be a problem in the stream if it isn't working with rtpjitterbuffer. Could you suggest what might be the possible issues on the sender's side here?


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Sebastian Dröge-3
On Mi, 2016-05-11 at 13:09 +0530, Anuj Pahuja wrote:

> I tried using a V4L2 stream instead of a custom appsrc. I'm still
> facing the same issue (frame drop/video tearing) while using sdpdemux
> or rtpjitterbuffer with my receiver's pipeline. No issue when
> rtpjitterbuffer is not there. Here are the pipelines I'm using:
>
> Sender side:
> gst-launch-1.0 v4l2src device=/dev/video0 ! videoscale ! videoconvert
> ! video/x-raw, width=720, height=576, format=UYVY ! rtpvrawpay !
> udpsink host=239.192.1.6 port=5004 -v
>
> Receiver's side:
> gst-launch-1.0 udpsrc address=239.192.1.6 port=5004 ! "application/x-
> rtp, media=(string)video, clock-rate=(int)90000, encoding-
> name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8,
> width=(string)720, height=(string)576, colorimetry=(string)BT601-5,
> payload=(int)96" ! rtpjitterbuffer ! rtpvrawdepay ! videoconvert !
> autovideosink
>
> As you said, there must be a problem in the stream if it isn't
> working with rtpjitterbuffer. Could you suggest what might be the
> possible issues on the sender's side here?
Can you also reproduce it with a videotestsrc, e.g. when using
  videotestsrc ! "video/x-raw,width=720,height=576" ! ...
?

The logs you get with GST_DEBUG=3,rtpjitterbufer:6 might also be
useful.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (968 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Anuj Pahuja

On Wed, May 11, 2016 at 1:54 PM, Sebastian Dröge <[hidden email]> wrote:
On Mi, 2016-05-11 at 13:09 +0530, Anuj Pahuja wrote:
> I tried using a V4L2 stream instead of a custom appsrc. I'm still
> facing the same issue (frame drop/video tearing) while using sdpdemux
> or rtpjitterbuffer with my receiver's pipeline. No issue when
> rtpjitterbuffer is not there. Here are the pipelines I'm using:
>
> Sender side:
> gst-launch-1.0 v4l2src device=/dev/video0 ! videoscale ! videoconvert
> ! video/x-raw, width=720, height=576, format=UYVY ! rtpvrawpay !
> udpsink host=239.192.1.6 port=5004 -v
>
> Receiver's side:
> gst-launch-1.0 udpsrc address=239.192.1.6 port=5004 ! "application/x-
> rtp, media=(string)video, clock-rate=(int)90000, encoding-
> name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8,
> width=(string)720, height=(string)576, colorimetry=(string)BT601-5,
> payload=(int)96" ! rtpjitterbuffer ! rtpvrawdepay ! videoconvert !
> autovideosink
>
> As you said, there must be a problem in the stream if it isn't
> working with rtpjitterbuffer. Could you suggest what might be the
> possible issues on the sender's side here?

Can you also reproduce it with a videotestsrc, e.g. when using
  videotestsrc ! "video/x-raw,width=720,height=576" ! ...
?

Yes, I can reproduce it with videotestsrc also.
 

The logs you get with GST_DEBUG=3,rtpjitterbufer:6 might also be
useful.

I got a lot of debug messages after enabling that. Here's a bit of that:

0:00:05.729438230  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 357, 1, 33662, 0:00:01.626756110
0:00:05.729438230  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 358, 1, 33663, 0:00:01.626756357
0:00:05.729446434  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 359, 1, 33664, 0:00:01.626756604
0:00:05.729454907  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 360, 1, 33665, 0:00:01.626756851
0:00:05.729463187  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 361, 1, 33666, 0:00:01.626757098
0:00:05.729471554  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 362, 1, 33667, 0:00:01.626757345
0:00:05.729479770  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 363, 1, 33668, 0:00:01.626757592
0:00:05.729488141  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3294:do_lost_timeout:<rtpjitterbuffer0> Packet #33656 lost
0:00:05.729500432  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:1948:remove_timer:<rtpjitterbuffer0> removed index 351
0:00:05.729509156  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3416:wait_next_timeout:<rtpjitterbuffer0> now 0:00:05.702634355
0:00:05.729517244  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 0, 1, 34019, 0:00:01.626844289
0:00:05.729525904  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3450:wait_next_timeout:<rtpjitterbuffer0> new best 0
0:00:05.729532770  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 1, 1, 34018, 0:00:01.626844042
0:00:05.729541166  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3450:wait_next_timeout:<rtpjitterbuffer0> new best 1
0:00:05.729547959  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 2, 1, 34017, 0:00:01.626843795
0:00:05.729556229  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3450:wait_next_timeout:<rtpjitterbuffer0> new best 2
0:00:05.729563019  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 3, 1, 34016, 0:00:01.626843548

I'm assuming it's losing some packets. Can some more sense be made out of the logs?


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Anuj Pahuja
I tried v4l2src once again with rtpjitterbuffer and it seems to work better now. I'm not sure why it wasn't working before. But the problem
 with playing sdp file using sdpdemux still persists. Video tearing and packet loss is still happening when playing using sdpdemux for a v4l2src.

On Wed, May 11, 2016 at 2:44 PM, Anuj Pahuja <[hidden email]> wrote:

On Wed, May 11, 2016 at 1:54 PM, Sebastian Dröge <[hidden email]> wrote:
On Mi, 2016-05-11 at 13:09 +0530, Anuj Pahuja wrote:
> I tried using a V4L2 stream instead of a custom appsrc. I'm still
> facing the same issue (frame drop/video tearing) while using sdpdemux
> or rtpjitterbuffer with my receiver's pipeline. No issue when
> rtpjitterbuffer is not there. Here are the pipelines I'm using:
>
> Sender side:
> gst-launch-1.0 v4l2src device=/dev/video0 ! videoscale ! videoconvert
> ! video/x-raw, width=720, height=576, format=UYVY ! rtpvrawpay !
> udpsink host=239.192.1.6 port=5004 -v
>
> Receiver's side:
> gst-launch-1.0 udpsrc address=239.192.1.6 port=5004 ! "application/x-
> rtp, media=(string)video, clock-rate=(int)90000, encoding-
> name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8,
> width=(string)720, height=(string)576, colorimetry=(string)BT601-5,
> payload=(int)96" ! rtpjitterbuffer ! rtpvrawdepay ! videoconvert !
> autovideosink
>
> As you said, there must be a problem in the stream if it isn't
> working with rtpjitterbuffer. Could you suggest what might be the
> possible issues on the sender's side here?

Can you also reproduce it with a videotestsrc, e.g. when using
  videotestsrc ! "video/x-raw,width=720,height=576" ! ...
?

Yes, I can reproduce it with videotestsrc also.
 

The logs you get with GST_DEBUG=3,rtpjitterbufer:6 might also be
useful.

I got a lot of debug messages after enabling that. Here's a bit of that:

0:00:05.729438230  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 357, 1, 33662, 0:00:01.626756110
0:00:05.729438230  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 358, 1, 33663, 0:00:01.626756357
0:00:05.729446434  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 359, 1, 33664, 0:00:01.626756604
0:00:05.729454907  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 360, 1, 33665, 0:00:01.626756851
0:00:05.729463187  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 361, 1, 33666, 0:00:01.626757098
0:00:05.729471554  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 362, 1, 33667, 0:00:01.626757345
0:00:05.729479770  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 363, 1, 33668, 0:00:01.626757592
0:00:05.729488141  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3294:do_lost_timeout:<rtpjitterbuffer0> Packet #33656 lost
0:00:05.729500432  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:1948:remove_timer:<rtpjitterbuffer0> removed index 351
0:00:05.729509156  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3416:wait_next_timeout:<rtpjitterbuffer0> now 0:00:05.702634355
0:00:05.729517244  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 0, 1, 34019, 0:00:01.626844289
0:00:05.729525904  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3450:wait_next_timeout:<rtpjitterbuffer0> new best 0
0:00:05.729532770  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 1, 1, 34018, 0:00:01.626844042
0:00:05.729541166  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3450:wait_next_timeout:<rtpjitterbuffer0> new best 1
0:00:05.729547959  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 2, 1, 34017, 0:00:01.626843795
0:00:05.729556229  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3450:wait_next_timeout:<rtpjitterbuffer0> new best 2
0:00:05.729563019  6615      0x1f8e0a0 DEBUG        rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 3, 1, 34016, 0:00:01.626843548

I'm assuming it's losing some packets. Can some more sense be made out of the logs?



_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Nicolas Dufresne-4
Le mercredi 11 mai 2016 à 15:14 +0530, Anuj Pahuja a écrit :
> I tried v4l2src once again with rtpjitterbuffer and it seems to work
> better now. I'm not sure why it wasn't working before. But the
> problem
>  with playing sdp file using sdpdemux still persists. Video tearing
> and packet loss is still happening when playing using sdpdemux for a
> v4l2src.

A plausible cause is that your sockets (udpsrc) have to small buffers
to be able to hold on a frame, while this frame is being processed. You
could try and increase the buffer-size property (it's by default 0,
which is system default, you can also increase you system default if
you prefer).

For the non-v4l2src case, you probably have timestamp issues.

Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Anuj Pahuja
A plausible cause is that your sockets (udpsrc) have to small buffers
to be able to hold on a frame, while this frame is being processed. You
could try and increase the buffer-size property (it's by default 0,
which is system default, you can also increase you system default if
you prefer).

For the non-v4l2src case, you probably have timestamp issues.

So I've tried most of the combinations now. I used a different machine as the client this time, earlier I was streaming out and in from the same machine, which doesn't work out correctly in case of multicast. I connected just two of them to a Gigabit router via GIgabit ethernet cables to ensure there's no packet loss due to network traffic. I used these pipelines on the source machine -

gst-launch-1.0 v4l2src device=/dev/video0 ! videoscale ! videoconvert ! video/x-raw, height=576, width=720, format=UYVY ! rtpvrawpay ! udpsink host=224.1.1.1 port=5004


gst-launch-1.0 filesrc location=test_video.mp4 ! decodebin !  videoconvert ! videoscale ! video/x-raw, height=576, width=720, format=UYVY ! rtpvrawpay ! udpsink host=224.1.1.1 port=5004

On the client machine, I played them using:

gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)720, height=(string)576, colorimetry=(string)BT601-5, payload=(int)96" ! rtpvrawdepay ! videoconvert ! autovideosink

(using rtpjitterbuffer also)
gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)720, height=(string)576, colorimetry=(string)BT601-5, payload=(int)96" ! rtpjitterbuffer ! rtpvrawdepay ! videoconvert ! autovideosink

Results:
1. When I don't use rtpjitterbuffer, there are no lacerations in the video or video tearing, but the sink displays a still frame in 70% of the window, and a continuous video stream on 30% of the sink window. I thought that was a sink issue and tested it out on a custom appsink, which converts it to RGB and renders it onto a Qt display. This improved the video a lot. I got a continuous stream with minor data loss which is tolerable I think in case of RTP/UDP, but I'm not entirely sure if even that should be there. This was the best result that I got.

2. There is a lot of video tearing (maybe because of data loss) using rtpjitterbuffer. It plays for some time but eventually stops the stream completely.

3. Then I tried increasing buffer-size of udpsrc to 100, 1000, 50000 but I'm getting green lines on most of the video.

Is there anything wrong in the pipelines I'm using? Is there a working example of multicast send and receive using rtpvrawpay/depay or using sdp file? Appreciate any help on this!

Thanks,
Anuj

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Nicolas Dufresne-4
I've looked into the payloader/depayload code, I'm not sure this is done correctly. First part is on the payloader, my quick read tells me it's splitting frames in 10 chunks rather then calculating the appropriate divider from the MTU size. This means the sender may endup producing large UDP packets that get split by the protocol. It also produce burst. On the receiver side, the depayloader does not expose it's latency. Unless the video frame fit into a single UDP packet, it's impossible to reconstruct that frame without inducing latency. The latency the number of chunk per frame - 1 (we usually round up to the number of chunk so we give the display a little more time to react). For the tearing and green bar at start, that would seem the result of missing packets at start and drops. I think the depayloader should just drop the packet until it gets the first line of the following frame. I doubt it's doing that atm. You will still get green lines whenever a packet was dropped, though drops should be rate if you get the burst part fixed.

Please, file those bugs to bugs.gnome.org on GStreamer product and gst-plugins-good component. Feel free to offer you own pathes by attaching them to the created bugs.

regards,
Nicolas

Le mercredi 18 mai 2016 à 11:51 +0530, Anuj Pahuja a écrit :
A plausible cause is that your sockets (udpsrc) have to small buffers
to be able to hold on a frame, while this frame is being processed. You
could try and increase the buffer-size property (it's by default 0,
which is system default, you can also increase you system default if
you prefer).

For the non-v4l2src case, you probably have timestamp issues.

So I've tried most of the combinations now. I used a different machine as the client this time, earlier I was streaming out and in from the same machine, which doesn't work out correctly in case of multicast. I connected just two of them to a Gigabit router via GIgabit ethernet cables to ensure there's no packet loss due to network traffic. I used these pipelines on the source machine -

gst-launch-1.0 v4l2src device=/dev/video0 ! videoscale ! videoconvert ! video/x-raw, height=576, width=720, format=UYVY ! rtpvrawpay ! udpsink host=224.1.1.1 port=5004


gst-launch-1.0 filesrc location=test_video.mp4 ! decodebin !  videoconvert ! videoscale ! video/x-raw, height=576, width=720, format=UYVY ! rtpvrawpay ! udpsink host=224.1.1.1 port=5004

On the client machine, I played them using:

gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)720, height=(string)576, colorimetry=(string)BT601-5, payload=(int)96" ! rtpvrawdepay ! videoconvert ! autovideosink

(using rtpjitterbuffer also)
gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)720, height=(string)576, colorimetry=(string)BT601-5, payload=(int)96" ! rtpjitterbuffer ! rtpvrawdepay ! videoconvert ! autovideosink

Results:
1. When I don't use rtpjitterbuffer, there are no lacerations in the video or video tearing, but the sink displays a still frame in 70% of the window, and a continuous video stream on 30% of the sink window. I thought that was a sink issue and tested it out on a custom appsink, which converts it to RGB and renders it onto a Qt display. This improved the video a lot. I got a continuous stream with minor data loss which is tolerable I think in case of RTP/UDP, but I'm not entirely sure if even that should be there. This was the best result that I got.

2. There is a lot of video tearing (maybe because of data loss) using rtpjitterbuffer. It plays for some time but eventually stops the stream completely.

3. Then I tried increasing buffer-size of udpsrc to 100, 1000, 50000 but I'm getting green lines on most of the video.

Is there anything wrong in the pipelines I'm using? Is there a working example of multicast send and receive using rtpvrawpay/depay or using sdp file? Appreciate any help on this!

Thanks,
Anuj
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Nicolas Dufresne-4
In reply to this post by Anuj Pahuja
I've looked into the payloader/depayload code, I'm not sure this is done correctly. First part is on the payloader, my quick read tells me it's splitting frames in 10 chunks rather then calculating the appropriate divider from the MTU size. This means the sender may endup producing large UDP packets that get split by the protocol. It also produce burst. On the receiver side, the depayloader does not expose it's latency. Unless the video frame fit into a single UDP packet, it's impossible to reconstruct that frame without inducing latency. The latency the number of chunk per frame - 1 (we usually round up to the number of chunk so we give the display a little more time to react). For the tearing and green bar at start, that would seem the result of missing packets at start and drops. I think the depayloader should just drop the packet until it gets the first line of the following frame. I doubt it's doing that atm. You will still get green lines whenever a packet was dropped, though drops should be rate if you get the burst part fixed.

Please, file those bugs to bugs.gnome.org on GStreamer product and gst-plugins-good component. Feel free to offer you own pathes by attaching them to the created bugs.

regards,
Nicolas

Le mercredi 18 mai 2016 à 11:51 +0530, Anuj Pahuja a écrit :
A plausible cause is that your sockets (udpsrc) have to small buffers
to be able to hold on a frame, while this frame is being processed. You
could try and increase the buffer-size property (it's by default 0,
which is system default, you can also increase you system default if
you prefer).

For the non-v4l2src case, you probably have timestamp issues.

So I've tried most of the combinations now. I used a different machine as the client this time, earlier I was streaming out and in from the same machine, which doesn't work out correctly in case of multicast. I connected just two of them to a Gigabit router via GIgabit ethernet cables to ensure there's no packet loss due to network traffic. I used these pipelines on the source machine -

gst-launch-1.0 v4l2src device=/dev/video0 ! videoscale ! videoconvert ! video/x-raw, height=576, width=720, format=UYVY ! rtpvrawpay ! udpsink host=224.1.1.1 port=5004


gst-launch-1.0 filesrc location=test_video.mp4 ! decodebin !  videoconvert ! videoscale ! video/x-raw, height=576, width=720, format=UYVY ! rtpvrawpay ! udpsink host=224.1.1.1 port=5004

On the client machine, I played them using:

gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)720, height=(string)576, colorimetry=(string)BT601-5, payload=(int)96" ! rtpvrawdepay ! videoconvert ! autovideosink

(using rtpjitterbuffer also)
gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)720, height=(string)576, colorimetry=(string)BT601-5, payload=(int)96" ! rtpjitterbuffer ! rtpvrawdepay ! videoconvert ! autovideosink

Results:
1. When I don't use rtpjitterbuffer, there are no lacerations in the video or video tearing, but the sink displays a still frame in 70% of the window, and a continuous video stream on 30% of the sink window. I thought that was a sink issue and tested it out on a custom appsink, which converts it to RGB and renders it onto a Qt display. This improved the video a lot. I got a continuous stream with minor data loss which is tolerable I think in case of RTP/UDP, but I'm not entirely sure if even that should be there. This was the best result that I got.

2. There is a lot of video tearing (maybe because of data loss) using rtpjitterbuffer. It plays for some time but eventually stops the stream completely.

3. Then I tried increasing buffer-size of udpsrc to 100, 1000, 50000 but I'm getting green lines on most of the video.

Is there anything wrong in the pipelines I'm using? Is there a working example of multicast send and receive using rtpvrawpay/depay or using sdp file? Appreciate any help on this!

Thanks,
Anuj
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Tim Müller
In reply to this post by Anuj Pahuja
On Wed, 2016-05-18 at 11:51 +0530, Anuj Pahuja wrote:

Hi,

> Results:
> 1. When I don't use rtpjitterbuffer, there are no lacerations in the
> video or video tearing, but the sink displays a still frame in 70% of
> the window, and a continuous video stream on 30% of the sink window.
> I thought that was a sink issue and tested it out on a custom
> appsink, which converts it to RGB and renders it onto a Qt display.
> This improved the video a lot. I got a continuous stream with minor
> data loss which is tolerable I think in case of RTP/UDP, but I'm not
> entirely sure if even that should be there. This was the best result
> that I got. 
>
> 2. There is a lot of video tearing (maybe because of data loss) using
> rtpjitterbuffer. It plays for some time but eventually stops the
> stream completely.
>
> 3. Then I tried increasing buffer-size of udpsrc to 100, 1000, 50000
> but I'm getting green lines on most of the video. 
>
> Is there anything wrong in the pipelines I'm using? Is there a
> working example of multicast send and receive using rtpvrawpay/depay
> or using sdp file? Appreciate any help on this!

These buffer-size values are waaaay too small. You want them as big as
possible, try multiple MB.

You may also need to do something like this:

sudo /sbin/sysctl -w net.core.rmem_max=33554432
sudo /sbin/sysctl -w net.core.wmem_max=33554432

to allow larger values for the udp send/receive buffers.

On the sender side I would also recommend a queue between payloader and
udpsink, and v4l2src and the converters/payloader.

On the receiver side I would also recommend a queue right after udpsrc
if there's no jitterbuffer (make it unlimited with queue max-size-
bytes=0 max-size-buffers=0 max-size-time=0).

Cheers
 -Tim

-- 
Tim Müller, Centricular Ltd - http://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Frame drop when playing RTP stream using SDP file

Tim Müller
In reply to this post by Nicolas Dufresne-4
On Wed, 2016-05-18 at 08:54 -0400, Nicolas Dufresne wrote:

> I've looked into the payloader/depayload code, I'm not sure this is
> done correctly. First part is on the payloader, my quick read tells
> me it's splitting frames in 10 chunks rather then calculating the
> appropriate divider from the MTU size. This means the sender may
> endup producing large UDP packets that get split by the protocol. It
> also produce burst. On the receiver side, the depayloader does not
> expose it's latency. Unless the video frame fit into a single UDP
> packet, it's impossible to reconstruct that frame without inducing
> latency. The latency the number of chunk per frame - 1 (we usually
> round up to the number of chunk so we give the display a little more
> time to react). For the tearing and green bar at start, that would
> seem the result of missing packets at start and drops. I think the
> depayloader should just drop the packet until it gets the first line
> of the following frame. I doubt it's doing that atm. You will still
> get green lines whenever a packet was dropped, though drops should be
> rate if you get the burst part fixed.

In case of rtpvrawpay, a raw video frame might be split up into many
thousand RTP/UDP packets.

What the payloader is trying to do is not split up the frame as a whole
into N thousand packets, and then send those all at once by pushing
them to the udpsink in one go, but rather payload a part of the frame,
then send those RTP/UDP packets already to udpsink to send out, then go
and payload the next part of the frame, etc.

This should reduce burstiness and latency.

The behaviour can also be configured with the "chunks-per-frame"
property.

Cheers
 -Tim


--
Tim Müller, Centricular Ltd - http://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel