Hi, In my server-side application, I'm generating a raw video stream using appsrc and streaming it out on a udpsink which is a multicast address. So the pipeline conceptually looks like this:gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)640, height=(string)480, colorimetry=(string)BT601-5, payload=(int)96" ! rtpvrawdepay ! videoconvert ! autovideosink I then converted my media information to an SDP file so that I can play the stream without specifying the caps every time. Here's the resultant SDP file: v=0 o=- 21821257167 21821257167 IN IP4 192.168.1.4 s=Raw-stream c=IN IP4 224.1.1.1 t=0 0 m=video 5004 RTP/AVP 96 a=rtpmap:96 RAW/90000 a=fmtp:96 media=video; sampling=YCbCr-4:2:2; depth=8; width=720; height=576; colorimetry=BT601-5 and I play this file using gst-launch-1.0 -v filesrc location=stream.sdp ! sdpdemux ! rtpvrawdepay ! videoconvert ! autovideosink The video stream plays but I'm getting a significant frame drop in the stream. Am I doing anything wrong? I tried adding queue elements in the above pipeline but there wasn't much change. Any suggestions would be much appreciated. Thanks. Cheers, Anuj _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Minor correction: gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)720, height=(string)576, colorimetry=(string)BT601-5, payload=(int)96" ! rtpvrawdepay ! videoconvert ! autovideosink _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
On Fr, 2016-05-06 at 11:16 +0530, Anuj Pahuja wrote:
> > On the receiver's side, I can play this stream properly using this > > pipeline: > > > > gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x- > > rtp, media=(string)video, clock-rate=(int)90000, encoding- > > name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, > > width=(string)640, height=(string)480, colorimetry=(string)BT601-5, > > payload=(int)96" ! rtpvrawdepay ! videoconvert ! autovideosink > > > Minor correction: > gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x- > rtp, media=(string)video, clock-rate=(int)90000, encoding- > name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, > width=(string)720, height=(string)576, colorimetry=(string)BT601-5, > payload=(int)96" ! rtpvrawdepay ! videoconvert ! autovideosink what exactly is the reason for the frame drops? Also note that sdpdemux is using an rtpbin, which especially includes an rtpjitterbuffer element. Does your manual pipeline fail in similar ways if you add an rtpjitterbuffer before the depayloader? -- Sebastian Dröge, Centricular Ltd · http://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel signature.asc (968 bytes) Download Attachment |
Also note that sdpdemux is using an rtpbin, which especially includes I think that's the reason. Manual pipeline with rtpjitterbuffer gives the same results as playing using sdpdemux. Is there any way around it? _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
On Fr, 2016-05-06 at 12:13 +0530, Anuj Pahuja wrote:
> > > Also note that sdpdemux is using an rtpbin, which especially > > includes > > an rtpjitterbuffer element. Does your manual pipeline fail in > > similar > > ways if you add an rtpjitterbuffer before the depayloader? > I think that's the reason. Manual pipeline with rtpjitterbuffer gives > the same results as playing using sdpdemux. Is there any way around > it? If your stream does not work with the rtpjitterbuffer, then something's clearly wrong and that should be fixed. Especially for raw video streams it is important to have an rtpjitterbuffer there. So what exactly does not work? Do you get packet loss after the jitterbuffer, or even before? You will get a lot of useful debug output from rtpjitterbuffer and rtpbasedepayload here for finding the cause. -- Sebastian Dröge, Centricular Ltd · http://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel signature.asc (968 bytes) Download Attachment |
So what exactly does not work? Do you get packet loss after the I'm not sure what's not working, Could you tell me how can I enable additional debug output for rtpjitterbuffer and rtpvrawdepay? Is it using --gst-debug=rtpjitterbuffer in the pipeline? Also, I get a better result when I use sync=false with rtpjitterbuffer. Then I get a result similar to the pipeline without rtpjitterbuffer. _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
On Fr, 2016-05-06 at 15:31 +0530, Anuj Pahuja wrote:
> > So what exactly does not work? Do you get packet loss after the > > jitterbuffer, or even before? You will get a lot of useful debug > > output > > from rtpjitterbuffer and rtpbasedepayload here for finding the > > cause. > I'm not sure what's not working, Could you tell me how can I enable > additional debug output for rtpjitterbuffer and rtpvrawdepay? Is it > using --gst-debug=rtpjitterbuffer in the pipeline? Setting GST_DEBUG=3,rtpjitterbuffer:6,rtpbasedepayload:6 in the environment should give you something. > Also, I get a better result when I use sync=false with > rtpjitterbuffer. Then I get a result similar to the pipeline without > rtpjitterbuffer. That points at rtpjitterbuffer having problems producing correct timestamps for your stream. -- Sebastian Dröge, Centricular Ltd · http://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel signature.asc (968 bytes) Download Attachment |
I tried using a V4L2 stream instead of a custom appsrc. I'm still facing the same issue (frame drop/video tearing) while using sdpdemux or rtpjitterbuffer with my receiver's pipeline. No issue when rtpjitterbuffer is not there. Here are the pipelines I'm using: Sender side:gst-launch-1.0 v4l2src device=/dev/video0 ! videoscale ! videoconvert ! video/x-raw, width=720, height=576, format=UYVY ! rtpvrawpay ! udpsink host=239.192.1.6 port=5004 -v gst-launch-1.0 udpsrc address=239.192.1.6 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)720, height=(string)576, colorimetry=(string)BT601-5, payload=(int)96" ! rtpjitterbuffer ! rtpvrawdepay ! videoconvert ! autovideosink As you said, there must be a problem in the stream if it isn't working with rtpjitterbuffer. Could you suggest what might be the possible issues on the sender's side here? _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
On Mi, 2016-05-11 at 13:09 +0530, Anuj Pahuja wrote:
> I tried using a V4L2 stream instead of a custom appsrc. I'm still > facing the same issue (frame drop/video tearing) while using sdpdemux > or rtpjitterbuffer with my receiver's pipeline. No issue when > rtpjitterbuffer is not there. Here are the pipelines I'm using: > > Sender side: > gst-launch-1.0 v4l2src device=/dev/video0 ! videoscale ! videoconvert > ! video/x-raw, width=720, height=576, format=UYVY ! rtpvrawpay ! > udpsink host=239.192.1.6 port=5004 -v > > Receiver's side: > gst-launch-1.0 udpsrc address=239.192.1.6 port=5004 ! "application/x- > rtp, media=(string)video, clock-rate=(int)90000, encoding- > name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, > width=(string)720, height=(string)576, colorimetry=(string)BT601-5, > payload=(int)96" ! rtpjitterbuffer ! rtpvrawdepay ! videoconvert ! > autovideosink > > As you said, there must be a problem in the stream if it isn't > working with rtpjitterbuffer. Could you suggest what might be the > possible issues on the sender's side here? videotestsrc ! "video/x-raw,width=720,height=576" ! ... ? The logs you get with GST_DEBUG=3,rtpjitterbufer:6 might also be useful. -- Sebastian Dröge, Centricular Ltd · http://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel signature.asc (968 bytes) Download Attachment |
On Wed, May 11, 2016 at 1:54 PM, Sebastian Dröge <[hidden email]> wrote: On Mi, 2016-05-11 at 13:09 +0530, Anuj Pahuja wrote: Yes, I can reproduce it with videotestsrc also.
I got a lot of debug messages after enabling that. Here's a bit of that: 0:00:05.729438230 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 357, 1, 33662, 0:00:01.626756110 0:00:05.729438230 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 358, 1, 33663, 0:00:01.626756357 0:00:05.729446434 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 359, 1, 33664, 0:00:01.626756604 0:00:05.729454907 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 360, 1, 33665, 0:00:01.626756851 0:00:05.729463187 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 361, 1, 33666, 0:00:01.626757098 0:00:05.729471554 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 362, 1, 33667, 0:00:01.626757345 0:00:05.729479770 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 363, 1, 33668, 0:00:01.626757592 0:00:05.729488141 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3294:do_lost_timeout:<rtpjitterbuffer0> Packet #33656 lost 0:00:05.729500432 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:1948:remove_timer:<rtpjitterbuffer0> removed index 351 0:00:05.729509156 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3416:wait_next_timeout:<rtpjitterbuffer0> now 0:00:05.702634355 0:00:05.729517244 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 0, 1, 34019, 0:00:01.626844289 0:00:05.729525904 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3450:wait_next_timeout:<rtpjitterbuffer0> new best 0 0:00:05.729532770 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 1, 1, 34018, 0:00:01.626844042 0:00:05.729541166 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3450:wait_next_timeout:<rtpjitterbuffer0> new best 1 0:00:05.729547959 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 2, 1, 34017, 0:00:01.626843795 0:00:05.729556229 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3450:wait_next_timeout:<rtpjitterbuffer0> new best 2 0:00:05.729563019 6615 0x1f8e0a0 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3425:wait_next_timeout:<rtpjitterbuffer0> 3, 1, 34016, 0:00:01.626843548 I'm assuming it's losing some packets. Can some more sense be made out of the logs? _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
I tried v4l2src once again with rtpjitterbuffer and it seems to work better now. I'm not sure why it wasn't working before. But the problem with playing sdp file using sdpdemux still persists. Video tearing and packet loss is still happening when playing using sdpdemux for a v4l2src. On Wed, May 11, 2016 at 2:44 PM, Anuj Pahuja <[hidden email]> wrote:
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Le mercredi 11 mai 2016 à 15:14 +0530, Anuj Pahuja a écrit :
> I tried v4l2src once again with rtpjitterbuffer and it seems to work > better now. I'm not sure why it wasn't working before. But the > problem > with playing sdp file using sdpdemux still persists. Video tearing > and packet loss is still happening when playing using sdpdemux for a > v4l2src. A plausible cause is that your sockets (udpsrc) have to small buffers to be able to hold on a frame, while this frame is being processed. You could try and increase the buffer-size property (it's by default 0, which is system default, you can also increase you system default if you prefer). For the non-v4l2src case, you probably have timestamp issues. Nicolas _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
A plausible cause is that your sockets (udpsrc) have to small buffers So I've tried most of the combinations now. I used a different machine as the client this time, earlier I was streaming out and in from the same machine, which doesn't work out correctly in case of multicast. I connected just two of them to a Gigabit router via GIgabit ethernet cables to ensure there's no packet loss due to network traffic. I used these pipelines on the source machine - gst-launch-1.0 v4l2src device=/dev/video0 ! videoscale ! videoconvert ! video/x-raw, height=576, width=720, format=UYVY ! rtpvrawpay ! udpsink host=224.1.1.1 port=5004 gst-launch-1.0 filesrc location=test_video.mp4 ! decodebin ! videoconvert ! videoscale ! video/x-raw, height=576, width=720, format=UYVY ! rtpvrawpay ! udpsink host=224.1.1.1 port=5004 gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)720, height=(string)576, colorimetry=(string)BT601-5, payload=(int)96" ! rtpvrawdepay ! videoconvert ! autovideosink (using rtpjitterbuffer also) gst-launch-1.0 udpsrc address=224.1.1.1 port=5004 ! "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)RAW, sampling=(string)YCbCr-4:2:2, depth=(string)8, width=(string)720, height=(string)576, colorimetry=(string)BT601-5, payload=(int)96" ! rtpjitterbuffer ! rtpvrawdepay ! videoconvert ! autovideosink Results: 1. When I don't use rtpjitterbuffer, there are no lacerations in the video or video tearing, but the sink displays a still frame in 70% of the window, and a continuous video stream on 30% of the sink window. I thought that was a sink issue and tested it out on a custom appsink, which converts it to RGB and renders it onto a Qt display. This improved the video a lot. I got a continuous stream with minor data loss which is tolerable I think in case of RTP/UDP, but I'm not entirely sure if even that should be there. This was the best result that I got. 2. There is a lot of video tearing (maybe because of data loss) using rtpjitterbuffer. It plays for some time but eventually stops the stream completely. 3. Then I tried increasing buffer-size of udpsrc to 100, 1000, 50000 but I'm getting green lines on most of the video. Is there anything wrong in the pipelines I'm using? Is there a working example of multicast send and receive using rtpvrawpay/depay or using sdp file? Appreciate any help on this! Thanks, Anuj _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
I've looked into the payloader/depayload code, I'm not sure this is done correctly. First part is on the payloader, my quick read tells me it's splitting frames in 10 chunks rather then calculating the appropriate divider from the MTU size. This means the sender may endup producing large UDP packets that get split by the protocol. It also produce burst. On the receiver side, the depayloader does not expose it's latency. Unless the video frame fit into a single UDP packet, it's impossible to reconstruct that frame without inducing latency. The latency the number of chunk per frame - 1 (we usually round up to the number of chunk so we give the display a little more time to react). For the tearing and green bar at start, that would seem the result of missing packets at start and drops. I think the depayloader should just drop the packet until it gets the first line of the following frame. I doubt it's doing that atm. You will still get green lines whenever a packet was dropped, though drops should be rate if you get the burst part fixed. Please, file those bugs to bugs.gnome.org on GStreamer product and gst-plugins-good component. Feel free to offer you own pathes by attaching them to the created bugs. regards, Nicolas Le mercredi 18 mai 2016 à 11:51 +0530, Anuj Pahuja a écrit :
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
In reply to this post by Anuj Pahuja
I've looked into the payloader/depayload code, I'm not sure this is done correctly. First part is on the payloader, my quick read tells me it's splitting frames in 10 chunks rather then calculating the appropriate divider from the MTU size. This means the sender may endup producing large UDP packets that get split by the protocol. It also produce burst. On the receiver side, the depayloader does not expose it's latency. Unless the video frame fit into a single UDP packet, it's impossible to reconstruct that frame without inducing latency. The latency the number of chunk per frame - 1 (we usually round up to the number of chunk so we give the display a little more time to react). For the tearing and green bar at start, that would seem the result of missing packets at start and drops. I think the depayloader should just drop the packet until it gets the first line of the following frame. I doubt it's doing that atm. You will still get green lines whenever a packet was dropped, though drops should be rate if you get the burst part fixed. Please, file those bugs to bugs.gnome.org on GStreamer product and gst-plugins-good component. Feel free to offer you own pathes by attaching them to the created bugs. regards, Nicolas Le mercredi 18 mai 2016 à 11:51 +0530, Anuj Pahuja a écrit :
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
In reply to this post by Anuj Pahuja
On Wed, 2016-05-18 at 11:51 +0530, Anuj Pahuja wrote:
Hi, > Results: > 1. When I don't use rtpjitterbuffer, there are no lacerations in the > video or video tearing, but the sink displays a still frame in 70% of > the window, and a continuous video stream on 30% of the sink window. > I thought that was a sink issue and tested it out on a custom > appsink, which converts it to RGB and renders it onto a Qt display. > This improved the video a lot. I got a continuous stream with minor > data loss which is tolerable I think in case of RTP/UDP, but I'm not > entirely sure if even that should be there. This was the best result > that I got. > > 2. There is a lot of video tearing (maybe because of data loss) using > rtpjitterbuffer. It plays for some time but eventually stops the > stream completely. > > 3. Then I tried increasing buffer-size of udpsrc to 100, 1000, 50000 > but I'm getting green lines on most of the video. > > Is there anything wrong in the pipelines I'm using? Is there a > working example of multicast send and receive using rtpvrawpay/depay > or using sdp file? Appreciate any help on this! These buffer-size values are waaaay too small. You want them as big as possible, try multiple MB. You may also need to do something like this: sudo /sbin/sysctl -w net.core.rmem_max=33554432 sudo /sbin/sysctl -w net.core.wmem_max=33554432 to allow larger values for the udp send/receive buffers. On the sender side I would also recommend a queue between payloader and udpsink, and v4l2src and the converters/payloader. On the receiver side I would also recommend a queue right after udpsrc if there's no jitterbuffer (make it unlimited with queue max-size- bytes=0 max-size-buffers=0 max-size-time=0). Cheers -Tim -- Tim Müller, Centricular Ltd - http://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
In reply to this post by Nicolas Dufresne-4
On Wed, 2016-05-18 at 08:54 -0400, Nicolas Dufresne wrote:
> I've looked into the payloader/depayload code, I'm not sure this is > done correctly. First part is on the payloader, my quick read tells > me it's splitting frames in 10 chunks rather then calculating the > appropriate divider from the MTU size. This means the sender may > endup producing large UDP packets that get split by the protocol. It > also produce burst. On the receiver side, the depayloader does not > expose it's latency. Unless the video frame fit into a single UDP > packet, it's impossible to reconstruct that frame without inducing > latency. The latency the number of chunk per frame - 1 (we usually > round up to the number of chunk so we give the display a little more > time to react). For the tearing and green bar at start, that would > seem the result of missing packets at start and drops. I think the > depayloader should just drop the packet until it gets the first line > of the following frame. I doubt it's doing that atm. You will still > get green lines whenever a packet was dropped, though drops should be > rate if you get the burst part fixed. In case of rtpvrawpay, a raw video frame might be split up into many thousand RTP/UDP packets. What the payloader is trying to do is not split up the frame as a whole into N thousand packets, and then send those all at once by pushing them to the udpsink in one go, but rather payload a part of the frame, then send those RTP/UDP packets already to udpsink to send out, then go and payload the next part of the frame, etc. This should reduce burstiness and latency. The behaviour can also be configured with the "chunks-per-frame" property. Cheers -Tim -- Tim Müller, Centricular Ltd - http://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Free forum by Nabble | Edit this page |