RTP video stream corruption
Locked
 
|
Hello, I have a pipeline that looks like this: rtspsrc ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! video/x-raw,format=RGBA ! appsink which I use to receive a 4k video stream from a camera on the LAN (or to test - a stream from localhost). Occasionally, I see that the video stream gets 'corrupted' - moving objects get smeared & then sometimes jump back/forth in time momentarily. It looks like perhaps some I-frame information gets dropped & the h264 decoder just does the best it can until the next I-frame. This would be okay for something like a video-conferencing solution, but my application takes decoded frames & does object recognition & temporal tracking, and the smearing & jumping really messes with this downstream processing - it would be much better to have no frames at all than these 'corrupt' ones. I have tried all the properties & knobs that I know of w.r.t rtspsrc ( udp-buffer-size, latency, drop-on-latency, buffer-mode, retransmissions), and have tried adding queues in the pipeline, etc.. No matter what I do, this occasionally happens when there is any other significant usage of cpu (for example a multi-job compilation). I do know that I have enough CPU time 'on-average' to keep up with the stream rate, so it seems like with correct buffering in the right places, I shouldn't have to drop any information. I have three questions: 1) Is there some way to configure my gstreamer pipeline so that I don't drop this information (I suspect it is the rtsp jitterbuffer dropping, but I'm not sure), and therefore don't have any problem. 2) If I can't eliminate this via #1, can I somehow configure different information to be dropped (essentially p-frames, but not i-frames). 3) If not #1 or #2, is there a way I can detect downstream from the video-decoder that the frame is 'corrupt' - I think the decoder should know when something is missing from the h264 stream, right? Thanks in advance to anyone who can help shed some light on this issue. -Joel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
On Wed, 2018-01-10 at 14:27 -0500, Joel Keller wrote:
Hi Joel, > which I use to receive a 4k video stream from a camera on the LAN (or > to test - a stream from localhost). Occasionally, I see that the > video stream gets 'corrupted' (...) > > I have tried all the properties & knobs that I know of w.r.t rtspsrc > ( udp-buffer-size, latency, drop-on-latency, buffer-mode, > retransmissions), and have tried adding queues in the pipeline, > etc.. No matter what I do, this occasionally happens when there is > any other significant usage of cpu (for example a multi-job > compilation). I do know that I have enough CPU time 'on-average' to > keep up with the stream rate, so it seems like with correct buffering > in the right places, I shouldn't have to drop any information. Have you also checked whether these parameters were actually effective? e.g. udp buffer-size might be restricted and you might need something like sudo /sbin/sysctl -w net.core.rmem_max=33554432 to up the max. Did you check if the camera supports interleaved RTSP so that the data is sent over the TCP connection instead via UDP? That might be the easiest workaround in your case. You can try by using rtspsrc location=rtspt://... instead of rtsp:// One would have to check if the omx decoder supports not outputting corrupted frames. Cheers -Tim -- Tim Müller, Centricular Ltd - http://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
AW: RTP video stream corruption
|
|
In reply to this post by Joel Keller
Hi, Does the h264 source contain b-frames? It could be a time-stamping issue between DTS and PTS if b frames are involved Von: gstreamer-devel [mailto:[hidden email]]
Im Auftrag von Joel Keller Hello, I have a pipeline that looks like this: rtspsrc ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! video/x-raw,format=RGBA ! appsink which I use to receive a 4k video stream from a camera on the LAN (or to test - a stream from localhost). Occasionally, I see that the video stream gets 'corrupted' - moving objects get smeared & then sometimes jump back/forth in time
momentarily. It looks like perhaps some I-frame information gets dropped & the h264 decoder just does the best it can until the next I-frame. This would be okay for something like a video-conferencing solution, but my application takes decoded frames & does
object recognition & temporal tracking, and the smearing & jumping really messes with this downstream processing - it would be much better to have no frames at all than these 'corrupt' ones. I have tried all the properties & knobs that I know of w.r.t rtspsrc ( udp-buffer-size, latency, drop-on-latency, buffer-mode, retransmissions), and have tried adding queues in the pipeline, etc.. No matter what I do, this occasionally
happens when there is any other significant usage of cpu (for example a multi-job compilation). I do know that I have enough CPU time 'on-average' to keep up with the stream rate, so it seems like with correct buffering in the right places, I shouldn't have
to drop any information. I have three questions: 1) Is there some way to configure my gstreamer pipeline so that I don't drop this information (I suspect it is the rtsp jitterbuffer dropping, but I'm not sure), and therefore don't have any problem. 2) If I can't eliminate this via #1, can I somehow configure different information to be dropped (essentially p-frames, but not i-frames). 3) If not #1 or #2, is there a way I can detect downstream from the video-decoder that the frame is 'corrupt' - I think the decoder should know when something is missing from the h264 stream, right? Thanks in advance to anyone who can help shed some light on this issue. -Joel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
| Free forum by Nabble | Edit this page |