video interrupted,timestamping issue?

[Katerina Voulgary-2]

Hello everyone,

I have these pipelines and video doesn't stream smoothly to a webrtc client:

Sender pipeline in python(goes through opencv):

         self._videoRGBString = "v4l2src device=/dev/video9 !
video/x-raw,format=YUY2, width=1920, height=1080,framerate=15/1 !
videoconvert ! queue ! video/x-raw,format=BGR ! appsink "


         self._writerRGBString = "appsrc caps=video/x-raw,format=BGR \
                                 ! queue ! videoconvert !
video/x-raw,format=I420,width=1920,height=1080,framerate=15/1  \
                                 ! queue ! omxh264enc profile=baseline
control-rate=1 qp-range=35,45:35,45:35,45 peak-bitrate=1500000
preset-level=0 \
                                 ! video/x-h264,
stream-format=byte-stream, alignment=au,
width=1920,height=1080,framerate=15/1\
                                 ! queue ! h264parse ! video/x-h264,
alignment=au, stream-format=byte-stream,
width=1920,height=1080,framerate=15/1\
                                 ! queue ! rtph264pay config-interval=-1
! queue ! udpsink "+ self._config['janus']['url_RGB'] +"  \
                                  async=false"

cv2.VideoWriter(self._writerRGBString,0,15.0,(1920,1080))


Receiver pipeline:

     gst-launch-1.0 udpsrc port=8006 !
application/x-rtp,encoding-name=H264,payload=96 ! rtpjitterbuffer mode=1
! rtph264depay ! decodebin ! videoconvert ! glimagesink

Some logs on the receiver:

0:03:54.409415392 2343207 0x5561344bb860 WARN rtpjitterbuffer
rtpjitterbuffer.c:749:rtp_jitter_buffer_calculate_pts: backward
timestamps at server, schedule resync
0:03:54.409648475 2343207 0x5561344bb860 INFO rtpjitterbuffer
rtpjitterbuffer.c:780:rtp_jitter_buffer_calculate_pts: resync to time
0:03:54.290964724, rtptime 9:57:44.938222222
0:03:55.516084124 2343207 0x5561344bb860 WARN rtpjitterbuffer
rtpjitterbuffer.c:572:calculate_skew: delta - skew: 0:00:01.040029654
too big, reset skew

The video stream has to go to a JanusWebrtc server. The stream
frequently stops(we see no image or a still frame depending on the
browser) like it is missing data. I think it is a timestamp issue that
happens because of the latency(4" to the screen) that the processing
introduces and the webrtc server cannot handle. The sender machine will
never have a correct utc time if it is of interest.

Can I get an insight of what is missing to run smoothly in the client? I
can give any other logs/info as needed.

The machine is a jetson xavier with  GStreamer 1.14.5.


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[gotsring]

Just guessing here, but did you try setting buffer timestamps before pushing
the buffers through the appsrc? Probably just pts is needed, see docs here:
https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=python#GstBuffer




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Katerina Voulgary-2]

Hello,

the only evidence I have is the interrupted video on the webrtc client
and it is random.

The processing of the frames is not consistent, so appsrc is not
producing frames at a stable framerate.

I changed the pipeline to "appsrc
caps=video/x-raw,format=BGR,format=GST_FORMAT_TIME,do-timestamp=true,stream-type=0,is-live=true",
I suppose that sets the pts? I also added a videorate element. Still it
happens.

Would anyone know what could cause these interrupts and how to debug?

Sender pipeline:


         self._videoRGBString = "v4l2src device=/dev/video9 !
video/x-raw,format=YUY2, width=1920, height=1080,framerate=15/1 !
videoconvert ! queue ! video/x-raw,format=BGR ! appsink "


         self._writerRGBString = "appsrc
caps=video/x-raw,format=BGR,do-timestamp=true,stream-type=0 \
                                 ! queue ! videoconvert !  videorate !
video/x-raw,format=I420,width=1920,height=1080,framerate=15/1  \
                                 ! queue ! omxh264enc profile=baseline
control-rate=1 qp-range=35,45:35,45:35,45 peak-bitrate=1500000
preset-level=0 \
                                 ! video/x-h264,
stream-format=byte-stream, alignment=au,
width=1920,height=1080,framerate=15/1\
                                 ! queue ! h264parse ! video/x-h264,
alignment=au, stream-format=byte-stream,
width=1920,height=1080,framerate=15/1\
                                 ! queue ! rtph264pay config-interval=-1
! queue ! udpsink "+ self._config['janus']['url_RGB'] +"  \
                                  async=false"

cv2.VideoWriter(self._writerRGBString,0,15.0,(1920,1080))


Receiver pipeline:

     gst-launch-1.0 udpsrc port=8006 !
application/x-rtp,encoding-name=H264,payload=96 ! rtpjitterbuffer mode=1
! rtph264depay ! decodebin ! videoconvert ! fakesink dump=1

Στις 26/1/21 5:31 μ.μ., o/η gotsring έγραψε:

> Just guessing here, but did you try setting buffer timestamps before pushing
> the buffers through the appsrc? Probably just pts is needed, see docs here:
> https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html?gi-language=python#GstBuffer
>
>
>
>
> --
> Sent from:http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel