GStreamer-devel

Only first frame drawn gst-rtsp-server

classic Classic list List threaded Threaded
♦
♦
 3 messages Options
Shishigami
Reply | Threaded
Open this post in threaded view
♦
♦
|

Only first frame drawn gst-rtsp-server

Shishigami
I'm on Windows 10 using Gstreamer 1.16.1.

I'm trying to run two pipelines in parallel. Sender gets data from appsrc
(OpenCV) and writes them to UDP sink:

appsrc is-live=true format=time ! videoconvert ! video/x-raw, format=I420 !
x264enc speed-preset=superfast tune=zerolatency byte-stream=true
key-int-max=15 ! rtph264pay pt=96 config-interval=5 ! udpsink sync=false
host=localhost port=8553

Receiver reads from UDP source and uses gst-rtsp-server to provide an RTSP
stream:

udpsrc address=localhost port=8553 caps="application/x-rtp,
clock-rate=90000, payload=96" ! rtpjitterbuffer ! rtph264depay ! rtph264pay
name=pay0 pt=96

Opening the stream in VLC media player, the resulting frame from OpenCV does
show up, but it never changes. In Unreal Engine (target environment) the
stream opens fine, but it only shows a gray image.

I'm not sure if this is related, but I set GST_DEBUG and get the following
output from gst-rstp-server:

0:00:03.329599300  6124 0000024F68D88700 FIXME              rtspmedia
rtsp-media.c:4168:gst_rtsp_media_suspend: suspend for dynamic pipelines
needs fixing
0:00:03.329990500  6124 0000024F68D88700 WARN               rtspmedia
rtsp-media.c:4194:gst_rtsp_media_suspend: media 0000024F68DB3220 was not
prepared
0:00:03.339047900  6124 0000024F68D88700 FIXME             rtspclient
rtsp-client.c:1818:handle_play_request:<GstRTSPClient@0000024F68D89850> Add
support for seek style (null)
0:00:03.339776300  6124 0000024F68D88700 FIXME              rtspmedia
rtsp-media.c:2684:gst_rtsp_media_seek_full:<GstRTSPMedia@0000024F68DB3220>
Handle going back to 0 for none live not seekable streams.

I tried using fpsdisplaysink with the following pipeline (instead of
receiver) and that works:

gst-launch-1.0 -v udpsrc address=localhost port=8553
caps="application/x-rtp,clock-rate=90000,payload=96" ! rtpjitterbuffer !
rtph264depay ! decodebin ! videoconvert ! fpsdisplaysink

But the FPS is pretty bad, averaging at 2.29, which is a far cry from the 30
FPS I'm looking to achieve:
last-message = rendered: 240, dropped: 0, current: 2.25, average: 2.29

Why does it work with fpsdisplaysink but not RTSP? I'd highly highly
appreciate any help!



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Michael MacIntosh
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Only first frame drawn gst-rtsp-server

Michael MacIntosh

Generally when i have issues where only the first frame gets drawn, it is a time stamping issue.

The pipeline will preroll and pull the first frame, but then wait for data to match the current time clock.  The buffers you are supplying to the appsrc should be timestamped.  The videorate plugin might also work (it works really well with sources that sometimes have timestamps).  But when you tell appsrc format=time, it is expecting timestamps.  Or at least that is my understanding.

In my situation, I have a pipeline that I pass frames to after the main pipeline has started, so I have to adjust each timestamp by an offset (this is on the new-sample callback on my appsink which feeds my appsrc).

GST_BUFFER_PTS(buffer) = GST_BUFFER_PTS(buffer) - offset;

Hopefully this helps!

On 11/18/2019 7:10 AM, Shishigami wrote:
I'm on Windows 10 using Gstreamer 1.16.1.

I'm trying to run two pipelines in parallel. Sender gets data from appsrc
(OpenCV) and writes them to UDP sink:

appsrc is-live=true format=time ! videoconvert ! video/x-raw, format=I420 !
x264enc speed-preset=superfast tune=zerolatency byte-stream=true
key-int-max=15 ! rtph264pay pt=96 config-interval=5 ! udpsink sync=false
host=localhost port=8553

Receiver reads from UDP source and uses gst-rtsp-server to provide an RTSP
stream:

udpsrc address=localhost port=8553 caps="application/x-rtp,
clock-rate=90000, payload=96" ! rtpjitterbuffer ! rtph264depay ! rtph264pay
name=pay0 pt=96

Opening the stream in VLC media player, the resulting frame from OpenCV does
show up, but it never changes. In Unreal Engine (target environment) the
stream opens fine, but it only shows a gray image.

I'm not sure if this is related, but I set GST_DEBUG and get the following
output from gst-rstp-server:

0:00:03.329599300  6124 0000024F68D88700 FIXME              rtspmedia
rtsp-media.c:4168:gst_rtsp_media_suspend: suspend for dynamic pipelines
needs fixing
0:00:03.329990500  6124 0000024F68D88700 WARN               rtspmedia
rtsp-media.c:4194:gst_rtsp_media_suspend: media 0000024F68DB3220 was not
prepared
0:00:03.339047900  6124 0000024F68D88700 FIXME             rtspclient
rtsp-client.c:1818:handle_play_request:<GstRTSPClient@0000024F68D89850> Add
support for seek style (null)
0:00:03.339776300  6124 0000024F68D88700 FIXME              rtspmedia
rtsp-media.c:2684:gst_rtsp_media_seek_full:<GstRTSPMedia@0000024F68DB3220>
Handle going back to 0 for none live not seekable streams.

I tried using fpsdisplaysink with the following pipeline (instead of
receiver) and that works:

gst-launch-1.0 -v udpsrc address=localhost port=8553
caps="application/x-rtp,clock-rate=90000,payload=96" ! rtpjitterbuffer !
rtph264depay ! decodebin ! videoconvert ! fpsdisplaysink

But the FPS is pretty bad, averaging at 2.29, which is a far cry from the 30
FPS I'm looking to achieve:
last-message = rendered: 240, dropped: 0, current: 2.25, average: 2.29

Why does it work with fpsdisplaysink but not RTSP? I'd highly highly
appreciate any help!



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Shishigami
Reply | Threaded
Open this post in threaded view
♦
♦
|

Re: Only first frame drawn gst-rtsp-server

Shishigami
Thank you for your advice, I really appreciate it! I have gotten similiar
advice on IRC, unfortunately I am not sure howto timestamp the buffers since
I am not using GStreamer directly.
To clarify, I have compiled OpenCV with GStreamer support and simply write
to the pipeline like so:

VideoWriter writer("appsrc ...... udpsink", CAP_GSTREAMER, 0, 30, Size(640,
480), true);
writer << foreground;

I have not found any examples that show howto add the timestamps with
OpenCV.

I have also tried adding do-timestamp=true to appsrc and udpsrc but it did
not help. I will try out videorate, thank you!


Michael MacIntosh wrote
> But when you tell appsrc format=time, it is expecting timestamps.

Yes, I added it because I got errors on the receiving side stating that
timestamps were sent backwards or something along those lines.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Free forum by Nabble Edit this page