shmsink stops working on change in RTP stream

[Erwin]

Hi everyone,

I'm trying to process a RTP stream with gstreamer that's forwarded from
a Janus WebRTC gateway. Due to its WebRTC origin, the properties of the
stream can change from time to time, e.g. the width and height change.
That's not a problem when I play the stream on my local system with the
following pipeline:

gst-launch-1.0 -v rtpbin name=rtpbin latency=100 \
 udpsrc port=60000 caps="application/x-rtp,
media=audio,encoding-name=OPUS, clock-rate=48000" ! \
 rtpbin.recv_rtp_sink_0 \
 udpsrc port=60001 caps="application/x-rtcp" ! rtpbin.recv_rtcp_sink_0 \
 udpsrc port=60002 caps="application/x-rtp,
media=video,encoding-name=VP8, clock-rate=90000" ! \
 rtpbin.recv_rtp_sink_1 \
 udpsrc port=60003 caps="application/x-rtcp" ! rtpbin.recv_rtcp_sink_1 \
 rtpbin. ! rtpvp8depay ! avdec_vp8 ! videoscale ! videoconvert ! \
 video/x-raw,format=BGRA,pixel-aspect-ratio=1/1,\
  interlace-mode=progressive, width=1280, height=720 ! \
 videoconvert ! autovideosink \
 rtpbin. ! rtpopusdepay ! avdec_opus ! audioconvert ! \
 audioresample ! autoaudiosink


I can see the stream properties change as there are messages like these:

/GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:src: caps =
video/x-vp8, framerate=(fraction)0/1, height=(int)540, width=(int)960,
profile=(string)0
/GstPipeline:pipeline0/avdec_vp8:avdec_vp8-0.GstPad:sink: caps =
video/x-vp8, framerate=(fraction)0/1, height=(int)540, width=(int)960,
profile=(string)0
/GstPipeline:pipeline0/avdec_vp8:avdec_vp8-0.GstPad:src: caps =
video/x-raw, format=(string)I420, width=(int)960, height=(int)540,
interlace-mode=(string)progressive, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg,
colorimetry=(string)bt601, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:src: caps =
video/x-raw, format=(string)I420, width=(int)1280, height=(int)720,
interlace-mode=(string)progressive, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg,
colorimetry=(string)bt601, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps =
video/x-raw, format=(string)I420, width=(int)1280, height=(int)720,
interlace-mode=(string)progressive, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg,
colorimetry=(string)bt601, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstVideoScale:videoscale0.GstPad:sink: caps =
video/x-raw, format=(string)I420, width=(int)960, height=(int)540,
interlace-mode=(string)progressive, multiview-mode=(string)mono,
multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,
pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg,
colorimetry=(string)bt601, framerate=(fraction)0/1

So the resolution changes to 960x540, but my display resolution stays
the same thanks to the videoscale element. That's important because my
actual goal is to ingest this stream into Snowmix using shmsink.
Therefore the scaling and the conversion to BGRA. But when I replace the
autovideosink with a shmsink, the pipeline stops working on the first
change of resolution. Here's my pipeline:

gst-launch-1.0 -v \
 rtpbin name=rtpbin latency=100 \
 udpsrc port=60000 caps="application/x-rtp, media=audio,
encoding-name=OPUS, clock-rate=48000" ! rtpbin.recv_rtp_sink_0 \
 udpsrc port=60001 caps="application/x-rtcp" ! rtpbin.recv_rtcp_sink_0 \
 udpsrc port=60002 caps="application/x-rtp, media=video,
encoding-name=VP8, clock-rate=90000" ! rtpbin.recv_rtp_sink_1 \
 udpsrc port=60003 caps="application/x-rtcp" ! rtpbin.recv_rtcp_sink_1 \
 rtpbin. ! rtpvp8depay ! avdec_vp8 ! \
 videoscale ! videoconvert  ! \
 video/x-raw,format=BGRA,pixel-aspect-ratio=1/1,interlace-mode=progressive,width=1280,height=720 ! \
 shmsink socket-path=/tmp/rtptest shm-size=95846400
wait-for-connection=1 sync=true \
 rtpbin. ! rtpopusdepay ! avdec_opus  ! audioconvert ! audioresample ! \
 fakesink

When I watch the stream using another pipeline with shmsrc it works
until the first change of resolution and then the video stops. Also I
don't see any further messages regarding resolution changes in the
pipeline's output like I do when feeding the stream to autovideosink.

As I'm new to gstreamer I'm not sure how to debug this problem. I've
recorded a short RTP stream using tcpdump that can be replayed with
udpreplay to reproduce the problem. Is there a way to "neutralize" those
resolution changes, to keep the pipeline running with shmsink?

Thanks,

Erwin
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[AzazelN28]

Hi Erwin,

I know that this thread is 3 years old but I think I'm having the same issue and I'm completely lost, I tried a lot of different gstreamer setups and as soon as the caps change, shmsink stops working. Did you found a solution to this?