Loss of interleaved stereo audio after ~20 Minutes

[Wilkins White]

We have a robot that uses gstreamer to send back audio and video data over
rtp.  Video works great, but we've been having trouble with the stereo
audio.

The audio is clear when we first boot up, but after awhile we start to lose
packets until there's nothing but static coming through.  If we kill the
pipeline and restart everything is clear once more.  The problem is
repeatable in our lab with the bot connected via Ethernet, so it's not a
bandwidth issue.

This is my pipeline


And here is the error once the pipeline starts to fail.


I went through and trimmed elements out of the pipeline until everything
worked.  It seems like the culprit is the audiointerleave element.  Since
only pulsesrc0 is dropping samples I assume that means pulsesrc1 is not
producing audio fast enough, so the audiointerleave element is forced to
wait for it.

Does anyone have advice on how I could improve my pipeline?  Perhaps make
the audiointerleave element not block if it can't find a sample from one of
the inputs? We want the operator to be able to hear what's going on around
the robot, which means sync'd stereo audio, but one ear cutting out every so
often wouldn't be a big deal.

Thanks!





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim Müller]

On Mon, 2018-06-04 at 16:11 -0700, Wilkins White wrote:

Hi,

> We have a robot that uses gstreamer to send back audio and video data
> over rtp.  Video works great, but we've been having trouble with the
> stereo audio.
>
> The audio is clear when we first boot up, but after awhile we start
> to lose packets until there's nothing but static coming through.  If
> we kill the pipeline and restart everything is clear once more.  The
> problem is repeatable in our lab with the bot connected via Ethernet,
> so it's not a bandwidth issue.
>
> This is my pipeline

Your pipeline didn't make it to the mailing list, and I can't see it on
  the nabble web page either.

It would be helpful if you could describe your sender pipeline and your
receiver pipeline/application.

> And here is the error once the pipeline starts to fail.

Same for this.


> I went through and trimmed elements out of the pipeline until
> everything worked.  It seems like the culprit is the audiointerleave
> element.  Since only pulsesrc0 is dropping samples I assume that
> means pulsesrc1 is not producing audio fast enough, so the
> audiointerleave element is forced to wait for it.
>
> Does anyone have advice on how I could improve my pipeline?  Perhaps
> make the audiointerleave element not block if it can't find a sample
> from one of the inputs?

audiointerleave *should* already handle this properly, as it is based
on the aggregator base class, but maybe you need to set some extra
latency on it via the property, but again details of the pipeline and
the error message would help.

Cheers
-Tim

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim Müller]

On Tue, 2018-06-05 at 10:52 +0100, Tim Müller wrote:

PS: without knowing your pipelines, the error, or if the problem is
sender or receiver side: my first question would be whether you have an
rtpjitterbuffer (or rtpbin) in your receiver pipeline.

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[virtually_me]

I have been experiencing a similar problem. I am sending raw audio (no video) via RTP in UDP packets, and using an rtpjitterbuffer on the RX machine. The audio would just abruptly cease at random times, sometimes after many hours or sometimes after only 10 minutes. After seeing this email I increased the latency of the audiointerleave elements I am using in both the TX and RX pipelines (with UPD inbetween them) but this did not at all solve the problem. Then last night I also increased the latency of the rtpjitterbuffer to 500msec (it was previously about 100-150msec) and left the system streaming overnight. This morning, and in the ensuing few hours, the audio is perfect. For me it looks like it was the rtpjitterbuffer causing the problem. Likely the audiointerleave latencies are not responsible, but I haven't dialed them back yet to check. I didn't look for any errors produced so this may not be all that useful for others except as a hint. I assume that the rtpjitterbuffer should gracefully recover even in the event of starving or blocking conditions, but perhaps not.



-----Original Message-----
From: gstreamer-devel <[hidden email]> On Behalf Of Tim Müller
Sent: Tuesday, June 5, 2018 2:57 AM
To: Discussion of the development of and with GStreamer <[hidden email]>
Subject: Re: Loss of interleaved stereo audio after ~20 Minutes

On Tue, 2018-06-05 at 10:52 +0100, Tim Müller wrote:

PS: without knowing your pipelines, the error, or if the problem is sender or receiver side: my first question would be whether you have an rtpjitterbuffer (or rtpbin) in your receiver pipeline.

--
Tim Müller, Centricular Ltd - http://www.centricular.com _______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Wilkins White]

the Raw tag must not have worked

Pipeline:

gst-launch-1.0 -vvv audiointerleave name=i \
        ! queue \
        ! opusenc audio-type="voice" perfect-timestamp=true complexity=0
bitrate=$BITRATE \
        ! rtpopuspay \
        ! udpsink host=$CLIENT_IP port=$PORT sync=false async=false \
      pulsesrc do-timestamp=true device=$AUDIO0 buffer-time=20000 \
        ! audioconvert \
        ! "audio/x-raw,channels=1" \
        ! i.sink_0 \
      pulsesrc do-timestamp=true device=$AUDIO1 buffer-time=20000 \
        ! audioconvert \
        ! "audio/x-raw,channels=1" \
        ! i.sink_1

Error (repeating):

WARNING: from element /GstPipeline:pipeline0/GstPulseSrc:pulsesrc0: Can't
record audio fast enough
Additional debug info:
gstaudiobasesrc.c(869): gst_audio_base_src_create ():
/GstPipeline:pipeline0/GstPulseSrc:pulsesrc0:
Dropped 882 samples. This is most likely because downstream can't keep up
and is consuming samples too slowly.




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Wilkins White]

The problem still occurs when removing all the encoding elements and piping
data into a fakesink like so:

gst-launch-1.0 -vvv audiointerleave name=i \
        ! queue \
        ! fakesink \
      pulsesrc do-timestamp=true device=$AUDIO0 buffer-time=20000 \
        ! audioconvert \
        ! "audio/x-raw,channels=1" \
        ! i.sink_0 \
      pulsesrc do-timestamp=true device=$AUDIO1 buffer-time=20000 \
        ! audioconvert \
        ! "audio/x-raw,channels=1" \
        ! i.sink_1




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel