Audio stalling when using Opus to send audio to Raspberry Pi 3

[zacattackcit]

Hi all--

I've been having problems with my audio pipeline from a remote computer to my Raspberry Pi 3.  I first noticed the problem on Gstreamer 1.10.1, but it is also present on the HEAD of the 1.10 branch.

The minimal pipelines I've found to reproduce this are:

(remote computer):
gst-launch-1.0 audiotestsrc ! opusenc ! rtpopuspay ! udpsink host=<IP of RPi> port=5555

(RPi, connected via WiFi, which might be important to introduce packet loss/latency?):
GST_DEBUG=4 gst-launch-1.0 udpsrc port=5555 caps="application/x-rtp" ! rtpopusdepay ! opusdec ! audioconvert ! pulsesink

The audio crackles a bit, eventually stops completely, and never recovers.  I see log messages like this:

0:00:03.822817883  1616   0xcd9260 WARN           audiobasesink gstaudiobasesink.c:1512:gst_audio_base_sink_skew_slaving:<pulsesink0> correct clock skew -0:00:00.020306372 < -+0:00:00.020000000
0:00:06.345903418  1616   0xcd9260 WARN           audiobasesink gstaudiobasesink.c:1484:gst_audio_base_sink_skew_slaving:<pulsesink0> correct clock skew +0:00:00.020106622 > +0:00:00.020000000
0:00:18.374848155  1616 0x70b01320 WARN                   pulse pulsesink.c:702:gst_pulsering_stream_underflow_cb:<pulsesink0> Got underflow
0:00:18.680042928  1616 0x70b01320 WARN                   pulse pulsesink.c:702:gst_pulsering_stream_underflow_cb:<pulsesink0> Got underflow
0:00:18.708299984  1616   0xcd9260 WARN           audiobasesink gstaudiobasesink.c:1807:gst_audio_base_sink_get_alignment:<pulsesink0> Unexpected discontinuity in audio timestamps of +0:00:00.082708333, resyncing

This pipeline didn't seem to give me trouble on earlier versions of GStreamer (around 1.9).  Any ideas what could be causing the issues?  I've tried adding queues and jitterbuffers, which seem to help somewhat, but eventually I get the same stalling behavior.

Thanks,

-Zac

[Sebastian Dröge-3]

On Tue, 2016-12-20 at 09:29 -0800, zacattackcit wrote:
> Hi all--
>
> I've been having problems with my audio pipeline from a remote computer to
> my Raspberry Pi 3.  I first noticed the problem on Gstreamer 1.10.1, but it
> is also present on the HEAD of the 1.10 branch.
> [...]

What is the exact last version that is known to work for you? We should
try to find the change that broke it for you.

Also you should usually have an rtpjitterbuffer between the udpsrc and
the depayloader.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[zacattackcit]

Sebastian--

I finally got around to testing this on previous versions of GStreamer.  I tested 1.9.2, 1.8.0, and 1.4.4 (both compiled from source and from the Raspbian deb files).  The issue was present in all versions tested, so doesn't look like a regression after all!  I must have just missed it before.

Adding an rtpjitterbuffer helps, but eventually (sometimes after minutes) the audio cuts out permanently, with log messages similar to those I posted above.  I tried changing various properties on the rtpjitterbuffer, but none completely resolved the issue for me.

I don't care so much about occasional audio crackles, but I don't want the pipeline to stall completely.  Is there something else I can add to my pipeline that will prevent this?  Or failing that, something that will restart things when the audio gets stalled?

Thanks!

[zacattackcit]

I did some more testing with this and found a workaround for this issue.  If I use alsasink the same pipeline is stable.  So, it would seem the problem is somewhere in the pulsesink element.  My observation that the issue is present in (at least) all the way back to 1.4.4 makes more sense now--there have only been 4 commits to pulesink.c in that time!

I'd prefer to use pulseaudio if possible, but for now my workaround is fine.

[zacattackcit]

Any other ideas on this?  I'm running into limitations of ALSA in my application and would like to use pulseaudio among other things, for the nice webrtc echo cancelling package.  Thank you!