RTP audio packets are sent as a burst, not in real-time, with rawaudioparse

[wegherfe]

Hi all.

I have to implement a RTP streaming pipeline for S16LE 48KHz raw audio content, sourced from a Unix named pipe. My pipeline is the following:

gst-launch-1.0 blocksize=1388 location=/home/wegherfe/pipe-sink ! queue ! "audio/x-unaligned-raw, format=S16LE, channels=2, rate=48000, layout=interleaved" ! rawaudioparse use-sink-caps=true ! audiorate ! audioconvert dithering=0 noise-shaping=0 ! rtpL16pay pt=99 ! queue ! udpsink host=192.168.1.2 port=6004

In short, the pipeline sets the named pipe as file source, raw parses it with caps, converts to S16 BE format and encapsulates data into RTP payload. The two queues are used to split job into 3 tasks: reading, processing, transmitting.

In order to test the pipeline, just create the Unix named pipe (as root) and write raw audio content into it (I have attached the sample raw audio I used in my tests: it says "front center", taken from my Ubuntu PC: audio.raw):

mknode /home/wegherfe/pipe-sink p
cat audio.raw > /home/wegherfe/pipe-sink

The server pipeline and test commands are run on my laptop, which is connected to a small embedded device, working as RTP client, which has limited buffering and processing capabilities.

The problem is the following: audio is played fluently and properly at first invocation of cat command, but it is played with glitches and bad timing sometimes at following invocations. Usually, the first word "front" is eaten, cannot be heard well, while "center" is usually fine.

By running tcpdump, I noticed that, with following invocations of cat command, UDP packets are sent as a burst and not in real-time. I think this causes issue to my client device, which has a little buffer.

Notices:
1) if I remove rawaudioparse and use wavparse, audio is always ok:

gst-launch-1.0 filesrc location=/home/wegherfe/Front_Center.wav ! wavparse ! queue ! audioresample ! audioconvert dithering=0 noise-shaping=0 ! rtpL16pay pt=99 ! queue ! udpsink host=192.168.1.2 port=6004

so I think the problem is in timestamps generated by rawaudioparse;

2) if I set blocksize to smaller values, the client device does not work properly
3) queue elements seems to mitigate the issue a bit

[Sebastian Dröge-3]

On Thu, 2017-04-27 at 02:28 -0700, wegherfe wrote:
>
> [...]
> Notices:
> 1) if I remove rawaudioparse and use wavparse, audio is always ok:

I don't see anything fundamentally wrong with your pipeline.
rawaudioparse should put proper timestamps on the audio, and udpsink
then should sync those to the clock and send them out in real-time
according to the timestamps.

So as you also guess, something must be wrong with the timestamps.


Can you file a bug here with instructions how to reproduce?
https://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[wegherfe]

Hi Sebastian.

Thanks for your reply. I analysed the timestamps and indeed the problem is due to the improper use of rawaudioparse with a source which is not complete, like a file for example, but rather live.
By debugging udpsink, I noticed that during second play audio packets are marked as "too late" and, instead of being dropped (max-lateness=-1), they are sent out of sync as a burst. This happens because there are no silence audio data between the two audio playbacks, so timestamps of audio samples of second play are written as consecutive by rawaudioparse, as if there were no pause between the two plays (as those data come from a file). But in the meantime the system clock went on, so udpsink detects those samples as too late.
Is it possible that rawaudioparse cannot be used with live sources?

At the moment I am working on a new pipeline, without rawaudioparse, with property sync=false for udpsink and with appsrc, sending audio frames in realtime (by queuing data and scheduling the push-buffers in time for rt behavior). I will be back with results asap.

I have already opened a bug there. Please look at: https://bugzilla.gnome.org/show_bug.cgi?id=781824

Regards,
Federico