Choppy Audio over UDP
Locked
 
Administrator
|
Hello All,
I have a Leopardboard 365 (Ridgerun Leopardboard-365 Free SDK includes prebuilt Gstreamer) transmitting linein audio with the following pipeline: gst-launch alsasrc do-timestamps=true ! queue2 ! audioconvert ! audioresample \ ! volume volume=2 ! dmaienc_aac outputBufferSize=10000000 outputformat=2 \ ! rtpmp4apay ! queue2 ! udpsink port=5555 host=10.253.5.156 dmaienc_aac is a hardware accelerated aac encoder. For outputformat, 2 is adts. ("man page" is below) My receiver (PC with Ubuntu 10.04 Gstreamer 0.10.31) uses this pipeline: gst-launch-0.10 udpsrc port=5555 caps="application/x-rtp, media=audio, clock-rate=44100, \ encoding-name=MP4A-LATM, cpresent=0, config=40002420, payload=96" \ ! queue2 max-size-bytes=1000000 low-percent=40 high-percent=75 \ ! rtpmp4adepay ! queue2 max-size-bytes=1000000 ! decodebin \ ! alsasink sync=false The output is choppy and sped-up. I tried sync=true, but that doesn't seem to help. The choppiness is worse with long periods of silence punctuated by blips and short bursts of music. I added queue2 max-size-bytes=1000000 after alsasrc and before udpsink (sending pipeline). Did not help. I feel like there is some synchronization step I'm missing, but I have been staring at it too long today. Can someone offer me advice? Wes <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><> dmaienc_aac "man page" <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><> DMAIENC_AAC Help Factory Details: Long name: DMAI xDM 1.0 AAC Encoder Class: Codec/Encoder/Audio Description: DMAI AAC encoder Author(s): Don Darling; Texas Instruments, Inc., Diego Dompe; RidgeRun Engineering Rank: primary (256) Plugin Details: Name: TICodecPlugin Description: Plugin for TI xDM-Based Codecs Filename: /usr/lib/gstreamer-0.10/libgstticodecplugin.so Version: 0.10.0.1 License: LGPL Source module: gstticodecplugin Binary package: TI / RidgeRun Origin URL: http://www.ti.com/, http://www.ridgerun.com GObject +----GstObject +----GstElement +----dmaienc_aac Pad Templates: SRC template: 'src' Availability: Always Capabilities: audio/mpeg mpegversion: 4 channels: [ 1, 2 ] rate: [ 16000, 96000 ] SINK template: 'sink' Availability: Always Capabilities: audio/x-raw-int width: 16 depth: 16 endianness: 1234 channels: [ 1, 2 ] rate: [ 8000, 96000 ] Element Flags: no flags set Element Implementation: Has change_state() function: 0x40649818 Has custom save_thyself() function: gst_element_save_thyself Has custom restore_thyself() function: gst_element_restore_thyself Element has no clocking capabilities. Element has no indexing capabilities. Element has no URI handling capabilities. Pads: SRC: 'src' Implementation: Pad Template: 'src' SINK: 'sink' Implementation: Has chainfunc(): gst_tidmaienc_chain Has custom eventfunc(): gst_tidmaienc_sink_event Pad Template: 'sink' Element Properties: name : The name of the object flags: readable, writable String. Default: null Current: "dmaienc_aac0" outputBufferSize : Size of the output buffer flags: readable, writable Integer. Range: 0 - 2147483647 Default: 0 Current: 0 copyOutput : Boolean that set if the output buffers should be copied into standard gst buffers flags: readable, writable Boolean. Default: false Current: false bitrate : Average bit rate in bps flags: readable, writable Integer. Range: 0 - 2147483647 Default: 128000 Current: 32000 maxbitrate : Max bitrate for VBR encoding flags: readable, writable Integer. Range: 0 - 2147483647 Default: 128000 Current: 32000 downmix : Option to enable downmix flags: readable, writable Boolean. Default: false Current: false fnostereoprocessing : Use stereo preprocessing flag: Only applicable when sampleRate <24000 Hz and bitRate < 60000 bps. flags: readable, writable Boolean. Default: false Current: false invquant : Inverse quantization level flags: readable, writable Integer. Range: 0 - 2 Default: 0 Current: 0 tns : Flag for TNS enable flags: readable, writable Boolean. Default: true Current: true outputformat : Output format: 0 - RAW, 1 - ADIF, 2 - ADTS flags: readable, writable Integer. Range: 0 - 2 Default: 0 Current: 0 fullbandwidth : Flag to enable full bandwidth flags: readable, writable Boolean. Default: false Current: false |
|
Administrator
|
All,
Progress! I added an gstrtpjitterbuffer with various latencies. Helped a lot but I still get dropout and the audio plays justa little too fast. Does require sync=true on alsasink. Any additional suggestion? Thanks, Wes |
|
Hi,
On Sat, Oct 16, 2010 at 12:58 AM, Wes Miller <[hidden email]> wrote:
Yes, it does ;)
UDP does not include timestamps out of the box, so you'll very likely always experience dropouts with such a setup. What you definitely need there for good timing is RTP. I suggest you to take advantage of the rtpbin which covers all the basic aspects of multimedia transmission over the network (yes, there's even a jitterbuffer there ;) ). More infos and examples at: http://www.gstreamer.net/data/doc/gstreamer/head/gst-plugins-good-plugins/html/gst-plugins-good-plugins-gstrtpbin.html Regards
------------------------------------------------------------------------------ Download new Adobe(R) Flash(R) Builder(TM) 4 The new Adobe(R) Flex(R) 4 and Flash(R) Builder(TM) 4 (formerly Flex(R) Builder(TM)) enable the development of rich applications that run across multiple browsers and platforms. Download your free trials today! http://p.sf.net/sfu/adobe-dev2dev _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
|
Administrator
|
Marco,
Thanks for the pointer to gstrtpbin. i will give it a try as soon as I figure out what all the bit and pieces mean. I must say, the examples seen to be more mocha-almond-fudge than vanilla or even chocolate. Wes |
|
Administrator
|
Marco et al,
Ok, new pipelines with rtpbin. These work though I am not sure they are completely correct. SENDER <><><><><><><><><><><><><><><><> gst-launch-0.10 -v gstrtpbin name=rtpbin \ alsasrc do-timestamp=true \ ! queue \ ! audioresample \ ! audioconvert \ ! dmaienc_aac outputBufferSize=10000000 outputformat=2 \ ! rtpmp4apay \ ! rtpbin.send_rtp_sink_1 \ rtpbin.send_rtp_src_1 \ ! udpsink port=5002 host=$1 ts-offset=0 name=artpsink \ rtpbin.send_rtcp_src_1 \ ! udpsink port=5003 host=$1 sync=false async=false name=artcpsink \ udpsrc port=5007 name=artpsrc \ ! rtpbin.recv_rtcp_sink_1 RECEIVER <><><><><><><><><><><><><><><> gst-launch-0.10 -v gstrtpbin name=rtpbin latency=2000 \ udpsrc caps="application/x-rtp,media=audio,clock-rate=44100,encoding-name=MP4A-LATM,payload=96" port=5002 \ ! rtpbin.recv_rtp_sink_1 \ rtpbin. \ ! rtpmp4adepay \ ! decodebin \ ! audioconvert \ ! audioresample \ ! alsasink sync=true \ udpsrc port=5002 \ ! rtpbin.recv_rtcp_sink_1 \ rtpbin.send_rtcp_src_1 \ ! udpsink port=5003 host=$1 sync=false async=false I am still getting playback that is too fast and choppy. Sounds like the receiver pipe is just playing each packet too fast. I don't think I'm loosing any of the music, it just plays each packet too fast and has a dead spot in between them. Imagine the Moody Blues sung by the Chipmunks with hiccups. So, first, do I haave the ports mapped correctly? Am I missing any connections between the rtpbin in's and out's? Secondly, is there a retimesynch trick I can use? Replacing alsasink with fakesink -v tells me that the packets are timestamped and that the durations match the time between timestamps. I really intend to replace the mp3 player I am using with a microphone. The Leopardboard only has linelevel input so I can't use a mic right now. If passing speech, would I be more likely to hear the speed=up and gaps? Will speex through an rtpxxxpayloader still cause me to need 4k of Caps? Thanks, Wes |
|
|
Hi,
On Mon, Oct 18, 2010 at 9:03 PM, Wes Miller <[hidden email]> wrote:
Ok now I understand what's the issue. If it appears to be between elves or ogres either you're reading the Lord of Rings or the sample rate differs between encoder and decoder. You may try and explicitly set the RTP clock in the receiving pipeline with something lik: caps="application/x-rtp,media=(string)audio,clock-rate=(int)8000,encoding-name=(string)MPEG4"Even though these should already be the default values. Another (more likely) possibility is that the encoder you're using is working at a default sample rate different from the one of the decoder. You should thus checker whether any compile/run time options make it possible for you to use the same values. Regards
------------------------------------------------------------------------------ Download new Adobe(R) Flash(R) Builder(TM) 4 The new Adobe(R) Flex(R) 4 and Flash(R) Builder(TM) 4 (formerly Flex(R) Builder(TM)) enable the development of rich applications that run across multiple browsers and platforms. Download your free trials today! http://p.sf.net/sfu/adobe-dev2dev _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
|
|
In reply to this post by Wes Miller
Hi,
sorry for double posting, I've forgot to add a few comments in my previous email. On Mon, Oct 18, 2010 at 9:03 PM, Wes Miller <[hidden email]> wrote:
Are the audioresample/audioconvert necessary? What if you use pulsesrc / pulsesink and remove them?
here clock-rate is the RTP clock, you should set it to a value according with your sample rate, check the caps negotiated from the source element on the sender side to have an hint. You can do this using gst-launch -v payload=96" you don't need decodebin here. Just put the decoder element. ! audioconvert \ Again, try replacing audioconvert/audioresample/alsasink with a plain pulsesink (if you can ;) ). Regards, Marco udpsrc port=5002 \ ------------------------------------------------------------------------------ Download new Adobe(R) Flash(R) Builder(TM) 4 The new Adobe(R) Flex(R) 4 and Flash(R) Builder(TM) 4 (formerly Flex(R) Builder(TM)) enable the development of rich applications that run across multiple browsers and platforms. Download your free trials today! http://p.sf.net/sfu/adobe-dev2dev _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
|
Administrator
|
Marco,
Better, still not quite right. Removing audioconvert and audioresample on both sender and receiver seem to have little or no effect, so they are now out. Pulsesink is working on the receiver (my Linux workstation/host). I can use pulsesrc on the sender wince Ti/RidgeRun don't seem to include the pulse stuff in their ports of gst. I keep eading about alsa hardware on the Leopardboard...??? I used fakesink to get the sender caps (from fakesink0:Gstpad:sink) and I notice that the ssrc, clock-base and seqnum change every time I run the pipeline. If the clock-base is different each time I start the sender, how can the receiver ever actually match the sender? Is there a tcp-ish way to pass the caps to the receiver and insert them in the receiver pipeline? (sounds like a great, first, element writing project, doesn't it?) I've tried to find out what ssrc is/are and can't find a description. So what is it? Does it matter? As ever, many thanks, Wes |
|
Administrator
|
Hi All,
Two additional bit of information: 1. "TURN THAT $@!# THING DOWN" I added volume elements in both sener and receiver pipes. Made a big difference. Guess I was overdriving everything. 2. For the aac pipes above, I had to slow down the clock-rate on the receiver to about 20400 to get something that sounded even remotely like it was matched to the clock-rate=44100 sender. So, it's still awfully jittery. On a whim, I tried going back to just using udpsrc/udpsink without rtpbin. Still poor quality. Then I took out the dmaienc_aac and replaced it with several different encoders (aka, whatever TI and RidgeRun managed to stick in the GST packages). Finally landed on alawenc/dec. Suitably altered the clock-rate and nixed gstrtpbin and behold, pretty good sound. A mite echoy but WAY better. So, these are the best pipes I have right now: SENDER: gst-launch-0.10 -e -v \ alsasrc do-timestamp=true \ ! queue2 \ ! alawenc \ ! udpsink port=5002 host=$1 RECEIVER: gst-launch-0.10 -v \ udpsrc caps="audio/x-alaw, channels=2, rate=29000" \ port=5002 \ ! queue2 \ ! alawdec \ ! volume volume=0.1 \ ! queue2 \ ! pulsesink Thanks for all the help, M4arco. Wes |
|
|
In reply to this post by Wes Miller
Hi Wes,
I see from your next post you've made a few progresses with alaw which is, (maybe) unfortunately, a codec working @ quite low bitrates. If it's enough for you to work at such levels you may want to give a try to g729 as well. On Tue, Oct 19, 2010 at 5:18 PM, Wes Miller <[hidden email]> wrote:
I just suggest you to fix the samplerate between alsasrc and the encoder with fixed caps, something like: gst-launch-0.10 alsasrc ! "audio/x-raw-int, rate=44100" ! pulsesink at least you'll know which sample rate you're working at ;)
They're expected to change each time. Some references: http://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/gst/rtp/README
as clock-base is the (stream) initial time, it's expected to change slightly each time (I guess mainly because of source buffering/processing jitter). Is there a tcp-ish way to pass the caps to the receiver and insert them in yes, there are plenty of standard ways. SIP is an example, but also RTSP and MMS. All of those are excellently implmemented in many gstreamer -based applications and elements.
SSRC is the stream identification used to distinguish it among different communications about two endpoints (thinki about a videocall: audio and video are separate streams). More references here: http://www.ietf.org/rfc/rfc3550.txt Regards
------------------------------------------------------------------------------ Nokia and AT&T present the 2010 Calling All Innovators-North America contest Create new apps & games for the Nokia N8 for consumers in U.S. and Canada $10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store http://p.sf.net/sfu/nokia-dev2dev _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
«
Return to GStreamer-devel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |