RTP audio stream to filesink, issue with packet loss

[Thorsten Brendt]

Hi!

I'm using Gstreamer to save RTP audio streams to wav files.  This works in
general, however, I need to deal with packet loss and in this case the pipeline
I'm currently using fails:

  filesrc location=foobar.pcap ! pcapparse
   ! "application/x-rtp, payload=0, clock-rate=8000"
   ! gstrtpjitterbuffer ! rtppcmudepay ! mulawdec ! audioconvert
   ! audioresample ! wavenc ! filesink location=foobar.wav

This produces audio files that are shorter than the original RTP stream,
e.g. for a certain amount of consecutively lost packets a second of audio is
missing from the wav file.  For a real audio sink this doesn't happen as it
plays audio when something is available, the filesink however -- I assume --
writes data sequentially and only when data is actually available.  By the
way: the effect is the same for udpsrc as it is for filesrc+pcapparse.

I tried the above pipeline with filesink's sync property set to true ([1]), this
didn't make any difference.  I read somewhere in the Gstreamer docs that the
global clock is [usually] dependent on the sink element.  Do I need to reverse
this and make the sink's clock dependent on the source's clock?

1. <http://www.gstreamer.net/data/doc/gstreamer/head/gstreamer-libs/html/GstBaseSink.html#gst-base-sink-set-sync>

Maybe I'm totally off in this regard and there is no implicit way to achieve
what I need.  Ultimately some element in the pipeline needs to produce
"silence/empty/zero data" if its input buffer is empty.  Is there an explicit
way (helper element/attribute) to do this?

Thorsten

------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand
malware threats, the impact they can have on your business, and how you
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Tim-Philipp Müller-2]

On Wed, 2011-01-12 at 22:21 +0100, Thorsten Brendt wrote:

Hi,

> (snip)
>
> Ultimately some element in the pipeline needs to produce  
>  "silence/empty/zero data" if its input buffer is empty.  Is there an
>   explicit way (helper element/attribute) to do this?

Tried adding an audiorate element?

 Cheers
  -Tim


------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand
malware threats, the impact they can have on your business, and how you
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[shiva varma]

You can take the output of the rtppcmudepay directly to wavenc...

The below pipeline should work...

filesrc location=foobar.pcap ! pcapparse
   ! "application/x-rtp, payload=0, clock-rate=8000"
   ! gstrtpjitterbuffer ! rtppcmudepay ! wavenc ! filesink location=foobar.wav

[Thorsten Brendt]

On Thu, Jan 13, 2011 at 12:16 AM, Tim-Philipp Müller <[hidden email]> wrote:

> On Wed, 2011-01-12 at 22:21 +0100, Thorsten Brendt wrote:
>
> Hi,
>
>> (snip)
>>
>> Ultimately some element in the pipeline needs to produce
>>  "silence/empty/zero data" if its input buffer is empty.  Is there an
>>   explicit way (helper element/attribute) to do this?
>
> Tried adding an audiorate element?

Do you mean a caps-filter element like this?:

'audio/x-raw-int, width=16, depth=16, rate=8000, endianness=1234,
signed=(boolean)true, channels=1'

The right place to put this seems to be before the encoder element
(wavenc in my case), however in that case the pipeline can't be linked
together.  It works before an autoaudiosink though.

Anyhow, I assume that the audio's sampling rate is known to the
encoder already, as it both given explicitly as a clock-rate attribute
in a previous caps-filter and should be implicitly known by the
mulaw-decoder (plus the audio file "sounds right" in general).

------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand
malware threats, the impact they can have on your business, and how you
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Thorsten Brendt]

On Thu, Jan 13, 2011 at 12:50 AM, shiva varma <[hidden email]> wrote:
>
> You can take the output of the rtppcmudepay directly to wavenc...
>
> The below pipeline should work...
>
> filesrc location=foobar.pcap ! pcapparse
>   ! "application/x-rtp, payload=0, clock-rate=8000"
>   ! gstrtpjitterbuffer ! rtppcmudepay ! wavenc ! filesink
> location=foobar.wav

Thanks for the suggestion, but I need to have uncompressed PCM audio
in the wav file.  Should have stated that more clearly, "a wav file"
is really kind of a noop.

------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand
malware threats, the impact they can have on your business, and how you
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Tim-Philipp Müller-2]

On Thu, 2011-01-13 at 03:10 +0100, Thorsten Brendt wrote:

> > Tried adding an audiorate element?
>
> Do you mean a caps-filter element like this?:
>
> 'audio/x-raw-int, width=16, depth=16, rate=8000, endianness=1234,
> signed=(boolean)true, channels=1'

No, I meant an "audiorate" element, which is part of gst-plugins-base.

Factory Details:
  Long name: Audio rate adjuster
  Class: Filter/Effect/Audio
  Description: Drops/duplicates/adjusts timestamps on audio samples to
make a perfect stream

Cheers
 -Tim



------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand
malware threats, the impact they can have on your business, and how you
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Thorsten Brendt]

On Thu, Jan 13, 2011 at 10:46 AM, Tim-Philipp Müller <[hidden email]> wrote:

> On Thu, 2011-01-13 at 03:10 +0100, Thorsten Brendt wrote:
>
>> > Tried adding an audiorate element?
>>
>> Do you mean a caps-filter element like this?:
>>
>> 'audio/x-raw-int, width=16, depth=16, rate=8000, endianness=1234,
>> signed=(boolean)true, channels=1'
>
> No, I meant an "audiorate" element, which is part of gst-plugins-base.
>
> Factory Details:
>  Long name:    Audio rate adjuster
>  Class:        Filter/Effect/Audio
>  Description:  Drops/duplicates/adjusts timestamps on audio samples to
> make a perfect stream

Ah, I should've looked at the gstreamer sources instead of searching
for "gstreamer audiorate"... lazy me!

Anyway, this was a great pointer!  Just adding audiorate to the
pipeline didn't help.  However, I could trace down the issue to the
jitter buffer: I needed to set do-lost=true to ask the jitter buffer
element to send packet-lost events downstream.  Now this combination
works just like I expected.  For the sake of completeness here's the
pipeline I used:

filesrc location=foobar.pcap ! pcapparse
  ! "application/x-rtp, payload=0, clock-rate=8000"
  ! gstrtpjitterbuffer do-lost=true ! rtppcmudepay ! mulawdec
  ! audiorate ! wavenc ! filesink location=foobar.wav

Tim, thank you very much for your help!

Cheers,
Thorsten

------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand
malware threats, the impact they can have on your business, and how you
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel