Hi,
in some <span id="spans0e0" tooltip="{<b>use cases</b>} <i>replaces</i> <br/>{<b><s style='color:Red;'>usecases</s></b>}: wrong word" onmouseover="showtooltip(this,300);" onmouseout="hideddrivetip();" class="sac" onclick="showMenu(this);">use cases (broadcast digital radio or IP radio), the transmitter is sending an audio stream with a <span id="spans0e1" tooltip="{<b>nominal</b>}: relating to or constituting or bearing or giving a name<br/><i>replaces</i><br/> {<b><s style='color:Red;'>norminal</s></b>}: wrong word" onmouseover="showtooltip(this,300);" onmouseout="hideddrivetip();" class="sac" onclick="showMenu(this);">nominal sampling rate (e.g. 48kHz). When a receiver, expecting 48KHz, wants to play <span id="spans0e2" tooltip="{<b>these samples</b>} <i>replaces</i> <br/>{<b><s style='color:Red;'>this samples</s></b>}: wrong word" onmouseover="showtooltip(this,300);" onmouseout="hideddrivetip();" class="sac" onclick="showMenu(this);">these samples on a local sound sink, then there is always the problem, that the transmitter clock <span id="spans0e3" tooltip="{<b>slightly differs</b>} <i>replaces</i> <br/>{<b><s style='color:Red;'>sligly differ</s></b>}: wrong word" onmouseover="showtooltip(this,300);" onmouseout="hideddrivetip();" class="sac" onclick="showMenu(this);">slightly differs from the receiver clock. For that issue, I need <span id="spans0e4" tooltip="{<b>an</b>} <i>replaces</i> <br/>{<b><s style='color:Red;'>a</s></b>}: wrong word" onmouseover="showtooltip(this,300);" onmouseout="hideddrivetip();" class="sac" onclick="showMenu(this);">an <span id="spans0e5" tooltip="{<b>asynchronous</b>}: (digital communication) pertaining to a transmission technique that does not require a common clock between the communicating devices; timing signals are derived from special characters in the data stream itself<br/><i>replaces</i><br/> {<b><s style='color:Red;'>asynchrounous</s></b>}: wrong word" onmouseover="showtooltip(this,300);" onmouseout="hideddrivetip();" class="sac" onclick="showMenu(this);">asynchronous sampling rate <span id="spans0e6" tooltip="{<b>converter</b>}: a device for changing one substance or form or state into another<br/><i>replaces</i><br/> {<b><s style='color:Red;'>conmverter</s></b>}: wrong word" onmouseover="showtooltip(this,300);" onmouseout="hideddrivetip();" class="sac" onclick="showMenu(this);">converter! Is this possible the available <span style="text-decoration:none;background-color:yellow;" tooltip="{gstreamer} : unknown word" onmouseover="showtooltip(this,300);" onmouseout="hideddrivetip();">gstreamer plugins? <span id="spans0e8" tooltip="{<b>Do</b>} <i>replaces</i> <br/>{<b><s style='color:Red;'>Does</s></b>}: wrong word" onmouseover="showtooltip(this,300);" onmouseout="hideddrivetip();" class="sac" onclick="showMenu(this);">Do <span style="text-decoration:none;background-color:yellow;" tooltip="{audioresample} : unknown word" onmouseover="showtooltip(this,300);" onmouseout="hideddrivetip();">audioresample support <span id="spans0e10" tooltip="{<b>asynchronous</b>}: (digital communication) pertaining to a transmission technique that does not require a common clock between the communicating devices; timing signals are derived from special characters in the data stream itself<br/><i>replaces</i><br/> {<b><s style='color:Red;'>asynchrounous</s></b>}: wrong word" onmouseover="showtooltip(this,300);" onmouseout="hideddrivetip();" class="sac" onclick="showMenu(this);">asynchronous clocks? My example: My Transmitter (sending samples with (48-1)kHz): gst-launch-0.10 audiotestsrc freq=300 ! audio/x-raw-int,
endianness="(int)1234", signed="(boolean)true", width="(int)16",
depth="(int)16", rate="(int)47000",
channels="(int)1" ! tcpclientsink host=localhost port=3000
My Receiver: (Expeting 48kHz)
gst-launch-0.10 tcpserversrc host=localhost port=3000 !
audio/x-raw-int, endianness="(int)1234", signed="(boolean)true",
width="(int)16", depth="(int)16", rate="(int)48000",
channels="(int)1" ! audioconvert ! audioresample !
audio/x-raw-int, endianness="(int)1234", signed="(boolean)true",
width="(int)16", depth="(int)16", rate="(int)48000",
channels="(int)1" ! autoaudiosink
Because the transmitter clock (47kHz) <span id="spans0e0" tooltip="{<b>differs</b>} <i>replaces</i> <br/>{<b><s style='color:Red;'>differ</s></b>}: wrong word" onmouseover="showtooltip(this,300);" onmouseout="hideddrivetip();" class="sac" onclick="showMenu(this);">differs from the receiver clock (48kHz), the sound drops <span id="spans0e2" tooltip="{<b>over</b>} <i>replaces</i> <br/>{<b><s style='color:Red;'>after</s></b>}: wrong word" onmouseover="showtooltip(this,300);" onmouseout="hideddrivetip();" class="sac" onclick="showMenu(this);">over short time. Thank you for any hint in advance. Maik _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
On Fr, 2016-04-29 at 21:11 +0200, Maik Scholz wrote:
> Hi, > > in some use cases (broadcast digital radio or IP radio), the > transmitter is sending an audio stream with a nominal sampling rate > (e.g. 48kHz). > When a receiver, expecting 48KHz, wants to play these samples on a > local sound sink, then there is always the problem, that the > transmitter clock slightly differs from the receiver clock. > For that issue, I need an asynchronous sampling rate converter! > > Is this possible the available gstreamer plugins? > Do audioresample support asynchronous clocks? of work (but in your case you can use audioresample to resample the audio as you know the ratio between sender and receiver rate). This is something that can be implemented on various layers, e.g. with RTP this would happen based on the RTP timestamps and arrival times inside the rtpjitterbuffer. You don't have any timestamps inside the stream so that won't do it. In your case you would have to implement something similar somewhere downstream of the tcpserversrc. Something that correlates the amount of data received per time unit with the duration of that time unit and then extrapolates the relative rate difference between the server and receiver clock, and then accordingly timestamps the media, or directly resamples the media (for which you can again use audioresample here). One main problem in your case is that you use TCP, and due to all the buffering that happens in TCP you will have it harder to calculate this than with UDP. And as you don't have timestamps inside the stream, you will have to assume that your media is continuous which might also not be true. And then (if you don't resample anywhere but just adjust timestamps), it's the job of the audio sink in the end of make up for the difference between timestamps and actual samples received. It will be default just "skip" (skew) but you can configure it also to resample, this can be done with the slave-method property on all audio sinks. However the resampling that is currently implemented there is very minimal and audioresample will give you better quality, but that's something that should be fixed. Does that answer your question? There's no standard solution for this (as your streaming protocol is not and has many use case specific assumptions) and you have to implement something yourself with the help of existing GStreamer code. -- Sebastian Dröge, Centricular Ltd · http://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel signature.asc (968 bytes) Download Attachment |
In reply to this post by Maik Scholz
On Fr, 2016-04-29 at 21:11 +0200, Maik Scholz wrote:
> > My example: > My Transmitter (sending samples with (48-1)kHz): > gst-launch-0.10 audiotestsrc freq=300 ! audio/x-raw-int, > endianness="(int)1234", signed="(boolean)true", width="(int)16", > depth="(int)16", rate="(int)47000", channels="(int)1" ! tcpclientsink > host=localhost port=3000 > > My Receiver: (Expeting 48kHz) > gst-launch-0.10 tcpserversrc host=localhost port=3000 ! audio/x-raw- > int, endianness="(int)1234", signed="(boolean)true", width="(int)16", > depth="(int)16", rate="(int)48000", channels="(int)1" ! audioconvert > ! audioresample ! audio/x-raw-int, endianness="(int)1234", > signed="(boolean)true", width="(int)16", depth="(int)16", > rate="(int)48000", channels="(int)1" ! autoaudiosink use GStreamer 0.10 but use a recent 1.x version, e.g. 1.8.1. 0.10 is no longer maintained since more than 3 years and you're unlikely to get much support for it anywhere. Then, you should make sure that downstream of tcpserversrc always a multiple of full samples arrives. That is, in your case an even buffer size. This is not guaranteed because TCP is a stream protocol. The audioparse element can ensure that this is the case for you. -- Sebastian Dröge, Centricular Ltd · http://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel signature.asc (968 bytes) Download Attachment |
Hi,
thank you very much for the hints. With your help, my pipeline is running quite well with ONE audio source. My target is, having multiply audio sources with different sample rates and different external clock sources. That includes small sample rate mismatches (+- 50Hz) and drifts. My mixer pipeline prototype: GSTLAUNCH=gst-launch-1.0 STREAMDECHOST=127.0.0.1 PORT_SRC1=41001 PORT_SRC2=42001 PORT_SINK_DEBUG_1=45002 ${GSTLAUNCH} \ audiomixer name=sink_audiomixer latency=256000000 ! \ tee name=sink_tee \ \ sink_tee. ! \ queue name=outtcp_queue ! \ tcpserversink name=tcpserversink host="0.0.0.0" port=${PORT_SINK_DEBUG_1} \ \ sink_tee. ! \ queue name=outdsp_queue ! \ autoaudiosink name=autoaudiosink sync=true slave-method=resample \ \ sink_tee. ! \ queue name=file_queue ! \ audioconvert name=fileconvert ! \ filesink name=file_filesink location=/dev/null \ \ udpsrc name=src1_src port=${PORT_SRC1} do-timestamp=true ! \ audio/x-raw,format=S16LE,channels=2,rate=32000 ! \ queue name=src1_queue min-threshold-time=0 ! \ audiorate name=src1_audiorate tolerance=256000000 skip-to-first=false ! \ audioconvert name=src1_audioconvert ! \ audioresample name=src1_audioresample quality=1 ! \ audio/x-raw,format=S16LE,channels=2,rate=48000 ! \ tee name=src1_tee \ \ src1_tee. ! \ volume name=src1_volume volume=0.5 ! \ sink_audiomixer. \ \ udpsrc name=src2_src port=${PORT_SRC2} do-timestamp=true ! \ audio/x-raw,format=S16LE,channels=2,rate=48000 ! \ queue name=src2_queue min-threshold-time=0 ! \ audiorate name=src2_audiorate tolerance=256000000 skip-to-first=false ! \ audioconvert name=src2_audioconvert ! \ audioresample name=src2_audioresample quality=4 ! \ audio/x-raw,format=S16LE,channels=2,rate=48000 ! \ tee name=src2_tee \ \ src2_tee. ! \ volume name=src2_volume volume=0.5 ! \ sink_audiomixer. My SRC1 audio transmitter (32000 - 50Hz): gst-launch-1.0 audiotestsrc freq=100 ! audio/x-raw,format=S16LE,channels=2,rate=31950 ! udpsink host="127.0.0.1" port=41001 My SRC2 audio transmitter (48000 + 50Hz): gst-launch-1.0 audiotestsrc freq=300 ! audio/x-raw,format=S16LE,channels=2,rate=48050 ! udpsink host="127.0.0.1" port=42001 When I run these pipelines with even the latest gst version, then I hear "clicks" from time to time. I expect, that the audiomixer has some problems with the differ sampling frequencies. My assumption is, I need a asynchronous SRC for each input (src1_audioresample+src2_audioresample). How can I implement this with available elements. Or if not available, what is the best way to implement these? Why is the audiomixer located in the bad plugins? What means bad? |
In most cases, there is only one clock (-> audio clock = pipeline
clock), but in your case, the pipeline clock is the monotonic system clock, so audiobasesink needs to compensate for the drift. The clicks are most likely coming from the "skew" drift compensation method in the audiobasesink. This method looks at the drift, and if a threshold is reached, moves the playout pointer (= an offset withing a ring buffer), effectively inserting nullsamples or discarding samples, depending on the skew direction. You can hook in your own drift compensation method if you set the audiobasesink's "slave-method" property to "custom". We use that in a project to finetune the audio clock's speed via a PLL by an integer number of ppm. It would be the same with an ASRC. On 2016-05-25 15:04, Maik Scholz wrote: > Hi, > thank you very much for the hints. > With your help, my pipeline is running quite well with ONE audio source. > > My target is, having multiply audio sources with different sample rates and > different external clock sources. > That includes small sample rate mismatches (+- 50Hz) and drifts. > > *My mixer pipeline prototype:* > GSTLAUNCH=gst-launch-1.0 > STREAMDECHOST=127.0.0.1 > PORT_SRC1=41001 > PORT_SRC2=42001 > PORT_SINK_DEBUG_1=45002 > > ${GSTLAUNCH} \ > audiomixer name=sink_audiomixer latency=256000000 ! \ > tee name=sink_tee \ > \ > sink_tee. ! \ > queue name=outtcp_queue ! \ > tcpserversink name=tcpserversink host="0.0.0.0" > port=${PORT_SINK_DEBUG_1} \ > \ > sink_tee. ! \ > queue name=outdsp_queue ! \ > autoaudiosink name=autoaudiosink sync=true > slave-method=resample \ > \ > sink_tee. ! \ > queue name=file_queue ! \ > audioconvert name=fileconvert ! \ > filesink name=file_filesink location=/dev/null \ > \ > udpsrc name=src1_src port=${PORT_SRC1} do-timestamp=true ! \ > audio/x-raw,format=S16LE,channels=2,rate=32000 ! \ > queue name=src1_queue min-threshold-time=0 ! \ > audiorate name=src1_audiorate tolerance=256000000 > skip-to-first=false ! \ > audioconvert name=src1_audioconvert ! \ > audioresample name=src1_audioresample quality=1 ! \ > audio/x-raw,format=S16LE,channels=2,rate=48000 ! \ > tee name=src1_tee \ > \ > src1_tee. ! \ > volume name=src1_volume volume=0.5 ! \ > sink_audiomixer. \ > \ > udpsrc name=src2_src port=${PORT_SRC2} do-timestamp=true ! \ > audio/x-raw,format=S16LE,channels=2,rate=48000 ! \ > queue name=src2_queue min-threshold-time=0 ! \ > audiorate name=src2_audiorate tolerance=256000000 > skip-to-first=false ! \ > audioconvert name=src2_audioconvert ! \ > audioresample name=src2_audioresample quality=4 ! \ > audio/x-raw,format=S16LE,channels=2,rate=48000 ! \ > tee name=src2_tee \ > \ > src2_tee. ! \ > volume name=src2_volume volume=0.5 ! \ > sink_audiomixer. > > > *My SRC1 audio transmitter (32000 - 50Hz):* > gst-launch-1.0 audiotestsrc freq=100 ! > audio/x-raw,format=S16LE,channels=2,rate=31950 ! udpsink host="127.0.0.1" > port=41001 > > *My SRC2 audio transmitter (48000 + 50Hz):* > gst-launch-1.0 audiotestsrc freq=300 ! > audio/x-raw,format=S16LE,channels=2,rate=48050 ! udpsink host="127.0.0.1" > port=42001 > > When I run these pipelines with even the latest gst version, then I hear > "clicks" from time to time. > I expect, that the audiomixer has some problems with the differ sampling > frequencies. > > My assumption is, I need a asynchronous SRC for each input > (src1_audioresample+src2_audioresample). > How can I implement this with available elements. Or if not available, what > is the best way to implement these? > > Why is the audiomixer located in the bad plugins? What means bad? > > > > > -- > View this message in context: http://gstreamer-devel.966125.n4.nabble.com/Asynchrounous-Audio-Sample-Rate-Conversion-with-gstreamer-tp4677252p4677765.html > Sent from the GStreamer-devel mailing list archive at Nabble.com. > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Hi,
My pipeline simplified: SRC1: sine with 32kHz+50Hz=> NETWORK=> udpsrc(expected sample rate is 32kHz)=> queue=> audiorate=> audioconvert=> audioresample=>| |audiomixer=>queue=>autoaudiosink(sync=true,slave-method=resample)=>Audio-Out audioresample=>| audioconvert=> audiorate=> queue=> udpsrc(expected sample rate is 48kHz)=> NETWORK=> SRC2: sine with 48kHz-50Hz=> >You can hook in your own drift compensation method if you set the >audiobasesink's "slave-method" property to "custom". We use that in a >project to finetune the audio clock's speed via a PLL by an integer >number of ppm. It would be the same with an ASRC. Sorry, I have some concerns about this proposal. How should this compensate the clock drifts between (Audio-Out) and two different inputs (SRC1 and SRC2)? My expectation is, that I need something in the SRC pipeline parts. This compensates each SRC clock to the main pipeline clock. The ASRC in the basesink/autoaudiosink is responsible to synchronize the main pipeline clock to the sink clock. In the gstreamer audiomixer manual: >Unlike the adder element audiomixer properly synchronises all input streams. What does this mean in this context? Is there a ASRC inside the audiomixer? An idea for would be adding a custom element in front of each "audioresample" element. This "ASRC-SampleRate-Checker", could observe the SRCx buffer timestamps and calculate the real input sample rate. By knowing this, the element could adapt/change the rate in the output caps in a way, that the following audioresample is compensating the source clock difference. Could this work (inaudible)? If there is any other solution for this, I am happy for all proposals. Maik |
Free forum by Nabble | Edit this page |