webrtcdsp echo cancellation not working
Locked
 
|
Hi! I'm trying to use webrtcdsp echo cancellation with my Raspberry Pi based project. I'm using the Janus server for WebRTC interaction and GStreamer for AV processing. Here's my GStreamer pipeline:
gst-launch-1.0 \ rtpbin name=rtpbin latency=100 \ udpsrc port=50000 caps="application/x-rtp, media=audio, encoding-name=OPUS, clock-rate=48000" ! rtpbin.recv_rtp_sink_0 \ udpsrc port=50001 caps="application/x-rtcp" ! rtpbin.recv_rtcp_sink_0 \ rtpbin. ! rtpopusdepay ! queue ! opusdec ! audioresample ! audio/x-raw,format=S16LE,layout=interleaved,rate=16000,channels=1 ! webrtcechoprobe ! alsasink \ alsasrc device=plughw:0,0 ! audio/x-raw,format=S16LE,layout=interleaved,rate=16000,channels=1 ! volume volume=10 ! volume volume=3 ! webrtcdsp delay-agnostic=true noise-suppression-level=high ! \ opusenc ! rtpopuspay ! udpsink host=127.0.0.1 port=8005 I'm using rtpbin to receive audio data from remote clients which gets played back using the default alsa device. It gets forwarded over the local UDP port from Janus. I'm also recording from the default alsa device and sending it to a local UDP port to get picked up by Janus. I'm using an I2S MEMS microphone which by default has extremely low volume coming from the device but the volume boost gets me very clear audio. As you can see, the echo probe is located right before the sound gets pumped out to the alsasink. The actual DSP is applied right after the initial volume boost. I know the DSP is working because I can hear the noise cancellation doing its job. However, the echo canceller itself seems to not be working at all. I tried adjusting the playback volume with no effect. I also tried placing the DSP before the volume boost, also to no effect. Lastly, I tried enabling delay-agnostic, but that also didn't affect the result. I'm wondering if the DSP's and the echo probe's timing is not in sync, but I'm not sure how to adjust that. Any advice would be greatly appreciated! _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: webrtcdsp echo cancellation not working
|
|
Le samedi 16 janvier 2021 à 19:54 -0500, Nik Ivanov a écrit :
Even with "delay-agnostic" mode set, the default latency for alsasink i sway too high to remove echo. I would suggest to reduce the "buffer-time", which is 200ms by default to something saner like 80ms (80000).
About delay-agnostic, you may want to retry without it, as when it works (when the alsa driver has precise enough delay report), it will start removing echo immediatly.
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
Thank you! Buffer-time was definitely the key here. However, I had to set the
buffer-time to 20ms (20000) before I could get semi-acceptable results. And it's still just a little bit too much - it catches a good chunk of the echo, but I can still hear some parts. The size of the parts shrink down as the delay goes down, but for some reason, if I set the buffer-time to something less than 20000, my stream stops recording altogether. It doesn't crash, but it does not record any sound whatsoever: gst-launch-1.0 \ rtpbin name=rtpbin latency=100 \ udpsrc port=50000 caps="application/x-rtp, media=audio, encoding-name=OPUS, clock-rate=48000" ! rtpbin.recv_rtp_sink_0 \ udpsrc port=50001 caps="application/x-rtcp" ! rtpbin.recv_rtcp_sink_0 \ rtpbin. ! rtpopusdepay ! queue ! opusdec ! audioresample ! audio/x-raw,format=S16LE,layout=interleaved,rate=16000,channels=1 ! webrtcechoprobe ! alsasink \ alsasrc device=plughw:0,0 buffer-time=20000 ! audio/x-raw,format=S16LE,layout=interleaved,rate=16000,channels=1 ! webrtcdsp noise-suppression-level=high echo-suppression-level=high ! volume volume=10 ! volume volume=3 ! \ queue ! opusenc ! rtpopuspay ! udpsink host=127.0.0.1 port=8005 Note that I increased the echo-suppresion-level to high and also put the volume amplification after the DSP - this allowed me to have the speaker at the maximum volume while maintaining acceptable results. Do you know why buffer-time cannot be set below 20000? I also tried enabling delay-agnostic, but that made things a lot worse - I think I'm better off leaving that flag off. Thank you for your help so far! -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: webrtcdsp echo cancellation not working
|
|
Le dim. 17 janv. 2021 02 h 15, nivanov <[hidden email]> a écrit : Thank you! Buffer-time was definitely the key here. However, I had to set the The buffer-time must at least be twice the latency-time for things to work.
Here you configure the alsasrc, I was particularly aiming at alsasink in my comment. I realise it wasn't clear. audio/x-raw,format=S16LE,layout=interleaved,rate=16000,channels=1 ! Any modifications to the stream (volume, resampling format conversion) may distort the signal and reduce echo cancellation efficiency. So volume after is just the right and logical thing to do. why
I also tried enabling delay- agnosticccccccc,
_______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
I tried setting buffer-time on alsasink but the best results so far are
alsasrc having buffer-time of 20000 and alsasink having default buffer-time. I was expecting that those two can compensate for one another - i could either decrease latency on recording, or decrease latency on playback - either way, the probe and the DSP get the same time shift. However, that doesn't seem to be the case: my best result so far is decreasing recording delay by 180ms (default of 200000 - 20000). I tried splitting the difference and setting buffer-time of alsasrc to 110000 (default of 200000 - 180000/2) and setting same 110000 buffer-time on alsasink, but the results are bad. I thought maybe I'm not understanding the time skew correctly, so maybe I have to increase the sink time, so I tried 290000 buffer-time on alsasink but the results were also bad. All this led me to a thought - wouldn't it make sense for the DSP to have a setting for the echo delay? You could specify the time difference between when the probe captures audio and when recorded audio hits DSP. Also, it's probably worth noting that I'm using: I2S Mic: https://www.seeedstudio.com/Sipeed-I2S-Mic-for-MAIX-Dev-Boards-p-2887.html I2S Amp: https://learn.adafruit.com/adafruit-max98357-i2s-class-d-mono-amp all with googlevoicehat-soundcard dt-overlay driver. I suspect that there might be some additional buffering / delays happening on that level as well, or maybe even just the I2S bus. -- Sent from: http://gstreamer-devel.966125.n4.nabble.com/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
«
Return to GStreamer-devel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |