DTLS-SRTP handshake sometimes does not complete
Locked
 
|
Hi,
I am still trying to communicate GStreamer with FreeSwitch via WebRTC. It turned out that DTLS-SRTP handshake may not be completed properly, resulting in one-way audio (from GStreamer to FreeSwitch only). When my test app is run without debug logs, it reproduces in about 3 times per 4 attempts. With debug logs enabled on everything except GST_PADS, and piped via tee to file and stdout handshake was always completed, so it looks like some race condition for me. Here are packets exchanged between GST and FS: Good scenario: FS->GST: Binding Request GST->FS: Binding Success GST->FS: Binding Request FS->GST: Binding Success FS->GST: DTLS Handshake GST->FS: Binding Indication GST->FS: DTLS Handshake GST->FS: DTLS Handshake GST->FS: DTLS Handshake GST->FS: DTLS Handshake GST->FS: DTLS Handshake GST->FS: DTLS Handshake FS->GST: DTLS Handshake FS->GST: DTLS Handshake GST->FS: DTLS Handshake GST->FS: DTLS Handshake GST->FS: DTLS Handshake GST->FS: DTLS Handshake GST->FS: DTLS Handshake GST->FS: DTLS Handshake GST->FS: DTLS Change Cipher Spec GST->FS: SRTP/SRTCP FS->GST: SRTP/SRTCP ... more 2-way media here Bad scenario: FS->GST: Binding Request GST->FS: Binding Success GST->FS: Binding Request FS->GST: Binding Success FS->GST: DTLS Handshake GST->FS: Binding Indication GST->FS: DTLS Handshake GST->FS: DTLS Handshake GST->FS: DTLS Handshake GST->FS: DTLS Handshake GST->FS: DTLS Handshake GST->FS: DTLS Handshake FS->GST: DTLS Handshake FS->GST: DTLS Handshake GST->FS: SRTP/SRTCP ... more media from GST here In bad scenario GST did not sent 6 DTLS Handshake packets and DTLS Change Cipher Spec; instead it started sending media. I have uploaded logs from both scenarios there. https://pastebin.com/TF3gnK6A https://pastebin.com/EqkN4naP Is there any workaround for this? Regards, Daniel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
W dniu 2018-04-04 17:13, [hidden email] napisał(a):
> Hi, > I am still trying to communicate GStreamer with FreeSwitch via WebRTC. > It turned out that DTLS-SRTP handshake may not be completed properly, > resulting in one-way audio (from GStreamer to FreeSwitch only). When > my test app is run without debug logs, it reproduces in about 3 times > per 4 attempts. With debug logs enabled on everything except GST_PADS, > and piped via tee to file and stdout handshake was always completed, > so it looks like some race condition for me. > > Here are packets exchanged between GST and FS: > > Good scenario: > FS->GST: Binding Request > GST->FS: Binding Success > GST->FS: Binding Request > FS->GST: Binding Success > FS->GST: DTLS Handshake > GST->FS: Binding Indication > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > FS->GST: DTLS Handshake > FS->GST: DTLS Handshake > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > GST->FS: DTLS Change Cipher Spec > GST->FS: SRTP/SRTCP > FS->GST: SRTP/SRTCP > ... more 2-way media here > > > Bad scenario: > FS->GST: Binding Request > GST->FS: Binding Success > GST->FS: Binding Request > FS->GST: Binding Success > FS->GST: DTLS Handshake > GST->FS: Binding Indication > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > GST->FS: DTLS Handshake > FS->GST: DTLS Handshake > FS->GST: DTLS Handshake > GST->FS: SRTP/SRTCP > ... more media from GST here > > In bad scenario GST did not sent 6 DTLS Handshake packets and DTLS > Change Cipher Spec; instead it started sending media. > > I have uploaded logs from both scenarios there. > https://pastebin.com/TF3gnK6A > https://pastebin.com/EqkN4naP > > Is there any workaround for this? Did you have chance to look on this? I would like to get some hints here. I checked timestamps in logs and found that in successful scenario delay between first "SSL negotiation finished successfully" and first "Encoding RTP buffer of size 182" is about 1ms, and in unsuccessful one it is about 0.4ms. This supports hypothesis that there is some race condition there, and for some reason DTLS packets with New Session Ticket and other required records are not sent to the wire at all. I checked that gstdtlssrtpenc internally uses funnel block so all there is no hard switch which could cut off some packets, but maybe I missed something. I also checked code of gstdtlsconnection.c and found that custom BIO implemented there does not have flush operation implemented. Maybe it should be added there? I also found this issue logged against OpenSSL. Looks that it is not able to resend lost packets with New Session Ticket: https://github.com/openssl/openssl/issues/5409 In the GST logs I saw some entries that FreSwitch was trying to recover and sent extra DTLS Handshake packets, but again handshake did not complete. Regards, Daniel > > Regards, > Daniel > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
W dniu 2018-04-09 17:50, [hidden email] napisał(a):
> W dniu 2018-04-04 17:13, [hidden email] napisał(a): >> Hi, >> I am still trying to communicate GStreamer with FreeSwitch via WebRTC. >> It turned out that DTLS-SRTP handshake may not be completed properly, >> resulting in one-way audio (from GStreamer to FreeSwitch only). When >> my test app is run without debug logs, it reproduces in about 3 times >> per 4 attempts. With debug logs enabled on everything except GST_PADS, >> and piped via tee to file and stdout handshake was always completed, >> so it looks like some race condition for me. >> >> Here are packets exchanged between GST and FS: >> >> Good scenario: >> FS->GST: Binding Request >> GST->FS: Binding Success >> GST->FS: Binding Request >> FS->GST: Binding Success >> FS->GST: DTLS Handshake >> GST->FS: Binding Indication >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> FS->GST: DTLS Handshake >> FS->GST: DTLS Handshake >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> GST->FS: DTLS Change Cipher Spec >> GST->FS: SRTP/SRTCP >> FS->GST: SRTP/SRTCP >> ... more 2-way media here >> >> >> Bad scenario: >> FS->GST: Binding Request >> GST->FS: Binding Success >> GST->FS: Binding Request >> FS->GST: Binding Success >> FS->GST: DTLS Handshake >> GST->FS: Binding Indication >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> GST->FS: DTLS Handshake >> FS->GST: DTLS Handshake >> FS->GST: DTLS Handshake >> GST->FS: SRTP/SRTCP >> ... more media from GST here >> >> In bad scenario GST did not sent 6 DTLS Handshake packets and DTLS >> Change Cipher Spec; instead it started sending media. >> >> I have uploaded logs from both scenarios there. >> https://pastebin.com/TF3gnK6A >> https://pastebin.com/EqkN4naP >> >> Is there any workaround for this? > > Did you have chance to look on this? I would like to get some hints > here. > > I checked timestamps in logs and found that in successful scenario > delay between first "SSL negotiation finished successfully" and first > "Encoding RTP buffer of size 182" is about 1ms, and in unsuccessful > one it is about 0.4ms. This supports hypothesis that there is some > race condition there, and for some reason DTLS packets with New > Session Ticket and other required records are not sent to the wire at > all. I checked that gstdtlssrtpenc internally uses funnel block so all > there is no hard switch which could cut off some packets, but maybe I > missed something. > > I also checked code of gstdtlsconnection.c and found that custom BIO > implemented there does not have flush operation implemented. Maybe it > should be added there? > > I also found this issue logged against OpenSSL. Looks that it is not > able to resend lost packets with New Session Ticket: > https://github.com/openssl/openssl/issues/5409 > In the GST logs I saw some entries that FreSwitch was trying to > recover and sent extra DTLS Handshake packets, but again handshake did > not complete. > > Regards, > Daniel Hi all, I have more logs for this. I started tracking calls to selected socket functions, and SSL_do_handshake. It turned out that handshake packets are created and put in GStreamer buffers, but for some reason may not be sent to the wire. I have uploaded new log here: https://pastebin.com/E7HiAfd7 Note: for calls to sendmsg/recvmsg I log initial 4 bytes of data only. This is enough to distinguish STUN/DTLS/SRTP packets. Regards, Daniel > >> >> Regards, >> Daniel >> _______________________________________________ >> gstreamer-devel mailing list >> [hidden email] >> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
«
Return to GStreamer-devel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |