MPEG-TS and Rust
Locked
MPEG-TS and Rust
 
|
Hi,
I am finding that from Rust code, via gstreamer_mpegts_sys, gst_mpegts_descriptor_parse_dvb_component works fine if there is no text in the component descriptor. However as soon as there is any text in the content descriptor then I get a SIGSEGV from, according to the CLion stack trace: __gconv iconv g_convert_with_iconv ??? ??? gst_mpegts_descriptor_parse_dvb_component Has anyone any idea how to find out why there is a SIGSEGV, the code is just supposed to collect together n (known) bytes into a nul terminated C string. I am guessing there are no tests for this C code. :-( -- Russel. =========================================== Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: MPEG-TS and Rust
|
|
On Mon, 2019-07-15 at 20:04 +0100, Russel Winder wrote:
> Hi, > > I am finding that from Rust code, via gstreamer_mpegts_sys, > gst_mpegts_descriptor_parse_dvb_component works fine if there is no text in > the component descriptor. However as soon as there is any text in the content > descriptor then I get a SIGSEGV from, according to the CLion stack trace: > > __gconv > iconv > g_convert_with_iconv > ??? > ??? > gst_mpegts_descriptor_parse_dvb_component > > Has anyone any idea how to find out why there is a SIGSEGV, the code is just > supposed to collect together n (known) bytes into a nul terminated C string. what goes on here :) You'd need to provide at least a full backtrace with line numbers and the input, or even better a testcase to reproduce the crash you're seeing. > I am guessing there are no tests for this C code. :-( No, but there's tests/examples/mpegts/ts-parser.c at least using this. -- Sebastian Dröge, Centricular Ltd · https://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: MPEG-TS and Rust
|
|
On Mon, 2019-07-15 at 22:24 +0300, Sebastian Dröge wrote:
[…] > > With that little context, one would need a crystal ball to understand > what goes on here :) Actually I thought I had explained quite well. However… > You'd need to provide at least a full backtrace with line numbers and > the input, or even better a testcase to reproduce the crash you're > seeing. I suspect it will never be possible to get a full stack trace with line numbers because the project is in Rust calling a compiled shared library. Two testcase codes is however really straightforward. I have attached the two files. > No, but there's tests/examples/mpegts/ts-parser.c at least using this. Indeed, that's the other thread. :-) -- Russel. =========================================== Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel  fails.rs (1K) Download Attachment succeeds.rs (1K) Download Attachment signature.asc (849 bytes) Download Attachment |
Re: MPEG-TS and Rust
|
|
Re: MPEG-TS and Rust
|
|
In reply to this post by Russel Winder
On Mon, 2019-07-15 at 22:22 +0100, Russel Winder wrote:
> On Mon, 2019-07-15 at 22:24 +0300, Sebastian Dröge wrote: > […] > > With that little context, one would need a crystal ball to understand > > what goes on here :) > > Actually I thought I had explained quite well. However… Thanks. The problem here is that you don't provide valid data to the descriptor, please check the MPEG-TS and DVB specs or at least the parsing code. One place where this is described is ETSI EN 300 468 in annex A. The first byte of the text is supposed to signify the charset encoding that is used (and it might be more than a single byte). Of course this shouldn't crash if random other data is found there nonetheless. Can you create an issue with your testcase? > > You'd need to provide at least a full backtrace with line numbers and > > the input, or even better a testcase to reproduce the crash you're > > seeing. > > I suspect it will never be possible to get a full stack trace with line > numbers because the project is in Rust calling a compiled shared library. That seems like a problem in your setup. You can get proper stack traces with gdb/lldb and even MSVC on Windows in mixed Rust/C/C++ projects. -- Sebastian Dröge, Centricular Ltd · https://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: MPEG-TS and Rust
|
|
In reply to this post by Nirbheek Chauhan
On Tue, 2019-07-16 at 03:00 +0530, Nirbheek Chauhan wrote:
> […] > > Both gdb and Visual Studio are able to generate backtraces in code > that calls into Rust and/or C. You just have to ensure that all your > code is built with debugging options enabled. > I think the problem is that I am using production GStreamer (Debian packages 1.16.0) and it comes without debugging symbols. I have a request in the the JetBrains CLion folk to see if they have any thoughts on getting proper stack traces from C code called by Rust code. -- Russel. =========================================== Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: MPEG-TS and Rust
|
|
In reply to this post by Sebastian Dröge-3
On Tue, 2019-07-16 at 09:23 +0300, Sebastian Dröge wrote:
> […] > > Thanks. The problem here is that you don't provide valid data to the > descriptor, please check the MPEG-TS and DVB specs or at least the > parsing code. One place where this is described is ETSI EN 300 468 in > annex A. I am using ETSI EN 300 468 V1.15.1 (2016-03) as my reference for the packets. > The first byte of the text is supposed to signify the charset encoding > that is used (and it might be more than a single byte). The standard says Appendix A.0 that if the first character of the string is not an encoding byte then there is a default encoding, Latin. Thus my data conforms to the standard as far as I know. > Of course this shouldn't crash if random other data is found there > nonetheless. Can you create an issue with your testcase? OK. Later today though, I have to take my car in for a recall update. […] > That seems like a problem in your setup. You can get proper stack > traces with gdb/lldb and even MSVC on Windows in mixed Rust/C/C++ > projects. It would seen that the Debina packages of gstreamer 1.16.0 do not come with debugging symbols, at least that would be my guess as to why (*). I am solely on Linux and have no access to Windows or MSVC. I have a request in to the JetBrains CLion folk on this issue – CLion should be able to deal with C calls from Rust, but only as long as the debic symbols are available. (*) I be I have just failed to install the debug symbol packages. Have you moved gstreamer 1.16.0 from experimental to sid now that the Debian freeze is over? -- Russel. =========================================== Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: MPEG-TS and Rust
|
|
In reply to this post by Russel Winder
On Tue, 2019-07-16 at 09:24 +0100, Russel Winder wrote:
> On Tue, 2019-07-16 at 03:00 +0530, Nirbheek Chauhan wrote: > […] > > Both gdb and Visual Studio are able to generate backtraces in code > > that calls into Rust and/or C. You just have to ensure that all your > > code is built with debugging options enabled. > > > > I think the problem is that I am using production GStreamer (Debian packages > 1.16.0) and it comes without debugging symbols. The debug symbols are available on Debian via additional debug packages that you can install. -- Sebastian Dröge, Centricular Ltd · https://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: MPEG-TS and Rust
|
|
In reply to this post by Russel Winder
On Tue, 2019-07-16 at 09:32 +0100, Russel Winder wrote:
> On Tue, 2019-07-16 at 09:23 +0300, Sebastian Dröge wrote: > > […] > > > > Thanks. The problem here is that you don't provide valid data to the > > descriptor, please check the MPEG-TS and DVB specs or at least the > > parsing code. One place where this is described is ETSI EN 300 468 in > > annex A. > > I am using ETSI EN 300 468 V1.15.1 (2016-03) as my reference for the packets. > > > The first byte of the text is supposed to signify the charset encoding > > that is used (and it might be more than a single byte). > > The standard says Appendix A.0 that if the first character of the string is > not an encoding byte then there is a default encoding, Latin. Thus my data > conforms to the standard as far as I know. 6937 in that case. Unclear why it still crashes, needs an issue in GitLab and someone to debug it. The function is also not handling the control characters from A.1 as far as I can see, that should probably also be fixed. > (*) I be I have just failed to install the debug symbol packages. Have you > moved gstreamer 1.16.0 from experimental to sid now that the Debian freeze is > over? Had no time for that yet, sorry. -- Sebastian Dröge, Centricular Ltd · https://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: MPEG-TS and Rust
|
|
In reply to this post by Sebastian Dröge-3
On Tue, 2019-07-16 at 11:58 +0300, Sebastian Dröge wrote:
> […] > The debug symbols are available on Debian via additional debug packages > that you can install. Currently these are 1.14.? on Sid not the 1.16.0 on Experimental, will 1.16.0 migrate to Sid soon? -- Russel. =========================================== Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: MPEG-TS and Rust
|
|
In reply to this post by Sebastian Dröge-3
On Tue, 2019-07-16 at 12:03 +0300, Sebastian Dröge wrote:
> […] > > Indeed, and the function is actually doing that and defaulting to ISO > 6937 in that case. Unclear why it still crashes, needs an issue in > GitLab and someone to debug it. I'll add an issue attaching the source code. I'll see if I can get a better stack trace. > The function is also not handling the control characters from A.1 as > far as I can see, that should probably also be fixed. I hadn't come across that, I only use Latin or UTF-8. > > (*) I be I have just failed to install the debug symbol packages. Have you > > moved gstreamer 1.16.0 from experimental to sid now that the Debian freeze > > is > > over? > > Had no time for that yet, sorry. OK. No rush for me I have pulled in the debug packages from Experimental. -- Russel. =========================================== Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: MPEG-TS and Rust
|
|
On Tue, 2019-07-16 at 16:57 +0100, Russel Winder wrote:
> On Tue, 2019-07-16 at 12:03 +0300, Sebastian Dröge wrote: > > […] > > > > Indeed, and the function is actually doing that and defaulting to ISO > > 6937 in that case. Unclear why it still crashes, needs an issue in > > GitLab and someone to debug it. > > I'll add an issue attaching the source code. I'll see if I can get a better > stack trace. I shall refrain from adding an issue, at least due to the current problem I had. It turns out that my problem was that gst_mpegts_sys::gst_mpegts_initialize was not being called. So clearly the initialisation is doing something very profound affecting text processing. To put it another way, I have been an idiot. -- Russel. =========================================== Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: MPEG-TS and Rust
|
|
On Tue, 2019-07-16 at 19:45 +0100, Russel Winder wrote:
> On Tue, 2019-07-16 at 16:57 +0100, Russel Winder wrote: > > On Tue, 2019-07-16 at 12:03 +0300, Sebastian Dröge wrote: > > > […] > > > > > > Indeed, and the function is actually doing that and defaulting to ISO > > > 6937 in that case. Unclear why it still crashes, needs an issue in > > > GitLab and someone to debug it. > > > > I'll add an issue attaching the source code. I'll see if I can get a > > better > > stack trace. > > I shall refrain from adding an issue, at least due to the current problem I > had. > > It turns out that my problem was that gst_mpegts_sys::gst_mpegts_initialize > was not being called. So clearly the initialisation is doing something very > profound affecting text processing. > > To put it another way, I have been an idiot. initialisation! -- Russel. =========================================== Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: MPEG-TS and Rust
|
|
On Tue, 2019-07-16 at 19:55 +0100, Russel Winder wrote:
> On Tue, 2019-07-16 at 19:45 +0100, Russel Winder wrote: > > On Tue, 2019-07-16 at 16:57 +0100, Russel Winder wrote: > > > On Tue, 2019-07-16 at 12:03 +0300, Sebastian Dröge wrote: > > > > […] > > > > > > > > Indeed, and the function is actually doing that and defaulting to ISO > > > > 6937 in that case. Unclear why it still crashes, needs an issue in > > > > GitLab and someone to debug it. > > > > > > I'll add an issue attaching the source code. I'll see if I can get a > > > better > > > stack trace. > > > > I shall refrain from adding an issue, at least due to the current problem I > > had. > > > > It turns out that my problem was that gst_mpegts_sys::gst_mpegts_initialize > > was not being called. So clearly the initialisation is doing something very > > profound affecting text processing. > > > > To put it another way, I have been an idiot. > > I guess the question is why the success code worked without the > initialisation! -- Sebastian Dröge, Centricular Ltd · https://www.centricular.com _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: MPEG-TS and Rust
|
|
On Tue, 2019-07-16 at 23:03 +0300, Sebastian Dröge wrote:
> […] > > It was not doing any charset encoding processing. "statin the bleedin' obvious" :-) I have looked at the source yet, I'll try to later today, but there is a real question of why you need initialisation in order for charset processing not to SIGSEGV. I would never have connected this SIGSEGV with lack of initialisation without a lot of luck. -- Russel. =========================================== Dr Russel Winder t: +44 20 7585 2200 41 Buckmaster Road m: +44 7770 465 077 London SW11 1EN, UK w: www.russel.org.uk _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
| Free forum by Nabble | Edit this page |