DVB Tables

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

DVB Tables

Russel Winder
I thought I'd take another go at getting an EPG from a DVB-T/DVB-T2
device. I am sure there are many instances of doing this sensibly and
even in C++, but Google seems not to have indexed them. So not a lot of
code to "rip off" :-)

I have EN 300 468 1.15.1 which is dry and boring, but contains the
definite statement of what should be.

Reading  https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-
plugins-bad-libs/html/gst-plugins-bad-libs-DVB-variants-of-MPEG-TS-
sections.html seems to indicate that the GStreamer EIT section for
example is not structured as the EIT table is specified, and all the
names are a bit different. This apparent lack of consistency between
the standard and the GStreamer stuff is making this much harder work
that I thought it should be.

Unless I am just misunderstanding.


--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:[hidden email]
41 Buckmaster Road    m: +44 7770 465 077   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: DVB Tables

Nuno Mota
I would suggest looking into dvblast. It will give you EIT tables in XML format under a C source code.
I've done a sample to extract some of the EIT tags to a file a couple of years ago, that's the best way I found at the time.

Been using this ever since.

Russel Winder <[hidden email]> escreveu no dia quinta, 25/05/2017 às 13:44:
I thought I'd take another go at getting an EPG from a DVB-T/DVB-T2
device. I am sure there are many instances of doing this sensibly and
even in C++, but Google seems not to have indexed them. So not a lot of
code to "rip off" :-)

I have EN 300 468 1.15.1 which is dry and boring, but contains the
definite statement of what should be.

Reading  https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-
plugins-bad-libs/html/gst-plugins-bad-libs-DVB-variants-of-MPEG-TS-
sections.html
seems to indicate that the GStreamer EIT section for
example is not structured as the EIT table is specified, and all the
names are a bit different. This apparent lack of consistency between
the standard and the GStreamer stuff is making this much harder work
that I thought it should be.

Unless I am just misunderstanding.


--
Russel.
=============================================================================
Dr Russel Winder      t: <a href="tel:+44%2020%207585%202200" value="+442075852200" target="_blank">+44 20 7585 2200   voip: [hidden email]
41 Buckmaster Road    m: <a href="tel:+44%207770%20465077" value="+447770465077" target="_blank">+44 7770 465 077   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: DVB Tables

Russel Winder
On Thu, 2017-05-25 at 18:32 +0000, Nuno Mota wrote:

> I would suggest looking into dvblast. It will give you EIT tables in
> XML
> format under a C source code.
> I've done a sample to extract some of the EIT tags to a file a couple
> of
> years ago, that's the best way I found at the time.
>
> Been using this ever since.
> https://mailman.videolan.org/pipermail/dvblast-devel/2013-October/001
> 232.html
>
Certainly could be useful for a check of correctness of the extraction
of an EIT, but the code is VideoLAN rather than GStreamer so can't
really give an insight into how the GStreamer sections need to be
handled to get the data specified in the standard.

GNOME DVBDaemon was a code I was looking at but it is Vala and doesn't
deal with the streams directly in code it just creates a shell pipeline
as a subprocess and handles the results from that.

--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:[hidden email]
41 Buckmaster Road    m: +44 7770 465 077   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: DVB Tables

Edward Hervey-4
In reply to this post by Russel Winder
Hi,

On Thu, 2017-05-25 at 13:38 +0100, Russel Winder wrote:

> I thought I'd take another go at getting an EPG from a DVB-T/DVB-T2
> device. I am sure there are many instances of doing this sensibly and
> even in C++, but Google seems not to have indexed them. So not a lot
> of
> code to "rip off" :-)
>
> I have EN 300 468 1.15.1 which is dry and boring, but contains the
> definite statement of what should be.
>
> Reading  https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gs
> t-
> plugins-bad-libs/html/gst-plugins-bad-libs-DVB-variants-of-MPEG-TS-
> sections.html seems to indicate that the GStreamer EIT section for
> example is not structured as the EIT table is specified, and all the
> names are a bit different. This apparent lack of consistency between
> the standard and the GStreamer stuff is making this much harder work
> that I thought it should be.
>
> Unless I am just misunderstanding.
  I did my best to make it a 1:1 mapping with the field naming in the
specs. Which part don't you think is similar ?

   Edward

>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: DVB Tables

Russel Winder
On Fri, 2017-05-26 at 08:01 +0200, Edward Hervey wrote:
>
[…]
>   I did my best to make it a 1:1 mapping with the field naming in the
> specs. Which part don't you think is similar ?
>

It may well be then that I am just misunderstanding – actually I hope
so, as it will easier to fix that. :-)

The standard presents an EIT section as follows (Table 7 of the
standard):

    event_information_section(){
      table_id
      section_syntax_indicator
      reserved_future_use
      reserved
      section_length
      service_id
      reserved
      version_number
      current_next_indicator
      section_number
      last_section_number
      transport_stream_id
      original_network_id
      segment_last_section_number
      last_table_id
      for(i=0;i<N;i++){
        event_id
        start_time
        duration
        running_status
        free_CA_mode
        descriptors_loop_length
        for(i=0;i<N;i++){
          descriptor()
        }
      }
      CRC_32
    }

The GstMpegtsEIT structure is though:

    struct GstMpegtsEIT {
      guint16        transport_stream_id;
      guint16        original_network_id;
      guint8         segment_last_section_number;
      guint8         last_table_id;

      gboolean       actual_stream;
      gboolean       present_following;

      GPtrArray     *events;
    };

which doesn't seem to map to any bits of the standard's Table 7 –
though it is clear that the events, a sequence of GstMpegtsEITEvent
structures does somewhat map to the N events in the section:

    struct GstMpegtsEITEvent {
      guint16      event_id;

      GstDateTime *start_time;
      guint32      duration;

      GstMpegtsRunningStatus running_status;
      gboolean     free_CA_mode;

      GPtrArray   *descriptors;
    };

My code is getting the events, seemingly correctly (*), but it is the
SID and stuff like that that give the channel numbers that I cannot see
how to extract. The EIT section should have the service_id, but the
GstMpegtsEIT structure does not appear to offer it.

I guess I had expected the GStreamer structures to map more or less
exactly to the tables in the standard, but this seems to be only
partially true. Where it is true, it is easier to work things out.
Where it apparently is not the case, I am at a bit of a loss as to how
to proceed processing the events on the bus so as to collate the data
into an in-memory representation of the EIT.



(*) Though I regularly get memory errors due to all the malloc and free
stuff that I really do not properly understand yet. Oh for doing this
in D with it's garbage collector.

--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:[hidden email]
41 Buckmaster Road    m: +44 7770 465 077   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: DVB Tables

Edward Hervey-4
Hi,

On Fri, 2017-05-26 at 07:31 +0100, Russel Winder wrote:

> On Fri, 2017-05-26 at 08:01 +0200, Edward Hervey wrote:
> >
>
> […]
> >   I did my best to make it a 1:1 mapping with the field naming in
> > the
> > specs. Which part don't you think is similar ?
> >
>
> It may well be then that I am just misunderstanding – actually I hope
> so, as it will easier to fix that. :-)
>
> The standard presents an EIT section as follows (Table 7 of the
> standard):
>
>     event_information_section(){
>       table_id
>       section_syntax_indicator
>       reserved_future_use
>       reserved
>       section_length
>       service_id
>       reserved
>       version_number
>       current_next_indicator
>       section_number
>       last_section_number
>       transport_stream_id
>       original_network_id
>       segment_last_section_number
>       last_table_id
>       for(i=0;i<N;i++){
>         event_id
>         start_time
>         duration
>         running_status
>         free_CA_mode
>         descriptors_loop_length
>         for(i=0;i<N;i++){
>           descriptor()
>         }
>       }
>       CRC_32
>     }
>
> The GstMpegtsEIT structure is though:
>
>     struct GstMpegtsEIT {
>       guint16        transport_stream_id;
>       guint16        original_network_id;
>       guint8         segment_last_section_number;
>       guint8         last_table_id;
>
>       gboolean       actual_stream;
>       gboolean       present_following;
>
>       GPtrArray     *events;
>     };
>
> which doesn't seem to map to any bits of the standard's Table 7
  All the standard bits are in the GstMpegtsSection : https://gstreamer
.freedesktop.org/data/doc/gstreamer/head/gst-plugins-bad-libs/html/gst-
plugins-bad-libs-Base-MPEG-TS-sections.html#GstMpegtsSection-struct

  Just get them directly :)
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: DVB Tables

Russel Winder
On Fri, 2017-05-26 at 09:54 +0200, Edward Hervey wrote:
> […]
>
>   All the standard bits are in the GstMpegtsSection : https://gstream
> er
> .freedesktop.org/data/doc/gstreamer/head/gst-plugins-bad-
> libs/html/gst-
> plugins-bad-libs-Base-MPEG-TS-sections.html#GstMpegtsSection-struct
>
>   Just get them directly :)

But that means processing all sections requiring a full-blown state
machine to correctly put together all the sections for all the tables.
I had hoped the GStreamer bus and the section type system would have
made things easier than that.

--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:[hidden email]
41 Buckmaster Road    m: +44 7770 465 077   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: DVB Tables

Baby Octopus
Administrator
Have you looked into ts-parser to see if it gives you all the info? https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/mpegts

~BO
Reply | Threaded
Open this post in threaded view
|

Re: DVB Tables

Edward Hervey-4
Thanks :)

  Was going to point to that. Hopefully it should make the usage
clearer.

   Edward

On Fri, 2017-05-26 at 02:23 -0700, Baby Octopus wrote:

> Have you looked into ts-parser to see if it gives you all the info?
> https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/exa
> mples/mpegts
>
> ~BO
>
>
>
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble
> .com/DVB-Tables-tp4683104p4683123.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: DVB Tables

Russel Winder
In reply to this post by Baby Octopus
On Fri, 2017-05-26 at 02:23 -0700, Baby Octopus wrote:
> Have you looked into ts-parser to see if it gives you all the info?
> https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/exa
> mples/mpegts
>

No I hadn't looked at that, I think that will be most helpful. Thank
you for pointing me to it.

As well as being helpful, it also proves C is a really dreadful
programming language for all this stuff. :-(

--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:[hidden email]
41 Buckmaster Road    m: +44 7770 465 077   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: DVB Tables

Russel Winder
In reply to this post by Edward Hervey-4
On Fri, 2017-05-26 at 11:52 +0200, Edward Hervey wrote:
> Thanks :)
>
>   Was going to point to that. Hopefully it should make the usage
> clearer.
>
>    Edward
>

Yes. And No.

I pulled out the ts-parser.c file, made a meson.build so as to create
an executable outside the plugins hierarchy. I ran it with a playbin
using the dvb URI scheme and it dumps out all the good stuff you'd
expect. And doesn't seem to fail – which is good. :-)

If in my code I just look at the DVB short event a lot of the time it
works but very quickly I get a memory allocation failure.

(me-tv:6718): GLib-ERROR **: /build/glib2.0-B1uXKV/glib2.0-2.50.3/./glib/gmem.c:165: failed to allocate 20401094656 bytes
Trace/breakpoint trap

Now whilst I didn't cut and paste the ts-parser C code I cannot see any
difference between it and what I have that would matter. So the
implication is that each EIT section on the bus cannot be treated
independently, that there is some non-local interaction happening – or
that there is something really weird with the whole descriptor system.

I am guessing that you really do have to fully process many different
messages to avoid the problem with the short event messages.


--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:[hidden email]
41 Buckmaster Road    m: +44 7770 465 077   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (849 bytes) Download Attachment