Rewrite of DVB related elements

[Russel Winder]

Hi,

I am in the process of rewriting (well nigh on starting from scratch)
Me TV (https://github/Me-TV/Me-TV ) to replace all the hardwired Linux
API and Xine/VLC/GStreamer for video rendering use with a much deeper
tie in with GStreamer, dropping all the Xine and VLC stuff. However,
GStreamer is based on Linux API use but mostly DVBv3 with a DVBv5 "hack
layer".  I believe this needs to be updated. However this would be a
revolutionary change since I believe the way forward on this is to use
libdvbv5 and not deal with the Linux API directly.

libdvbv5 (linuxtv.org/docs/libdvbv5) is maintained by the Linux API
team to provide a better API for applications to work with.

If anyone is interested in collaborating on this, feel free to contact
me directly.
--
Russel.
=============================================================================
Dr Russel Winder      t: [phone]   voip: sip:[hidden email]
41 Buckmaster Road    m: [phone]   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Do, 2016-03-24 at 15:21 +0000, Russel Winder wrote:

> Hi,
>
> I am in the process of rewriting (well nigh on starting from scratch)
> Me TV (https://github/Me-TV/Me-TV ) to replace all the hardwired Linux
> API and Xine/VLC/GStreamer for video rendering use with a much deeper
> tie in with GStreamer, dropping all the Xine and VLC stuff. However,
> GStreamer is based on Linux API use but mostly DVBv3 with a DVBv5 "hack
> layer".  I believe this needs to be updated. However this would be a
> revolutionary change since I believe the way forward on this is to use
> libdvbv5 and not deal with the Linux API directly.
>
> libdvbv5 (linuxtv.org/docs/libdvbv5) is maintained by the Linux API
> team to provide a better API for applications to work with.

If I'm not mistaken this would also mean that a) a lot of code from the
dvb plugin would disappear as it is handled in libdvbv5 already and b)
that we finally handle modern channel files. But would we still handle
old channel files?

Also it's important that the MPEG-TS bits are still exposed in the
normal way by the dvb elements, and not somehow handled/hidden by
libdvbv5.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Russel Winder]

On Fri, 2016-03-25 at 10:53 +0200, Sebastian Dröge wrote:

[…]

> If I'm not mistaken this would also mean that a) a lot of code from
> the
> dvb plugin would disappear as it is handled in libdvbv5 already and
> b)
> that we finally handle modern channel files. But would we still
> handle
> old channel files?

libdvbv5 has a DVBv3 compatibility mode so yes old zap format files are
handled.

Currently GStreamer is DVBv3 with a DVB-T2 added bit, that doesn't
really work. This whole thing definitely needs to be DVBv5 since this
would give usable DVB-T2. So the current code has to either be
rewritten or replaced for GStreamer to be useful going forward.

I appreciate the licence issue and have raised this on the libdvbv5
emailing list. If this cannot be resolved then GStreamer will have to
have a different implementation of the exact same code that is in
libdvbv5.

> Also it's important that the MPEG-TS bits are still exposed in the
> normal way by the dvb elements, and not somehow handled/hidden by
> libdvbv5.

libdvbv5 does have it's own MPEG-TS codes. So the GStreamer MPEG-TS
code is replicating libdvbv5 code for that just as for the frontend
managment. I am not clear yet if the libdvbv5 code is harmonious with
the needs of GStreamer – more research needed here.


--
Russel.
=============================================================================
Dr Russel Winder      t: [phone]   voip: sip:[hidden email]
41 Buckmaster Road    m: [phone]   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Sa, 2016-03-26 at 14:43 +0000, Russel Winder wrote:
> 
> I appreciate the licence issue and have raised this on the libdvbv5
> emailing list. If this cannot be resolved then GStreamer will have to
> have a different implementation of the exact same code that is in
> libdvbv5.

For background info, libdvbv5 is GPL licensed.

> >
> > Also it's important that the MPEG-TS bits are still exposed in the
> > normal way by the dvb elements, and not somehow handled/hidden by
> > libdvbv5.
> libdvbv5 does have it's own MPEG-TS codes. So the GStreamer MPEG-TS
> code is replicating libdvbv5 code for that just as for the frontend
> managment. I am not clear yet if the libdvbv5 code is harmonious with
> the needs of GStreamer – more research needed here.

That would definitely be a requirement. Our MPEG-TS code is of course
also used for other things than DVB, so it has to stay in any case and
we should try to use the same MPEG-TS code for everything.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Russel Winder]

On Sat, 2016-03-26 at 20:20 +0200, Sebastian Dröge wrote:
[…]
> That would definitely be a requirement. Our MPEG-TS code is of course
> also used for other things than DVB, so it has to stay in any case
> and
> we should try to use the same MPEG-TS code for everything.

I guess the question is whether the libdvbv5 MPEG-TS code works for all
GStreamer cases. Maybe not, in which case a parallel implementation
would be appropriate. I am guessing the current libdvbv5 and GStreamer
implementations are more or less identical in terms of processing
content since it is all standard driven.

One thing worrying me just now is that the DVB EIT data structures in
GStreamer seem different to the libdvbv5 ones and the latter reflect
the standard whilst the former appear not to. I need to check stuff in
more detail though. This is next item on the agenda once I have the
tuning event handling working as required.

--
Russel.
=============================================================================
Dr Russel Winder      t: [phone]   voip: sip:[hidden email]
41 Buckmaster Road    m: [phone]   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On So, 2016-03-27 at 15:11 +0100, Russel Winder wrote:

> On Sat, 2016-03-26 at 20:20 +0200, Sebastian Dröge wrote:
> […]
> >
> > That would definitely be a requirement. Our MPEG-TS code is of course
> > also used for other things than DVB, so it has to stay in any case
> > and
> > we should try to use the same MPEG-TS code for everything.
> I guess the question is whether the libdvbv5 MPEG-TS code works for all
> GStreamer cases. Maybe not, in which case a parallel implementation
> would be appropriate. I am guessing the current libdvbv5 and GStreamer
> implementations are more or less identical in terms of processing
> content since it is all standard driven.

Note that MPEG-TS is not just DVB. And most of the MPEG-TS code used by
the DVB elements is shared with the demuxers, which handles all kinds
of MPEG-TS streams.

> One thing worrying me just now is that the DVB EIT data structures in
> GStreamer seem different to the libdvbv5 ones and the latter reflect
> the standard whilst the former appear not to. I need to check stuff in
> more detail though. This is next item on the agenda once I have the
> tuning event handling working as required.

What does different mean here? The content of the EIT, or the
structure? Do you have some examples?

Might be just a bug in GStreamer.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Russel Winder]

On Sun, 2016-03-27 at 19:20 +0300, Sebastian Dröge wrote:
[…]
> Note that MPEG-TS is not just DVB. And most of the MPEG-TS code used
> by
> the DVB elements is shared with the demuxers, which handles all kinds
> of MPEG-TS streams.

I would hope the libdvbv5 implementation was of full MPEG-TS and not
just DVB usage. However given GStreamer already has a MPEG-TS
implementation, there is less pressure to switch to using libdvbv5.

The dvbsrc code definitely needs a rewrite to bring it up to date with
the Linux API. This debate is rapidly moving us to re-implementing what
is in libdvbv5, not least because of the licencing issue.

[…]
> What does different mean here? The content of the EIT, or the
> structure? Do you have some examples?
>
> Might be just a bug in GStreamer.

The structs representing the packet data structures appear not, at
first sight, to reflect the data structures as per the standard. I will
investigate this further in the next few days now that I have the
tuning callbacks working.

(Though I am having some problems with "tuning-timeout" property
proxied by dvbbasebin.)

--
Russel.
=============================================================================
Dr Russel Winder      t: [phone]   voip: sip:[hidden email]
41 Buckmaster Road    m: [phone]   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Russel Winder]

On Sat, 2016-03-26 at 20:20 +0200, Sebastian Dröge wrote:
> […]
> For background info, libdvbv5 is GPL licensed.

The Linux Media email list members are not picking up on my question
about licencing, but it has been an Easter break for the bulk of them
so I guess leave it a few more days to see if anyone addresses the
issue.

If libdvbv5 remains GPLv2 then clearly a reimplementation of the DVBv5
"middleware" is required for GStreamer since it require LGPL 2.1 or
more permissive dependencies. As with libdvbv5 this will have to be
principally DVBv5 but with a DVBv3 backward compatibility layer.

libdvbv5 has to be pure C with no dependencies other than libc. This
leads to what appears to be very primitive (i.e. 1960s) ways of
realizing some algorithms. Given GStreamer relies on GLib, a GStreamer
implementation of the DVBv5 algorithms would obviously be realized with
GObjects making the middleware API a lot more harmonious with the rest
of GStreamer, and just better in general.

Given the licencing issue and the MPEG-TS issue, there may then be a
strong argument for reimplementing the DVBv5 middleware, at the expense
of having to maintain it. However ongoing maintenance is a small price
given that currently the DVBv3 → DVBv5 problem is a huge problem for
GStreamer.

I will most likely look at parsing DVBv5 files (real channel and
virtual channel) using C (this is going to be very, very painful :-(
with a view to creating GObjects that are effectively maps (just
properties à la other GStreamer types).

Is there a preferred test framework for codes such as this? I generally
test C codes with Python+pytest.
  
--
Russel.
=============================================================================
Dr Russel Winder      t: [phone]   voip: sip:[hidden email]
41 Buckmaster Road    m: [phone]   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Thiago Santos-3]

On 03/29/2016 04:16 AM, Russel Winder wrote:

On Sat, 2016-03-26 at 20:20 +0200, Sebastian Dröge wrote:

[…]
For background info, libdvbv5 is GPL licensed.

The Linux Media email list members are not picking up on my question
about licencing, but it has been an Easter break for the bulk of them
so I guess leave it a few more days to see if anyone addresses the
issue.

If libdvbv5 remains GPLv2 then clearly a reimplementation of the DVBv5
"middleware" is required for GStreamer since it require LGPL 2.1 or
more permissive dependencies. As with libdvbv5 this will have to be
principally DVBv5 but with a DVBv3 backward compatibility layer.

libdvbv5 has to be pure C with no dependencies other than libc. This
leads to what appears to be very primitive (i.e. 1960s) ways of
realizing some algorithms. Given GStreamer relies on GLib, a GStreamer
implementation of the DVBv5 algorithms would obviously be realized with
GObjects making the middleware API a lot more harmonious with the rest
of GStreamer, and just better in general.

Given the licencing issue and the MPEG-TS issue, there may then be a
strong argument for reimplementing the DVBv5 middleware, at the expense
of having to maintain it. However ongoing maintenance is a small price
given that currently the DVBv3 → DVBv5 problem is a huge problem for
GStreamer.

I will most likely look at parsing DVBv5 files (real channel and
virtual channel) using C (this is going to be very, very painful :-(
with a view to creating GObjects that are effectively maps (just
properties à la other GStreamer types).

FWIW I have a version of dvbsrc that is able to parse v5 files using GKeyFile. Code is at https://cgit.freedesktop.org/~thiagoss/gst-plugins-bad/log/?h=dvb-v5-file in case it could be helpful.

Perhaps some of it can help you. I was going to propose merging it upstream but it is better to settle on a plan here for the future of our DVB implementation before adding more small features.

Is there a preferred test framework for codes such as this? I generally
test C codes with Python+pytest.
  

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Russel Winder]

On Tue, 2016-03-29 at 18:25 -0300, Thiago Sousa Santos wrote:

> […]
> FWIW I have a version of dvbsrc that is able to parse v5 files using 
> GKeyFile. Code is at 
> https://cgit.freedesktop.org/~thiagoss/gst-plugins-bad/log/?h=dvb-v5-
> file in 
> case it could be helpful.
>
> Perhaps some of it can help you. I was going to propose merging it 
> upstream but it is better to settle on a plan here for the future of
> our 
> DVB implementation before adding more small features.

Reading, and writing, DVBv5 format files, is clearly crucial. I will
take a look at your code, thanks for pointing it out.

However there is something even more crucial about rewriting dvbsrc and
that is the change in the Linux DVB API: basically dvbsrc needs a total
rewrite, much of it replicating code that is already in libdvbv5.
  
--
Russel.
=============================================================================
Dr Russel Winder      t: [phone]   voip: sip:[hidden email]
41 Buckmaster Road    m: [phone]   xmpp: [hidden email]
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel