GStreamer and Closed-Captioning support

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

GStreamer and Closed-Captioning support

Bryen Yunashko
Can anyone send me information on whether gstreamer supports closed-captioning with gstreamer?  Particularly, I'm trying to find it in Totem.

Thanks!
Bryen

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: GStreamer and Closed-Captioning support

Bryen Yunashko
I'm assuming since I haven't heard anything back, that GStreamer does not support closed-captioning for the deaf community.  Is this correct?


On Jan 4, 2008 5:27 PM, Bryen Yunashko < [hidden email]> wrote:
Can anyone send me information on whether gstreamer supports closed-captioning with gstreamer?  Particularly, I'm trying to find it in Totem.

Thanks!
Bryen


-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: GStreamer and Closed-Captioning support

Stefan Sauer
In reply to this post by Bryen Yunashko
Hi,

Quoting Bryen Yunashko <[hidden email]>:

> Can anyone send me information on whether gstreamer supports
> closed-captioning with gstreamer?  Particularly, I'm trying to find it in
> Totem.
Could you elaborate a bit what that is? Is it reading out subtitles?  
Any pointers? I want to prepare a list of taks for the next google  
summer-of-code application. This could be a potential subproject.

Stefan

>
> Thanks!
> Bryen
>



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: GStreamer and Closed-Captioning support

Stefan Sauer
Hi again,

Quoting Stefan Kost <[hidden email]>:

> Hi,
>
> Quoting Bryen Yunashko <[hidden email]>:
>
>> Can anyone send me information on whether gstreamer supports
>> closed-captioning with gstreamer?  Particularly, I'm trying to find it in
>> Totem.
> Could you elaborate a bit what that is? Is it reading out subtitles?
> Any pointers? I want to prepare a list of taks for the next google
> summer-of-code application. This could be a potential subproject.

Replying to myself:
http://en.wikipedia.org/wiki/Closed_captioning

Bryen, do you have any info about how this differs from subtitles? Do  
you have some sample content that has closed captioning tracks that  
you could give to the gstreamer developers?

Stefan

>
> Stefan
>
>>
>> Thanks!
>> Bryen
>>
>
>
>
> -------------------------------------------------------------------------
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>



-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

from pipeline to code

Daniel Shachrur

look at:
gst-launch-0.10 udpsrc port=50000 caps=application/x-rtp ! rtpmp4vdepay ! video/mpeg,mpegversion=4,width=720,height=576,framerate=25/1 ! ffdec_mpeg4 ! videoflip method=clockwise ! xvimagesink sync=false

this pipe;ine can be coded as following:
videodepay = gst_element_factory_make ("rtpmp4vdepay", "vdepay");
decoder = gst_element_factory_make ("ffdec_mpeg4", "ffmpeg-decoder");
etc...

how can i code the following parts: video/mpeg,mpegversion=4,width=720,height=576,framerate=25/1 and
                                                        videoflip method=clockwise

what are there types? what gst functions are needed?


thanx

daniel

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE!
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: GStreamer and Closed-Captioning support

Zaheer Merali
In reply to this post by Stefan Sauer
On 1/8/08, Stefan Kost <[hidden email]> wrote:

> Hi again,
>
> Quoting Stefan Kost <[hidden email]>:
> > Hi,
> >
> > Quoting Bryen Yunashko <[hidden email]>:
> >
> >> Can anyone send me information on whether gstreamer supports
> >> closed-captioning with gstreamer?  Particularly, I'm trying to find it in
> >> Totem.
> > Could you elaborate a bit what that is? Is it reading out subtitles?
> > Any pointers? I want to prepare a list of taks for the next google
> > summer-of-code application. This could be a potential subproject.
>
> Replying to myself:
> http://en.wikipedia.org/wiki/Closed_captioning
>
> Bryen, do you have any info about how this differs from subtitles? Do
> you have some sample content that has closed captioning tracks that
> you could give to the gstreamer developers?
>
> Stefan
> >
> > Stefan
> >
> >>
> >> Thanks!
> >> Bryen
> >>
> >
> >
> >
> > -------------------------------------------------------------------------
> > Check out the new SourceForge.net Marketplace.
> > It's the best place to buy or sell services for
> > just about anything Open Source.
> > http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
> >
>
I can provide plenty of mpeg transport streams that have closed
caption in it. Currently the mpeg demuxers we have do not demux the
closed caption out, but nothing technically is stopping them doing it.

Zaheer

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: GStreamer and Closed-Captioning support

David I. Lehn-2
In reply to this post by Stefan Sauer
* Stefan Kost <[hidden email]> [2008-01-08T04:10:44-0500]:

> Quoting Stefan Kost <[hidden email]>:
> > Quoting Bryen Yunashko <[hidden email]>:
> >
> >> Can anyone send me information on whether gstreamer supports
> >> closed-captioning with gstreamer?  Particularly, I'm trying to find it in
> >> Totem.
> > Could you elaborate a bit what that is? Is it reading out subtitles?
> > Any pointers? I want to prepare a list of taks for the next google
> > summer-of-code application. This could be a potential subproject.
>
> Replying to myself:
> http://en.wikipedia.org/wiki/Closed_captioning
>
> Bryen, do you have any info about how this differs from subtitles? Do  
> you have some sample content that has closed captioning tracks that  
> you could give to the gstreamer developers?
>

When I was active here back in the day (years ago) I was playing with CC
code in GStreamer.  I think some of my code may have been lost to time
though.

At one point I had GStreamer decoding the CC data from DVDs.  It wasn't
doing full positional on-screen rendering but just dumping the raw text
to the console.  At the time I was quite thrilled to get that far but
moved on to other projects.

To get it working I made a specific Bin source on the mpeg demuxer (I
think?) to output the "user_data" packets.  On a DVD that is how they
store the VBI data.  I then stole the VBI code from the very excellent
tvtime project to do the hard work.  I think it's still in the "bad"
plugins repo in the vbidec plugin.  Then you just route the user_data to
vbidec and the output of vbidec to a console.  (Or close to that...
might have been stripping off control codes to just print the text.)  It
worked just fine but was mostly just a testing hack for GStreamer, DVD
code, Python bindings, and other stuff... all at once.  Would be easier
today.

To make this code useful there's lots of work to be done.

- Update vbidec to latest plugin standards and APIs.  I'm sure it's
  quite bitrotten by now.
- Look at status of the "user_data" mpeg fields.  I have no idea what it
  is these days.  I think it was ripped out at some point due to the
  core not being able to handle multiple bin sources at once properly or
  something.  Might be easier today or already done.  This is the
  easiest way to test CC data.  It's a correct VBI bitstream on any DVD
  with CC data.  Live video signals would work too but might have errors
  in the VBI data.  Though it's supposed to handle some level of errors.
- Try pushing V4L streams through vbidec.  I don't recall if I ever
  tried this.  I also can't remember if you have to strip out the VBI
  info or if the vbidec code can find that for you.
- Take the decoded VBI info and do something more useful with it.  There
  is data in there for display timing, colors, symbols, fonts,
  positional info... etc etc.  TVTime also supported other type of info.
  For instance the tv channel/program info (XDS) and the menu system
  more popular in .eu than .us.
- Create something to render the CC data.  (Re-use subtitle related
  work)
- Create something to composite that rendered data stream on top of a
  video stream properly.  (Re-use subtitle related work)
- Hook it all up, integrate into DVD and V4L apps and add some controls.
- Check with the tvtime.sourceforge.net project to see if they have any
  newer code.  Also you might need to rip more code to support more than
  just CC data.  TVTime would be _excellent_ reference code for the
  non-GStreamer-specific parts of this task.

There may be an issue with the vbidec code being in GStreamer at all.
It comes from a GPL project.  I'm not sure what the licensing issues are
these days with that.  Might need to convert bits of tvtime into an
external library and keep GStreamer specific parts in gst plugins.

This would all make an interesting and useful project.  With example
code like TVTime out there it's probably not extremely hard either.

I had always tried to convince vektor (Billy Biggs) from tvtime fame to
just use GStreamer as a base for tvtime but that never happened.  They
were focused on making the most elite tv app ever and succeeded.  Might
be possible at this point to port over pieces of it beyond just the CC
related parts.  There are some issues with GStreamers frame based
algorithms (I assume it's still like that?) and tvtimes scan-line
algorithms.  Fun stuff.  Good luck.

-dave

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: from pipeline to code

Daniel Díaz-2
In reply to this post by Daniel Shachrur
Hello.


On Jan 8, 2008 4:16 AM, Daniel Shachrur <[hidden email]> wrote:

> look at:
> gst-launch-0.10 udpsrc port=50000 caps=application/x-rtp ! rtpmp4vdepay ! video/mpeg,mpegversion=4,width=720,height=576,framerate=25/1 ! ffdec_mpeg4 ! videoflip method=clockwise ! xvimagesink sync=false
>
> this pipe;ine can be coded as following:
> videodepay = gst_element_factory_make ("rtpmp4vdepay", "vdepay");
> decoder = gst_element_factory_make ("ffdec_mpeg4", "ffmpeg-decoder");
> etc...
>
> how can i code the following parts: video/mpeg,mpegversion=4,width=720,height=576,framerate=25/1 and
>                                                         videoflip method=clockwise

Those are GStreamer elements as well, and you should instantiate one
for each. Do:
  gst-inspect capsfilter
  gst-inspect videoflip

This:
  video/mpeg,mpegversion=4,width=720,height=576,framerate=25/1
is equivalent to:
  capsfilter caps=video/mpeg,mpegversion=4,width=720,height=576,framerate=25/1

Make sure you set the properties (both for the capsfilter and the
videoflip element) when creating the pipeline.

Greetings!

Daniel Díaz
[hidden email]



> what are there types? what gst functions are needed?
> thanx
> daniel

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: GStreamer and Closed-Captioning support

Bryen Yunashko
In reply to this post by David I. Lehn-2


On Jan 8, 2008 10:01 AM, David I. Lehn <[hidden email]> wrote:
* Stefan Kost <[hidden email]> [2008-01-08T04:10:44-0500]:
> Quoting Stefan Kost <[hidden email]>:
> > Quoting Bryen Yunashko <[hidden email]>:

> >
> >> Can anyone send me information on whether gstreamer supports
> >> closed-captioning with gstreamer?  Particularly, I'm trying to find it in
> >> Totem.
> > Could you elaborate a bit what that is? Is it reading out subtitles?
> > Any pointers? I want to prepare a list of taks for the next google
> > summer-of-code application. This could be a potential subproject.
>
> Replying to myself:
> http://en.wikipedia.org/wiki/Closed_captioning
>
> Bryen, do you have any info about how this differs from subtitles? Do
> you have some sample content that has closed captioning tracks that
> you could give to the gstreamer developers?
>

When I was active here back in the day (years ago) I was playing with CC
code in GStreamer.  I think some of my code may have been lost to time
though.

At one point I had GStreamer decoding the CC data from DVDs.  It wasn't
doing full positional on-screen rendering but just dumping the raw text
to the console.  At the time I was quite thrilled to get that far but
moved on to other projects.

To get it working I made a specific Bin source on the mpeg demuxer (I
think?) to output the "user_data" packets.  On a DVD that is how they
store the VBI data.  I then stole the VBI code from the very excellent
tvtime project to do the hard work.  I think it's still in the "bad"
plugins repo in the vbidec plugin.  Then you just route the user_data to
vbidec and the output of vbidec to a console.  (Or close to that...
might have been stripping off control codes to just print the text.)  It
worked just fine but was mostly just a testing hack for GStreamer, DVD
code, Python bindings, and other stuff... all at once.  Would be easier
today.

To make this code useful there's lots of work to be done.

- Update vbidec to latest plugin standards and APIs.  I'm sure it's
 quite bitrotten by now.
- Look at status of the "user_data" mpeg fields.  I have no idea what it
 is these days.  I think it was ripped out at some point due to the
 core not being able to handle multiple bin sources at once properly or
 something.  Might be easier today or already done.  This is the
 easiest way to test CC data.  It's a correct VBI bitstream on any DVD
 with CC data.  Live video signals would work too but might have errors
 in the VBI data.  Though it's supposed to handle some level of errors.
- Try pushing V4L streams through vbidec.  I don't recall if I ever
 tried this.  I also can't remember if you have to strip out the VBI
 info or if the vbidec code can find that for you.
- Take the decoded VBI info and do something more useful with it.  There
 is data in there for display timing, colors, symbols, fonts,
 positional info... etc etc.  TVTime also supported other type of info.
 For instance the tv channel/program info (XDS) and the menu system
 more popular in .eu than .us.
- Create something to render the CC data.  (Re-use subtitle related
 work)
- Create something to composite that rendered data stream on top of a
 video stream properly.  (Re-use subtitle related work)
- Hook it all up, integrate into DVD and V4L apps and add some controls.
- Check with the tvtime.sourceforge.net project to see if they have any
 newer code.  Also you might need to rip more code to support more than
 just CC data.  TVTime would be _excellent_ reference code for the
 non-GStreamer-specific parts of this task.

There may be an issue with the vbidec code being in GStreamer at all.
It comes from a GPL project.  I'm not sure what the licensing issues are
these days with that.  Might need to convert bits of tvtime into an
external library and keep GStreamer specific parts in gst plugins.

This would all make an interesting and useful project.  With example
code like TVTime out there it's probably not extremely hard either.

I had always tried to convince vektor (Billy Biggs) from tvtime fame to
just use GStreamer as a base for tvtime but that never happened.  They
were focused on making the most elite tv app ever and succeeded.  Might
be possible at this point to port over pieces of it beyond just the CC
related parts.  There are some issues with GStreamers frame based
algorithms (I assume it's still like that?) and tvtimes scan-line
algorithms.  Fun stuff.  Good luck.

-dave

-------------------------------------------------------------------------
Well, Dave certainly gave lots of additional information.  :-)  Do you need anything additional from me?  

I do want to make one thing clear.  subtitling and CC are not the same.  CC uses a black block background, which myself and many in the US prefer because subtitling uses white font colors which can blend into the media if there's white or light colors behind it, making the subtitling completely useless for the moment.

Also, in the US, all broadcast programming is required under the FCC to broadcast in CC, and all televisions sold in the US are required to have the built-in CC chip.  This has been so since the early 90's.

DVDs come distributed with either CC capabilities, subtitling capabilities, or both.  Whenever we have a choice, we typically choose CC.

I did find a reference on this page that claims gstreamer has built in CC support:
http://www.linuxtv.org/v4lwiki/index.php/Text_capture.

Bryen

-------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It's the best place to buy or sell services for
just about anything Open Source.
http://ad.doubleclick.net/clk;164216239;13503038;w?http://sf.net/marketplace
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel