VANC (Ancillary data, e.g. captions)

[jpoet]

What is the best way to handle VANC?

An ex-coworker came up with a patch to add VANC processing to GStreamer version 1.2.3 for Blackmagic cards.   Unfortunately, he never pushed that code upstream before he left.

With his patch, the VANC was extracted from the Blackmagic driver and then pushed onto the gstreamer pipeline -- similar to the way video and audio was pushed onto the pipeline in V1.2.3 of Gstreamer.  He then added a VANC sink pad to ffmpegenc to encode the captions.

Since version 1.2.3, the GStreamer Blackmagic support has been completely re-written, and it no longer works the same way.  I would like to take the work that my ex-coworker did, and adapt it to the latest version of GStreamer, but I am unclear on what the proper approach would be.  Should I effectively clone gstdecklinkvideosrc into gstdecklinkvancsrc and tweak it as appropriate?

I don't see any established support for processing captions.  Has a roadmap been defined?  If I get this working, I am hoping that my work will be accepted upstream, so I would like some guidance on how to do it right.

Thanks,

John

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Baby Octopus]

Since the new decklinkvideosrc and decklinkaudiosrc are written on top of basesrc, you can have only one src pad. Having said that, you have multiple options
1. You can send VANC data through gstMeta along with video data in decklinkvideosrsc element
2. Another option is to create decklinkmetasrc or decklinkvancsrc as a separate src element

I guess #1 would be easier. Please file a bug in bugzilla for the same and upload the patch so that it can be pushed upstream after review :)

[Sebastian Dröge-3]

On Di, 2016-05-17 at 22:24 +0000, John P Poet wrote:

>
> What is the best way to handle VANC?
>
> An ex-coworker came up with a patch to add VANC processing to
> GStreamer version 1.2.3 for Blackmagic cards.   Unfortunately, he
> never pushed that code upstream before he left.
>
> With his patch, the VANC was extracted from the Blackmagic driver and
> then pushed onto the gstreamer pipeline -- similar to the way video
> and audio was pushed onto the pipeline in V1.2.3 of Gstreamer.  He
> then added a VANC sink pad to ffmpegenc to encode the captions.
>
> Since version 1.2.3, the GStreamer Blackmagic support has been
> completely re-written, and it no longer works the same way.  I would
> like to take the work that my ex-coworker did, and adapt it to the
> latest version of GStreamer, but I am unclear on what the proper
> approach would be.  Should I effectively clone gstdecklinkvideosrc
> into gstdecklinkvancsrc and tweak it as appropriate?
>
> I don't see any established support for processing captions.  Has a
> roadmap been defined?  If I get this working, I am hoping that my
> work will be accepted upstream, so I would like some guidance on how
> to do it right.

There's
  https://bugzilla.gnome.org/show_bug.cgi?id=766419
for adding specific "processing captions": timecodes :)

To me it seems like VANC shouldn't be a separate stream but also
GstMeta on the audio/video buffers. As it is metadata about those and
not an independently useful streams.

What do you think? What are your use cases for VANC?

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[jpoet]

On Wed, May 18, 2016 at 7:13 AM Sebastian Dröge <[hidden email]> wrote:

On Di, 2016-05-17 at 22:24 +0000, John P Poet wrote:
>
> What is the best way to handle VANC?
>
> An ex-coworker came up with a patch to add VANC processing to
> GStreamer version 1.2.3 for Blackmagic cards.   Unfortunately, he
> never pushed that code upstream before he left.
>
> With his patch, the VANC was extracted from the Blackmagic driver and
> then pushed onto the gstreamer pipeline -- similar to the way video
> and audio was pushed onto the pipeline in V1.2.3 of Gstreamer.  He
> then added a VANC sink pad to ffmpegenc to encode the captions.
>
> Since version 1.2.3, the GStreamer Blackmagic support has been
> completely re-written, and it no longer works the same way.  I would
> like to take the work that my ex-coworker did, and adapt it to the
> latest version of GStreamer, but I am unclear on what the proper
> approach would be.  Should I effectively clone gstdecklinkvideosrc
> into gstdecklinkvancsrc and tweak it as appropriate?
>
> I don't see any established support for processing captions.  Has a
> roadmap been defined?  If I get this working, I am hoping that my
> work will be accepted upstream, so I would like some guidance on how
> to do it right.

There's
  https://bugzilla.gnome.org/show_bug.cgi?id=766419
for adding specific "processing captions": timecodes :)

That is a possibility.  I would have figuring out how to convert VANC captions into SMPTE time code format.  Or, just add the VANC as more metadata...

 

To me it seems like VANC shouldn't be a separate stream but also
GstMeta on the audio/video buffers. As it is metadata about those and
not an independently useful streams.

I will look into that.  Either way, the next step is adding sinks for the appropriate encoders/muxers to deal with that data.

 

What do you think? What are your use cases for VANC?

I want to have the closed captions available in the resulting mpeg2 and/or mpeg4 and/or transport stream.  When I play the resulting file in vlc, for example, I want to be able to select which captions I want to see.

 Thanks for the pointers.

John

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel