muxing proposal

[David Schleef]

Attempting to get Dirac muxing working correctly with most containers
has been a difficult task recently.  GStreamer has been the easiest
media framework *by* *far*; other frameworks like DirectShow and
QuickTime really only allow muxing into one specific container.  (In
some ways, gstreamer is the same way, with ogg.)

The main problems, as I see it, are:

 - All our muxers suck.

 - Each container format has special mapping requirements for streams.
   AVI needs special chunking of Dirac packets, MPEG-TS needs
   presentation timestamps, Ogg needs granulepos, etc.  Right now,
   each muxer reparses and/or attempts to reconstruct this information
   in the muxer (but mostly just does it wrong).

 - Adding a new format requires modifying the muxer.  This is the same
   problem as convincing Apple to add Dirac to their mp4 muxer, except
   that I'm lucky enough to have commit access here.


What I propose is this:

 - Muxers should be modified to implement a native stream format, e.g.,
   video/x-avi-part for avimux, with a fourcc describing the stream.
   Muxers may optionally accept other formats as they do now, but are
   responsible for parsing and discovering values that the container
   needs to store.  These streams have a well-defined structure, with
   the intention that buffers are directly encapsulated into output
   packets.
   
 - For each of these muxer substream formats, define additional
   stream-specific buffer flags as necessary and override
   GST_BUFFER_OFFSET_END() for stream- specific purposes (like granulepos.)

 - A set of new elements (or mods to parsers) that parse encoded streams
   into the correct format for muxers.  Thus, diracparseforavi would
   parse a dirac stream into the correct chunks to go into AVI, as well
   as setting the key frame flags correctly.  (Key frame flags in AVI and
   QuickTime are more restricted than MPEG or Ogg.)  diracparseforogg
   would add Ogg granulepos to each buffer.

   (I'm indifferent to using separate elements or a single diracparse
   element that can output many parsed formats.)

 - Theoretically, this should be backward compatible.  We can move toward
   using the native stream formats gradually.

 - Ogg will still require stream-specific code in the muxer.

Once parsers are responsible for delivering buffers to the muxer that
the muxer understands, I'm hoping that writing a muxer becomes a lot
closer to "while(1) { collect a buffer; encapsulate; push }".



dave...


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Benoit Fouet]

Hi,

David Schleef wrote:

> Attempting to get Dirac muxing working correctly with most containers
> has been a difficult task recently.  GStreamer has been the easiest
> media framework *by* *far*; other frameworks like DirectShow and
> QuickTime really only allow muxing into one specific container.  (In
> some ways, gstreamer is the same way, with ogg.)
>
> The main problems, as I see it, are:
>
>  - All our muxers suck.
>
>  - Each container format has special mapping requirements for streams.
>    AVI needs special chunking of Dirac packets, MPEG-TS needs
>    presentation timestamps, Ogg needs granulepos, etc.  Right now,
>    each muxer reparses and/or attempts to reconstruct this information
>    in the muxer (but mostly just does it wrong).
>
>  - Adding a new format requires modifying the muxer.  This is the same
>    problem as convincing Apple to add Dirac to their mp4 muxer, except
>    that I'm lucky enough to have commit access here.
>
>
>  

there will still be the problem of extradata that the muxer has to write.
or do you also plan to have the new elements (or modified parsers) to
take care of the extradata too ?

> What I propose is this:
>
>  - Muxers should be modified to implement a native stream format, e.g.,
>    video/x-avi-part for avimux, with a fourcc describing the stream.
>    Muxers may optionally accept other formats as they do now, but are
>    responsible for parsing and discovering values that the container
>    needs to store.  These streams have a well-defined structure, with
>    the intention that buffers are directly encapsulated into output
>    packets.
>    
>  

Doesn't that mean code duplication in multiple plugins ?

>  - For each of these muxer substream formats, define additional
>    stream-specific buffer flags as necessary and override
>    GST_BUFFER_OFFSET_END() for stream- specific purposes (like granulepos.)
>
>  - A set of new elements (or mods to parsers) that parse encoded streams
>    into the correct format for muxers.  Thus, diracparseforavi would
>    parse a dirac stream into the correct chunks to go into AVI, as well
>    as setting the key frame flags correctly.  (Key frame flags in AVI and
>    QuickTime are more restricted than MPEG or Ogg.)  diracparseforogg
>    would add Ogg granulepos to each buffer.
>
>    (I'm indifferent to using separate elements or a single diracparse
>    element that can output many parsed formats.)
>
>  - Theoretically, this should be backward compatible.  We can move toward
>    using the native stream formats gradually.
>
>  - Ogg will still require stream-specific code in the muxer.
>
> Once parsers are responsible for delivering buffers to the muxer that
> the muxer understands, I'm hoping that writing a muxer becomes a lot
> closer to "while(1) { collect a buffer; encapsulate; push }".
>
>  

if one is willing to support a specific stream encapsulation in a
specific container, I don't see why (s)he would not do it directly in
the muxer
(but I might have overlooked something)

--
Benoit Fouet
Purple Labs S.A.
www.purplelabs.com

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel