what format is exchanged between qtdemux and ffdec_h264

[franchan]

Hello,

I'm asked to create a gstreamer plug-in using a proprietary h264
decoder library (binary only :-(.
I based myself on the many gstreamer plug-ins.
So implementing this until getting the chain function to work was smooth.

Then now I'm blocked.
The data that is passed from the qtdemuxer into the plug-in's chain function
does not seem to contain any NAL headers or NAL unit.
Can somebody explain what data format the qtdemux outputs?

As a working reference I used this working pipeline:
gst-launch filesrc location=small.mp4 ! qtdemu name demux
demux.VIDEO_00 ! queue ! ffdec_h264 ! ffmpegcolorspace ! xvimagesink

I dumped some GstBuffer data and I think it is part
of the 'mdat' mp4 box for the mp4 container (small.mp4)
What is passed to the ffdec_h264 is the same as what it passed to my
plug-in wrapper.
The proprietary decoder library hints that it does not understand the
syntax of the content.
I already tried to glue some 00 00 01 NAL prefix in front but
this does not solve the problem.

Thanks in advance,
franchan

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Felipe Contreras]

On Thu, Oct 2, 2008 at 10:53 AM, franchan <[hidden email]> wrote:

> Hello,
>
> I'm asked to create a gstreamer plug-in using a proprietary h264
> decoder library (binary only :-(.
> I based myself on the many gstreamer plug-ins.
> So implementing this until getting the chain function to work was smooth.
>
> Then now I'm blocked.
> The data that is passed from the qtdemuxer into the plug-in's chain function
> does not seem to contain any NAL headers or NAL unit.
> Can somebody explain what data format the qtdemux outputs?
>
> As a working reference I used this working pipeline:
> gst-launch filesrc location=small.mp4 ! qtdemu name demux
> demux.VIDEO_00 ! queue ! ffdec_h264 ! ffmpegcolorspace ! xvimagesink
>
> I dumped some GstBuffer data and I think it is part
> of the 'mdat' mp4 box for the mp4 container (small.mp4)
> What is passed to the ffdec_h264 is the same as what it passed to my
> plug-in wrapper.
> The proprietary decoder library hints that it does not understand the
> syntax of the content.
> I already tried to glue some 00 00 01 NAL prefix in front but
> this does not solve the problem.

Aside from raw frames the capabilities are also passed. Parts of the
capabilities is the codec-data, maybe that's what you want:
270000010fc2fe0af08f88800000010e18642bc23c80

--
Felipe Contreras

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Jan Schmidt-6]

On Thu, 2008-10-02 at 16:04 +0300, Felipe Contreras wrote:

> On Thu, Oct 2, 2008 at 10:53 AM, franchan <[hidden email]> wrote:
> > I dumped some GstBuffer data and I think it is part
> > of the 'mdat' mp4 box for the mp4 container (small.mp4)
> > What is passed to the ffdec_h264 is the same as what it passed to my
> > plug-in wrapper.
> > The proprietary decoder library hints that it does not understand the
> > syntax of the content.
> > I already tried to glue some 00 00 01 NAL prefix in front but
> > this does not solve the problem.
>
> Aside from raw frames the capabilities are also passed. Parts of the
> capabilities is the codec-data, maybe that's what you want:
> 270000010fc2fe0af08f88800000010e18642bc23c80

The data emerging from qtdemux isn't raw H.264 frames - it's
encapsulated into a packed format for the mp4 container. You need to
read out the NAL units by parsing the codec_data to get some detail
about the bytestream and to extract the SPS and PPS packets.

Check out:

http://www.geocities.com/xhelmboyx/quicktime/formats/mp4-layout.txt

Cheers,
Jan.
--
Jan Schmidt <[hidden email]>


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Wim Taymans]

On Thu, 2008-10-02 at 09:53 +0200, franchan wrote:

> Hello,
>
> I'm asked to create a gstreamer plug-in using a proprietary h264
> decoder library (binary only :-(.
> I based myself on the many gstreamer plug-ins.
> So implementing this until getting the chain function to work was smooth.
>
> Then now I'm blocked.
> The data that is passed from the qtdemuxer into the plug-in's chain function
> does not seem to contain any NAL headers or NAL unit.
> Can somebody explain what data format the qtdemux outputs?
>
> As a working reference I used this working pipeline:
> gst-launch filesrc location=small.mp4 ! qtdemu name demux
> demux.VIDEO_00 ! queue ! ffdec_h264 ! ffmpegcolorspace ! xvimagesink
>
> I dumped some GstBuffer data and I think it is part
> of the 'mdat' mp4 box for the mp4 container (small.mp4)
> What is passed to the ffdec_h264 is the same as what it passed to my
> plug-in wrapper.
> The proprietary decoder library hints that it does not understand the
> syntax of the content.
> I already tried to glue some 00 00 01 NAL prefix in front but
> this does not solve the problem.

H264 in quicktime is what we call in gstreamer 'packtized' h264. A
codec_data is exchanged in the caps that contains, among other things,
the nal_length_size field and SPS, PPS.

The data consists of a nal_length_size header containing the length of
the NAL unit that immediatly follows the size header.

Inserting the SPS,PPS (after prefixing them with escape codes) and
exchanging the size header with escape codes is a valid way to transform
a packetized stream into a byte stream. h264parse in gst-plugins-bad
will show you how this is done in code.

Wim

>
> Thanks in advance,
> franchan
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel