Setting codec_data in h264 caps for qtmux

[laena]

Hi all,

I am trying to set the correct caps for the following pipeline: "appsrc - qtmux - filesink", where appsrc receives a raw h264 video input stream.
The pipeline runs through, but the output cannot displayed properly.

I still get the warning "no codec data in h264 caps", so this missing field is probably the problem.
(see also http://gstreamer-devel.966125.n4.nabble.com/Problems-muxing-raw-h264-stream-with-qtmux-tc4417169.html).

How do I create such a codec_data buffer? In which order do I need to provide the SPS/PPS information? Do you have some example code for that?

cheers and thanks in advance,
Lena

[coezbek]

From other emails I have gathered that the codec_data should be set to the AVCDecoderConfigurationRecord from ISO 14496-15 Section 5.2.4:

aligned(8) class AVCDecoderConfigurationRecord {
  unsigned int(8) configurationVersion = 1;
  unsigned int(8) AVCProfileIndication;
  unsigned int(8) profile_compatibility;
  unsigned int(8) AVCLevelIndication;
  bit(6) reserved = ‘111111’b;  
  unsigned int(2) lengthSizeMinusOne;
  bit(3) reserved = ‘111’b;  
  unsigned int(5) numOfSequenceParameterSets;
  for (i=0; i< numOfSequenceParameterSets; i++) {
    unsigned int(16) sequenceParameterSetLength ;
    bit(8*sequenceParameterSetLength) sequenceParameterSetNALUnit;
  }
unsigned int(8) numOfPictureParameterSets;
  for (i=0; i< numOfPictureParameterSets; i++) {  
    unsigned int(16) pictureParameterSetLength;
    bit(8*pictureParameterSetLength) pictureParameterSetNALUnit;
  }
}

Does GStreamer have any helper function / datastructure to construct this header?

Does qtmux expect the record to be given as a raw buffer or wrapped somehow?

Any help would be appreciated.

Christopher

[Mohammed Sameer]

On Tue, Apr 03, 2012 at 12:57:44AM -0700, coezbek wrote:
[,,,]
> Does GStreamer have any helper function / datastructure to construct this
> header?

I hope there will be. I'm not aware of any helpers and I had to do it manually.

However you can check how gst-dsp constructs the header here:
https://github.com/felipec/gst-dsp/blob/master/tidsp/td_h264enc.c#L189

> Does qtmux expect the record to be given as a raw buffer or wrapped somehow?

I was doing the same a few weeks ago. Constructed the structure, copied it to a GstBuffer
and handed it over to mp4mux and it was happy so I can say that no wrapping was needed.

Cheers,

--
GPG-Key: 0xA3FD0DF7 - 9F73 032E EAC9 F7AD 951F  280E CB66 8E29 A3FD 0DF7
Debian User and Developer.
Homepage: www.foolab.org
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Stefan Sauer]

On 04/03/2012 09:57 AM, coezbek wrote:

> From other emails I have gathered that the codec_data should be set to the
> AVCDecoderConfigurationRecord from ISO 14496-15 Section 5.2.4:
>
> aligned(8) class AVCDecoderConfigurationRecord {
>   unsigned int(8) configurationVersion = 1;
>   unsigned int(8) AVCProfileIndication;
>   unsigned int(8) profile_compatibility;
>   unsigned int(8) AVCLevelIndication;
>   bit(6) reserved = ‘111111’b;  
>   unsigned int(2) lengthSizeMinusOne;
>   bit(3) reserved = ‘111’b;  
>   unsigned int(5) numOfSequenceParameterSets;
>   for (i=0; i< numOfSequenceParameterSets; i++) {
>     unsigned int(16) sequenceParameterSetLength ;
>     bit(8*sequenceParameterSetLength) sequenceParameterSetNALUnit;
>   }
> unsigned int(8) numOfPictureParameterSets;
>   for (i=0; i< numOfPictureParameterSets; i++) {  
>     unsigned int(16) pictureParameterSetLength;
>     bit(8*pictureParameterSetLength) pictureParameterSetNALUnit;
>   }
> }
>
> Does GStreamer have any helper function / datastructure to construct this
> header?

A couple of them are here:
http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-libs/html/gst-plugins-base-libs-gstpbutilscodecutils.html

Stefan

> Does qtmux expect the record to be given as a raw buffer or wrapped somehow?
>
> Any help would be appreciated.
>
> Christopher
>
> --
> View this message in context: http://gstreamer-devel.966125.n4.nabble.com/Setting-codec-data-in-h264-caps-for-qtmux-tp4525597p4528058.html
> Sent from the GStreamer-devel mailing list archive at Nabble.com.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[coezbek]

Thanks Mohammed and Stefan!

With the link from Mohammend I came up with the following (C++ albeit) to construct a AVCDecoderConfigurationRecord for use as a codec_data for qtmux.

I am not sure whether lengthSizeMinusOne will always indicate 4bytes.

GstBuffer* BitstreamMuxer::createCodecData(GstBuffer* sps, GstBuffer* pps)
{
  unsigned int offset = 0;

  GstBuffer* codec_data_buffer = gst_buffer_new_and_alloc(sps->size + pps->size + 11);
  guint8* codec_data = codec_data_buffer->data;

  codec_data[offset++] = 0x01; // Configuration Version
  codec_data[offset++] = sps->data[1]; // AVCProfileIndication
  codec_data[offset++] = sps->data[2]; // profile_compatibility
  codec_data[offset++] = sps->data[3]; // AVCLevelIndication
  codec_data[offset++] = 0xff; // lengthSizeMinusOne == 3 -> length == 4 byte

  // SPS
  codec_data[offset++] = 0xe1; // numOfSequenceParameterSets | b11100000 -> numSPS == 1
 
  assert(sps->size <= 0xffff);
  codec_data[offset++] = (sps->size >> 8) & 0xff; // numOfSequenceParameterSets high 8bit
  codec_data[offset++] = sps->size & 0xff; // numOfSequenceParameterSets low 8bit
  memcpy(codec_data + offset, sps->data, sps->size);
  offset += sps->size;

  // PPS
  codec_data[offset++] = 0x1; // numOfPictureParameterSets == 1
 
  assert(pps->size <= 0xffff);
  codec_data[offset++] = (pps->size >> 8) & 0xff;
  codec_data[offset++] = pps->size & 0xff;
  memcpy(codec_data + offset, pps->data, pps->size);
  offset += pps->size;
 
  assert(offset == codec_data_buffer->size);

  return codec_data_buffer;
}