AV Sync for demuxed data

[rpavithra.87]

I am trying to test a decoder element.I am dumping the demuxed data (data,size and timestamps).
After that i am feeding the demuxed data to the following pipeline containing two  bins as described below.
 Appsrc - Video dec - Video sink
 Appsrc - Audio dec - Audio sink

it plays well for two or three sec with audio and video sync . But after that it throws the following error messages :
0:00:05.834803752 29674       0x8b9c20 WARN           baseaudiosink gstbaseaudiosink.c:1518:gst_base_audio_sink_render:<alsasink0> warning: Compensating for audio synchronisation problems
0:00:05.834846589 29674       0x8b9c20 WARN           baseaudiosink gstbaseaudiosink.c:1518:gst_base_audio_sink_render:<alsasink0> warning: Unexpected discontinuity in audio timestamps of more than half a second (0:00:04.294979166), resyncing and the actual timestamp is  (0:00:00.001032704)
0:00:05.836275768 29674       0x8b9c20 WARN           baseaudiosink gstbaseaudiosink.c:1518:gst_base_audio_sink_render:<alsasink0> warning: Compensating for audio synchronisation problems
0:00:05.836289732 29674       0x8b9c20 WARN           baseaudiosink gstbaseaudiosink.c:1518:gst_base_audio_sink_render:<alsasink0> warning: Unexpected discontinuity in audio timestamps of more than half a second (0:00:04.294958333), resyncing and the actual timestamp is  (0:00:00.002065408)


and the video decoder also starts dropping the frames.

 ffmpeg gstffmpegdec.c:1966:gst_ffmpegdec_video_frame:<ffdec_h2640> Dropping non-keyframe (seek/init)

Without timestamps all the video frames and audio samples are played out without sync.
Should i need to do something extra in order to achieve avsync? Please help me.

[Sandeep Prakash]

Hi,

Make sure that:
1. The timestamps from the demuxed data is properly injected into the pipeline. Also the framerate and
    sampling rates has to be set on the caps of respective appsrc so that its propagated properly to the
    respective sinks.
2. You are feeding full frames into appsrc. If you are not sure, better to use the respective
    elementary stream parsers before each decoder so that the decoder will have a frame of data to decode.
    "h264parse" in case of h264 streams, "aacparse" to parse aac streams. Your pipeline will look like:
        Appsrc - Elementary Stream Parser - Video dec - Video sink
        Appsrc - Elementary Stream Parser - Audio dec - Audio sink

Regards,
Sandeep Prakash
http://sandeepprakash.homeip.net

[rpavithra.87]

I compared the timestamps with the timestamps passed in a standard gstreamer pipeline(playbin).The timestamps are correct. And i am setting the caps using a capsfilter which is present in between appsrc and decoders.

Actually i dumped the buffer size, buffer data ,codec private data for h264 and timestamps in separate files.Whenever need data callback comes, i read the size from the sizefile and depending on the size, i read the data from the data file and feed it to the appsrc.  

When played without timestamps , all the frames and samples are played out. But only when i feed the timestamps , this problem occurs.

But the same file plays well when i use a playbin.

So i hope the demux does something extra .

[rpavithra.87]

Hi sandeep,

Thanks for the information.  I tried using the parsers infront of the decoders. Actually in case of audio some more samples were played out(i hope this is because video gets struck in the middle. otherwise all the samples would have been played out). But if i use the h264parse element the application hangs.

How to feed the codec private data to h264parse? Currently i am having a caps filter in between decoder and appsrc. And i am setting the codec private data as part of the caps.

[Sandeep Prakash]

Hi,

rpavithra.87 wrote

How to feed the codec private data to h264parse? Currently i am having a caps filter in between decoder and appsrc. And i am setting the codec private data as part of the caps.

h264parse expects the codec private data as "codec_data" which should be part of the caps. Unless the stream is a bytestream in which case the codec_data might be part of the 1st frame, codec_data is compulsory as part of the caps. Plz refer any of the standard demuxers/parser on how they construct this codec_data. There is a format that need to be followed for codec_data (In h264 case). You can start of here:
http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst/h264parse/gsth264parse.c#n1115

Regards,
Sandeep Prakash
http://sandeepprakash.homeip.net