Video frame timings (revisited)

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Video frame timings (revisited)

Darren Staples
Hi, I'm hoping someone can help with this please, as I'm struggling...

When doing audio and video decoding of a H.264 MP4 file I find that
the timing of the video frames seems to be affected by the processing
of the audio frames.
My pipeline looks like this:

                    / ffmpegcolorspace - queue - appsink
uridecodebin
                    \ audioconvert - queue - appsink

I have setup callbacks on both the audio and video appsinks.
In the video callback I get the buffer from the appsink and then call
gst_element_query_position() to get the frame time.
In the audio callback I'm just getting the buffer and discarding it (for now).

The frame times I see in the video callback seem to be affected by the
audio decoding i.e. I don't see the regular frame times that I
expected. They're quite variable and there seem to be extended periods
(1-3s) between some frames.
If I remove the audio branch of the pipeline I see perfectly regular
(exactly 40ms or 25fps) video frame times.

Am I missing some vital component from my piepline ?
Can anyone suggest, please, where I might be going wrong ?

Thanks in advance,
Darren

------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand
malware threats, the impact they can have on your business, and how you
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Video frame timings (revisited)

michael smith-6-3
On Wed, Jan 12, 2011 at 7:50 AM, Darren Staples
<[hidden email]> wrote:

> Hi, I'm hoping someone can help with this please, as I'm struggling...
>
> When doing audio and video decoding of a H.264 MP4 file I find that
> the timing of the video frames seems to be affected by the processing
> of the audio frames.
> My pipeline looks like this:
>
>                     / ffmpegcolorspace - queue - appsink
> uridecodebin
>                    \ audioconvert - queue - appsink
>
> I have setup callbacks on both the audio and video appsinks.
> In the video callback I get the buffer from the appsink and then call
> gst_element_query_position() to get the frame time.
> In the audio callback I'm just getting the buffer and discarding it (for now).

Don't use gst_element_query_position, then. It's not doing what you want.

Instead, use the buffer timestamps directly (GST_BUFFER_TIMESTAMP(buffer)).

Mike

------------------------------------------------------------------------------
Protect Your Site and Customers from Malware Attacks
Learn about various malware tactics and how to avoid them. Understand
malware threats, the impact they can have on your business, and how you
can protect your company and customers by using code signing.
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel