Audio lags video for avi movie

[William Salibrici]

I have the following recording pipeline [in C code] which runs ok on my windows 7 x86system. The recording captures audio and video from my computer’s mic and camera and creates a movie file. The appsrc is running in push mode and the video buffer timestamps are correct. Each video buffer timestamp matches the moment this data has been captured according to the pipeline clock, minus the configured base time. The directsoundsrc element is providing the clock for the pipeline.

 appsrc is-live=TRUE ! videoconvert ! vp8enc ! queue !

 mux. directsoundsrc device-name="Microphone Array" !

 audioconvert ! audio/x-raw,format=S16LE,rate=8000,channels=1 !

opusenc ! queue ! mux. oggmux name=mux !

 filesink location=C:/movie.ogg

 

I use the following pipeline with the launch tool for playback and all is well. Audio and video are in sync.

 filesrc location=C:/movie.ogg do-timestamp=TRUE !

 oggdemux name=demux demux. ! queue ! vp8dec ! videoconvert !

 d3dvideosink demux. ! queue ! opusdec ! audioconvert ! directsoundsink

 

The problem occurred when I changed the recording pipeline [in C code] to use different video encoding, no audio encoding [for smallest audio latency], and a different muxer. Notice I kept the audio format caps filter the same. Recording runs ok but playback has the audio lagging the video by a small amount [a few seconds] that appears to be constant. Playback using the standard windows media player has the same result.

 appsrc is-live=TRUE ! videoconvert ! openh264enc ! h264parse ! queue !

 mux. directsoundsrc device-name="Microphone Array" !

 audio/x-raw,format=S16LE,rate=8000,channels=1 !

audioconvert ! mux. avimux name=mux !

 filesink location=C:/movie.avi

 

Here is the playback pipeline [using the launch tool] for the second recording:

filesrc location=C:/movie.avi ! avidemux name=demux demux. !

video/x-h264,stream-format=byte-stream,alignment=au ! h264parse !

queue ! openh264dec ! videoconvert ! d3dvideosink demux. !

queue ! audioconvert ! directsoundsink

 

Here are my two questions for the second record pipeline where audio lags video:

1. What is causing the audio to lag the video?
2. What would be the best way to add a fixed latency to the video leg in the recording pipeline so that audio and video are in sync for playback? My customer application requires the standard windows media player for playback so I can’t tweak the playback properties.

 

I’m using GStreamer 1.6.4 with your windows pre-built binaries.

Thank you,

Bill Salibrici

Lattice Incorporated

http://www.latticeinc.com/

 

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim Müller]

On Thu, 2016-06-09 at 21:36 +0000, William Salibrici wrote:

Hi Bill,

 Here are my two questions for the second record pipeline where audio lags video:

1. What is causing the audio to lag the video?
2. What would be the best way to add a fixed latency to the video leg in the recording pipeline so that audio and video are in sync for playback? My customer application requires the standard windows media player for playback so I can’t tweak the playback properties.

 

I’m using GStreamer 1.6.4 with your windows pre-built binaries.

Try adding a videorate element before the video encoder and an audiorate element before the audio encoder or muxer (if you mux raw audio), to see if that makes a difference.

Cheers

 -Tim

-- 
Tim Müller, Centricular Ltd - http://www.centricular.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[William Salibrici]

Hi Tim,

Thank you for your reply.

I did as you suggested and yes it fixed the problem!

Playback is fine now using my play pipeline as well as the windows media player.

Any further insight as to why audio was lagging?

At first I was using mulawenc but I removed it to further reduce audio leg latency.

If anything, I would have expected video to be lagging due to the greater computational overhead in the video leg.

Perhaps one of the elements I’m using is not reporting latency correctly – I don’t know.

Regards,

 

Bill

 

From: gstreamer-devel [mailto:[hidden email]] On Behalf Of Tim Müller
Sent: Thursday, June 09, 2016 6:10 PM
To: Discussion of the development of and with GStreamer <[hidden email]>
Subject: Re: Audio lags video for avi movie

 

On Thu, 2016-06-09 at 21:36 +0000, William Salibrici wrote:

 

Hi Bill,

 

 Here are my two questions for the second record pipeline where audio lags video:

 

1. What is causing the audio to lag the video?
2. What would be the best way to add a fixed latency to the video leg in the recording pipeline so that audio and video are in sync for playback? My customer application requires the standard windows media player for playback so I can’t tweak the playback properties.

 

I’m using GStreamer 1.6.4 with your windows pre-built binaries.

Try adding a videorate element before the video encoder and an audiorate element before the audio encoder or muxer (if you mux raw audio), to see if that makes a difference.

Cheers

 -Tim

-- 

Tim Müller, Centricular Ltd - http://www.centricular.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel