Inconsistent video frame rate and timestamp in v4l2src, and qtmux when both audio and video are captured

[yiliang]

Hi experts,

I try to capture both video and audio using the following pipeline:

gst-launch -e v4l2src ! video/x-raw-yuv, framerate=30000/1001 ! ffmpegcolorspace ! queue ! qtmux name=mux alsasrc device=hw:1,0 ! audio/x-raw-int, rate=8000 ! queue ! mux. mux. ! filesink location=a.mov

Here are the printed messages from v4l2src in function gst_v4l2src_create():

current = gst_clock_get_time (clock)
timestamp = GST_BUFFER_TIMESTAMP (*buf)
duration = GST_BUFFER_DURATION (*buf)

v4l2, current:  0.000000, ts:  0.000000, duration: 0.033367.
v4l2, current:  0.050000, ts:  0.016633, duration: 0.033367.
v4l2, current:  0.050000, ts:  0.016633, duration: 0.033367.
v4l2, current:  0.050000, ts:  0.016633, duration: 0.033367.
v4l2, current:  0.100000, ts:  0.066633, duration: 0.033367.
v4l2, current:  0.150000, ts:  0.116633, duration: 0.033367.
v4l2, current:  0.150000, ts:  0.116633, duration: 0.033367.

The first issue is that ts and duration are not consistent. It seems that the video timestamp is aligned with audio timestamp with some offset adjustment.

When I play back the video using totem, it reports that the frame rate is 60FPS. I am wondering if it is calculated from the first timestamp difference which is 0.016633.

Is this possible to make the timestamp and frame rate more consistent in gstreamer pipeline? Currently it looks very confusing.

Thanks,
Yiliang