Slight audio/video sync problem - any suggestions?
Locked
Slight audio/video sync problem - any suggestions?
 
|
I recently acquired a Logitech Quickcam Pro for Notebooks, and have been
trying to use gstreamer to capture synchronized audio/video from the device. The camera is a USB Video/Audio device, and I am using the v4l2 driver. My general configuration is: Ubuntu 7.10 > dpkg --list 'gstreamer*' | grep "^ii" ii gstreamer0.10-alsa 0.10.14-1ubuntu3 GStreamer plugin for ALSA ii gstreamer0.10-esd 0.10.6-0ubuntu4 GStreamer plugin for ESD ii gstreamer0.10-ffmpeg 0.10.2-2ubuntu1 FFmpeg plugin for GStreamer ii gstreamer0.10-gnomevfs 0.10.14-1ubuntu3 GStreamer plugin for GnomeVFS ii gstreamer0.10-gnonlin 0.10.9-1 non-linear editing module for GStreamer ii gstreamer0.10-plugins-bad 0.10.5-4ubuntu1 GStreamer plugins from the "bad" set ii gstreamer0.10-plugins-bad-multiverse 0.10.5-1 GStreamer plugins from the "bad" set (Multiv ii gstreamer0.10-plugins-base 0.10.14-1ubuntu3 GStreamer plugins from the "base" set ii gstreamer0.10-plugins-base-apps 0.10.14-1ubuntu3 GStreamer helper programs from the "base" se ii gstreamer0.10-plugins-good 0.10.6-0ubuntu4 GStreamer plugins from the "good" set ii gstreamer0.10-plugins-ugly 0.10.6-0ubuntu2 GStreamer plugins from the "ugly" set ii gstreamer0.10-plugins-ugly-multiverse 0.10.6-0ubuntu1 GStreamer plugins from the "ugly" set (Multi ii gstreamer0.10-tools 0.10.14-1ubuntu3 Tools for use with GStreamer ii gstreamer0.10-x 0.10.14-1ubuntu3 GStreamer plugins for X11 and Pango The command I am using is: gst-launch-0.10 v4l2src ! queue ! videorate ! video/x-raw-yuv,width=800,height=600,framerate=15/1 \ ! ffmpegcolorspace ! theoraenc ! queue ! oggmux name=mux alsasrc device="hw:1,0" \ ! audiorate ! audio/x-raw-int,rate=16000,channels=1,depth=16 ! queue ! audioconvert ! vorbisenc \ ! mux. mux. ! queue ! filesink sync=true location=test.ogg The result of that can be seen: http://www.buberel.org/test.ogg As you can see, there is a slight but significant sync problem between the video and audio. Are there any additional options that I can or should be using in order to better synchronize the audio and video? Thanks, Jason ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
Re: Slight audio/video sync problem - any suggestions?
|
|
Here is what I have confirmed - the following command will result in a VERY clear high quality audio-only recording on my configuration: > gst-launch-0.10 alsasrc device="hw:1,0" ! audio/x-raw-int,rate=16000,channels=1,depth=16 ! audioconvert ! vorbisenc ! oggmux ! filesink location=audio-only.ogg Which you can hear: http://www.buberel.org/audio-only.ogg But when I use Nei's command below, which has essentially the same audio parameters but with the additional 'queue' element to record audio and video, I see many of the following error messages: WARNING: from element /pipeline0/alsasrc0: Can't record audio fast enough Additional debug info: gstbaseaudiosrc.c(603): gst_base_audio_src_create (): /pipeline0/alsasrc0: dropped 13600 samples WARNING: from element /pipeline0/alsasrc0: Can't record audio fast enough Additional debug info: gstbaseaudiosrc.c(603): gst_base_audio_src_create (): /pipeline0/alsasrc0: dropped 12000 samples WARNING: from element /pipeline0/alsasrc0: Can't record audio fast enough Additional debug info: I am going to work on an ascii-art diagram of this pipeline in the hopes that it will help others see the mistake I am making. -jason Nie Jun wrote: gst-launch-0.10 v4l2src ! queue ! videorate ! video/x-raw-yuv,width=800,height=600,framerate=15/1 ! ffmpegcolorspace ! theoraenc ! queue ! oggmux name=mux alsasrc device="hw:1,0" ! audiorate ! audio/x-raw-int,rate=16000,channels=1,depth=16 ! queue ! audioconvert ! vorbisenc ! QUEUE ! mux. mux. ! queue ! filesink sync=true location=test.ogg hope it helps you. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
Re: Slight audio/video sync problem - any suggestions?
|
|
This could be another example of what is mentioned in the Note at the end of
section http://gentrans.sourceforge.net/docs/head/manual/html/howto.html#sect-recording (observe that introducing a queue actually entails a "separate thread", as it is mentioned in this context). In itself, that may not provide more of an explanation :-(, but may give some ideas/inspiration to variations/alternatives. Regards, Mark. Jason L. Buberel wrote: > Oddly enough, that makes things worse :( > > Here is what I have confirmed - the following command will result in a > VERY clear high quality audio-only recording on my configuration: > >> gst-launch-0.10 alsasrc device="hw:1,0" ! > audio/x-raw-int,rate=16000,channels=1,depth=16 ! audioconvert ! > vorbisenc ! oggmux ! filesink location=audio-only.ogg > > Which you can hear: > > http://www.buberel.org/audio-only.ogg > > But when I use Nei's command below, which has essentially the same audio > parameters but with the additional 'queue' element to record audio and > video, I see many of the following error messages: > > WARNING: from element /pipeline0/alsasrc0: Can't record audio fast enough > Additional debug info: > gstbaseaudiosrc.c(603): gst_base_audio_src_create (): /pipeline0/alsasrc0: > dropped 13600 samples > WARNING: from element /pipeline0/alsasrc0: Can't record audio fast enough > Additional debug info: > gstbaseaudiosrc.c(603): gst_base_audio_src_create (): /pipeline0/alsasrc0: > dropped 12000 samples > WARNING: from element /pipeline0/alsasrc0: Can't record audio fast enough > Additional debug info: > > I am going to work on an ascii-art diagram of this pipeline in the hopes > that it will help others see the mistake I am making. > > -jason > > > Nie Jun wrote: >> gst-launch-0.10 v4l2src ! queue ! videorate ! >> video/x-raw-yuv,width=800,height=600,framerate=15/1 ! >> ffmpegcolorspace ! theoraenc ! queue ! oggmux name=mux alsasrc >> device="hw:1,0" ! audiorate ! >> audio/x-raw-int,rate=16000,channels=1,depth=16 ! queue ! audioconvert >> ! vorbisenc ! QUEUE ! mux. mux. ! queue ! filesink sync=true >> location=test.ogg >> >> hope it helps you. >> > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > > > ------------------------------------------------------------------------ > > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.sourceforge.net/lists/listinfo/gstreamer-devel ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
Re: Slight audio/video sync problem - any suggestions?
|
|
Ack - that feels SOO CLOSE to
what I need. Unfortunately, the element being discussed - 'stamp' - no
longer appears to be part of gstreamer v0.10:
WARNING: erroneous pipeline: no element "stamp" And sure enough 'gst-inspect' does not include any reference to a 'stamp' element either. I shall keep digging! -jason Mark Nauwelaerts wrote: This could be another example of what is mentioned in the Note at the end of section http://gentrans.sourceforge.net/docs/head/manual/html/howto.html#sect-recording (observe that introducing a queue actually entails a "separate thread", as it is mentioned in this context). In itself, that may not provide more of an explanation :-(, but may give some ideas/inspiration to variations/alternatives. Regards, Mark. Jason L. Buberel wrote: ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
Re: Slight audio/video sync problem - any suggestions?
|
|
Well, I know :) , "stamp" never has been. That is, it is not part of a usual distro GStreamer installation (involving core, gst-plugins-good, etc). However, it is part of the gst-entrans collection also found at the given sourceforge URL (which would then require download, compiling, etc). You might do a temporary proof-of-concept (sort of) test instead that forces a copy using: - efence; pretty abusive and hits memory a bit - a combination of 2 x videocrop (adding and dropping a mock border) - some other creative combination ... However, neither of these has any of stamp's "fancy sync features" (if that's what you refer to), which is why it was made ... Regards, Mark. Jason L. Buberel wrote: > Ack - that feels SOO CLOSE to what I need. Unfortunately, the element > being discussed - 'stamp' - no longer appears to be part of gstreamer v0.10: > > WARNING: erroneous pipeline: no element "stamp" > > And sure enough 'gst-inspect' does not include any reference to a > 'stamp' element either. > > I shall keep digging! > > -jason > > Mark Nauwelaerts wrote: >> This could be another example of what is mentioned in the Note at the end of >> section >> http://gentrans.sourceforge.net/docs/head/manual/html/howto.html#sect-recording >> (observe that introducing a queue actually entails a "separate thread", as it is >> mentioned in this context). >> >> In itself, that may not provide more of an explanation :-(, >> but may give some ideas/inspiration to variations/alternatives. >> >> Regards, >> Mark. >> >> Jason L. Buberel wrote: >> >>> Oddly enough, that makes things worse :( >>> >>> Here is what I have confirmed - the following command will result in a >>> VERY clear high quality audio-only recording on my configuration: >>> >>> >>>> gst-launch-0.10 alsasrc device="hw:1,0" ! >>>> >>> audio/x-raw-int,rate=16000,channels=1,depth=16 ! audioconvert ! >>> vorbisenc ! oggmux ! filesink location=audio-only.ogg >>> >>> Which you can hear: >>> >>> http://www.buberel.org/audio-only.ogg >>> >>> But when I use Nei's command below, which has essentially the same audio >>> parameters but with the additional 'queue' element to record audio and >>> video, I see many of the following error messages: >>> >>> WARNING: from element /pipeline0/alsasrc0: Can't record audio fast enough >>> Additional debug info: >>> gstbaseaudiosrc.c(603): gst_base_audio_src_create (): /pipeline0/alsasrc0: >>> dropped 13600 samples >>> WARNING: from element /pipeline0/alsasrc0: Can't record audio fast enough >>> Additional debug info: >>> gstbaseaudiosrc.c(603): gst_base_audio_src_create (): /pipeline0/alsasrc0: >>> dropped 12000 samples >>> WARNING: from element /pipeline0/alsasrc0: Can't record audio fast enough >>> Additional debug info: >>> >>> I am going to work on an ascii-art diagram of this pipeline in the hopes >>> that it will help others see the mistake I am making. >>> >>> -jason >>> >>> >>> Nie Jun wrote: >>> >>>> gst-launch-0.10 v4l2src ! queue ! videorate ! >>>> video/x-raw-yuv,width=800,height=600,framerate=15/1 ! >>>> ffmpegcolorspace ! theoraenc ! queue ! oggmux name=mux alsasrc >>>> device="hw:1,0" ! audiorate ! >>>> audio/x-raw-int,rate=16000,channels=1,depth=16 ! queue ! audioconvert >>>> ! vorbisenc ! QUEUE ! mux. mux. ! queue ! filesink sync=true >>>> location=test.ogg >>>> >>>> hope it helps you. >>>> >>>> >>> ------------------------------------------------------------------------ >>> >>> ------------------------------------------------------------------------- >>> This SF.net email is sponsored by: Microsoft >>> Defy all challenges. Microsoft(R) Visual Studio 2008. >>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ >>> >>> >>> ------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> gstreamer-devel mailing list >>> [hidden email] >>> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel >>> > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > > > ------------------------------------------------------------------------ > > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.sourceforge.net/lists/listinfo/gstreamer-devel ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
Re: Slight audio/video sync problem - any suggestions?
|
|
Thanks to Mark for his wonderful
gentrans plugin, I have declared success!
You can read all about it at: http://www.buberel.org/serendipity/ The short version is that I now have a working gstreamer-based command that will capture very high quality video (high resolution, very few compression artifacts) and audio with near perfect audio/video sync using Theora/Vorbis/Ogg encoding. The resulting output can then be uploaded DIRECTLY to YouTube and plays back flawlessly. All of this using nothing more than a cheap USB webcam on Ubuntu. -Jason Mark Nauwelaerts wrote: Well, I know :) , "stamp" never has been. That is, it is not part of a usual distro GStreamer installation (involving core, gst-plugins-good, etc). However, it is part of the gst-entrans collection also found at the given sourceforge URL (which would then require download, compiling, etc). You might do a temporary proof-of-concept (sort of) test instead that forces a copy using: - efence; pretty abusive and hits memory a bit - a combination of 2 x videocrop (adding and dropping a mock border) - some other creative combination ... However, neither of these has any of stamp's "fancy sync features" (if that's what you refer to), which is why it was made ... Regards, Mark. Jason L. Buberel wrote: ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
Slight audio/video sync problem - any suggestions?
|
|
Hi, nice your blog http://www.buberel.org/serendipity/ and nice http://gentrans.sourceforge.net/ project But I think, unfortunately, it doesn't help for something more robust and professional. In case of your pipeline: gst-launch-0.10 v4l2src queue-size=16 ! stamp sync-margin=1 sync-interval=1 ! video/x-raw-yuv,width=800,height=600,framerate=15/1 ! queue2 max-size-buffers=1000 max-size-bytes=0 max-size-time=0 ! ffmpegcolorspace ! theoraenc quality=60 name=venc alsasrc device="hw:1,0" ! audio/x-raw-int,rate=16000,channels=1,depth=16 ! audioconvert ! queue2 max-size-buffers=1000 max-size-bytes=0 max-size-time=0 ! vorbisenc quality=0.9 name=aenc oggmux name=mux ! filesink location=test.ogg aenc. ! mux. venc. ! mux Considering you have 100% CPU usage, what happens is that you are delaying the problem a bit by using a large queue (1000) and again delaying the problem a bit with "stamp" element Please try with the following pipeline: gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv, format=\(fourcc\)I420, width=640, height=480 ! tee name=t ! queue2 ! timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 ! videoscale ! xvimagesink sync=false Then try change the resolution to the one your camera accept (width=800,height=600,framerate=15/1) and see if you can remove ffmpegcolorspace and videoscale if you don't have 100% you will have best result. if you have 100% then try to increase the queue size (or also use stamp) so it would be ok to make short videos (as you already did). - - - - - - - - - - - - - - - - - -- Now I would like discuss what would be the best way to record audio and video in 100% cpu case. if I use gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv, format=\(fourcc\)I420, width=1280, height=960 ! tee name=t ! queue2 ! timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 ! videoscale ! xvimagesink sync=false then my file isn't recorded properly (Can't record audio fast enough...this discontinuity breaks the recording) BUT gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv, format=\(fourcc\)I420, width=1280, height=960 ! tee name=t ! queue2 ! timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg alsasrc ! queue2 ! audioconvert ! vorbisenc ! oggmux ! filesink location=2.ogg mux. t. ! queue leaky=2 ! videoscale ! xvimagesink sync=false Both audio and video file are recorded fine (individually). In addition, the audio file has duration greater than video file. I think it happens because of video dropping. But I think it is close to the best thing that could be done. So what I would like to do is to give priority to audio as much as possible, and drop videos, but still having audio and video sync. So, How to do it? Best Regards, Edgard ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
Re: Slight audio/video sync problem - any suggestions?
|
|
Testing with the following command (very slightly modified from your suggestion): gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv,format=\(fourcc\)I420,width=800,height=600,framerate=15/1 ! tee name=t ! queue2 ! timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 ! videoscale ! xvimagesink sync=false Produced the following behavior: 1. A display window was opened, showing a static (single frame) of output from the video camera. 2. The initial output from gst-launch was: Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstAudioSrcClock 3. After waiting a while, I pressed Ctrl-C: Caught interrupt -- handling interrupt. Interrupt: Setting pipeline to PAUSED ... Execution ended after 6333893000 ns. Setting pipeline to PAUSED ... Setting pipeline to READY ... 4. After a little while longer, I pressed Ctrl-C again. 5. The output file '1.ogg' was zero bytes. Would this indicates that the 'tee' element is not quite configured correct? I've never used that before... -jason Edgard Lima wrote: Hi, nice your blog http://www.buberel.org/serendipity/ and nice http://gentrans.sourceforge.net/ project But I think, unfortunately, it doesn't help for something more robust and professional. In case of your pipeline: gst-launch-0.10 v4l2src queue-size=16 ! stamp sync-margin=1 sync-interval=1 ! video/x-raw-yuv,width=800,height=600,framerate=15/1 ! queue2 max-size-buffers=1000 max-size-bytes=0 max-size-time=0 ! ffmpegcolorspace ! theoraenc quality=60 name=venc alsasrc device="hw:1,0" ! audio/x-raw-int,rate=16000,channels=1,depth=16 ! audioconvert ! queue2 max-size-buffers=1000 max-size-bytes=0 max-size-time=0 ! vorbisenc quality=0.9 name=aenc oggmux name=mux ! filesink location=test.ogg aenc. ! mux. venc. ! mux Considering you have 100% CPU usage, what happens is that you are delaying the problem a bit by using a large queue (1000) and again delaying the problem a bit with "stamp" element Please try with the following pipeline: gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv, format=\(fourcc\)I420, width=640, height=480 ! tee name=t ! queue2 ! timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 ! videoscale ! xvimagesink sync=false Then try change the resolution to the one your camera accept (width=800,height=600,framerate=15/1) and see if you can remove ffmpegcolorspace and videoscale if you don't have 100% you will have best result. if you have 100% then try to increase the queue size (or also use stamp) so it would be ok to make short videos (as you already did). - - - - - - - - - - - - - - - - - -- Now I would like discuss what would be the best way to record audio and video in 100% cpu case. if I use gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv, format=\(fourcc\)I420, width=1280, height=960 ! tee name=t ! queue2 ! timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 ! videoscale ! xvimagesink sync=false then my file isn't recorded properly (Can't record audio fast enough...this discontinuity breaks the recording) BUT gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv, format=\(fourcc\)I420, width=1280, height=960 ! tee name=t ! queue2 ! timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg alsasrc ! queue2 ! audioconvert ! vorbisenc ! oggmux ! filesink location=2.ogg mux. t. ! queue leaky=2 ! videoscale ! xvimagesink sync=false Both audio and video file are recorded fine (individually). In addition, the audio file has duration greater than video file. I think it happens because of video dropping. But I think it is close to the best thing that could be done. So what I would like to do is to give priority to audio as much as possible, and drop videos, but still having audio and video sync. So, How to do it? Best Regards, Edgard ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
Re: Slight audio/video sync problem - any suggestions?
|
|
In reply to this post by Edgard Lima
try exactly this one: gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv, format=\(fourcc\)I420, width=800, height=600 ! tee name=t ! queue2 ! timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 ! videoscale ! xvimagesink sync=false what happens? 1- works fine, ok done 2- 100% cpu ... not good result then decrease the resolution to see the possible resolution try GST_DEBUG=*v4l2*:3 gst-launch-0.10 v4l2src num-buffers=1 ! fakesink * I can't see how to solve the problem of 100% cpu without some QoS smartness to decrease the video framerate BR, Edgard ext Jason L. Buberel wrote: > Some additional information on my device and it's capabilities. > > According to 'v4l-info': > > video capture > VIDIOC_ENUM_FMT(0,VIDEO_CAPTURE) > index : 0 > type : VIDEO_CAPTURE > flags : 1 > description : "MJPEG" > pixelformat : 0x47504a4d [MJPG] > VIDIOC_ENUM_FMT(1,VIDEO_CAPTURE) > index : 1 > type : VIDEO_CAPTURE > flags : 0 > description : "YUV 4:2:2 (YUYV)" > pixelformat : 0x56595559 [YUYV] > > I am assuming that the second format corresponds to 'video/x-raw-yuv' in > the gstreamer pipeline. As far as I can tell, there is no gstreamer > support for reading the camera data using this format. > > Starting with the basic - the following command works: > >> gst-launch-0.10 v4l2src ! ffmpegcolorspace ! > video/x-raw-yuv,width=800,height=600 ! xvimagesink sync=false > > Adding the timeoverlay also works: > >> gst-launch-0.10 v4l2src ! ffmpegcolorspace ! > video/x-raw-yuv,width=800,height=600 ! timeoverlay ! xvimagesink sync=false > > Adding the format=I420 also works when 'ffmpegcolorspace' comes early in > the pipline: > >> gst-launch-0.10 -v v4l2src ! ffmpegcolorspace ! > video/x-raw-yuv,width=800,height=600,format=\(fourcc\)I420 ! timeoverlay > ! xvimagesink sync=false > > But when I then attempt to remove the ffmpegcolorspace: > >> gst-launch-0.10 -v v4l2src ! > video/x-raw-yuv,width=800,height=600,format=\(fourcc\)I420 ! timeoverlay > ! xvimagesink sync=false > > Setting pipeline to PAUSED ... > ERROR: Pipeline doesn't want to pause. > ERROR: from element /pipeline0/v4l2src0: Could not negotiate format > Additional debug info: > gstbasesrc.c(2062): gst_base_src_start (): /pipeline0/v4l2src0: > Check your filtered caps, if any > Setting pipeline to NULL ... > FREEING pipeline ... > > However, if I change the format to YUV2, it works: > >> gst-launch-0.10 -v v4l2src ! > video/x-raw-yuv,width=800,height=600,format=\(fourcc\)YUY2 ! > ffmpegcolorspace ! timeoverlay ! xvimagesink sync=false > > This tells me that the camera simply does not support the I420 pixel > format - only YUY. And because the 'timeoverlay' always expects an input > format of 'video/x-raw-yuv,format=I420', I have to put the > ffmpegcolorspace element after the video capture element. > > I'm going to work a bit more on your command to see if I can get the > 'tee' element working properly. > > -jason > > > > > Edgard Lima wrote: >> Hi, >> >> nice your blog >> >> http://www.buberel.org/serendipity/ >> >> and nice >> >> http://gentrans.sourceforge.net/ >> >> project >> >> But I think, unfortunately, it doesn't help for something more robust >> and professional. >> >> In case of your pipeline: >> >> gst-launch-0.10 v4l2src queue-size=16 ! stamp sync-margin=1 >> sync-interval=1 >> ! video/x-raw-yuv,width=800,height=600,framerate=15/1 >> ! queue2 max-size-buffers=1000 max-size-bytes=0 max-size-time=0 ! >> ffmpegcolorspace >> ! theoraenc quality=60 name=venc alsasrc device="hw:1,0" >> ! audio/x-raw-int,rate=16000,channels=1,depth=16 >> ! audioconvert ! queue2 max-size-buffers=1000 max-size-bytes=0 >> max-size-time=0 >> ! vorbisenc quality=0.9 name=aenc oggmux name=mux ! filesink >> location=test.ogg aenc. ! mux. venc. ! mux >> >> Considering you have 100% CPU usage, what happens is that you are >> delaying the problem a bit by using a large queue (1000) and again >> delaying the problem a bit with "stamp" element >> >> Please try with the following pipeline: >> >> gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv, >> format=\(fourcc\)I420, width=640, height=480 ! tee name=t ! queue2 ! >> timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg >> alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 ! >> videoscale ! xvimagesink sync=false >> >> Then try change the resolution to the one your camera accept >> (width=800,height=600,framerate=15/1) and see if you can remove >> ffmpegcolorspace and videoscale >> >> if you don't have 100% you will have best result. >> >> if you have 100% then try to increase the queue size (or also use stamp) >> so it would be ok to make short videos (as you already did). >> >> - - - - - - - - - - - - - - - - - -- >> >> Now I would like discuss what would be the best way to record audio and >> video in 100% cpu case. >> >> if I use >> >> gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv, >> format=\(fourcc\)I420, width=1280, height=960 ! tee name=t ! queue2 ! >> timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg >> alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 ! >> videoscale ! xvimagesink sync=false >> >> then my file isn't recorded properly (Can't record audio fast >> enough...this discontinuity breaks the recording) >> >> BUT >> >> gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv, >> format=\(fourcc\)I420, width=1280, height=960 ! tee name=t ! queue2 ! >> timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg >> alsasrc ! queue2 ! audioconvert ! vorbisenc ! oggmux ! filesink >> location=2.ogg mux. t. ! queue leaky=2 ! videoscale ! xvimagesink >> sync=false >> >> Both audio and video file are recorded fine (individually). In addition, >> the audio file has duration greater than video file. I think it happens >> because of video dropping. >> >> But I think it is close to the best thing that could be done. So what I >> would like to do is to give priority to audio as much as possible, and >> drop videos, but still having audio and video sync. So, How to do it? >> >> Best Regards, >> Edgard >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> > ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
Re: Slight audio/video sync problem - any suggestions?
|
|
There must be something a bit off
with that command.
When issued, I get one still frame of video onscreen and the output: Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstAudioSrcClock I then press Ctrl-C and I see: Caught interrupt -- handling interrupt. Interrupt: Setting pipeline to PAUSED ... Execution ended after 35390780000 ns. Setting pipeline to PAUSED ... Setting pipeline to READY ... I then press Ctrl-C again to exit. The resulting file '1.ogg' is empty. Using the debug level 4 (instead of 3) flag with 'framerate=30/1,width=800,height=600' I get: > gst-launch-0.10 v4l2src num-buffers=1 ! ffmpegcolorspace ! fakesink 0:00:00.338550000 9822 0x804e050 DEBUG v4l2src gstv4l2src.c:398:gst_v4l2src_fixate:<v4l2src0> fixating caps video/x-raw-yuv, format=(fourcc)YUY2, width=(int)160, height=(int)120, framerate=(fraction){ 30/1, 25/1, 20/1, 15/1, 10/1, 5/1 } 0:00:00.339356000 9822 0x804e050 DEBUG v4l2src gstv4l2src.c:854:gst_v4l2src_set_caps:<v4l2src0> trying to set_capture 160x120 at 30/1 fps, format YUV 4:2:2 (YUYV) But this command give a different result: > gst-launch-0.10 v4l2src num-buffers=1 ! fakesink 0:00:00.203129000 9849 0x804e050 DEBUG v4l2src gstv4l2src.c:398:gst_v4l2src_fixate:<v4l2src0> fixating caps image/jpeg, width=(int)160, height=(int)120, framerate=(fraction){ 30/1, 25/1, 20/1, 15/1, 10/1, 5/1 } 0:00:00.203615000 9849 0x804e050 DEBUG v4l2src gstv4l2src.c:854:gst_v4l2src_set_caps:<v4l2src0> trying to set_capture 160x120 at 30/1 fps, format MJPEG What video/x-raw-? parameter would I have to use in order to force v4l2 to use what the 'luvcview' command indicates it can handle: 30fps @ 960x720 in MJPEG format? -jason Edgard Lima wrote: try exactly this one: gst-launch-0.10 v4l2src ! ffmpegcolorspace ! video/x-raw-yuv, format=\(fourcc\)I420, width=800, height=600 ! tee name=t ! queue2 ! timeoverlay ! theoraenc ! oggmux name=mux ! filesink location=1.ogg alsasrc ! queue2 ! audioconvert ! vorbisenc ! mux. t. ! queue leaky=2 ! videoscale ! xvimagesink sync=false what happens? 1- works fine, ok done 2- 100% cpu ... not good result then decrease the resolution to see the possible resolution try GST_DEBUG=*v4l2*:3 gst-launch-0.10 v4l2src num-buffers=1 ! fakesink * I can't see how to solve the problem of 100% cpu without some QoS smartness to decrease the video framerate BR, Edgard ext Jason L. Buberel wrote: ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
«
Return to GStreamer-devel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |