audiosink causes video framerate to drop to undetermined level (not 30fps)

[Matt McIntyre]

That's the best I can determine so far at least by removing it.  I'm writing a program using the ruby-gnome2 gstreamer bindings that captures video (v4l) and audio (standard soundcard), displays the video onscreen, loops the audio back out for monitoring, and records to disk using avi/wav.  I'll be glad to provide the code, but in consideration of everyone's time I'll provide the gstreamer pipeline I'm first since I'm sure its something simple I'm not doing right.

My pipeline is separated into several bins and looks like this:

AudioBin: (source ghost pad to the tee src)

autoaudiosrc ! audio/x-raw-int ! level ! tee name=audiotee

audiotee ! autoaudiosink

VideoBin: (source ghost pad to the tee src)

v4l2src ! video/x-raw-yuv, format=(fourcc)YUY2, framerate=(fraction)30000/1001, width=(int)720, height=(int)480 ! videorate ! tee name=videotee

videotee ! xvimagesink

RecorderBin: (audio and video sink pads linked to the audio queue and video queue respectively)

queue name=audioqueue ! avimux name=mux

queue name=videoqueue ! jpegenc ! mux ! filesink

AudioBin ! VideoBin ! RecorderBin

The (horrible mess) of code that does all this is as follows:

    @pipeline = Gst::Pipeline.new

    

    @audiobin = Gst::Bin.new()

    @audiobin.name = "AudioBin"

    

    @autoaudiosrc = Gst::ElementFactory.make("autoaudiosrc")

    @audiocaps = Gst::ElementFactory.make("capsfilter")

    @audiocaps.caps = Gst::Caps.parse("audio/x-raw-int")

    @level = Gst::ElementFactory.make("level")

    @level.peak_ttl = 0

    @level.peak_falloff = 20

    @audiotee = Gst::ElementFactory.make("tee")

    @audiosink = Gst::ElementFactory.make("autoaudiosink")

    @audiobin.add(@autoaudiosrc)

    @audiobin.add(@audiocaps)

    @audiobin.add(@level)

    @audiobin.add(@audiotee)

    @audiobin.add(@audiosink)

    @autoaudiosrc >> @audiocaps >> @audiotee    

    @audiotee >> @level >> @audiosink

    @audiotee_src_pad = @audiotee.get_request_pad("src%d")

    @audiobin_ghost_pad = Gst::GhostPad.new('src', @audiotee_src_pad)

    @audiobin.add_pad(@audiobin_ghost_pad)

    @videobin = Gst::Bin.new()

    @videobin.name = "VideoBin"

    @v4l2src = Gst::ElementFactory.make("v4l2src")

    @videocaps = Gst::ElementFactory.make("capsfilter")

    @videocaps.caps = Gst::Caps.parse("video/x-raw-yuv, format=(fourcc)YUY2, framerate=(fraction)30000/1001, width=(int)720, height=(int)480")

    @videorate = Gst::ElementFactory.make("videorate")

    @videotee = Gst::ElementFactory.make("tee")

    @videosink = Gst::ElementFactory.make("xvimagesink")

    @videosink.set_xwindow_id(@video.window.xid)

    

    @videobin.add(@v4l2src)

    @videobin.add(@videocaps)

    @videobin.add(@videorate)

    @videobin.add(@videotee)

    @videobin.add(@videosink)

    @v4l2src >> @videocaps >> @videorate >> @videotee

    @videotee >> @videosink

    @videotee_src_pad = @videotee.get_request_pad("src%d")

    @videobin_ghost_pad = Gst::GhostPad.new('src',@videotee_src_pad)

    @videobin.add_pad(@videobin_ghost_pad)

    

    @recorderbin = Gst::Bin.new()

    @recorderbin.name = "RecorderBin"

    @audioqueue = Gst::ElementFactory.make("queue")

    @videoqueue = Gst::ElementFactory.make("queue")

    @jpegenc = Gst::ElementFactory.make("jpegenc")

    @avimux = Gst::ElementFactory.make("avimux")

    @filesink = Gst::ElementFactory.make("filesink")

    @filesink.stop

    @filesink.location = "test.avi"

    @recorderbin.add(@audioqueue)

    @recorderbin.add(@videoqueue)

    @recorderbin.add(@jpegenc)

    @recorderbin.add(@avimux)

    @recorderbin.add(@filesink)

    

    @audioqueue >> @avimux

    @videoqueue >> @jpegenc >> @avimux

    @avimux >> @filesink

    

    @audioqueue_audio_sink_pad = @audioqueue.get_static_pad("sink")

    @recorderbin_audio_ghost_pad = Gst::GhostPad.new('audio', @audioqueue_audio_sink_pad)

    @recorderbin.add_pad(@recorderbin_audio_ghost_pad)

    @videoqueue_video_sink_pad = @videoqueue.get_static_pad("sink")

    @recorderbin_video_ghost_pad = Gst::GhostPad.new('video', @videoqueue_video_sink_pad)

    @recorderbin.add_pad(@recorderbin_video_ghost_pad)

    

    # add objects to the main pipeline

    @pipeline.add(@audiobin)

    @pipeline.add(@videobin)

    @pipeline.add(@recorderbin)

    

    @audiobin >> @recorderbin

    @videobin >> @recorderbin

Thanks for any help you can provide,

 
Matt

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Marco Ballesio]

Hi,

I'm not an expert of Ruby, btw there are a few sanity checks you may run (see below)..

On Fri, May 20, 2011 at 11:57 PM, Matt McIntyre <[hidden email]> wrote:

That's the best I can determine so far at least by removing it.  I'm writing a program using the ruby-gnome2 gstreamer bindings that captures video (v4l) and audio (standard soundcard), displays the video onscreen, loops the audio back out for monitoring, and records to disk using avi/wav.  I'll be glad to provide the code, but in consideration of everyone's time I'll provide the gstreamer pipeline I'm first since I'm sure its something simple I'm not doing right.

My pipeline is separated into several bins and looks like this:

AudioBin: (source ghost pad to the tee src)

autoaudiosrc ! audio/x-raw-int ! level ! tee name=audiotee

audiotee ! autoaudiosink

VideoBin: (source ghost pad to the tee src)

v4l2src ! video/x-raw-yuv, format=(fourcc)YUY2, framerate=(fraction)30000/1001, width=(int)720, height=(int)480 ! videorate ! tee name=videotee

videotee ! xvimagesink

here you're dealing with raw 720x480 YUY2 @ 30fps.. I would run "top" and give a look at the CPU usage.
 

RecorderBin: (audio and video sink pads linked to the audio queue and video queue respectively)

queue name=audioqueue ! avimux name=mux

queue name=videoqueue ! jpegenc ! mux ! filesink

AudioBin ! VideoBin ! RecorderBin

running the pipelines after setting GST_DEBUG=2 (or more) will give you some help.

Regards

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel