Dropping Audio Samples, Downstream Can't Keep Up

[Eric-Stone]

CONTENTS DELETED

The author has deleted this message.

[Eric-Stone]

CONTENTS DELETED

The author has deleted this message.

[Dimitrios Katsaros]

The videorate element doesn't change the rate at which the hardware is capturing, so my guess is that it has something to do with that. It is possible that the muxer was being stalled which in turn would fill the audio pipeline and would eventually lead to sample loss on a live source. However I am not experienced with the windows elements so I can only speculate. One thing you may want to test is adding an audio rate in the pipeline with the videorate. Another would be to check what the actual framerate with the higher exposure is and setting the pipeline with that framerate rather than 30. The muxer may be expecting frames at the target rate but the source can't report that it can capture at a lower rate, leading to the audio piepline being stalled.

Dimitrios

On Fri, Jan 13, 2017 at 9:14 AM, Brendan Lockhart <[hidden email]> wrote:

Turns out what was happening is that the exposure on the webcam was too high. Lowering the exposure increased the framerate and fixed the warning.

That's strange, because it was happening even if I put a videorate element after the ksvideosrc.

Is this because ksvideosrc was producing bad timestamps? Is there a way that I can check that hypothesis?

On Thu, Jan 12, 2017 at 2:29 AM, Brendan Lockhart <[hidden email]> wrote:

Hello all,

I'm having an issue where when streaming video/audio with the pipeline:

gst-launch-1.0 ksvideosrc device-index=0 ! image/jpeg,width=1024,height=768,framerate=30/1 ! queue ! jpegparse ! jpegdec ! queue ! x264enc tune=zerolatency speed-preset=veryfast bitrate=4096 ! h264parse ! video/x-h264, profile=high ! queue ! flvmux streamable=true name=mux ! queue ! rtmpsink location=rtmp://<my-url> autoaudiosrc ! audio/x-raw ! queue ! audioconvert ! avenc_aac ! aacparse ! queue ! mux.

I eventually start generating errors:

  gst_audio_base_src_create (): /GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstDirectSoundSrc:autoaudiosrc0-actual-src-directsoun:
Dropped 1420020 samples. This is most likely because downstream can't keep up and is consuming samples too slowly.

 With this pipeline, my PC is at ~8% CPU utilization so I don't think it's because my PC isn't powerful enough.

I know this error has been brought up in the past, but unfortunately none of the proposed solutions seem to be fixing this problem. Does anyone have any insight as to what might be causing this error?

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Eric-Stone]

CONTENTS DELETED

The author has deleted this message.

[Dimitrios Katsaros]

Maybe the video frame timestamps are not being timestamped correctly, or you need to wait until the first frame is available. You can try changing the behavior of videorate to start from the first received frame using skip-to-first, but the default behavior should be doing what you need for the pipeline to work.

Another minor thing is since the camera has a variant rate you can try setting it before the videorate element to explicitly state that the rate is variant. I don't think it will have much of a difference but it is worth taking a shot. you can do that this way: "ksvideosrc device-index=0 ! image/jpeg,framerate=0/1 ! videorate ! image/jpeg,width=1024,height=768,framerate=30/1". Again, it's just a wild guess.

Also if you haven't done this yet, try debugging the demuxer using the "--gst-debug=flvmux:5" option on gst-launch. The last thing I can think of is that in some cases encoders/muxers need a certain amount of frames to start producing results. Try increasing the size of your queues in the audio pipeline to 5-10 seconds so that the video pipe has enough time to produce enough data.

If none of those work then I am kinda stumped. Maybe someone else can give a bit of insight on what we may be missing.

On Sat, Jan 14, 2017 at 10:22 AM, Brendan Lockhart <[hidden email]> wrote:

Thanks for the reply,

>The videorate element doesn't change the rate at which the hardware is capturing, so my guess is that it has something to do with that.

I know that it wouldn't change the hardware capture rate, but wouldn't it duplicate frames as needed to make the framerate constant? Does it not correctly set the timestamps?

>One thing you may want to test is adding an audio rate in the pipeline with the videorate.

I tried that, along with a few permutations of audioresample & audioconvert. Unfortunately that never fixed the issue.

 >Another would be to check what the actual framerate with the higher exposure is and setting the pipeline with that framerate rather than 30.

That's a good idea, but it wouldn't work in the case that the camera is on autoexposure, as the framerate varies with lighting.

>The muxer may be expecting frames at the target rate but the source can't report that it can capture at a lower rate, leading to the audio pipeline being stalled.

I agree that that's the most likely cause, but I still don't fully understand why videorate doesn't fix that.

On Fri, Jan 13, 2017 at 12:54 AM, Dimitrios Katsaros <[hidden email]> wrote:

The videorate element doesn't change the rate at which the hardware is capturing, so my guess is that it has something to do with that. It is possible that the muxer was being stalled which in turn would fill the audio pipeline and would eventually lead to sample loss on a live source. However I am not experienced with the windows elements so I can only speculate. One thing you may want to test is adding an audio rate in the pipeline with the videorate. Another would be to check what the actual framerate with the higher exposure is and setting the pipeline with that framerate rather than 30. The muxer may be expecting frames at the target rate but the source can't report that it can capture at a lower rate, leading to the audio piepline being stalled.

Dimitrios

On Fri, Jan 13, 2017 at 9:14 AM, Brendan Lockhart <[hidden email]> wrote:

Turns out what was happening is that the exposure on the webcam was too high. Lowering the exposure increased the framerate and fixed the warning.

That's strange, because it was happening even if I put a videorate element after the ksvideosrc.

Is this because ksvideosrc was producing bad timestamps? Is there a way that I can check that hypothesis?

On Thu, Jan 12, 2017 at 2:29 AM, Brendan Lockhart <[hidden email]> wrote:

Hello all,

I'm having an issue where when streaming video/audio with the pipeline:

gst-launch-1.0 ksvideosrc device-index=0 ! image/jpeg,width=1024,height=768,framerate=30/1 ! queue ! jpegparse ! jpegdec ! queue ! x264enc tune=zerolatency speed-preset=veryfast bitrate=4096 ! h264parse ! video/x-h264, profile=high ! queue ! flvmux streamable=true name=mux ! queue ! rtmpsink location=rtmp://<my-url> autoaudiosrc ! audio/x-raw ! queue ! audioconvert ! avenc_aac ! aacparse ! queue ! mux.

I eventually start generating errors:

  gst_audio_base_src_create (): /GstPipeline:pipeline0/GstAutoAudioSrc:autoaudiosrc0/GstDirectSoundSrc:autoaudiosrc0-actual-src-directsoun:
Dropped 1420020 samples. This is most likely because downstream can't keep up and is consuming samples too slowly.

 With this pipeline, my PC is at ~8% CPU utilization so I don't think it's because my PC isn't powerful enough.

I know this error has been brought up in the past, but unfortunately none of the proposed solutions seem to be fixing this problem. Does anyone have any insight as to what might be causing this error?

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel