Audio / Video Synchronization issue through HLS gstreamer

[mohcer]

Hi,

I am trying to perform HLS through gstreamer with audio and video within a single pipeline media stream with a requirement to create .ts segments which are finally played back through a playlist file(m3u8 format). I am trying to achieve this with the following pipeline design:

gst-launch-1.5 v4l2src do-timestamp=true ! videoconvert ! videoscale ! video/x-raw,framerate=30/1 ! timeoverlay shaded-background=1 ! queue ! x264enc ! queue ! mpegtsmux name=mux alsasrc ! queue ! audioconvert ! queue ! audioresample ! queue ! audiorate ! voaacenc bitrate=160000 ! queue ! aacparse ! queue ! mux. mux. ! queue ! hlssink sync=false


The pipeline is launched successfully with audio / video in place but after some time the audio and video does not synchronize and end up in droping many audio frames based on the video timestamp in order to create a smooth audio stream with many pauses.

I get the following information in the debug log:

 WARNING: from element /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: Can't record audio fast enough
Additional debug info:
gstaudiobasesrc.c(869): gst_audio_base_src_create (): /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0:
Dropped 9261 samples. This is most likely because downstream can't keep up and is consuming samples too slowly.
WARNING: from element /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: Can't record audio fast enough
Additional debug info:
gstaudiobasesrc.c(869): gst_audio_base_src_create (): /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0:
Dropped 11025 samples. This is most likely because downstream can't keep up and is consuming samples too slowly.
WARNING: from element /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: Can't record audio fast enough
Additional debug info:
gstaudiobasesrc.c(869): gst_audio_base_src_create (): /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0:
Dropped 9261 samples. This is most likely because downstream can't keep up and is consuming samples too slowly.
WARNING: from element /GstPipeline:pipeline0/GstAlsaSrc:alsasrc0: Can't record audio fast enough

Can anyone suggest what is going wrong in this direction and in general how to improve audio / video synchronization.

[Nicolas Dufresne-4]

Le vendredi 30 septembre 2016 à 06:39 -0700, mohcer a écrit :
> gst-launch-1.5 v4l2src do-timestamp=true ! videoconvert ! videoscale

do-timestamp should never be set with v4l2src.

> !
> video/x-raw,framerate=30/1 ! timeoverlay shaded-background=1 ! queue
> !
> x264enc ! queue ! mpegtsmux name=mux alsasrc ! queue ! audioconvert !

Without any configuration, x264enc will pick a very complex compression
algorithm and introduce a significant latency. Configure that element
to reduce the CPU load and latency. Things like bitrate, setting
tune=zerolatency, adding a capsilter after to pick the H264 profile,
change the quantizer (if you feel like an expert) etc.

> queue
> ! audioresample ! queue ! audiorate ! voaacenc bitrate=160000 ! queue
> !
> aacparse ! queue ! mux. mux. ! queue ! hlssink sync=false

Try to stay on sync, don't diable sync. Note that mpegtsmux does not do
any type of synchronization other then marking the time on each stream
properly. As you have picked a very high latency video encoding method,
you'll get a huge drift between audio and video. Normally, sync should
not be affected, but at a certain level, it will.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel