Transmit Audio + Video - Redistributing Latency

[stproj]

Greetings,

Im trying to transmit both audio and video over rtp but the pipeline is stuck in PAUSE state. The pipeline that i use follows:

gst-launch-1.0 filesrc location=vid2.mp4 ! qtdemux name=demux      demux.audio_0 ! queue ! aacparse ! avdec_aac ! audioconvert ! rtpL16pay ! udpsink host=localhost port=5000     demux.video_0 ! queue ! h264parse ! avdec_h264 ! videoconvert ! x264enc ! rtph264pay ! udpsink host=127.0.0.1 port = 5002

If i cut off a branch i can view or listen the file on the other side, but if i have the whole pipeline the end results is:

Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Redistribute latency...
Redistribute latency...

Do you know why this happens? Or what should i do avoid the error when transmitting both audio and video?

[Sebastian Dröge-3]

On Tue, 2016-10-25 at 01:10 -0700, stproj wrote:

> Greetings,
>
> Im trying to transmit both audio and video over rtp but the pipeline is
> stuck in PAUSE state. The pipeline that i use follows:
>
> gst-launch-1.0 filesrc location=vid2.mp4 ! qtdemux name=demux     
> demux.audio_0 ! queue ! aacparse ! avdec_aac ! audioconvert ! rtpL16pay !
> udpsink host=localhost port=5000     demux.video_0 ! queue ! h264parse !
> avdec_h264 ! videoconvert ! x264enc ! rtph264pay ! udpsink host=127.0.0.1
> port = 5002
>
> If i cut off a branch i can view or listen the file on the other side, but
> if i have the whole pipeline the end results is:
>
> Setting pipeline to PAUSED ...
> Pipeline is PREROLLING ...
> Redistribute latency...
> Redistribute latency...
>
> Do you know why this happens? Or what should i do avoid the error when
> transmitting both audio and video? 

x264enc by default has ~3s of latency. Use a bigger queue for the
audio, or reduce the latency by e.g. setting tune=zerolatency on
x264enc.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[stproj]

Sebastian Dröge-3 wrote

On Tue, 2016-10-25 at 01:10 -0700, stproj wrote:
> Greetings,
>
> Im trying to transmit both audio and video over rtp but the pipeline is
> stuck in PAUSE state. The pipeline that i use follows:
>
> gst-launch-1.0 filesrc location=vid2.mp4 ! qtdemux name=demux     
> demux.audio_0 ! queue ! aacparse ! avdec_aac ! audioconvert ! rtpL16pay !
> udpsink host=localhost port=5000     demux.video_0 ! queue ! h264parse !
> avdec_h264 ! videoconvert ! x264enc ! rtph264pay ! udpsink host=127.0.0.1
> port = 5002
>
> If i cut off a branch i can view or listen the file on the other side, but
> if i have the whole pipeline the end results is:
>
> Setting pipeline to PAUSED ...
> Pipeline is PREROLLING ...
> Redistribute latency...
> Redistribute latency...
>
> Do you know why this happens? Or what should i do avoid the error when
> transmitting both audio and video? 

x264enc by default has ~3s of latency. Use a bigger queue for the
audio, or reduce the latency by e.g. setting tune=zerolatency on
x264enc.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


signature.asc (949 bytes) <http://gstreamer-devel.966125.n4.nabble.com/attachment/4680216/0/signature.asc>

Thank you for your answer,

Yes, both a bigger queue and tune = zerolatency on x264enc worked for me on video, but the results on audio is that i can listen to the sound in "slow motion", meaning that the sound is crippled. What i cannot tell is if its a problem from my receiver pipeline or sender. Im providing both and the gst_discoverer on the test video i am working. Do you think its a problem on source or destination pipelines?

Thank you in advance.


source pipeline:
gst-launch-1.0 filesrc location=vid2.mp4 ! qtdemux name=demux  \
    demux.audio_0 ! queue max-size-time = 3000000000 ! aacparse ! avdec_aac ! audioconvert ! rtpL16pay ! udpsink host=localhost port=5002 \
    demux.video_0 ! queue ! h264parse ! avdec_h264 ! videoconvert ! x264enc ! rtph264pay ! udpsink host=127.0.0.1 port = 5000

receiver audio pipeline
gst-launch-1.0 -v udpsrc port=5000 ! "application/x-rtp,media=(string)audio, clock-rate=(int)44100, width=16, height=16, encoding-name=(string)L16, encoding-params=(string)1, channels=(int)1, channel-positions=(int)1, payload=(int)96" ! rtpL16depay ! audioconvert ! autoaudiosink

gst-discoverer information on the video im making test:
$ gst-discoverer-1.0 vid2.mp4

Topology:
  container: Quicktime
    audio: MPEG-4 AAC
    video: H.264 (Constrained Baseline Profile)

Properties:
  Duration: 0:00:11.005000000
  Seekable: yes
  Tags:
      audio codec: MPEG-4 AAC audio
      maximum bitrate: 103608
      bitrate: 96408
      datetime: 2015-11-17T13:28:06Z
      QT atom: buffer of 25 bytes
      container format: ISO MP4/M4A
      video codec: H.264 / AVC

[Sebastian Dröge-3]

On Wed, 2016-10-26 at 03:24 -0700, stproj wrote:
>
> Thank you for your answer,
>
> Yes, both a bigger queue and tune = zerolatency on x264enc worked for me on
> video, but the results on audio is that i can listen to the sound in "slow
> motion", meaning that the sound is crippled. What i cannot tell is if its a
> problem from my receiver pipeline or sender. Im providing both and the
> gst_discoverer on the test video i am working. Do you think its a problem on
> source or destination pipelines? 

Is it also slow-motion if you only send audio, or only receive audio?

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[stproj]

It is in slow motion even when i cutoff the video branch.

[Sebastian Dröge-3]

On Mon, 2016-10-31 at 18:18 -0700, stproj wrote:
> It is in slow motion even when i cutoff the video branch.

Does it also happen if you use an rtpjitterbuffer before the
depayloader? Does it happen with any input file?

If it does, please file a bug here with a testcase to reproduce it:
  https://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[mudassar]

Hi,

I am streaming video and audio over rtmp to Wowza. Both video and audio are
captured using appsrc in my application with the following pipeline:

"appsrc name=videosrc format=3 is-live=true do-timestamp=true ! video/x-raw,
width=%ld, height=%ld, framerate=20/1, format=ARGB ! queue
max-size-buffers=1 ! videoconvert  ! vtenc_h264 realtime=true
max-keyframe-interval=60 bitrate=1000 ! h264parse ! flvmux name=mux
streamable=true  ! queue ! rtmpsink name=sink sync=true location=%s  appsrc
name=audiosrc format=3  blocksize=8192 do-timestamp=true is-live=true    !
queue ! audio/x-raw, channels=1, rate=44100, format=F64LE, width=64,
depth=64 ! audioconvert ! audiorate ! audioresample ! audio/x-raw,
channels=1, rate=44100, format=F32LE ! queue ! avenc_aac  ! aacparse ! mux."

The problem is that audio is always slightly earlier than audio in the live
stream. I have tried adding more queue in audio pipeline, I have tried to
set minimum latency=100000000 at videosrc as well as audiosrc with different
combinations ,nothing worked.

What should I do ?




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le lun. 2 juil. 2018 09:27, mudassar <[hidden email]> a écrit :

Hi,

I am streaming video and audio over rtmp to Wowza. Both video and audio are
captured using appsrc in my application with the following pipeline:

"appsrc name=videosrc format=3 is-live=true do-timestamp=true ! video/x-raw,
width=%ld, height=%ld, framerate=20/1, format=ARGB ! queue
max-size-buffers=1 ! videoconvert  ! vtenc_h264 realtime=true
max-keyframe-interval=60 bitrate=1000 ! h264parse ! flvmux name=mux
streamable=true  ! queue ! rtmpsink name=sink sync=true location=%s  appsrc
name=audiosrc format=3  blocksize=8192 do-timestamp=true is-live=true    !
queue ! audio/x-raw, channels=1, rate=44100, format=F64LE, width=64,
depth=64 ! audioconvert ! audiorate ! audioresample ! audio/x-raw,
channels=1, rate=44100, format=F32LE ! queue ! avenc_aac  ! aacparse ! mux."

The problem is that audio is always slightly earlier than audio in the live
stream. I have tried adding more queue in audio pipeline, I have tried to
set minimum latency=100000000 at videosrc as well as audiosrc with different
combinations ,nothing worked.

What should I do ?

It's the timestamp that will disctate synchronization of the audio and video. If you use do-timestamp, you then need to sync audio/video before pushing into appsrc.

--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel