Merging two stream from the same rtsp source into a matroska container

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Merging two stream from the same rtsp source into a matroska container

marom
I have a problem with the following pipeline:

gst-launch-1.0 -e uridecodebin caps="video/x-h264, parsed=true; audio/x-raw"
uri=<camera url> name=rtspsrc   matroskamux name=mux ! filesink
location=test.mkv  rtspsrc. ! identity ! mux.   rtspsrc. ! audioresample !
audio/x-raw,rate=16000 ! voaacenc ! aacparse ! mux.

I am getting two streams from an rtsp soruce and placing them into a
matroska container which is saved to a file. Essentially a diamond-shaped
pipeline:

                audio
 rtspsrc<             > matroska -> file
               video

The pipeline works, creates a playable file. But there is a problem with
rtpjitterbuffer. By setting
GST_DEBUG=2,rtpjit*:5 I can see that on some cameras audio packets come out
later and later, they are actually stacking inside Jitter buffer as
indicated by lines like the following:

rtpjitterbuffer
gstrtpjitterbuffer.c:3268:gst_rtp_jitter_buffer_chain:<rtpjitterbuffer1>
Pushed packet #18896, now 31 packets, head: 0, percent -1

Over time I see that the number of packets inside rtpjitterbuffer1 keeps
rising.
On the other hand if I simply play the same source like in the following
pipeline:

gst-launch-1.0 -e uridecodebin caps="video/x-h264, parsed=true; audio/x-raw"
uri=<camera url> name=rtspsrc rtspsrc. !  avdec_h264 ! videoconvert !
autovideosink rtspsrc. ! audioresample ! audioconvert ! autoaudiosink

then all is OK, audio plays correctly with no problem/delay- Looks like the
insertion of the matroska container forces some kind of synchronization
between the two sources that is badly estimated on audio side. Note:
inserting a queue on both branches does not help

Not all cameras exhibit this problem; one some it works without any problem.

Note: this is not my real problem, I am working with a pipeline build
programmatically where I noticed that audio packet come out later and later.
The above pipeline (with the GST_DEBUG set accordingly) is the simplest way
to reproduce the problem I have found.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Merging two stream from the same rtsp source into a matroska container

marom
Problem solved, or at least fully identified. The problem was the DTS of the
ho264 frame. It was far from PTS (about half the value) and matroska muxer
was  placing together H62 with this (fake) DTS with audio packets with a
different value.

So for example audio at second 5 was placed next to video at second 10
(since video at second 10 had DTS about 5). Hence all my troubles.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel