Buffer PTS drifts into the future

[mb2016]

Hi All,

I have a very simple Gstreamer (using version 1.6.0 at present) pipeline running on an Embedded ARM system:

rtspsrc -> rtph264depay -> appsink

        -> rtppcmudepay -> appsink

In the appsink I pass the buffers to my application for processing. However, over time the PTS value of the buffers gradually drift into the future.

My RTSP source is an IP Camera, providing H.264 video (1080P, 15fps) and PCM mulaw audio (single channel, 8000Hz).

Ideally I need to have the buffer PTS values in real time so I can match video frames (and audio) up to externally timed events, such that I'm given a Linux epoch where I need to find the appropriate video frames for within a buffer in my application.

I was under the impression that the jitter buffer should be compensating for any drift or mismatch in clocks?

I've tried a few things but nothing seems to resolve this problem! Before setting sync=false on my sinks, lag would build up in the live output from the appsink...and over the course of a few days I'd be 10, or so, minutes behind real time and the application's memory usage would increase accordingly (I believe it was stacking up the rtspsrc somewhere). I've tried adding queues between the depayloaders and sinks, and again no difference. (I used the camera's overlay feature to place the date and time in the video frame, comparing that time value to the timestamp shows the same problem - at the start they are in line, but after some running time, the disparity builds).

I've had the pipeline use the GST_CLOCK_TYPE_REALTIME (since I want buffer PTS with real time values) and I've also just tried using the defaults to no avail... In the appsink I decided to print out the pipeline clock time, the base time and the PTS of the buffer:

Example after running for a couple of minutes:
    Pipeline Clock:[16905692471587] Pipeline Base Time:[16662001463649] Buffer PTS:[243730006699]. A little calculation:
        clock - base = running time
        (16905692471587 - 16662001463649) = 243691007938, which is less than PTS: 243730006699 (bad).

And just for reference - the first buffer to arrive at the appsink:
    Pipeline Clock:[16663381713212] Pipeline Base Time:[16662001463649] Buffer PTS[480006699]. A little calculation:
        clock - base = running time

        (16663381713212 - 16662001463649) = 1380249563, which is greater than the PTS: 480006699 (good).

I've found some improvement by setting the buffer-mode to 0 and by setting ntp-time-source to 3. But essentially am just staving off the inevitable.

Any ideas?

Best Regards,

Martin

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Thu, 2016-10-20 at 14:30 +0100, Martin Beynon wrote:
> Hi All,
>
> I have a very simple Gstreamer (using version 1.6.0 at present)
> pipeline running on an Embedded ARM system:
> [...]

You should try with 1.8 ideally, there were many RTP related fixes.
Nothing specifically about this here, but it might still help.

> Example after running for a couple of minutes: 
>     Pipeline Clock:[16905692471587] Pipeline Base
> Time:[16662001463649] Buffer PTS:[243730006699]. A little
> calculation: 
>         clock - base = running time 
>         (16905692471587 - 16662001463649) = 243691007938, which is
> less than PTS: 243730006699 (bad).
>
> And just for reference - the first buffer to arrive at the appsink:
>     Pipeline Clock:[16663381713212] Pipeline Base
> Time:[16662001463649] Buffer PTS[480006699]. A little calculation: 
>         clock - base = running time 
>         (16663381713212 - 16662001463649) = 1380249563, which is
> greater than the PTS: 480006699 (good).

How do you calculate the PTS? In an rtspsrc pipeline, the buffer PTS
and running time of the buffer should always be the same.

Or do you mean at the appsink the packet is arriving late, i.e. buffer
running time is less than clock running time, and that difference is
steadily increasing?

If that is the case, you'll have to check the PTS calculation in
gst-plugins-good/gst/rtpmanager/rtpjitterbuffer.c to see where things
go wrong there.

Also is this with a TCP based or UDP based RTP stream? For TCP based
streams we currently don't do proper translation between the sender and
receiver clocks, which might cause such problems. However for TCP this
is also far from trivial due to all the buffering in the protocol and
the inherent non-realtime'ness. Detecting and fixing up cases where
things slowly drift apart should be possible to detect though once it
reaches a specific threshold, that would have to be implemented there
though.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[keyclip]

Hi Martin, did you ever make any progress on this issue? I have encountered
the exact same problem and am searching for a fix. Thanks.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel