Timestamp H.264 frame ramp up?

[pisymbol .]

Here's a weird thing:

I have a pipeline like so (this is the pertinent part):

... ! omxh264enc bitrate=10000000 ! video/x-h264, stream-format=(string)byte-stream ! h264parse ! identity name=h264id ! muxer.video_0

I use the "identity" plugin to tap the pipeline and count frames like so:

[
    {
        "delta": 0,
        "frame": 0,
        "timestamp": 769318722
    },
    {
        "delta": 19584421,
        "frame": 1,
        "timestamp": 788903143
    },
    {
        "delta": 6983473,
        "frame": 2,,
        "timestamp": 795886616
    },
    {
        "delta": 8273539,
        "frame": 3,
        "timestamp": 804160155
    },
    {
        "delta": 37436828,
        "frame": 4,
        "timestamp": 841596983
    },
    {
        "delta": 35012695,
        "frame": 5,
        "timestamp": 876609678
    },
    {
        "delta": 32337205,
        "frame": 6,
        "timestamp": 908946883
    },
    {
        "delta": 36766563,
        "frame": 7,
        "timestamp": 945713446
    },
    {
        "delta": 28523040,
        "frame": 8,,
        "timestamp": 974236486
    },
    {
        "delta": 35687439,
        "frame": 9,
        "timestamp": 1009923925
    },

I'm doing a fixed 30fps. However, notice the first few frames have odd deltas (delta is the difference between the current buf.pts minus the last frame's buf.pts).

But after about 3-4 frames, my deltas start to look good (~0.03 seconds or 1/30 or 30fps).

For why? Is there some kind of buffering going on I'm not aware of? How can I avoid this ramp up?

-aps

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Vinod Kesti]

pisymbol

It may be source giving wrong delta.

What is a source in your pipeline
Do similar analysis immediately after the source. That would give you some
Idea.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[pisymbol .]

On Thu, Jul 4, 2019 at 11:44 PM Vinod Kesti <[hidden email]> wrote:

pisymbol

It may be source giving wrong delta.

What is a source in your pipeline
Do similar analysis immediately after the source. That would give you some
Idea.

Vinod that is a great question. I am going to exactly that. Will report back.

-aps

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[pisymbol .]

On Fri, Jul 5, 2019 at 7:36 PM pisymbol . <[hidden email]> wrote:

On Thu, Jul 4, 2019 at 11:44 PM Vinod Kesti <[hidden email]> wrote:

pisymbol

It may be source giving wrong delta.

What is a source in your pipeline
Do similar analysis immediately after the source. That would give you some
Idea.

Vinod that is a great question. I am going to exactly that. Will report back.

[
    {
        "delta": 0,
        "frame": 0,
        "timestamp": 710794680
    },
    {
        "delta": 31127306,
        "frame": 1,
        "timestamp": 741921986
    },
    {
        "delta": 8828365,
        "frame": 2,
        "timestamp": 750750351
    },
    {
        "delta": 21636578,
        "frame": 3,
        "timestamp": 772386929
    },
    {
        "delta": 32609730,
        "frame": 4,
        "timestamp": 804996659
    },
    {
        "delta": 33260030,
        "frame": 5,
        "timestamp": 838256689
    },
    {
        "delta": 32897792,
        "frame": 6,
        "timestamp": 871154481
    },

...

So it's slightly better. But notice frame 2 and 3 have some kinda weird buf.pts timestamps before it starts to level out to about 30fps. The source is nvcamerasrc (i.e. nvcamerasrc (bunch of options) ! identity ..).

Is this just some kind of pipeline jitter?

-aps

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le sam. 6 juill. 2019 08 h 10, pisymbol . <[hidden email]> a écrit :

On Fri, Jul 5, 2019 at 7:36 PM pisymbol . <[hidden email]> wrote:

On Thu, Jul 4, 2019 at 11:44 PM Vinod Kesti <[hidden email]> wrote:

pisymbol

It may be source giving wrong delta.

What is a source in your pipeline
Do similar analysis immediately after the source. That would give you some
Idea.

Vinod that is a great question. I am going to exactly that. Will report back.

[
    {
        "delta": 0,
        "frame": 0,
        "timestamp": 710794680
    },
    {
        "delta": 31127306,
        "frame": 1,
        "timestamp": 741921986
    },
    {
        "delta": 8828365,
        "frame": 2,
        "timestamp": 750750351
    },
    {
        "delta": 21636578,
        "frame": 3,
        "timestamp": 772386929
    },
    {
        "delta": 32609730,
        "frame": 4,
        "timestamp": 804996659
    },
    {
        "delta": 33260030,
        "frame": 5,
        "timestamp": 838256689
    },
    {
        "delta": 32897792,
        "frame": 6,
        "timestamp": 871154481
    },

...

So it's slightly better. But notice frame 2 and 3 have some kinda weird buf.pts timestamps before it starts to level out to about 30fps. The source is nvcamerasrc (i.e. nvcamerasrc (bunch of options) ! identity ..).

Is this just some kind of pipeline jitter?

This looks like frame drops. GStreamer uses a push back model. So what I imagine here is that nvcamsrc produced a buffer, push which simply get queued. Then it produces a second buffer, but this time it's block for a certain amount of time (about 2-3 frames, probably initialization time, NVidia stuff is know for having long alloc/init time). In parallel to that, the capture HW ends up reusing that current buffer, because it might not have any other to capture in, so buffers are effectively being dropped. 

This is one a the wide variety of possibilities. Without traces from nvcamsrc and it's code, we are limited to guessing.

Nicolas

-aps

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[pisymbol .]

On Sat, Jul 6, 2019 at 9:12 AM Nicolas Dufresne <[hidden email]> wrote:

Le sam. 6 juill. 2019 08 h 10, pisymbol . <[hidden email]> a écrit :

On Fri, Jul 5, 2019 at 7:36 PM pisymbol . <[hidden email]> wrote:

On Thu, Jul 4, 2019 at 11:44 PM Vinod Kesti <[hidden email]> wrote:

pisymbol

It may be source giving wrong delta.

What is a source in your pipeline
Do similar analysis immediately after the source. That would give you some
Idea.

Vinod that is a great question. I am going to exactly that. Will report back.

[
    {
        "delta": 0,
        "frame": 0,
        "timestamp": 710794680
    },
    {
        "delta": 31127306,
        "frame": 1,
        "timestamp": 741921986
    },
    {
        "delta": 8828365,
        "frame": 2,
        "timestamp": 750750351
    },
    {
        "delta": 21636578,
        "frame": 3,
        "timestamp": 772386929
    },
    {
        "delta": 32609730,
        "frame": 4,
        "timestamp": 804996659
    },
    {
        "delta": 33260030,
        "frame": 5,
        "timestamp": 838256689
    },
    {
        "delta": 32897792,
        "frame": 6,
        "timestamp": 871154481
    },

...

So it's slightly better. But notice frame 2 and 3 have some kinda weird buf.pts timestamps before it starts to level out to about 30fps. The source is nvcamerasrc (i.e. nvcamerasrc (bunch of options) ! identity ..).

Is this just some kind of pipeline jitter?

This looks like frame drops. GStreamer uses a push back model. So what I imagine here is that nvcamsrc produced a buffer, push which simply get queued. Then it produces a second buffer, but this time it's block for a certain amount of time (about 2-3 frames, probably initialization time, NVidia stuff is know for having long alloc/init time). In parallel to that, the capture HW ends up reusing that current buffer, because it might not have any other to capture in, so buffers are effectively being dropped. 

This is one a the wide variety of possibilities. Without traces from nvcamsrc and it's code, we are limited to guessing.

Let me ask NVidia directly since you maybe right: This may have more to do with the hardware and their plugin than gstreamer itself.

One thing I also noticed using nvcamerasrc is that when I use max-file-duration of 10s I don't get 300 frames per capture file (I seem to be off anywhere from 30-50 frames during each rollover despite my pipeline fixed at 30fps).

Not sure yet what this is all about either.

-aps

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel