1000 hours offset in timestamp of h264 stream

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

1000 hours offset in timestamp of h264 stream

GStreamer-devel mailing list
I have the following pipeline (using interpipesrc/sink from RidgeRun):

gst-launch-1.0 videotestsrc is-live=true pattern=ball !
video/x-raw,framerate=30/1,format=NV12 ! timeoverlay ! tee name=t ! queue !
videoconvert ! ximagesink sync=0 t. ! queue ! x264enc key-int-max=30
speed-preset=1 tune=zerolatency ! video/x-h264,profile=high ! h264parse !
queue ! interpipesink name=vtest1 sync=true async=false interpipesrc
listen-to=vtest1 name=vselect is-live=true stream-sync=passthrough-ts !
queue ! fakesink dump=1 sync=1  --gst-debug=*:3,*interpipe*:5

When sync=1 in the fakesink it doesn't dump any data, if I change to sync=0
it does.

If I remove the encoder (use RAW data) it dumps data for both cases.
The same is true if I remove the interpipesrc/sink and just plug the
fakesink directly into the encoder.

Also if I use recorded h264 data from a multifilesrc, instead of having an
encoder, it dumps data for both cases.

gst-launch-1.0 multifilesrc loop=1 name=replay location=/home/msb/test.ts !
identity sync=true ! tsdemux : identity sync=true ! h264parse ! queue !
interpipesink name=vreplay sync=false async=false interpipesrc
listen-to=vreplay is-live=true stream-sync=passthrough-ts ! fakesink dump=1
sync=1 --gst-debug=*:3,*interpipe*:5

The debug information from the interpipe elements reports timestamps with
1000 hours offset which I think it is the cause of the problem:

0:00:01.714147464 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vtest1> Incoming
serialized event tag
0:00:01.714232382 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vtest1> Event
timestamp 1000:00:01.633333333
0:00:01.714275617 1339403 0x55b0aa657520 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<vselect> Event tag with
calculated timestamp 1000:00:01.633333333 enqueued on serial pending events
0:00:02.030453992 1339403 0x55b0aa657700 INFO           interpipesink
gstinterpipesink.c:389:gst_inter_pipe_sink_intersect_listener_caps:<vtest1>
Listener vselect caps: ANY
0:00:02.030554449 1339403 0x55b0aa657700 INFO           interpipesink
gstinterpipesink.c:443:gst_inter_pipe_sink_get_caps:<vtest1> Caps
negotiated: video/x-h264, profile=(string)high, framerate=(fraction)[ 0/1,
2147483647/1 ], width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ],
parsed=(boolean)true, stream-format=(string){ avc, avc3, byte-stream },
alignment=(string){ au, nal }
0:00:02.047439856 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vtest1> Incoming
serialized event tag
0:00:02.047497306 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vtest1> Event
timestamp 1000:00:01.966666666
0:00:02.047530207 1339403 0x55b0aa657520 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<vselect> Event tag with
calculated timestamp 1000:00:01.966666666 enqueued on serial pending events

If I use RAW data the offset is not present, and if I use h264 encoded data
coming from the multifilesrc the offset is not present either:

0:00:01.911863974 1339557 0x559c28433d20 DEBUG           interpipesrc
gstinterpipesrc.c:481:gst_inter_pipe_src_create:<interpipesrc0> Got event
with timestamp 0:00:01.258333333
0:00:01.911912193 1339557 0x559c28433d20 DEBUG           interpipesrc
gstinterpipesrc.c:489:gst_inter_pipe_src_create:<interpipesrc0> Sending
Serial Event tag
0:00:02.111425433 1339557 0x559c28433d80 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vreplay> Incoming
serialized event tag
0:00:02.111489516 1339557 0x559c28433d80 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vreplay> Event
timestamp 0:00:01.458333333
0:00:02.111523291 1339557 0x559c28433d80 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<interpipesrc0> Event
tag with calculated timestamp 0:00:01.458333333 enqueued on serial pending
events

I think that the problem is that the interpipe elements are not handling
properly the timestamp offset caused by the encoder.
My question is, how can they (or maybe I) fix it? How to handle this offset
properly?
Some existing elements must be doing it, where can I look?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

AW: 1000 hours offset in timestamp of h264 stream

GStreamer-devel mailing list
Hi!

You are right
Noticed an timestamp offset using  x264enc connected through an appsink/appsrc pair quite a while a ago
The only way I found to "fix" it was by subtracting 3600.000.000.000.000ns from the timestamp prior to pushing it into an appsrc

the only documentation I found was following statement in the 1.18 release notes:

avvidenc: shift output buffer timestamps and output segment by 1h just like x264enc does, to allow for negative DTS.




Von: gstreamer-devel <[hidden email]> im Auftrag von Marianna S. Buschle via gstreamer-devel <[hidden email]>
Gesendet: Montag, 31. Mai 2021 09:01
An: [hidden email] <[hidden email]>
Cc: Marianna S. Buschle <[hidden email]>
Betreff: 1000 hours offset in timestamp of h264 stream
 
I have the following pipeline (using interpipesrc/sink from RidgeRun):

gst-launch-1.0 videotestsrc is-live=true pattern=ball !
video/x-raw,framerate=30/1,format=NV12 ! timeoverlay ! tee name=t ! queue !
videoconvert ! ximagesink sync=0 t. ! queue ! x264enc key-int-max=30
speed-preset=1 tune=zerolatency ! video/x-h264,profile=high ! h264parse !
queue ! interpipesink name=vtest1 sync=true async=false interpipesrc
listen-to=vtest1 name=vselect is-live=true stream-sync=passthrough-ts !
queue ! fakesink dump=1 sync=1  --gst-debug=*:3,*interpipe*:5

When sync=1 in the fakesink it doesn't dump any data, if I change to sync=0
it does.

If I remove the encoder (use RAW data) it dumps data for both cases.
The same is true if I remove the interpipesrc/sink and just plug the
fakesink directly into the encoder.

Also if I use recorded h264 data from a multifilesrc, instead of having an
encoder, it dumps data for both cases.

gst-launch-1.0 multifilesrc loop=1 name=replay location=/home/msb/test.ts !
identity sync=true ! tsdemux : identity sync=true ! h264parse ! queue !
interpipesink name=vreplay sync=false async=false interpipesrc
listen-to=vreplay is-live=true stream-sync=passthrough-ts ! fakesink dump=1
sync=1 --gst-debug=*:3,*interpipe*:5

The debug information from the interpipe elements reports timestamps with
1000 hours offset which I think it is the cause of the problem:

0:00:01.714147464 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vtest1> Incoming
serialized event tag
0:00:01.714232382 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vtest1> Event
timestamp 1000:00:01.633333333
0:00:01.714275617 1339403 0x55b0aa657520 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<vselect> Event tag with
calculated timestamp 1000:00:01.633333333 enqueued on serial pending events
0:00:02.030453992 1339403 0x55b0aa657700 INFO           interpipesink
gstinterpipesink.c:389:gst_inter_pipe_sink_intersect_listener_caps:<vtest1>
Listener vselect caps: ANY
0:00:02.030554449 1339403 0x55b0aa657700 INFO           interpipesink
gstinterpipesink.c:443:gst_inter_pipe_sink_get_caps:<vtest1> Caps
negotiated: video/x-h264, profile=(string)high, framerate=(fraction)[ 0/1,
2147483647/1 ], width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ],
parsed=(boolean)true, stream-format=(string){ avc, avc3, byte-stream },
alignment=(string){ au, nal }
0:00:02.047439856 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vtest1> Incoming
serialized event tag
0:00:02.047497306 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vtest1> Event
timestamp 1000:00:01.966666666
0:00:02.047530207 1339403 0x55b0aa657520 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<vselect> Event tag with
calculated timestamp 1000:00:01.966666666 enqueued on serial pending events

If I use RAW data the offset is not present, and if I use h264 encoded data
coming from the multifilesrc the offset is not present either:

0:00:01.911863974 1339557 0x559c28433d20 DEBUG           interpipesrc
gstinterpipesrc.c:481:gst_inter_pipe_src_create:<interpipesrc0> Got event
with timestamp 0:00:01.258333333
0:00:01.911912193 1339557 0x559c28433d20 DEBUG           interpipesrc
gstinterpipesrc.c:489:gst_inter_pipe_src_create:<interpipesrc0> Sending
Serial Event tag
0:00:02.111425433 1339557 0x559c28433d80 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vreplay> Incoming
serialized event tag
0:00:02.111489516 1339557 0x559c28433d80 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vreplay> Event
timestamp 0:00:01.458333333
0:00:02.111523291 1339557 0x559c28433d80 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<interpipesrc0> Event
tag with calculated timestamp 0:00:01.458333333 enqueued on serial pending
events

I think that the problem is that the interpipe elements are not handling
properly the timestamp offset caused by the encoder.
My question is, how can they (or maybe I) fix it? How to handle this offset
properly?
Some existing elements must be doing it, where can I look?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: 1000 hours offset in timestamp of h264 stream

GStreamer-devel mailing list
After a lot of digging I have managed to find the bug related to this 1000
hours offset:
https://github.com/GStreamer/gst-plugins-ugly/blob/master/ext/x264/gstx264enc.c#L1383

So I can see how x264 is adding the offset and why.

But so far I haven't found how other elements are ignoring/removing this
offset.

I can see that if I add `mpegtsmux ! tsdemux` to the pipeline after the
`h264parse` the offset seems to be removed by the `mux` and things then work
as expected.

But I cant really see how it is doing it or how I should go about properly
fixing the interpipe elements.

So far my backup plan is to, as you say, to try re-timstamp the frames by
subtracting the offset...



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

AW: AW: 1000 hours offset in timestamp of h264 stream

GStreamer-devel mailing list
mpegtsmux interprets the first incoming timestamp as 0 ( the start-time-selection property defaults to zero)
same goes for flvmux
so basically both muxer offset the timestamps by default


Von: gstreamer-devel <[hidden email]> im Auftrag von Marianna S. Buschle via gstreamer-devel <[hidden email]>
Gesendet: Montag, 31. Mai 2021 12:49
An: [hidden email] <[hidden email]>
Cc: Marianna S. Buschle <[hidden email]>
Betreff: Re: AW: 1000 hours offset in timestamp of h264 stream
 
After a lot of digging I have managed to find the bug related to this 1000
hours offset:
https://github.com/GStreamer/gst-plugins-ugly/blob/master/ext/x264/gstx264enc.c#L1383

So I can see how x264 is adding the offset and why.

But so far I haven't found how other elements are ignoring/removing this
offset.

I can see that if I add `mpegtsmux ! tsdemux` to the pipeline after the
`h264parse` the offset seems to be removed by the `mux` and things then work
as expected.

But I cant really see how it is doing it or how I should go about properly
fixing the interpipe elements.

So far my backup plan is to, as you say, to try re-timstamp the frames by
subtracting the offset...



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: 1000 hours offset in timestamp of h264 stream

GStreamer-devel mailing list
Yes, I seem to have found where 'mpegtsmux' is doing the "magic":

0:00:03.550340073 600306 0x564a3d45b990 LOG                basetsmux
gstbasetsmux.c:1578:gst_base_ts_mux_clip:<mpegtsmux0:sink_0> buffer pts
1000:00:02.633333333 ->  0:00:02.647277730 running time
0:00:03.550395376 600306 0x564a3d45b990 LOG                basetsmux
gstbasetsmux.c:1602:gst_base_ts_mux_clip:<mpegtsmux0:sink_0> buffer dts
1000:00:02.633333333 -> +0:00:02.647277730 running time
0:00:03.550425254 600306 0x564a3d45b990 DEBUG              basetsmux
gstbasetsmux.c:1676:gst_base_ts_mux_find_best_pad:<mpegtsmux0> Best pad
found with 0:00:02.647277730: <mpegtsmux0:sink_0>
0:00:03.550450924 600306 0x564a3d45b990 DEBUG              basetsmux
gstbasetsmux.c:1094:gst_base_ts_mux_aggregate_buffer:<mpegtsmux0> Pads
collected
0:00:03.550471499 600306 0x564a3d45b990 DEBUG              basetsmux
gstbasetsmux.c:1187:gst_base_ts_mux_aggregate_buffer:<mpegtsmux0:sink_0>
Chose stream for output (PID: 0x0041)
0:00:03.550498231 600306 0x564a3d45b990 DEBUG              basetsmux
gstbasetsmux.c:1201:gst_base_ts_mux_aggregate_buffer:<mpegtsmux0> Buffer has
PTS  0:00:02.647277730 pts 238254

https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/blob/master/gst/mpegtsmux/gstbasetsmux.c#L1618

time = gst_segment_to_running_time (&agg_pad->segment, GST_FORMAT_TIME,
time);



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 1000 hours offset in timestamp of h264 stream

GStreamer-devel mailing list
In reply to this post by GStreamer-devel mailing list
Le lundi 31 mai 2021 à 02:01 -0500, Marianna S. Buschle via gstreamer-devel a
écrit :

> I have the following pipeline (using interpipesrc/sink from RidgeRun):
>
> gst-launch-1.0 videotestsrc is-live=true pattern=ball !
> video/x-raw,framerate=30/1,format=NV12 ! timeoverlay ! tee name=t ! queue !
> videoconvert ! ximagesink sync=0 t. ! queue ! x264enc key-int-max=30
> speed-preset=1 tune=zerolatency ! video/x-h264,profile=high ! h264parse !
> queue ! interpipesink name=vtest1 sync=true async=false interpipesrc
> listen-to=vtest1 name=vselect is-live=true stream-sync=passthrough-ts !
> queue ! fakesink dump=1 sync=1  --gst-debug=*:3,*interpipe*:5
>
> When sync=1 in the fakesink it doesn't dump any data, if I change to sync=0
> it does.

I'm guessing that interpipesink element, from RidgeRun do not handle the
GstSegment properly. It's been over 4 years now that x264enc add 1 day to the
timestamp in order to support negative DTS. The offset is balanced through the
GstSegment. Ignoring the segment is a bug, timestamp have no meaning without
their segment.

Please report to RidgeRun github.

>
> If I remove the encoder (use RAW data) it dumps data for both cases.
> The same is true if I remove the interpipesrc/sink and just plug the
> fakesink directly into the encoder.
>
> Also if I use recorded h264 data from a multifilesrc, instead of having an
> encoder, it dumps data for both cases.
>
> gst-launch-1.0 multifilesrc loop=1 name=replay location=/home/msb/test.ts !
> identity sync=true ! tsdemux : identity sync=true ! h264parse ! queue !
> interpipesink name=vreplay sync=false async=false interpipesrc
> listen-to=vreplay is-live=true stream-sync=passthrough-ts ! fakesink dump=1
> sync=1 --gst-debug=*:3,*interpipe*:5
>
> The debug information from the interpipe elements reports timestamps with
> 1000 hours offset which I think it is the cause of the problem:
>
> 0:00:01.714147464 1339403 0x55b0aa657520 INFO           interpipesink
> gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vtest1> Incoming
> serialized event tag
> 0:00:01.714232382 1339403 0x55b0aa657520 INFO           interpipesink
> gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vtest1> Event
> timestamp 1000:00:01.633333333
> 0:00:01.714275617 1339403 0x55b0aa657520 DEBUG           interpipesrc
> gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<vselect> Event tag with
> calculated timestamp 1000:00:01.633333333 enqueued on serial pending events
> 0:00:02.030453992 1339403 0x55b0aa657700 INFO           interpipesink
> gstinterpipesink.c:389:gst_inter_pipe_sink_intersect_listener_caps:<vtest1>
> Listener vselect caps: ANY
> 0:00:02.030554449 1339403 0x55b0aa657700 INFO           interpipesink
> gstinterpipesink.c:443:gst_inter_pipe_sink_get_caps:<vtest1> Caps
> negotiated: video/x-h264, profile=(string)high, framerate=(fraction)[ 0/1,
> 2147483647/1 ], width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ],
> parsed=(boolean)true, stream-format=(string){ avc, avc3, byte-stream },
> alignment=(string){ au, nal }
> 0:00:02.047439856 1339403 0x55b0aa657520 INFO           interpipesink
> gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vtest1> Incoming
> serialized event tag
> 0:00:02.047497306 1339403 0x55b0aa657520 INFO           interpipesink
> gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vtest1> Event
> timestamp 1000:00:01.966666666
> 0:00:02.047530207 1339403 0x55b0aa657520 DEBUG           interpipesrc
> gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<vselect> Event tag with
> calculated timestamp 1000:00:01.966666666 enqueued on serial pending events
>
> If I use RAW data the offset is not present, and if I use h264 encoded data
> coming from the multifilesrc the offset is not present either:
>
> 0:00:01.911863974 1339557 0x559c28433d20 DEBUG           interpipesrc
> gstinterpipesrc.c:481:gst_inter_pipe_src_create:<interpipesrc0> Got event
> with timestamp 0:00:01.258333333
> 0:00:01.911912193 1339557 0x559c28433d20 DEBUG           interpipesrc
> gstinterpipesrc.c:489:gst_inter_pipe_src_create:<interpipesrc0> Sending
> Serial Event tag
> 0:00:02.111425433 1339557 0x559c28433d80 INFO           interpipesink
> gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vreplay> Incoming
> serialized event tag
> 0:00:02.111489516 1339557 0x559c28433d80 INFO           interpipesink
> gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vreplay> Event
> timestamp 0:00:01.458333333
> 0:00:02.111523291 1339557 0x559c28433d80 DEBUG           interpipesrc
> gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<interpipesrc0> Event
> tag with calculated timestamp 0:00:01.458333333 enqueued on serial pending
> events
>
> I think that the problem is that the interpipe elements are not handling
> properly the timestamp offset caused by the encoder.
> My question is, how can they (or maybe I) fix it? How to handle this offset
> properly?
> Some existing elements must be doing it, where can I look?
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: 1000 hours offset in timestamp of h264 stream

GStreamer-devel mailing list
In reply to this post by GStreamer-devel mailing list
Le lundi 31 mai 2021 à 10:27 +0000, Winand Appelhoff via gstreamer-devel a écrit :
Hi!

You are right
Noticed an timestamp offset using  x264enc connected through an appsink/appsrc pair quite a while a ago
The only way I found to "fix" it was by subtracting 3600.000.000.000.000ns from the timestamp prior to pushing it into an appsrc

Same here, you application is likely ignoring the segment event, giving a meaning to timestamp is only possible while using the date in the segment event.


the only documentation I found was following statement in the 1.18 release notes:

avvidenc: shift output buffer timestamps and output segment by 1h just like x264enc does, to allow for negative DTS.




Von: gstreamer-devel <[hidden email]> im Auftrag von Marianna S. Buschle via gstreamer-devel <[hidden email]>
Gesendet: Montag, 31. Mai 2021 09:01
An: [hidden email] <[hidden email]>
Cc: Marianna S. Buschle <[hidden email]>
Betreff: 1000 hours offset in timestamp of h264 stream
 
I have the following pipeline (using interpipesrc/sink from RidgeRun):

gst-launch-1.0 videotestsrc is-live=true pattern=ball !
video/x-raw,framerate=30/1,format=NV12 ! timeoverlay ! tee name=t ! queue !
videoconvert ! ximagesink sync=0 t. ! queue ! x264enc key-int-max=30
speed-preset=1 tune=zerolatency ! video/x-h264,profile=high ! h264parse !
queue ! interpipesink name=vtest1 sync=true async=false interpipesrc
listen-to=vtest1 name=vselect is-live=true stream-sync=passthrough-ts !
queue ! fakesink dump=1 sync=1  --gst-debug=*:3,*interpipe*:5

When sync=1 in the fakesink it doesn't dump any data, if I change to sync=0
it does.

If I remove the encoder (use RAW data) it dumps data for both cases.
The same is true if I remove the interpipesrc/sink and just plug the
fakesink directly into the encoder.

Also if I use recorded h264 data from a multifilesrc, instead of having an
encoder, it dumps data for both cases.

gst-launch-1.0 multifilesrc loop=1 name=replay location=/home/msb/test.ts !
identity sync=true ! tsdemux : identity sync=true ! h264parse ! queue !
interpipesink name=vreplay sync=false async=false interpipesrc
listen-to=vreplay is-live=true stream-sync=passthrough-ts ! fakesink dump=1
sync=1 --gst-debug=*:3,*interpipe*:5

The debug information from the interpipe elements reports timestamps with
1000 hours offset which I think it is the cause of the problem:

0:00:01.714147464 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vtest1> Incoming
serialized event tag
0:00:01.714232382 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vtest1> Event
timestamp 1000:00:01.633333333
0:00:01.714275617 1339403 0x55b0aa657520 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<vselect> Event tag with
calculated timestamp 1000:00:01.633333333 enqueued on serial pending events
0:00:02.030453992 1339403 0x55b0aa657700 INFO           interpipesink
gstinterpipesink.c:389:gst_inter_pipe_sink_intersect_listener_caps:<vtest1>
Listener vselect caps: ANY
0:00:02.030554449 1339403 0x55b0aa657700 INFO           interpipesink
gstinterpipesink.c:443:gst_inter_pipe_sink_get_caps:<vtest1> Caps
negotiated: video/x-h264, profile=(string)high, framerate=(fraction)[ 0/1,
2147483647/1 ], width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ],
parsed=(boolean)true, stream-format=(string){ avc, avc3, byte-stream },
alignment=(string){ au, nal }
0:00:02.047439856 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vtest1> Incoming
serialized event tag
0:00:02.047497306 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vtest1> Event
timestamp 1000:00:01.966666666
0:00:02.047530207 1339403 0x55b0aa657520 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<vselect> Event tag with
calculated timestamp 1000:00:01.966666666 enqueued on serial pending events

If I use RAW data the offset is not present, and if I use h264 encoded data
coming from the multifilesrc the offset is not present either:

0:00:01.911863974 1339557 0x559c28433d20 DEBUG           interpipesrc
gstinterpipesrc.c:481:gst_inter_pipe_src_create:<interpipesrc0> Got event
with timestamp 0:00:01.258333333
0:00:01.911912193 1339557 0x559c28433d20 DEBUG           interpipesrc
gstinterpipesrc.c:489:gst_inter_pipe_src_create:<interpipesrc0> Sending
Serial Event tag
0:00:02.111425433 1339557 0x559c28433d80 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vreplay> Incoming
serialized event tag
0:00:02.111489516 1339557 0x559c28433d80 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vreplay> Event
timestamp 0:00:01.458333333
0:00:02.111523291 1339557 0x559c28433d80 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<interpipesrc0> Event
tag with calculated timestamp 0:00:01.458333333 enqueued on serial pending
events

I think that the problem is that the interpipe elements are not handling
properly the timestamp offset caused by the encoder.
My question is, how can they (or maybe I) fix it? How to handle this offset
properly?
Some existing elements must be doing it, where can I look?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: 1000 hours offset in timestamp of h264 stream

GStreamer-devel mailing list
In reply to this post by GStreamer-devel mailing list
Le lundi 31 mai 2021 à 11:16 +0000, Winand Appelhoff via gstreamer-devel a
écrit :
> mpegtsmux interprets the first incoming timestamp as 0 ( the start-time-
> selection property defaults to zero)
> same goes for flvmux
> so basically both muxer offset the timestamps by default

Not entirely accurate, the muxers uses stream-time, which is a time scale
converted using the Segment. See gst_segment* APIs.

>
> Von: gstreamer-devel <[hidden email]> im
> Auftrag von Marianna S. Buschle via gstreamer-devel <
> [hidden email]>
> Gesendet: Montag, 31. Mai 2021 12:49
> An: [hidden email] <
> [hidden email]>
> Cc: Marianna S. Buschle <[hidden email]>
> Betreff: Re: AW: 1000 hours offset in timestamp of h264 stream 
> After a lot of digging I have managed to find the bug related to this 1000
> hours offset:
> https://github.com/GStreamer/gst-plugins-ugly/blob/master/ext/x264/gstx264enc.c#L1383
>
> So I can see how x264 is adding the offset and why.
>
> But so far I haven't found how other elements are ignoring/removing this
> offset.
>
> I can see that if I add `mpegtsmux ! tsdemux` to the pipeline after the
> `h264parse` the offset seems to be removed by the `mux` and things then work
> as expected.
>
> But I cant really see how it is doing it or how I should go about properly
> fixing the interpipe elements.
>
> So far my backup plan is to, as you say, to try re-timstamp the frames by
> subtracting the offset...
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: 1000 hours offset in timestamp of h264 stream

GStreamer-devel mailing list
thanks, I have opened https://github.com/RidgeRun/gst-interpipe/issues/89



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

AW: AW: AW: 1000 hours offset in timestamp of h264 stream

GStreamer-devel mailing list
Hi,
I too am using interpipes (GStreamer 1.18.4 and the latest Ridgerun Interpipes) for recording, streaming and displaying live video. All our recordins are playable with Mediaplayer /VLC and ffplay.
Gruesse

-----Ursprüngliche Nachricht-----
Von: gstreamer-devel <[hidden email]> Im Auftrag von Marianna S. Buschle via gstreamer-devel
Gesendet: Dienstag, 1. Juni 2021 09:35
An: [hidden email]
Cc: Marianna S. Buschle <[hidden email]>
Betreff: Re: AW: AW: 1000 hours offset in timestamp of h264 stream

thanks, I have opened https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FRidgeRun%2Fgst-interpipe%2Fissues%2F89&amp;data=04%7C01%7C%7Cabafb55a740445169d5308d924d99655%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C637581339226984103%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=knUfGecZ5M0x%2BBuJBcEYBHblALAC1VtXj1tu6QqCE0E%3D&amp;reserved=0



--
Sent from: https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgstreamer-devel.966125.n4.nabble.com%2F&amp;data=04%7C01%7C%7Cabafb55a740445169d5308d924d99655%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C637581339226984103%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=WO92k1WlF4BFHdctNnXOQzWhBE3RPBHiC6rfw6U64tc%3D&amp;reserved=0
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fgstreamer-devel&amp;data=04%7C01%7C%7Cabafb55a740445169d5308d924d99655%7C28042244bb514cd680347776fa3703e8%7C1%7C0%7C637581339226984103%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=cOjyMVWRfU3B8hC2Ojbk0XQR5BSxulKRGLc%2F3CljgZY%3D&amp;reserved=0
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

AW: AW: 1000 hours offset in timestamp of h264 stream

GStreamer-devel mailing list
In reply to this post by GStreamer-devel mailing list
Hi Nicolas,

could you elaborate on this a little bit more?
I thought GstSegment was only relevant when processing seek events , at least that's how I interpreted the documentation

This helper structure holds the relevant values for tracking the region of interest in a media file, called a segment.

The structure can be used for two purposes:

  • performing seeks (handling seek events)
  • tracking playback regions (handling newsegment events)

so, basically I should be using  gst_segment_to_stream_time(<GstSegment>, <timestamp of GstBuffer>) to get the "true" stream timestamp instead?

(sorry for hijacking this thread)

Von: Nicolas Dufresne <[hidden email]>
Gesendet: Montag, 31. Mai 2021 15:55
An: Discussion of the development of and with GStreamer <[hidden email]>
Cc: Winand Appelhoff <[hidden email]>; Marianna S. Buschle <[hidden email]>
Betreff: Re: AW: 1000 hours offset in timestamp of h264 stream
 
Le lundi 31 mai 2021 à 10:27 +0000, Winand Appelhoff via gstreamer-devel a écrit :
Hi!

You are right
Noticed an timestamp offset using  x264enc connected through an appsink/appsrc pair quite a while a ago
The only way I found to "fix" it was by subtracting 3600.000.000.000.000ns from the timestamp prior to pushing it into an appsrc

Same here, you application is likely ignoring the segment event, giving a meaning to timestamp is only possible while using the date in the segment event.


the only documentation I found was following statement in the 1.18 release notes:

avvidenc: shift output buffer timestamps and output segment by 1h just like x264enc does, to allow for negative DTS.




Von: gstreamer-devel <[hidden email]> im Auftrag von Marianna S. Buschle via gstreamer-devel <[hidden email]>
Gesendet: Montag, 31. Mai 2021 09:01
An: [hidden email] <[hidden email]>
Cc: Marianna S. Buschle <[hidden email]>
Betreff: 1000 hours offset in timestamp of h264 stream
 
I have the following pipeline (using interpipesrc/sink from RidgeRun):

gst-launch-1.0 videotestsrc is-live=true pattern=ball !
video/x-raw,framerate=30/1,format=NV12 ! timeoverlay ! tee name=t ! queue !
videoconvert ! ximagesink sync=0 t. ! queue ! x264enc key-int-max=30
speed-preset=1 tune=zerolatency ! video/x-h264,profile=high ! h264parse !
queue ! interpipesink name=vtest1 sync=true async=false interpipesrc
listen-to=vtest1 name=vselect is-live=true stream-sync=passthrough-ts !
queue ! fakesink dump=1 sync=1  --gst-debug=*:3,*interpipe*:5

When sync=1 in the fakesink it doesn't dump any data, if I change to sync=0
it does.

If I remove the encoder (use RAW data) it dumps data for both cases.
The same is true if I remove the interpipesrc/sink and just plug the
fakesink directly into the encoder.

Also if I use recorded h264 data from a multifilesrc, instead of having an
encoder, it dumps data for both cases.

gst-launch-1.0 multifilesrc loop=1 name=replay location=/home/msb/test.ts !
identity sync=true ! tsdemux : identity sync=true ! h264parse ! queue !
interpipesink name=vreplay sync=false async=false interpipesrc
listen-to=vreplay is-live=true stream-sync=passthrough-ts ! fakesink dump=1
sync=1 --gst-debug=*:3,*interpipe*:5

The debug information from the interpipe elements reports timestamps with
1000 hours offset which I think it is the cause of the problem:

0:00:01.714147464 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vtest1> Incoming
serialized event tag
0:00:01.714232382 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vtest1> Event
timestamp 1000:00:01.633333333
0:00:01.714275617 1339403 0x55b0aa657520 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<vselect> Event tag with
calculated timestamp 1000:00:01.633333333 enqueued on serial pending events
0:00:02.030453992 1339403 0x55b0aa657700 INFO           interpipesink
gstinterpipesink.c:389:gst_inter_pipe_sink_intersect_listener_caps:<vtest1>
Listener vselect caps: ANY
0:00:02.030554449 1339403 0x55b0aa657700 INFO           interpipesink
gstinterpipesink.c:443:gst_inter_pipe_sink_get_caps:<vtest1> Caps
negotiated: video/x-h264, profile=(string)high, framerate=(fraction)[ 0/1,
2147483647/1 ], width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ],
parsed=(boolean)true, stream-format=(string){ avc, avc3, byte-stream },
alignment=(string){ au, nal }
0:00:02.047439856 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vtest1> Incoming
serialized event tag
0:00:02.047497306 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vtest1> Event
timestamp 1000:00:01.966666666
0:00:02.047530207 1339403 0x55b0aa657520 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<vselect> Event tag with
calculated timestamp 1000:00:01.966666666 enqueued on serial pending events

If I use RAW data the offset is not present, and if I use h264 encoded data
coming from the multifilesrc the offset is not present either:

0:00:01.911863974 1339557 0x559c28433d20 DEBUG           interpipesrc
gstinterpipesrc.c:481:gst_inter_pipe_src_create:<interpipesrc0> Got event
with timestamp 0:00:01.258333333
0:00:01.911912193 1339557 0x559c28433d20 DEBUG           interpipesrc
gstinterpipesrc.c:489:gst_inter_pipe_src_create:<interpipesrc0> Sending
Serial Event tag
0:00:02.111425433 1339557 0x559c28433d80 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vreplay> Incoming
serialized event tag
0:00:02.111489516 1339557 0x559c28433d80 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vreplay> Event
timestamp 0:00:01.458333333
0:00:02.111523291 1339557 0x559c28433d80 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<interpipesrc0> Event
tag with calculated timestamp 0:00:01.458333333 enqueued on serial pending
events

I think that the problem is that the interpipe elements are not handling
properly the timestamp offset caused by the encoder.
My question is, how can they (or maybe I) fix it? How to handle this offset
properly?
Some existing elements must be doing it, where can I look?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: AW: AW: 1000 hours offset in timestamp of h264 stream

GStreamer-devel mailing list
Le mardi 01 juin 2021 à 09:28 +0000, Winand Appelhoff a écrit :
Hi Nicolas,

could you elaborate on this a little bit more?
I thought GstSegment was only relevant when processing seek events , at least that's how I interpreted the documentation

GstSegment contains the required information to interpret the following buffers timestamp. I guess the doc could be improve, interpreting GstBuffer timestamp is not that common, so it wasn't included there.


This helper structure holds the relevant values for tracking the region of interest in a media file, called a segment.

The structure can be used for two purposes:

  • performing seeks (handling seek events)
  • tracking playback regions (handling newsegment events)

so, basically I should be using  gst_segment_to_stream_time(<GstSegment>, <timestamp of GstBuffer>) to get the "true" stream timestamp instead?

Correct, note I made a typo, most applicaiton will likely want to deal with running-time. The stream-time is like the position within a stream (e.g. if you are playing a file with a container like ISOMP4, it would translate the timestamp into the position inside the current file). The running-time is the time you pipeline have been playing. So starting from 0 as you asked for.


(sorry for hijacking this thread)

Von: Nicolas Dufresne <[hidden email]>
Gesendet: Montag, 31. Mai 2021 15:55
An: Discussion of the development of and with GStreamer <[hidden email]>
Cc: Winand Appelhoff <[hidden email]>; Marianna S. Buschle <[hidden email]>
Betreff: Re: AW: 1000 hours offset in timestamp of h264 stream
 
Le lundi 31 mai 2021 à 10:27 +0000, Winand Appelhoff via gstreamer-devel a écrit :
Hi!

You are right
Noticed an timestamp offset using  x264enc connected through an appsink/appsrc pair quite a while a ago
The only way I found to "fix" it was by subtracting 3600.000.000.000.000ns from the timestamp prior to pushing it into an appsrc

Same here, you application is likely ignoring the segment event, giving a meaning to timestamp is only possible while using the date in the segment event.


the only documentation I found was following statement in the 1.18 release notes:

avvidenc: shift output buffer timestamps and output segment by 1h just like x264enc does, to allow for negative DTS.




Von: gstreamer-devel <[hidden email]> im Auftrag von Marianna S. Buschle via gstreamer-devel <[hidden email]>
Gesendet: Montag, 31. Mai 2021 09:01
An: [hidden email] <[hidden email]>
Cc: Marianna S. Buschle <[hidden email]>
Betreff: 1000 hours offset in timestamp of h264 stream
 
I have the following pipeline (using interpipesrc/sink from RidgeRun):

gst-launch-1.0 videotestsrc is-live=true pattern=ball !
video/x-raw,framerate=30/1,format=NV12 ! timeoverlay ! tee name=t ! queue !
videoconvert ! ximagesink sync=0 t. ! queue ! x264enc key-int-max=30
speed-preset=1 tune=zerolatency ! video/x-h264,profile=high ! h264parse !
queue ! interpipesink name=vtest1 sync=true async=false interpipesrc
listen-to=vtest1 name=vselect is-live=true stream-sync=passthrough-ts !
queue ! fakesink dump=1 sync=1  --gst-debug=*:3,*interpipe*:5

When sync=1 in the fakesink it doesn't dump any data, if I change to sync=0
it does.

If I remove the encoder (use RAW data) it dumps data for both cases.
The same is true if I remove the interpipesrc/sink and just plug the
fakesink directly into the encoder.

Also if I use recorded h264 data from a multifilesrc, instead of having an
encoder, it dumps data for both cases.

gst-launch-1.0 multifilesrc loop=1 name=replay location=/home/msb/test.ts !
identity sync=true ! tsdemux : identity sync=true ! h264parse ! queue !
interpipesink name=vreplay sync=false async=false interpipesrc
listen-to=vreplay is-live=true stream-sync=passthrough-ts ! fakesink dump=1
sync=1 --gst-debug=*:3,*interpipe*:5

The debug information from the interpipe elements reports timestamps with
1000 hours offset which I think it is the cause of the problem:

0:00:01.714147464 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vtest1> Incoming
serialized event tag
0:00:01.714232382 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vtest1> Event
timestamp 1000:00:01.633333333
0:00:01.714275617 1339403 0x55b0aa657520 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<vselect> Event tag with
calculated timestamp 1000:00:01.633333333 enqueued on serial pending events
0:00:02.030453992 1339403 0x55b0aa657700 INFO           interpipesink
gstinterpipesink.c:389:gst_inter_pipe_sink_intersect_listener_caps:<vtest1>
Listener vselect caps: ANY
0:00:02.030554449 1339403 0x55b0aa657700 INFO           interpipesink
gstinterpipesink.c:443:gst_inter_pipe_sink_get_caps:<vtest1> Caps
negotiated: video/x-h264, profile=(string)high, framerate=(fraction)[ 0/1,
2147483647/1 ], width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ],
parsed=(boolean)true, stream-format=(string){ avc, avc3, byte-stream },
alignment=(string){ au, nal }
0:00:02.047439856 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vtest1> Incoming
serialized event tag
0:00:02.047497306 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vtest1> Event
timestamp 1000:00:01.966666666
0:00:02.047530207 1339403 0x55b0aa657520 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<vselect> Event tag with
calculated timestamp 1000:00:01.966666666 enqueued on serial pending events

If I use RAW data the offset is not present, and if I use h264 encoded data
coming from the multifilesrc the offset is not present either:

0:00:01.911863974 1339557 0x559c28433d20 DEBUG           interpipesrc
gstinterpipesrc.c:481:gst_inter_pipe_src_create:<interpipesrc0> Got event
with timestamp 0:00:01.258333333
0:00:01.911912193 1339557 0x559c28433d20 DEBUG           interpipesrc
gstinterpipesrc.c:489:gst_inter_pipe_src_create:<interpipesrc0> Sending
Serial Event tag
0:00:02.111425433 1339557 0x559c28433d80 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vreplay> Incoming
serialized event tag
0:00:02.111489516 1339557 0x559c28433d80 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vreplay> Event
timestamp 0:00:01.458333333
0:00:02.111523291 1339557 0x559c28433d80 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<interpipesrc0> Event
tag with calculated timestamp 0:00:01.458333333 enqueued on serial pending
events

I think that the problem is that the interpipe elements are not handling
properly the timestamp offset caused by the encoder.
My question is, how can they (or maybe I) fix it? How to handle this offset
properly?
Some existing elements must be doing it, where can I look?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel



_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: 1000 hours offset in timestamp of h264 stream

GStreamer-devel mailing list
Hello, thanks for the report. Nicolas is correct, there are some values that are
being computed manually, that should use the segment instead. Precisely because
of these scenarios.

Let us take a look.

On 1 Jun 2021, at 09:05, Nicolas Dufresne via gstreamer-devel <[hidden email]> wrote:


Le mardi 01 juin 2021 à 09:28 +0000, Winand Appelhoff a écrit :
Hi Nicolas,

could you elaborate on this a little bit more?
I thought GstSegment was only relevant when processing seek events , at least that's how I interpreted the documentation

GstSegment contains the required information to interpret the following buffers timestamp. I guess the doc could be improve, interpreting GstBuffer timestamp is not that common, so it wasn't included there.


This helper structure holds the relevant values for tracking the region of interest in a media file, called a segment.

The structure can be used for two purposes:

  • performing seeks (handling seek events)
  • tracking playback regions (handling newsegment events)

so, basically I should be using  gst_segment_to_stream_time(<GstSegment>, <timestamp of GstBuffer>) to get the "true" stream timestamp instead?

Correct, note I made a typo, most applicaiton will likely want to deal with running-time. The stream-time is like the position within a stream (e.g. if you are playing a file with a container like ISOMP4, it would translate the timestamp into the position inside the current file). The running-time is the time you pipeline have been playing. So starting from 0 as you asked for.


(sorry for hijacking this thread)

Von: Nicolas Dufresne <[hidden email]>
Gesendet: Montag, 31. Mai 2021 15:55
An: Discussion of the development of and with GStreamer <[hidden email]>
Cc: Winand Appelhoff <[hidden email]>; Marianna S. Buschle <[hidden email]>
Betreff: Re: AW: 1000 hours offset in timestamp of h264 stream
 
Le lundi 31 mai 2021 à 10:27 +0000, Winand Appelhoff via gstreamer-devel a écrit :
Hi!

You are right
Noticed an timestamp offset using  x264enc connected through an appsink/appsrc pair quite a while a ago
The only way I found to "fix" it was by subtracting 3600.000.000.000.000ns from the timestamp prior to pushing it into an appsrc

Same here, you application is likely ignoring the segment event, giving a meaning to timestamp is only possible while using the date in the segment event.


the only documentation I found was following statement in the 1.18 release notes:

avvidenc: shift output buffer timestamps and output segment by 1h just like x264enc does, to allow for negative DTS.




Von: gstreamer-devel <[hidden email]> im Auftrag von Marianna S. Buschle via gstreamer-devel <[hidden email]>
Gesendet: Montag, 31. Mai 2021 09:01
An: [hidden email] <[hidden email]>
Cc: Marianna S. Buschle <[hidden email]>
Betreff: 1000 hours offset in timestamp of h264 stream
 
I have the following pipeline (using interpipesrc/sink from RidgeRun):

gst-launch-1.0 videotestsrc is-live=true pattern=ball !
video/x-raw,framerate=30/1,format=NV12 ! timeoverlay ! tee name=t ! queue !
videoconvert ! ximagesink sync=0 t. ! queue ! x264enc key-int-max=30
speed-preset=1 tune=zerolatency ! video/x-h264,profile=high ! h264parse !
queue ! interpipesink name=vtest1 sync=true async=false interpipesrc
listen-to=vtest1 name=vselect is-live=true stream-sync=passthrough-ts !
queue ! fakesink dump=1 sync=1  --gst-debug=*:3,*interpipe*:5

When sync=1 in the fakesink it doesn't dump any data, if I change to sync=0
it does.

If I remove the encoder (use RAW data) it dumps data for both cases.
The same is true if I remove the interpipesrc/sink and just plug the
fakesink directly into the encoder.

Also if I use recorded h264 data from a multifilesrc, instead of having an
encoder, it dumps data for both cases.

gst-launch-1.0 multifilesrc loop=1 name=replay location=/home/msb/test.ts !
identity sync=true ! tsdemux : identity sync=true ! h264parse ! queue !
interpipesink name=vreplay sync=false async=false interpipesrc
listen-to=vreplay is-live=true stream-sync=passthrough-ts ! fakesink dump=1
sync=1 --gst-debug=*:3,*interpipe*:5

The debug information from the interpipe elements reports timestamps with
1000 hours offset which I think it is the cause of the problem:

0:00:01.714147464 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vtest1> Incoming
serialized event tag
0:00:01.714232382 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vtest1> Event
timestamp 1000:00:01.633333333
0:00:01.714275617 1339403 0x55b0aa657520 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<vselect> Event tag with
calculated timestamp 1000:00:01.633333333 enqueued on serial pending events
0:00:02.030453992 1339403 0x55b0aa657700 INFO           interpipesink
gstinterpipesink.c:389:gst_inter_pipe_sink_intersect_listener_caps:<vtest1>
Listener vselect caps: ANY
0:00:02.030554449 1339403 0x55b0aa657700 INFO           interpipesink
gstinterpipesink.c:443:gst_inter_pipe_sink_get_caps:<vtest1> Caps
negotiated: video/x-h264, profile=(string)high, framerate=(fraction)[ 0/1,
2147483647/1 ], width=(int)[ 1, 2147483647 ], height=(int)[ 1, 2147483647 ],
parsed=(boolean)true, stream-format=(string){ avc, avc3, byte-stream },
alignment=(string){ au, nal }
0:00:02.047439856 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vtest1> Incoming
serialized event tag
0:00:02.047497306 1339403 0x55b0aa657520 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vtest1> Event
timestamp 1000:00:01.966666666
0:00:02.047530207 1339403 0x55b0aa657520 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<vselect> Event tag with
calculated timestamp 1000:00:01.966666666 enqueued on serial pending events

If I use RAW data the offset is not present, and if I use h264 encoded data
coming from the multifilesrc the offset is not present either:

0:00:01.911863974 1339557 0x559c28433d20 DEBUG           interpipesrc
gstinterpipesrc.c:481:gst_inter_pipe_src_create:<interpipesrc0> Got event
with timestamp 0:00:01.258333333
0:00:01.911912193 1339557 0x559c28433d20 DEBUG           interpipesrc
gstinterpipesrc.c:489:gst_inter_pipe_src_create:<interpipesrc0> Sending
Serial Event tag
0:00:02.111425433 1339557 0x559c28433d80 INFO           interpipesink
gstinterpipesink.c:542:gst_inter_pipe_sink_forward_event:<vreplay> Incoming
serialized event tag
0:00:02.111489516 1339557 0x559c28433d80 INFO           interpipesink
gstinterpipesink.c:547:gst_inter_pipe_sink_forward_event:<vreplay> Event
timestamp 0:00:01.458333333
0:00:02.111523291 1339557 0x559c28433d80 DEBUG           interpipesrc
gstinterpipesrc.c:729:gst_inter_pipe_src_push_event:<interpipesrc0> Event
tag with calculated timestamp 0:00:01.458333333 enqueued on serial pending
events

I think that the problem is that the interpipe elements are not handling
properly the timestamp offset caused by the encoder.
My question is, how can they (or maybe I) fix it? How to handle this offset
properly?
Some existing elements must be doing it, where can I look?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel