How to seek pipeline with appsrc?

[Halley Zhao]

Hi expert:

I have special pipeline: appsrc-->h264parse-->h264dec-->videosink

and use a customized clock to control the pipeline.

when I want to do some seek operation, it doesn't work well. I tried to dig into gstbasesink, but  still not a workable solution yet.

after play 3 seconds, I try to seek to 0 (file start)

approach 1: I also reset my clock time to restart from 0.

but, I found, the first buffer (pts=0) and other buffer before 3 second timestamp are dropped. and after no video update for 3 seconds, video playback continues with 3s+ frames.

I added some debug log to gstbasesink, and found that video frames before 3s are treated as 'too late' and are dropped.

I tried to compare it with seek of a normal pipeline (filesrc-->qtdemux-->h264parse-->h264dec-->videosink), and found that the base_time of gstbasesink is updated to the previous played time. 

so I try to gst_element_set_base_time() for the elements of my appsrc pipeline.

then the first video frame isn't dropped, but gst_base_sink_wait_clock() from gst_base_sink_do_sync() is blocked for 3 seconds. then the first video frame is rendered.

approach 2: I tried to create a new clock for the pipeline

after seek, I tried to use a new clock start from zero. it doesn't work well either, because element base_time keeps big number, gst_base_sink_wait_clock() is blocked for 3 seconds as well

to be honest, I don't know what's the correct way to seek a pipeline.

could anyone educate me on it?

maybe some segment event is required for seek, where is some reference code?

my test code see attachment.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Mathieu Duponchelle]

As far as I remember, appsrc is not really well suited for seeking, you
may want to consider writing a GstBaseSrc subclass.

On 6/14/19 1:09 PM, Halley Zhao wrote:

Hi expert:

I have special pipeline: appsrc-->h264parse-->h264dec-->videosink

and use a customized clock to control the pipeline.

when I want to do some seek operation, it doesn't work well. I tried to dig into gstbasesink, but  still not a workable solution yet.

after play 3 seconds, I try to seek to 0 (file start)

approach 1: I also reset my clock time to restart from 0.

but, I found, the first buffer (pts=0) and other buffer before 3 second timestamp are dropped. and after no video update for 3 seconds, video playback continues with 3s+ frames.

I added some debug log to gstbasesink, and found that video frames before 3s are treated as 'too late' and are dropped.

I tried to compare it with seek of a normal pipeline (filesrc-->qtdemux-->h264parse-->h264dec-->videosink), and found that the base_time of gstbasesink is updated to the previous played time. 

so I try to gst_element_set_base_time() for the elements of my appsrc pipeline.

then the first video frame isn't dropped, but gst_base_sink_wait_clock() from gst_base_sink_do_sync() is blocked for 3 seconds. then the first video frame is rendered.

approach 2: I tried to create a new clock for the pipeline

after seek, I tried to use a new clock start from zero. it doesn't work well either, because element base_time keeps big number, gst_base_sink_wait_clock() is blocked for 3 seconds as well

to be honest, I don't know what's the correct way to seek a pipeline.

could anyone educate me on it?

maybe some segment event is required for seek, where is some reference code?

my test code see attachment.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Halley Zhao]

thanks,

yes, appsrc is not suitable for seek;

so, I want to know which additional work should be done to support seek. for example, how to manage pipeline clock, element base_time.

however, I got confused after read gst doc and code.

Mathieu Duponchelle <[hidden email]> 于2019年6月14日周五 下午11:55写道：

As far as I remember, appsrc is not really well suited for seeking, you
may want to consider writing a GstBaseSrc subclass.

On 6/14/19 1:09 PM, Halley Zhao wrote:

Hi expert:

I have special pipeline: appsrc-->h264parse-->h264dec-->videosink

and use a customized clock to control the pipeline.

when I want to do some seek operation, it doesn't work well. I tried to dig into gstbasesink, but  still not a workable solution yet.

after play 3 seconds, I try to seek to 0 (file start)

approach 1: I also reset my clock time to restart from 0.

but, I found, the first buffer (pts=0) and other buffer before 3 second timestamp are dropped. and after no video update for 3 seconds, video playback continues with 3s+ frames.

I added some debug log to gstbasesink, and found that video frames before 3s are treated as 'too late' and are dropped.

I tried to compare it with seek of a normal pipeline (filesrc-->qtdemux-->h264parse-->h264dec-->videosink), and found that the base_time of gstbasesink is updated to the previous played time. 

so I try to gst_element_set_base_time() for the elements of my appsrc pipeline.

then the first video frame isn't dropped, but gst_base_sink_wait_clock() from gst_base_sink_do_sync() is blocked for 3 seconds. then the first video frame is rendered.

approach 2: I tried to create a new clock for the pipeline

after seek, I tried to use a new clock start from zero. it doesn't work well either, because element base_time keeps big number, gst_base_sink_wait_clock() is blocked for 3 seconds as well

to be honest, I don't know what's the correct way to seek a pipeline.

could anyone educate me on it?

maybe some segment event is required for seek, where is some reference code?

my test code see attachment.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Halley Zhao]

I tried some work around, and it works:

    /* gst clock is designed to be monotonous increasing,
     * if getMyClockTime() returns a time less than myGstClock last_time,
     * the final gst_audio_clock_get_internal_time() will be clamp to last_time.
     * I can't handle this non-linear timeline well.
     * to be simple for my simple usage, I pretend to increase clock time
     * basing on the the timestamp before seek. and buffer timestamp should be increased basing on
     * timestamp before seek as well.
     * suppose the decent solution is to re-anchor both the element base time (gst_element_set_basetime)
     * and the clock time_offset(gst_audio_clock_reset). however, I didn't succeed it yet.
     */

Halley Zhao <[hidden email]> 于2019年6月17日周一 上午8:32写道：

thanks,

yes, appsrc is not suitable for seek;

so, I want to know which additional work should be done to support seek. for example, how to manage pipeline clock, element base_time.

however, I got confused after read gst doc and code.

Mathieu Duponchelle <[hidden email]> 于2019年6月14日周五 下午11:55写道：

As far as I remember, appsrc is not really well suited for seeking, you
may want to consider writing a GstBaseSrc subclass.

On 6/14/19 1:09 PM, Halley Zhao wrote:

Hi expert:

I have special pipeline: appsrc-->h264parse-->h264dec-->videosink

and use a customized clock to control the pipeline.

when I want to do some seek operation, it doesn't work well. I tried to dig into gstbasesink, but  still not a workable solution yet.

after play 3 seconds, I try to seek to 0 (file start)

approach 1: I also reset my clock time to restart from 0.

but, I found, the first buffer (pts=0) and other buffer before 3 second timestamp are dropped. and after no video update for 3 seconds, video playback continues with 3s+ frames.

I added some debug log to gstbasesink, and found that video frames before 3s are treated as 'too late' and are dropped.

I tried to compare it with seek of a normal pipeline (filesrc-->qtdemux-->h264parse-->h264dec-->videosink), and found that the base_time of gstbasesink is updated to the previous played time. 

so I try to gst_element_set_base_time() for the elements of my appsrc pipeline.

then the first video frame isn't dropped, but gst_base_sink_wait_clock() from gst_base_sink_do_sync() is blocked for 3 seconds. then the first video frame is rendered.

approach 2: I tried to create a new clock for the pipeline

after seek, I tried to use a new clock start from zero. it doesn't work well either, because element base_time keeps big number, gst_base_sink_wait_clock() is blocked for 3 seconds as well

to be honest, I don't know what's the correct way to seek a pipeline.

could anyone educate me on it?

maybe some segment event is required for seek, where is some reference code?

my test code see attachment.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel