RTSP stream and seeking operations

[Aleksey Yulin]

I'm implementing a video player application that should be able to play mpeg2ts from RTSP server. Everything seems to be working fine. The only problem I've faced is that SEEK operation doesn't work when rendering RTSP streams (in the same time it works just fine with local mpeg4 files). gst_element_seek call just fails and rendering is stopped.

My pipeline looks like follows:

source -> decodebin -> videobin (queue -> ffmpegcolorspace -> autovideosink)
                               -> audiobin (queue -> audioconvert -> audioresample -> volume -> autoaudiosink)

source is created using ::gst_element_make_from_uri() function. I beleive it's rtspsrc in my case.

I've checked what RTSP requests are being sent to the server and as I could see there's a valid seek request (i'm seeking to the 5th second from the beginning of the stream):

PLAY
Range: npt=5.0000-

Then the server responds with 200 OK:

status line: 200 OK
Range: npt=4-579

The communication seems to look OK, but gst_element_seek fails then and stream is stopped (I don't see any state change notifications though).

I tried to play the same rtsp stream using Totem player, but the seek operation seems to be unavailabe there too. This looks really strange as RTSP server supports seeking and the rtspsrc seems to send seek command correctly.

Where I might be wrong here? May be seeking RTSP streams require some additional actions to do?

Thanks in advance!

[Wim Taymans]

On Tue, 2009-02-24 at 01:08 -0800, Aleksey Yulin wrote:

> I'm implementing a video player application that should be able to play
> mpeg2ts from RTSP server. Everything seems to be working fine. The only
> problem I've faced is that SEEK operation doesn't work when rendering RTSP
> streams (in the same time it works just fine with local mpeg4 files).
> gst_element_seek call just fails and rendering is stopped.
>
> My pipeline looks like follows:
>
> source -> decodebin -> videobin (queue -> ffmpegcolorspace -> autovideosink)
>                                -> audiobin (queue -> audioconvert ->
> audioresample -> volume -> autoaudiosink)
>
> source is created using ::gst_element_make_from_uri() function. I beleive
> it's rtspsrc in my case.
>
> I've checked what RTSP requests are being sent to the server and as I could
> see there's a valid seek request (i'm seeking to the 5th second from the
> beginning of the stream):
>
> PLAY
> Range: npt=5.0000-
>
> Then the server responds with 200 OK:
>
> status line: 200 OK
> Range: npt=4-579
>
> The communication seems to look OK, but gst_element_seek fails then and
> stream is stopped (I don't see any state change notifications though).
>
> I tried to play the same rtsp stream using Totem player, but the seek
> operation seems to be unavailabe there too. This looks really strange as
> RTSP server supports seeking and the rtspsrc seems to send seek command
> correctly.
>
> Where I might be wrong here? May be seeking RTSP streams require some
> additional actions to do?

It's probably because seeking in rtspsrc is not fully working and needs
some more code here and there.

Wim

>
> Thanks in advance!
>


------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Aleksey Yulin]

Wim Taymans-2 wrote

It's probably because seeking in rtspsrc is not fully working and needs
some more code here and there.

Wim

I really need this feature so may be you could tell me where I could read about how to fully implement it? Do I need to modify rtspsrc plugin sources (or event implement some unimplemented stuff) ? Or I could just call some additional plugin methods?

[Aleksey Yulin]

Wim Taymans-2 wrote

It's probably because seeking in rtspsrc is not fully working and needs
some more code here and there.

As I understood the problem begins when an 'xvimagesink' appears in the pipeline (SEEK works fine when 'filesink' is used). So it looks like some clock synchronization problem occurs when videosink tries to render the video with it's actual FPS.

May be there some quick workaroung for this issue exists?

Thanks,
Aleksey

[Aleksey Yulin]

As far as we've investigated passing start_segment event created in the rtspsrc from the streaming thread context is nearly not enough: all newsegment events pushed from the rtspsrc seems to be lost by the basertpdepayload (gstbasertpdepayload.c, line: 417).

What do we have:

1. We're streaming MPEG2 TS file
2. First TS frame has a timestamp at about 7:26:30.100055555
3. flutsdemux is used as a TS demuxer

What do we see:

1. Before seek we see that mpeg2 decoder element receives frames timestamps starting from 7:26:30.100055555
2. The initial newsegment event the mpeg decoder receives also has a segment starting from the 7:26:30.100055555
3. after seek performed by the rtspsrc a newsegment event is being sent by the RTP depayloader. This event contains 0 as a segment start, and a valid new seek position (10 seconds) in 'position' field.
4. decoder receives that segment but continues to get frames with timestamps starting from the 7:26:40 (as we've made a seek 10 seconds forward)
5. as soon as decoder thinks that newsegment being played is starting from 0, it starts to skip all the frames as their timestamps are far beyond the segment boundaries

So as we could see there could be several causes for this issue:

1. The mpeg2 ts demuxer (flutsdemux) should update the newsegment received from the depayloader, shifting all its timestamps to the first TS frame timestamp (7:26:30)
2. The mpeg2 ts demuxer should initially send downstream a newsegment starting not from the 7:26:30, but from 0 shifting the TS frames timestamp by this value each time.
3. The demuxer should update the base time of the downstream element (decoder) after seek operation so the decoder will adjust frame timestamp using this value

Is there any documents describing these elements valid behavior?

Any help will be appreciated!

[Aleksey Yulin]

Further investigations show that there's the same SEEK issue while trying to stream mpeg TS over using the HTTP streaming.

So it seems that the problem is actually in the Mpeg Transport Stream demuxer (we use 'mpegtsdemux').

As we could see the demuxer creates an initial NEWSEGMENT starting from the first TS timestamp for the downstream elements. That works fine, but when SEEK is performed the new NEWSEGMENT event is generated before the demuxer - in the RTP depayload element. This segment has a 0 as start and actual seek position as a 'position'.
But the demuxer instead of patching this NEWSEGMENT event (shifting to the next known TS packet timestamp) seems to just pass it downstream. This would be OK, but the demuxer continues to set buffers timestamps to the actual TS packets timestamp so for downstream elements it looks like out-of-valid-range buffers are coming so they whether start to skip these buffers (like mpeg2decoder) or just returns UNEXPECTED (as mpegaudio decoder) causing the pipeline to pause forever.

We've tried to send the valid NEWSEGMENT event after SEEK (from the streaming thread context) and got the following: after seek the picture has paused, but after some time (comparable to the new position value) the playback started from the new position. This happens only with the first SEEK - any subsequential seeks causes the pipeline to pause forever as before.

[Edward Hervey]

On Mon, 2009-03-16 at 00:09 -0700, Aleksey Yulin wrote:
> Further investigations show that there's the same SEEK issue while trying to
> stream mpeg TS over using the HTTP streaming.
>
> So it seems that the problem is actually in the Mpeg Transport Stream
> demuxer (we use 'mpegtsdemux').

  That would be "Bug 550634 – [mpeg ts demuxer] Doesn't support seeking
and duration reporting":
  http://bugzilla.gnome.org/show_bug.cgi?id=550634

>
> As we could see the demuxer creates an initial NEWSEGMENT starting from the
> first TS timestamp for the downstream elements. That works fine, but when
> SEEK is performed the new NEWSEGMENT event is generated before the demuxer -
> in the RTP depayload element. This segment has a 0 as start and actual seek
> position as a 'position'.

  Wait, you're using a http source or a rtp source ? Why do you need a
rtp depayloader ?

> But the demuxer instead of patching this NEWSEGMENT event (shifting to the
> next known TS packet timestamp) seems to just pass it downstream. This would
> be OK, but the demuxer continues to set buffers timestamps to the actual TS
> packets timestamp so for downstream elements it looks like
> out-of-valid-range buffers are coming so they whether start to skip these
> buffers (like mpeg2decoder) or just returns UNEXPECTED (as mpegaudio
> decoder) causing the pipeline to pause forever.
>
> We've tried to send the valid NEWSEGMENT event after SEEK (from the
> streaming thread context) and got the following: after seek the picture has
> paused, but after some time (comparable to the new position value) the
> playback started from the new position. This happens only with the first
> SEEK - any subsequential seeks causes the pipeline to pause forever as
> before.

  Doing proper seeking in push-mode (you can do 'immediate'
random-access with an http source) is going to be a lot trickier than
implementing it for pull-mode (from a local file) though.

  If you're interested in getting this feature fixed, I'd recommend
carrying on the discussions in the bug report.

   Edward


------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel