seeking in mpegtsdemux

[Janne Grunau]

Hi,

I'm currently writing seeking support for the MPEG TS demuxer rewrite.
Since MPEG TS as streaming format lacks any index the seeking
implementation has to build it's own index.

The basic algorithm I intend to use is an hinted binary search. For
constant bitrate streams estimating the position and a linear search
from the estimated position should work well. Variable bitrate streams
can vary too much (factor 3 is not unusual for HD H.264 broadcast).

So the idea is to do a binary search but instead of choosing the
midpoint for the next iteration a educated guess based on the average
bitrate of the segment is taken. The average bitrate will converge to
the actual bitrate as the segments get smaller. Of course only up to a
certain point, 2 B-frames will be smaller than an I-frame but at this
point a linear search should be used.

For large files this algorithm should need less seeks than a simple
binary at the file system level. I've already started and hope to have
something working at the end of the week.

There are currently two issues I would like to get some feedback on.

*handling of multiple program/streams:
  I can't think of a valid example where multiple programs in a single
multiplex have different durations (assuming they both stay "on-air").
Unless someone can think of valid use case, the scaffold will be per
file and not per program/stream.

*frame accurate seeking:
  seeking to the frame with the desired PTS is unfortunately only part
of the solution. If the application wants to seek to a correctly decoded
(bi-)predicted frame the demuxer needs to return all reference frames
too. This is not too hard for MPEG2 video but MPEG2 is becoming
obsolete. H264 is unfortunately much more complex (not speaking about
other video codecs). The random access indicator in the adaption field
is unfortunately not reliable. I haven't seen it set.

I'll probably have more questions as I implement it.

Janne


_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Edward Hervey]

On Tue, 2011-02-22 at 15:17 +0100, Janne Grunau wrote:

> Hi,
>
> I'm currently writing seeking support for the MPEG TS demuxer rewrite.
> Since MPEG TS as streaming format lacks any index the seeking
> implementation has to build it's own index.
>
> The basic algorithm I intend to use is an hinted binary search. For
> constant bitrate streams estimating the position and a linear search
> from the estimated position should work well. Variable bitrate streams
> can vary too much (factor 3 is not unusual for HD H.264 broadcast).
>
> So the idea is to do a binary search but instead of choosing the
> midpoint for the next iteration a educated guess based on the average
> bitrate of the segment is taken. The average bitrate will converge to
> the actual bitrate as the segments get smaller. Of course only up to a
> certain point, 2 B-frames will be smaller than an I-frame but at this
> point a linear search should be used.
>
> For large files this algorithm should need less seeks than a simple
> binary at the file system level. I've already started and hope to have
> something working at the end of the week.
>
> There are currently two issues I would like to get some feedback on.
>
> *handling of multiple program/streams:
>   I can't think of a valid example where multiple programs in a single
> multiplex have different durations (assuming they both stay "on-air").
> Unless someone can think of valid use case, the scaffold will be per
> file and not per program/stream.

  Programs and streams are meant to use the same clock rate, therefore
you shouldn't have to worry about that.
  In pull-mode, you should only have to worry about the selected program
(or the first one you encounter).

>
> *frame accurate seeking:
>   seeking to the frame with the desired PTS is unfortunately only part
> of the solution. If the application wants to seek to a correctly decoded
> (bi-)predicted frame the demuxer needs to return all reference frames
> too. This is not too hard for MPEG2 video but MPEG2 is becoming
> obsolete. H264 is unfortunately much more complex (not speaking about
> other video codecs). The random access indicator in the adaption field
> is unfortunately not reliable. I haven't seen it set.

   I checked the usage of the random_access_indicator on various files
locally, and it indeed doesn't seem to be used.

   At least you can speed up the scanning process by only looking for
packets with have payload_unit_start_indicator set.

>
> I'll probably have more questions as I implement it.
>
> Janne
>
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Akihiro TSUKADA]

Hi Janne,

I also wrote a seeking support for mpegtsdemux,
although it is non index based, non frame-accurate, bitrate based seek.
On writing this feature, I peeked in the code of gst-ffmpeg,
which also seemed to support the indexed seek.
So you can get some hints from those code and ffmpeg, JFYI.

regards,
Akihiro
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel