Multithreaded fast-forward possible?

[Peter Rennert]

Dear all,

I want to increase the playback of a video quite drastically. At the
moment I am playing a test video sequence with

gst-launch-0.10 filesrc location=/path/to/my.mp4 ! qtdemux ! h264parse !
ffdec_h264 name=dec ! ffmpegcolorspace ! deinterlace ! xvimagesink

and at some point the video speed cannot get increased further, because
it uses only one CPU core to decode the video. Gstreamer starts
complaining of getting behind the timestamps of the stream and crashes.

I could think of two solutions:
     - Is there a "native" way of making use of more than just a single
core to decode the video frames? I tried to set "max-threads" of
ffdec_h264 to 4, but it still only uses a single core.

or,
     - Is there a way of skipping frames? As I need only a "effective"
framerate on the screen of about 25fps I could just decode the frames I
need and skip the other frames. Then I would not need more CPU power
than for realtime playback.

or.
     - Is there a way to split the stream after the qtdemux and use
several decoders in a kind of decoder pool (distribute the frames
between them) and unify the stream after?

My videos are in H264 format. I am not sure if the non-keyframes are
encoded with respect to the previous frame or with respect to the last
key-frame. In that case I could try to filter the key-frames and send
them to every decoder in the pool and decode the any frame with that as
reference until the next key-frame. With some management I could try to
merge the output of the pool decoder with a input-selector. But that
sounds bulky :\ So if one knows how to get to one of the first two
solutions, I would be very happy...


Cheers,

Peter

PS I am happy to provide an example video. Maybe something is wrong with
the encoding there that prevents ffdec_h264 to use multiple cores
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Juraj Holtak]

Hi,

Maybe worth a try:

Put an "queue" element before and after ffdec_h264 and use max-threads=<your_cpu_count>. I imagine it worked like this for me but maybe I just had luck...

Juraj

On Dec 13, 2012 2:03 AM, "Peter Rennert" <[hidden email]> wrote:

Dear all,

I want to increase the playback of a video quite drastically. At the moment I am playing a test video sequence with

gst-launch-0.10 filesrc location=/path/to/my.mp4 ! qtdemux ! h264parse ! ffdec_h264 name=dec ! ffmpegcolorspace ! deinterlace ! xvimagesink

and at some point the video speed cannot get increased further, because it uses only one CPU core to decode the video. Gstreamer starts complaining of getting behind the timestamps of the stream and crashes.

I could think of two solutions:
    - Is there a "native" way of making use of more than just a single core to decode the video frames? I tried to set "max-threads" of ffdec_h264 to 4, but it still only uses a single core.

or,
    - Is there a way of skipping frames? As I need only a "effective" framerate on the screen of about 25fps I could just decode the frames I need and skip the other frames. Then I would not need more CPU power than for realtime playback.

or.
    - Is there a way to split the stream after the qtdemux and use several decoders in a kind of decoder pool (distribute the frames between them) and unify the stream after?

My videos are in H264 format. I am not sure if the non-keyframes are encoded with respect to the previous frame or with respect to the last key-frame. In that case I could try to filter the key-frames and send them to every decoder in the pool and decode the any frame with that as reference until the next key-frame. With some management I could try to merge the output of the pool decoder with a input-selector. But that sounds bulky :\ So if one knows how to get to one of the first two solutions, I would be very happy...


Cheers,

Peter

PS I am happy to provide an example video. Maybe something is wrong with the encoding there that prevents ffdec_h264 to use multiple cores
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Juraj Holtak]

And maybe the xvimagesink should have sync=false set too... maybe...

On Dec 13, 2012 12:15 PM, "Juraj Holtak" <[hidden email]> wrote:

Hi,

Maybe worth a try:

Put an "queue" element before and after ffdec_h264 and use max-threads=<your_cpu_count>. I imagine it worked like this for me but maybe I just had luck...

Juraj

On Dec 13, 2012 2:03 AM, "Peter Rennert" <[hidden email]> wrote:

Dear all,

I want to increase the playback of a video quite drastically. At the moment I am playing a test video sequence with

gst-launch-0.10 filesrc location=/path/to/my.mp4 ! qtdemux ! h264parse ! ffdec_h264 name=dec ! ffmpegcolorspace ! deinterlace ! xvimagesink

and at some point the video speed cannot get increased further, because it uses only one CPU core to decode the video. Gstreamer starts complaining of getting behind the timestamps of the stream and crashes.

I could think of two solutions:
    - Is there a "native" way of making use of more than just a single core to decode the video frames? I tried to set "max-threads" of ffdec_h264 to 4, but it still only uses a single core.

or,
    - Is there a way of skipping frames? As I need only a "effective" framerate on the screen of about 25fps I could just decode the frames I need and skip the other frames. Then I would not need more CPU power than for realtime playback.

or.
    - Is there a way to split the stream after the qtdemux and use several decoders in a kind of decoder pool (distribute the frames between them) and unify the stream after?

My videos are in H264 format. I am not sure if the non-keyframes are encoded with respect to the previous frame or with respect to the last key-frame. In that case I could try to filter the key-frames and send them to every decoder in the pool and decode the any frame with that as reference until the next key-frame. With some management I could try to merge the output of the pool decoder with a input-selector. But that sounds bulky :\ So if one knows how to get to one of the first two solutions, I would be very happy...


Cheers,

Peter

PS I am happy to provide an example video. Maybe something is wrong with the encoding there that prevents ffdec_h264 to use multiple cores
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Peter Rennert]

Thanks Juraj these were good tips.

I needed to add a queue before ffdec_h264, but there must not be one after ffdec_264. Furthermore, I needed to set sync=false at the image sink. However, now it seems as the filesink (or qtdemux/h264parse?) is not feeding the data quickly enough. (I am just at x2 speed, my disk I/O maximum should far ahead). My ffdec_264 sends now the following warning message when I try to increase the speed:

0:00:06.193160955  3058      0x27f08f0 WARN            videodecoder gstvideodecoder.c:2847:gst_video_decoder_alloc_output_frame:<dec> failed to get buffer wrong-state
0:00:06.193239481  3058      0x27f08f0 WARN                  ffmpeg gstffmpegviddec.c:1218:gst_ffmpegviddec_frame:<dec> ffdec_h264: decoding error (len: -1, have_data: 0)

I got rid of those warnings when I add a queue after the filesink.

Still there are problems with my pipeline:

First problem is that the standard speed seems to be twice as normal.

Second, although I see that the computations are now spread over several cores and I do not get anymore warnings from ffdec_h264 after keyPress (and respective the seek event), the playback speed does not change at all anymore.

I put my test code below. Its in python (my first attempt of python and gstreamer), but I think it should be readable for people who use C or C++, too. I suspect I might use wrong flags in gst.event_new_seek().

P.S.: just to make it complete for documentation purposes, playing a bit around with it and using

self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME,
            gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE,
            gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE))

instead of

self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME,
            gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0,
            gst.SEEK_TYPE_NONE, 0))

gives me a python error:

Traceback (most recent call last):
  File "pyGstViewer.py", line 61, in keyPress
    self.increasePlaybackSpeed()
  File "pyGstViewer.py", line 70, in increasePlaybackSpeed
    gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE))


===== CODE =====

import sys, os
import pygst
pygst.require("0.10")
import gst
import pygtk, gobject
import gtk

class GTK_Main:
    def __init__(self):
        self.window = gtk.Window(gtk.WINDOW_TOPLEVEL)
        self.window.connect("destroy", gtk.main_quit, "WM destroy")
        vbox = gtk.VBox()
        self.window.add(vbox)
        vbox.pack_start(gtk.Label("Please type text"))
        entry = gtk.Entry()
        vbox.pack_start(entry)
        self.window.connect("key-press-event", self.keyPress)

        self.window.show_all()

        self.pipeline =  gst.parse_launch("filesrc " +
            "location=/home/peter/vid/20121207/00/2012-12-07.00-00-00.mp4 ! " +
            "queue ! qtdemux ! h264parse ! queue ! ffdec_h264 name=dec ! " +
            "ffmpegcolorspace ! deinterlace ! xvimagesink sync=false")

        self.dec = self.pipeline.get_by_name("dec")

        self.dec.set_property("max-threads", 8);
        
        self.pbRate = 1

        self.pipeline.set_state(gst.STATE_PLAYING)

        bus = self.pipeline.get_bus()
        bus.add_signal_watch()
        bus.connect("message", self.onMessage)


    def keyPress(self, widget, event):
        print "keypress event!!"
        key = gtk.gdk.keyval_name(event.keyval)
        if key == "d":
            self.increasePlaybackSpeed()
        else:
            self.decreasePlaybackSpeed()

    def increasePlaybackSpeed(self):
        print str(self.pbRate) + " --> " + str(self.pbRate + 4)
        self.pbRate += 4
        self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME,
            gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0,
            gst.SEEK_TYPE_NONE, 0))
        print "increased playback speed!!!"
       
    def decreasePlaybackSpeed(self):  
        self.pbRate -= 2
        self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME,
            gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0,
            gst.SEEK_TYPE_NONE, 0))
        print "decreased playback speed!!!"

    def onMessage(self, bus, message):
        t = message.type
        if t == gst.MESSAGE_EOS:
            self.pipeline.set_state(gst.STATE_NULL)
        elif t == gst.MESSAGE_ERROR:
            self.pipeline.set_state(gst.STATE_NULL)
            err, debug = message.parse_error()
            print "Error: %s" % err, debug

GTK_Main()
gtk.gdk.threads_init()
gtk.main()



On 12/13/2012 11:18 AM, Juraj Holtak wrote:

And maybe the xvimagesink should have sync=false set too... maybe...

On Dec 13, 2012 12:15 PM, "Juraj Holtak" <[hidden email]> wrote:

Hi,

Maybe worth a try:

Put an "queue" element before and after ffdec_h264 and use max-threads=<your_cpu_count>. I imagine it worked like this for me but maybe I just had luck...

Juraj

On Dec 13, 2012 2:03 AM, "Peter Rennert" <[hidden email]> wrote:

Dear all,

I want to increase the playback of a video quite drastically. At the moment I am playing a test video sequence with

gst-launch-0.10 filesrc location=/path/to/my.mp4 ! qtdemux ! h264parse ! ffdec_h264 name=dec ! ffmpegcolorspace ! deinterlace ! xvimagesink

and at some point the video speed cannot get increased further, because it uses only one CPU core to decode the video. Gstreamer starts complaining of getting behind the timestamps of the stream and crashes.

I could think of two solutions:
    - Is there a "native" way of making use of more than just a single core to decode the video frames? I tried to set "max-threads" of ffdec_h264 to 4, but it still only uses a single core.

or,
    - Is there a way of skipping frames? As I need only a "effective" framerate on the screen of about 25fps I could just decode the frames I need and skip the other frames. Then I would not need more CPU power than for realtime playback.

or.
    - Is there a way to split the stream after the qtdemux and use several decoders in a kind of decoder pool (distribute the frames between them) and unify the stream after?

My videos are in H264 format. I am not sure if the non-keyframes are encoded with respect to the previous frame or with respect to the last key-frame. In that case I could try to filter the key-frames and send them to every decoder in the pool and decode the any frame with that as reference until the next key-frame. With some management I could try to merge the output of the pool decoder with a input-selector. But that sounds bulky :\ So if one knows how to get to one of the first two solutions, I would be very happy...


Cheers,

Peter

PS I am happy to provide an example video. Maybe something is wrong with the encoding there that prevents ffdec_h264 to use multiple cores
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Peter Rennert]

Although in my code it does not change anything of the behaviour, I just found the theoretical solution to my idea of skipping frames for faster playback:

"When doing fast forward (rate > 1.0) or fast reverse (rate < -1.0) trickmode playback, the GST_SEEK_FLAG_SKIP flag can be used to instruct decoders and demuxers to adjust the playback rate by skipping frames. This can improve performance and decrease CPU usage because not all frames need to be decoded." [1]

[1] http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-GstEvent.html#GstSeekFlags


On 12/13/2012 01:08 PM, Peter Rennert wrote:

Thanks Juraj these were good tips.

I needed to add a queue before ffdec_h264, but there must not be one after ffdec_264. Furthermore, I needed to set sync=false at the image sink. However, now it seems as the filesink (or qtdemux/h264parse?) is not feeding the data quickly enough. (I am just at x2 speed, my disk I/O maximum should far ahead). My ffdec_264 sends now the following warning message when I try to increase the speed:

0:00:06.193160955  3058      0x27f08f0 WARN            videodecoder gstvideodecoder.c:2847:gst_video_decoder_alloc_output_frame:<dec> failed to get buffer wrong-state
0:00:06.193239481  3058      0x27f08f0 WARN                  ffmpeg gstffmpegviddec.c:1218:gst_ffmpegviddec_frame:<dec> ffdec_h264: decoding error (len: -1, have_data: 0)

I got rid of those warnings when I add a queue after the filesink.

Still there are problems with my pipeline:

First problem is that the standard speed seems to be twice as normal.

Second, although I see that the computations are now spread over several cores and I do not get anymore warnings from ffdec_h264 after keyPress (and respective the seek event), the playback speed does not change at all anymore.

I put my test code below. Its in python (my first attempt of python and gstreamer), but I think it should be readable for people who use C or C++, too. I suspect I might use wrong flags in gst.event_new_seek().

P.S.: just to make it complete for documentation purposes, playing a bit around with it and using

self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME,
            gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE,
            gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE))

instead of

self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME,
            gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0,
            gst.SEEK_TYPE_NONE, 0))

gives me a python error:

Traceback (most recent call last):
  File "pyGstViewer.py", line 61, in keyPress
    self.increasePlaybackSpeed()
  File "pyGstViewer.py", line 70, in increasePlaybackSpeed
    gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE))


===== CODE =====

import sys, os
import pygst
pygst.require("0.10")
import gst
import pygtk, gobject
import gtk

class GTK_Main:
    def __init__(self):
        self.window = gtk.Window(gtk.WINDOW_TOPLEVEL)
        self.window.connect("destroy", gtk.main_quit, "WM destroy")
        vbox = gtk.VBox()
        self.window.add(vbox)
        vbox.pack_start(gtk.Label("Please type text"))
        entry = gtk.Entry()
        vbox.pack_start(entry)
        self.window.connect("key-press-event", self.keyPress)

        self.window.show_all()

        self.pipeline =  gst.parse_launch("filesrc " +
            "location=/home/peter/vid/20121207/00/2012-12-07.00-00-00.mp4 ! " +
            "queue ! qtdemux ! h264parse ! queue ! ffdec_h264 name=dec ! " +
            "ffmpegcolorspace ! deinterlace ! xvimagesink sync=false")

        self.dec = self.pipeline.get_by_name("dec")

        self.dec.set_property("max-threads", 8);
        
        self.pbRate = 1

        self.pipeline.set_state(gst.STATE_PLAYING)

        bus = self.pipeline.get_bus()
        bus.add_signal_watch()
        bus.connect("message", self.onMessage)


    def keyPress(self, widget, event):
        print "keypress event!!"
        key = gtk.gdk.keyval_name(event.keyval)
        if key == "d":
            self.increasePlaybackSpeed()
        else:
            self.decreasePlaybackSpeed()

    def increasePlaybackSpeed(self):
        print str(self.pbRate) + " --> " + str(self.pbRate + 4)
        self.pbRate += 4
        self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME,
            gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0,
            gst.SEEK_TYPE_NONE, 0))
        print "increased playback speed!!!"
       
    def decreasePlaybackSpeed(self):  
        self.pbRate -= 2
        self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME,
            gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0,
            gst.SEEK_TYPE_NONE, 0))
        print "decreased playback speed!!!"

    def onMessage(self, bus, message):
        t = message.type
        if t == gst.MESSAGE_EOS:
            self.pipeline.set_state(gst.STATE_NULL)
        elif t == gst.MESSAGE_ERROR:
            self.pipeline.set_state(gst.STATE_NULL)
            err, debug = message.parse_error()
            print "Error: %s" % err, debug

GTK_Main()
gtk.gdk.threads_init()
gtk.main()



On 12/13/2012 11:18 AM, Juraj Holtak wrote:

And maybe the xvimagesink should have sync=false set too... maybe...

On Dec 13, 2012 12:15 PM, "Juraj Holtak" <[hidden email]> wrote:

Hi,

Maybe worth a try:

Put an "queue" element before and after ffdec_h264 and use max-threads=<your_cpu_count>. I imagine it worked like this for me but maybe I just had luck...

Juraj

On Dec 13, 2012 2:03 AM, "Peter Rennert" <[hidden email]> wrote:

Dear all,

I want to increase the playback of a video quite drastically. At the moment I am playing a test video sequence with

gst-launch-0.10 filesrc location=/path/to/my.mp4 ! qtdemux ! h264parse ! ffdec_h264 name=dec ! ffmpegcolorspace ! deinterlace ! xvimagesink

and at some point the video speed cannot get increased further, because it uses only one CPU core to decode the video. Gstreamer starts complaining of getting behind the timestamps of the stream and crashes.

I could think of two solutions:
    - Is there a "native" way of making use of more than just a single core to decode the video frames? I tried to set "max-threads" of ffdec_h264 to 4, but it still only uses a single core.

or,
    - Is there a way of skipping frames? As I need only a "effective" framerate on the screen of about 25fps I could just decode the frames I need and skip the other frames. Then I would not need more CPU power than for realtime playback.

or.
    - Is there a way to split the stream after the qtdemux and use several decoders in a kind of decoder pool (distribute the frames between them) and unify the stream after?

My videos are in H264 format. I am not sure if the non-keyframes are encoded with respect to the previous frame or with respect to the last key-frame. In that case I could try to filter the key-frames and send them to every decoder in the pool and decode the any frame with that as reference until the next key-frame. With some management I could try to merge the output of the pool decoder with a input-selector. But that sounds bulky :\ So if one knows how to get to one of the first two solutions, I would be very happy...


Cheers,

Peter

PS I am happy to provide an example video. Maybe something is wrong with the encoding there that prevents ffdec_h264 to use multiple cores
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Peter Rennert]

I rewrote my script in C++ to make sure that I am not messing up things with python, but it did not change anything. But I solved the problem.

If I compile the playback speed tutorial [1] that uses a playbin2 on my file, I can do some kind of fast forward. If I test it with a file [2] with low resolution, I can do fast-forward till about 90x (using GST_SEEK_FLAG_SKIP does not change anything in my case). As I increase the playback speed, the CPU use changes only slightly. But then suddenly it seems that the threading overhead in ffdec_h264 becomes too big and and a single thread goes to full load and blocks the playback. Fast forward at about 50x starts to become instable and gets blocked some time. I have the impression that flushing the pipeline helps in that case. I will try tomorrow to install a handler that flushes the pipeline in such cases.

Using my HD file, in the same setup allows me to speed the playback up only until 4x speed. Even though the cores are under relative low use (below 50%) one of the threads starts to block the playback. Anyway it works somehow.

One remark to setting xvimagesink to sync=false. In some cases it makes the pipeline play at a high speed (seems to depend on the speed that ffdec_h264 decodes), but the speed is not controllable. In a reduced pipeline (see below) everything works. Therefore for me the multithreading problem is more or less solved by using this pipeline:

filesrc location=/path/to/my.mp4 ! qtdemux ! h264parse ! ffdec_h264 name=dec ! autovideosink sync=false

It might have been the deinterlace or ffmpegcolorspace that caused ffdec_h264 to decode only with a single thread...

[1] http://docs.gstreamer.com/display/GstSDK/Basic+tutorial+13%3A+Playback+speed
[2] http://archive.org/details/ThePhantomoftheOpera


On 12/13/2012 01:35 PM, Peter Rennert wrote:

Although in my code it does not change anything of the behaviour, I just found the theoretical solution to my idea of skipping frames for faster playback:

"When doing fast forward (rate > 1.0) or fast reverse (rate < -1.0) trickmode playback, the GST_SEEK_FLAG_SKIP flag can be used to instruct decoders and demuxers to adjust the playback rate by skipping frames. This can improve performance and decrease CPU usage because not all frames need to be decoded." [1]

[1] http://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer/html/gstreamer-GstEvent.html#GstSeekFlags


On 12/13/2012 01:08 PM, Peter Rennert wrote:

Thanks Juraj these were good tips.

I needed to add a queue before ffdec_h264, but there must not be one after ffdec_264. Furthermore, I needed to set sync=false at the image sink. However, now it seems as the filesink (or qtdemux/h264parse?) is not feeding the data quickly enough. (I am just at x2 speed, my disk I/O maximum should far ahead). My ffdec_264 sends now the following warning message when I try to increase the speed:

0:00:06.193160955  3058      0x27f08f0 WARN            videodecoder gstvideodecoder.c:2847:gst_video_decoder_alloc_output_frame:<dec> failed to get buffer wrong-state
0:00:06.193239481  3058      0x27f08f0 WARN                  ffmpeg gstffmpegviddec.c:1218:gst_ffmpegviddec_frame:<dec> ffdec_h264: decoding error (len: -1, have_data: 0)

I got rid of those warnings when I add a queue after the filesink.

Still there are problems with my pipeline:

First problem is that the standard speed seems to be twice as normal.

Second, although I see that the computations are now spread over several cores and I do not get anymore warnings from ffdec_h264 after keyPress (and respective the seek event), the playback speed does not change at all anymore.

I put my test code below. Its in python (my first attempt of python and gstreamer), but I think it should be readable for people who use C or C++, too. I suspect I might use wrong flags in gst.event_new_seek().

P.S.: just to make it complete for documentation purposes, playing a bit around with it and using

self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME,
            gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE,
            gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE))

instead of

self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME,
            gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0,
            gst.SEEK_TYPE_NONE, 0))

gives me a python error:

Traceback (most recent call last):
  File "pyGstViewer.py", line 61, in keyPress
    self.increasePlaybackSpeed()
  File "pyGstViewer.py", line 70, in increasePlaybackSpeed
    gst.SEEK_TYPE_NONE, gst.CLOCK_TIME_NONE))


===== CODE =====

import sys, os
import pygst
pygst.require("0.10")
import gst
import pygtk, gobject
import gtk

class GTK_Main:
    def __init__(self):
        self.window = gtk.Window(gtk.WINDOW_TOPLEVEL)
        self.window.connect("destroy", gtk.main_quit, "WM destroy")
        vbox = gtk.VBox()
        self.window.add(vbox)
        vbox.pack_start(gtk.Label("Please type text"))
        entry = gtk.Entry()
        vbox.pack_start(entry)
        self.window.connect("key-press-event", self.keyPress)

        self.window.show_all()

        self.pipeline =  gst.parse_launch("filesrc " +
            "location=/home/peter/vid/20121207/00/2012-12-07.00-00-00.mp4 ! " +
            "queue ! qtdemux ! h264parse ! queue ! ffdec_h264 name=dec ! " +
            "ffmpegcolorspace ! deinterlace ! xvimagesink sync=false")

        self.dec = self.pipeline.get_by_name("dec")

        self.dec.set_property("max-threads", 8);
        
        self.pbRate = 1

        self.pipeline.set_state(gst.STATE_PLAYING)

        bus = self.pipeline.get_bus()
        bus.add_signal_watch()
        bus.connect("message", self.onMessage)


    def keyPress(self, widget, event):
        print "keypress event!!"
        key = gtk.gdk.keyval_name(event.keyval)
        if key == "d":
            self.increasePlaybackSpeed()
        else:
            self.decreasePlaybackSpeed()

    def increasePlaybackSpeed(self):
        print str(self.pbRate) + " --> " + str(self.pbRate + 4)
        self.pbRate += 4
        self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME,
            gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0,
            gst.SEEK_TYPE_NONE, 0))
        print "increased playback speed!!!"
       
    def decreasePlaybackSpeed(self):  
        self.pbRate -= 2
        self.pipeline.send_event(gst.event_new_seek(self.pbRate, gst.FORMAT_TIME,
            gst.SEEK_FLAG_FLUSH, gst.SEEK_TYPE_NONE, 0,
            gst.SEEK_TYPE_NONE, 0))
        print "decreased playback speed!!!"

    def onMessage(self, bus, message):
        t = message.type
        if t == gst.MESSAGE_EOS:
            self.pipeline.set_state(gst.STATE_NULL)
        elif t == gst.MESSAGE_ERROR:
            self.pipeline.set_state(gst.STATE_NULL)
            err, debug = message.parse_error()
            print "Error: %s" % err, debug

GTK_Main()
gtk.gdk.threads_init()
gtk.main()



On 12/13/2012 11:18 AM, Juraj Holtak wrote:

And maybe the xvimagesink should have sync=false set too... maybe...

On Dec 13, 2012 12:15 PM, "Juraj Holtak" <[hidden email]> wrote:

Hi,

Maybe worth a try:

Put an "queue" element before and after ffdec_h264 and use max-threads=<your_cpu_count>. I imagine it worked like this for me but maybe I just had luck...

Juraj

On Dec 13, 2012 2:03 AM, "Peter Rennert" <[hidden email]> wrote:

Dear all,

I want to increase the playback of a video quite drastically. At the moment I am playing a test video sequence with

gst-launch-0.10 filesrc location=/path/to/my.mp4 ! qtdemux ! h264parse ! ffdec_h264 name=dec ! ffmpegcolorspace ! deinterlace ! xvimagesink

and at some point the video speed cannot get increased further, because it uses only one CPU core to decode the video. Gstreamer starts complaining of getting behind the timestamps of the stream and crashes.

I could think of two solutions:
    - Is there a "native" way of making use of more than just a single core to decode the video frames? I tried to set "max-threads" of ffdec_h264 to 4, but it still only uses a single core.

or,
    - Is there a way of skipping frames? As I need only a "effective" framerate on the screen of about 25fps I could just decode the frames I need and skip the other frames. Then I would not need more CPU power than for realtime playback.

or.
    - Is there a way to split the stream after the qtdemux and use several decoders in a kind of decoder pool (distribute the frames between them) and unify the stream after?

My videos are in H264 format. I am not sure if the non-keyframes are encoded with respect to the previous frame or with respect to the last key-frame. In that case I could try to filter the key-frames and send them to every decoder in the pool and decode the any frame with that as reference until the next key-frame. With some management I could try to merge the output of the pool decoder with a input-selector. But that sounds bulky :\ So if one knows how to get to one of the first two solutions, I would be very happy...


Cheers,

Peter

PS I am happy to provide an example video. Maybe something is wrong with the encoding there that prevents ffdec_h264 to use multiple cores
_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel