decodebin2 manipulation of multiqueue

[Conrad Cooke]

I recently debugged an issue where playback was failing for certain mpeg-4 container files.  Turned out to be hitting a limitation placed on multiqueue.
This is a result of how the file was encoded.  While it is a legal containerised media file, there is a large difference between audio and video timestamps.
It is the purpose of multiqueue to realign the audio and video bins, I believe.

multiqueue has various parameters which determine how much difference there can be between video and audio.
>From gst-inspect:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: null Current: "multiqueue0"
  extra-size-bytes    : Amount of data the queues can grow if one of them is empty (bytes, 0=disable)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 10485760 Current: 10485760
  extra-size-buffers  : Amount of buffers the queues can grow if one of them is empty (0=disable)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 5 Current: 5
  extra-size-time     : Amount of time the queues can grow if one of them is empty (in ns, 0=disable)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 3000000000 Current: 3000000000
  max-size-bytes      : Max. amount of data in the queue (bytes, 0=disable)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 10485760 Current: 10485760
  max-size-buffers    : Max. number of buffers in the queue (0=disable)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 5 Current: 5
  max-size-time       : Max. amount of data in the queue (in ns, 0=disable)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 2000000000 Current: 2000000000

Seems the last three properties are not implemented but were intended to support 'poorly encoded media'

The three supported properties are set to their defaults on construction of the multiqueue.  They are also set twice by decodebin, once when a group is created (gst_decode_group_new) and once when its pads are exposed (gst_decode_group_expose)
The second case set the values to

        "max-size-bytes", 2 * 1024 * 1024,
        "max-size-time", 2 * GST_SECOND,
        "max-size-buffers", 5, NULL;

It seems for some media, there is a greater than 2 second difference betwee audio and video, so I chnged max-size-time to be 50 seconds.

My questions are
Why is decodebin2 directly modifying this component at all ?
Why is a 2 second maximum imposed - is the only real limitation is the maximum memory that can be used ?

Here is my change:

component: gst-plugins-base-0.10.21

file: gstdecodebin2 line 1983

  if (group->multiqueue) {
    /* update runtime limits. At runtime, we try to keep the amount of buffers
     * in the queues as low as possible (but at least 5 buffers). */
    g_object_set (G_OBJECT (group->multiqueue),
        "max-size-bytes", 2 * 1024 * 1024,
        "max-size-time", 50 * GST_SECOND, "max-size-buffers", 5, NULL);
    /* we can now disconnect any overrun signal, which is used to expose the
     * group. */
    if (group->overrunsig) {
      GST_LOG ("Disconnecting overrun");
      g_signal_handler_disconnect (group->multiqueue, group->overrunsig);
      group->overrunsig = 0;
    }
  }


Best Wishes,

Conrad Cooke
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Wim Taymans]

On Tue, 2009-10-20 at 10:10 -0700, Conrad Cooke wrote:

Thanks for the patch but this issue has been fixed already. Newest
decodebin2 does not set an upper limit on time anymore, just bytes.

Wim

> I recently debugged an issue where playback was failing for certain mpeg-4 container files.  Turned out to be hitting a limitation placed on multiqueue.
> This is a result of how the file was encoded.  While it is a legal containerised media file, there is a large difference between audio and video timestamps.
> It is the purpose of multiqueue to realign the audio and video bins, I believe.
>
> multiqueue has various parameters which determine how much difference there can be between video and audio.
> >From gst-inspect:
>   name                : The name of the object
>                         flags: readable, writable
>                         String. Default: null Current: "multiqueue0"
>   extra-size-bytes    : Amount of data the queues can grow if one of them is empty (bytes, 0=disable)
>                         flags: readable, writable
>                         Unsigned Integer. Range: 0 - 4294967295 Default: 10485760 Current: 10485760
>   extra-size-buffers  : Amount of buffers the queues can grow if one of them is empty (0=disable)
>                         flags: readable, writable
>                         Unsigned Integer. Range: 0 - 4294967295 Default: 5 Current: 5
>   extra-size-time     : Amount of time the queues can grow if one of them is empty (in ns, 0=disable)
>                         flags: readable, writable
>                         Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 3000000000 Current: 3000000000
>   max-size-bytes      : Max. amount of data in the queue (bytes, 0=disable)
>                         flags: readable, writable
>                         Unsigned Integer. Range: 0 - 4294967295 Default: 10485760 Current: 10485760
>   max-size-buffers    : Max. number of buffers in the queue (0=disable)
>                         flags: readable, writable
>                         Unsigned Integer. Range: 0 - 4294967295 Default: 5 Current: 5
>   max-size-time       : Max. amount of data in the queue (in ns, 0=disable)
>                         flags: readable, writable
>                         Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 2000000000 Current: 2000000000
>
> Seems the last three properties are not implemented but were intended to support 'poorly encoded media'
>
> The three supported properties are set to their defaults on construction of the multiqueue.  They are also set twice by decodebin, once when a group is created (gst_decode_group_new) and once when its pads are exposed (gst_decode_group_expose)
> The second case set the values to
>
>         "max-size-bytes", 2 * 1024 * 1024,
>         "max-size-time", 2 * GST_SECOND,
>         "max-size-buffers", 5, NULL;
>
> It seems for some media, there is a greater than 2 second difference betwee audio and video, so I chnged max-size-time to be 50 seconds.
>
> My questions are
> Why is decodebin2 directly modifying this component at all ?
> Why is a 2 second maximum imposed - is the only real limitation is the maximum memory that can be used ?
>
> Here is my change:
>
> component: gst-plugins-base-0.10.21
>
> file: gstdecodebin2 line 1983
>
>   if (group->multiqueue) {
>     /* update runtime limits. At runtime, we try to keep the amount of buffers
>      * in the queues as low as possible (but at least 5 buffers). */
>     g_object_set (G_OBJECT (group->multiqueue),
>         "max-size-bytes", 2 * 1024 * 1024,
>         "max-size-time", 50 * GST_SECOND, "max-size-buffers", 5, NULL);
>     /* we can now disconnect any overrun signal, which is used to expose the
>      * group. */
>     if (group->overrunsig) {
>       GST_LOG ("Disconnecting overrun");
>       g_signal_handler_disconnect (group->multiqueue, group->overrunsig);
>       group->overrunsig = 0;
>     }
>   }
>
>
> Best Wishes,
>
> Conrad Cooke
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[firerock]

Forward this mail to [hidden email].

2011/3/2 firerock <[hidden email]>

Hi, Wim.

 

I met similar issue with Gstreamer 0.10.28 on Ubuntu 10.04. The difference is that it was bounded by the "max-size-bytes"(2 * 1024 * 1024) instead of "max-size-time"(2 * GST_SECOND). In Gstreamer 0.10.28, there are no upper limits on time difference for multiqueue and number of buffers in the queue ( the max-size-buffers can be increased one by one if the queue is full but the total bytes is smaller than max-size-bytes). It has only  limititation for max-size-bytes in decodebin2.

 

So, in my case, when playing a WMV file which is not muxed well enough using "gst-launch playbin2 uri=file:///", the audio would disappear after a while.  The phenomena is that, at some point, the audio queue is empty while video queue is full, the max-size-buffers of video queue would increase 1 and continue until the bytes of buffer in video queue is about to be larger than max-size-bytes(2MB). Then the number of video queue reaches muximum and the demuxer can not put more video data down to multiqueue in decodebin2, which leads no enough audio data was sended to mutiqueue and then audio decoder and audio sink. And the audio would disappear.

 

I have tried to increase the  max-size-bytes of decodebin2 and there was no audio disappearing issue.

# gst-launch filesrc location=/test.wmv ! decodebin2 max-size-bytes=8000000 name=demux ! pulsesink demux. ! xvimagesink

 

And I have checked that the default value for max-size-bytes in multiqueue is 10 * 1024 * 1024 (10MB), why did we set the default value  for max-size-bytes  to 2MB in decodebin2 ?

 

How about to set the default value  for max-size-bytes  to 10MB in decodebin2 as what we did in multiqueue or set it to unlimited value in decodebin2 ?

 

 

 

2009/10/21 Wim Taymans <[hidden email]>

On Tue, 2009-10-20 at 10:10 -0700, Conrad Cooke wrote:

Thanks for the patch but this issue has been fixed already. Newest
decodebin2 does not set an upper limit on time anymore, just bytes.

Wim

> I recently debugged an issue where playback was failing for certain mpeg-4 container files.  Turned out to be hitting a limitation placed on multiqueue.
> This is a result of how the file was encoded.  While it is a legal containerised media file, there is a large difference between audio and video timestamps.
> It is the purpose of multiqueue to realign the audio and video bins, I believe.
>
> multiqueue has various parameters which determine how much difference there can be between video and audio.
> >From gst-inspect:
>   name                : The name of the object
>                         flags: readable, writable
>                         String. Default: null Current: "multiqueue0"
>   extra-size-bytes    : Amount of data the queues can grow if one of them is empty (bytes, 0=disable)
>                         flags: readable, writable
>                         Unsigned Integer. Range: 0 - 4294967295 Default: 10485760 Current: 10485760
>   extra-size-buffers  : Amount of buffers the queues can grow if one of them is empty (0=disable)
>                         flags: readable, writable
>                         Unsigned Integer. Range: 0 - 4294967295 Default: 5 Current: 5
>   extra-size-time     : Amount of time the queues can grow if one of them is empty (in ns, 0=disable)
>                         flags: readable, writable
>                         Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 3000000000 Current: 3000000000
>   max-size-bytes      : Max. amount of data in the queue (bytes, 0=disable)
>                         flags: readable, writable
>                         Unsigned Integer. Range: 0 - 4294967295 Default: 10485760 Current: 10485760
>   max-size-buffers    : Max. number of buffers in the queue (0=disable)
>                         flags: readable, writable
>                         Unsigned Integer. Range: 0 - 4294967295 Default: 5 Current: 5
>   max-size-time       : Max. amount of data in the queue (in ns, 0=disable)
>                         flags: readable, writable
>                         Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 2000000000 Current: 2000000000
>
> Seems the last three properties are not implemented but were intended to support 'poorly encoded media'
>
> The three supported properties are set to their defaults on construction of the multiqueue.  They are also set twice by decodebin, once when a group is created (gst_decode_group_new) and once when its pads are exposed (gst_decode_group_expose)
> The second case set the values to
>
>         "max-size-bytes", 2 * 1024 * 1024,
>         "max-size-time", 2 * GST_SECOND,
>         "max-size-buffers", 5, NULL;
>
> It seems for some media, there is a greater than 2 second difference betwee audio and video, so I chnged max-size-time to be 50 seconds.
>
> My questions are
> Why is decodebin2 directly modifying this component at all ?
> Why is a 2 second maximum imposed - is the only real limitation is the maximum memory that can be used ?
>
> Here is my change:
>
> component: gst-plugins-base-0.10.21
>
> file: gstdecodebin2 line 1983
>
>   if (group->multiqueue) {
>     /* update runtime limits. At runtime, we try to keep the amount of buffers
>      * in the queues as low as possible (but at least 5 buffers). */
>     g_object_set (G_OBJECT (group->multiqueue),
>         "max-size-bytes", 2 * 1024 * 1024,
>         "max-size-time", 50 * GST_SECOND, "max-size-buffers", 5, NULL);
>     /* we can now disconnect any overrun signal, which is used to expose the
>      * group. */
>     if (group->overrunsig) {
>       GST_LOG ("Disconnecting overrun");
>       g_signal_handler_disconnect (group->multiqueue, group->overrunsig);
>       group->overrunsig = 0;
>     }
>   }
>
>
> Best Wishes,
>
> Conrad Cooke
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel



------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel