RFC - possible new vmethod for GstBaseSink

[Tom Bailey]

Hi all,

I would like to see if there is any support in the developer community for a new vmethod on GstBaseSink to allow derived sinks to indicate their pre-roll status to the base sink.

I've made this change as a patch to GstBaseSink and it seems to work well for our application. Our application runs on a set-top box and uses a custom sink element which wraps some proprietary APIs which give access to a hardware decoder which accepts MPEG TS or ES. Our custom sink is derived from GstBaseSink and runs unsynchronized (sync = FALSE).

We were sometimes seeing unacceptably long delays when the pipeline transitioned from PLAYING to PAUSED, which I tracked down to the fact that when the sink was asked to transition to PAUSED it would sometimes report that it needed to wait for preroll, even though video was currently playing. This meant that the state transition was delayed until a buffer arrived, which could be several seconds in some cases.

I tried to work out the logic in GstBaseSink by which it decides whether the element needs to preroll or not, and there doesn't seem to be any way for a derived sink to indicate that following a call to the render method it still has frames of video queued for display, and therefore doesn't need to pre-roll. So I tried adding a new vmethod, is_prerolled to GstBaseSink, and changed the logic in gst_base_sink_needs_preroll to look like this:

static gboolean
gst_base_sink_needs_preroll (GstBaseSink * basesink)
{
  gboolean is_prerolled, res, derived_needs_preroll = FALSE;
  GstBaseSinkClass *bclass;

  bclass = GST_BASE_SINK_GET_CLASS (basesink);

  /* we have 2 cases where the PREROLL_LOCK is released:
   *  1) we are blocking in the PREROLL_LOCK and thus are prerolled.
   *  2) we are syncing on the clock
   */
  is_prerolled = basesink->have_preroll || basesink->priv->received_eos ||
      basesink->priv->received_segment_done;

  if (!is_prerolled && bclass->is_prerolled) {
    derived_needs_preroll = !bclass->is_prerolled (basesink);
    res = derived_needs_preroll;
  } else {
    res = !is_prerolled;
  }

  GST_DEBUG_OBJECT (basesink, "have_preroll: %d, EOS: %d SEGMENT_DONE: %d "
      "derived_needs_preroll: %d => needs preroll: %d", basesink->have_preroll,
      basesink->priv->received_eos, basesink->priv->received_segment_done,
      derived_needs_preroll, res);

  return res;
}

The implementation of the is_prerolled vmethod in our derived sink now queries the hardware decoder, and if there are decoded frames of video queued for display it returns true. This has massively improved the performance of the application when transitioning from playing to paused.

Does this seem like a sensible enhancement to GstBaseSink for derived sinks which run with sync=FALSE? If so I can submit a patch.

If people don't think the change makes sense I'd be interested to hear what other approaches are available for sink elements which perform their own synchronization and which have to manage a queue of decoded frames.

Best regards

Tom

This transmission contains information that may be confidential and contain personal views which are not necessarily those of YouView TV Ltd. YouView TV Ltd (Co No:7308805) is a limited liability company registered in England and Wales with its registered address at YouView TV Ltd, 3rd Floor, 10 Lower Thames Street, London, EC3R 6YT. For details see our web site at http://www.youview.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le lundi 11 décembre 2017 à 12:37 +0000, Tom Bailey a écrit :
> The implementation of the is_prerolled vmethod in our derived sink
> now queries the hardware decoder, and if there are decoded frames of
> video queued for display it returns true. This has massively improved
> the performance of the application when transitioning from playing to
> paused.
>
> Does this seem like a sensible enhancement to GstBaseSink for derived
> sinks which run with sync=FALSE? If so I can submit a patch.

I'm not against, but it does not seems complete, specially for those
that want to implement proper clock slaving hense sync=TRUE. Best is to
file a bug so we can discuss the options.

Last time I had to deal with similar stack, I ended up forcing
async=FALSE on the base class, and implementing my own asynchronous
state change. It then worked seamlessly, because I could switch paused
whenever a frame was displayed (prerolled), and from paused to playing,
there was no code from the base class waiting. I understand this is
difficult, but I'm wondering if we could have helpers in that direction
instead.

Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel