appsrc usage (push and pull mode)

[distran]

Hello.

I have some doubts regarding appsrc development and using appsrc in push vs pull mode. I need to inject frames from a live source to a pipeline. Basically, I have my source (producer) generating frames in one thread, and putting them in a buffer (let's say these are just OpenCV generated images). GStreamer thread should take the frame out of the buffer and push it to the pipeline.

I noticed there are esencially two ways you can use appsrc - "push" mode (for live sources) and "pull" mode (from non-live sources). I can understand how should I use them but I am missing the practical point on when to use one implementation over another. Let me give you two examples (I tried to leave only the essence of my doubts)). Please correct me in my understanding, if anything is wrong:

AppSrc in "pull" mode:
  - we connect a function "cb_need_data" to a signal "need_data" from a source
  - if an appsrc needs data, it will emit "need-data" signal and then our function ("cb_need_data") is invoked, where "push-buffer" signal is emmited with a new buffer to transmit

static void cb_need_data (GstElement *appsrc)
{
  // prepare buffer

  g_signal_emit_by_name (appsrc, "push-buffer", buffer, &ret);
}

gint main (gint   argc, gchar *argv[])
{
  // initialize GStreamer, pipeline and stuff
  
  /* setup appsrc */
  g_object_set (G_OBJECT (appsrc),
		"stream-type", 0,
		"format", GST_FORMAT_TIME, NULL);
  g_signal_connect (appsrc, "need-data", G_CALLBACK (cb_need_data), NULL);

  gst_element_set_state (app->pipeline, GST_STATE_PLAYING);
  g_main_loop_run (app->loop);

  // cleanup at the end
}

AppSrc in "push" mode:
  - we connect a functions "start_feed" and "stop_feed" to a signals "need_data" and "enough-data", within these functions we add function "read_data" to Glib idle loop. Subsequently, "read_data" will be invoked (according to documentation) "whenever there are no higher priority events pending to the default main loop."

static gboolean read_data(gst_app_t *app)
{
  // prepare buffer 

  g_signal_emit_by_name (app->appsrc, "push-buffer", buffer, &ret);
}

static void start_feed (GstElement * pipeline, guint size, gst_app_t *app)
{
    if (app->sourceid == 0) {
        GST_DEBUG ("start feeding");
        app->sourceid = g_idle_add
        g_idle_add ((GSourceFunc) read_data, app);
    }
}

static void stop_feed (GstElement * pipeline, gst_app_t *app)
{
    if (app->sourceid != 0) {
        GST_DEBUG ("stop feeding");
        g_source_remove (app->sourceid);
        app->sourceid = 0;
    }
}

int main(int argc, char *argv[])
{
  // initialize GStreamer, pipeline and stuff

  g_signal_connect(app->src, "need-data", G_CALLBACK(start_feed), app);
  g_signal_connect(app->src, "enough-data", G_CALLBACK(stop_feed), app);

  gst_element_set_state (app->pipeline, GST_STATE_PLAYING);
  g_main_loop_run (app->loop);

  // cleanup at the end
}

Questions:
  - In "push" mode - how often is actually "push-buffer" signal invoked? What, if there are a lot of "higher priority events"? How precisely can we control the rate with which "push-buffer" signal is emitted (for the pipeline it is usually required to send frames with approximately specified rate)?
  - What is the difference between g_signal_emit_by_name (app->appsrc, "push-buffer", buffer, &ret); and using directly ret = gst_app_src_push_buffer(app->src, buffer); ?
  - In both modes function, that emits "push-buffer" signal is invoked somewhat automatically (in "pull" mode, when we receive "need-data" signal, and in "push" mode when there are no higher priority events). Is it possible to directly emit "push-buffer" signal when we have our frame ready to emit? (as I said before, in my architecture I have an element, that generates frames constantly with a certain frame rate. Would it be possible to emit "push-buffer" every time frame is generated? What are the drawbacks of such solution? - I tried emitting "push-buffer" signal directly from producer thread and it seems to work well).

Thanks for all clarifications!

Regards,
Michal

[Sebastian Dröge-3]

On Mo, 2013-10-21 at 07:46 -0700, distran wrote:

> [...]
> Questions:
>   - In "push" mode - how often is actually "push-buffer" signal invoked?
> What, if there are a lot of "higher priority events"? How precisely can we
> control the rate with which "push-buffer" signal is emitted (for the
> pipeline it is usually required to send frames with approximately specified
> rate)?

push-buffer is an action signal, you invoke/emit it and not appsrc. It
happens as often as you do that. And will block if the appsrc buffer
queue is filled until downstream handles another buffer.

>   - What is the difference between g_signal_emit_by_name (app->appsrc,
> "push-buffer", buffer, &ret); and using directly ret =
> gst_app_src_push_buffer(app->src, buffer); ?

None at all

>   - In both modes function, that emits "push-buffer" signal is invoked
> somewhat automatically (in "pull" mode, when we receive "need-data" signal,
> and in "push" mode when there are no higher priority events). Is it possible
> to directly emit "push-buffer" signal when we have our frame ready to emit?
> (as I said before, in my architecture I have an element, that generates
> frames constantly with a certain frame rate. Would it be possible to emit
> "push-buffer" every time frame is generated? What are the drawbacks of such
> solution? - I tried emitting "push-buffer" signal directly from producer
> thread and it seems to work well).

Yes, that's exactly what you should do. Start a thread and from that
thread push buffers to appsrc as fast as possible.

--
Sebastian Dröge <[hidden email]>
Centricular Ltd - http://www.centricular.com
Expertise, Straight from the Source

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim Müller]

On Sun, 2013-10-27 at 21:42 +0000, Sebastian Dröge wrote:

> >   - What is the difference between g_signal_emit_by_name (app->appsrc,
> > "push-buffer", buffer, &ret); and using directly ret =
> > gst_app_src_push_buffer(app->src, buffer); ?
>
> None at all

Just one tiny difference: g_signal_emit_by_name() does not take
ownership of the buffer (meaning: you have to unref it afterwards),
while gst_app_src_push_buffer() does.

Cheers
 -Tim

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[HermannSW]

This thread is old, but really interesting.

I tried to use  gst_app_src_push_buffer()  and   g_idle_add(), details and
source files attached here:
https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=197124&p=1234930#p1234930
<https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=197124&p=1234930#p1234930>  

I seem to be missing a step that is needed in order to not freeze pipeline.
The diff to working sample is very small.

What needs to be done to get  gst_app_src_push_buffer()  program streaming?

Hermann.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Antonio Ospite-2]

On Tue, 14 Nov 2017 17:22:34 -0700 (MST)
HermannSW <[hidden email]> wrote:

> This thread is old, but really interesting.
>
> I tried to use  gst_app_src_push_buffer()  and   g_idle_add(), details and
> source files attached here:
> https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=197124&p=1234930#p1234930
> <https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=197124&p=1234930#p1234930>  
>
> I seem to be missing a step that is needed in order to not freeze pipeline.
> The diff to working sample is very small.
>
> What needs to be done to get  gst_app_src_push_buffer()  program streaming?
>

The problem is not about gst_app_src_push_buffer() but about
read_data().

The callback you set with g_idle_add() needs to return TRUE (or its
alias G_SOURCE_CONTINUE) if you want it to be called again and
G_SOURCE_REMOVE if you don't.

See the documentation:
https://developer.gnome.org/glib/stable/glib-The-Main-Event-Loop.html#g-idle-add
https://developer.gnome.org/glib/stable/glib-The-Main-Event-Loop.html#GSourceFunc

In a _synthetic_ example for push mode like yours you may also want some
rate limiting (e.g. a sleep) to avoid thrashing your system.

And about pull mode (appsrc-launch2.c), remember that when the
cb_need_data() function returns you must _always_ have pushed a buffer
(or sent an EOS), otherwise the pipeline stalls. Your simple example
does that, but when there is more logic involved this detail may be
overlooked, at least it happened to me in
https://git.ao2.it/gst-aseq-appsrc.git/

Ciao,
   Antonio

P.S. use "diff -u" when you can.

--
Antonio Ospite
https://ao2.it
https://twitter.com/ao2it

A: Because it messes up the order in which people normally read text.
   See http://en.wikipedia.org/wiki/Posting_style
Q: Why is top-posting such a bad thing?
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[HermannSW]

Antonio Ospite-2 wrote:
> The problem is not about gst_app_src_push_buffer() but about
> read_data().
>
> The callback you set with g_idle_add() needs to return TRUE (or its
> alias G_SOURCE_CONTINUE) if you want it to be called again and
> G_SOURCE_REMOVE if you don't.
>
Thanks, returning G_SOURCE_CONTINUE in callback funtion made it work
immediately!

> ...
> In a _synthetic_ example for push mode like yours you may also want some
> rate limiting (e.g. a sleep) to avoid thrashing your system.
>
Of course I did nearly freeze my Pi Zero due to running out of memory
several times before getting rate limiting right.
[not a good idea to stream 100.000 frames per second into a gstreamer
pipeline without that :) ]

The result is far better than what I need, was able to push 216KB frames
with 626fps on average!
And without memory leak, I did babysit a 441.100 push frames run with top,
details here:
https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=197124&p=1235322#p1235322

Hermann.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel