GStreamer and gjs

[Tony Houghton]

I'm having some trouble using GStreamer with gjs. I wasn't sure whether it was supposed top be supported at all, because Javascript isn't mentioned in its list of bindings, and other bindings, like python, seem to need help from custom bindings instead of being able to rely on gi alone. But the API docs support Javascript, so it seems like gjs is intended to be supported.

However, problems started almost immediately. The first is that if I call Gst.init(ARGV) it crashes with an attempt to free a block of memory that isn't freeable. I solved that by simply passing null instead, and I guess I could find a workaround if I really do need it to process ARGV.

More problems occurred when I attempted to write a very simple pipeline, similar to the playbin example:

function bus_cb(bus, message) {
    log(`Got GstMessage type ${gst.message_type_get_name(message.type)}`);

    ...
    return true;
}

const playbin = gst.ElementFactory.make("playbin", "play");
playbin.get_bus().add_watch(glib.PRIORITY_DEFAULT, bus_cb);
playbin.set_property("uri", uri);
playbin.set_state(gst.State.PLAYING);

const playbin = gst.ElementFactory.make("playbin", "play");
playbin.get_bus().add_watch_full(glib.PRIORITY_DEFAULT, bus_cb);
playbin.set_property("uri", uri);
playbin.set_state(gst.State.PLAYING);

(The URI I'm using is HTTP to stream DVB as an MPEG transport stream from tvheadend, but I have the same problem if I try to use a file:/// for a saved mpeg.ts instead).

First, note that although the API docs say you need to use add_watch_full(), AFAICT you actually have to use add_watch() with arguments as for add_watch_full(), because the latter doesn't exist in gjs. This is something to do with the 'full' version shadowing the other, which I don't fully understand, but I managed to work it out by looking at the output of ts-for-gjs.

The main problem though is that the pipeline only runs for a few seconds, then crashes with this error:

Trying to dispose element inputselector2, but it is in PLAYING instead of the NULL state.

I haven't tested whether this is specific to gjs, or also occurs in python or even C, but I get the same error in OS X as well as Arch Linux. Both are using gstreamer 1.18.0.

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Mon, 2020-09-21 at 16:55 +0100, Tony Houghton wrote:
> I'm having some trouble using GStreamer with gjs. I wasn't sure
> whether it was supposed top be supported at all, because Javascript
> isn't mentioned in its list of bindings, and other bindings, like
> python, seem to need help from custom bindings instead of being able
> to rely on gi alone. But the API docs support Javascript, so it seems
> like gjs is intended to be supported.

It's supposed to work, yes. There might be some bugs though, it's not
the most widely used language for GStreamer.

> However, problems started almost immediately. The first is that if I
> call Gst.init(ARGV) it crashes with an attempt to free a block of
> memory that isn't freeable. I solved that by simply passing null
> instead, and I guess I could find a workaround if I really do need it
> to process ARGV.

Can you report that to gjs? That would be a bug on the bindings side
  https://gitlab.gnome.org/GNOME/gjs

> More problems occurred when I attempted to write a very simple
> pipeline, similar to the playbin example:
>
> [...]
>
> First, note that although the API docs say you need to use
> add_watch_full(), AFAICT you actually have to use add_watch() with
> arguments as for add_watch_full(), because the latter doesn't exist
> in gjs. This is something to do with the 'full' version shadowing the
> other, which I don't fully understand, but I managed to work it out
> by looking at the output of ts-for-gjs.

That seems like a problem in the documentation indeed. The non-full
version is not usable from bindings, so the full version is configured
to shadow/hide the non-full one.

Where is that in the documentation? Can you create an issue for that at
  https://gitlab.freedesktop.org/gstreamer/gst-docs/

> The main problem though is that the pipeline only runs for a few
> seconds, then crashes with this error:
>
> Trying to dispose element inputselector2, but it is in PLAYING
> instead of the NULL state.
>
> I haven't tested whether this is specific to gjs, or also occurs in
> python or even C, but I get the same error in OS X as well as Arch
> Linux. Both are using gstreamer 1.18.0.

Your code doesn't have any main loop or similar, so after setting the
pipeline to PLAYING it simply exits. You probably want to wait in one
way or another until the EOS (or ERROR) message is received from the
bus.

Then before shutting down the application you need to set the state of
the pipeline to NULL. When the last reference goes out of scope in gjs,
the object will be destroyed. But GStreamer requires for all elements
that their state is NULL before they're destroyed. This is something
you need to take care of here.

--
Sebastian Dröge, Centricular Ltd · https://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tony Houghton]

On Mon, 21 Sep 2020 at 17:10, Sebastian Dröge <[hidden email]> wrote:

On Mon, 2020-09-21 at 16:55 +0100, Tony Houghton wrote:
> The first is that if I
> call Gst.init(ARGV) it crashes with an attempt to free a block of
> memory that isn't freeable. I solved that by simply passing null
> instead, and I guess I could find a workaround if I really do need it
> to process ARGV.

Can you report that to gjs? That would be a bug on the bindings side
  https://gitlab.gnome.org/GNOME/gjs

OK, I've done that: https://gitlab.gnome.org/GNOME/gjs/-/issues/350

 

> First, note that although the API docs say you need to use
> add_watch_full(), AFAICT you actually have to use add_watch() with
> arguments as for add_watch_full(), because the latter doesn't exist
> in gjs. This is something to do with the 'full' version shadowing the
> other, which I don't fully understand, but I managed to work it out
> by looking at the output of ts-for-gjs.

That seems like a problem in the documentation indeed. The non-full
version is not usable from bindings, so the full version is configured
to shadow/hide the non-full one.

Where is that in the documentation? Can you create an issue for that at
  https://gitlab.freedesktop.org/gstreamer/gst-docs/

I've reported that too. I've included a link to the page where I found the mistake in my report: https://gitlab.freedesktop.org/gstreamer/gst-docs/-/issues/72

 

> The main problem though is that the pipeline only runs for a few
> seconds, then crashes with this error:
>
> Trying to dispose element inputselector2, but it is in PLAYING
> instead of the NULL state.
>
> I haven't tested whether this is specific to gjs, or also occurs in
> python or even C, but I get the same error in OS X as well as Arch
> Linux. Both are using gstreamer 1.18.0.

Your code doesn't have any main loop or similar, so after setting the
pipeline to PLAYING it simply exits. You probably want to wait in one
way or another until the EOS (or ERROR) message is received from the
bus.

Sorry, my code does include a GMainLoop, but I missed it out here for brevity. Without the watch callback my code seems to work OK, but with the callback it only runs for a few seconds (it does open a window and play back the stream for a while) before stopping with the above error.

I've tried a few other things. Equivalents in C and python both run successfully. The Javascript version also runs OK if I use add_signal_watch() instead of add_watch(). So I think the problem is that gjs can't cope with a callback being called off the main thread. This could theoretically cause problems in python too, but I presume it works either because this has been addressed in its custom Gst bindings, or at a lower level. I'm writing a companion library in introspectable C for my app, so if I need synchronous callbacks for something like embedding video in a window of my choice, I can probably add support functions in the library.

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Mon, 2020-09-21 at 20:20 +0100, Tony Houghton wrote:

>
> > Your code doesn't have any main loop or similar, so after setting the
> > pipeline to PLAYING it simply exits. You probably want to wait in one
> > way or another until the EOS (or ERROR) message is received from the
> > bus.
>
> Sorry, my code does include a GMainLoop, but I missed it out here for
> brevity. Without the watch callback my code seems to work OK, but
> with the callback it only runs for a few seconds (it does open a
> window and play back the stream for a while) before stopping with the
> above error.
>
> I've tried a few other things. Equivalents in C and python both run
> successfully. The Javascript version also runs OK if I use
> add_signal_watch() instead of add_watch(). So I think the problem is
> that gjs can't cope with a callback being called off the main thread.
> This could theoretically cause problems in python too, but I presume
> it works either because this has been addressed in its custom Gst
> bindings, or at a lower level. I'm writing a companion library in
> introspectable C for my app, so if I need synchronous callbacks for
> something like embedding video in a window of my choice, I can
> probably add support functions in the library.

In Python that's not a problem, the callbacks can come from any thread.
There's just Python's global interpreter lock that will block
everything else while some other thread calls into Python code.

Can you provide a full gjs example for the application that causes the
problem above? Seems like another bug in gjs to me though but I can
take a short look first :)


Thanks for reporting the issues btw!

--
Sebastian Dröge, Centricular Ltd · https://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tony Houghton]

On Mon, 21 Sep 2020 at 21:25, Sebastian Dröge <[hidden email]> wrote:

Can you provide a full gjs example for the application that causes the
problem above? Seems like another bug in gjs to me though but I can
take a short look first :)

I've tried distilling it down to a small standalone script, but now I can't reproduce the problem, even in the full version if I replace the signal-based watch with the threaded one. I'll attach the simple version anyway. Another, rarer, symptom is that sometimes the window doesn't open and the messages stop appearing.

One possible reason for the original error message could be that the callback was getting EOS or ERROR messages and quitting without stopping the pipeline, and the threading was causing the log of the message to get lost. But there must have been something wrong for the pipeline to have been sending such messages or otherwise quitting. I'll ask for clarification about threaded callbacks in gjs on the GNOME discourse bindings forum.

 

Thanks for reporting the issues btw!

The least I could do after you helpfully provided links :-).

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tony Houghton]

I've learnt the hard way that it's important to be aware of floating references with gstreamer. In a simple gjs script to run a playbin I was having problems with various symptoms, including the pipeline usually stalling or deadlocking or something, that I couldn't track down. Then on a different machine the symptoms included lots of errors about unreffing objects while the pipeline was still in PLAYING state. I added gst_object_ref_sink() where appropriate and all the problems went away. But I have to use ref_sink() in the Javascript too. If I miss out the second of these two lines, I get all the nasty symptoms back;

    const playbin = Gst.ElementFactory.make("playbin", "play");
    playbin.ref_sink();

I think bindings are supposed to be able to work without explicit (user) calls to ref_sink() when they encounter '(transfer floating)' aren't they? So does gjs not support this (yet)? 

On Mon, 21 Sep 2020 at 16:55, Tony Houghton <[hidden email]> wrote:

I'm having some trouble using GStreamer with gjs. I wasn't sure whether it was supposed top be supported at all, because Javascript isn't mentioned in its list of bindings, and other bindings, like python, seem to need help from custom bindings instead of being able to rely on gi alone. But the API docs support Javascript, so it seems like gjs is intended to be supported.

-- 

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel