Pipeline Editor as Web App / UI

[Ralf Sippl]

Hi all,

GStreamer is extremely powerful and flexible, but to the average user, it's
extremely hard to understand and use. gst-inspect-1.0, gst-launch-1.0, even
those commands are impossibly hard to type :)

What it lacks is a friendly UI. Imagine a drag-and-drop pipeline editor in
the browser, with the default pipeline sending video and audio to the
browser e.g. using the newly added webrtcbin. As default source, it would
use either the camera, or videotestsrc (or was it testvideosrc? See what I
mean??)

The app could have four layers:

Layer 0: GStreamer. This could be the local GStreamer install via the OS's
package manager. Or, if that version is too old as in my case, a gst-build
'sandbox'.

Layer 1: (optional) your own set of GStreamer plugins. As an example, I have
a set of deep learning/inference plugins (face reco, obj detection, body
pose etc.).

Layer 2: HTTP API. Simply put, a gst-inspect/gst-launch wrapper written in
Python e.g. as RESTful API. Provides all data needed by Layer 3.

Layer 3: the fancy UI, HTML5 + JS (Vue.JS, React, whatever). Gives you
access to all available GStreamer elements (in a sane way), to the pipeline
as an editable graph, to each element and its properties (click on graph
node opens dialog), launch button starts the pipeline (opens video window);
get the textual notation of the pipeline (... ! ...), see the GStreamer log
in an extra window etc.

(Layer 4: package everything as a Docker container)

The only open source project I'm aware of that resembles this is
https://github.com/bbc/brave, but it works at a higher abstraction level
(which can be both good and bad). There were other attempts at implementing
editors, all of them native (e.g. based on Qt), but none of those I'm aware
of were successful.

Is there any open source project with the aim to implement a web-based
pipeline editor? Is there any interest to work on (or fund) such a project?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[R C]

not only the "average user" ...  I design design special purpose HPCs (HPC clusters, and

parallel file systems) for a living ..   and I can't figure gstreamer out either, while what

I am trying to do should not even be that hard. (getting gstreamer to stream an IP camera feed

to be embedded in a html/web page.)

Some sort of gui would definitely help.

As for the docker container...   yeah good luck with that, I know a bunch of people that are trying to build a robust, production level containers, so far containers don't deliver,  especially when it comes to compatibility with older software.

Ron

On 5/5/19 3:29 AM, gripsynth wrote:

Hi all, 

GStreamer is extremely powerful and flexible, but to the average user, it's
extremely hard to understand and use. gst-inspect-1.0, gst-launch-1.0, even
those commands are impossibly hard to type :) 

What it lacks is a friendly UI. Imagine a drag-and-drop pipeline editor in
the browser, with the default pipeline sending video and audio to the
browser e.g. using the newly added webrtcbin. As default source, it would
use either the camera, or videotestsrc (or was it testvideosrc? See what I
mean??) 

The app could have four layers: 

Layer 0: GStreamer. This could be the local GStreamer install via the OS's
package manager. Or, if that version is too old as in my case, a gst-build
'sandbox'. 

Layer 1: (optional) your own set of GStreamer plugins. As an example, I have
a set of deep learning/inference plugins (face reco, obj detection, body
pose etc.). 

Layer 2: HTTP API. Simply put, a gst-inspect/gst-launch wrapper written in
Python e.g. as RESTful API. Provides all data needed by Layer 3. 

Layer 3: the fancy UI, HTML5 + JS (Vue.JS, React, whatever). Gives you
access to all available GStreamer elements (in a sane way), to the pipeline
as an editable graph, to each element and its properties (click on graph
node opens dialog), launch button starts the pipeline (opens video window);
get the textual notation of the pipeline (... ! ...), see the GStreamer log
in an extra window etc. 

(Layer 4: package everything as a Docker container) 

The only open source project I'm aware of that resembles this is
https://github.com/bbc/brave, but it works at a higher abstraction level
(which can be both good and bad). There were other attempts at implementing
editors, all of them native (e.g. based on Qt), but none of those I'm aware
of were successful. 

Is there any open source project with the aim to implement a web-based
pipeline editor? Is there any interest to work on (or fund) such a project?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Ralf Sippl]

"Some sort of gui" is what didn't work that well in the past. It needs to be
a GUI which is fairly easy to set up and use by new users, works on every
OS, and is flexible enough to support more complex setups. Like, run locally
or remote (or on a Raspberry Pi), headless with scripts talking to the REST
API, with the default JS UI or with an UI tailored to your needs etc.

As for the issues you mentioned, a WebRTC based solution should be ok for
you, if not directly using webrtcbin (which BTW works well for me), then via
a server like Janus. In your thread, I just posted a pipeline which streams
to janus via udpsink.

Regarding Docker, projects like https://github.com/mpromonet/webrtc-streamer
show that containers work well, if done right...



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[coderofsalvation]

> What it lacks is a friendly UI.
 
I know this feeling, however I came to the conclusion that it's a tricky
topic.
While there are great GUI graph-processing solutions out there
(touchdesigner for windows, praxislive for linux), almost none of them can
run headless.
Gstreamer is positioned much closer to the metal, from the beginning.

I think the typical gstreamer-user first educates him/herself with less
powerful gui tools.
Personally, it took me 8 years to finally see the value of using gstreamer.

Having that all said, for a webbased future I could imagine gstreamer-nodes
to land in https://nodered.org.
Under the hood these nodes can simply execute gst-launch using
https://www.npmjs.com/package/gstreamer-superficial.
An interesting project in the desktop-space seems:

https://github.com/virinext/pipeviz



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[CreativeDataStudio]

A well-designed UI creates a seamless and intuitive user experience, making it easy for users to navigate, interact with, and accomplish tasks within the product. This leads to higher user satisfaction and engagement.

In a crowded marketplace, an excellent UI sets a product apart from competitors. It becomes a competitive advantage by providing a superior user experience and attracting users who appreciate the attention to design and usability.

A visually appealing and user-friendly UI attracts users and encourages them to explore and use the product. When users have a positive experience, they are more likely to continue using the product, leading to increased user retention and loyalty.

An intuitive UI design with clear feedback and guidance helps prevent user errors and reduces frustration. This results in a smoother user experience and saves time and effort for both users and support teams.