ximagesrc at 144 fps

[Xiaojiba]

Hello,
I have been looking all around for one week now but no one seems to be
needing a ximagesrc at more than 30 fps... I saw one at 60fps but it's all
fine until 60 so, it was not related to my problem.

I have a pipeline to capture my screen to a file :

``
gst-launch-1.0 -e ximagesrc ! \
        "video/x-raw, framerate=144/1" ! \
        jpegenc quality=5 ! avimux ! \
        filesink location=file.avi
``

This compiles, but when I open the .avi with for example VLC,
I have a framerate of 144, BUT it takes 60 frames of the first second, 60 of
the second one and 24 if the third one

Easier with 120 fps : It takes 2s at 60 fps to have one full second on VLC
I believe ximagesrc is capped at 60 frame per second, am I correct ? Or my
pipeline is wrong ?

Thanks for help,



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le mercredi 22 avril 2020 à 10:01 -0500, Xiaojiba a écrit :

> Hello,
> I have been looking all around for one week now but no one seems to be
> needing a ximagesrc at more than 30 fps... I saw one at 60fps but it's all
> fine until 60 so, it was not related to my problem.
>
> I have a pipeline to capture my screen to a file :
>
> ``
> gst-launch-1.0 -e ximagesrc ! \
> "video/x-raw, framerate=144/1" ! \
> jpegenc quality=5 ! avimux ! \
> filesink location=file.avi
> ``
>
> This compiles, but when I open the .avi with for example VLC,
> I have a framerate of 144, BUT it takes 60 frames of the first second, 60 of
> the second one and 24 if the third one
>
> Easier with 120 fps : It takes 2s at 60 fps to have one full second on VLC
> I believe ximagesrc is capped at 60 frame per second, am I correct ? Or my
> pipeline is wrong ?

ximagesrc is capped by (first one hit) your CPU speed, compositor rate,
display rate. Many Linux compositor will hardcode 60, regardless of
your screen refresh rate.

If you are interested in better performance and better rate, you will
have to move on with gnome-shell Wayland and use the Screencast portal
to request a pipewire stream. This requires some programming though, of
course a project to make a gst source could be nice.

>
> Thanks for help,
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Daniel Johnson]

On Fri, Apr 24, 2020, 12:15 AM Xiaojiba <[hidden email]> wrote:

Hello,
I have been looking all around for one week now but no one seems to be
needing a ximagesrc at more than 30 fps... I saw one at 60fps but it's all
fine until 60 so, it was not related to my problem.

I have a pipeline to capture my screen to a file :

``
gst-launch-1.0 -e ximagesrc ! \
        "video/x-raw, framerate=144/1" ! \
        jpegenc quality=5 ! avimux ! \
        filesink location=file.avi
``

This compiles, but when I open the .avi with for example VLC,
I have a framerate of 144, BUT it takes 60 frames of the first second, 60 of
the second one and 24 if the third one

Easier with 120 fps : It takes 2s at 60 fps to have one full second on VLC
I believe ximagesrc is capped at 60 frame per second, am I correct ? Or my
pipeline is wrong ?

Thanks for help,

Ximagesrc is broken in a few ways, and once I understand how to write plugins better I want to rewrite it. One of the ways it's broken is by default it has use-damage on by default, and does non sensible stuff with that. What "damage" is used for on X is if say the only thing that changed is say an icon changed from play to pause it only propagates those changed pixels. Ximgsrc instead of sending through say a 400 pixel buffer for the new icon it sends a copy of the whole desktop. This would be fine if it did it at most once per screen update, but if multiple small changes happen in a single screen draw you get the whole desktop each time. So in your case it's probably getting 60 small changes in a single frame update and you get 60 frames of the same. Set use-damage=false and that might be enough.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Daniel Johnson]

If you are interested in better performance and better rate, you will
have to move on with gnome-shell Wayland and use the Screencast portal
to request a pipewire stream. This requires some programming though, of
course a project to make a gst source could be nice.

Unless they have massively fixed it since the last time I tried it I wouldn't recommend the gnome-shell recorder. It tended to instantly freeze my desktop so that I couldn't recover without hitting the power button. I was horrified when I looked at the source code. Basically if I remember correctly dropped in a chunk of gst-launch passed in from another program as a string via dbus and ran it directly in the same thread that listened for keypresses and drew the screen. So any gstreamer pipeline issues that keep the thread busy will mean no recovery. 

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le sam. 25 avr. 2020 22 h 00, Daniel Johnson <[hidden email]> a écrit :

If you are interested in better performance and better rate, you will
have to move on with gnome-shell Wayland and use the Screencast portal
to request a pipewire stream. This requires some programming though, of
course a project to make a gst source could be nice.

Unless they have massively fixed it since the last time I tried it I wouldn't recommend the gnome-shell recorder. It tended to instantly freeze my desktop so that I couldn't recover without hitting the power button. I was horrified when I looked at the source code. Basically if I remember correctly dropped in a chunk of gst-launch passed in from another program as a string via dbus and ran it directly in the same thread that listened for keypresses and drew the screen. So any gstreamer pipeline issues that keep the thread busy will mean no recovery. 

I'm referring to Screencast portal. This comment seems to mix multiple topics together in order to support a rant.

No one have touched ximagesrc for a long time, feel free to give it some love, it's old and slightly deprecated, but it haven't worked well since the introduction of GL compositors (at least that's what my memory thinks is the trip point). Note that normally how damage should work would be to download the damage from xserver and apply the delta on top of a copy of the full previous frames. That's what we do in rfbsrc (known as VNC). GStreamer does not have mechanisms to stream damages. This also mean it would be a very bad base for VNC server, or other damage.

With the Screencast portal (also works on KDE btw), the compositor can zero-copy the full rendered image through pipewire, which is a better fit for streaming. There is still some work to enable zero-copy with another GL pipeline, or VAAPI encoder, but this is being worked on. The portal implement X11 fallback for folks on X11.

I use this regularly inside video conference and for wifi display, I doubt my experience is unique. It also ensure that as a user I'm aware that someone is spying on my screen. With X11, any background process can do that silently.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Daniel Johnson]

I'm referring to Screencast portal. This comment seems to mix multiple topics together in order to support a rant.

I was trying to save the person asking the question from the pain of installing Wayland if they don't actually need it.

No one have touched ximagesrc for a long time, feel free to give it some love, it's old and slightly deprecated, but it haven't worked well since the introduction of GL compositors (at least that's what my memory thinks is the trip point). Note that normally how damage should work would be to download the damage from xserver and apply the delta on top of a copy of the full previous frames. That's what we do in rfbsrc (known as VNC). GStreamer does not have mechanisms to stream damages. This also mean it would be a very bad base for VNC server, or other damage.

Yeah gstreamer has no support for partial buffer updates so I don't understand why it's on by default when most users would want time based updates. It would make sense if they are trying to debug some aspect of screen updates and the want a frame for each update, but it's more data than most systems can handle so still not very useful. Weston has a screen recorder that captures to wcap format essentially each damage event but it only captures the damage so it isn't overloaded.

With the Screencast portal (also works on KDE btw), the compositor can zero-copy the full rendered image through pipewire, which is a better fit for streaming. There is still some work to enable zero-copy with another GL pipeline, or VAAPI encoder, but this is being worked on. The portal implement X11 fallback for folks on X11.

I use this regularly inside video conference and for wifi display, I doubt my experience is unique. It also ensure that as a user I'm aware that someone is spying on my screen. With X11, any background process can do that silently.

The thing I was talking about doesn't use pipewire so apparently we are talking about different features.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel