vaapi and x-forwarding

[GStreamer-devel mailing list]

Hi

I'm doing remote development and testing so I'm using X forwarding.

This works fine:

gst-launch-1.0 videotestsrc is-live=true pattern=ball !
video/x-raw,framerate=30/1,format=NV12 ! tee name=t ! queue ! x264enc
key-int-max=30 speed-preset=1 tune=zerolatency ! video/x-h264,profile=high !
h264parse ! fakesink t. ! queue ! videoconvert ! timeoverlay ! textoverlay
text=src ! ximagesink sync=0

This however doesn't:

gst-launch-1.0 videotestsrc is-live=true pattern=ball !
video/x-raw,framerate=30/1,format=NV12 ! tee name=t ! queue ! vaapih264enc
rate-control=4 bitrate=15000 ! video/x-h264,profile=high ! h264parse !
fakesink t. ! queue ! videoconvert ! timeoverlay ! textoverlay text=src !
ximagesink sync=0

Setting pipeline to PAUSED ...
error: XDG_RUNTIME_DIR not set in the environment.
X Error of failed request:  BadRequest (invalid request code or no such
operation)
  Major opcode of failed request:  154 (DRI2)
  Minor opcode of failed request:  1 (DRI2Connect)
  Serial number of failed request:  12
  Current serial number in output stream:  12

Both commands are run on a remote machine by SSH with "-X"

The problem seems to be related to my graphics card (NVIDIA) because of
direct/indirect rendering.
And I have already spend several days updating drivers and trying different
fixes without any luck.

I can accept that is problematic to forward stuff related to 3D graphics,
but this is not what I'm trying to do.
The encode/decode should happen in the remote machine, and I would expect
only the RAW frames to be sent by X to be rendered on my machine.

Using fx `glimagesink` works fine as well.
It is only VAAPI that is the problem, as soon as I add any VAAPi element to
the pipe I get that error.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[GStreamer-devel mailing list]

I guess the problems caused by the `SSH with "-X"`
The vaapi elements need the DRI2 protocal support, which can not span
the machine. It only support local X server.

You can run this command on the local machine or compile the VAAPI
plugins without X11 feature.



On Mon, 2021-06-14 at 06:22 -0500, Marianna S. Buschle via gstreamer-
devel wrote:

> Hi
>
> I'm doing remote development and testing so I'm using X forwarding.
>
> This works fine:
>
> gst-launch-1.0 videotestsrc is-live=true pattern=ball !
> video/x-raw,framerate=30/1,format=NV12 ! tee name=t ! queue ! x264enc
> key-int-max=30 speed-preset=1 tune=zerolatency ! video/x-
> h264,profile=high !
> h264parse ! fakesink t. ! queue ! videoconvert ! timeoverlay !
> textoverlay
> text=src ! ximagesink sync=0
>
> This however doesn't:
>
> gst-launch-1.0 videotestsrc is-live=true pattern=ball !
> video/x-raw,framerate=30/1,format=NV12 ! tee name=t ! queue !
> vaapih264enc
> rate-control=4 bitrate=15000 ! video/x-h264,profile=high ! h264parse
> !
> fakesink t. ! queue ! videoconvert ! timeoverlay ! textoverlay
> text=src !
> ximagesink sync=0
>
> Setting pipeline to PAUSED ...
> error: XDG_RUNTIME_DIR not set in the environment.
> X Error of failed request:  BadRequest (invalid request code or no
> such
> operation)
>   Major opcode of failed request:  154 (DRI2)
>   Minor opcode of failed request:  1 (DRI2Connect)
>   Serial number of failed request:  12
>   Current serial number in output stream:  12
>
> Both commands are run on a remote machine by SSH with "-X"
>
> The problem seems to be related to my graphics card (NVIDIA) because
> of
> direct/indirect rendering.
> And I have already spend several days updating drivers and trying
> different
> fixes without any luck.
>
> I can accept that is problematic to forward stuff related to 3D
> graphics,
> but this is not what I'm trying to do.
> The encode/decode should happen in the remote machine, and I would
> expect
> only the RAW frames to be sent by X to be rendered on my machine.
>
> Using fx `glimagesink` works fine as well.
> It is only VAAPI that is the problem, as soon as I add any VAAPi
> element to
> the pipe I get that error.
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[GStreamer-devel mailing list]

I need to run on the remote machine, I'm doing remote development on specific
HW.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[GStreamer-devel mailing list]

And one of my colleagues is able to run the same command over X forwarding.

So it must be a compatibility issue between the HW on my machine and the
remote machine, since it wants to forward the VAAPI commands (which I don't
quite get why unless it was the vaapisink I was using).

But you say that if we compile VAAPI without X support then it won't try to
forward those?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[GStreamer-devel mailing list]

Vaapi element inside use gst_vaapi_create_display() to create the
VADisplay. It will try wayland, x11, drm one by one. In your case, the
x11 way return success and so choose. But later, when vaInitialize, the
it fails to init because no DRI2 extersion support.

The best way is disabling the X11 support in build. And because you do
not need to use vaapisink, it is OK.


On Mon, 2021-06-14 at 07:54 -0500, Marianna S. Buschle via gstreamer-
devel wrote:

> And one of my colleagues is able to run the same command over X
> forwarding.
>
> So it must be a compatibility issue between the HW on my machine and
> the
> remote machine, since it wants to forward the VAAPI commands (which I
> don't
> quite get why unless it was the vaapisink I was using).
>
> But you say that if we compile VAAPI without X support then it won't
> try to
> forward those?
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[GStreamer-devel mailing list]

Is that only possible at compile time?
Or can I do something programmatically (in a python app)?

I ask that because I also plan to have decoupled pipelines using something
like the intervideo or interpipe elements.
And I was told that in that case I also need to do some things (handle the
got context message on the bus?) in order to make sure that fx a
vaapih264enc and vaapih264dec in different pipelines can share the same
VASurface.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[GStreamer-devel mailing list]

I have managed to get somewhat the behavior I want by doing something like:

gst-launch-1.0 videotestsrc is-live=true pattern=ball !
video/x-raw,framerate=30/1,format=NV12 ! tee name=t ! queue ! vaapih264enc
rate-control=4 bitrate=5000 ! video/x-h264,profile=high ! queue ! h264parse
! queue ! vaapih264dec low-latency=1 ! queue !
video/x-raw,framerate=30/1,format=NV12 ! queue ! videoconvert ! queue !
ximagesink sync=0 fakesrc ! vaapisink display=drm --gst-debug=*:3

The 'fakesrc ! vaapisink display=drm' forces the VADisplay to DRM and that
gets shared on the bus and set automatically on the
vaapih264enc/vaapih264dec

This pipeline dies right away:

0:00:00.167233055  5878 0x564d0741e4f0 WARN               vaapisink
gstvaapipluginbase.c:1260:gst_vaapi_plugin_base_pad_get_input_buffer:<vaapisink0>
error: failed to validate source buffer
0:00:00.167252497  5878 0x564d0741e4f0 WARN               vaapisink
gstvaapipluginbase.c:1260:gst_vaapi_plugin_base_pad_get_input_buffer:<vaapisink0>
error: failed to validate source buffer
ERROR: from element /GstPipeline:pipeline0/GstVaapiSink:vaapisink0: failed
to validate source buffer
Additional debug info:
../gstreamer-vaapi-1.18.2/gst/vaapi/gstvaapipluginbase.c(1260):
gst_vaapi_plugin_base_pad_get_input_buffer ():
/GstPipeline:pipeline0/GstVaapiSink:vaapisink0:
failed to validate source buffer
ERROR: pipeline doesn't want to preroll.
0:00:00.167433979  5878 0x564d0741e4f0 WARN                 basesrc
gstbasesrc.c:3127:gst_base_src_loop:<fakesrc0> error: Internal data stream
error.
0:00:00.167453711  5878 0x564d0741e4f0 WARN                 basesrc
gstbasesrc.c:3127:gst_base_src_loop:<fakesrc0> error: streaming stopped,
reason error (-5)
ERROR: from element /GstPipeline:pipeline0/GstFakeSrc:fakesrc0: Internal
data stream error.

But the rest continues to play fine.

Now I would like a bit more elegant solution, since I need to handle anyway
the HAVE_CONTEXT and NEED_CONTEXT messages on the bus because I want to
decouple my pipelines.
So I would like to do something like
https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/blob/master/tests/examples/test-vaapicontext.c#L127
but with the 'vaGetDisplayDRM()'.
However I can't seem to find python bindings for the VAAPI API?

And if I end up sticking with the solution of having a "dummy" pipeline like
'fakesrc ! vaapisink display=drm' just to create the DRM VADisplay so I can
set it in the others is there any penalty on doing that?



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[GStreamer-devel mailing list]

On Tue, 15 Jun 2021 at 03:32, Marianna S. Buschle via gstreamer-devel wrote:
>
> Now I would like a bit more elegant solution, since I need to handle anyway
> the HAVE_CONTEXT and NEED_CONTEXT messages on the bus because I want to
> decouple my pipelines.
> So I would like to do something like
> https://gitlab.freedesktop.org/gstreamer/gstreamer-vaapi/-/blob/master/tests/examples/test-vaapicontext.c#L127

Yes, that's the way to choose programmatically and specific display
(DRM in your case).

> but with the 'vaGetDisplayDRM()'.
> However I can't seem to find python bindings for the VAAPI API?

I'm not a python expert but  this looks the way

https://docs.python.org/3.8/library/ctypes.html

>>> from ctypes import *
>>> va_drm = cdll.LoadLibrary("/usr/lib/x86_64-linux-gnu/libva-drm.so")
>>> va_drm.vaGetDisplayDRM(opened_fd)

vmjl
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[GStreamer-devel mailing list]

I have just tried the solution with having multiple pipelines and using the
HAVE / NEED_CONTEXT messages on the bus.

pipelines.append("videotestsrc num-buffers=1 ! vaapisink display=drm
name=vaapi_dummy")

pipelines.append("videotestsrc is-live=true pattern=ball !
video/x-raw,framerate=30/1,format=NV12 ! "
                 "tee name=t ! queue ! vaapih264enc rate-control=4
bitrate=15000 ! video/x-h264,profile=high ! h264parse ! "
                 "interpipesink name=vtest sync=false async=false "
                 "t. ! queue ! videoconvert ! timeoverlay ! textoverlay
text=src ! ximagesink sync=0 ")

pipelines.append("interpipesrc listen-to=vtest is-live=false
stream-sync=restart-ts ! "
                 "identity sync=true ! h264parse ! vaapih264dec
low-latency=true ! queue ! videoconvert ! timeoverlay ! textoverlay
name=text_out text=out ! ximagesink sync=0 ")

The first pipeline goes to playing from start, and then after I got the
context from it I start the other 2.

However it doesn't work, I end up getting the error as soon as the 2 other
pipelines go into playing/ready/paused, before I get the NEED_CONTEXT
message from them...

X Error of failed request:  BadRequest (invalid request code or no such
operation)
  Major opcode of failed request:  154 (DRI2)
  Minor opcode of failed request:  1 (DRI2Connect)
  Serial number of failed request:  12
  Current serial number in output stream:  12

The only way I can avoid the error is by adding fx "videotestsrc
num-buffers=1 ! vaapisink display=drm" to each pipeline.
Then I can see the NEED/HAVE context messages from all the vaapisinks, but
not from encoder/decoder.
I guess their contexts are being set by upstream/downstream queries.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel