Trying to understand how to make alpha & ximagesrc work efficiently

[Alain Culos]

Hi,

I am a recent newcomer to gstreamer.
Trying to use `gst-launch-1.0` to chroma-key my webcam and insert a pdf as background.

My attempt is very imperfect:
1/ I have not been able to find the right parameters to make the window sizes match.
2/ gst-launch occasionally complains it loses a lot of buffers.
3/ the video output only updates seemingly randomly, infrequently, and seems to (but not always) needs the pdf window to have focus and the mouse to be over it (but not exclusively).

Maybe there is a better solution.
My desktop is quite powerful: a modern (about 1yo, i7, 6-core, hyperthreaded, 64GB RAM), 4k display, 1080p USB webcam, SSD.
Linux 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
()

The commands I ran are:

1/ testing the chroma-key with a chequered background
gst-launch-1.0 \
  videotestsrc pattern=checkers-8 ! video/x-raw,width=800,height=640 ! mixer.sink_0 \
  autovideosrc ! video/x-raw,width=800 ! alpha method=custom target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
  videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 ! videoconvert ! autovideosink
-> Very slow video, does not chroma-key on white!
gst-launch recurrent message: "A lot of buffers are being dropped"

2/ testing the chroma-key (2nd attempt)
gst-launch-1.0 \
  videotestsrc pattern=checkers-8 ! video/x-raw,width=800,height=640,framerate=1/1 ! mixer.sink_0 \
  autovideosrc ! video/x-raw,width=800 ! alpha method=custom target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
  videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 ! videoconvert ! autovideosink
-> Same as above

3/ testing my video
gst-launch-1.0 autovideosrc ! video/x-raw,width=800 ! videoconvert ! autovideosink
-> Showing my webcam is almost real-time, but has a bit of lag
The same test at 1080p is very laggy

4/ testing a source application window (evince, showing a pdf file)
gst-launch-1.0 \
  ximagesrc xid=0x06800008 ! mixer.sink_0 \
  autovideosrc ! video/x-raw,width=800 ! alpha method=custom target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
  videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 ! videoconvert ! autovideosink
-> Does not update the video in realtime (only updates rarely, at most every few seconds, when the application window has the focus or redraw events (possibly and/or mouse over)), does not allow me to resize the application window video stream.


Hoping to find here some pointers to solve the issues highlighted above, most importantly performance (framerate).

Next steps: a/ including the audio stream, b/ outputting to a video device so I can use this as a source for a video conference (zoom/skype/jitsi), c/ using an actual green screen.

Thanks for any help, pointers to documentation or tutorials.
Regards,
Alain.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Alain Culos]

Dear All,

I just discovered GStreamer, so apologies if this is a basic error, but no matter how hard I try I can't find what is wrong with my set-up / pipeline.

===========
Summary:
When using ximagesrc, error messages appear to tell me buffers are lost and that there may be timestamping issues or that my computer is slow.

I have a beefy desktop, 1yo i7, 6-core hyperthreaded, 64GB RAM ubuntu 18.04).
GStreamer 1.14.5

===========
In detail:

The video shown with this command line is very laggy (it hardly uses any RAM or CPU // other note: my desktop easily copes with e.g. zoom meetings):

    gst-launch-1.0 autovideosrc ! videoconvert ! autovideosink # shows no error message
    gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! autovideosink # does exactly the same


Preparing the ground for chroma-keying a green screen background, this example works the same way the first two do (laggy but good quality & no error message showing):
    w=1280; h=800; x0=450; y0=200; gst-launch-1.0 \
        v4l2src device=/dev/video0 \
        ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
        ! alpha method=green angle=70 \
        ! videoconvert \
        ! autovideosink

The background to replace the green screen will be taken from an application window.
This causes issues with buffers dropped & it complains about timestamping issues.
    gst-launch-1.0   ximagesrc xid=0x02a00006 ! videoconvert ! autovideosink

Below is the output of the command above:
    Setting pipeline to PAUSED ...
    Pipeline is live and does not need PREROLL ...
    Setting pipeline to PLAYING ...
    New clock: GstSystemClock
    WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
    Additional debug info:
    gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
    There may be a timestamping problem, or this computer is too slow.


Even if I impose a low frame rate, it outputs the same error messages as above, command line:
    gst-launch-1.0   ximagesrc xid=0x02a00006 ! video/x-raw,framerate=5/1 ! videoconvert ! autovideosink

Obviously, if I combine the two, performance gets worse, despite still not hitting the CPU or RAM, and the following command shows exactly the same messages as above:
    w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
        v4l2src device=/dev/video0 \
        ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
        ! alpha method=green angle=70 \
        ! mixer.sink_1   \
        \
        ximagesrc xid=0x02a00006 ! videoscale \
        ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
        \
        videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
        ! videoconvert ! autovideosink

Whether I put ximagesrc before v4l2src or after changes nothing.

(next challenges: 1/ channelling this to /dev/video10; 2/ channeling the audio from the webcam (/dev/video0) to the virtual webcam (dev/video10))

For now I  would most appreciate some help with making the above commands works without error/warning messages and good performance. Or in other words how do I resolve timestamping with ximagesrc?

Best regards,
Alain.


On Wed, 1 Apr 2020, at 02:42, Alain Culos wrote:

> Hi,
>
> I am a recent newcomer to gstreamer.
> Trying to use `gst-launch-1.0` to chroma-key my webcam and insert a pdf
> as background.
>
> My attempt is very imperfect:
> 1/ I have not been able to find the right parameters to make the window
> sizes match.
> 2/ gst-launch occasionally complains it loses a lot of buffers.
> 3/ the video output only updates seemingly randomly, infrequently, and
> seems to (but not always) needs the pdf window to have focus and the
> mouse to be over it (but not exclusively).
>
> Maybe there is a better solution.
> My desktop is quite powerful: a modern (about 1yo, i7, 6-core,
> hyperthreaded, 64GB RAM), 4k display, 1080p USB webcam, SSD.
> Linux 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC
> 2020 x86_64 x86_64 x86_64 GNU/Linux
> ()
>
> The commands I ran are:
>
> 1/ testing the chroma-key with a chequered background
> gst-launch-1.0 \
>   videotestsrc pattern=checkers-8 ! video/x-raw,width=800,height=640 !
> mixer.sink_0 \
>   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
>   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> videoconvert ! autovideosink
> -> Very slow video, does not chroma-key on white!
> gst-launch recurrent message: "A lot of buffers are being dropped"
>
> 2/ testing the chroma-key (2nd attempt)
> gst-launch-1.0 \
>   videotestsrc pattern=checkers-8 !
> video/x-raw,width=800,height=640,framerate=1/1 ! mixer.sink_0 \
>   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
>   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> videoconvert ! autovideosink
> -> Same as above
>
> 3/ testing my video
> gst-launch-1.0 autovideosrc ! video/x-raw,width=800 ! videoconvert !
> autovideosink
> -> Showing my webcam is almost real-time, but has a bit of lag
> The same test at 1080p is very laggy
>
> 4/ testing a source application window (evince, showing a pdf file)
> gst-launch-1.0 \
>   ximagesrc xid=0x06800008 ! mixer.sink_0 \
>   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
>   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> videoconvert ! autovideosink
> -> Does not update the video in realtime (only updates rarely, at most
> every few seconds, when the application window has the focus or redraw
> events (possibly and/or mouse over)), does not allow me to resize the
> application window video stream.
>
>
> Hoping to find here some pointers to solve the issues highlighted
> above, most importantly performance (framerate).
>
> Next steps: a/ including the audio stream, b/ outputting to a video
> device so I can use this as a source for a video conference
> (zoom/skype/jitsi), c/ using an actual green screen.
>
> Thanks for any help, pointers to documentation or tutorials.
> Regards,
> Alain.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Desouza, Ederson]

Hi,

On Sun, 2020-04-05 at 01:20 +0100, Alain Culos wrote:
> Dear All,
>
> I just discovered GStreamer, so apologies if this is a basic error, but no matter how hard I try I can't find what is wrong with my set-up / pipeline.
>
> ===========
> Summary:
> When using ximagesrc, error messages appear to tell me buffers are lost and that there may be timestamping issues or that my computer is slow.

Have you tried using "use-damage=false" parameter on ximagesrc?

>
> I have a beefy desktop, 1yo i7, 6-core hyperthreaded, 64GB RAM ubuntu 18.04).
> GStreamer 1.14.5
>
> ===========
> In detail:
>
> The video shown with this command line is very laggy (it hardly uses any RAM or CPU // other note: my desktop easily copes with e.g. zoom meetings):
>
>     gst-launch-1.0 autovideosrc ! videoconvert ! autovideosink # shows no error message
>     gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! autovideosink # does exactly the same
>
>
> Preparing the ground for chroma-keying a green screen background, this example works the same way the first two do (laggy but good quality & no error message showing):
>     w=1280; h=800; x0=450; y0=200; gst-launch-1.0 \
>         v4l2src device=/dev/video0 \
>         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
>         ! alpha method=green angle=70 \
>         ! videoconvert \
>         ! autovideosink
>
> The background to replace the green screen will be taken from an application window.
> This causes issues with buffers dropped & it complains about timestamping issues.
>     gst-launch-1.0   ximagesrc xid=0x02a00006 ! videoconvert ! autovideosink
>
> Below is the output of the command above:
>     Setting pipeline to PAUSED ...
>     Pipeline is live and does not need PREROLL ...
>     Setting pipeline to PLAYING ...
>     New clock: GstSystemClock
>     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
>     Additional debug info:
>     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
>     There may be a timestamping problem, or this computer is too slow.
>
>
> Even if I impose a low frame rate, it outputs the same error messages as above, command line:
>     gst-launch-1.0   ximagesrc xid=0x02a00006 ! video/x-raw,framerate=5/1 ! videoconvert ! autovideosink
>
> Obviously, if I combine the two, performance gets worse, despite still not hitting the CPU or RAM, and the following command shows exactly the same messages as above:
>     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
>         v4l2src device=/dev/video0 \
>         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
>         ! alpha method=green angle=70 \
>         ! mixer.sink_1   \
>         \
>         ximagesrc xid=0x02a00006 ! videoscale \
>         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
>         \
>         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
>         ! videoconvert ! autovideosink
>
> Whether I put ximagesrc before v4l2src or after changes nothing.
>
> (next challenges: 1/ channelling this to /dev/video10; 2/ channeling the audio from the webcam (/dev/video0) to the virtual webcam (dev/video10))
>
> For now I  would most appreciate some help with making the above commands works without error/warning messages and good performance. Or in other words how do I resolve timestamping with ximagesrc?
>
> Best regards,
> Alain.
>
>
> On Wed, 1 Apr 2020, at 02:42, Alain Culos wrote:
> > Hi,
> >
> > I am a recent newcomer to gstreamer.
> > Trying to use `gst-launch-1.0` to chroma-key my webcam and insert a pdf
> > as background.
> >
> > My attempt is very imperfect:
> > 1/ I have not been able to find the right parameters to make the window
> > sizes match.
> > 2/ gst-launch occasionally complains it loses a lot of buffers.
> > 3/ the video output only updates seemingly randomly, infrequently, and
> > seems to (but not always) needs the pdf window to have focus and the
> > mouse to be over it (but not exclusively).
> >
> > Maybe there is a better solution.
> > My desktop is quite powerful: a modern (about 1yo, i7, 6-core,
> > hyperthreaded, 64GB RAM), 4k display, 1080p USB webcam, SSD.
> > Linux 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC
> > 2020 x86_64 x86_64 x86_64 GNU/Linux
> > ()
> >
> > The commands I ran are:
> >
> > 1/ testing the chroma-key with a chequered background
> > gst-launch-1.0 \
> >   videotestsrc pattern=checkers-8 ! video/x-raw,width=800,height=640 !
> > mixer.sink_0 \
> >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > videoconvert ! autovideosink
> > -> Very slow video, does not chroma-key on white!
> > gst-launch recurrent message: "A lot of buffers are being dropped"
> >
> > 2/ testing the chroma-key (2nd attempt)
> > gst-launch-1.0 \
> >   videotestsrc pattern=checkers-8 !
> > video/x-raw,width=800,height=640,framerate=1/1 ! mixer.sink_0 \
> >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > videoconvert ! autovideosink
> > -> Same as above
> >
> > 3/ testing my video
> > gst-launch-1.0 autovideosrc ! video/x-raw,width=800 ! videoconvert !
> > autovideosink
> > -> Showing my webcam is almost real-time, but has a bit of lag
> > The same test at 1080p is very laggy
> >
> > 4/ testing a source application window (evince, showing a pdf file)
> > gst-launch-1.0 \
> >   ximagesrc xid=0x06800008 ! mixer.sink_0 \
> >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > videoconvert ! autovideosink
> > -> Does not update the video in realtime (only updates rarely, at most
> > every few seconds, when the application window has the focus or redraw
> > events (possibly and/or mouse over)), does not allow me to resize the
> > application window video stream.
> >
> >
> > Hoping to find here some pointers to solve the issues highlighted
> > above, most importantly performance (framerate).
> >
> > Next steps: a/ including the audio stream, b/ outputting to a video
> > device so I can use this as a source for a video conference
> > (zoom/skype/jitsi), c/ using an actual green screen.
> >
> > Thanks for any help, pointers to documentation or tutorials.
> > Regards,
> > Alain.
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Alain Culos]

Thanks for this tip,

On Mon, 6 Apr 2020, at 17:26, Desouza, Ederson wrote:

> Hi,
>
> On Sun, 2020-04-05 at 01:20 +0100, Alain Culos wrote:
> > Dear All,
> >
> > I just discovered GStreamer, so apologies if this is a basic error, but no matter how hard I try I can't find what is wrong with my set-up / pipeline.
> >
> > ===========
> > Summary:
> > When using ximagesrc, error messages appear to tell me buffers are lost and that there may be timestamping issues or that my computer is slow.
>
> Have you tried using "use-damage=false" parameter on ximagesrc?

I had not, I now have.

    gst-launch-1.0   ximagesrc xid=0x02a00006 use-damage=false ! videoconvert ! autovideosink

The simple pipeline above now works flawlessly, with near real-time mirroring of the original window, great, now I need to make the more complex pipeline work better.

The mixer pipeline below, now updates about once per second (much better than without the "use-damage=false"), but still reports the timestamping error message.

    w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
        v4l2src device=/dev/video0 \
        ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
        ! alpha method=green angle=70 \
        ! mixer.sink_1   \
        \
        ximagesrc xid=0x02a00006 use-damage=false ! videoscale \
        ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
        \
        videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
        ! videoconvert ! autovideosink

Output:

    Setting pipeline to PAUSED ...
    Pipeline is live and does not need PREROLL ...
    Setting pipeline to PLAYING ...
    New clock: GstSystemClock
{
    WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
    Additional debug info:
    gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
    There may be a timestamping problem, or this computer is too slow.
} * repeats every second or so.

Thanks & regards,
Alain.

> > I have a beefy desktop, 1yo i7, 6-core hyperthreaded, 64GB RAM ubuntu 18.04).
> > GStreamer 1.14.5
> >
> > ===========
> > In detail:
> >
> > The video shown with this command line is very laggy (it hardly uses any RAM or CPU // other note: my desktop easily copes with e.g. zoom meetings):
> >
> >     gst-launch-1.0 autovideosrc ! videoconvert ! autovideosink # shows no error message
> >     gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! autovideosink # does exactly the same
> >
> >
> > Preparing the ground for chroma-keying a green screen background, this example works the same way the first two do (laggy but good quality & no error message showing):
> >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0 \
> >         v4l2src device=/dev/video0 \
> >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> >         ! alpha method=green angle=70 \
> >         ! videoconvert \
> >         ! autovideosink
> >
> > The background to replace the green screen will be taken from an application window.
> > This causes issues with buffers dropped & it complains about timestamping issues.
> >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! videoconvert ! autovideosink
> >
> > Below is the output of the command above:
> >     Setting pipeline to PAUSED ...
> >     Pipeline is live and does not need PREROLL ...
> >     Setting pipeline to PLAYING ...
> >     New clock: GstSystemClock
> >     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
> >     Additional debug info:
> >     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
> >     There may be a timestamping problem, or this computer is too slow.
> >
> >
> > Even if I impose a low frame rate, it outputs the same error messages as above, command line:
> >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! video/x-raw,framerate=5/1 ! videoconvert ! autovideosink
> >
> > Obviously, if I combine the two, performance gets worse, despite still not hitting the CPU or RAM, and the following command shows exactly the same messages as above:
> >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> >         v4l2src device=/dev/video0 \
> >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> >         ! alpha method=green angle=70 \
> >         ! mixer.sink_1   \
> >         \
> >         ximagesrc xid=0x02a00006 ! videoscale \
> >         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
> >         \
> >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> >         ! videoconvert ! autovideosink
> >
> > Whether I put ximagesrc before v4l2src or after changes nothing.
> >
> > (next challenges: 1/ channelling this to /dev/video10; 2/ channeling the audio from the webcam (/dev/video0) to the virtual webcam (dev/video10))
> >
> > For now I  would most appreciate some help with making the above commands works without error/warning messages and good performance. Or in other words how do I resolve timestamping with ximagesrc?
> >
> > Best regards,
> > Alain.
> >
> >
> > On Wed, 1 Apr 2020, at 02:42, Alain Culos wrote:
> > > Hi,
> > >
> > > I am a recent newcomer to gstreamer.
> > > Trying to use `gst-launch-1.0` to chroma-key my webcam and insert a pdf
> > > as background.
> > >
> > > My attempt is very imperfect:
> > > 1/ I have not been able to find the right parameters to make the window
> > > sizes match.
> > > 2/ gst-launch occasionally complains it loses a lot of buffers.
> > > 3/ the video output only updates seemingly randomly, infrequently, and
> > > seems to (but not always) needs the pdf window to have focus and the
> > > mouse to be over it (but not exclusively).
> > >
> > > Maybe there is a better solution.
> > > My desktop is quite powerful: a modern (about 1yo, i7, 6-core,
> > > hyperthreaded, 64GB RAM), 4k display, 1080p USB webcam, SSD.
> > > Linux 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC
> > > 2020 x86_64 x86_64 x86_64 GNU/Linux
> > > ()
> > >
> > > The commands I ran are:
> > >
> > > 1/ testing the chroma-key with a chequered background
> > > gst-launch-1.0 \
> > >   videotestsrc pattern=checkers-8 ! video/x-raw,width=800,height=640 !
> > > mixer.sink_0 \
> > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > videoconvert ! autovideosink
> > > -> Very slow video, does not chroma-key on white!
> > > gst-launch recurrent message: "A lot of buffers are being dropped"
> > >
> > > 2/ testing the chroma-key (2nd attempt)
> > > gst-launch-1.0 \
> > >   videotestsrc pattern=checkers-8 !
> > > video/x-raw,width=800,height=640,framerate=1/1 ! mixer.sink_0 \
> > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > videoconvert ! autovideosink
> > > -> Same as above
> > >
> > > 3/ testing my video
> > > gst-launch-1.0 autovideosrc ! video/x-raw,width=800 ! videoconvert !
> > > autovideosink
> > > -> Showing my webcam is almost real-time, but has a bit of lag
> > > The same test at 1080p is very laggy
> > >
> > > 4/ testing a source application window (evince, showing a pdf file)
> > > gst-launch-1.0 \
> > >   ximagesrc xid=0x06800008 ! mixer.sink_0 \
> > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > videoconvert ! autovideosink
> > > -> Does not update the video in realtime (only updates rarely, at most
> > > every few seconds, when the application window has the focus or redraw
> > > events (possibly and/or mouse over)), does not allow me to resize the
> > > application window video stream.
> > >
> > >
> > > Hoping to find here some pointers to solve the issues highlighted
> > > above, most importantly performance (framerate).
> > >
> > > Next steps: a/ including the audio stream, b/ outputting to a video
> > > device so I can use this as a source for a video conference
> > > (zoom/skype/jitsi), c/ using an actual green screen.
> > >
> > > Thanks for any help, pointers to documentation or tutorials.
> > > Regards,
> > > Alain.
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > [hidden email]
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > >
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Desouza, Ederson]

On Mon, 2020-04-06 at 20:39 +0100, Alain Culos wrote:

> Thanks for this tip,
>
> On Mon, 6 Apr 2020, at 17:26, Desouza, Ederson wrote:
> > Hi,
> >
> > On Sun, 2020-04-05 at 01:20 +0100, Alain Culos wrote:
> > > Dear All,
> > >
> > > I just discovered GStreamer, so apologies if this is a basic error, but no matter how hard I try I can't find what is wrong with my set-up / pipeline.
> > >
> > > ===========
> > > Summary:
> > > When using ximagesrc, error messages appear to tell me buffers are lost and that there may be timestamping issues or that my computer is slow.
> >
> > Have you tried using "use-damage=false" parameter on ximagesrc?
>
> I had not, I now have.
>
>     gst-launch-1.0   ximagesrc xid=0x02a00006 use-damage=false ! videoconvert ! autovideosink
>
> The simple pipeline above now works flawlessly, with near real-time mirroring of the original window, great, now I need to make the more complex pipeline work better.
>
> The mixer pipeline below, now updates about once per second (much better than without the "use-damage=false"), but still reports the timestamping error message.
>
>     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
>         v4l2src device=/dev/video0 \
>         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
>         ! alpha method=green angle=70 \
>         ! mixer.sink_1   \
>         \
>         ximagesrc xid=0x02a00006 use-damage=false ! videoscale \
>         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
>         \
>         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
>         ! videoconvert ! autovideosink
>

Try matching the framerate of both ximagesrc and v4l2src elements.
Maybe add some queues to the pipeline too?

> Output:
>
>     Setting pipeline to PAUSED ...
>     Pipeline is live and does not need PREROLL ...
>     Setting pipeline to PLAYING ...
>     New clock: GstSystemClock
> {
>     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
>     Additional debug info:
>     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
>     There may be a timestamping problem, or this computer is too slow.
> } * repeats every second or so.
>
> Thanks & regards,
> Alain.
>
>
> > > I have a beefy desktop, 1yo i7, 6-core hyperthreaded, 64GB RAM ubuntu 18.04).
> > > GStreamer 1.14.5
> > >
> > > ===========
> > > In detail:
> > >
> > > The video shown with this command line is very laggy (it hardly uses any RAM or CPU // other note: my desktop easily copes with e.g. zoom meetings):
> > >
> > >     gst-launch-1.0 autovideosrc ! videoconvert ! autovideosink # shows no error message
> > >     gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! autovideosink # does exactly the same
> > >
> > >
> > > Preparing the ground for chroma-keying a green screen background, this example works the same way the first two do (laggy but good quality & no error message showing):
> > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0 \
> > >         v4l2src device=/dev/video0 \
> > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > >         ! alpha method=green angle=70 \
> > >         ! videoconvert \
> > >         ! autovideosink
> > >
> > > The background to replace the green screen will be taken from an application window.
> > > This causes issues with buffers dropped & it complains about timestamping issues.
> > >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! videoconvert ! autovideosink
> > >
> > > Below is the output of the command above:
> > >     Setting pipeline to PAUSED ...
> > >     Pipeline is live and does not need PREROLL ...
> > >     Setting pipeline to PLAYING ...
> > >     New clock: GstSystemClock
> > >     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
> > >     Additional debug info:
> > >     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
> > >     There may be a timestamping problem, or this computer is too slow.
> > >
> > >
> > > Even if I impose a low frame rate, it outputs the same error messages as above, command line:
> > >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! video/x-raw,framerate=5/1 ! videoconvert ! autovideosink
> > >
> > > Obviously, if I combine the two, performance gets worse, despite still not hitting the CPU or RAM, and the following command shows exactly the same messages as above:
> > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > >         v4l2src device=/dev/video0 \
> > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > >         ! alpha method=green angle=70 \
> > >         ! mixer.sink_1   \
> > >         \
> > >         ximagesrc xid=0x02a00006 ! videoscale \
> > >         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
> > >         \
> > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > >         ! videoconvert ! autovideosink
> > >
> > > Whether I put ximagesrc before v4l2src or after changes nothing.
> > >
> > > (next challenges: 1/ channelling this to /dev/video10; 2/ channeling the audio from the webcam (/dev/video0) to the virtual webcam (dev/video10))
> > >
> > > For now I  would most appreciate some help with making the above commands works without error/warning messages and good performance. Or in other words how do I resolve timestamping with ximagesrc?
> > >
> > > Best regards,
> > > Alain.
> > >
> > >
> > > On Wed, 1 Apr 2020, at 02:42, Alain Culos wrote:
> > > > Hi,
> > > >
> > > > I am a recent newcomer to gstreamer.
> > > > Trying to use `gst-launch-1.0` to chroma-key my webcam and insert a pdf
> > > > as background.
> > > >
> > > > My attempt is very imperfect:
> > > > 1/ I have not been able to find the right parameters to make the window
> > > > sizes match.
> > > > 2/ gst-launch occasionally complains it loses a lot of buffers.
> > > > 3/ the video output only updates seemingly randomly, infrequently, and
> > > > seems to (but not always) needs the pdf window to have focus and the
> > > > mouse to be over it (but not exclusively).
> > > >
> > > > Maybe there is a better solution.
> > > > My desktop is quite powerful: a modern (about 1yo, i7, 6-core,
> > > > hyperthreaded, 64GB RAM), 4k display, 1080p USB webcam, SSD.
> > > > Linux 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC
> > > > 2020 x86_64 x86_64 x86_64 GNU/Linux
> > > > ()
> > > >
> > > > The commands I ran are:
> > > >
> > > > 1/ testing the chroma-key with a chequered background
> > > > gst-launch-1.0 \
> > > >   videotestsrc pattern=checkers-8 ! video/x-raw,width=800,height=640 !
> > > > mixer.sink_0 \
> > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > videoconvert ! autovideosink
> > > > -> Very slow video, does not chroma-key on white!
> > > > gst-launch recurrent message: "A lot of buffers are being dropped"
> > > >
> > > > 2/ testing the chroma-key (2nd attempt)
> > > > gst-launch-1.0 \
> > > >   videotestsrc pattern=checkers-8 !
> > > > video/x-raw,width=800,height=640,framerate=1/1 ! mixer.sink_0 \
> > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > videoconvert ! autovideosink
> > > > -> Same as above
> > > >
> > > > 3/ testing my video
> > > > gst-launch-1.0 autovideosrc ! video/x-raw,width=800 ! videoconvert !
> > > > autovideosink
> > > > -> Showing my webcam is almost real-time, but has a bit of lag
> > > > The same test at 1080p is very laggy
> > > >
> > > > 4/ testing a source application window (evince, showing a pdf file)
> > > > gst-launch-1.0 \
> > > >   ximagesrc xid=0x06800008 ! mixer.sink_0 \
> > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > videoconvert ! autovideosink
> > > > -> Does not update the video in realtime (only updates rarely, at most
> > > > every few seconds, when the application window has the focus or redraw
> > > > events (possibly and/or mouse over)), does not allow me to resize the
> > > > application window video stream.
> > > >
> > > >
> > > > Hoping to find here some pointers to solve the issues highlighted
> > > > above, most importantly performance (framerate).
> > > >
> > > > Next steps: a/ including the audio stream, b/ outputting to a video
> > > > device so I can use this as a source for a video conference
> > > > (zoom/skype/jitsi), c/ using an actual green screen.
> > > >
> > > > Thanks for any help, pointers to documentation or tutorials.
> > > > Regards,
> > > > Alain.
> > > > _______________________________________________
> > > > gstreamer-devel mailing list
> > > > [hidden email]
> > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > >
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > [hidden email]
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Alain Culos]

On Mon, 6 Apr 2020, at 21:44, Desouza, Ederson wrote:

> On Mon, 2020-04-06 at 20:39 +0100, Alain Culos wrote:
> > Thanks for this tip,
> >
> > On Mon, 6 Apr 2020, at 17:26, Desouza, Ederson wrote:
> > > Hi,
> > >
> > > On Sun, 2020-04-05 at 01:20 +0100, Alain Culos wrote:
> > > > Dear All,
> > > >
> > > > I just discovered GStreamer, so apologies if this is a basic error, but no matter how hard I try I can't find what is wrong with my set-up / pipeline.
> > > >
> > > > ===========
> > > > Summary:
> > > > When using ximagesrc, error messages appear to tell me buffers are lost and that there may be timestamping issues or that my computer is slow.
> > >
> > > Have you tried using "use-damage=false" parameter on ximagesrc?
> >
> > I had not, I now have.
> >
> >     gst-launch-1.0   ximagesrc xid=0x02a00006 use-damage=false ! videoconvert ! autovideosink
> >
> > The simple pipeline above now works flawlessly, with near real-time mirroring of the original window, great, now I need to make the more complex pipeline work better.
> >
> > The mixer pipeline below, now updates about once per second (much better than without the "use-damage=false"), but still reports the timestamping error message.
> >
> >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> >         v4l2src device=/dev/video0 \
> >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> >         ! alpha method=green angle=70 \
> >         ! mixer.sink_1   \
> >         \
> >         ximagesrc xid=0x02a00006 use-damage=false ! videoscale \
> >         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
> >         \
> >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> >         ! videoconvert ! autovideosink
> >
>
> Try matching the framerate of both ximagesrc and v4l2src elements.
> Maybe add some queues to the pipeline too?

Admission: I have no idea how to use queues, the effect or impact they have.
Could it be a timestamping issue due to the fact that I do not carry the audio channel?
I so, I have no idea how to bring it in, I think I need a demux + mux, but which, how and where?

Trying (see below, 2nd block, ximagesrc, 2nd line, framerate) and (2nd block, 1st line, queue) - one or the other or both - all yield the same result as previously:
    w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
        v4l2src device=/dev/video0 \
        ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
        ! alpha method=green angle=70 \
        ! mixer.sink_1   \
        \
        ximagesrc xid=0x02a00006 use-damage=false ! queue ! videoscale \
        ! video/x-raw,width=$w,height=$h,framerate=30/1 ! mixer.sink_0   \
        \
        videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
        ! videoconvert ! autovideosink

Also tried with (alone and also with the queue after ximagesrc):
        v4l2src device=/dev/video0 ! queue \

Same result again.

> > Output:
> >
> >     Setting pipeline to PAUSED ...
> >     Pipeline is live and does not need PREROLL ...
> >     Setting pipeline to PLAYING ...
> >     New clock: GstSystemClock
> > {
> >     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
> >     Additional debug info:
> >     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
> >     There may be a timestamping problem, or this computer is too slow.
> > } * repeats every second or so.
> >
> > Thanks & regards,
> > Alain.
> >
> >
> > > > I have a beefy desktop, 1yo i7, 6-core hyperthreaded, 64GB RAM ubuntu 18.04).
> > > > GStreamer 1.14.5
> > > >
> > > > ===========
> > > > In detail:
> > > >
> > > > The video shown with this command line is very laggy (it hardly uses any RAM or CPU // other note: my desktop easily copes with e.g. zoom meetings):
> > > >
> > > >     gst-launch-1.0 autovideosrc ! videoconvert ! autovideosink # shows no error message
> > > >     gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! autovideosink # does exactly the same
> > > >
> > > >
> > > > Preparing the ground for chroma-keying a green screen background, this example works the same way the first two do (laggy but good quality & no error message showing):
> > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0 \
> > > >         v4l2src device=/dev/video0 \
> > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > >         ! alpha method=green angle=70 \
> > > >         ! videoconvert \
> > > >         ! autovideosink
> > > >
> > > > The background to replace the green screen will be taken from an application window.
> > > > This causes issues with buffers dropped & it complains about timestamping issues.
> > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! videoconvert ! autovideosink
> > > >
> > > > Below is the output of the command above:
> > > >     Setting pipeline to PAUSED ...
> > > >     Pipeline is live and does not need PREROLL ...
> > > >     Setting pipeline to PLAYING ...
> > > >     New clock: GstSystemClock
> > > >     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
> > > >     Additional debug info:
> > > >     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
> > > >     There may be a timestamping problem, or this computer is too slow.
> > > >
> > > >
> > > > Even if I impose a low frame rate, it outputs the same error messages as above, command line:
> > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! video/x-raw,framerate=5/1 ! videoconvert ! autovideosink
> > > >
> > > > Obviously, if I combine the two, performance gets worse, despite still not hitting the CPU or RAM, and the following command shows exactly the same messages as above:
> > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > > >         v4l2src device=/dev/video0 \
> > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > >         ! alpha method=green angle=70 \
> > > >         ! mixer.sink_1   \
> > > >         \
> > > >         ximagesrc xid=0x02a00006 ! videoscale \
> > > >         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
> > > >         \
> > > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > > >         ! videoconvert ! autovideosink
> > > >
> > > > Whether I put ximagesrc before v4l2src or after changes nothing.
> > > >
> > > > (next challenges: 1/ channelling this to /dev/video10; 2/ channeling the audio from the webcam (/dev/video0) to the virtual webcam (dev/video10))
> > > >
> > > > For now I  would most appreciate some help with making the above commands works without error/warning messages and good performance. Or in other words how do I resolve timestamping with ximagesrc?
> > > >
> > > > Best regards,
> > > > Alain.
> > > >
> > > >
> > > > On Wed, 1 Apr 2020, at 02:42, Alain Culos wrote:
> > > > > Hi,
> > > > >
> > > > > I am a recent newcomer to gstreamer.
> > > > > Trying to use `gst-launch-1.0` to chroma-key my webcam and insert a pdf
> > > > > as background.
> > > > >
> > > > > My attempt is very imperfect:
> > > > > 1/ I have not been able to find the right parameters to make the window
> > > > > sizes match.
> > > > > 2/ gst-launch occasionally complains it loses a lot of buffers.
> > > > > 3/ the video output only updates seemingly randomly, infrequently, and
> > > > > seems to (but not always) needs the pdf window to have focus and the
> > > > > mouse to be over it (but not exclusively).
> > > > >
> > > > > Maybe there is a better solution.
> > > > > My desktop is quite powerful: a modern (about 1yo, i7, 6-core,
> > > > > hyperthreaded, 64GB RAM), 4k display, 1080p USB webcam, SSD.
> > > > > Linux 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC
> > > > > 2020 x86_64 x86_64 x86_64 GNU/Linux
> > > > > ()
> > > > >
> > > > > The commands I ran are:
> > > > >
> > > > > 1/ testing the chroma-key with a chequered background
> > > > > gst-launch-1.0 \
> > > > >   videotestsrc pattern=checkers-8 ! video/x-raw,width=800,height=640 !
> > > > > mixer.sink_0 \
> > > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > > videoconvert ! autovideosink
> > > > > -> Very slow video, does not chroma-key on white!
> > > > > gst-launch recurrent message: "A lot of buffers are being dropped"
> > > > >
> > > > > 2/ testing the chroma-key (2nd attempt)
> > > > > gst-launch-1.0 \
> > > > >   videotestsrc pattern=checkers-8 !
> > > > > video/x-raw,width=800,height=640,framerate=1/1 ! mixer.sink_0 \
> > > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > > videoconvert ! autovideosink
> > > > > -> Same as above
> > > > >
> > > > > 3/ testing my video
> > > > > gst-launch-1.0 autovideosrc ! video/x-raw,width=800 ! videoconvert !
> > > > > autovideosink
> > > > > -> Showing my webcam is almost real-time, but has a bit of lag
> > > > > The same test at 1080p is very laggy
> > > > >
> > > > > 4/ testing a source application window (evince, showing a pdf file)
> > > > > gst-launch-1.0 \
> > > > >   ximagesrc xid=0x06800008 ! mixer.sink_0 \
> > > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > > videoconvert ! autovideosink
> > > > > -> Does not update the video in realtime (only updates rarely, at most
> > > > > every few seconds, when the application window has the focus or redraw
> > > > > events (possibly and/or mouse over)), does not allow me to resize the
> > > > > application window video stream.
> > > > >
> > > > >
> > > > > Hoping to find here some pointers to solve the issues highlighted
> > > > > above, most importantly performance (framerate).
> > > > >
> > > > > Next steps: a/ including the audio stream, b/ outputting to a video
> > > > > device so I can use this as a source for a video conference
> > > > > (zoom/skype/jitsi), c/ using an actual green screen.
> > > > >
> > > > > Thanks for any help, pointers to documentation or tutorials.
> > > > > Regards,
> > > > > Alain.
> > > > > _______________________________________________
> > > > > gstreamer-devel mailing list
> > > > > [hidden email]
> > > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > > >
> > > > _______________________________________________
> > > > gstreamer-devel mailing list
> > > > [hidden email]
> > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > [hidden email]
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > >
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Desouza, Ederson]

On Mon, 2020-04-06 at 23:16 +0100, Alain Culos wrote:

> On Mon, 6 Apr 2020, at 21:44, Desouza, Ederson wrote:
> > On Mon, 2020-04-06 at 20:39 +0100, Alain Culos wrote:
> > > Thanks for this tip,
> > >
> > > On Mon, 6 Apr 2020, at 17:26, Desouza, Ederson wrote:
> > > > Hi,
> > > >
> > > > On Sun, 2020-04-05 at 01:20 +0100, Alain Culos wrote:
> > > > > Dear All,
> > > > >
> > > > > I just discovered GStreamer, so apologies if this is a basic error, but no matter how hard I try I can't find what is wrong with my set-up / pipeline.
> > > > >
> > > > > ===========
> > > > > Summary:
> > > > > When using ximagesrc, error messages appear to tell me buffers are lost and that there may be timestamping issues or that my computer is slow.
> > > >
> > > > Have you tried using "use-damage=false" parameter on ximagesrc?
> > >
> > > I had not, I now have.
> > >
> > >     gst-launch-1.0   ximagesrc xid=0x02a00006 use-damage=false ! videoconvert ! autovideosink
> > >
> > > The simple pipeline above now works flawlessly, with near real-time mirroring of the original window, great, now I need to make the more complex pipeline work better.
> > >
> > > The mixer pipeline below, now updates about once per second (much better than without the "use-damage=false"), but still reports the timestamping error message.
> > >
> > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > >         v4l2src device=/dev/video0 \
> > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > >         ! alpha method=green angle=70 \
> > >         ! mixer.sink_1   \
> > >         \
> > >         ximagesrc xid=0x02a00006 use-damage=false ! videoscale \
> > >         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
> > >         \
> > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > >         ! videoconvert ! autovideosink
> > >
> >
> > Try matching the framerate of both ximagesrc and v4l2src elements.
> > Maybe add some queues to the pipeline too?
>
> Admission: I have no idea how to use queues, the effect or impact they have.
> Could it be a timestamping issue due to the fact that I do not carry the audio channel?
> I so, I have no idea how to bring it in, I think I need a demux + mux, but which, how and where?

You can take a look at this link for a nice overview of queues:
https://gstreamer.freedesktop.org/documentation/tutorials/basic/multithreading-and-pad-availability.html?gi-language=c
I suggested it so that you may have more than on thread in your
pipeline. It _may_ help - or not =D

>
> Trying (see below, 2nd block, ximagesrc, 2nd line, framerate) and (2nd block, 1st line, queue) - one or the other or both - all yield the same result as previously:
>     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
>         v4l2src device=/dev/video0 \
>         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
>         ! alpha method=green angle=70 \
>         ! mixer.sink_1   \
>         \
>         ximagesrc xid=0x02a00006 use-damage=false ! queue ! videoscale \
>         ! video/x-raw,width=$w,height=$h,framerate=30/1 ! mixer.sink_0   \
>         \
>         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
>         ! videoconvert ! autovideosink
>

I see that you fixed the framerate for the ximagesrc - are you sure
that v4l2src has the same framerate? Anyway, another thing to try is to
lower both framerates. You may need to use "videorate" element to
reduce v4l2src framerate.

> Also tried with (alone and also with the queue after ximagesrc):
>         v4l2src device=/dev/video0 ! queue \
>
> Same result again.
>
> > > Output:
> > >
> > >     Setting pipeline to PAUSED ...
> > >     Pipeline is live and does not need PREROLL ...
> > >     Setting pipeline to PLAYING ...
> > >     New clock: GstSystemClock
> > > {
> > >     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
> > >     Additional debug info:
> > >     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
> > >     There may be a timestamping problem, or this computer is too slow.
> > > } * repeats every second or so.
> > >
> > > Thanks & regards,
> > > Alain.
> > >
> > >
> > > > > I have a beefy desktop, 1yo i7, 6-core hyperthreaded, 64GB RAM ubuntu 18.04).
> > > > > GStreamer 1.14.5
> > > > >
> > > > > ===========
> > > > > In detail:
> > > > >
> > > > > The video shown with this command line is very laggy (it hardly uses any RAM or CPU // other note: my desktop easily copes with e.g. zoom meetings):
> > > > >
> > > > >     gst-launch-1.0 autovideosrc ! videoconvert ! autovideosink # shows no error message
> > > > >     gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! autovideosink # does exactly the same
> > > > >
> > > > >
> > > > > Preparing the ground for chroma-keying a green screen background, this example works the same way the first two do (laggy but good quality & no error message showing):
> > > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0 \
> > > > >         v4l2src device=/dev/video0 \
> > > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > > >         ! alpha method=green angle=70 \
> > > > >         ! videoconvert \
> > > > >         ! autovideosink
> > > > >
> > > > > The background to replace the green screen will be taken from an application window.
> > > > > This causes issues with buffers dropped & it complains about timestamping issues.
> > > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! videoconvert ! autovideosink
> > > > >
> > > > > Below is the output of the command above:
> > > > >     Setting pipeline to PAUSED ...
> > > > >     Pipeline is live and does not need PREROLL ...
> > > > >     Setting pipeline to PLAYING ...
> > > > >     New clock: GstSystemClock
> > > > >     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
> > > > >     Additional debug info:
> > > > >     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
> > > > >     There may be a timestamping problem, or this computer is too slow.
> > > > >
> > > > >
> > > > > Even if I impose a low frame rate, it outputs the same error messages as above, command line:
> > > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! video/x-raw,framerate=5/1 ! videoconvert ! autovideosink
> > > > >
> > > > > Obviously, if I combine the two, performance gets worse, despite still not hitting the CPU or RAM, and the following command shows exactly the same messages as above:
> > > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > > > >         v4l2src device=/dev/video0 \
> > > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > > >         ! alpha method=green angle=70 \
> > > > >         ! mixer.sink_1   \
> > > > >         \
> > > > >         ximagesrc xid=0x02a00006 ! videoscale \
> > > > >         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
> > > > >         \
> > > > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > > > >         ! videoconvert ! autovideosink
> > > > >
> > > > > Whether I put ximagesrc before v4l2src or after changes nothing.
> > > > >
> > > > > (next challenges: 1/ channelling this to /dev/video10; 2/ channeling the audio from the webcam (/dev/video0) to the virtual webcam (dev/video10))
> > > > >
> > > > > For now I  would most appreciate some help with making the above commands works without error/warning messages and good performance. Or in other words how do I resolve timestamping with ximagesrc?
> > > > >
> > > > > Best regards,
> > > > > Alain.
> > > > >
> > > > >
> > > > > On Wed, 1 Apr 2020, at 02:42, Alain Culos wrote:
> > > > > > Hi,
> > > > > >
> > > > > > I am a recent newcomer to gstreamer.
> > > > > > Trying to use `gst-launch-1.0` to chroma-key my webcam and insert a pdf
> > > > > > as background.
> > > > > >
> > > > > > My attempt is very imperfect:
> > > > > > 1/ I have not been able to find the right parameters to make the window
> > > > > > sizes match.
> > > > > > 2/ gst-launch occasionally complains it loses a lot of buffers.
> > > > > > 3/ the video output only updates seemingly randomly, infrequently, and
> > > > > > seems to (but not always) needs the pdf window to have focus and the
> > > > > > mouse to be over it (but not exclusively).
> > > > > >
> > > > > > Maybe there is a better solution.
> > > > > > My desktop is quite powerful: a modern (about 1yo, i7, 6-core,
> > > > > > hyperthreaded, 64GB RAM), 4k display, 1080p USB webcam, SSD.
> > > > > > Linux 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC
> > > > > > 2020 x86_64 x86_64 x86_64 GNU/Linux
> > > > > > ()
> > > > > >
> > > > > > The commands I ran are:
> > > > > >
> > > > > > 1/ testing the chroma-key with a chequered background
> > > > > > gst-launch-1.0 \
> > > > > >   videotestsrc pattern=checkers-8 ! video/x-raw,width=800,height=640 !
> > > > > > mixer.sink_0 \
> > > > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > > > videoconvert ! autovideosink
> > > > > > -> Very slow video, does not chroma-key on white!
> > > > > > gst-launch recurrent message: "A lot of buffers are being dropped"
> > > > > >
> > > > > > 2/ testing the chroma-key (2nd attempt)
> > > > > > gst-launch-1.0 \
> > > > > >   videotestsrc pattern=checkers-8 !
> > > > > > video/x-raw,width=800,height=640,framerate=1/1 ! mixer.sink_0 \
> > > > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > > > videoconvert ! autovideosink
> > > > > > -> Same as above
> > > > > >
> > > > > > 3/ testing my video
> > > > > > gst-launch-1.0 autovideosrc ! video/x-raw,width=800 ! videoconvert !
> > > > > > autovideosink
> > > > > > -> Showing my webcam is almost real-time, but has a bit of lag
> > > > > > The same test at 1080p is very laggy
> > > > > >
> > > > > > 4/ testing a source application window (evince, showing a pdf file)
> > > > > > gst-launch-1.0 \
> > > > > >   ximagesrc xid=0x06800008 ! mixer.sink_0 \
> > > > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > > > videoconvert ! autovideosink
> > > > > > -> Does not update the video in realtime (only updates rarely, at most
> > > > > > every few seconds, when the application window has the focus or redraw
> > > > > > events (possibly and/or mouse over)), does not allow me to resize the
> > > > > > application window video stream.
> > > > > >
> > > > > >
> > > > > > Hoping to find here some pointers to solve the issues highlighted
> > > > > > above, most importantly performance (framerate).
> > > > > >
> > > > > > Next steps: a/ including the audio stream, b/ outputting to a video
> > > > > > device so I can use this as a source for a video conference
> > > > > > (zoom/skype/jitsi), c/ using an actual green screen.
> > > > > >
> > > > > > Thanks for any help, pointers to documentation or tutorials.
> > > > > > Regards,
> > > > > > Alain.
> > > > > > _______________________________________________
> > > > > > gstreamer-devel mailing list
> > > > > > [hidden email]
> > > > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > > > >
> > > > > _______________________________________________
> > > > > gstreamer-devel mailing list
> > > > > [hidden email]
> > > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > > _______________________________________________
> > > > gstreamer-devel mailing list
> > > > [hidden email]
> > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > >
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > [hidden email]
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Alain Culos]

On Tue, 7 Apr 2020, at 00:11, Desouza, Ederson wrote:

> On Mon, 2020-04-06 at 23:16 +0100, Alain Culos wrote:
> > On Mon, 6 Apr 2020, at 21:44, Desouza, Ederson wrote:
> > > On Mon, 2020-04-06 at 20:39 +0100, Alain Culos wrote:
> > > > Thanks for this tip,
> > > >
> > > > On Mon, 6 Apr 2020, at 17:26, Desouza, Ederson wrote:
> > > > > Hi,
> > > > >
> > > > > On Sun, 2020-04-05 at 01:20 +0100, Alain Culos wrote:
> > > > > > Dear All,
> > > > > >
> > > > > > I just discovered GStreamer, so apologies if this is a basic error, but no matter how hard I try I can't find what is wrong with my set-up / pipeline.
> > > > > >
> > > > > > ===========
> > > > > > Summary:
> > > > > > When using ximagesrc, error messages appear to tell me buffers are lost and that there may be timestamping issues or that my computer is slow.
> > > > >
> > > > > Have you tried using "use-damage=false" parameter on ximagesrc?
> > > >
> > > > I had not, I now have.
> > > >
> > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 use-damage=false ! videoconvert ! autovideosink
> > > >
> > > > The simple pipeline above now works flawlessly, with near real-time mirroring of the original window, great, now I need to make the more complex pipeline work better.
> > > >
> > > > The mixer pipeline below, now updates about once per second (much better than without the "use-damage=false"), but still reports the timestamping error message.
> > > >
> > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > > >         v4l2src device=/dev/video0 \
> > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > >         ! alpha method=green angle=70 \
> > > >         ! mixer.sink_1   \
> > > >         \
> > > >         ximagesrc xid=0x02a00006 use-damage=false ! videoscale \
> > > >         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
> > > >         \
> > > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > > >         ! videoconvert ! autovideosink
> > > >
> > >
> > > Try matching the framerate of both ximagesrc and v4l2src elements.
> > > Maybe add some queues to the pipeline too?
> >
> > Admission: I have no idea how to use queues, the effect or impact they have.
> > Could it be a timestamping issue due to the fact that I do not carry the audio channel?
> > I so, I have no idea how to bring it in, I think I need a demux + mux, but which, how and where?
>
> You can take a look at this link for a nice overview of queues:
> https://gstreamer.freedesktop.org/documentation/tutorials/basic/multithreading-and-pad-availability.html?gi-language=c
> I suggested it so that you may have more than on thread in your
> pipeline. It _may_ help - or not =D

Great resource, clear.
I moved both queues to the end of my pipelines, just before the input into the mixer sinks, it did not change anything.

> > Trying (see below, 2nd block, ximagesrc, 2nd line, framerate) and (2nd block, 1st line, queue) - one or the other or both - all yield the same result as previously:
> >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> >         v4l2src device=/dev/video0 \
> >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> >         ! alpha method=green angle=70 \
> >         ! mixer.sink_1   \
> >         \
> >         ximagesrc xid=0x02a00006 use-damage=false ! queue ! videoscale \
> >         ! video/x-raw,width=$w,height=$h,framerate=30/1 ! mixer.sink_0   \
> >         \
> >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> >         ! videoconvert ! autovideosink
> >
>
> I see that you fixed the framerate for the ximagesrc - are you sure
> that v4l2src has the same framerate? Anyway, another thing to try is to
> lower both framerates. You may need to use "videorate" element to
> reduce v4l2src framerate.

You were right to question the framerate.

I kept the queues and added "! videorate max-rate=5" (if I use rate instead of max-rate, it freezes the video output) after both ximagesrc and v42src, it made things worse (slower than 1 refresh per second) + same error messages.

> > Also tried with (alone and also with the queue after ximagesrc):
> >         v4l2src device=/dev/video0 ! queue \
> >
> > Same result again.
> >
> > > > Output:
> > > >
> > > >     Setting pipeline to PAUSED ...
> > > >     Pipeline is live and does not need PREROLL ...
> > > >     Setting pipeline to PLAYING ...
> > > >     New clock: GstSystemClock
> > > > {
> > > >     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
> > > >     Additional debug info:
> > > >     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
> > > >     There may be a timestamping problem, or this computer is too slow.
> > > > } * repeats every second or so.
> > > >
> > > > Thanks & regards,
> > > > Alain.
> > > >
> > > >
> > > > > > I have a beefy desktop, 1yo i7, 6-core hyperthreaded, 64GB RAM ubuntu 18.04).
> > > > > > GStreamer 1.14.5
> > > > > >
> > > > > > ===========
> > > > > > In detail:
> > > > > >
> > > > > > The video shown with this command line is very laggy (it hardly uses any RAM or CPU // other note: my desktop easily copes with e.g. zoom meetings):
> > > > > >
> > > > > >     gst-launch-1.0 autovideosrc ! videoconvert ! autovideosink # shows no error message
> > > > > >     gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! autovideosink # does exactly the same
> > > > > >
> > > > > >
> > > > > > Preparing the ground for chroma-keying a green screen background, this example works the same way the first two do (laggy but good quality & no error message showing):
> > > > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0 \
> > > > > >         v4l2src device=/dev/video0 \
> > > > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > > > >         ! alpha method=green angle=70 \
> > > > > >         ! videoconvert \
> > > > > >         ! autovideosink
> > > > > >
> > > > > > The background to replace the green screen will be taken from an application window.
> > > > > > This causes issues with buffers dropped & it complains about timestamping issues.
> > > > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! videoconvert ! autovideosink
> > > > > >
> > > > > > Below is the output of the command above:
> > > > > >     Setting pipeline to PAUSED ...
> > > > > >     Pipeline is live and does not need PREROLL ...
> > > > > >     Setting pipeline to PLAYING ...
> > > > > >     New clock: GstSystemClock
> > > > > >     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
> > > > > >     Additional debug info:
> > > > > >     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
> > > > > >     There may be a timestamping problem, or this computer is too slow.
> > > > > >
> > > > > >
> > > > > > Even if I impose a low frame rate, it outputs the same error messages as above, command line:
> > > > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! video/x-raw,framerate=5/1 ! videoconvert ! autovideosink
> > > > > >
> > > > > > Obviously, if I combine the two, performance gets worse, despite still not hitting the CPU or RAM, and the following command shows exactly the same messages as above:
> > > > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > > > > >         v4l2src device=/dev/video0 \
> > > > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > > > >         ! alpha method=green angle=70 \
> > > > > >         ! mixer.sink_1   \
> > > > > >         \
> > > > > >         ximagesrc xid=0x02a00006 ! videoscale \
> > > > > >         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
> > > > > >         \
> > > > > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > > > > >         ! videoconvert ! autovideosink
> > > > > >
> > > > > > Whether I put ximagesrc before v4l2src or after changes nothing.
> > > > > >
> > > > > > (next challenges: 1/ channelling this to /dev/video10; 2/ channeling the audio from the webcam (/dev/video0) to the virtual webcam (dev/video10))
> > > > > >
> > > > > > For now I  would most appreciate some help with making the above commands works without error/warning messages and good performance. Or in other words how do I resolve timestamping with ximagesrc?
> > > > > >
> > > > > > Best regards,
> > > > > > Alain.
> > > > > >
> > > > > >
> > > > > > On Wed, 1 Apr 2020, at 02:42, Alain Culos wrote:
> > > > > > > Hi,
> > > > > > >
> > > > > > > I am a recent newcomer to gstreamer.
> > > > > > > Trying to use `gst-launch-1.0` to chroma-key my webcam and insert a pdf
> > > > > > > as background.
> > > > > > >
> > > > > > > My attempt is very imperfect:
> > > > > > > 1/ I have not been able to find the right parameters to make the window
> > > > > > > sizes match.
> > > > > > > 2/ gst-launch occasionally complains it loses a lot of buffers.
> > > > > > > 3/ the video output only updates seemingly randomly, infrequently, and
> > > > > > > seems to (but not always) needs the pdf window to have focus and the
> > > > > > > mouse to be over it (but not exclusively).
> > > > > > >
> > > > > > > Maybe there is a better solution.
> > > > > > > My desktop is quite powerful: a modern (about 1yo, i7, 6-core,
> > > > > > > hyperthreaded, 64GB RAM), 4k display, 1080p USB webcam, SSD.
> > > > > > > Linux 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC
> > > > > > > 2020 x86_64 x86_64 x86_64 GNU/Linux
> > > > > > > ()
> > > > > > >
> > > > > > > The commands I ran are:
> > > > > > >
> > > > > > > 1/ testing the chroma-key with a chequered background
> > > > > > > gst-launch-1.0 \
> > > > > > >   videotestsrc pattern=checkers-8 ! video/x-raw,width=800,height=640 !
> > > > > > > mixer.sink_0 \
> > > > > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > > > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > > > > videoconvert ! autovideosink
> > > > > > > -> Very slow video, does not chroma-key on white!
> > > > > > > gst-launch recurrent message: "A lot of buffers are being dropped"
> > > > > > >
> > > > > > > 2/ testing the chroma-key (2nd attempt)
> > > > > > > gst-launch-1.0 \
> > > > > > >   videotestsrc pattern=checkers-8 !
> > > > > > > video/x-raw,width=800,height=640,framerate=1/1 ! mixer.sink_0 \
> > > > > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > > > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > > > > videoconvert ! autovideosink
> > > > > > > -> Same as above
> > > > > > >
> > > > > > > 3/ testing my video
> > > > > > > gst-launch-1.0 autovideosrc ! video/x-raw,width=800 ! videoconvert !
> > > > > > > autovideosink
> > > > > > > -> Showing my webcam is almost real-time, but has a bit of lag
> > > > > > > The same test at 1080p is very laggy
> > > > > > >
> > > > > > > 4/ testing a source application window (evince, showing a pdf file)
> > > > > > > gst-launch-1.0 \
> > > > > > >   ximagesrc xid=0x06800008 ! mixer.sink_0 \
> > > > > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > > > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > > > > videoconvert ! autovideosink
> > > > > > > -> Does not update the video in realtime (only updates rarely, at most
> > > > > > > every few seconds, when the application window has the focus or redraw
> > > > > > > events (possibly and/or mouse over)), does not allow me to resize the
> > > > > > > application window video stream.
> > > > > > >
> > > > > > >
> > > > > > > Hoping to find here some pointers to solve the issues highlighted
> > > > > > > above, most importantly performance (framerate).
> > > > > > >
> > > > > > > Next steps: a/ including the audio stream, b/ outputting to a video
> > > > > > > device so I can use this as a source for a video conference
> > > > > > > (zoom/skype/jitsi), c/ using an actual green screen.
> > > > > > >
> > > > > > > Thanks for any help, pointers to documentation or tutorials.
> > > > > > > Regards,
> > > > > > > Alain.
> > > > > > > _______________________________________________
> > > > > > > gstreamer-devel mailing list
> > > > > > > [hidden email]
> > > > > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > > > > >
> > > > > > _______________________________________________
> > > > > > gstreamer-devel mailing list
> > > > > > [hidden email]
> > > > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > > > _______________________________________________
> > > > > gstreamer-devel mailing list
> > > > > [hidden email]
> > > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > > >
> > > > _______________________________________________
> > > > gstreamer-devel mailing list
> > > > [hidden email]
> > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > [hidden email]
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > >
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Desouza, Ederson]

On Tue, 2020-04-07 at 00:52 +0100, Alain Culos wrote:

> On Tue, 7 Apr 2020, at 00:11, Desouza, Ederson wrote:
> > On Mon, 2020-04-06 at 23:16 +0100, Alain Culos wrote:
> > > On Mon, 6 Apr 2020, at 21:44, Desouza, Ederson wrote:
> > > > On Mon, 2020-04-06 at 20:39 +0100, Alain Culos wrote:
> > > > > Thanks for this tip,
> > > > >
> > > > > On Mon, 6 Apr 2020, at 17:26, Desouza, Ederson wrote:
> > > > > > Hi,
> > > > > >
> > > > > > On Sun, 2020-04-05 at 01:20 +0100, Alain Culos wrote:
> > > > > > > Dear All,
> > > > > > >
> > > > > > > I just discovered GStreamer, so apologies if this is a basic error, but no matter how hard I try I can't find what is wrong with my set-up / pipeline.
> > > > > > >
> > > > > > > ===========
> > > > > > > Summary:
> > > > > > > When using ximagesrc, error messages appear to tell me buffers are lost and that there may be timestamping issues or that my computer is slow.
> > > > > >
> > > > > > Have you tried using "use-damage=false" parameter on ximagesrc?
> > > > >
> > > > > I had not, I now have.
> > > > >
> > > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 use-damage=false ! videoconvert ! autovideosink
> > > > >
> > > > > The simple pipeline above now works flawlessly, with near real-time mirroring of the original window, great, now I need to make the more complex pipeline work better.
> > > > >
> > > > > The mixer pipeline below, now updates about once per second (much better than without the "use-damage=false"), but still reports the timestamping error message.
> > > > >
> > > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > > > >         v4l2src device=/dev/video0 \
> > > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > > >         ! alpha method=green angle=70 \
> > > > >         ! mixer.sink_1   \
> > > > >         \
> > > > >         ximagesrc xid=0x02a00006 use-damage=false ! videoscale \
> > > > >         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
> > > > >         \
> > > > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > > > >         ! videoconvert ! autovideosink
> > > > >
> > > >
> > > > Try matching the framerate of both ximagesrc and v4l2src elements.
> > > > Maybe add some queues to the pipeline too?
> > >
> > > Admission: I have no idea how to use queues, the effect or impact they have.
> > > Could it be a timestamping issue due to the fact that I do not carry the audio channel?
> > > I so, I have no idea how to bring it in, I think I need a demux + mux, but which, how and where?
> >
> > You can take a look at this link for a nice overview of queues:
> > https://gstreamer.freedesktop.org/documentation/tutorials/basic/multithreading-and-pad-availability.html?gi-language=c
> > I suggested it so that you may have more than on thread in your
> > pipeline. It _may_ help - or not =D
>
> Great resource, clear.
> I moved both queues to the end of my pipelines, just before the input into the mixer sinks, it did not change anything.
>
> > > Trying (see below, 2nd block, ximagesrc, 2nd line, framerate) and (2nd block, 1st line, queue) - one or the other or both - all yield the same result as previously:
> > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > >         v4l2src device=/dev/video0 \
> > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > >         ! alpha method=green angle=70 \
> > >         ! mixer.sink_1   \
> > >         \
> > >         ximagesrc xid=0x02a00006 use-damage=false ! queue ! videoscale \
> > >         ! video/x-raw,width=$w,height=$h,framerate=30/1 ! mixer.sink_0   \
> > >         \
> > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > >         ! videoconvert ! autovideosink
> > >
> >
> > I see that you fixed the framerate for the ximagesrc - are you sure
> > that v4l2src has the same framerate? Anyway, another thing to try is to
> > lower both framerates. You may need to use "videorate" element to
> > reduce v4l2src framerate.
>
> You were right to question the framerate.
>
> I kept the queues and added "! videorate max-rate=5" (if I use rate instead of max-rate, it freezes the video output) after both ximagesrc and v42src, it made things worse (slower than 1 refresh per second) + same error messages.

I imagined something like:

 ... ! videorate ! video/x-raw,framerate=10/1 ! ...

For the v4l2src branch. I think that you don't need the videorate on
the ximagesrc branch, only the caps.

>
>
> > > Also tried with (alone and also with the queue after ximagesrc):
> > >         v4l2src device=/dev/video0 ! queue \
> > >
> > > Same result again.
> > >
> > > > > Output:
> > > > >
> > > > >     Setting pipeline to PAUSED ...
> > > > >     Pipeline is live and does not need PREROLL ...
> > > > >     Setting pipeline to PLAYING ...
> > > > >     New clock: GstSystemClock
> > > > > {
> > > > >     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
> > > > >     Additional debug info:
> > > > >     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
> > > > >     There may be a timestamping problem, or this computer is too slow.
> > > > > } * repeats every second or so.
> > > > >
> > > > > Thanks & regards,
> > > > > Alain.
> > > > >
> > > > >
> > > > > > > I have a beefy desktop, 1yo i7, 6-core hyperthreaded, 64GB RAM ubuntu 18.04).
> > > > > > > GStreamer 1.14.5
> > > > > > >
> > > > > > > ===========
> > > > > > > In detail:
> > > > > > >
> > > > > > > The video shown with this command line is very laggy (it hardly uses any RAM or CPU // other note: my desktop easily copes with e.g. zoom meetings):
> > > > > > >
> > > > > > >     gst-launch-1.0 autovideosrc ! videoconvert ! autovideosink # shows no error message
> > > > > > >     gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! autovideosink # does exactly the same
> > > > > > >
> > > > > > >
> > > > > > > Preparing the ground for chroma-keying a green screen background, this example works the same way the first two do (laggy but good quality & no error message showing):
> > > > > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0 \
> > > > > > >         v4l2src device=/dev/video0 \
> > > > > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > > > > >         ! alpha method=green angle=70 \
> > > > > > >         ! videoconvert \
> > > > > > >         ! autovideosink
> > > > > > >
> > > > > > > The background to replace the green screen will be taken from an application window.
> > > > > > > This causes issues with buffers dropped & it complains about timestamping issues.
> > > > > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! videoconvert ! autovideosink
> > > > > > >
> > > > > > > Below is the output of the command above:
> > > > > > >     Setting pipeline to PAUSED ...
> > > > > > >     Pipeline is live and does not need PREROLL ...
> > > > > > >     Setting pipeline to PLAYING ...
> > > > > > >     New clock: GstSystemClock
> > > > > > >     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
> > > > > > >     Additional debug info:
> > > > > > >     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
> > > > > > >     There may be a timestamping problem, or this computer is too slow.
> > > > > > >
> > > > > > >
> > > > > > > Even if I impose a low frame rate, it outputs the same error messages as above, command line:
> > > > > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! video/x-raw,framerate=5/1 ! videoconvert ! autovideosink
> > > > > > >
> > > > > > > Obviously, if I combine the two, performance gets worse, despite still not hitting the CPU or RAM, and the following command shows exactly the same messages as above:
> > > > > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > > > > > >         v4l2src device=/dev/video0 \
> > > > > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > > > > >         ! alpha method=green angle=70 \
> > > > > > >         ! mixer.sink_1   \
> > > > > > >         \
> > > > > > >         ximagesrc xid=0x02a00006 ! videoscale \
> > > > > > >         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
> > > > > > >         \
> > > > > > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > > > > > >         ! videoconvert ! autovideosink
> > > > > > >
> > > > > > > Whether I put ximagesrc before v4l2src or after changes nothing.
> > > > > > >
> > > > > > > (next challenges: 1/ channelling this to /dev/video10; 2/ channeling the audio from the webcam (/dev/video0) to the virtual webcam (dev/video10))
> > > > > > >
> > > > > > > For now I  would most appreciate some help with making the above commands works without error/warning messages and good performance. Or in other words how do I resolve timestamping with ximagesrc?
> > > > > > >
> > > > > > > Best regards,
> > > > > > > Alain.
> > > > > > >
> > > > > > >
> > > > > > > On Wed, 1 Apr 2020, at 02:42, Alain Culos wrote:
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I am a recent newcomer to gstreamer.
> > > > > > > > Trying to use `gst-launch-1.0` to chroma-key my webcam and insert a pdf
> > > > > > > > as background.
> > > > > > > >
> > > > > > > > My attempt is very imperfect:
> > > > > > > > 1/ I have not been able to find the right parameters to make the window
> > > > > > > > sizes match.
> > > > > > > > 2/ gst-launch occasionally complains it loses a lot of buffers.
> > > > > > > > 3/ the video output only updates seemingly randomly, infrequently, and
> > > > > > > > seems to (but not always) needs the pdf window to have focus and the
> > > > > > > > mouse to be over it (but not exclusively).
> > > > > > > >
> > > > > > > > Maybe there is a better solution.
> > > > > > > > My desktop is quite powerful: a modern (about 1yo, i7, 6-core,
> > > > > > > > hyperthreaded, 64GB RAM), 4k display, 1080p USB webcam, SSD.
> > > > > > > > Linux 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC
> > > > > > > > 2020 x86_64 x86_64 x86_64 GNU/Linux
> > > > > > > > ()
> > > > > > > >
> > > > > > > > The commands I ran are:
> > > > > > > >
> > > > > > > > 1/ testing the chroma-key with a chequered background
> > > > > > > > gst-launch-1.0 \
> > > > > > > >   videotestsrc pattern=checkers-8 ! video/x-raw,width=800,height=640 !
> > > > > > > > mixer.sink_0 \
> > > > > > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > > > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > > > > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > > > > > videoconvert ! autovideosink
> > > > > > > > -> Very slow video, does not chroma-key on white!
> > > > > > > > gst-launch recurrent message: "A lot of buffers are being dropped"
> > > > > > > >
> > > > > > > > 2/ testing the chroma-key (2nd attempt)
> > > > > > > > gst-launch-1.0 \
> > > > > > > >   videotestsrc pattern=checkers-8 !
> > > > > > > > video/x-raw,width=800,height=640,framerate=1/1 ! mixer.sink_0 \
> > > > > > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > > > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > > > > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > > > > > videoconvert ! autovideosink
> > > > > > > > -> Same as above
> > > > > > > >
> > > > > > > > 3/ testing my video
> > > > > > > > gst-launch-1.0 autovideosrc ! video/x-raw,width=800 ! videoconvert !
> > > > > > > > autovideosink
> > > > > > > > -> Showing my webcam is almost real-time, but has a bit of lag
> > > > > > > > The same test at 1080p is very laggy
> > > > > > > >
> > > > > > > > 4/ testing a source application window (evince, showing a pdf file)
> > > > > > > > gst-launch-1.0 \
> > > > > > > >   ximagesrc xid=0x06800008 ! mixer.sink_0 \
> > > > > > > >   autovideosrc ! video/x-raw,width=800 ! alpha method=custom
> > > > > > > > target-r=255 target-g=255 target-b=255 ! mixer.sink_1 \
> > > > > > > >   videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 !
> > > > > > > > videoconvert ! autovideosink
> > > > > > > > -> Does not update the video in realtime (only updates rarely, at most
> > > > > > > > every few seconds, when the application window has the focus or redraw
> > > > > > > > events (possibly and/or mouse over)), does not allow me to resize the
> > > > > > > > application window video stream.
> > > > > > > >
> > > > > > > >
> > > > > > > > Hoping to find here some pointers to solve the issues highlighted
> > > > > > > > above, most importantly performance (framerate).
> > > > > > > >
> > > > > > > > Next steps: a/ including the audio stream, b/ outputting to a video
> > > > > > > > device so I can use this as a source for a video conference
> > > > > > > > (zoom/skype/jitsi), c/ using an actual green screen.
> > > > > > > >
> > > > > > > > Thanks for any help, pointers to documentation or tutorials.
> > > > > > > > Regards,
> > > > > > > > Alain.
> > > > > > > > _______________________________________________
> > > > > > > > gstreamer-devel mailing list
> > > > > > > > [hidden email]
> > > > > > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > > > > > >
> > > > > > > _______________________________________________
> > > > > > > gstreamer-devel mailing list
> > > > > > > [hidden email]
> > > > > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > > > > _______________________________________________
> > > > > > gstreamer-devel mailing list
> > > > > > [hidden email]
> > > > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > > > >
> > > > > _______________________________________________
> > > > > gstreamer-devel mailing list
> > > > > [hidden email]
> > > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > > _______________________________________________
> > > > gstreamer-devel mailing list
> > > > [hidden email]
> > > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > > >
> > > _______________________________________________
> > > gstreamer-devel mailing list
> > > [hidden email]
> > > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> >
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Alain Culos]

On Tue, 7 Apr 2020, at 01:24, Desouza, Ederson wrote:

> On Tue, 2020-04-07 at 00:52 +0100, Alain Culos wrote:
> > On Tue, 7 Apr 2020, at 00:11, Desouza, Ederson wrote:
> > > On Mon, 2020-04-06 at 23:16 +0100, Alain Culos wrote:
> > > > On Mon, 6 Apr 2020, at 21:44, Desouza, Ederson wrote:
> > > > > On Mon, 2020-04-06 at 20:39 +0100, Alain Culos wrote:
> > > > > > On Mon, 6 Apr 2020, at 17:26, Desouza, Ederson wrote:
> > > > > > > On Sun, 2020-04-05 at 01:20 +0100, Alain Culos wrote:
> > > > > > > > Summary:
> > > > > > > > When using ximagesrc, error messages appear to tell me buffers are lost and that there may be timestamping issues or that my computer is slow.
> > > > > > > Have you tried using "use-damage=false" parameter on ximagesrc?
> > > > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 use-damage=false ! videoconvert ! autovideosink
> > > > > > The simple pipeline above now works flawlessly, with near real-time mirroring of the original window, great, now I need to make the more complex pipeline work better.
> > > > > >
> > > > > > The mixer pipeline below, now updates about once per second (much better than without the "use-damage=false"), but still reports the timestamping error message.
> > > > > >
> > > > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > > > > >         v4l2src device=/dev/video0 \
> > > > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > > > >         ! alpha method=green angle=70 \
> > > > > >         ! mixer.sink_1   \
> > > > > >         \
> > > > > >         ximagesrc xid=0x02a00006 use-damage=false ! videoscale \
> > > > > >         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
> > > > > >         \
> > > > > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > > > > >         ! videoconvert ! autovideosink
> > > > >
> > > > > Try matching the framerate of both ximagesrc and v4l2src elements.
> > > > > Maybe add some queues to the pipeline too?
> > > You can take a look at this link for a nice overview of queues:
> > > https://gstreamer.freedesktop.org/documentation/tutorials/basic/multithreading-and-pad-availability.html?gi-language=c
> > > I suggested it so that you may have more than on thread in your
> > > pipeline. It _may_ help - or not =D
> > Great resource, clear.
> > I moved both queues to the end of my pipelines, just before the input into the mixer sinks, it did not change anything.
> >
> > > > Trying (see below, 2nd block, ximagesrc, 2nd line, framerate) and (2nd block, 1st line, queue) - one or the other or both - all yield the same result as previously:
> > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > > >         v4l2src device=/dev/video0 \
> > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > >         ! alpha method=green angle=70 \
> > > >         ! mixer.sink_1   \
> > > >         \
> > > >         ximagesrc xid=0x02a00006 use-damage=false ! queue ! videoscale \
> > > >         ! video/x-raw,width=$w,height=$h,framerate=30/1 ! mixer.sink_0   \
> > > >         \
> > > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > > >         ! videoconvert ! autovideosink
> > > >
> > >
> > > I see that you fixed the framerate for the ximagesrc - are you sure
> > > that v4l2src has the same framerate? Anyway, another thing to try is to
> > > lower both framerates. You may need to use "videorate" element to
> > > reduce v4l2src framerate.
> > I kept the queues and added "! videorate max-rate=5" (if I use rate instead of max-rate, it freezes the video output) after both ximagesrc and v42src, it made things worse (slower than 1 refresh per second) + same error messages.
> I imagined something like:
>  ... ! videorate ! video/x-raw,framerate=10/1 ! ...
>
> For the v4l2src branch. I think that you don't need the videorate on
> the ximagesrc branch, only the caps.

Ok, I removed the videorate from the ximagesrc branch, using caps to set the framerate to 5/1 or 10/1.
I tried the "! videorate ! video/x-raw,framerate=5/1" (or 10/1) before or after (or also without) the videocrop element.
In all cases I still get the error messages and warnings, as well as a very laggy video stream.
With framerate at 10/1 it also scales the v4l2src to something much smaller.

(thanks so much for trying to help and find a solution, but I am not quite there yet)

    w=1280; h=800; x0=450; y0=200; gst-launch-1.0
           v4l2src device=/dev/video0  ! videorate ! video/x-raw,framerate=5/1 \
           ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
           ! alpha method=green angle=70 \
           ! queue ! mixer.sink_1 \
            \
           ximagesrc xid=0x06600003 use-damage=false ! videoscale \
           ! video/x-raw,width=$w,height=$h,framerate=5/1 ! queue ! mixer.sink_0 \
            \
           videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
           ! videoconvert ! autovideosink -v

> > > > > > Output:
> > > > > >
> > > > > >     Setting pipeline to PAUSED ...
> > > > > >     Pipeline is live and does not need PREROLL ...
> > > > > >     Setting pipeline to PLAYING ...
> > > > > >     New clock: GstSystemClock
> > > > > > {
> > > > > >     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
> > > > > >     Additional debug info:
> > > > > >     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
> > > > > >     There may be a timestamping problem, or this computer is too slow.
> > > > > > } * repeats every second or so.
> > > > > >
> > > > > > Thanks & regards,
> > > > > > Alain.
> > > > > >
> > > > > >
> > > > > > > > I have a beefy desktop, 1yo i7, 6-core hyperthreaded, 64GB RAM ubuntu 18.04).
> > > > > > > > GStreamer 1.14.5
> > > > > > > >
> > > > > > > > ===========
> > > > > > > > In detail:
> > > > > > > >
> > > > > > > > The video shown with this command line is very laggy (it hardly uses any RAM or CPU // other note: my desktop easily copes with e.g. zoom meetings):
> > > > > > > >
> > > > > > > >     gst-launch-1.0 autovideosrc ! videoconvert ! autovideosink # shows no error message
> > > > > > > >     gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! autovideosink # does exactly the same
> > > > > > > >
> > > > > > > >
> > > > > > > > Preparing the ground for chroma-keying a green screen background, this example works the same way the first two do (laggy but good quality & no error message showing):
> > > > > > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0 \
> > > > > > > >         v4l2src device=/dev/video0 \
> > > > > > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > > > > > >         ! alpha method=green angle=70 \
> > > > > > > >         ! videoconvert \
> > > > > > > >         ! autovideosink
> > > > > > > >
> > > > > > > > The background to replace the green screen will be taken from an application window.
> > > > > > > > This causes issues with buffers dropped & it complains about timestamping issues.
> > > > > > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! videoconvert ! autovideosink

This point is solved by "use-damage=false"

> > > > > > > > Below is the output of the command above:
> > > > > > > >     Setting pipeline to PAUSED ...
> > > > > > > >     Pipeline is live and does not need PREROLL ...
> > > > > > > >     Setting pipeline to PLAYING ...
> > > > > > > >     New clock: GstSystemClock
> > > > > > > >     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
> > > > > > > >     Additional debug info:
> > > > > > > >     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
> > > > > > > >     There may be a timestamping problem, or this computer is too slow.
> > > > > > > >
> > > > > > > >
> > > > > > > > Even if I impose a low frame rate, it outputs the same error messages as above, command line:
> > > > > > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! video/x-raw,framerate=5/1 ! videoconvert ! autovideosink
> > > > > > > >
> > > > > > > > Obviously, if I combine the two, performance gets worse, despite still not hitting the CPU or RAM, and the following command shows exactly the same messages as above:
> > > > > > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > > > > > > >         v4l2src device=/dev/video0 \
> > > > > > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > > > > > >         ! alpha method=green angle=70 \
> > > > > > > >         ! mixer.sink_1   \
> > > > > > > >         \
> > > > > > > >         ximagesrc xid=0x02a00006 ! videoscale \
> > > > > > > >         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
> > > > > > > >         \
> > > > > > > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > > > > > > >         ! videoconvert ! autovideosink
> > > > > > > >
> > > > > > > > Whether I put ximagesrc before v4l2src or after changes nothing.
> > > > > > > >
> > > > > > > > (next challenges: 1/ channelling this to /dev/video10; 2/ channeling the audio from the webcam (/dev/video0) to the virtual webcam (dev/video10))
> > > > > > > >
> > > > > > > > For now I  would most appreciate some help with making the above commands works without error/warning messages and good performance. Or in other words how do I resolve timestamping with ximagesrc?
> > > > > > > >
> > > > > > > > Best regards,
> > > > > > > > Alain.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Arnaud Loonstra]

On 08-04-2020 12:38, Alain Culos wrote:
[big snip]

I tried your pipeline but couldn't get it to work. I think it'll run a
lot smoother by adding sync=false to your imagesink.

I'm also trying to create something like your setup. We could definitely
use something for linux like camtwist on OSX. It should be easily doable
with gstreamer and v4l2loopback. It's quite tricky to get stuff like
this working with gst-launch. I just played a bit with pipeviz and
immediatelly got two streams working with the videomixer.

https://github.com/virinext/pipeviz

Rg,

Arnaud
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Desouza, Ederson]

On Wed, 2020-04-08 at 11:38 +0100, Alain Culos wrote:

> On Tue, 7 Apr 2020, at 01:24, Desouza, Ederson wrote:
> > On Tue, 2020-04-07 at 00:52 +0100, Alain Culos wrote:
> > > On Tue, 7 Apr 2020, at 00:11, Desouza, Ederson wrote:
> > > > On Mon, 2020-04-06 at 23:16 +0100, Alain Culos wrote:
> > > > > On Mon, 6 Apr 2020, at 21:44, Desouza, Ederson wrote:
> > > > > > On Mon, 2020-04-06 at 20:39 +0100, Alain Culos wrote:
> > > > > > > On Mon, 6 Apr 2020, at 17:26, Desouza, Ederson wrote:
> > > > > > > > On Sun, 2020-04-05 at 01:20 +0100, Alain Culos wrote:
> > > > > > > > > Summary:
> > > > > > > > > When using ximagesrc, error messages appear to tell me buffers are lost and that there may be timestamping issues or that my computer is slow.
> > > > > > > > Have you tried using "use-damage=false" parameter on ximagesrc?
> > > > > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 use-damage=false ! videoconvert ! autovideosink
> > > > > > > The simple pipeline above now works flawlessly, with near real-time mirroring of the original window, great, now I need to make the more complex pipeline work better.
> > > > > > >
> > > > > > > The mixer pipeline below, now updates about once per second (much better than without the "use-damage=false"), but still reports the timestamping error message.
> > > > > > >
> > > > > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > > > > > >         v4l2src device=/dev/video0 \
> > > > > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > > > > >         ! alpha method=green angle=70 \
> > > > > > >         ! mixer.sink_1   \
> > > > > > >         \
> > > > > > >         ximagesrc xid=0x02a00006 use-damage=false ! videoscale \
> > > > > > >         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
> > > > > > >         \
> > > > > > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > > > > > >         ! videoconvert ! autovideosink
> > > > > >
> > > > > > Try matching the framerate of both ximagesrc and v4l2src elements.
> > > > > > Maybe add some queues to the pipeline too?
> > > > You can take a look at this link for a nice overview of queues:
> > > > https://gstreamer.freedesktop.org/documentation/tutorials/basic/multithreading-and-pad-availability.html?gi-language=c
> > > > I suggested it so that you may have more than on thread in your
> > > > pipeline. It _may_ help - or not =D
> > > Great resource, clear.
> > > I moved both queues to the end of my pipelines, just before the input into the mixer sinks, it did not change anything.
> > >
> > > > > Trying (see below, 2nd block, ximagesrc, 2nd line, framerate) and (2nd block, 1st line, queue) - one or the other or both - all yield the same result as previously:
> > > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > > > >         v4l2src device=/dev/video0 \
> > > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > > >         ! alpha method=green angle=70 \
> > > > >         ! mixer.sink_1   \
> > > > >         \
> > > > >         ximagesrc xid=0x02a00006 use-damage=false ! queue ! videoscale \
> > > > >         ! video/x-raw,width=$w,height=$h,framerate=30/1 ! mixer.sink_0   \
> > > > >         \
> > > > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > > > >         ! videoconvert ! autovideosink
> > > > >

Your pipeline does work for me (I just need to scale v4l2src output, as
it doesn't output a big enough video for you parameters). I'm afraid
I'm out of ideas now... =/

> > > >
> > > > I see that you fixed the framerate for the ximagesrc - are you sure
> > > > that v4l2src has the same framerate? Anyway, another thing to try is to
> > > > lower both framerates. You may need to use "videorate" element to
> > > > reduce v4l2src framerate.
> > > I kept the queues and added "! videorate max-rate=5" (if I use rate instead of max-rate, it freezes the video output) after both ximagesrc and v42src, it made things worse (slower than 1 refresh per second) + same error messages.
> > I imagined something like:
> >  ... ! videorate ! video/x-raw,framerate=10/1 ! ...
> >
> > For the v4l2src branch. I think that you don't need the videorate on
> > the ximagesrc branch, only the caps.
>
> Ok, I removed the videorate from the ximagesrc branch, using caps to set the framerate to 5/1 or 10/1.
> I tried the "! videorate ! video/x-raw,framerate=5/1" (or 10/1) before or after (or also without) the videocrop element.
> In all cases I still get the error messages and warnings, as well as a very laggy video stream.
> With framerate at 10/1 it also scales the v4l2src to something much smaller.
>
> (thanks so much for trying to help and find a solution, but I am not quite there yet)
>
>     w=1280; h=800; x0=450; y0=200; gst-launch-1.0
>            v4l2src device=/dev/video0  ! videorate ! video/x-raw,framerate=5/1 \
>            ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
>            ! alpha method=green angle=70 \
>            ! queue ! mixer.sink_1 \
>             \
>            ximagesrc xid=0x06600003 use-damage=false ! videoscale \
>            ! video/x-raw,width=$w,height=$h,framerate=5/1 ! queue ! mixer.sink_0 \
>             \
>            videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
>            ! videoconvert ! autovideosink -v
>
> > > > > > > Output:
> > > > > > >
> > > > > > >     Setting pipeline to PAUSED ...
> > > > > > >     Pipeline is live and does not need PREROLL ...
> > > > > > >     Setting pipeline to PLAYING ...
> > > > > > >     New clock: GstSystemClock
> > > > > > > {
> > > > > > >     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
> > > > > > >     Additional debug info:
> > > > > > >     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
> > > > > > >     There may be a timestamping problem, or this computer is too slow.
> > > > > > > } * repeats every second or so.
> > > > > > >
> > > > > > > Thanks & regards,
> > > > > > > Alain.
> > > > > > >
> > > > > > >
> > > > > > > > > I have a beefy desktop, 1yo i7, 6-core hyperthreaded, 64GB RAM ubuntu 18.04).
> > > > > > > > > GStreamer 1.14.5
> > > > > > > > >
> > > > > > > > > ===========
> > > > > > > > > In detail:
> > > > > > > > >
> > > > > > > > > The video shown with this command line is very laggy (it hardly uses any RAM or CPU // other note: my desktop easily copes with e.g. zoom meetings):
> > > > > > > > >
> > > > > > > > >     gst-launch-1.0 autovideosrc ! videoconvert ! autovideosink # shows no error message
> > > > > > > > >     gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert ! autovideosink # does exactly the same
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Preparing the ground for chroma-keying a green screen background, this example works the same way the first two do (laggy but good quality & no error message showing):
> > > > > > > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0 \
> > > > > > > > >         v4l2src device=/dev/video0 \
> > > > > > > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > > > > > > >         ! alpha method=green angle=70 \
> > > > > > > > >         ! videoconvert \
> > > > > > > > >         ! autovideosink
> > > > > > > > >
> > > > > > > > > The background to replace the green screen will be taken from an application window.
> > > > > > > > > This causes issues with buffers dropped & it complains about timestamping issues.
> > > > > > > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! videoconvert ! autovideosink
>
> This point is solved by "use-damage=false"
>
> > > > > > > > > Below is the output of the command above:
> > > > > > > > >     Setting pipeline to PAUSED ...
> > > > > > > > >     Pipeline is live and does not need PREROLL ...
> > > > > > > > >     Setting pipeline to PLAYING ...
> > > > > > > > >     New clock: GstSystemClock
> > > > > > > > >     WARNING: from element /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage: A lot of buffers are being dropped.
> > > > > > > > >     Additional debug info:
> > > > > > > > >     gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstAutoVideoSink:autovideosink0/GstXvImageSink:autovideosink0-actual-sink-xvimage:
> > > > > > > > >     There may be a timestamping problem, or this computer is too slow.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Even if I impose a low frame rate, it outputs the same error messages as above, command line:
> > > > > > > > >     gst-launch-1.0   ximagesrc xid=0x02a00006 ! video/x-raw,framerate=5/1 ! videoconvert ! autovideosink
> > > > > > > > >
> > > > > > > > > Obviously, if I combine the two, performance gets worse, despite still not hitting the CPU or RAM, and the following command shows exactly the same messages as above:
> > > > > > > > >     w=1280; h=800; x0=450; y0=200; gst-launch-1.0   \
> > > > > > > > >         v4l2src device=/dev/video0 \
> > > > > > > > >         ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
> > > > > > > > >         ! alpha method=green angle=70 \
> > > > > > > > >         ! mixer.sink_1   \
> > > > > > > > >         \
> > > > > > > > >         ximagesrc xid=0x02a00006 ! videoscale \
> > > > > > > > >         ! video/x-raw,width=$w,height=$h ! mixer.sink_0   \
> > > > > > > > >         \
> > > > > > > > >         videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
> > > > > > > > >         ! videoconvert ! autovideosink
> > > > > > > > >
> > > > > > > > > Whether I put ximagesrc before v4l2src or after changes nothing.
> > > > > > > > >
> > > > > > > > > (next challenges: 1/ channelling this to /dev/video10; 2/ channeling the audio from the webcam (/dev/video0) to the virtual webcam (dev/video10))
> > > > > > > > >
> > > > > > > > > For now I  would most appreciate some help with making the above commands works without error/warning messages and good performance. Or in other words how do I resolve timestamping with ximagesrc?
> > > > > > > > >
> > > > > > > > > Best regards,
> > > > > > > > > Alain.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Alain Culos]

Thank you both Ederson & Arnaud,

This pipeline now works without error messages:

w=1280; h=800; x0=450; y0=200; gst-launch-1.0
    v4l2src device=/dev/video0 \
    ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 bottom=$(( 1080 - $h - $y0 )) \
    ! alpha method=green angle=70 \
    ! queue \
    ! mixer.sink_1 \
    \
    ximagesrc xid=0x06600003 use-damage=false \
    ! videoscale \
    ! video/x-raw,width=$w,height=$h \
    ! queue \
    ! mixer.sink_0 \
    \
    videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
    ! videoconvert \
    ! autovideosink -v sync=false

The time-lag is significant and the displayed framerate is poor (maybe 5/1 to 10/1), but it works.
Note that my desktop hardly works at all (overall CPU well below 10%, 1 of 12 cores at 10-20% max).

1/ It would be great if someone could chime in with ideas to help improve the real-time performance of this pipeline - ideally resulting in a framerate of at least 15 or 20.

2/ My next step is to use the "alpha" element properly.
With the pipeline above my green screen gets replaced with a shaded background - i.e. alpha is not 0 or 1, but something in between.
I already pushed the "angle" value as far as I can get without becoming semi-transparent myself.
How do I increase the alpha channel values without becoming transparent?

Thanks,
Alain.


On Wed, 8 Apr 2020, at 13:27, Arnaud Loonstra wrote:

> On 08-04-2020 12:38, Alain Culos wrote:
> [big snip]
>
> I tried your pipeline but couldn't get it to work. I think it'll run a
> lot smoother by adding sync=false to your imagesink.
>
> I'm also trying to create something like your setup. We could definitely
> use something for linux like camtwist on OSX. It should be easily doable
> with gstreamer and v4l2loopback. It's quite tricky to get stuff like
> this working with gst-launch. I just played a bit with pipeviz and
> imediatelly got two streams working with the videomixer.
>
> https://github.com/virinext/pipeviz
>
> Rg,
> Arnaud

How does my pipeline fail for you?
If it fails on v4l2src, try autoviedosrc instead.
If it fails on ximagesrc, it could be because you have the wrong window id?
You can drop the videocrop this is to extract the portion of the webcam video of interest to me (zooming in) - and with it you can drop the videoscale and caps to rescale the ximagesrc.
You can drop the alpha, it should work similarly, albeit without green screen replacement, but that would test the pipeline with more simplicity.
Check your window ids with "wmctrl -l".


On Wed, 8 Apr 2020, at 17:14, Desouza, Ederson wrote:
> Your pipeline does work for me (I just need to scale v4l2src output, as
> it doesn't output a big enough video for you parameters). I'm afraid
> I'm out of ideas now... =/

Different webcams, I suppose. Mine is a 1080p.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Alain Culos]

On Wed, 8 Apr 2020, at 18:14, Alain Culos wrote:

> Thank you both Ederson & Arnaud,
>
> This pipeline now works without error messages:
>
> w=1280; h=800; x0=450; y0=200; gst-launch-1.0
>     v4l2src device=/dev/video0 \
>     ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0
> bottom=$(( 1080 - $h - $y0 )) \
>     ! alpha method=green angle=70 \
>     ! queue \
>     ! mixer.sink_1 \
>     \
>     ximagesrc xid=0x06600003 use-damage=false \
>     ! videoscale \
>     ! video/x-raw,width=$w,height=$h \
>     ! queue \
>     ! mixer.sink_0 \
>     \
>     videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
>     ! videoconvert \
>     ! autovideosink -v sync=false
>
> The time-lag is significant and the displayed framerate is poor (maybe
> 5/1 to 10/1), but it works.
> Note that my desktop hardly works at all (overall CPU well below 10%, 1
> of 12 cores at 10-20% max).
>
> 1/ It would be great if someone could chime in with ideas to help
> improve the real-time performance of this pipeline - ideally resulting
> in a framerate of at least 15 or 20.
>
> 2/ My next step is to use the "alpha" element properly.
> With the pipeline above my green screen gets replaced with a shaded
> background - i.e. alpha is not 0 or 1, but something in between.
> How do I increase the alpha channel values?

I said something wrong about becoming transparent. I pushed the angle to 90, and the alpha channel still isn't 100% transparent. I am fine though.

> Thanks,
> Alain.
>
>
> On Wed, 8 Apr 2020, at 13:27, Arnaud Loonstra wrote:
> > On 08-04-2020 12:38, Alain Culos wrote:
> > [big snip]
> >
> > I tried your pipeline but couldn't get it to work. I think it'll run a
> > lot smoother by adding sync=false to your imagesink.
> >
> > I'm also trying to create something like your setup. We could definitely
> > use something for linux like camtwist on OSX. It should be easily doable
> > with gstreamer and v4l2loopback. It's quite tricky to get stuff like
> > this working with gst-launch. I just played a bit with pipeviz and
> > imediatelly got two streams working with the videomixer.
> >
> > https://github.com/virinext/pipeviz
> >
> > Rg,
> > Arnaud
>
> How does my pipeline fail for you?
> If it fails on v4l2src, try autoviedosrc instead.
> If it fails on ximagesrc, it could be because you have the wrong window
> id?
> You can drop the videocrop this is to extract the portion of the webcam
> video of interest to me (zooming in) - and with it you can drop the
> videoscale and caps to rescale the ximagesrc.
> You can drop the alpha, it should work similarly, albeit without green
> screen replacement, but that would test the pipeline with more
> simplicity.
> Check your window ids with "wmctrl -l".
>
>
> On Wed, 8 Apr 2020, at 17:14, Desouza, Ederson wrote:
> > Your pipeline does work for me (I just need to scale v4l2src output, as
> > it doesn't output a big enough video for you parameters). I'm afraid
> > I'm out of ideas now... =/
>
> Different webcams, I suppose. Mine is a 1080p.
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
>

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Alain Culos]

On Wed, 8 Apr 2020, at 13:27, Arnaud Loonstra wrote:
> I'm also trying to create something like your setup. We could definitely
> use something for linux like camtwist on OSX. It should be easily doable
> with gstreamer and v4l2loopback.

I meant to say that I found OBS - https://obsproject.com/ - which might answer your requirements or not.
It more or less does what I want - with an add-on, https://github.com/CatxFish/obs-v4l2sink.git - but I would much prefer to be able to do this with gstreamer, starting with gst-launch for proof of concept before I move on to more involved programming whether C or python.

> https://github.com/virinext/pipeviz

I meant to say thank you - while I have not taken the time to check it out yet, I will.

Regards,
Alain.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Arnaud Loonstra]

On 09-04-2020 21:55, Alain Culos wrote:
> On Wed, 8 Apr 2020, at 13:27, Arnaud Loonstra wrote:
>> I'm also trying to create something like your setup. We could definitely
>> use something for linux like camtwist on OSX. It should be easily doable
>> with gstreamer and v4l2loopback.
>
> I meant to say that I found OBS - https://obsproject.com/ - which might answer your requirements or not.
> It more or less does what I want - with an add-on, https://github.com/CatxFish/obs-v4l2sink.git - but I would much prefer to be able to do this with gstreamer, starting with gst-launch for proof of concept before I move on to more involved programming whether C or python.

Couldn't agree more, but the gst-launch commands get very big and
cumbersome easily. If had the time I would start building something in
Python.

I already know OBS studio but I never gave it a try up until now. It's
quite cool. Going to play with it :)

>
>> https://github.com/virinext/pipeviz
>
> I meant to say thank you - while I have not taken the time to check it out yet, I will.

I really like it. It's much more intuitive to experiment with than
gst-launch. I think gstreamer could really benefit from having such a
tool. Pipewire is however in a proof of concept stage IMHO.

Rg,

Arnaud
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Alain Culos]

On Thu, 9 Apr 2020, at 22:13, Arnaud Loonstra wrote:
> > On Wed, 8 Apr 2020, at 13:27, Arnaud Loonstra wrote:
> >> https://github.com/virinext/pipeviz
> I really like it. It's much more intuitive to experiment with than
> gst-launch. I think gstreamer could really benefit from having such a
> tool. Pipewire is however in a proof of concept stage IMHO.

I just tried. It does not agree with me, unlike you I find the command line easier to deal with, faster.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Daniel Johnson]

On Thu, Apr 9, 2020, 2:00 PM Alain Culos <[hidden email]> wrote:

I meant to say that I found OBS - https://obsproject.com/ - which might answer your requirements or not.
It more or less does what I want - with an add-on, https://github.com/CatxFish/obs-v4l2sink.git - but I would much prefer to be able to do this with gstreamer, starting with gst-launch for proof of concept before I move on to more involved programming whether C or python.

I mostly prototyped out the pipelines with a live streaming command line app. I did the compositing in OpenGL and used used Intel hardware video encoder. I tried to use hardware acceleration wherever possible which had the negative side effects of it being broken on anything other than Intel graphics.

So glupload ! glvideomixer

vaapiencode

Etc

When someone mentioned queues that's important because without them stuff will generally be running on the same cpu core so your 6 cores won't help you. Queues allow things on either side of the queue to be on different threads. If the queues add too much latency from buffers that's tuneable. 

I'm currently starting over with a cross platform UI, but maybe you can fork the code to get what you need. 

https://github.com/teknotus/bitcorder

What's still broken in your pipeline?

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Alain Culos]

On Mon, 13 Apr 2020, at 20:31, Daniel Johnson wrote:

On Thu, Apr 9, 2020, 2:00 PM Alain Culos <[hidden email]> wrote:

I meant to say that I found OBS -https://obsproject.com/ - which might answer your requirements or not.

It more or less does what I want - with an add-on, https://github.com/CatxFish/obs-v4l2sink.git - but I would much prefer to be able to do this with gstreamer, starting with gst-launch for proof of concept before I move on to more involved programming whether C or python.

I mostly prototyped out the pipelines with a live streaming command line app. I did the compositing in OpenGL and used used Intel hardware video encoder. I tried to use hardware acceleration wherever possible which had the negative side effects of it being broken on anything other than Intel graphics.

So glupload ! glvideomixer

vaapiencode

Etc

When someone mentioned queues that's important because without them stuff will generally be running on the same cpu core so your 6 cores won't help you. Queues allow things on either side of the queue to be on different threads. If the queues add too much latency from buffers that's tuneable. 

I'm currently starting over with a cross platform UI, but maybe you can fork the code to get what you need. 

https://github.com/teknotus/bitcorder

What's still broken in your pipeline?

bitcoder sounds really interesting, but I am not ready to delve into this kind of coding at this stage. One question though: does your code include being able to output to a video device (/dev/videoXX)?

I'll also look at the gl elements, but presumably glupload is additional on both streams upstream of the mixer, right?

What is vaapiencode for, can you elaborate?

My pipeline now works without error messages:

w=1280; h=800; x0=450; y0=200; gst-launch-1.0
    v4l2src device=/dev/video0 \
    ! videocrop left=$x0 right=$(( 1920 - $w - $x0 )) top=$y0 
bottom=$(( 1080 - $h - $y0 )) \
    ! alpha method=green angle=70 \
    ! queue \
    ! mixer.sink_1 \
    \
    ximagesrc xid=0x06600003 use-damage=false \
    ! videoscale \
    ! video/x-raw,width=$w,height=$h \
    ! queue \
    ! mixer.sink_0 \
    \
    videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 \
    ! videoconvert \
    ! autovideosink -v sync=false

The time-lag is significant and the displayed framerate is poor (maybe 5 to 10 fps), but it works.
Note that my desktop hardly works at all (overall CPU average well below 10%, the 1 highest of 12 cores at 10-20% max).

1/ It would be great if someone could chime in with ideas to help improve the real-time performance of this pipeline - ideally resulting in a framerate of at least 15 or 20.

2/ My next step is to use the "alpha" element properly.

With the pipeline above my green screen gets replaced with a shaded background - i.e. alpha is not 0 or 1, but something in between.

How do I increase the alpha channel values?

3/ My following step is to be able to output to /dev/video11 (v4l2loopback). It currently blocks because of the ximagesrc, but I do not understand why.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel