Reduce to 2 colors only (black+white) based on threshhold (x-raw GRAY8 format)?

[HermannSW]

I get GRAY8 format by either of these lines on my Pi Zero:
$ gst-launch-1.0 rpicamsrc ! video/x-raw,format=GRAY8,width=640,height=480 !
videoconvert ! ...
$ gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert !
video/x-raw,format=GRAY8,width=640,height=480 ! ...

I need to convert GRAY8 format image to a black+white only (still GRAY8
format) for a specified threshold and searched on google for a long time.

https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-good/html/gst-plugins-good-plugins-videobalance.html
says "This pipeline converts the image to black and white by setting the
saturation to 0.0.".
But it only converts to GRAY which I already have with GRAY8 format.

Is there is a plugin for allowing black+white(threshold) conversion, or do I
need to code a new plugin for that?

Hermann.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Josh Doe]

On Sat, Nov 18, 2017 at 3:52 PM, HermannSW <[hidden email]> wrote:

Is there is a plugin for allowing black+white(threshold) conversion, or do I
need to code a new plugin for that?

What first comes to mind is OpenCV. I don't see a threshold element there, but you could easily make one by taking one of the existing elements like cvsobel and modifying it just a bit to get what you want.

What I'd rather see is a GEGL-based plugin. (GEGL is a filter engine not unlike GStreamer which is used by GIMP and other programs.) GEGL already has thresholding. There's an old patch (for 0.10) that hasn't seen any attention recently:

https://bugzilla.gnome.org/show_bug.cgi?id=650750

-Josh

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le samedi 18 novembre 2017 à 13:52 -0700, HermannSW a écrit :

> I get GRAY8 format by either of these lines on my Pi Zero:
> $ gst-launch-1.0 rpicamsrc ! video/x-raw,format=GRAY8,width=640,height=480 !
> videoconvert ! ...
> $ gst-launch-1.0 v4l2src device=/dev/video0 ! videoconvert !
> video/x-raw,format=GRAY8,width=640,height=480 ! ...
>
> I need to convert GRAY8 format image to a black+white only (still GRAY8
> format) for a specified threshold and searched on google for a long time.
>
> https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-good/html/gst-plugins-good-plugins-videobalance.html
> says "This pipeline converts the image to black and white by setting the
> saturation to 0.0.".
> But it only converts to GRAY which I already have with GRAY8 format.
>
> Is there is a plugin for allowing black+white(threshold) conversion, or do I
> need to code a new plugin for that?

There is frei0r-filter-threshold0r that can do that, but the element
only inputs RGBA, which forces double color conversion. It's a bit
inconvenient. I wonder if it's a Frei0r limitation, or GStreamer here.

gst-launch-1.0 videotestsrc ! video/x-raw,format=GRAY8 ! videoconvert ! frei0r-filter-threshold0r threshold=0.2 ! glimagesink

>
> Hermann.
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Josh Doe]

On Mon, Nov 20, 2017 at 2:06 PM, Nicolas Dufresne <[hidden email]> wrote:

Le samedi 18 novembre 2017 à 13:52 -0700, HermannSW a écrit :
> Is there is a plugin for allowing black+white(threshold) conversion, or do I
> need to code a new plugin for that?

There is frei0r-filter-threshold0r that can do that, but the element
only inputs RGBA, which forces double color conversion. It's a bit
inconvenient. I wonder if it's a Frei0r limitation, or GStreamer here.

It's a frei0r limitation. A while back I made a MATLAB script which would turn a MATLAB M-file into a frei0r plugin to be used in GStreamer, and found there was no GRAY8 or GRAY16_*. It's a simple #define that's needed to add additional color models, though frei0r.h hasn't been touched since 2012. 

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le lundi 20 novembre 2017 à 14:23 -0500, Josh Doe a écrit :

> > There is frei0r-filter-threshold0r that can do that, but the
> > element
> > only inputs RGBA, which forces double color conversion. It's a bit
> > inconvenient. I wonder if it's a Frei0r limitation, or GStreamer
> > here.
>
> It's a frei0r limitation. A while back I made a MATLAB script which
> would turn a MATLAB M-file into a frei0r plugin to be used in
> GStreamer, and found there was no GRAY8 or GRAY16_*. It's a simple
> #define that's needed to add additional color models, though frei0r.h
> hasn't been touched since 2012.

That's unfortunate. Well, writing a GstVideoFilter element for that is
rather trivial though. I like the use of a double for the threshold.

Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[HermannSW]

Thanks Josh and Nicolas.

The most important for me is high speed low latency, since this b/w 2 color
reduction will be part of a high framerate gstreamer robot control pipeline.

I don't know yet how to test the timing of a single gstreamer plugin. But I
tried same pipeline based on Nicolas' pipeline, with 320x240 and 90fps
against fpsdisplaysink="fakesink" to minimize overhead.

Without the frei0r plugin I get 90fps as requested:
pi@raspberrypi02:~ $ gst-launch-1.0 -v videotestsrc !
video/x-raw,format=GRAY8,width=320,height=240,framerate=90/1 ! videoconvert
! fpsdisplaysink video-sink="fakesink"
...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message =
rendered: 820, dropped: 0, current: 89.77, average: 90.11
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay:
text = rendered: 866, dropped: 0, current: 90.23, average: 90.11
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message =
rendered: 866, dropped: 0, current: 90.23, average: 90.11
...

With frei0r plugin only 55fps:
pi@raspberrypi02:~ $ gst-launch-1.0 -v videotestsrc !
video/x-raw,format=GRAY8,width=320,height=240,framerate=90/1 ! videoconvert
! frei0r-filter-threshold0r threshold=0.2 ! fpsdisplaysink
video-sink="fakesink"
...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message =
rendered: 83, dropped: 0, current: 55.62, average: 53.95
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay:
text = rendered: 111, dropped: 0, current: 55.40, average: 54.31
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message =
rendered: 111, dropped: 0, current: 55.40, average: 54.31
...

I will write my own plugin (GRAY8 in, GRAY8 out) and compare its performance
with similar pipeline then.

Hermann.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Josh Doe]

On Tue, Nov 21, 2017 at 8:07 AM, HermannSW <[hidden email]> wrote:

I will write my own plugin (GRAY8 in, GRAY8 out) and compare its performance
with similar pipeline then.

threshold0r uses a LUT which should be quite fast. Of course in your case there's a GRAY8->RGBA->GRAY8 conversion, so direct on GRAY8 should be faster than what you're seeing with frei0r. Consider adding this in the opencv plugin located in gst-plugins-bad, it will make some things a little easier, and would be nice to add, or at least use GstVideoFilter. Be sure to implement both transform and transform_ip. If you roll your own, I'd be curious how orc's cmpgtsb opcode compares to a LUT, both should be very fast.

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel