Is there a GStreamer filter that does normalization on an image?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Is there a GStreamer filter that does normalization on an image?

chrisp
Hello GStreamer Devel World!

I'm trying to create a GStreamer pipeline that can do image processing.
Specifically, I'm doing image pre-processing to be used by machine learning
networks. So this pipeline will do all the necessary pre-processing steps
before it is fed to the neural network.
Right now, I'm trying to figure out if there are any GStreamer plugins that
can simply do normalization.
GStreamer does allow programmers to create custom plugins for applications.
This is one solution to this question, but I want to find out if there are
any pre-existing solutions.

I've looked into GStreamers list of plugins on their website but there are a
lot and some descriptions aren't clear, to me, what they do or how to use
them.

I assume that the pipeline would look something like this:
gst-launch-1.0 videotestsrc ! decodebin ! imagefreeze ! videoconvert !
<PREPROCESS FILTER> ! videoconvert ! autovideosink

If anyone is curious, this is the normalization method that I want to
replicate for my GStreamer pipeline.

# Python code for image preprocessing
def preprocess(img_data):
    mean_vec = np.array([0.485, 0.456, 0.406])
    stddev_vec = np.array([0.229, 0.224, 0.225])
    norm_img_data = np.zeros(img_data.shape).astype('float32')
    for i in range(img_data.shape[0]):  
         # for each pixel in each channel, divide the value by 255 to get
value between [0, 1] and then normalize
        norm_img_data[i,:,:] = (img_data[i,:,:]/255 - mean_vec[i]) /
stddev_vec[i]
    return norm_img_data

Anything might help!!! I'm still new to GStreamer and still learning. Thank
you in advance!




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Is there a GStreamer filter that does normalization on an image?

Mathieu Duponchelle
Hello,

Wikipedia tells me that image normalization is also sometimes called histogram stretching,
and searching for "histogram" in the documentation leads to
<https://gstreamer.freedesktop.org/documentation/frei0r/frei0r-filter-equaliz0r.html?gi-language=c#frei0r-filter-equaliz0r>,
maybe that's what you're looking for?

--
Mathieu Duponchelle · https://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Is there a GStreamer filter that does normalization on an image?

chrisp
Hi Mathieu,

This has helped me out quite a bit to be honest.  It's not exactly what I
was looking for but this has helped me move more in the right direction.
I'm currently trying two approaches to this problem.  One is the filter you
suggested and other filters that the frei0r plugin has to offer.  Then the
other is a custom plugin that I'm writing in python.  However, since I am
still fairly new to gst, its taking me a bit longer then I expected.  Thank
you for your help.  




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
Reply | Threaded
Open this post in threaded view
|

Re: Is there a GStreamer filter that does normalization on an image?

Mathieu Duponchelle
Hi Chris,

Turns out there's also another element in the opencv plugin that might interest you,
cvequalizehist[1], as far as I can tell from [2] it does exactly what you're looking for :)

Regarding writing elements in python, it can make it pretty easy to prototype your elements,
as long as you're aware of the caveats it implies (eg. the terrible performance :)

Cheers,
--
Mathieu Duponchelle · https://www.centricular.com

[1]: https://gstreamer.freedesktop.org/documentation/opencv/cvequalizehist.html?gi-language=c#page-description
[2]: https://docs.opencv.org/3.1.0/d4/d1b/tutorial_histogram_equalization.html

On 7/1/19 10:17 PM, chrisp wrote:

> Hi Mathieu,
>
> This has helped me out quite a bit to be honest.  It's not exactly what I
> was looking for but this has helped me move more in the right direction.
> I'm currently trying two approaches to this problem.  One is the filter you
> suggested and other filters that the frei0r plugin has to offer.  Then the
> other is a custom plugin that I'm writing in python.  However, since I am
> still fairly new to gst, its taking me a bit longer then I expected.  Thank
> you for your help.  
>
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel