Rendering mjpeg streaming video in the defined alpha channel of a jpeg

[thelenko]

What I want to do is render a crosshair image on top of a mjpeg video feed from a camera.  Since I am very new to gstreamer I proceeded to do this in baby steps, but run into issues with my mjpeg video stream.  I am going to outline my steps below so you can see what I tried, what worked, and where I got stuck:

Sorry, it is a bit of a long read.  You can probably skip to the last command if you desire.  Thanks in advance for your time and help :)

I am able to get the video feed to properly display using the following command:

gst-launch gnomevfssrc location=http://ccd1611-403/axis-cgi/mjpg/video.cgi?resolution=1280x1024 ! decodebin ! ffmpegcolorspace ! xvimagesink

I am able to display the image as a video using the following command:

gst-launch multifilesrc location="crosshair.jpg" caps="image/jpeg,framerate=1/1" ! decodebin ! xvimagesink

Taking a look at the official documents I found an example for displaying a video within a defined alpha channel.  The following example displays a snow test video in the green channel of a pattern test image:

gst-launch videotestsrc pattern=smpte75 ! alpha method=green ! videomixer name=mix ! ffmpegcolorspace ! xvimagesink videotestsrc pattern=snow ! mix.

I was able to transplant my image into the example like so:

gst-launch multifilesrc location="crosshair.jpg" caps="image/jpeg,framerate=1/1" ! decodebin ! ffmpegcolorspace ! alphacolor ! alpha method=green ! queue ! videomixer name=mixer ! ffmpegcolorspace ! xvimagesink videotestsrc pattern=snow ! mixer.

I now had my cross-hair appearing on top of the snow test video.  I next tried to switch the snow test video with my video feed:

gst-launch multifilesrc location="crosshair2.jpg" caps="image/jpeg,framerate=1/1" ! decodebin ! ffmpegcolorspace ! alphacolor ! alpha method=green ! queue ! videomixer name=mixer ! ffmpegcolorspace ! autovideosink gnomevfssrc location=http://ccd1611-403/axis-cgi/mjpg/video.cgi ! decodebin ! ffmpegcolorspace ! queue ! mixer.

When I run this I get the following error:

ERROR: from element /GstPipeline:pipeline0/GstGnomeVFSSrc:gnomevfssrc0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2507): gst_base_src_loop (): /GstPipeline:pipeline0/GstGnomeVFSSrc:gnomevfssrc0:
streaming task paused, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.

I assume there is something about the format of my video that it is not liking, but I am not sure how to remedy it.

Any suggestions would be appreciated.. Thank you for your time and help :)

[Andrés González]

  On 26/08/10 18:58, thelenko wrote:
> What I want to do is render a crosshair image on top of a mjpeg video feed
> from a camera.  Since I am very new to gstreamer I proceeded to do this in
> baby steps, but run into issues with my mjpeg video stream.  I am going to
> outline my steps below so you can see what I tried, what worked, and where I
> got stuck:

Hi,
I am not a GStreamer expert, but recently I have been working with
videomixer and had similar problems. That "Task paused, reason
not-negotiated (-4)" happened a lot. It looks to me like in one of the
branches of your pipeline (that which gnomevfssrc0 belongs to) there is
no way to get all the elements working together, because for each thing
the upstream elements can produce, the downstream elements can't manage
it (don't take this too seriously, I'm just writing what I think...).
So in a few words, I think there is some element missing, possibly a
videoscale element in some point between videomixer and gnomevfssrc0, or
maybe between videomixer and multifilesrc. Maybe then you will also want
to specify the capabilities (image size, for example).

Surely the big guys can correct me if I'm wrong :-)

Good luck,
Andrés

> Sorry, it is a bit of a long read.  You can probably skip to the last
> command if you desire.  Thanks in advance for your time and help :)
>
> I am able to get the video feed to properly display using the following
> command:
>
> gst-launch gnomevfssrc
> location=http://ccd1611-403/axis-cgi/mjpg/video.cgi?resolution=1280x1024 !
> decodebin ! ffmpegcolorspace ! xvimagesink
>
> I am able to display the image as a video using the following command:
>
> gst-launch multifilesrc location="crosshair.jpg"
> caps="image/jpeg,framerate=1/1" ! decodebin ! xvimagesink
>
> Taking a look at the official documents I found an example for displaying a
> video within a defined alpha channel.  The following example displays a snow
> test video in the green channel of a pattern test image:
>
> gst-launch videotestsrc pattern=smpte75 ! alpha method=green ! videomixer
> name=mix ! ffmpegcolorspace ! xvimagesink videotestsrc pattern=snow ! mix.
>
> I was able to transplant my image into the example like so:
>
> gst-launch multifilesrc location="crosshair.jpg"
> caps="image/jpeg,framerate=1/1" ! decodebin ! ffmpegcolorspace ! alphacolor
> ! alpha method=green ! queue ! videomixer name=mixer ! ffmpegcolorspace !
> xvimagesink videotestsrc pattern=snow ! mixer.
>
> I now had my cross-hair appearing on top of the snow test video.  I next
> tried to switch the snow test video with my video feed:
>
> gst-launch multifilesrc location="crosshair2.jpg"
> caps="image/jpeg,framerate=1/1" ! decodebin ! ffmpegcolorspace ! alphacolor
> ! alpha method=green ! queue ! videomixer name=mixer ! ffmpegcolorspace !
> autovideosink gnomevfssrc
> location=http://ccd1611-403/axis-cgi/mjpg/video.cgi ! decodebin !
> ffmpegcolorspace ! queue ! mixer.
>
> When I run this I get the following error:
>
> ERROR: from element /GstPipeline:pipeline0/GstGnomeVFSSrc:gnomevfssrc0:
> Internal data flow error.
> Additional debug info:
> gstbasesrc.c(2507): gst_base_src_loop ():
> /GstPipeline:pipeline0/GstGnomeVFSSrc:gnomevfssrc0:
> streaming task paused, reason not-negotiated (-4)
> ERROR: pipeline doesn't want to preroll.
>
> I assume there is something about the format of my video that it is not
> liking, but I am not sure how to remedy it.
>
> Any suggestions would be appreciated.. Thank you for your time and help :)

------------------------------------------------------------------------------
Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
Be part of this innovative community and reach millions of netbook users
worldwide. Take advantage of special opportunities to increase revenue and
speed time-to-market. Join now, and jumpstart your future.
http://p.sf.net/sfu/intel-atom-d2d
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[thelenko]

I did some tweaking and am now at:

gst-launch -v multifilesrc location="crosshair.jpg" caps="image/jpeg,framerate=1/1" ! decodebin ! videomixer name=mixer ! xvimagesink gnomevfssrc location=http://ccd1611-403/axis-cgi/mjpg/video.cgi?.mjpg ! decodebin ! mixer.

This is able to launch without error.  Note that I removed the 'alpha method=green' method, so now I simply have a video overlayed on the crosshair graphic instead of the video appearing in the defined alpha channel of the graphic.

A secondary problem I am having now is that when using videomixer both my video feed and image lose their color and framerate.

Andres Gonzalez wrote

Hi,
I am not a GStreamer expert, but recently I have been working with
videomixer and had similar problems. That "Task paused, reason
not-negotiated (-4)" happened a lot. It looks to me like in one of the
branches of your pipeline (that which gnomevfssrc0 belongs to) there is
no way to get all the elements working together, because for each thing
the upstream elements can produce, the downstream elements can't manage
it (don't take this too seriously, I'm just writing what I think...).
So in a few words, I think there is some element missing, possibly a
videoscale element in some point between videomixer and gnomevfssrc0, or
maybe between videomixer and multifilesrc. Maybe then you will also want
to specify the capabilities (image size, for example).

Surely the big guys can correct me if I'm wrong :-)

Good luck,
Andrés

[Luciana Fujii Pontello]

On Thu, 2010-08-26 at 09:58 -0700, thelenko wrote:

> gst-launch multifilesrc location="crosshair2.jpg"
> caps="image/jpeg,framerate=1/1" ! decodebin ! ffmpegcolorspace ! alphacolor
> ! alpha method=green ! queue ! videomixer name=mixer ! ffmpegcolorspace !
> autovideosink gnomevfssrc
> location=http://ccd1611-403/axis-cgi/mjpg/video.cgi ! decodebin !
> ffmpegcolorspace ! queue ! mixer.
>
> When I run this I get the following error:
>
> ERROR: from element /GstPipeline:pipeline0/GstGnomeVFSSrc:gnomevfssrc0:
> Internal data flow error.
> Additional debug info:
> gstbasesrc.c(2507): gst_base_src_loop ():
> /GstPipeline:pipeline0/GstGnomeVFSSrc:gnomevfssrc0:
> streaming task paused, reason not-negotiated (-4)
> ERROR: pipeline doesn't want to preroll.
>
> I assume there is something about the format of my video that it is not
> liking, but I am not sure how to remedy it.

Have you tried using an alpha element in the branch you have your video?
videotestsrc can provide video with alpha, but your video probably
doesn't have one, and that's probably why you're having this error.
Videomixer expects both videos to have the same colorspace, but it is
getting one video with alpha channel and one video without it.

Regards,

Luciana Fujii

------------------------------------------------------------------------------
Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
Be part of this innovative community and reach millions of netbook users
worldwide. Take advantage of special opportunities to increase revenue and
speed time-to-market. Join now, and jumpstart your future.
http://p.sf.net/sfu/intel-atom-d2d
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[Andrés González]

  On 27/08/10 18:12, thelenko wrote:

> I did some tweaking and am now at:
>
> gst-launch -v multifilesrc location="crosshair.jpg"
> caps="image/jpeg,framerate=1/1" ! decodebin ! videomixer name=mixer !
> xvimagesink gnomevfssrc
> location=http://ccd1611-403/axis-cgi/mjpg/video.cgi?.mjpg ! decodebin !
> mixer.
>
> This is able to launch without error.  Note that I removed the 'alpha
> method=green' method, so now I simply have a video overlayed on the
> crosshair graphic instead of the video appearing in the defined alpha
> channel of the graphic.
>

Sorry, I don't have any idea about this.
> A secondary problem I am having now is that when using videomixer both my
> video feed and image lose their color and framerate.
>
I had a very similar problem (video losing color in videomixer) a few
days ago. Are you using Ubuntu? Maybe you want to download the updated
packages:

https://launchpad.net/~gstreamer-developers/+archive/ppa/+packages?field.name_filter=&field.status_filter=published&field.series_filter=lucid




------------------------------------------------------------------------------
Sell apps to millions through the Intel(R) Atom(Tm) Developer Program
Be part of this innovative community and reach millions of netbook users
worldwide. Take advantage of special opportunities to increase revenue and
speed time-to-market. Join now, and jumpstart your future.
http://p.sf.net/sfu/intel-atom-d2d
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel