GStreamer to combine 2 streams into Side-by-Side Stereo3D?

[Nathan Clark]

Hi Guys,

My name is Nathan.

I am building a Fully OpenSource 3D Cinema Camera-Rig as part of the Apertus "Open Source Cinema" project.

I am writing to introduce myself, with hopes to muster up some enthusiasm for the project; and also to seek assistance in the development of a tool, built on GStreamer- for monitoring the stereoscopic rig during production.

As I do not have any experience with program development, I am here to see if:
a) this tool is easy/difficult to create, and
b) if anyone is interested helping with it's development.

The basic function of the tool ("S3DView") is as follows:

2 x synchronized IP cameras send 2 x unicast rtsp streams over the network

rtsp://192.168.0.8:554 (left)
rtsp://192.168.0.9:554 (right)

"S3DView" takes the 2 streams and displays them in one of 4 ways:

1. Side-by-Side

Left image is squashed 50% and positioned to the left
Right image is squashed 50% and positioned to the right

2. Overlay

Both images are displayed- one overlaid on the other, with 50% opacity

3. Left Only

Left image only is show (with unobtrusive text overlay - "left")

3. Right Only

Right image only is show (with unobtrusive text overlay - "right")

These four display modes could then be switched on the fly using shortcuts for example:

Side-by-Side: Up Arrow
Overlay: Down Arrow
Left Only: Left Arrow
Right Only: Right Arrow

The window should be able to optionally go into "fullscreen" mode and retain the ability to switch between the 4 viewing modes.

Until this point, i have been attempting to use VLC's mosaic to do this, but very quickly have I realized that GStreamer looks to be far better suited to the job.

Unfortunately, it is beyond my scope to develop the "S3DView" tool myself- if anyone is interested in becoming involved, or if anyone has any questions / ideas / input-- please let me know.

Thanks for your time.

Kind regards,
Nathan Clark

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nathan Clark]

Hi Martin,

Thanks for your brilliantly fragmented email! :D
Indeed that was me that contacted you re. your anaglyph filter for GStreamer.
Its great to hear of your continued enthusiasm! The Apertus Open Source Cinema Stereoscopic Rig has come quite a way since then!

You can see some test footage on youtube (link 1 below) or vimeo (link 2). Please excuse the excessive disparity (ie. broken background), I couldnt get the cameras close together on this occasion.

I have been working closely with Sebastian over at Apertus.org and he has implemented stereoscopic support into the existing ElphelVision software- which has enabled us to capture synchronized, raw, stereoscopic footage! The next step is sorting out a solution for monitoring the cameras on location.

Please don't worry about insulting me, as I can confirm that I am completely uneducated with regards to programming / development. So my capacity to develop this small app, as basic as it may be- depends solely on the patience and assistance of the community!

Luckily, my partner is doing her PHD in stereoscopic cinema and has several, particularly geeky friends. One of whom has agreed to have a look at the requirements and may be able to assist in developing the tool.

Since my initial email to this mailing list, the scope of the project has grown to include a live debayering plugin (link 3)- prior to assembling them into a stereo pair. This process is quite cpu intensive- but it allows us to shoot and monitor in RAW, which significantly improves the final quality of image!

Furthermore, I have thought it would be ideal if the resulting stereo pair could be transcoded to a lower resolution and restreamed over the local network- for monitoring on less powerful devices.

Finally, the implementation of your anaglyph plug-ins would be the icing on the cake! I do have concerns, however, about the CPU usage becoming and issue. (is your anaglyph plugin cpu intensive?)

All in all, the concept has matured considerably, but if it manages to come into fruition- I belive it will be of great significance to the open source cinema community!

(Link 1) http://www.youtube.com/watch?v=popTxxJ4zpo
(Link 2) http://vimeo.com/24735939
(Link 3) http://code.google.com/p/gst-plugins-elphel/

PS #1 ;)
I know this email has gone off on a bit of a tangent- and I apologize for rambling! but I did think it might be of interest to provide a bit more of a detailed account of where the project is at the moment!

PS #2
Back on topic now: Hopefully my partners friend will take up the development of the actual software tool, within which GStreamer will be implemented. He is a seasoned linux developer and as such will not require any basic instruction / tuition regarding programming (unlike myself!) But he is not familiar with GStreamer as of yet, so he will undoubtedly require some pointers along the way- especially when it comes to the inclusion of your anaglyph plugin. Would you be happy for me to suggest to him to use this thread to ask any technical questions of you?

On Fri, Jun 24, 2011 at 3:31 PM, Martin Bisson <[hidden email]> wrote:

I worked on the GStreamer side-branch for stereo video.

But what you are looking for is really "basic" GStreamer usage, i.e. copying video streams to sub parts of the screen.  What I mean by "basic" is that I don't think you would have to create new elements in the pipelines to do what you described, you just need to use existing ones and configure an appropriate pipeline.  Take a look at the way to create a basic pipeline, the way to use the video scale, video crop, etc, elements to do what you want to do.  I don't have access to it right now, but I could send you the pipeline I used to create that video which seems to be a bit what you are trying to do :

http://misc.martinbisson.com/voyage/presentation/Knights_Quest_half.wmv

(If I remember correctly : top left = left view, top right = right view, bottom left = "gray" anaglyph, bottom right = color anaglyph)

However, you'd probably have to understand the pipeline workflow in order to be able to do what you're trying to do...

Hope this helps, available to answer some questions,

Mart


P.S.  I wrote this e-mail, but then when I got internet access again I re-read it before sending it to you and realized I might have read your e-mail too quickly and I might not be answering your actual question...

There are probably many ways to to what you want to do, the easiest I see right now is to have an embedded GStreamer window in your application.  Your application would be managing events (such as the arrow key events) and setting the GStreamer pipeline to use accordingly.

Anyway, hope this helps a little.  I don't know how comfortable you are with GStreamer or general programming, so I don't want to waste my time/insult you by giving you very basic information, but I'm willing to help you by answering questions and pointing your to the right sources to do what you want to do.

End of P.S.!

P.S. #2 another view you could add to your software is the anaglyph view ;).  But it's not integrated in the main GStreamer branch yet.  But the more people want it, the more it will be prioritized and the quicker it will get "accepted".

P.S. #3 I just remembered...  Someone from Apartus contacted me regarding possible uses of GStreamer for stereo video in Apertus.  I'm pretty sure it was you.  I re-iterate my strong interest in the project, I think what you guys are doing is very cool and would like to help as much as I can, but I have very, very limited free time these days so I can't do actual development.  I'd be glad to orient you in the right direction if I can.

Sorry for "split/incoherent email", I was offline for a while...

On 20/06/2011 11:24 AM, Nathan Clark wrote:

Hi Guys,

My name is Nathan.

I am building a Fully OpenSource 3D Cinema Camera-Rig as part of the Apertus "Open Source Cinema" project.

I am writing to introduce myself, with hopes to muster up some enthusiasm for the project; and also to seek assistance in the development of a tool, built on GStreamer- for monitoring the stereoscopic rig during production.

As I do not have any experience with program development, I am here to see if:
a) this tool is easy/difficult to create, and
b) if anyone is interested helping with it's development.

The basic function of the tool ("S3DView") is as follows:

2 x synchronized IP cameras send 2 x unicast rtsp streams over the network

rtsp://192.168.0.8:554 (left)
rtsp://192.168.0.9:554 (right)

"S3DView" takes the 2 streams and displays them in one of 4 ways:

1. Side-by-Side

Left image is squashed 50% and positioned to the left
Right image is squashed 50% and positioned to the right

2. Overlay

Both images are displayed- one overlaid on the other, with 50% opacity

3. Left Only

Left image only is show (with unobtrusive text overlay - "left")

3. Right Only

Right image only is show (with unobtrusive text overlay - "right")

These four display modes could then be switched on the fly using shortcuts for example:

Side-by-Side: Up Arrow
Overlay: Down Arrow
Left Only: Left Arrow
Right Only: Right Arrow

The window should be able to optionally go into "fullscreen" mode and retain the ability to switch between the 4 viewing modes.

Until this point, i have been attempting to use VLC's mosaic to do this, but very quickly have I realized that GStreamer looks to be far better suited to the job.

Unfortunately, it is beyond my scope to develop the "S3DView" tool myself- if anyone is interested in becoming involved, or if anyone has any questions / ideas / input-- please let me know.

Thanks for your time.

Kind regards,
Nathan Clark

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel