Live Video Stream Mixer
Locked
 
|
Hi all,
I'm a final year Engineering undergraduate and have decided to spend my final year project designing and building an open source software suite which will hopefully allow for the cheap production of professional-style live internet TV broadcasts using consumer hardware. I intend to use GStreamer as the main framework to help build the software and was hoping I may be able to get some advice from you guys about whether the idea is actually feasible with GStreamer.
Here's a bit more detail: The idea is that input from consumer video cameras will be transcoded and streamed live via LAN/internet to a central "production server". The main aim of my project is to write the video mixer software for this "production server" that will allow for multiple video streams (with audio) to be cut/mixed together live into a single stream that will be output as an internet TV channel. In the future I intend to extend this to allow overlay of 2D and 3D moving graphics (such as tickers, logos etc.), allow video fades, wipes etc. between streams and for playback of local video files for mixing into the content (for VTs, ad breaks etc.). I hope it will eventually become an open source software version of a live video mixer/effects, opening up live TV production to the masses with little cost involved!
So the technical issues - (and apologies if the q's seem obvious...I'm a bit of a novice with audio/video software development!) 1) Is live mixing of video streams going to be possible using the GStreamer framework and are there any suggestions on how I should look into implementing this? Can I simply cut between pipeline frames once the video has been decoded?
2) Are fades/wipes and motion graphics overlays going to be possible without using things like OpenGL (or similar) to display the video and then capture the frames back into a new pipeline (like screen capturing)?
3) Low latency is desirable, how powerful a PC would be required for this decoding, mixing and recoding multiple low-res live video streams and is there any graphics hardware acceleration available with GStreamer using CUDA, OpenCL or similar?
4) In your opinions is this something that would be worth setting up as an open source project in the future once I have done the ground work? Many thanks in advance for any help and feedback!
Nick
------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
|
|
Nick Daniels schrieb:
> Hi all, > > I'm a final year Engineering undergraduate and have decided to spend my > final year project designing and building an open source software suite > which will hopefully allow for the cheap production of > professional-style live internet TV broadcasts using consumer hardware. > I intend to use GStreamer as the main framework to help build the > software and was hoping I may be able to get some advice from you guys > about whether the idea is actually feasible with GStreamer. > > Here's a bit more detail: The idea is that input from consumer video > cameras will be transcoded and streamed live via LAN/internet to a > central "production server". The main aim of my project is to write the > video mixer software for this "production server" that will allow for > multiple video streams (with audio) to be cut/mixed together live into a > single stream that will be output as an internet TV channel. In the > future I intend to extend this to allow overlay of 2D and 3D moving > graphics (such as tickers, logos etc.), allow video fades, wipes etc. > between streams and for playback of local video files for mixing into > the content (for VTs, ad breaks etc.). I hope it will eventually become > an open source software version of a live video mixer/effects, opening > up live TV production to the masses with little cost involved! > This is quite a project. Not sure if it fits a final year project. I am not a video expert, but answer as no one else did so far. > So the technical issues - (and apologies if the q's seem obvious...I'm a > bit of a novice with audio/video software development!) > 1) Is live mixing of video streams going to be possible using the > GStreamer framework and are there any suggestions on how I should look > into implementing this? Can I simply cut between pipeline frames once > the video has been decoded? There is a videomixer element that works on raw video. So you have to decode the video anyway. Wheter you can cut on arbitrary positions depends at bit on the container format and codec. Try it and file bugs where it does not work. > 2) Are fades/wipes and motion graphics overlays going to be possible > without using things like OpenGL (or similar) to display the video and > then capture the frames back into a new pipeline (like screen capturing)? Everything is possible. It might be slow though. we have a couple of fades and wipes. Overlays are just mixing another stream. It does not matter too much wheter its animated or not. > 3) Low latency is desirable, how powerful a PC would be required for > this decoding, mixing and recoding multiple low-res live video streams > and is there any graphics hardware acceleration available with GStreamer > using CUDA, OpenCL or similar? There is not opencl for linux as far as I know. Besides all the gfx accelerated video stuff is quite new on linux, as hardware vendors are not really supportive here. > 4) In your opinions is this something that would be worth setting up as > an open source project in the future once I have done the ground work? Sure, sounds like fun. I see some overlap with flumotion though. Stefan > > Many thanks in advance for any help and feedback! > Nick > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry(R) Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9 - 12, 2009. Register now! > http://p.sf.net/sfu/devconference > > > ------------------------------------------------------------------------ > > _______________________________________________ > gstreamer-devel mailing list > [hidden email] > https://lists.sourceforge.net/lists/listinfo/gstreamer-devel ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
|
|
2009/10/27 Stefan Kost <[hidden email]>
Nick Daniels schrieb: Hi, Just a note about that. Actually, I have tested OpenCL on a kubuntu, last week. It requires to install the last nvidia driver which is a beta: ( http://www.nvidia.com/object/cuda_opencl.html ) Linux
I tried some of the examples and it works pretty well.Anyway, it's not used in gstreamer yet. Julien ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
|
|
In reply to this post by Stefan Sauer
Hi,
> This is quite a project. Not sure if it fits a final year project. I am not a > video expert, but answer as no one else did so far. I agree it is a large scope ; i tried something similar in the past. What i found out is that synchronization and mixing from network sources is really a challenge with Gstreamer (according to my experience), especially with videomixer etc... >> 1) Is live mixing of video streams going to be possible using the >> GStreamer framework and are there any suggestions on how I should look >> into implementing this? Can I simply cut between pipeline frames once >> the video has been decoded? It depends on what you will choose; if you need 'videomixer', then each of your input streams will have to be decoded and keyed before mixing (e.g. using 'alpha'); you could also try the same with 'switcher', although i never tried to use it. Also, audio will have to be downmixed by using 'adder' plus 'volume' on input streams for example... If you can cope with raw DV, you could try switching the DV packets in real time in gstreamer (like [1]). Seems like an easier project to me. This way, you could have a simple pipeline, which would also be very light in processing. However, you will not be able to have swipes/openGL effects etc... (which would need decoding, mixing and reencoding in any case). >> 3) Low latency is desirable, how powerful a PC would be required for >> this decoding, mixing and recoding multiple low-res live video streams How many streams ? Which resolution and framerate, origin codec ? Which target codec ? These questions only make sense with numbers, but expect the need for powerful machines. As for latency, it will depend of the number of filters and operations you will have between the senders and the encoder. >> and is there any graphics hardware acceleration available with GStreamer >> using CUDA, OpenCL or similar? Keep in mind that gstreamer is heavily multi-threaded (and able), which is already a nice feature performance-wise. There currently are some efforts to integrate deblocking and debayering in opengl in gst-plugins-gl for Elphel cameras with GLSL, you have some hardware-accelerated colorspace conversion in some plugins (cluttergst, glupload)... And nothing prevents you to try coding the first OpenCL gstreamer plugin. In other terms, there are alternatives for accelerated processing. > Besides all the gfx accelerated > video stuff is quite new on linux, as hardware vendors are not really supportive > here. Most of the hardware graphics vendors are accelerating stream decoding and display (vdpau/...), there are currently no linux-supported encoding features (AFAIK). Some input hardware can be found on [2]. Cheers Florent [1] http://dvswitch.alioth.debian.org/wiki/ [2] http://www.mythtv.org/wiki/Category:Hardware ------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
|
|
In reply to this post by Nick Daniels
Hi,
Many thanks for the replies, they have been extremely useful! >> This is quite a project. Not sure if it fits a final year project. I am not a >> video expert, but answer as no one else did so far. > I agree it is a large scope ; i tried something similar in the past. > What i found out is that synchronization and mixing from network > sources is really a challenge with Gstreamer (according to my > experience), especially with videomixer etc... Taking these comments into account I've decided to reduce the scope of the project to focus solely on the network stream mixing and synchronisation problem. I will write the code with future features such as graphics, fades etc. in mind but by the sounds of it I won't have time to implement them before my project deadline.
>>> 3) Low latency is desirable, how powerful a PC would be required for >>> this decoding, mixing and recoding multiple low-res live video streams > How many streams ? Which resolution and framerate, origin codec ? > Which target codec ? These questions only make sense with numbers, but > expect the need for powerful machines. As for latency, it will depend > of the number of filters and operations you will have between the > senders and the encoder. Yeh this was a pretty vague question wasn't it! I suppose this is something I really need to wait for, and test out once I've written some of the code.
I have almost finished my initial research and should be starting to code next week hopefully...I expect I'll be back with more specific problems then!
Many thanks,
Nick
------------------------------------------------------------------------------ Come build with us! The BlackBerry(R) Developer Conference in SF, CA is the only developer event you need to attend this year. Jumpstart your developing skills, take BlackBerry mobile applications to market and stay ahead of the curve. Join us from November 9 - 12, 2009. Register now! http://p.sf.net/sfu/devconference _______________________________________________ gstreamer-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gstreamer-devel |
«
Return to GStreamer-devel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |