GStreamer-devel

Icecast a dv stream via ogg

Classic

List

Threaded

9 messages Options

matheist76

Icecast a dv stream via ogg

Hi, I'm wondering if anyone can help me. I want to, using gst-launch, send a dv (firewire) stream from a video camera convert it to an ogg and then send it to an icecast server.

I have tried several things. The main problem seems to lie with converting the dv video stream into a theora stream.

So I ran some tests. These I ran on an Macbook Pro with an Intel Dual core 2.4GHz Processor and 4GB of ram and my old Toshiba Dual Core 1.8GHz processor and 512MB of ram

So the tests were:

1. Just test the decoding of the dv stream.
gst-launch-0.10 dv1394src ! dvdemux ! dvdec ! fakesink

Mac: gives me 50% and 4% CPU usage. video quality unknown
Tosh: same

2. Do the full conversion.
gst-launch-0.10 dv1394src ! dvdemux ! dvdec ! video/x-raw-yuv ! ffmpegcolorspace ! theoraenc ! oggmux name=mux ! filesink location=test.ogg

Mac: That gives me 100% and 4% usage and real crap video.
Tosh: Same :-(

I have tried various other things like used queues and set the dv stream to drop frame rates however it ends up that it kinda works but the file plays for only 7 secs when I've recorded 30 secs worth or it records the video for the right length but misses showing most of the frames.

Matthew Franklin.

Edward Hervey

Re: Icecast a dv stream via ogg

Administrator

Hi,

On Tue, 2010-05-11 at 14:27 +0800, [hidden email] wrote:

> Hi, I'm wondering if anyone can help me. I want to, using gst-launch,
> send a dv (firewire) stream from a video camera convert it to an ogg
> and then send it to an icecast server.
>
> I have tried several things. The main problem seems to lie with
> converting the dv video stream into a theora stream.
>
> So I ran some tests. These I ran on an Macbook Pro with an Intel Dual
> core 2.4GHz Processor and 4GB of ram and my old Toshiba Dual Core
> 1.8GHz processor and 512MB of ram
>
> So the tests were:
>
> 1. Just test the decoding of the dv stream.
> gst-launch-0.10 dv1394src ! dvdemux ! dvdec ! fakesink

Try again with a queue between the source and the demuxer. Here only
one thread is being used to:
* wait for the next frame from the camera
* demux it
* decode it

Also, replace dvdec by ffdec_dvvideo (it's faster)
>
> Mac: gives me 50% and 4% CPU usage. video quality unknown
> Tosh: same
>
> 2. Do the full conversion.
> gst-launch-0.10 dv1394src ! dvdemux ! dvdec ! video/x-raw-yuv !
> ffmpegcolorspace ! theoraenc ! oggmux name=mux ! filesink
> location=test.ogg

Same as above, you're having it do *everything* in one thread
(capturing/demuxing/decoding/colorspace
conversion/encoding/muxing/writing). Regardless of having multiple
cores, you want to have a dedicated thread for:
* capturing
* decoding
* encoding

Replace dvdec by ffdec_dvvideo (provided with gst-ffmpeg), and you
don't need to specify you want video/x-raw-yuv (dv decodes to YUV and
theoraenc accepts YUV). I'm not even certain you need ffmpegcolorspace,
but let's leave it (won't take any cpu if it doesn't have to transform).

This pipeline should do it:
dv1394src ! queue ! dvdemux ! dvdec ! queue ! ffmpegcolorspace !
theoraenc ! queue ! oggmux ! filesink
>
> Mac: That gives me 100% and 4% usage and real crap video.
> Tosh: Same :-(
>
> I have tried various other things like used queues and set the dv
> stream to drop frame rates however it ends up that it kinda works but
> the file plays for only 7 secs when I've recorded 30 secs worth or it
> records the video for the right length but misses showing most of the
> frames.

Try again with the advices above, if the output is still garbled, then
there's definitely a problem

Also mention what version of gstreamer+plugins you're using

Edward

>
> Matthew Franklin.
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

matheist76

Re: Icecast a dv stream via ogg

In reply to this post by matheist76

Hi Edward,
I've done what you have suggested and replaced dvdec with ffdec_dvvideo. I've also added in a queue between the source and the demuxer. It produces much the same thing except with a lower cpu usage.

I'm using gstreamer version 0.10.1 . I think the real problem is with the theora encoding. Are there any other plugins that will do the same job?

Matt

----- Original Message -----
From: "Edward Hervey" <[hidden email]>
To: "Discussion of the development of GStreamer" <[hidden email]>
Sent: Tuesday, 11 May, 2010 4:59:25 PM GMT +10:00 Brisbane
Subject: Re: [gst-devel] Icecast a dv stream via ogg

Hi,

On Tue, 2010-05-11 at 14:27 +0800, [hidden email] wrote:

------------------------------------------------------------------------------

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

Edward Hervey

Re: Icecast a dv stream via ogg

Administrator

On Tue, 2010-05-11 at 16:15 +0800, [hidden email] wrote:
> Hi Edward,
> I've done what you have suggested and replaced dvdec with ffdec_dvvideo. I've also added in a queue between the source and the demuxer. It produces much the same thing except with a lower cpu usage.
>
> I'm using gstreamer version 0.10.1 . I think the real problem is with the theora encoding. Are there any other plugins that will do the same job?

0.10.1 ? Seriously ? That's... 5 years old almost. Switch to a recent
(i.e >= 0.10.25) gstreamer and plugins. That's most likely the source of
all your issues.

Edward

>
> Matt
>
> ----- Original Message -----
> From: "Edward Hervey" <[hidden email]>
> To: "Discussion of the development of GStreamer" <[hidden email]>
> Sent: Tuesday, 11 May, 2010 4:59:25 PM GMT +10:00 Brisbane
> Subject: Re: [gst-devel] Icecast a dv stream via ogg
>
> Hi,
>
> On Tue, 2010-05-11 at 14:27 +0800, [hidden email] wrote:
> > Hi, I'm wondering if anyone can help me. I want to, using gst-launch,
> > send a dv (firewire) stream from a video camera convert it to an ogg
> > and then send it to an icecast server.
> >
> > I have tried several things. The main problem seems to lie with
> > converting the dv video stream into a theora stream.
> >
> > So I ran some tests. These I ran on an Macbook Pro with an Intel Dual
> > core 2.4GHz Processor and 4GB of ram and my old Toshiba Dual Core
> > 1.8GHz processor and 512MB of ram
> >
> > So the tests were:
> >
> > 1. Just test the decoding of the dv stream.
> > gst-launch-0.10 dv1394src ! dvdemux ! dvdec ! fakesink
>
> Try again with a queue between the source and the demuxer. Here only
> one thread is being used to:
> * wait for the next frame from the camera
> * demux it
> * decode it
>
> Also, replace dvdec by ffdec_dvvideo (it's faster)
> >
> > Mac: gives me 50% and 4% CPU usage. video quality unknown
> > Tosh: same
> >
> > 2. Do the full conversion.
> > gst-launch-0.10 dv1394src ! dvdemux ! dvdec ! video/x-raw-yuv !
> > ffmpegcolorspace ! theoraenc ! oggmux name=mux ! filesink
> > location=test.ogg
>
> Same as above, you're having it do *everything* in one thread
> (capturing/demuxing/decoding/colorspace
> conversion/encoding/muxing/writing). Regardless of having multiple
> cores, you want to have a dedicated thread for:
> * capturing
> * decoding
> * encoding
>
> Replace dvdec by ffdec_dvvideo (provided with gst-ffmpeg), and you
> don't need to specify you want video/x-raw-yuv (dv decodes to YUV and
> theoraenc accepts YUV). I'm not even certain you need ffmpegcolorspace,
> but let's leave it (won't take any cpu if it doesn't have to transform).
>
> This pipeline should do it:
> dv1394src ! queue ! dvdemux ! dvdec ! queue ! ffmpegcolorspace !
> theoraenc ! queue ! oggmux ! filesink
> >
> > Mac: That gives me 100% and 4% usage and real crap video.
> > Tosh: Same :-(
> >
> > I have tried various other things like used queues and set the dv
> > stream to drop frame rates however it ends up that it kinda works but
> > the file plays for only 7 secs when I've recorded 30 secs worth or it
> > records the video for the right length but misses showing most of the
> > frames.
>
> Try again with the advices above, if the output is still garbled, then
> there's definitely a problem
>
> Also mention what version of gstreamer+plugins you're using
>
>
> Edward
>
> >
> > Matthew Franklin.
> >
> >
> > ------------------------------------------------------------------------------
> >
> > _______________________________________________
> > gstreamer-devel mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

matheist76

Re: Icecast a dv stream via ogg

In reply to this post by matheist76

Sorry I gave you the version number. These are my real version numbers.

Gstreamer Base is 0.10.28-1
Gstreamer Ugly is 0.10.14-1
Gstreamer Good is 0.10.21-1ubuntu2

I am running Ubuntu 10.04 on the Toshiba and Ubuntu 9.10 on the Mac.

Thanks Edward.

Matt

----- Original Message -----
From: "Edward Hervey" <[hidden email]>
To: "Discussion of the development of GStreamer" <[hidden email]>
Sent: Tuesday, 11 May, 2010 7:36:39 PM GMT +10:00 Brisbane
Subject: Re: [gst-devel] Icecast a dv stream via ogg

On Tue, 2010-05-11 at 16:15 +0800, [hidden email] wrote:
> Hi Edward,
> I've done what you have suggested and replaced dvdec with ffdec_dvvideo. I've also added in a queue between the source and the demuxer. It produces much the same thing except with a lower cpu usage.
>
> I'm using gstreamer version 0.10.1 . I think the real problem is with the theora encoding. Are there any other plugins that will do the same job?

0.10.1 ? Seriously ? That's... 5 years old almost. Switch to a recent
(i.e >= 0.10.25) gstreamer and plugins. That's most likely the source of
all your issues.

Edward

Benjamin Schwartz

Re: Icecast a dv stream via ogg

In reply to this post by matheist76

On Tue, 2010-05-11 at 16:15 +0800, [hidden email] wrote:
> It produces much the same thing except with a lower cpu usage.

Is the CPU usage below 100% on all cores? Most gstreamer elements just
fail hard when the system runs out of CPU.

BTW, you really want a lot of queues: one before DV decode, another
before Theora encode, and another before writing to disk. You could
even have one between theoraenc and oggmux.

> I think the real problem is with the theora encoding. Are there any other plugins that will do the same job?

Nope, theoraenc is the one and only. If you post a sample output ogg we
might be better able to diagnose.

--Ben

------------------------------------------------------------------------------

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

matheist76

Re: Icecast a dv stream via ogg

In reply to this post by matheist76

Okay. I've attached some sample files.

test.ogg was created using this script, CPU 32% and 100%
gst-launch-0.10 dv1394src ! queue ! dvdemux ! queue ! ffdec_dvvideo ! queue ! theoraenc quality=4 ! oggmux name=mux ! filesink location=test.ogg

test2.ogg was created using a pre saved dv file, CPU 100 and 27%
gst-launch filesrc location=test.dv ! dvdemux ! queue ! ffdec_dvvideo ! queue ! theoraenc quality=4 ! oggmux name=mux ! filesink location=test2.ogg

And finally I created a mpg from the live dv stream. CPU 32 and 36%
gst-launch filesrc dv1394src ! dvdemux ! queue ! ffdec_dvvideo ! queue ! fenc_mpeg2video ! mpegtsmux ! filesink location=test.mpg

Thanks guys.

Matt

----- Original Message -----
From: "Ben Schwartz" <[hidden email]>
To: "Discussion of the development of GStreamer" <[hidden email]>
Sent: Tuesday, 11 May, 2010 11:45:24 PM GMT +10:00 Brisbane
Subject: Re: [gst-devel] Icecast a dv stream via ogg

On Tue, 2010-05-11 at 16:15 +0800, [hidden email] wrote:
> It produces much the same thing except with a lower cpu usage.

Is the CPU usage below 100% on all cores? Most gstreamer elements just
fail hard when the system runs out of CPU.

BTW, you really want a lot of queues: one before DV decode, another
before Theora encode, and another before writing to disk. You could
even have one between theoraenc and oggmux.

> I think the real problem is with the theora encoding. Are there any other plugins that will do the same job?

Nope, theoraenc is the one and only. If you post a sample output ogg we
might be better able to diagnose.

--Ben

------------------------------------------------------------------------------

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

test.ogg (3M) Download Attachment

test.mpg (1M) Download Attachment

test2.ogg (1M) Download Attachment

Benjamin Schwartz

Re: Icecast a dv stream via ogg

On Wed, 2010-05-12 at 02:48 +0800, [hidden email] wrote:
> test.ogg was created using this script, CPU 32% and 100%
> gst-launch-0.10 dv1394src ! queue ! dvdemux ! queue ! ffdec_dvvideo ! queue ! theoraenc quality=4 ! oggmux name=mux ! filesink location=test.ogg

> ----- Original Message -----
> From: "Ben Schwartz" <[hidden email]>
> Is the CPU usage below 100% on all cores? Most gstreamer elements just
> fail hard when the system runs out of CPU.

You don't have enough CPU to run theoraenc at 720x480x30fps in real
time, which is what ffdec_dvvideo is providing. You can try invoking
theoraenc with speed-level=2, which is far less bitrate-efficient but
about twice as fast.

Really, though, you should be downscaling the input. Specifically, you
should throw away one field, and then probably rescale to 320x240 before
encoding. That (with a queue) will give you enough CPU, and also get
rid of the terrible interlacing artifacts.

--Ben

------------------------------------------------------------------------------

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

matheist76

Re: Icecast a dv stream via ogg

Thanks for that. It's now working real well.

Matt.
----- Original Message -----
From: "Ben Schwartz" <[hidden email]>
To: "Discussion of the development of GStreamer" <[hidden email]>
Sent: Wednesday, 12 May, 2010 5:12:42 AM GMT +10:00 Brisbane
Subject: Re: [gst-devel] Icecast a dv stream via ogg

On Wed, 2010-05-12 at 02:48 +0800, [hidden email] wrote:
> test.ogg was created using this script, CPU 32% and 100%
> gst-launch-0.10 dv1394src ! queue ! dvdemux ! queue ! ffdec_dvvideo ! queue ! theoraenc quality=4 ! oggmux name=mux ! filesink location=test.ogg

> ----- Original Message -----
> From: "Ben Schwartz" <[hidden email]>
> Is the CPU usage below 100% on all cores? Most gstreamer elements just
> fail hard when the system runs out of CPU.

You don't have enough CPU to run theoraenc at 720x480x30fps in real
time, which is what ffdec_dvvideo is providing. You can try invoking
theoraenc with speed-level=2, which is far less bitrate-efficient but
about twice as fast.

Really, though, you should be downscaling the input. Specifically, you
should throw away one field, and then probably rescale to 320x240 before
encoding. That (with a queue) will give you enough CPU, and also get
rid of the terrible interlacing artifacts.

--Ben

------------------------------------------------------------------------------

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

------------------------------------------------------------------------------

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel