GStreamer-devel

A question about h264 encoder

Classic

List

Threaded

8 messages Options

张若

A question about h264 encoder

Hi,

I am working on the imx6 board. There are two h264/x264 encoder, HW encoder named imxvpuenc_h264 and SW encoder named x264enc. I tried both of them. But problem appears.

The pipeline looks like this: (the video camera capture is 720p)

imxv4l2videosrc -> capsfilter1 -> videoscale -> videorate -> videoconvert -> capsfilter2 -> imxvpuenc_h264 (or x264enc) -> queue -> h264parse -> mpegtsmux -> rtpmp2tpay -> udpsink

While using imxvpuenc_h264, there is no large latency, but using x264enc will cause a really large latency (sometimes around 10s and frame stucks).

When I set "capsfilter2" to lower resolution such as 320x240, the latency comes much shorter for x264enc. On the other hand, if I remove the "videoscale -> videorate -> videoconvert" part while using x264enc, there almost no latency for the streaming.

For x264enc, I set "tune=zerolatency, key-int-max=32, bframes=0, speed-preset=1, bitrate=4000 (may changes from 1000 to 10000)".

I don't know the reason why it happens and how can I deal with that. Please help!

Thanks in advance!

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

张若

Re: A question about h264 encoder

I found the solution myself. But still not understand the reason thoroughly.

The problem is with the videorate, it needs to be set with drop-only=true, especially for the board. Otherwise, it seems to duplicate several frames and cause a large delay as well.

But I still don't why does this happen? and why there is no problem while using imxvpuenc_h264?

Hope somebody could explain that!

Thanks!

2016-10-13 11:32 GMT+02:00 张若 <[hidden email]>:

Hi,

I am working on the imx6 board. There are two h264/x264 encoder, HW encoder named imxvpuenc_h264 and SW encoder named x264enc. I tried both of them. But problem appears.

The pipeline looks like this: (the video camera capture is 720p)

imxv4l2videosrc -> capsfilter1 -> videoscale -> videorate -> videoconvert -> capsfilter2 -> imxvpuenc_h264 (or x264enc) -> queue -> h264parse -> mpegtsmux -> rtpmp2tpay -> udpsink

While using imxvpuenc_h264, there is no large latency, but using x264enc will cause a really large latency (sometimes around 10s and frame stucks).

When I set "capsfilter2" to lower resolution such as 320x240, the latency comes much shorter for x264enc. On the other hand, if I remove the "videoscale -> videorate -> videoconvert" part while using x264enc, there almost no latency for the streaming.

For x264enc, I set "tune=zerolatency, key-int-max=32, bframes=0, speed-preset=1, bitrate=4000 (may changes from 1000 to 10000)".

I don't know the reason why it happens and how can I deal with that. Please help!

Thanks in advance!

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Nuno Mota

Re: A question about h264 encoder

It is quite simple.

imxvpuenc_h264 uses imx hardware encoder.
x264enc uses the cpu, so It's software encoder. You wont be able to use it on an ARM CPU. Imx just cant handle it.

Check "top" and you will se the cpu load/usage...

On Oct 13, 2016 1:17 PM, "张若" <[hidden email]> wrote:

I found the solution myself. But still not understand the reason thoroughly.

The problem is with the videorate, it needs to be set with drop-only=true, especially for the board. Otherwise, it seems to duplicate several frames and cause a large delay as well.
But I still don't why does this happen? and why there is no problem while using imxvpuenc_h264?

Hope somebody could explain that!

Thanks!

2016-10-13 11:32 GMT+02:00 张若 <[hidden email]>:
Hi,

I am working on the imx6 board. There are two h264/x264 encoder, HW encoder named imxvpuenc_h264 and SW encoder named x264enc. I tried both of them. But problem appears.

The pipeline looks like this: (the video camera capture is 720p)

imxv4l2videosrc -> capsfilter1 -> videoscale -> videorate -> videoconvert -> capsfilter2 -> imxvpuenc_h264 (or x264enc) -> queue -> h264parse -> mpegtsmux -> rtpmp2tpay -> udpsink

While using imxvpuenc_h264, there is no large latency, but using x264enc will cause a really large latency (sometimes around 10s and frame stucks).

When I set "capsfilter2" to lower resolution such as 320x240, the latency comes much shorter for x264enc. On the other hand, if I remove the "videoscale -> videorate -> videoconvert" part while using x264enc, there almost no latency for the streaming.

For x264enc, I set "tune=zerolatency, key-int-max=32, bframes=0, speed-preset=1, bitrate=4000 (may changes from 1000 to 10000)".

I don't know the reason why it happens and how can I deal with that. Please help!

Thanks in advance!

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

张若

Re: A question about h264 encoder

Anyway, thanks!

But.. the board has both HW and SW 264 encoder on it. There is no problem to use x264enc.

The problem is with videorate. If I use videorate and x264enc without setting drop-only as true, there will be problems while streaming. But that works fine without videorate, or works fine with using imxvpuenc_h264 instead.

I don't know why. I think that could be some problems with framerate, either with camera or encoder.

2016-10-13 14:26 GMT+02:00 Nuno Mota <[hidden email]>:

It is quite simple.

imxvpuenc_h264 uses imx hardware encoder.
x264enc uses the cpu, so It's software encoder. You wont be able to use it on an ARM CPU. Imx just cant handle it.

Check "top" and you will se the cpu load/usage...

On Oct 13, 2016 1:17 PM, "张若" <[hidden email]> wrote:
I found the solution myself. But still not understand the reason thoroughly.

The problem is with the videorate, it needs to be set with drop-only=true, especially for the board. Otherwise, it seems to duplicate several frames and cause a large delay as well.
But I still don't why does this happen? and why there is no problem while using imxvpuenc_h264?

Hope somebody could explain that!

Thanks!

2016-10-13 11:32 GMT+02:00 张若 <[hidden email]>:
Hi,

I am working on the imx6 board. There are two h264/x264 encoder, HW encoder named imxvpuenc_h264 and SW encoder named x264enc. I tried both of them. But problem appears.

The pipeline looks like this: (the video camera capture is 720p)

imxv4l2videosrc -> capsfilter1 -> videoscale -> videorate -> videoconvert -> capsfilter2 -> imxvpuenc_h264 (or x264enc) -> queue -> h264parse -> mpegtsmux -> rtpmp2tpay -> udpsink

While using imxvpuenc_h264, there is no large latency, but using x264enc will cause a really large latency (sometimes around 10s and frame stucks).

When I set "capsfilter2" to lower resolution such as 320x240, the latency comes much shorter for x264enc. On the other hand, if I remove the "videoscale -> videorate -> videoconvert" part while using x264enc, there almost no latency for the streaming.

For x264enc, I set "tune=zerolatency, key-int-max=32, bframes=0, speed-preset=1, bitrate=4000 (may changes from 1000 to 10000)".

I don't know the reason why it happens and how can I deal with that. Please help!

Thanks in advance!

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Tonu Jaansoo-3

Re: A question about h264 encoder

Why would you want to NOT use HW encoder, if it is available and working? As it was pointed out, CPU can't handle encoding the stream in realtime from source such as as a camera. (thats why there is HW encoder)

Tõnu

On 13 Oct 2016, at 14:58, 张若 <[hidden email]> wrote:

Anyway, thanks!
But.. the board has both HW and SW 264 encoder on it. There is no problem to use x264enc.
The problem is with videorate. If I use videorate and x264enc without setting drop-only as true, there will be problems while streaming. But that works fine without videorate, or works fine with using imxvpuenc_h264 instead.
I don't know why. I think that could be some problems with framerate, either with camera or encoder.

2016-10-13 14:26 GMT+02:00 Nuno Mota <[hidden email]>:
It is quite simple.
imxvpuenc_h264 uses imx hardware encoder.
x264enc uses the cpu, so It's software encoder. You wont be able to use it on an ARM CPU. Imx just cant handle it.
Check "top" and you will se the cpu load/usage...

On Oct 13, 2016 1:17 PM, "张若" <[hidden email]> wrote:
I found the solution myself. But still not understand the reason thoroughly.

The problem is with the videorate, it needs to be set with drop-only=true, especially for the board. Otherwise, it seems to duplicate several frames and cause a large delay as well.
But I still don't why does this happen? and why there is no problem while using imxvpuenc_h264?

Hope somebody could explain that!

Thanks!

2016-10-13 11:32 GMT+02:00 张若 <[hidden email]>:
Hi,

I am working on the imx6 board. There are two h264/x264 encoder, HW encoder named imxvpuenc_h264 and SW encoder named x264enc. I tried both of them. But problem appears.

The pipeline looks like this: (the video camera capture is 720p)

imxv4l2videosrc -> capsfilter1 -> videoscale -> videorate -> videoconvert -> capsfilter2 -> imxvpuenc_h264 (or x264enc) -> queue -> h264parse -> mpegtsmux -> rtpmp2tpay -> udpsink

While using imxvpuenc_h264, there is no large latency, but using x264enc will cause a really large latency (sometimes around 10s and frame stucks).

When I set "capsfilter2" to lower resolution such as 320x240, the latency comes much shorter for x264enc. On the other hand, if I remove the "videoscale -> videorate -> videoconvert" part while using x264enc, there almost no latency for the streaming.

For x264enc, I set "tune=zerolatency, key-int-max=32, bframes=0, speed-preset=1, bitrate=4000 (may changes from 1000 to 10000)".

I don't know the reason why it happens and how can I deal with that. Please help!

Thanks in advance!

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

张若

Re: A question about h264 encoder

Actually, we need to use both the SW and HW encoder at the same time. And we want the SW encoder for streaming and HW encoder for storage.

The SW encoder can have more options and parameters to control and change, especially for a low delay. That's why we want to use SW for real time streaming to client.

The HW encoder will only be used for on board sd card storage of capture video.

2016-10-13 16:17 GMT+02:00 Tonu Jaansoo <[hidden email]>:

Why would you want to NOT use HW encoder, if it is available and working? As it was pointed out, CPU can't handle encoding the stream in realtime from source such as as a camera. (thats why there is HW encoder)

Tõnu

On 13 Oct 2016, at 14:58, 张若 <[hidden email]> wrote:

Anyway, thanks!
But.. the board has both HW and SW 264 encoder on it. There is no problem to use x264enc.
The problem is with videorate. If I use videorate and x264enc without setting drop-only as true, there will be problems while streaming. But that works fine without videorate, or works fine with using imxvpuenc_h264 instead.
I don't know why. I think that could be some problems with framerate, either with camera or encoder.

2016-10-13 14:26 GMT+02:00 Nuno Mota <[hidden email]>:
It is quite simple.
imxvpuenc_h264 uses imx hardware encoder.
x264enc uses the cpu, so It's software encoder. You wont be able to use it on an ARM CPU. Imx just cant handle it.
Check "top" and you will se the cpu load/usage...

On Oct 13, 2016 1:17 PM, "张若" <[hidden email]> wrote:
I found the solution myself. But still not understand the reason thoroughly.

The problem is with the videorate, it needs to be set with drop-only=true, especially for the board. Otherwise, it seems to duplicate several frames and cause a large delay as well.
But I still don't why does this happen? and why there is no problem while using imxvpuenc_h264?

Hope somebody could explain that!

Thanks!

2016-10-13 11:32 GMT+02:00 张若 <[hidden email]>:
Hi,

I am working on the imx6 board. There are two h264/x264 encoder, HW encoder named imxvpuenc_h264 and SW encoder named x264enc. I tried both of them. But problem appears.

The pipeline looks like this: (the video camera capture is 720p)

imxv4l2videosrc -> capsfilter1 -> videoscale -> videorate -> videoconvert -> capsfilter2 -> imxvpuenc_h264 (or x264enc) -> queue -> h264parse -> mpegtsmux -> rtpmp2tpay -> udpsink

While using imxvpuenc_h264, there is no large latency, but using x264enc will cause a really large latency (sometimes around 10s and frame stucks).

When I set "capsfilter2" to lower resolution such as 320x240, the latency comes much shorter for x264enc. On the other hand, if I remove the "videoscale -> videorate -> videoconvert" part while using x264enc, there almost no latency for the streaming.

For x264enc, I set "tune=zerolatency, key-int-max=32, bframes=0, speed-preset=1, bitrate=4000 (may changes from 1000 to 10000)".

I don't know the reason why it happens and how can I deal with that. Please help!

Thanks in advance!

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Thornton, Keith

AW: A question about h264 encoder

Hi, why do you not just encode once and then tee so that you can write to file on one branch and stream using the other branch. Do you need different sizes or framerates, or do you just need to puch the second branch through H264parse to recode it from avc to byte-stream

Regards

Keith

Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von ??
Gesendet: Donnerstag, 13. Oktober 2016 16:32
An: Discussion of the development of and with GStreamer <[hidden email]>
Betreff: Re: A question about h264 encoder

Actually, we need to use both the SW and HW encoder at the same time. And we want the SW encoder for streaming and HW encoder for storage.

The SW encoder can have more options and parameters to control and change, especially for a low delay. That's why we want to use SW for real time streaming to client.

The HW encoder will only be used for on board sd card storage of capture video.

2016-10-13 16:17 GMT+02:00 Tonu Jaansoo <[hidden email]>:

Why would you want to NOT use HW encoder, if it is available and working? As it was pointed out, CPU can't handle encoding the stream in realtime from source such as as a camera. (thats why there is HW encoder)

Tõnu

On 13 Oct 2016, at 14:58, 张若 <[hidden email]> wrote:

Anyway, thanks!

But.. the board has both HW and SW 264 encoder on it. There is no problem to use x264enc.

The problem is with videorate. If I use videorate and x264enc without setting drop-only as true, there will be problems while streaming. But that works fine without videorate, or works fine with using imxvpuenc_h264 instead.

I don't know why. I think that could be some problems with framerate, either with camera or encoder.

2016-10-13 14:26 GMT+02:00 Nuno Mota <[hidden email]>:

It is quite simple.

imxvpuenc_h264 uses imx hardware encoder.
x264enc uses the cpu, so It's software encoder. You wont be able to use it on an ARM CPU. Imx just cant handle it.

Check "top" and you will se the cpu load/usage...

On Oct 13, 2016 1:17 PM, "张若" <[hidden email]> wrote:

I found the solution myself. But still not understand the reason thoroughly.

The problem is with the videorate, it needs to be set with drop-only=true, especially for the board. Otherwise, it seems to duplicate several frames and cause a large delay as well.

But I still don't why does this happen? and why there is no problem while using imxvpuenc_h264?

Hope somebody could explain that!

Thanks!

2016-10-13 11:32 GMT+02:00 张若 <[hidden email]>:

Hi,

I am working on the imx6 board. There are two h264/x264 encoder, HW encoder named imxvpuenc_h264 and SW encoder named x264enc. I tried both of them. But problem appears.

The pipeline looks like this: (the video camera capture is 720p)

imxv4l2videosrc -> capsfilter1 -> videoscale -> videorate -> videoconvert -> capsfilter2 -> imxvpuenc_h264 (or x264enc) -> queue -> h264parse -> mpegtsmux -> rtpmp2tpay -> udpsink

While using imxvpuenc_h264, there is no large latency, but using x264enc will cause a really large latency (sometimes around 10s and frame stucks).

When I set "capsfilter2" to lower resolution such as 320x240, the latency comes much shorter for x264enc. On the other hand, if I remove the "videoscale -> videorate -> videoconvert" part while using x264enc, there almost no latency for the streaming.

For x264enc, I set "tune=zerolatency, key-int-max=32, bframes=0, speed-preset=1, bitrate=4000 (may changes from 1000 to 10000)".

I don't know the reason why it happens and how can I deal with that. Please help!

Thanks in advance!

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Nuno Mota

Re: A question about h264 encoder

imx6 is capable of more than one hardware encoding at the same time...

* There is a limit for the number of pipelines which can be run simultaneously, for high resolution input files, at most two for 1080p and four for 720p.

2016-10-14 9:54 GMT+01:00 Thornton, Keith <[hidden email]>:

Hi, why do you not just encode once and then tee so that you can write to file on one branch and stream using the other branch. Do you need different sizes or framerates, or do you just need to puch the second branch through H264parse to recode it from avc to byte-stream

Regards

Keith

Von: gstreamer-devel [mailto:[hidden email]] Im Auftrag von ??
Gesendet: Donnerstag, 13. Oktober 2016 16:32
An: Discussion of the development of and with GStreamer <[hidden email]>
Betreff: Re: A question about h264 encoder

Actually, we need to use both the SW and HW encoder at the same time. And we want the SW encoder for streaming and HW encoder for storage.

The SW encoder can have more options and parameters to control and change, especially for a low delay. That's why we want to use SW for real time streaming to client.

The HW encoder will only be used for on board sd card storage of capture video.

2016-10-13 16:17 GMT+02:00 Tonu Jaansoo <[hidden email]>:

Why would you want to NOT use HW encoder, if it is available and working? As it was pointed out, CPU can't handle encoding the stream in realtime from source such as as a camera. (thats why there is HW encoder)

Tõnu

On 13 Oct 2016, at 14:58, 张若 <[hidden email]> wrote:

Anyway, thanks!

But.. the board has both HW and SW 264 encoder on it. There is no problem to use x264enc.

The problem is with videorate. If I use videorate and x264enc without setting drop-only as true, there will be problems while streaming. But that works fine without videorate, or works fine with using imxvpuenc_h264 instead.

I don't know why. I think that could be some problems with framerate, either with camera or encoder.

2016-10-13 14:26 GMT+02:00 Nuno Mota <[hidden email]>:

It is quite simple.

imxvpuenc_h264 uses imx hardware encoder.
x264enc uses the cpu, so It's software encoder. You wont be able to use it on an ARM CPU. Imx just cant handle it.

Check "top" and you will se the cpu load/usage...

On Oct 13, 2016 1:17 PM, "张若" <[hidden email]> wrote:

I found the solution myself. But still not understand the reason thoroughly.

The problem is with the videorate, it needs to be set with drop-only=true, especially for the board. Otherwise, it seems to duplicate several frames and cause a large delay as well.

But I still don't why does this happen? and why there is no problem while using imxvpuenc_h264?

Hope somebody could explain that!

Thanks!

2016-10-13 11:32 GMT+02:00 张若 <[hidden email]>:

Hi,

I am working on the imx6 board. There are two h264/x264 encoder, HW encoder named imxvpuenc_h264 and SW encoder named x264enc. I tried both of them. But problem appears.

The pipeline looks like this: (the video camera capture is 720p)

imxv4l2videosrc -> capsfilter1 -> videoscale -> videorate -> videoconvert -> capsfilter2 -> imxvpuenc_h264 (or x264enc) -> queue -> h264parse -> mpegtsmux -> rtpmp2tpay -> udpsink

While using imxvpuenc_h264, there is no large latency, but using x264enc will cause a really large latency (sometimes around 10s and frame stucks).

When I set "capsfilter2" to lower resolution such as 320x240, the latency comes much shorter for x264enc. On the other hand, if I remove the "videoscale -> videorate -> videoconvert" part while using x264enc, there almost no latency for the streaming.

For x264enc, I set "tune=zerolatency, key-int-max=32, bframes=0, speed-preset=1, bitrate=4000 (may changes from 1000 to 10000)".

I don't know the reason why it happens and how can I deal with that. Please help!

Thanks in advance!

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel