vaapih264enc+hlssink output not playable in browsers

[Uwe Müller]

Hello,

I'm trying to encode a videostream to h.264, mux it with mpegtsmux and
use hlssink to generate a hls stream. The resulting files should be
playable by all common used browsers (some using hlsjs). While this
works flawless with x264enc, the vaapi264enc output is not playable at
all as soon as a browser is involved. When using hlsjs to play, then the
library produces "mediaError" events with no futher information about
it. A hls capable browser like safari just displays a black picture (but
plays sound if there is any soundtrack).
I was able to narrow the problem down to the vaapih264enc-element by
creating gst-launch lines looking like this:

    gst-launch-1.0 videotestsrc ! [ENCODER] ! h264parse ! mpegtsmux !
hlssink

by placing "x264enc" as encoder, everything works as expected. Placing
"vaapih264enc" produces the described mediaError / black picture
problem. I tried a bunch of different properties of vaapih264enc, but no
combination changes anything.

Then I created yet another gst-launch line that uses ffmpeg's
"h264_vaapi" as encoder:

    gst-launch-1.0 -q videotestsrc ! 'video/x-raw' ! matroskamux !
filesink location=/dev/stdout \
        | ffmpeg -vaapi_device /dev/dri/renderD128 -i - -vf
'format=nv12,hwupload' -c:v h264_vaapi -f mpegts - \
        | gst-launch-1.0 filesrc location=/dev/stdin ! tsdemux !
mpegtsmux ! hlssink

When running this i can see a +60% usage in intel_gpu_top, so it does in
fact use the gpu and not cpu. The files created by this are playable on
all my tested devices.
I created a few seconds with the 3 commands and uploaded it to a
webserver, including a flowplayer and video-tag version for safari:
http://88.99.27.80/gstreamer-hls/

Tested on two different systems with the same results:
gstreamer 1.10.2 (pkg from debian sid), libva 1.7.1 on ubuntu 16.10
(i7-6700 Skylake with HD Graphics 530)
gstreamer 1.11.0 (GIT), libva 1.7.3 on archlinux (i7-3615QM IvyBridge
with HD Graphics 4000)

Is there anything wrong with my command or is there a problem in the
vaapi plugin?

 -Uwe

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Victor Jaquez]

On 12/26/16 at 10:12pm, Uwe Müller wrote:

> Hello,
>
> I'm trying to encode a videostream to h.264, mux it with mpegtsmux and
> use hlssink to generate a hls stream. The resulting files should be
> playable by all common used browsers (some using hlsjs). While this
> works flawless with x264enc, the vaapi264enc output is not playable at
> all as soon as a browser is involved. When using hlsjs to play, then the
> library produces "mediaError" events with no futher information about
> it. A hls capable browser like safari just displays a black picture (but
> plays sound if there is any soundtrack).
> I was able to narrow the problem down to the vaapih264enc-element by
> creating gst-launch lines looking like this:
>
>     gst-launch-1.0 videotestsrc ! [ENCODER] ! h264parse ! mpegtsmux !
> hlssink
>
> by placing "x264enc" as encoder, everything works as expected. Placing
> "vaapih264enc" produces the described mediaError / black picture
> problem. I tried a bunch of different properties of vaapih264enc, but no
> combination changes anything.
>
> Then I created yet another gst-launch line that uses ffmpeg's
> "h264_vaapi" as encoder:
>
>     gst-launch-1.0 -q videotestsrc ! 'video/x-raw' ! matroskamux !
> filesink location=/dev/stdout \
>         | ffmpeg -vaapi_device /dev/dri/renderD128 -i - -vf
> 'format=nv12,hwupload' -c:v h264_vaapi -f mpegts - \
>         | gst-launch-1.0 filesrc location=/dev/stdin ! tsdemux !
> mpegtsmux ! hlssink
>
> When running this i can see a +60% usage in intel_gpu_top, so it does in
> fact use the gpu and not cpu. The files created by this are playable on
> all my tested devices.
> I created a few seconds with the 3 commands and uploaded it to a
> webserver, including a flowplayer and video-tag version for safari:
> http://88.99.27.80/gstreamer-hls/

If I open http://88.99.27.80/gstreamer-hls/vaapi/flowplayer.html with
Epiphany, it works.

With gst-play-1.0 http://88.99.27.80/gstreamer-hls/vaapi/playlist.m3u8 works
too.

If I play it with mpv (ffmeg) I got a segfault:

mpv http://88.99.27.80/gstreamer-hls/vaapi/playlist.m3u8
Playing: http://88.99.27.80/gstreamer-hls/vaapi/playlist.m3u8
[ffmpeg/video] h264: sps_id 0 out of range
[ffmpeg/video] h264: sps_id 0 out of range
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332108999, dts=332109000, size=3658
[ffmpeg/demuxer] mpegts: DTS discontinuity in stream 0: packet 3 with DTS 332109000, packet 4 with DTS 332111999
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332126999, dts=332127000, size=3632
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332144999, dts=332145000, size=3626
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332162999, dts=332163000, size=3669
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332180999, dts=332181000, size=3615
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332207999, dts=332208000, size=3629
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332225999, dts=332226000, size=3642
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332243999, dts=332244000, size=3641
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332261999, dts=332262000, size=3641
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332297999, dts=332298000, size=3664
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332315999, dts=332316000, size=3633
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332333999, dts=332334000, size=3644
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332351999, dts=332352000, size=3647
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332387999, dts=332388000, size=3665
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332405999, dts=332406000, size=3637
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332423999, dts=332424000, size=3657
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332441999, dts=332442000, size=3620
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332477999, dts=332478000, size=3654
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332495999, dts=332496000, size=3634
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332513999, dts=332514000, size=3624
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332531999, dts=332532000, size=3658
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332567999, dts=332568000, size=3636
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332585999, dts=332586000, size=3628
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332603999, dts=332604000, size=3639
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332621999, dts=332622000, size=3641
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332657999, dts=332658000, size=3659
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332675999, dts=332676000, size=3618
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332693999, dts=332694000, size=3642
[ffmpeg/demuxer] mpegts: Invalid timestamps stream=0, pts=332711999, dts=332712000, size=3650
[ffmpeg/video] h264: sps_id 0 out of range
[ffmpeg/demuxer] hls,applehttp: Invalid timestamps stream=0, pts=332108999, dts=332109000, size=3672
[ffmpeg/demuxer] hls,applehttp: DTS discontinuity in stream 0: packet 3 with DTS 332109000, packet 4 with DTS 332111999
[ffmpeg/demuxer] hls,applehttp: Invalid timestamps stream=0, pts=332126999, dts=332127000, size=3646
[ffmpeg/demuxer] hls,applehttp: Invalid timestamps stream=0, pts=332144999, dts=332145000, size=3640
 (+) Video --vid=1 (h264)
[ffmpeg/video] h264: sps_id 0 out of range
Could not open codec.
[ffmpeg/video] h264: sps_id 0 out of range
Could not open codec.
Video decoder init failed for lavc:h264
Segmentation fault (core dumped)

ffpmeg doesn't like the sps and the timestamps

>
> Tested on two different systems with the same results:
> gstreamer 1.10.2 (pkg from debian sid), libva 1.7.1 on ubuntu 16.10
> (i7-6700 Skylake with HD Graphics 530)
> gstreamer 1.11.0 (GIT), libva 1.7.3 on archlinux (i7-3615QM IvyBridge
> with HD Graphics 4000)
>
> Is there anything wrong with my command or is there a problem in the
> vaapi plugin?

I guess is the second. Perhaps the vaapi encoders are not smart enough
handling incomming timestamps. Can you fill a bug in bugzilla?

https://bugzilla.gnome.org/enter_bug.cgi?product=GStreamer&component=gstreamer-vaapi



>
>  -Uwe
>
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim Müller]

On Mon, 2016-12-26 at 22:12 +0100, Uwe Müller wrote:


Hi Uwe,

> I was able to narrow the problem down to the vaapih264enc-element by
> creating gst-launch lines looking like this:
>
>     gst-launch-1.0 videotestsrc ! [ENCODER] ! h264parse ! mpegtsmux !
> hlssink
>
> by placing "x264enc" as encoder, everything works as expected.
> Placing
> "vaapih264enc" produces the described mediaError / black picture
> problem. I tried a bunch of different properties of vaapih264enc, but
> no combination changes anything.
> (...)
> Is there anything wrong with my command or is there a problem in the
> vaapi plugin?

I suspect the problem is that vaapih264enc doesn't output AU delimiters
 by default, and it also seems to have no property for it ("aud"
property on x264enc which is set to true by default).

Compare working and non-working .ts segments with:

gst-launch-1.0 filesrc location=segment00000.ts ! tsdemux ! video/x-
h264 ! fakesink dump=true | grep ^00000000 | head

Note the 00 00 00 01 09 XX at the beginning of each packet/frame here.

Cheers
 -Tim

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Victor Jaquez]

On 12/28/16 at 07:39pm, Tim Müller wrote:

> On Mon, 2016-12-26 at 22:12 +0100, Uwe Müller wrote:
>
>
> Hi Uwe,
>
> > I was able to narrow the problem down to the vaapih264enc-element by
> > creating gst-launch lines looking like this:
> >
> >     gst-launch-1.0 videotestsrc ! [ENCODER] ! h264parse ! mpegtsmux !
> > hlssink
> >
> > by placing "x264enc" as encoder, everything works as expected.
> > Placing
> > "vaapih264enc" produces the described mediaError / black picture
> > problem. I tried a bunch of different properties of vaapih264enc, but
> > no combination changes anything.
> > (...)
> > Is there anything wrong with my command or is there a problem in the
> > vaapi plugin?
>
> I suspect the problem is that vaapih264enc doesn't output AU delimiters
>  by default, and it also seems to have no property for it ("aud"
> property on x264enc which is set to true by default).
>
> Compare working and non-working .ts segments with:
>
> gst-launch-1.0 filesrc location=segment00000.ts ! tsdemux ! video/x-
> h264 ! fakesink dump=true | grep ^00000000 | head
>
> Note the 00 00 00 01 09 XX at the beginning of each packet/frame here.

Thanks Tim!

If Uwe hasn't opened a bug, I will open one.

vmjl

>
> Cheers
>  -Tim
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Uwe Müller]

Thank you both for your help!

For the archive, the bug report including a workaround can be found
here: https://bugzilla.gnome.org/show_bug.cgi?id=776712

 -Uwe


On 29.12.2016 at 14:06, Víctor M. Jáquez L. wrote:

> On 12/28/16 at 07:39pm, Tim Müller wrote:
>> On Mon, 2016-12-26 at 22:12 +0100, Uwe Müller wrote:
>>
>>
>> Hi Uwe,
>>
>>> I was able to narrow the problem down to the vaapih264enc-element by
>>> creating gst-launch lines looking like this:
>>>
>>>     gst-launch-1.0 videotestsrc ! [ENCODER] ! h264parse ! mpegtsmux !
>>> hlssink
>>>
>>> by placing "x264enc" as encoder, everything works as expected.
>>> Placing
>>> "vaapih264enc" produces the described mediaError / black picture
>>> problem. I tried a bunch of different properties of vaapih264enc, but
>>> no combination changes anything.
>>> (...)
>>> Is there anything wrong with my command or is there a problem in the
>>> vaapi plugin?
>> I suspect the problem is that vaapih264enc doesn't output AU delimiters
>>  by default, and it also seems to have no property for it ("aud"
>> property on x264enc which is set to true by default).
>>
>> Compare working and non-working .ts segments with:
>>
>> gst-launch-1.0 filesrc location=segment00000.ts ! tsdemux ! video/x-
>> h264 ! fakesink dump=true | grep ^00000000 | head
>>
>> Note the 00 00 00 01 09 XX at the beginning of each packet/frame here.
> Thanks Tim!
>
> If Uwe hasn't opened a bug, I will open one.
>
> vmjl
>
>> Cheers
>>  -Tim
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel