No video display when watching HLS streams in browsers

[Chien-Ming Cheng]

Hello

I simply converted a media file (container: Quicktime, audio: MPEG-4
AAC, video: H.264) to HLS stream as follows.

gst-launch-1.0 filesrc location=media.mp4 ! qtdemux name=demux !
queue2 ! h264parse disable-passthrough=true ! mpegtsmux name=mux !
hlssink max-files=0 playlist-length=0 demux. ! queue2 ! aacparse !
mux.

I can watch the HLS stream via the playlist.m3u8 by using media
players such as ffplay and vlc.

Then I created a web page which includes videojs-contrib-hls to play
the playlist.m3u8. However, only audio can be played (no video
display) when watching the HLS stream via http://address/index.html in
browsers such as Chrome and Safari. On the other hand, both audio and
video are fine via the same web page when replacing the HLS playlist
and segment files with that generated by ffplay program.

I have tested GStreamer 1.6.x, 1.9.x, and the latest 1.10.3. All have
the same result.

Does anyone know how to watch HLS streams generated by GStreamer in
browsers? Did I make any mistakes (for example, the wrong GStreamer
pipeline)?

Thanks
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Rand Graham]

Hello,

Can you clarify what platform you are using?

I have observed a case where the videos play in Chrome on Android but not in Safari on iOS. On iOS I get audio playback only and no video.

My pipe line is as follows

gst-launch-1.0 -v -e mpegtsmux name=m ! hlssink max-files=0 playlist-length=0 filesrc location=bees_hevc.ts ! tsdemux name=dem dem. ! queue ! h265parse ! omxh265dec ! nvvidconv ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420' ! omxh264enc profile=8 ! m. dem. ! queue ! avdec_aac ! audioconvert ! voaacenc ! m.

Regarding your pipeline, I could not get it to run because I get a no more pads error when trying it on my machine.

Regards,
Rand
________________________________________
From: gstreamer-devel [[hidden email]] On Behalf Of Chien-Ming Cheng [[hidden email]]
Sent: Monday, February 06, 2017 3:56 AM
To: [hidden email]
Subject: No video display when watching HLS streams in browsers

Hello

I simply converted a media file (container: Quicktime, audio: MPEG-4
AAC, video: H.264) to HLS stream as follows.

gst-launch-1.0 filesrc location=media.mp4 ! qtdemux name=demux !
queue2 ! h264parse disable-passthrough=true ! mpegtsmux name=mux !
hlssink max-files=0 playlist-length=0 demux. ! queue2 ! aacparse !
mux.

I can watch the HLS stream via the playlist.m3u8 by using media
players such as ffplay and vlc.

Then I created a web page which includes videojs-contrib-hls to play
the playlist.m3u8. However, only audio can be played (no video
display) when watching the HLS stream via http://address/index.html in
browsers such as Chrome and Safari. On the other hand, both audio and
video are fine via the same web page when replacing the HLS playlist
and segment files with that generated by ffplay program.

I have tested GStreamer 1.6.x, 1.9.x, and the latest 1.10.3. All have
the same result.

Does anyone know how to watch HLS streams generated by GStreamer in
browsers? Did I make any mistakes (for example, the wrong GStreamer
pipeline)?

Thanks
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Chien-Ming Cheng]

Hi Rand,


Thanks for your reply. Let me describe my steps and further
experimental results.

I run GStreamer on a desktop server with Ubuntu 14. My objective is to
play HLS stream via a webpage (for example, http://address/index.html)
with various browsers. I refer to VideoJS HLS at
http://videojs.github.io/videojs-contrib-hls for creating the webpage
to play HLS stream in browsers without native HLS support (e.g.
Chrome).

The pipeline described in my previous email should be OK (kindly let
me know if it is not). As mentioned, however, I get only audio
playback but NO video display in browsers.

I further changed the pipeline from "demux and mux" to "decode first,
encode, and then mux" as follows (it does not matter although I
changed AAC encoder to faac).

gst-launch-1.0 filesrc location=media.mp4 ! decodebin name=decode !
queue2 ! videoconvert ! x264enc ! h264parse disable-passthrough=true !
mpegtsmux name=mux ! hlssink max-files=0 playlist-length=0 decode. !
queue2 ! audioconvert ! faac ! mux.

The HLS stream can be played well (both audio & video) in browsers (in
Chrome on Linux and Safari on iOS).

What is the difference between the first pipeline (only transmuxing)
and the second one (transmuxing plus re-encoding)? Does the
re-encoding process fix some errors in video stream and/or add some
missing information to video stream so that the video can be
displayed?

Thanks for any help or comments.

ps. I've tested version 1.10.3 and the one built from master branch (1.11.1.1).


On Wed, Feb 8, 2017 at 4:12 AM, Rand Graham <[hidden email]> wrote:

> Hello,
>
> Can you clarify what platform you are using?
>
> I have observed a case where the videos play in Chrome on Android but not in Safari on iOS. On iOS I get audio playback only and no video.
>
> My pipe line is as follows
>
> gst-launch-1.0 -v -e mpegtsmux name=m ! hlssink max-files=0 playlist-length=0 filesrc location=bees_hevc.ts ! tsdemux name=dem dem. ! queue ! h265parse ! omxh265dec ! nvvidconv ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420' ! omxh264enc profile=8 ! m. dem. ! queue ! avdec_aac ! audioconvert ! voaacenc ! m.
>
> Regarding your pipeline, I could not get it to run because I get a no more pads error when trying it on my machine.
>
> Regards,
> Rand
> ________________________________________
> From: gstreamer-devel [[hidden email]] On Behalf Of Chien-Ming Cheng [[hidden email]]
> Sent: Monday, February 06, 2017 3:56 AM
> To: [hidden email]
> Subject: No video display when watching HLS streams in browsers
>
> Hello
>
> I simply converted a media file (container: Quicktime, audio: MPEG-4
> AAC, video: H.264) to HLS stream as follows.
>
> gst-launch-1.0 filesrc location=media.mp4 ! qtdemux name=demux !
> queue2 ! h264parse disable-passthrough=true ! mpegtsmux name=mux !
> hlssink max-files=0 playlist-length=0 demux. ! queue2 ! aacparse !
> mux.
>
> I can watch the HLS stream via the playlist.m3u8 by using media
> players such as ffplay and vlc.
>
> Then I created a web page which includes videojs-contrib-hls to play
> the playlist.m3u8. However, only audio can be played (no video
> display) when watching the HLS stream via http://address/index.html in
> browsers such as Chrome and Safari. On the other hand, both audio and
> video are fine via the same web page when replacing the HLS playlist
> and segment files with that generated by ffplay program.
>
> I have tested GStreamer 1.6.x, 1.9.x, and the latest 1.10.3. All have
> the same result.
>
> Does anyone know how to watch HLS streams generated by GStreamer in
> browsers? Did I make any mistakes (for example, the wrong GStreamer
> pipeline)?
>
> Thanks
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Rand Graham]

Hello,

It sounds like you have a working pipeline now?

RE: VideoJS

I have used VideoJS successfully in the past to play back HLS streams. One way to gather more information about your failure case is to open the JavaScript console in Chrome and reload the page that is trying to playback the video. If you receive any error messages from VideoJS, you may get more information about the problem.

RE: your previous pipeline

Like I said, it did not work for me because I got a "no more pads" error. I am not sure the exact cause of "no more pads" errors. I was using gstreamer 1.8.2.

RE: re encoding

One major difference that I can think of is the encoder that is being used. You are using some input file but you have not mentioned how the input file was created. Specifically what encoder was used?

Your updated pipeline is using the x264enc software encoder that comes with gstreamer.

Through google searching I have seen that some h.264 encoders have shown problems creating video that plays well in HLS,

Here is an example of a problem that someone ran into previously (and there doesn't seem to be a solution)

https://www.raspberrypi.org/forums/viewtopic.php?t=116690&p=1049440

Regards,
Rand
________________________________________
From: gstreamer-devel [[hidden email]] On Behalf Of Chien-Ming Cheng [[hidden email]]
Sent: Thursday, February 09, 2017 11:31 PM
To: Discussion of the development of and with GStreamer
Subject: Re: No video display when watching HLS streams in browsers

Hi Rand,


Thanks for your reply. Let me describe my steps and further
experimental results.

I run GStreamer on a desktop server with Ubuntu 14. My objective is to
play HLS stream via a webpage (for example, http://address/index.html)
with various browsers. I refer to VideoJS HLS at
http://videojs.github.io/videojs-contrib-hls for creating the webpage
to play HLS stream in browsers without native HLS support (e.g.
Chrome).

The pipeline described in my previous email should be OK (kindly let
me know if it is not). As mentioned, however, I get only audio
playback but NO video display in browsers.

I further changed the pipeline from "demux and mux" to "decode first,
encode, and then mux" as follows (it does not matter although I
changed AAC encoder to faac).

gst-launch-1.0 filesrc location=media.mp4 ! decodebin name=decode !
queue2 ! videoconvert ! x264enc ! h264parse disable-passthrough=true !
mpegtsmux name=mux ! hlssink max-files=0 playlist-length=0 decode. !
queue2 ! audioconvert ! faac ! mux.

The HLS stream can be played well (both audio & video) in browsers (in
Chrome on Linux and Safari on iOS).

What is the difference between the first pipeline (only transmuxing)
and the second one (transmuxing plus re-encoding)? Does the
re-encoding process fix some errors in video stream and/or add some
missing information to video stream so that the video can be
displayed?

Thanks for any help or comments.

ps. I've tested version 1.10.3 and the one built from master branch (1.11.1.1).



_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Tim Müller]

On Mon, 2017-02-06 at 17:56 +0800, Chien-Ming Cheng wrote:

Hi,

> I simply converted a media file (container: Quicktime, audio: MPEG-4
> AAC, video: H.264) to HLS stream as follows.
>
> gst-launch-1.0 filesrc location=media.mp4 ! qtdemux name=demux !
> queue2 ! h264parse disable-passthrough=true ! mpegtsmux name=mux !
> hlssink max-files=0 playlist-length=0 demux. ! queue2 ! aacparse !
> mux.
>
> I can watch the HLS stream via the playlist.m3u8 by using media
> players such as ffplay and vlc.
>
> Then I created a web page which includes videojs-contrib-hls to play
> the playlist.m3u8. However, only audio can be played (no video
> display) when watching the HLS stream via http://address/index.html
> in browsers such as Chrome and Safari. On the other hand, both audio
> and video are fine via the same web page when replacing the HLS
> playlist and segment files with that generated by ffplay program.
>
> I have tested GStreamer 1.6.x, 1.9.x, and the latest 1.10.3. All have
> the same result.
>
> Does anyone know how to watch HLS streams generated by GStreamer in
> browsers? Did I make any mistakes (for example, the wrong GStreamer
> pipeline)?

I suspect the problem is missing AU delimiters. They don't exist in the
mp4 h264 stream, and h264parse doesn't insert them either [0]. If you
use an encoder it will depend on your encoder whether they get inserted
by default or not. x264enc should insert them by default these days.

Apple devices dont't like HLS streams without AU delimiters very much
if I remember correctly.

Cheers
 -Tim

[0] see https://bugzilla.gnome.org/show_bug.cgi?id=736213 for a patch
which might help

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Rand Graham]

Tim,

You are correct. When I used a version of libgstomx.so that had support for AU delimiters, the video plays on iOS.

Thanks for your help.

Regards,
Rand
________________________________________
From: gstreamer-devel [[hidden email]] On Behalf Of Tim Müller [[hidden email]]
Sent: Friday, February 10, 2017 10:22 AM
To: [hidden email]
Subject: Re: No video display when watching HLS streams in browsers

I suspect the problem is missing AU delimiters. They don't exist in the
mp4 h264 stream, and h264parse doesn't insert them either [0]. If you
use an encoder it will depend on your encoder whether they get inserted
by default or not. x264enc should insert them by default these days.

Apple devices dont't like HLS streams without AU delimiters very much
if I remember correctly.

Cheers
 -Tim

[0] see https://bugzilla.gnome.org/show_bug.cgi?id=736213 for a patch
which might help

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Chien-Ming Cheng]

Hi,

On Sat, Feb 11, 2017 at 12:22 AM, Tim Müller <[hidden email]> wrote:
> I suspect the problem is missing AU delimiters. They don't exist in the
> mp4 h264 stream, and h264parse doesn't insert them either [0]. If you
> use an encoder it will depend on your encoder whether they get inserted
> by default or not. x264enc should insert them by default these days.

Tim,

Thank you for the reference [0] too. It does help.

Yes, AU delimiters do not exist in the H.264 stream. After the patch
[0] is applied to h264parse, the HLS stream can be played well in
Safari.

On Fri, Feb 10, 2017 at 11:40 PM, Rand Graham <[hidden email]> wrote:
> RE: VideoJS
>
> I have used VideoJS successfully in the past to play back HLS streams. One way to gather more information about your failure case is to open the JavaScript console in Chrome and reload the page that is trying to playback the video. If you receive any error messages from VideoJS, you may get more information about the problem.

Rand,

When using VideoJS HLS, I always open the Developer Tool in Chrome to
find errors :)

I also use omxh264enc which has inline-au-delimiter support to
generate HLS stream. I can watch the generated HLS stream in Safari.
Unfortunately, I got the following errors in Chrome:

VIDEOJS: ERROR: (CODE:3 MEDIA_ERR_DECODE) The media playback was
aborted due to a corruption problem or because the media used features
your browser did not support.

I also got the same errors when using h264parse with
inline-au-delimiter support. I have no idea how to debug this errors
:(
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel