flvdemux working .. sometimes

[Peter Maersk-Moller-2]

Hi.

I'm playing a rtmp stream passed through nginx with the rtmp module added. My problem is that I can't play the stream reliably. The pipeline sometimes fail 3 out of 4 times with the messages:

Redistribute latency...
WARNING: from element /GstPipeline:pipeline0/GstFlvDemux:demux: Delayed linking failed.
Additional debug info:
./grammar.y(506): gst_parse_no_more_pads (): /GstPipeline:pipeline0/GstFlvDemux:demux:
failed delayed linking some pad of GstFlvDemux named demux to some pad of GstQueue named queue2

The playing pipeline is this:

QUEUE="queue max-size-time=0 max-size-bytes=0 max-size-buffers=0"
LOCATION=rtmp://192.168.3.100/live/mykey3

gst-launch-1.0 -v                        \
        rtmpsrc location=$LOCATION      !\
        $QUEUE                          !\
        flvdemux name=demux             !\
        $QUEUE                          !\
        aacparse                        !\
        avdec_aac                       !\
        autoaudiosink sync=0 demux.     !\
        $QUEUE                          !\
        h264parse                       !\
        avdec_h264                      !\
        $QUEUE                          !\
        videoconvert                    !\
        autovideosink sync=0

Is there a way to make the pipeline play the stream reliably (play every time)?

Enabling debugging suggest the demuxer get filled whatever that means. Debug:

0:00:06.298664848 28766      0x2573b70 WARN                 default grammar.y:506:gst_parse_no_more_pads:<demux> warning: Delayed linking failed.
0:00:06.298728893 28766      0x2573b70 WARN                 default grammar.y:506:gst_parse_no_more_pads:<demux> warning: failed delayed linking some pad of GstFlvDemux named demux to some pad of GstQueue named queue3
0:00:06.298792450 28766      0x2573b70 INFO        GST_ERROR_SYSTEM gstelement.c:1879:gst_element_message_full:<demux> posting message: Delayed linking failed.
0:00:06.298904128 28766      0x2573b70 INFO        GST_ERROR_SYSTEM gstelement.c:1902:gst_element_message_full:<demux> posted warning message: Delayed linking failed.
WARNING: from element /GstPipeline:pipeline0/GstFlvDemux:demux: Delayed linking failed.

The stream is generated with the following pipeline:

VIDSRC="v4l2src ! video/x-raw,width=320,height=240"
AUDSRC="alsasrc"
gst-launch-1.0 -v \
        $AUDSRC                                         !\
        'audio/x-raw,channels=2,rate=44100,layout=interleaved,format=S16LE' !\
        audioconvert                                    !\
        queue                                           !\
        faac bitrate=128000                             !\
        aacparse                                        !\
        queue ! muxer.                                   \
        $VIDSRC                                         !\
        queue                                           !\
        videoconvert                                    !\
        x264enc speed-preset=1 tune=zerolatency         !\
        h264parse                                       !\
        queue                                           !\
        flvmux name=muxer streamable=true               !\
        queue                                           !\
        rtmpsink location="rtmp://127.0.0.1/live/mykey3 app=live flashVer=FME/3.0%20(compatible;%20FMSc%201.0) swfUrl=rtmp://127.0.0.1/live" blocksize=4096 sync=0

The nginx rtmp relay is setup with the following config so absolutely nothing special. All it does is that it forward any rtmp stream sent to it as it is sent. That means that if you send using this location:

rtmp://127.0.0.1/live/mykey3

then you can retrieve it as rtmp://127.0.0.1/live/mykey3 using gstreamers rtmpsrc module:

rtmp {
        server {
                listen 1935;
                chunk_size 4096;
                application live {
                        live on;
                        record off;
                        allow publish 127.0.0.1;
                        allow publish 193.88.237.0/24;
                        allow publish 192.168.3.0/24;
                        allow publish 192.168.10.0/24;
                        deny publish all;
                }
        }
}

Any suggestion on how to make a pipeline playing the rtmp stream every single time?

Second question or perhaps a hint of the problem. When the pipeline plays the stream, audio frames starts flowing to the autoaudiosink at approx the same time the video start to appear, but while the video is only 200-300 ms behind, the audio is between 2-6 seconds behind. And here we have no sync what so ever on the audio and video sinks. If sync is enabled on video, video will lag not a few hundred ms but several seconds, if it plays at all.

Best regards

Peter

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Mo, 2016-05-30 at 15:12 +0200, Peter Maersk-Moller wrote:

> Hi.
>
> I'm playing a rtmp stream passed through nginx with the rtmp module
> added. My problem is that I can't play the stream reliably. The
> pipeline sometimes fail 3 out of 4 times with the messages:
>
> Redistribute latency...
> WARNING: from element /GstPipeline:pipeline0/GstFlvDemux:demux:
> Delayed linking failed.
> Additional debug info:
> ./grammar.y(506): gst_parse_no_more_pads ():
> /GstPipeline:pipeline0/GstFlvDemux:demux:
> failed delayed linking some pad of GstFlvDemux named demux to some
> pad of GstQueue named queue2
>
> The playing pipeline is this:
>
> QUEUE="queue max-size-time=0 max-size-bytes=0 max-size-buffers=0"
> LOCATION=rtmp://192.168.3.100/live/mykey3
>
> gst-launch-1.0 -v                        \
>         rtmpsrc location=$LOCATION      !\
>         $QUEUE                          !\
>         flvdemux name=demux             !\
>         $QUEUE                          !\
>         aacparse                        !\
>         avdec_aac                       !\
>         autoaudiosink sync=0 demux.     !\
>         $QUEUE                          !\
>         h264parse                       !\
>         avdec_h264                      !\
>         $QUEUE                          !\
>         videoconvert                    !\
>         autovideosink sync=0
> [...]

This means that one of the queues you have there is not linked until
no-more-pads happens on the demuxer. Give proper names to the queues to
know which one it is.

Most likely the problem here is that the audio or video stream are not
found in the container early enough, and then the demuxer decides that
there is no audio or video. This would be related to
  https://bugzilla.gnome.org/show_bug.cgi?id=764260

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Peter Maersk-Moller-2]

Hi Sebastian

On Tue, May 31, 2016 at 8:17 AM, Sebastian Dröge <[hidden email]> wrote:

> QUEUE="queue max-size-time=0 max-size-bytes=0 max-size-buffers=0"
> LOCATION=rtmp://192.168.3.100/live/mykey3
>
> gst-launch-1.0 -v                        \
>         rtmpsrc location=$LOCATION      !\
>         $QUEUE                          !\
>         flvdemux name=demux             !\
>         $QUEUE                          !\
>         aacparse                        !\
>         avdec_aac                       !\
>         autoaudiosink sync=0 demux.     !\
>         $QUEUE                          !\
>         h264parse                       !\
>         avdec_h264                      !\
>         $QUEUE                          !\
>         videoconvert                    !\
>         autovideosink sync=0

> [...]

This means that one of the queues you have there is not linked until
no-more-pads happens on the demuxer. Give proper names to the queues to
know which one it is.

I named the pipes uniquely and found the queue starving is the queue between the flvmux and h264parse.

0:00:06.310136630 10614       0xb6f990 WARN                 default grammar.y:506:gst_parse_no_more_pads:<demux> warning: Delayed linking failed.
0:00:06.310215513 10614       0xb6f990 WARN                 default grammar.y:506:gst_parse_no_more_pads:<demux> warning: failed delayed linking some pad of GstFlvDemux named demux to some pad of GstQueue named qh264
WARNING: from element /GstPipeline:pipeline0/GstFlvDemux:demux: Delayed linking failed.
Additional debug info:
./grammar.y(506): gst_parse_no_more_pads (): /GstPipeline:pipeline0/GstFlvDemux:demux:
failed delayed linking some pad of GstFlvDemux named demux to some pad of GstQueue named qh264
0:00:28.189631062 10614       0xb6f990 WARN                flvdemux gstflvdemux.c:1413:gst_flv_demux_parse_tag_video:<demux> Signaled no-more-pads already but had no video pad -- ignoring
 

Most likely the problem here is that the audio or video stream are not
found in the container early enough, and then the demuxer decides that
there is no audio or video. This would be related to
  https://bugzilla.gnome.org/show_bug.cgi?id=764260

Reading through it, it sounds like that. That bug however is marked as an enhancement and it appear not to be moving very fast :-( Note that the problematic rtmp/flv stream is muxed and produced by GStreamer and not an external tool. I tried to change the block-size in the flvmux, but that does not seem to have any effect on this problem.

Interesting thing for delay in sync. If I start op the pipeline producing the stream and then start the playing pipeline without sync for audio and video sink, the video displayed is only a few hundred ms after what the camera records. The audio is around 5 seconds late. But if I start the player first, several seconds before I start producing the stream, the audio is less than 1000 ms late. Maybe the demuxer introduces a delay it doesn't have to introduce?

Are there any plans to add a rtmpserversrc module to GStreamer, a module that could act as an rtmp server receiving a RTMP stream from a rtmpsink module or another third party encoded rtmp (some cameras does rtmp although the current issue with audio/video sync and no-pads doesn't make it very attractive) ?

Thanks for the help

Peter

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Mi, 2016-06-01 at 00:04 +0200, Peter Maersk-Moller wrote:

> 
> > Most likely the problem here is that the audio or video stream are
> > not
> > found in the container early enough, and then the demuxer decides
> > that
> > there is no audio or video. This would be related to
> >   https://bugzilla.gnome.org/show_bug.cgi?id=764260
>
> Reading through it, it sounds like that. That bug however is marked
> as an enhancement and it appear not to be moving very fast :-( Note
> that the problematic rtmp/flv stream is muxed and produced by
> GStreamer and not an external tool. I tried to change the block-size
> in the flvmux, but that does not seem to have any effect on this
> problem.

This bug should be more easy to fix once decodebin3/playbin3 are there.
It just will need someone to actually work on it and make "late added"
streams by demuxers work.

> Interesting thing for delay in sync. If I start op the pipeline
> producing the stream and then start the playing pipeline without sync
> for audio and video sink, the video displayed is only a few hundred
> ms after what the camera records. The audio is around 5 seconds late.
> But if I start the player first, several seconds before I start
> producing the stream, the audio is less than 1000 ms late. Maybe the
> demuxer introduces a delay it doesn't have to introduce?

That could be a problem, yes. The bug should only appear if audio and
video have a gap of more than 0.5 (iirc?) seconds. flvmux should
ideally not do that, that it does sounds like a bug.

> Are there any plans to add a rtmpserversrc module to GStreamer, a
> module that could act as an rtmp server receiving a RTMP stream from
> a rtmpsink module or another third party encoded rtmp (some cameras
> does rtmp although the current issue with audio/video sync and no-
> pads doesn't make it very attractive) ?

I'm not aware of such plans, but it would be a good addition to
GStreamer indeed. Unrelated to this problem though :)

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel