How to multiplex audio and video into mpegtsmux

[jmz]

Hi,

I can use gst-launch tool to save RTSP A/V stream to MPEG TS file, but fail
in GStreamer application.

In my application, I add rtspsrc, mpegtsmux, and filesink to a pipeline, and
then set the pipeline to PLAYING. On the "pad-added" signal of rtspsrc, I
create the corresponding elements for the audio and video (for example,
depayloader and parser), add elements to the pipeline, link elements
together, and sync state with pipeline.

If I handle only audio or video, the application works well. If both audio
and video are added to the pipeline in the pad-added callback, error occurs:

ERROR: from element /GstPipeline:pipeline/MpegTsMux:mpegtsmux0: Could not
multiplex stream.
Stream on pad <mpegtsmux0:sink_66> is not associated with any program
Additional debug info:
mpegtsmux.c(1192): mpegtsmux_collected_buffer ():
/GstPipeline:pipeline/MpegTsMux:mpegtsmux0

I am sure that all elements are linked correctly within the pipeline. The
error looks like mpegtsmux (as well as the pipeline) has been PLAYING after
one stream source is linked with it in the pad-added callback, and other
stream cannot be added to the playing mpegtsmux.

How to correctly multiplex audio and video into mpegtsmux in application?

Thanks for help.





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Gst-Geek]

Jmz,

The below error is reported when pad is added after mpegts muxer goes to
playing state. i.e you are adding the pad too late.


ERROR: from element /GstPipeline:pipeline/MpegTsMux:mpegtsmux0: Could not
multiplex stream.
Stream on pad <mpegtsmux0:sink_66> is not associated with any program
Additional debug info:
mpegtsmux.c(1192): mpegtsmux_collected_buffer ():
/GstPipeline:pipeline/MpegTsMux:mpegtsmux0

Which version of Gstreamer you are using ??

~ Vinod



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[jmz]

Hi Vinod


Vinod Kesti wrote
> The below error is reported when pad is added after mpegts muxer goes to
> playing state. i.e you are adding the pad too late.
>
> Which version of Gstreamer you are using ??


I use GStreamer 1.2, but get the same error "mpegtsmux0: Could not multiplex
stream." when using newer version such as 1.10. Is this a bug in older
mpegtsmux element?

As you mentioned, the error occurs when the second pad is added to mpegtsmux
after mpegts muxer goes to playing state. I change the flow to add rtspsrc,
mpegtsmux, and filesink to a pipeline at the beginning but set only rtspsrc
(not pipeline) to PLAYING. Both audio and video streams are also added to
the pipeline on the pad-added signal of rtspsrc. Finally the pipeline is set
to PLAYING on the no-more-pads signal of rtspsrc.

However, the pipeline is not playing (only PAUSED state?) and the following
warnings are shown continuously with GST_DEBUG=3:

WARN               rtpsource rtpsource.c:945:calculate_jitter: cannot get
current running_time

Please give me some instruction on how to multiplex RTSP audio and video
into mpegtsmux? Should I handle this in rtspsrc's pad-added, no-more-pads,
or other signals?

Thanks




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Gst-Geek]

Hi Jmz,

You can try this.
Set pipeline to Paused in the start and set to playing state in
"no-more-pads" call back of rtspsrc.
 



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[jmz]

Hi Vinod,


Vinod Kesti wrote
> You can try this.
> Set pipeline to Paused in the start and set to playing state in
> "no-more-pads" call back of rtspsrc.

Thank you for your reply. I modified my program as you suggested, but it
does not work. My program flow is as follows:

a) Create rtspsrc & mpegtsmux & sink and add them to pipeline
b) Set pipeline to PAUSED
c) Set rtspsrc to PLAYING
d) In pad-added callback of rtspsrc, add video and audio stream to pipeline,
link elements, and sync state with pipeline
e) In no-more-pads callback of rtspsrc, set pipeline to PLAYING

Step c is necessary because rtspsrc needs to prcoess stream. In such flow, I
still get the same error:
ERROR: from element /GstPipeline:pipeline/MpegTsMux:mpegtsmux0: Could not
multiplex stream.
Stream on pad <mpegtsmux0:sink_66> is not associated with any program

If step b is skipped (that is, except for rtspsrc, the other elements start
to change state from NULL to PLAYING in the step), the pipeline is not
playing and a warning message is shown continuously:
WARN               rtpsource rtpsource.c:920:calculate_jitter: cannot get
current running_time

If I use the same flow but handle ONLY ONE steam in step d, the pipeline is
playing.

Did I miss anything?




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Gst-Geek]

No need to set rtspsrc to PLAYING state. In PAUSE state on buffer can reach
sink.



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[jmz]

Hi Vinod,


Vinod Kesti wrote
> No need to set rtspsrc to PLAYING state. In PAUSE state on buffer can
> reach
> sink.

It is quite strange. If I do only step a and step b (the pipeline is in
PAUSED state), the pipeline gets stuck because rtspsrc does not emit
pad-added signal. I can see that manager, rtpssrcdemux*, and rtpsession*
(elements within rtspsrc) changed to PAUSED.

Even if I add only rtspsrc to the pipeline and set the pipeline to PAUSED,
rtspsrc does not emit pad-added signal. Without pad-added signal of rtspsrc,
I cannot add other elements to the pipeline.




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Nicolas Dufresne-5]

Le jeudi 12 octobre 2017 à 03:32 -0700, Vinod Kesti a écrit :
> No need to set rtspsrc to PLAYING state. In PAUSE state on buffer can
> reach
> sink.

Live source don't produce data in paused. rtspsrc is a live source.

>  
>
>
>
> --
> Sent from: http://gstreamer-devel.966125.n4.nabble.com/
> _______________________________________________
> gstreamer-devel mailing list
> [hidden email]
> https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[jmz]

Hi Nicolas,


Nicolas Dufresne-5 wrote
> Le jeudi 12 octobre 2017 à 03:32 -0700, Vinod Kesti a écrit :
>> No need to set rtspsrc to PLAYING state. In PAUSE state on buffer can
>> reach sink.
>
> Live source don't produce data in paused. rtspsrc is a live source.


Thank you for the clarification.

My initial question is I cannot multiplex RTSP audio and video into
mpegtsmux on the fly (as dynamic pipeline) in the following steps:

a) Create rtspsrc & mpegtsmux & sink and add them to pipeline
b) Set pipeline to PAUSED
c) Set rtspsrc to PLAYING (necessary because rtspsrc is live source)
d) In pad-added callback of rtspsrc, add video and audio stream to pipeline,
link elements, and sync state with pipeline
e) In no-more-pads callback of rtspsrc, set pipeline to PLAYING

I get error "mpegtsmux0: Could not multiplex stream." I can use gst-launch
tool to do the same thing. Do you or anyone know how mpegtsmux can be used
in dynamic pipeline?

Thanks for any help.





--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[jmz]

jmz wrote
> a) Create rtspsrc & mpegtsmux & sink and add them to pipeline
> b) Set pipeline to PAUSED
> c) Set rtspsrc to PLAYING (necessary because rtspsrc is live source)
> d) In pad-added callback of rtspsrc, add video and audio stream to
> pipeline,
> link elements, and sync state with pipeline
> e) In no-more-pads callback of rtspsrc, set pipeline to PLAYING


I changed my program flow as follows.
a) Add only rtspsrc to pipeline and set the pipeline PLAYING
b) In pad-added callback of rtspsrc, add depayloader & parser & mpegtsmux &
filesink to pipeline, link them together, and sync state (except for
mpegtsmux and filesink) with pipeline
c) In no-more-pads callback of rtspsrc, sync mpegtsmux and filesink state
with pipeline

In this flow, "mpegtsmux0: Could not multiplex stream." does not appear. But
the pipeline stays in PAUSED state and filesink stays in READY while the
other elements (including mpegtsmux) have been PLAYING. This does work :(

I further found in dot graph that the caps between parser (e.g. h264parse)
and mpegtsmux are not negotiated. How can I let the caps be negotiated?

In addition, I saw some logs as folows (where mpegtsmux0:sink_65 is linked
with h264parse0:src):
INFO               GST_EVENT
gstpad.c:5083:gst_pad_send_event_unchecked:<mpegtsmux0:sink_65> Received
event on flushing pad. Discarding

On the other hand, if mpegtsmux is set to PLAYING in the pad-added callback
of rtspsrc for video stream, the caps between h264parse and mpegtsmux are
negotiated. But, I cannot multiplex the audio stream into M2TS later in the
pad-added callback.

I would appreciate it if someone could help.




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Gst-Geek]

See if syncing pipeline childs helps in no-mode-pads

gst_bin_sync_children_states (pipeline);

~ Vinod



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[jmz]

Hi Vinod


Vinod Kesti wrote
> See if syncing pipeline childs helps in no-mode-pads
> gst_bin_sync_children_states (pipeline);

Thank you for your suggestion again. But it does not work :(

After trying in several days, I finally realized that we cannot mux more
than one stream from live source such as rtspsrc into mpegtsmux using
dynamic pipeline.

I tried encodebin with rtspsrc and also got the same error.

WARN               mpegtsmux
mpegtsmux.c:1192:mpegtsmux_collected_buffer:<muxer> error: Stream on pad
<muxer:sink_66> is not associated with any program
INFO        GST_ERROR_SYSTEM
gstelement.c:1835:gst_element_message_full:<muxer> posting message: Stream
on pad <muxer:sink_66> is not associated with any program
INFO        GST_ERROR_SYSTEM
gstelement.c:1858:gst_element_message_full:<muxer> posted error message:
Stream on pad <muxer:sink_66> is not associated with any program

On the other hand, encodebin works well with filesrc. I also attach the
error dot graph (someone may be interested in it). Please correct me if I am
not correct or make some mistakes.

By the way, I found that the encoding program in
gst-plugins-base/tests/examples does not work with rtsp:// source. It will
link uridecodebin0:src_1 with encodebin0:video_1 (should be
encodebin0:audio_1) when uridecodebin0:src_1 has caps "audio/x-raw." I have
to modify the program to let it work.

Thanks anyway

error.svg
<http://gstreamer-devel.966125.n4.nabble.com/file/t377835/error.svg>  




--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel