mpegtsmux with ximagesrc
Locked
 
|
Hi all,
I'm trying to capture desktop display (via H264) and system audio (via AAC) to send via TS->RTP->udp to another computer. The pipeline below works as long as something is moving within the desktop display. But, if display doesn't change, ximagesrc stops delivering frames (which saves needless thrash), and mpegtsmux output stops dead; mpegtsmux's collectpads won't do anything until *all* input pads (i.e. both audio *and* video) have data queued. Is this "sporadic", non-periodic video a valid usage case for TS? If so, any suggestions for an alternative/workaround? I've tried videorate in front of the H264 encoder, but that swamps the system with needless thrash. I'm also trying to hack mpegtsmux to allow it to package/forward audio frames even when video stops, but having problems getting things started (non-negotiated caps?? for the stream that starts second). Pipeline: video: ximagesrc ! ffmpegcolorspace ! x264enc ! queue ! mpegtsmux audio: autoaudiosrc ! ffenc_aac ! mpegtsmux transport: mpegtsmux ! rtpmp2tpay ! udpsink Thanks. -- Ben -- _______________________________________________ gstreamer-devel mailing list [hidden email] http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
Re: mpegtsmux with ximagesrc
|
|
On Mon, 2011-07-25 at 10:19 -0700, Cahill, Ben M wrote:
> I'm trying to capture desktop display (via H264) and system audio (via > AAC) to send via TS->RTP->udp to another computer. > > The pipeline below works as long as something is moving within the > desktop display. But, if display doesn't change, ximagesrc stops > delivering frames (which saves needless thrash), and mpegtsmux output > stops dead; mpegtsmux's collectpads won't do anything until *all* input > pads (i.e. both audio *and* video) have data queued. > > Is this "sporadic", non-periodic video a valid usage case for TS? > > If so, any suggestions for an alternative/workaround? I've tried > videorate in front of the H264 encoder, but that swamps the system with > needless thrash. I'm also trying to hack mpegtsmux to allow it to > package/forward audio frames even when video stops, but having problems > getting things started (non-negotiated caps?? for the stream that > starts second). There are known issues with GstCollectPads and handling of sparse streams, see for example [1], but there are probably ways this could be hacked around until something better comes along. Some random ideas: does ximagesrc send newsegment updates, to move time along when it doesn't output repeat frames? (Then the muxer/collectpads know when not to wait, but see above). Or maybe there's a way to tell x264enc that we want a repeat frame and/or that the frame being passed is identical to the previous frame? Then videorate could flag duplicates it creates and x264enc could pick up on the flag. Cheers -Tim [1] https://bugzilla.gnome.org/show_bug.cgi?id=415754 _______________________________________________ gstreamer-devel mailing list [hidden email] http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
>-----Original Message----- >From: >gstreamer-devel-bounces+bmcahill=[hidden email] >.org >[mailto:gstreamer-devel-bounces+bmcahill=[hidden email] >edesktop.org] On Behalf Of Tim-Philipp Müller >Sent: Tuesday, July 26, 2011 4:50 AM >To: [hidden email] >Subject: Re: mpegtsmux with ximagesrc > >On Mon, 2011-07-25 at 10:19 -0700, Cahill, Ben M wrote: > >> I'm trying to capture desktop display (via H264) and system >audio (via >> AAC) to send via TS->RTP->udp to another computer. >> >> The pipeline below works as long as something is moving within the >> desktop display. But, if display doesn't change, ximagesrc stops >> delivering frames (which saves needless thrash), and mpegtsmux output >> stops dead; mpegtsmux's collectpads won't do anything until >*all* input >> pads (i.e. both audio *and* video) have data queued. >> >> Is this "sporadic", non-periodic video a valid usage case for TS? >> >> If so, any suggestions for an alternative/workaround? I've tried >> videorate in front of the H264 encoder, but that swamps the >system with >> needless thrash. I'm also trying to hack mpegtsmux to allow it to >> package/forward audio frames even when video stops, but >having problems >> getting things started (non-negotiated caps?? for the stream that >> starts second). > >There are known issues with GstCollectPads and handling of sparse >streams, see for example [1], but there are probably ways this could be >hacked around until something better comes along. Thanks for the bugzilla reference. In my tarball source code (couple months old now) I see gstcollectpads2 is in gst-plugins-good-0.10.29/gst/videomixer, and is used only by videomixer2.c. Is anyone planning to port mpegtsmux to use gstcollectpads2? I could try it as an experiment. Last night I got something "working" here by hacking gstcollectpads.c to use a g_cond_timed_wait() instead of g_cond_wait() (in GST_COLLECT_PADS_WAIT()), and some other hacks to both gstcollectpads.c and mpegtsmux.c. A 100 msec timeout got things working afaict, but I wouldn't be surprised if there are some undesired side effects that I wouldn't understand yet (kind of newbie here). I'll guess gstcollectpads2.c provides a more well-thought-out solution. > >Some random ideas: does ximagesrc send newsegment updates, to move time >along when it doesn't output repeat frames? (Then the muxer/collectpads >know when not to wait, but see above). Or maybe there's a way to tell >x264enc that we want a repeat frame and/or that the frame being passed >is identical to the previous frame? Then videorate could flag >duplicates >it creates and x264enc could pick up on the flag. Thanks. -- Ben -- > > Cheers > -Tim > >[1] https://bugzilla.gnome.org/show_bug.cgi?id=415754 > >_______________________________________________ >gstreamer-devel mailing list >[hidden email] >http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > gstreamer-devel mailing list [hidden email] http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
|
|
Update to this situation of audio blocked by sparse video in mpegtsmux ... after some IRC discussion in late July, I've posted some patches at:
https://bugzilla.gnome.org/show_bug.cgi?id=415754 These include (paraphrased): Move GstCollectPads2 into gstreamer library Port mpegtsmux to use GstCollectPads2 Allow sparse (non-waiting) mode for any sink pad in mpegtsmux Use bitfield-based "collected" and "WAIT" algos in GstCollectPads2 Prototype "sparsesegment" inserter plugin The "sparsesegment" plugin is pretty primitive; it works for my purposes (so far), but as I'm segments noobie, it may break some usage models, and will probably need some additional development. This is all working for me (AFAICT), but I would appreciate any comments, testing, etc. -- Ben -- >-----Original Message----- >From: >gstreamer-devel-bounces+bmcahill=[hidden email] >.org >[mailto:gstreamer-devel-bounces+bmcahill=[hidden email] edesktop.org] On Behalf Of Cahill, Ben M >Sent: Tuesday, July 26, 2011 3:28 PM >To: Discussion of the development of and with GStreamer >Subject: RE: mpegtsmux with ximagesrc > > > >>-----Original Message----- >>From: >>gstreamer-devel-bounces+bmcahill=[hidden email] >>.org >>[mailto:gstreamer-devel-bounces+bmcahill=[hidden email] >>edesktop.org] On Behalf Of Tim-Philipp Müller >>Sent: Tuesday, July 26, 2011 4:50 AM >>To: [hidden email] >>Subject: Re: mpegtsmux with ximagesrc >> >>On Mon, 2011-07-25 at 10:19 -0700, Cahill, Ben M wrote: >> >>> I'm trying to capture desktop display (via H264) and system >>audio (via >>> AAC) to send via TS->RTP->udp to another computer. >>> >>> The pipeline below works as long as something is moving within the >>> desktop display. But, if display doesn't change, ximagesrc stops >>> delivering frames (which saves needless thrash), and >mpegtsmux output >>> stops dead; mpegtsmux's collectpads won't do anything until >>*all* input >>> pads (i.e. both audio *and* video) have data queued. >>> >>> Is this "sporadic", non-periodic video a valid usage case for TS? >>> >>> If so, any suggestions for an alternative/workaround? I've tried >>> videorate in front of the H264 encoder, but that swamps the >>system with >>> needless thrash. I'm also trying to hack mpegtsmux to allow it to >>> package/forward audio frames even when video stops, but >>having problems >>> getting things started (non-negotiated caps?? for the stream that >>> starts second). >> >>There are known issues with GstCollectPads and handling of sparse >>streams, see for example [1], but there are probably ways >this could be >>hacked around until something better comes along. > >Thanks for the bugzilla reference. In my tarball source code >(couple months old now) I see gstcollectpads2 is in >gst-plugins-good-0.10.29/gst/videomixer, and is used only by >videomixer2.c. > >Is anyone planning to port mpegtsmux to use gstcollectpads2? >I could try it as an experiment. > >Last night I got something "working" here by hacking >gstcollectpads.c to use a g_cond_timed_wait() instead of >g_cond_wait() (in GST_COLLECT_PADS_WAIT()), and some other >hacks to both gstcollectpads.c and mpegtsmux.c. A 100 msec >timeout got things working afaict, but I wouldn't be surprised >if there are some undesired side effects that I wouldn't >understand yet (kind of newbie here). > >I'll guess gstcollectpads2.c provides a more well-thought-out solution. > > >> >>Some random ideas: does ximagesrc send newsegment updates, to >move time >>along when it doesn't output repeat frames? (Then the >muxer/collectpads >>know when not to wait, but see above). Or maybe there's a way to tell >>x264enc that we want a repeat frame and/or that the frame being passed >>is identical to the previous frame? Then videorate could flag >>duplicates >>it creates and x264enc could pick up on the flag. > >Thanks. > >-- Ben -- > >> >> Cheers >> -Tim >> >>[1] https://bugzilla.gnome.org/show_bug.cgi?id=415754 >> >>_______________________________________________ >>gstreamer-devel mailing list >>[hidden email] >>http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel >> >_______________________________________________ >gstreamer-devel mailing list >[hidden email] >http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel > gstreamer-devel mailing list [hidden email] http://lists.freedesktop.org/mailman/listinfo/gstreamer-devel |
«
Return to GStreamer-devel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |