User question: remux MP4 file/stream (H.264/AAC) to MPEG2TS file/stream

[Chris Burns-3]

Hi all,

Sorry to post this on the devel list, but I'm not sure where else to
go...

I'm just getting started with gstreamer and am trying to create a
gst-launch pipeline to read an .m4v (MP4 file, but eventually an rtp or
udp stream, with H.264 & AAC data inside) and remux it to an MPEG2 TS
file (this will be changed to a UDP stream when it's working). I'm
testing with VLC v1.0.0 on WinXP.

On CentOS 5.3:

# yum list installed '*gstreamer*'
Loaded plugins: fastestmirror, priorities
Installed Packages
gstreamer.i386                                        0.10.20-3.el5
installed
gstreamer-ffmpeg.i386                                 0.10.2-2.el5.rf
installed
gstreamer-plugins-bad.i386                            0.10.8-3.el5.rf
installed
gstreamer-plugins-base.i386
0.10.20-3.0.1.el5_3                            installed
gstreamer-plugins-good.i386                           0.10.9-1.el5_3.2
installed
gstreamer-plugins-ugly.i386                           0.10.9-1.el5.rf
installed
gstreamer-tools.i386                                  0.10.20-3.el5
installed


Input file is 95sec of: 400Kb/s 480x320, 10fps H.264 video, with 20Kb/s
22050Hz, stereo AAC-LC audio.

# ls -l infile.m4v
-rw-r--r-- 1 root root 5300576 Jul  3 15:23 infile.m4v



[1] At first I hoped that the plumbing would sort itself out and tried
the really naive:

#  gst-launch-0.10 -v filesrc location=infile.m4v ! qtdemux  !
ffmux_mpegts ! filesink location=outfile.ts
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/pipeline0/ffmux_mpegts0.video_0: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 132343612 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
/pipeline0/ffmux_mpegts0.video_0: caps = NULL
/pipeline0/qtdemux0.audio_00: caps = NULL
/pipeline0/qtdemux0.video_00: caps = NULL
Setting pipeline to NULL ...
FREEING pipeline ...

# ls -l outfile.ts
-rw-r--r-- 1 root root 8036060 Jul 16 10:20 outfile.ts

This pipeline runs, but VLC can't play the larger (8MB, from a 5MB
source) outfile.ts... It opens the file, but has "0:00/0:00" in the time
field and while it scrolls the playback position from left to right in
approx the right time, there is no audio or video output.


[2] Next, after some googling, I tried:

# gst-launch-0.10 -v filesrc location=infile.m4v ! qtdemux name=d \
     d. ! video/x-h264 ! m. \
     d. ! audio/aac ! m. \
     ffmux_mpegts name=m ! filesink location=outfile.ts
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/pipeline0/capsfilter2: caps = video/x-h264
/pipeline0/capsfilter3: caps = audio/aac
/pipeline0/capsfilter4: caps = audio/aac
/pipeline0/capsfilter2.src: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
/pipeline0/capsfilter2.sink: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
/pipeline0/m.video_0: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 132576270 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
/pipeline0/m.video_0: caps = NULL
/pipeline0/capsfilter2.src: caps = NULL
/pipeline0/capsfilter2.sink: caps = NULL
/pipeline0/d.audio_00: caps = NULL
/pipeline0/d.video_00: caps = NULL
Setting pipeline to NULL ...
FREEING pipeline ...

# ls -l outfile.ts
-rw-r--r-- 1 root root 8036060 Jul 16 10:26 outfile.ts

Same result as [1] before...


[3] After some more googling and experimentation:

# gst-launch-0.10 -v filesrc location=infile.m4v ! qtdemux name=d \
     d.video_00 ! queue ! m.video_00 \
     d.audio_00 ! queue ! m.audio_00 \
     ffmux_mpegts name=m ! filesink location=outfile.ts
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/pipeline0/queue0.sink: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
/pipeline0/queue1.sink: caps = audio/mpeg, mpegversion=(int)4,
framed=(boolean)true, codec_data=(buffer)1290, rate=(int)32000,
channels=(int)2
/pipeline0/queue1.sink: caps = audio/mpeg, mpegversion=(int)4,
framed=(boolean)true, codec_data=(buffer)1290, rate=(int)32000,
channels=(int)2
/pipeline0/m.video_0: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
/pipeline0/queue1.src: caps = audio/mpeg, mpegversion=(int)4,
framed=(boolean)true, codec_data=(buffer)1290, rate=(int)32000,
channels=(int)2
/pipeline0/m.audio_0: caps = audio/mpeg, mpegversion=(int)4,
framed=(boolean)true, codec_data=(buffer)1290, rate=(int)32000,
channels=(int)2
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 189110943 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
/pipeline0/m.audio_0: caps = NULL
/pipeline0/m.video_0: caps = NULL
/pipeline0/queue1.src: caps = NULL
/pipeline0/queue1.sink: caps = NULL
/pipeline0/queue0.src: caps = NULL
/pipeline0/queue0.sink: caps = NULL
/pipeline0/d.audio_00: caps = NULL
/pipeline0/d.video_00: caps = NULL
Setting pipeline to NULL ...
FREEING pipeline ...

# ls -l outfile.ts
-rw-r--r-- 1 root root 6259272 Jul 16 10:34 outfile.ts

File size is better, but same VLC result as in [1]



[4] Switch to ffmpeg demux...

# gst-launch-0.10 -v filesrc location=infile.m4v !
ffdemux_mov_mp4_m4a_3gp_3g2_mj2 name=d \
     d.video_00 ! queue ! m.video_00 \
     d.audio_00 ! queue ! m.audio_00 \
     ffmux_mpegts name=m ! filesink location=outfile.ts
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/pipeline0/queue0.sink: caps = video/x-h264, width=(int)480,
height=(int)320, framerate=(fraction)10/1,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8
/pipeline0/queue1.sink: caps = audio/mpeg, rate=(int)32000,
channels=(int)2, mpegversion=(int)4, codec_data=(buffer)1290
/pipeline0/queue1.sink: caps = audio/mpeg, rate=(int)32000,
channels=(int)2, mpegversion=(int)4, codec_data=(buffer)1290
/pipeline0/m.video_0: caps = video/x-h264, width=(int)480,
height=(int)320, framerate=(fraction)10/1,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8
/pipeline0/queue1.src: caps = audio/mpeg, rate=(int)32000,
channels=(int)2, mpegversion=(int)4, codec_data=(buffer)1290
/pipeline0/m.audio_0: caps = audio/mpeg, rate=(int)32000,
channels=(int)2, mpegversion=(int)4, codec_data=(buffer)1290
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 180962744 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
/pipeline0/m.audio_0: caps = NULL
/pipeline0/m.video_0: caps = NULL
/pipeline0/queue1.src: caps = NULL
/pipeline0/queue1.sink: caps = NULL
/pipeline0/queue0.src: caps = NULL
/pipeline0/queue0.sink: caps = NULL
/pipeline0/d.audio_00: caps = NULL
/pipeline0/d.video_00: caps = NULL
Setting pipeline to NULL ...
FREEING pipeline ...

# ls -l outfile.ts
-rw-r--r-- 1 root root 6259272 Jul 16 10:36 outfile.ts

No difference from [3] (i.e. same VLC result as [1])



[5] Try muxing to MP4 container

# gst-launch-0.10 -v filesrc location=infile.m4v ! qtdemux name=d \
     d.video_00 ! queue ! m.video_0 \
     d.audio_00 ! queue ! m.audio_0 \
     ffmux_mp4 name=m ! filesink location=outfile.mp4
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/pipeline0/queue0.sink: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
/pipeline0/queue1.sink: caps = audio/mpeg, mpegversion=(int)4,
framed=(boolean)true, codec_data=(buffer)1290, rate=(int)32000,
channels=(int)2
/pipeline0/queue1.sink: caps = audio/mpeg, mpegversion=(int)4,
framed=(boolean)true, codec_data=(buffer)1290, rate=(int)32000,
channels=(int)2
/pipeline0/m.video_0: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
/pipeline0/m.video_0: caps = video/x-h264,
codec_data=(buffer)0142c01effe100176742c01e9a740f05342000000300200000030
291e2c5d401000468ce32c8, width=(int)480, height=(int)320,
framerate=(fraction)10/1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 200009206 ns.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
/pipeline0/m.audio_0: caps = NULL
/pipeline0/m.video_0: caps = NULL
/pipeline0/queue1.src: caps = NULL
/pipeline0/queue1.sink: caps = NULL
/pipeline0/queue0.src: caps = NULL
/pipeline0/queue0.sink: caps = NULL
/pipeline0/d.audio_00: caps = NULL
/pipeline0/d.video_00: caps = NULL
Setting pipeline to NULL ...
FREEING pipeline ...

# ls -l outfile.mp4
-rw-r--r-- 1 root root 5300514 Jul 16 10:39 outfile.mp4

outfile.mp4 is about the right size and plays back perfectly in VLC.
Crap! So the pipeline is basically ok, just the TS muxing is bad.



[6] Next, I moved to a FC10 machine and yum installed the latest
gstreamer RPMs. This let me use the mpegtsmux module...

# yum list installed '*gstreamer*'
Loaded plugins: refresh-packagekit
Installed Packages
PackageKit-gstreamer-plugin.i386                     0.3.9-4.fc10
installed
gstreamer.i386                                       0.10.21-2.fc10
installed
gstreamer-ffmpeg.i386                                0.10.5-1.fc10
@rpmfusion-free
gstreamer-plugins-bad.i386                           0.10.9-1.fc10
@rpmfusion-free
gstreamer-plugins-base.i386                          0.10.21-2.fc10
installed
gstreamer-plugins-flumpegdemux.i386                  0.10.15-4.fc10
installed
gstreamer-plugins-good.i386                          0.10.13-1.fc10
installed
gstreamer-plugins-ugly.i386                          0.10.10-2.fc10
@rpmfusion-free-updates
gstreamer-python.i386                                0.10.12-1.fc10
installed
gstreamer-tools.i386                                 0.10.21-2.fc10
installed
totem-gstreamer.i386                                 2.24.3-1.fc10
installed


Same infile.m4v:

# ls -l infile.m4v
-rw-r--r-- 1 root root 5300576 2009-07-03 15:23 infile.mv4

# gst-launch-0.10 filesrc location=infile.m4v ! qtdemux name=d \
    d.video_00 ! queue ! m.sink_0 \
    d.audio_00 ! queue ! m.sink_1 \
    mpegtsmux name=m ! filesink location=artbeats.ts
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
*** glibc detected *** gst-launch-0.10: free(): invalid next size
(fast): 0x09b6d400 ***
======= Backtrace: =========
/lib/libc.so.6[0x450394]
/lib/libc.so.6(cfree+0x96)[0x452346]
/lib/libglib-2.0.so.0(g_free+0x36)[0x3238e6]
/usr/lib/libgstreamer-0.10.so.0[0x491810b]
/usr/lib/libgstreamer-0.10.so.0(gst_mini_object_unref+0xb3)[0x49388a3]
/usr/lib/gstreamer-0.10/libgstmpegtsmux.so[0x600c0d]
/usr/lib/gstreamer-0.10/libgstmpegtsmux.so[0x6055e9]
/usr/lib/gstreamer-0.10/libgstmpegtsmux.so[0x605b28]
/usr/lib/gstreamer-0.10/libgstmpegtsmux.so[0x6047eb]
/usr/lib/gstreamer-0.10/libgstmpegtsmux.so[0x60254b]
/usr/lib/libgstbase-0.10.so.0[0x4a17377]
/usr/lib/libgstbase-0.10.so.0[0x4a17f50]
/usr/lib/libgstreamer-0.10.so.0[0x493e7d9]
/usr/lib/libgstreamer-0.10.so.0(gst_pad_push+0x33a)[0x493fa1a]
/usr/lib/gstreamer-0.10/libgstcoreelements.so[0x5b6202]
/usr/lib/libgstreamer-0.10.so.0[0x495ed33]
/lib/libglib-2.0.so.0[0x347346]
/lib/libglib-2.0.so.0[0x345caf]
/lib/libpthread.so.0[0x3cd51f]
/lib/libc.so.6(clone+0x5e)[0x4c501e]
======= Memory map: ========
00110000-00111000 r-xp 00110000 00:00 0          [vdso]
00111000-00115000 r-xp 00000000 09:00 212774
/lib/libgthread-2.0.so.0.1800.4
00115000-00116000 rw-p 00003000 09:00 212774
/lib/libgthread-2.0.so.0.1800.4
00116000-00119000 r-xp 00000000 09:00 212754
/lib/libgmodule-2.0.so.0.1800.4
00119000-0011a000 rw-p 00002000 09:00 212754
/lib/libgmodule-2.0.so.0.1800.4
0011a000-00122000 r-xp 00000000 09:00 214862     /lib/librt-2.9.so
00122000-00123000 r--p 00007000 09:00 214862     /lib/librt-2.9.so
00123000-00124000 rw-p 00008000 09:00 214862     /lib/librt-2.9.so
00124000-00127000 r-xp 00000000 09:00 214866     /lib/libdl-2.9.so
00127000-00128000 r--p 00002000 09:00 214866     /lib/libdl-2.9.so
00128000-00129000 rw-p 00003000 09:00 214866     /lib/libdl-2.9.so
0012f000-0014f000 r-xp 00000000 09:00 214859     /lib/ld-2.9.so
00150000-00151000 r--p 00020000 09:00 214859     /lib/ld-2.9.so
00151000-00152000 rw-p 00021000 09:00 214859     /lib/ld-2.9.so
00152000-0029d000 r-xp 00000000 09:00 409495
/usr/lib/libxml2.so.2.7.3
0029d000-002a2000 rw-p 0014b000 09:00 409495
/usr/lib/libxml2.so.2.7.3
002a2000-002a3000 rw-p 002a2000 00:00 0
002a3000-002e3000 r-xp 00000000 09:00 212770
/lib/libgobject-2.0.so.0.1800.4
002e3000-002e4000 rw-p 00040000 09:00 212770
/lib/libgobject-2.0.so.0.1800.4
002e4000-003c6000 r-xp 00000000 09:00 212750
/lib/libglib-2.0.so.0.1800.4
003c6000-003c7000 rw-p 000e2000 09:00 212750
/lib/libglib-2.0.so.0.1800.4
003c7000-003dd000 r-xp 00000000 09:00 214861     /lib/libpthread-2.9.so
003dd000-003de000 r--p 00015000 09:00 214861     /lib/libpthread-2.9.so
003de000-003df000 rw-p 00016000 09:00 214861     /lib/libpthread-2.9.so
003df000-003e1000 rw-p 003df000 00:00 0
003e1000-0054f000 r-xp 00000000 09:00 214860     /lib/libc-2.9.so
0054f000-00551000 r--p 0016e000 09:00 214860     /lib/libc-2.9.so
00551000-00552000 rw-p 00170000 09:00 214860     /lib/libc-2.9.so
00552000-00555000 rw-p 00552000 00:00 0
00555000-00568000 r-xp 00000000 09:00 214878     /lib/libz.so.1.2.3
00568000-00569000 rw-p 00012000 09:00 214878     /lib/libz.so.1.2.3
00569000-00590000 r-xp 00000000 09:00 214873     /lib/libm-2.9.so
00590000-00591000 r--p 00026000 09:00 214873     /lib/libm-2.9.so
00591000-00592000 rw-p 00027000 09:00 214873     /lib/libm-2.9.so
00592000-0059d000 r-xp 00000000 09:00 212762
/lib/libnss_files-2.9.so
0059d000-0059e000 r--p 0000a000 09:00 212762
/lib/libnss_files-2.9.so
0059e000-0059f000 rw-p 0000b000 09:00 212762
/lib/libnss_files-2.9.so
0059f000-005cb000 r-xp 00000000 09:00 500485
/usr/lib/gstreamer-0.10/libgstcoreelements.so
005cb000-005cc000 rw-p 0002c000 09:00 500485
/usr/lib/gstreamer-0.10/libgstcoreelements.so
005cc000-005ec000 r-xp 00000000 09:00 500757
/usr/lib/gstreamer-0.10/libgstqtdemux.so
005ec000-005ed000 rw-p 00020000 09:00 500757
/usr/lib/gstreamer-0.10/libgstqtdemux.so
005ed000-005fe000 r-xp 00000000 09:00 412179
/usr/lib/libgstrtp-0.10.sAborted

Whoops! So no luck there, either.


I downloaded the latest packages from
http://gstreamer.freedesktop.org/src/ but I couldn't find a new enough
flex package to build on CentOS 5.3 and could not figure out how to get
around the:

configure: No package 'gstreamer-0.10' found
configure: error: no gstreamer-0.10 >= 0.10.23 (GStreamer) found

when running gst-plugins-bad-0.10.13's ./configure on FC10.

Does my pipeline look ok towards the end? Is it looking like a muxing
problem?

Any ideas, suggestions, etc will be warmly received...

Thanks in advance,

ChrisB

--
Chris Burns
[hidden email]

------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time,
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

[tiki_ns]

Hello Chris,

I am using gstreamer for the similar purpose and I can "mux TS file" using this gst-launch command:

gst-launch filesrc location=file_name.mp4 ! qtdemux name=demux demux.video_00 ! queue ! mpegtsmux name=mux ! filesink location=out_file_name.ts sync=FALSE demux.audio_00 ! queue ! mux.

Also I found some issues in mpegtsmux which cause:
1. sync problems (PTSs are repeating because of unnecessary check in mpegtsmux_choose_best_stream() function)
2. with some mp4 files gstreamer is crashing (caused with buffer allocation in mpegtsmux_prepare_h264() function)
3. high CPU usage (caused with sending one by one TS packet)

If you have same issue I can send you my fixes. Regarding problem with building latest versions of gst-plugins you should first install the latest version of gstremer (0.10.23) and gst-plugins_base (0.10.23). For playback generated TS files I am using VLC player.