H264 stream to MP4 file without the first bytes on the H264 stream

[ivan-perez]

Good morning

I have a very simple pipeline which converts a H264 stream into a MP4 file. It looks like this:

    gst-launch-1.0 -e -v fdsrc
      ! video/x-h264,width=640,height=480,framerate=30/1,profile=baseline,stream-format=avc,alignment=au
      ! h264parse
      ! mp4mux
      ! filesink location=test.mp4

I send to this pipeline's standard input the H264 stream I want to convert. This works pretty well if I send the entire stream (starting at the very first byte) to GStreamer.

Now I need to do the same but on an on-going H264 stream, which can't be started at the same time as GStreamer (the stream is already active at the moment that GStreamer starts, and I can't save all the previous contents of the stream in order to send them to GStreamer at the beginning). So GStreamer doesn't have the first bytes of the stream, and therefore it cannot be processed. I get this when I run the pipeline:

    Setting pipeline to PAUSED ...
    Pipeline is PREROLLING ...
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "video/x-h264\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)30/1\,\ profile\=\(string\)baseline\,\ stream-format\=\(string\)avc\,\ alignment\=\(string\)au"
    /GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = "video/x-h264\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)30/1\,\ profile\=\(string\)baseline\,\ stream-format\=\(string\)avc\,\ alignment\=\(string\)au"

Is there any parameter which tells the pipeline to ignore all contents until it gets a valid frame, so it can start when that frame is detected?

Thanks in advance for your help!!

Regards

[Tim Müller]

On Fri, 2017-02-03 at 03:14 -0800, ivan-perez wrote:

Hi,

> I have a very simple pipeline which converts a H264 stream into a MP4
> file.
> It looks like this:
>
>     gst-launch-1.0 -e -v fdsrc
>       ! video/x-h264,stream-format=avc,alignment=au,...
>       ! h264parse
>       ! mp4mux
>       ! filesink location=test.mp4

So where does the data come from? What does the H.264 generation? Do
you have the option of feeding H.264 in byte-stream (annex b) format
instead?

The problem here is that you can't really send stream-format=avc over a
stream such as a pipe or such. You need to maintain the packetisation
of the avc frames, and streams won't do that (it might look like it
mostly works, but you can't rely on it unless you force a packet-based
transport).

If you can send the data in byte-stream format instead then h264parse
can sync onto a running stream in the middle. I might be mis-
understanding the problem though, I'm just guessing here.

Cheers
 -Tim

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[ivan-perez]

Tim Müller wrote

On Fri, 2017-02-03 at 03:14 -0800, ivan-perez wrote:

Hi,

> I have a very simple pipeline which converts a H264 stream into a MP4
> file.
> It looks like this:
>
>     gst-launch-1.0 -e -v fdsrc
>       ! video/x-h264,stream-format=avc,alignment=au,...
>       ! h264parse
>       ! mp4mux
>       ! filesink location=test.mp4

So where does the data come from? What does the H.264 generation? Do
you have the option of feeding H.264 in byte-stream (annex b) format
instead?

The problem here is that you can't really send stream-format=avc over a
stream such as a pipe or such. You need to maintain the packetisation
of the avc frames, and streams won't do that (it might look like it
mostly works, but you can't rely on it unless you force a packet-based
transport).

If you can send the data in byte-stream format instead then h264parse
can sync onto a running stream in the middle. I might be mis-
understanding the problem though, I'm just guessing here.

Cheers
 -Tim

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Hi Tim,

I'm going to test if using 'byte-stream' instead of 'avc' could do the trick.

For clarification, the H264 stream comes from the Raspberry camera, and I directly pipe it into two GStreamer instances:

* One instance is running all the time, as it takes care of producing a RTP stream which clients may see at any time.
* Another instance only runs if some conditions are met (for example, after some detection movement). In that case, I want to save a video of 30 seconds (in MP4). My idea is to use the stream directly taken from the Raspberry camera, but at the moment it doesn't work because -I think- in order to work properly it'd need the first bytes of the stream, which may contain some headers.

Let me try changing the 'stream-format' parameter and I'll post here if it works for me or not. Also I'm going to check if there are more options for 'raspivid' (the command I use to generate the H264 stream) to get this work.

Thanks!

Kind regards

[Tim Müller]

On Sun, 2017-02-05 at 11:45 -0800, ivan-perez wrote:

Hi,

> For clarification, the H264 stream comes from the Raspberry camera,
> and I directly pipe it into two GStreamer instances:
>
> * One instance is running all the time, as it takes care of producing
> a RTP stream which clients may see at any time.
> * Another instance only runs if some conditions are met (for example,
> after some detection movement). In that case, I want to save a video
> of 30 seconds (in MP4). My idea is to use the stream directly taken
> from the Raspberry camera, but at the moment it doesn't work because
> -I think- in order to work properly it'd need the first bytes of the
> stream, which may contain some headers.

You could also use rpicamsrc from

https://github.com/thaytan/gst-rpicamsrc

(which is preferably to raspivid) and run both things in the same
application?

e.g. here's a pipeline that can start/stop recording a stream (with a
few seconds of backlog even) on demand:

https://people.freedesktop.org/~tpm/code/test-backlog-recording-h264.c

Just have to switch out the bit at the beginning with rpicamsrc.

Cheers
 -Tim

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[ivan-perez]

Tim Müller wrote

On Sun, 2017-02-05 at 11:45 -0800, ivan-perez wrote:

Hi,

> For clarification, the H264 stream comes from the Raspberry camera,
> and I directly pipe it into two GStreamer instances:
>
> * One instance is running all the time, as it takes care of producing
> a RTP stream which clients may see at any time.
> * Another instance only runs if some conditions are met (for example,
> after some detection movement). In that case, I want to save a video
> of 30 seconds (in MP4). My idea is to use the stream directly taken
> from the Raspberry camera, but at the moment it doesn't work because
> -I think- in order to work properly it'd need the first bytes of the
> stream, which may contain some headers.

You could also use rpicamsrc from

https://github.com/thaytan/gst-rpicamsrc

(which is preferably to raspivid) and run both things in the same
application?

e.g. here's a pipeline that can start/stop recording a stream (with a
few seconds of backlog even) on demand:

https://people.freedesktop.org/~tpm/code/test-backlog-recording-h264.c

Just have to switch out the bit at the beginning with rpicamsrc.

Cheers
 -Tim

--
Tim Müller, Centricular Ltd - http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

Thanks for your reply, Tim. It looks very promising! I'm going to check it right now.

Kind regards