How to convert avc/au to byte-stream/au ?

[rland]

Hi,
AFAIK,Most of MediaCodec HW decoder seems only support stream-format = byte-stream(?), but when playing mp4 over http using playbin, it seems that h264parse is output stream-format = avc / avc1, causing h264parse<=>Mediacodec HW video decoder  nego failed.The same mp4 file, Nuplayer can use  HW decoder, but gstreamer can only use the SW decoder.

Non-custom pipeline, PLAYBIN2 automatically loaded plugin case,
is there any way to force h264parse(or more upstream element) to out byte-stream，then allows  HW decoding to be available. - -)

regards,
luckychou

[rland]

lucky chou wrote

Hi,
, but gstreamer can only use the SW decoder.

Can anyone help me on this?
Please confirm if any other information is required.

Regards,
luckychou

[msemsar]

Hi Lucky,

h264parse supports both avc and byte-stream, you can check it yourself by running  gst-inspect-1.0 h264parse.

Just set the stream-format of h264parse to byte-stream.

yours,

On Mon, Jun 12, 2017 at 9:51 AM, lucky chou <[hidden email]> wrote:

lucky chou wrote
> Hi,
> , but gstreamer can only use the SW decoder.

Can anyone help me on this?
Please confirm if any other information is required.

Regards,
luckychou



--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/How-to-convert-avc-au-to-byte-stream-au-tp4683201p4683313.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

--

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[rland]

Dear msemsar,
thanks for your comment.

msemsar wrote

Hi Lucky,
h264parse supports both avc and byte-stream, you can check it yourself by
running  gst-inspect-1.0 h264parse.
Just set the stream-format of h264parse to byte-stream.
yours,

Yes,I have checked, h264parse does support byte-stream.
In fact, I should describe it more clearly, sorry. If h264parse detected　stream is avc, how can it also let it output byte-stream?

Like the following log：
-------------
01-01 09:05:02.867 12796 13371 D gstplayer: enter setDataSource url=http://10.9.44.131/test.mp4 player_state=0
...
01-01 09:05:18.148 12796 13539 D [gsth264parse.c:331] <h264parse> parsing caps: video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3.1, profile=(string)high, codec_data=(buffer)0164001fffe1001a6764001facd1005005bb011000000300100000030328f162d12001000568ebe11cb0, width=(int)1280, height=(int)720, framerate=(fraction)25/1, pixel-aspect-ratio=(fraction)1/1
01-01 09:05:18.148 12796 13539 D [gsth264parse.c:2592] <h264parse> have packetized h264
01-01 09:05:18.148 12796 13539 D [gsth264parse.c:2622] <h264parse> profile 64001f
01-01 09:05:18.148 12796 13539 D [gsth264parse.c:2630] <h264parse> nal length size 4

...
01-01 09:05:19.812 12796 13539 D  [gstvideodecoder.c:537] <videodecoder> gst_video_decoder_init
01-01 09:05:19.812 12796 13539 D  [gstvideodecoder.c:1996] <videodecoder> reset full 1
01-01 09:05:19.813 12796 13539 D  [gstvideodecoder.c:1870] <videodecoder> received query 40963, accept-caps
01-01 09:05:19.813 12796 13539 D  [gstvideodecoder.c:1719] <videodecoder> received query 43523, caps
01-01 09:05:19.813 12796 13539 D  [gstvideodecoder.c:1996] <videodecoder> reset full 1
01-01 09:05:19.814 12796 13539 D  [gstvideodecoder.c:1379] <videodecoder> received event 10254, stream-start
01-01 09:05:19.814 12796 13539 D  [gstvideodecoder.c:1096] <videodecoder> received STREAM_START. Clearing taglist
01-01 09:05:19.814 12796 13539 D  [gstvideodecoder.c:852] <videodecoder> pushing event stream-start
01-01 09:05:19.814 12796 13539 D  [gstvideodecoder.c:1870] <videodecoder> received query 40963, accept-caps
01-01 09:05:19.814 12796 13539 D  [gstvideodecoder.c:1379] <videodecoder> received event 12814, caps
01-01 09:05:19.814 12796 13539 D  [gstvideodecoder.c:679] <videodecoder> setcaps video/x-h264, parsed=(boolean)true, stream-format=(string)avc, alignment=(string)au, width=(int)1280, height=(int)720, framerate=(fraction)25/1, profile=(string)high
01-01 09:05:19.928 12796 13539 D libchplayer: [gstvideodecoder.c:1719] <videodecoder> received query 43523, caps

As shown above, because the stream itself is avc, so h264parse very happy to choose  stream-format=avc, then  videodecoder in nego can only match avc, which causing HW decoder (amcvideodec*)  can't be selected.

If I understanding what is wrong, please help correct me,
thank you.

Regards,
luckychou

[Arjen Veenhuizen]

Try inserting a capsfilter after h264parse and setting its caps property to "video/x-h264,alignment=nal,stream-format=byte-stream". If that does not work, you can try with alignment=au. Not sure which one amcvideodec* accepts but I would expect nal-alignment to do the trick.

[rland]

Arjen Veenhuizen wrote

Try inserting a capsfilter after h264parse and setting its caps property to "video/x-h264,alignment=nal,stream-format=byte-stream". If that does not work, you can try with alignment=au. Not sure which one amcvideodec* accepts but I would expect nal-alignment to do the trick.

Hi ,Arjen
Well, uhuhuh... maybe the problem is that, I'm using playbin2,it seems that there's no easy way to add a capsfilter ?

[Arjen Veenhuizen]

Construct the pipeline yourself? Use this information to create an diagram of the pipeline created by playbin to see what is happening under the hood. That should give you all the information you need.

[rland]

Arjen Veenhuizen wrote

Construct the pipeline yourself? Use this information to create an diagram of the pipeline created by playbin to see what is happening under the hood. That should give you all the information you need.

 Arjen , Thanks  for your tips.
No, since I want to get a generic player, it is not a good idea to build a custom  pipeline.

[Nicolas Dufresne-5]

Le mercredi 14 juin 2017 à 00:57 -0700, lucky chou a écrit :

> Arjen Veenhuizen wrote
> > Construct the pipeline yourself? Use 
> > this
> > <https://developer.ridgerun.com/wiki/index.php/How_to_generate_a_Gs
> > treamer_pipeline_diagram_(graph)>  
> >  information to create an diagram of the pipeline created by
> > playbin to
> > see what is happening under the hood. That should give you all the
> > information you need.
>
>  Arjen , Thanks  for your tips.
> No, since I want to get a generic player, it is not a good idea to
> build a
> custom  pipeline.
>

decodebin should build a pipeline with:

  h264parse ! amch264...dec ! ...

And then h264parse should be able to negotiate the proper format and
will do the conversion automatically. When that does not work, it means
that there is a bug somewhere. First step, and what is being proposed
here, it to do some tests to figure-out what format works. Later you
should dig this up to find out were is the bug. It's likely that your
Android platform CODEC is not standard (e.g. expecting AVC instead of
byte-stream), for that you would implement a quirk in the amc elements.

regards,
Nicolas
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[rland]

Nicolas Dufresne-5 wrote

decodebin should build a pipeline with:

  h264parse ! amch264...dec ! ...

And then h264parse should be able to negotiate the proper format and
will do the conversion automatically. When that does not work, it means
that there is a bug somewhere. First step, and what is being proposed
here, it to do some tests to figure-out what format works. Later you
should dig this up to find out were is the bug. It's likely that your
Android platform CODEC is not standard (e.g. expecting AVC instead of
byte-stream), for that you would implement a quirk in the amc elements.

Nicolas,thanks for your attention.
I will follow your tips, first to check what format HW decoder can work properly, try to narrow the scope of the investigation. I have verified through other ways, I used CODEC only byte-stream can work properly, <stream-format=avc> mediacodec always reported can't find  frame start code, seems to be what we expected.

regards,
luckyc