Android Media amcvidenc-omxgoogleh264encoder plugin issues

[Stefan Persson]

I'm trying to replace x264enc in my Android app with the hardware accelerated
amcvidenc-omxgoogleh264encoder but I get error messages when I test it with
videotestsrc.

Pipeline for testing: videotestsrc ! amcvidenc-omxgoogleh264encoder !
rtph264pay ! udpsink host=192.168.1.3 port=47372

Error log:
D/MediaCodec: init()++ name=OMX.google.h264.encoder,nameIsType=0,encoder=0
I/ACodec: [OMX.google.h264.encoder] Now Loaded
D/MediaCodec: kWhatComponentAllocated received
D/MediaCodec: init()--
D/MediaCodec: kWhatConfigure received
I/ExtendedACodec: setupVideoEncoder()
I/ACodec: setupAVCEncoderParameters with [profile: Baseline] [level:
Level41]
I/ACodec: [OMX.google.h264.encoder] cannot encode color aspects. Ignoring.
I/ACodec: [OMX.google.h264.encoder] cannot encode HDR static metadata.
Ignoring.
I/ACodec: setupVideoEncoder succeeded
W/ExtendedACodec: Failed to get extension for extradata parameter
D/MediaCodec: onComponentConfigured()
D/MediaCodec: kWhatComponentConfigured received
D/MediaCodec: start()
D/MediaCodec: kWhatStart received
[OMX.google.h264.encoder] Now Idle->Executing
[OMX.google.h264.encoder] Now Executing
gstpad.c:4226:gst_pad_peer_query:<amcvideoenc-omxgoogleh264encoder0:src>
could not send sticky events
gstamcvideoenc.c:1143:gst_amc_video_enc_loop:<amcvideoenc-omxgoogleh264encoder0>
error: Internal data stream error.
gstamcvideoenc.c:1143:gst_amc_video_enc_loop:<amcvideoenc-omxgoogleh264encoder0>
error: streaming stopped, reason not-negotiated (-4)

How supported is the wrapper for amcvideoenc-omxgoogleh264encoder? I'd like
to get more debug logs from amcvideoenc-omxgoogleh264encoder to see if I'm
missing any parameters or undocumented caps. I've tried adding videoconvert
before the encoder and setting the caps for the encoder
("video/x-h264,width=320,height=240") but that didn't solve the issue.
I would of course like to replace videotestsrc with something more useful
like ahcsrc as source but I'm a bit confused that it doesn't even work with
the simple test source.

Thanks in advance.
Stefan



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Matthew Waters]

Just a note that omxgoogle elements are usually software implementations.

A GStreamer debug log is definitely a good start for looking into this.

Cheers
-Matt

On 15/5/19 12:35 am, Stefan Persson wrote:

> I'm trying to replace x264enc in my Android app with the hardware accelerated
> amcvidenc-omxgoogleh264encoder but I get error messages when I test it with
> videotestsrc.
>
> Pipeline for testing: videotestsrc ! amcvidenc-omxgoogleh264encoder !
> rtph264pay ! udpsink host=192.168.1.3 port=47372
>
> Error log:
> D/MediaCodec: init()++ name=OMX.google.h264.encoder,nameIsType=0,encoder=0
> I/ACodec: [OMX.google.h264.encoder] Now Loaded
> D/MediaCodec: kWhatComponentAllocated received
> D/MediaCodec: init()--
> D/MediaCodec: kWhatConfigure received
> I/ExtendedACodec: setupVideoEncoder()
> I/ACodec: setupAVCEncoderParameters with [profile: Baseline] [level:
> Level41]
> I/ACodec: [OMX.google.h264.encoder] cannot encode color aspects. Ignoring.
> I/ACodec: [OMX.google.h264.encoder] cannot encode HDR static metadata.
> Ignoring.
> I/ACodec: setupVideoEncoder succeeded
> W/ExtendedACodec: Failed to get extension for extradata parameter
> D/MediaCodec: onComponentConfigured()
> D/MediaCodec: kWhatComponentConfigured received
> D/MediaCodec: start()
> D/MediaCodec: kWhatStart received
> [OMX.google.h264.encoder] Now Idle->Executing
> [OMX.google.h264.encoder] Now Executing
> gstpad.c:4226:gst_pad_peer_query:<amcvideoenc-omxgoogleh264encoder0:src>
> could not send sticky events
> gstamcvideoenc.c:1143:gst_amc_video_enc_loop:<amcvideoenc-omxgoogleh264encoder0>
> error: Internal data stream error.
> gstamcvideoenc.c:1143:gst_amc_video_enc_loop:<amcvideoenc-omxgoogleh264encoder0>
> error: streaming stopped, reason not-negotiated (-4)
>
> How supported is the wrapper for amcvideoenc-omxgoogleh264encoder? I'd like
> to get more debug logs from amcvideoenc-omxgoogleh264encoder to see if I'm
> missing any parameters or undocumented caps. I've tried adding videoconvert
> before the encoder and setting the caps for the encoder
> ("video/x-h264,width=320,height=240") but that didn't solve the issue.
> I would of course like to replace videotestsrc with something more useful
> like ahcsrc as source but I'm a bit confused that it doesn't even work with
> the simple test source.
>
> Thanks in advance.
> Stefan

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Stefan Persson]

Thanks.

I've uploaded logs to https://paste.ee/p/vYUSE

They were taken using
https://gitlab.freedesktop.org/gstreamer/gst-docs/tree/master/examples/tutorials/android/android-tutorial-3
(with the surface layer removed) with GStreamer 1.16.0 and GST_DEBUG =
"amc*:6"
running the following pipeline:
videotestsrc ! videoconvert ! amcvidenc-omxgoogleh264encoder ! rtph264pay !
udpsink host=192.168.1.3 port=47372

The code for tutorial-3.c including my modifications can be found at
https://paste.ee/p/PZ6Iz

The plugins section in Android.mk also needs to be modified to include
Android Media. I'm using:
GSTREAMER_PLUGINS := $(GSTREAMER_PLUGINS_CORE) $(GSTREAMER_PLUGINS_PLAYBACK)
$(GSTREAMER_PLUGINS_EFFECTS) $(GSTREAMER_PLUGINS_CODECS)
$(GSTREAMER_PLUGINS_BAD) $(GSTREAMER_PLUGINS_NET) $(GSTREAMER_PLUGINS_SYS)
$(GSTREAMER_PLUGINS_CODECS_RESTRICTED)

Perhaps the most interesting parts from the log:
<amcvideoenc-omxgoogleh264encoder0> Got new output format:
{max-bitrate=2097152, csd-1=java.nio.HeapByteBuffer[pos=0 lim=10 cap=10],
mime=video/avc, width=320, bitrate=2097152, height=240,
csd-0=java.nio.HeapByteBuffer[pos=0 lim=17 cap=17]}
<amcvideoenc-omxgoogleh264encoder0:src> could not send sticky events
<amcvideoenc-omxgoogleh264encoder0> Waiting for available output buffer
<amcvideoenc-omxgoogleh264encoder0> Handling frame
<amcvideoenc-omxgoogleh264encoder0> Got output buffer at index 0: size 27
time 0 flags 0x00000002
<amcvideoenc-omxgoogleh264encoder0> got codecconfig in byte-stream format
<amcvideoenc-omxgoogleh264encoder0> Finished frame: not-negotiated
<amcvideoenc-omxgoogleh264encoder0> error: Internal data stream error.

I get similar messages when I try with amcvideoenc-omxqcomvideoencoderavc,
which is available on my phone.

With that element it starts well:
<amcvideoenc-omxqcomvideoencoderavc0> Setting new caps video/x-raw,
width=(int)320, height=(int)240, framerate=(fraction)30/1,
format=(string)NV12, multiview-mode=(string)mono,
pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive
<amcvideoenc-omxqcomvideoencoderavc0> picking an output format ...
<amcvideoenc-omxqcomvideoencoderavc0> chose caps video/x-h264, width=(int)[
16, 4096 ], height=(int)[ 16, 4096 ], framerate=(fraction)[ 0/1,
2147483647/1 ], parsed=(boolean)true, stream-format=(string)byte-stream,
alignment=(string)au, profile=(string)high, level=(string){ 1, 1b, 1.1, 1.2,
1.3, 2, 2.1, 2.2, 3, 3.1, 3.2, 4, 4.1, 4.2, 5, 5.1 }; video/x-h264,
width=(int)[ 16, 4096 ], height=(int)[ 16, 4096 ], framerate=(fraction)[
0/1, 2147483647/1 ], parsed=(boolean)true,
stream-format=(string)byte-stream, alignment=(string)au,
profile=(string)main, level=(string){ 1, 1b, 1.1, 1.2, 1.3, 2, 2.1, 2.2, 3,
3.1, 3.2, 4, 4.1, 4.2, 5, 5.1 }; video/x-h264, width=(int)[ 16, 4096 ],
height=(int)[ 16, 4096 ], framerate=(fraction)[ 0/1, 2147483647/1 ],
parsed=(boolean)true, stream-format=(string)byte-stream,
alignment=(string)au, profile=(string)baseline, level=(string){ 1, 1b, 1.1,
1.2, 1.3, 2, 2.1, 2.2, 3, 3.1, 3.2, 4,
<amcvideoenc-omxqcomvideoencoderavc0> Color format info: {color_format=21,
width=320, height=240, stride=320, slice-height=240, crop-left=0,
crop-top=0, crop-right=0, crop-bottom=0, frame-size=115200}
<amcvideoenc-omxqcomvideoencoderavc0> Configuring codec with format:
{color-format=21, slice-height=240, i-frame-interval=0, mime=video/avc,
width=320, bitrate=2097152, stride=320, frame-rate=30.0, height=240}

But still eventually:
<amcvideoenc-omxqcomvideoencoderavc0> Output format has changed
<amcvideoenc-omxqcomvideoencoderavc0> Got new output format:
{max-bitrate=2097152, csd-1=java.nio.HeapByteBuffer[pos=0 lim=8 cap=8],
mime=video/avc, width=320, bitrate=2097152, height=240,
csd-0=java.nio.HeapByteBuffer[pos=0 lim=20 cap=20]}
<amcvideoenc-omxqcomvideoencoderavc0:src> could not send sticky events
<amcvideoenc-omxqcomvideoencoderavc0> Waiting for available output buffer
<amcvideoenc-omxqcomvideoencoderavc0> Handling frame
<amcvideoenc-omxqcomvideoencoderavc0> Got output buffer at index 0: size 28
time 0 flags 0x00000002
<amcvideoenc-omxqcomvideoencoderavc0> got codecconfig in byte-stream format
<amcvideoenc-omxqcomvideoencoderavc0> Finished frame: not-negotiated
<amcvideoenc-omxqcomvideoencoderavc0> error: Internal data stream error.

Is there any element I need to insert between
amcvidenc-omxgoogleh264encoder/amcvideoenc-omxqcomvideoencoderavc and
rtph264pay, or between the latter and udpsink? It hasn't been needed for
x264enc.

Thanks!
Stefan



--
Sent from: http://gstreamer-devel.966125.n4.nabble.com/
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel