Android HW+ decoding problem

[Waziri]

I am working  android tv box(amlogic S912) device with Android 6.0. I used gstreamer binaries from https://gstreamer.freedesktop.org/data/pkg/android/ and tried last couple of versions.
To use Gstreamer from my application I used Android NDK and created pipelines with both "playbin" and "rtspsrc" to capture display from some rtsp/h264 security cameras. "playbin" uses "avdec_h264" to decode h264 stream. I tried also "openh264dec" to decode h264stream.
The problem is that both avdec_264 and openh264dec are software codecs and if i run multiple pipeline to display 2 or more cameras videos fall behind and come from back. I want to use hardware acceleration or hardware h264decoder or something like that but could not achieve yet. I need to learn hardware accelerated video decoding on android with gstreamer.
Media codec is not an solution because you can not open more than 5 instance with media codec. I need to open more.

[rland]

Maybe you have to build the SDK yourself using cerbero.
For some reasons, androidmedia has chosen to block non-google third-party
vendors's  HW decoder.

As noted in the comment,
gst-plugins-bad/sys/androidmedia/gstamc.c

static gboolean
scan_codecs (GstPlugin * plugin)
{
 ...
     /* FIXME: Non-Google codecs usually just don't work and hang forever
     * or crash when not used from a process that started the Java
     * VM via the non-public AndroidRuntime class. Can we somehow
     * initialize all this?
     */
       if (gst_amc_jni_is_vm_started () &&
       !g_str_has_prefix (name_str, "OMX.google.")) {
       GST_INFO ("Skipping non-Google codec '%s' in standalone mode",
name_str);
       valid_codec = FALSE;
       goto next_codec;
       }
   ...
｝  
Just annotate this piece of code, and then debug your HW  decoder to check
if it works.
Good luck. --)





--
View this message in context: http://gstreamer-devel.966125.n4.nabble.com/Android-HW-decoding-problem-tp4683293p4683320.html
Sent from the GStreamer-devel mailing list archive at Nabble.com.
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Waziri]

Thank you I will try and tell you. Thanks for your attention

[Sebastian Dröge-3]

On Mon, 2017-06-12 at 19:39 -0700, lucky chou wrote:
> Maybe you have to build the SDK yourself using cerbero.
> For some reasons, androidmedia has chosen to block non-google third-
> party vendors's  HW decoder.
> [...]

That's only in standalone mode as the comment suggests, i.e. when *not*
running GStreamer in a proper Android application but just on the
commandline.

For using the hardware codecs properly, you have to use an Android
application that is started via the app starter. Otherwise various
pieces are not initialized correctly.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Waziri]

I have an app already. When I try to decode 2MP 25FPS 4 cameras CPU cannot handle. So, I want to use hardware but I cannot use it with amlogic android app. Is there any way to activate it?

Also, I do not understand what you mean by App starter?

[Waziri]

I have developed ann app by using android NDK. If I use media codec it does not let to open many instances. I need to show 16 video. So, I am using gstreamer. It press to many to CPU so I need to use hw decoder but I do not know how to activate it.

[Sebastian Dröge-3]

On Tue, 2017-06-13 at 04:57 -0700, Waziri wrote:
> I have developed ann app by using android NDK. If I use media codec it does
> not let to open many instances. I need to show 16 video. So, I am using
> gstreamer. It press to many to CPU so I need to use hw decoder but I do not
> know how to activate it.

If your hardware can only be used for fewer videos in parallel than you
need and CPU decoding is not an option, you'll have to get different
hardware.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Waziri]

No there is an another app it supports more cameras than me, it uses ffmpeg. So it is about using hw decoder I guess. 

2017-06-13 15:35 GMT+03:00 Sebastian Dröge <[hidden email]>:

On Tue, 2017-06-13 at 04:57 -0700, Waziri wrote:
> I have developed ann app by using android NDK. If I use media codec it does
> not let to open many instances. I need to show 16 video. So, I am using
> gstreamer. It press to many to CPU so I need to use hw decoder but I do not
> know how to activate it.

If your hardware can only be used for fewer videos in parallel than you
need and CPU decoding is not an option, you'll have to get different
hardware.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Sebastian Dröge-3]

On Tue, 2017-06-13 at 15:36 +0300, Fatih Sabri AKTEPE wrote:
> No there is an another app it supports more cameras than me, it uses
> ffmpeg. So it is about using hw decoder I guess. 

You'll have to check how that other app does it then. The only *public*
API for using the hardware codecs is the MediaCodec API, and that's
also what ffmpeg is using. If MediaCodec can only be opened < 16 times
on your target hardware, then you have to find another solution.

Note that you might be able to use hardware specific APIs, but those
are not public, official Android APIs.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Waziri]

Okey thanks alot.

2017-06-13 15:40 GMT+03:00 Sebastian Dröge <[hidden email]>:

On Tue, 2017-06-13 at 15:36 +0300, Fatih Sabri AKTEPE wrote:
> No there is an another app it supports more cameras than me, it uses
> ffmpeg. So it is about using hw decoder I guess. 

You'll have to check how that other app does it then. The only *public*
API for using the hardware codecs is the MediaCodec API, and that's
also what ffmpeg is using. If MediaCodec can only be opened < 16 times
on your target hardware, then you have to find another solution.

Note that you might be able to use hardware specific APIs, but those
are not public, official Android APIs.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[rrazavi]

I'm using the same device, Android media plugin loads plugins dynamically, with gstreamer 1.10.2 SDK the element that you have to use is  amcviddec-omxamlogicavcdecoderawesome.
The device that I'm using is T95K Pro, if some how this element do not exist for your device it has something to do with Android Media API or your device ROM.

[Waziri]

Hi rrazavi, you mean that you can use hardware while decoding by that way?

[rrazavi]

Yes that enables you to use H264 hardware decoder. keep in mind this element will not get chosen automatically by decodebin and you have to construct the pipeline yourself like this:
... ! h264parse ! amcviddec-omxamlogicavcdecoderawesome ! ....

[Sebastian Dröge-3]

On Tue, 2017-06-13 at 07:08 -0700, rrazavi wrote:
> Yes that enables you to use H264 hardware decoder. keep in mind this
> element will not get chosen automatically by decodebin and you have
> to construct the pipeline yourself like this:
> ... ! h264parse ! amcviddec-omxamlogicavcdecoderawesome ! ....

It will be chosen automatically by decodebin/playbin if it's suitable
for the stream. That's the whole point of it :)

Why isn't it automatically chosen in your case?

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[Waziri]

Thanks all of you for your response. I will let you know after implementing this.

13 Haz 2017 Sal, saat 18:32 tarihinde Sebastian Dröge-3 [via GStreamer-devel] <[hidden email]> şunu yazdı:

On Tue, 2017-06-13 at 07:08 -0700, rrazavi wrote:
> Yes that enables you to use H264 hardware decoder. keep in mind this
> element will not get chosen automatically by decodebin and you have
> to construct the pipeline yourself like this:
> ... ! h264parse ! amcviddec-omxamlogicavcdecoderawesome ! ....

It will be chosen automatically by decodebin/playbin if it's suitable
for the stream. That's the whole point of it :)

Why isn't it automatically chosen in your case?

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com
_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

signature.asc (981 bytes) Download Attachment

If you reply to this email, your message will be added to the discussion below:

http://gstreamer-devel.966125.n4.nabble.com/Android-HW-decoding-problem-tp4683293p4683352.html

To unsubscribe from Android HW+ decoding problem, click here.
NAML

[rrazavi]

Well I'm new in using gstreamer in Android, Mostly I've used gst-remote-launch to explore it and because I still do not know how to get debug output of a specific element (http://gstreamer-devel.966125.n4.nabble.com/debuging-androidmedia-jpeg-decoder-in-gst-remote-launch-td4682806.html) I can not find the answer to this question that why this is happening.
There's a lot of other stuff about android that I have no Idea if the worth sharing or not like:
1- I changed cerbero gstreamer config file to use neon instructions for ARMv7 and I'm getting better performance special for jpegdec element(enables libjpeg turbo to use neon SIMD)
2- Compiling gstreamer andoid SDK using -fPIE enabled me to use it at runtime level
3- Problems with Android media at runtime level(https://bugzilla.gnome.org/show_bug.cgi?id=781860)
4- Android media completely fails to use hardware jpeg decoder(https://bugzilla.gnome.org/show_bug.cgi?id=781861)

[Waziri]

Hi, I do not know how to change gstreamer config. Can you send this changed module with cerbero?
This problem is very important for me if you can help me and send me how you use hardware in code I would be appreciated. Thank you very much.

2017-06-13 22:19 GMT+03:00 rrazavi [via GStreamer-devel] <[hidden email]>:

Well I'm new in using gstreamer in Android, Mostly I've used gst-remote-launch to explore it and because I still do not know how to get debug output of a specific element (http://gstreamer-devel.966125.n4.nabble.com/debuging-androidmedia-jpeg-decoder-in-gst-remote-launch-td4682806.html) I can not find the answer to this question that why this is happening.
There's a lot of other stuff about android that I have no Idea if the worth sharing or not like:
1- I changed cerbero gstreamer config file to use neon instructions for ARMv7 and I'm getting better performance special for jpegdec element(enables libjpeg turbo to use neon SIMD)
2- Compiling gstreamer andoid SDK using -fPIE enabled me to use it at runtime level
3- Problems with Android media at runtime level(https://bugzilla.gnome.org/show_bug.cgi?id=781860)
4- Android media completely fails to use hardware jpeg decoder(https://bugzilla.gnome.org/show_bug.cgi?id=781861)

---

If you reply to this email, your message will be added to the discussion below:

http://gstreamer-devel.966125.n4.nabble.com/Android-HW-decoding-problem-tp4683293p4683356.html

To unsubscribe from Android HW+ decoding problem, click here.
NAML

[rrazavi]

Does problems does not have anything to do with HW decoder support, I just wanted to bring attention to some other issues that I faced, sorry that I confused you by bringing them up here. using available gstreamer SDK should be enough to use HW acceleration. once again I'm using Amlogic S912 chip and hardware decoder element name will be different for other chips. as I told before I'm not writing any code and I'm just using https://github.com/sdroege/gst-launch-remote for my experiments.
you should be able to compile and install gst-remote-launch using android studio.
These are the two pipelines that I run:

nc 192.168.120.27 9123
filesrc location=/mnt/sdcard/Movies/bbb_sunflower_1080p_30fps_normal.mp4 ! decodebin ! queue ! glupload ! glimagesink
OK
+PLAY
OK
filesrc location=/mnt/sdcard/Movies/bbb_sunflower_1080p_30fps_normal.mp4 ! qtdemux ! h264parse ! amcviddec-omxamlogicavcdecoderawesome ! queue ! glupload ! glimagesink
OK
+PLAY
OK

The first pipeline put average CPU usage to 30% and playback is not smooth
The second one average CPU usage is below 10% and it gives you smooth playback

[Waziri]

I found that there is a xml file that contains decoder plugins which changes from device to device. You can find that file through etc/system/mediacodecs.xml file. So check that file and use that file for your app according to device in order to use hardware codecs.