Gstreamer and Android app

[de_ninja]

"Hey Guys,  i asked this Question in Stack Overflow: http://stackoverflow.com/questions/36969192/audio-data-from-gstreamer-to-android
Maybe you can tell me if audio streamed with gstreamer and encoded with faac has some sort of "extra Bytes" or "information Bytes" that I cant decode with the Android Decoder or if its just raw AAC. Maybe there are some commandline options that could help me. Or is it impossible to decode a gstreamer faac stream with the Media Codec in android?
Would be nice if you could help me!

[Sebastian Dröge-3]

On So, 2016-05-01 at 12:29 -0700, de_ninja wrote:
> "Hey Guys,  i asked this Question in Stack Overflow:
> http://stackoverflow.com/questions/36969192/audio-data-from-gstreamer-to-android
> Maybe you can tell me if audio streamed with gstreamer and encoded with faac
> has some sort of "extra Bytes" or "information Bytes" that I cant decode
> with the Android Decoder or if its just raw AAC. Maybe there are some
> commandline options that could help me. Or is it impossible to decode a
> gstreamer faac stream with the Media Codec in android?
> Would be nice if you could help me!

It's unlikely that the Android MediaExtractor supports raw AAC files
without any container. Try putting it into a MP4 container, then it
should work.

How are you generating the AAC on the GStreamer side, what is the
pipeline?

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com


_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[de_ninja]

Like you said I put the aac stream in a container with the following pipeine:

gst-launch-1.0 audiotestsrc ! faac ! "audio/mpeg, mpegversion=4" ! udpsink host=224.0.0.1 port=8001

Still the same error. I tried to write the incoming Bytes to an aac file and play it, but it didn't work. So I think the Android media extractor isn't the problem.
I write the data to the file with the following code:

String filename = "AudioData";

File file = new File(Environment.getExternalStorageDirectory(), filename);
FileOutputStream fos;
fos = new FileOutputStream(file, true);

int i = 0;
byte[] data;

while(i<=100) {

    multicastSocket.receive(packet);
    data = new byte[packet.getLength()];
    System.arraycopy(packet.getData(), packet.getOffset(), data, 0, packet.getLength());
    Log.e("UDPserver", data.length + " bytes received");
    fos.write(data);
    i++;
}


what do i have to change at the file that it is playable? (then it may work with media extractor too), i tried to name it .aac and .mp4 but both didn't work. Do I need a header in an aac file?

AudioData.AudioData

[Sebastian Dröge-3]

On Mo, 2016-05-02 at 11:36 -0700, de_ninja wrote:
> Like you said I put the aac stream in a container with the following
> pipeine:
>
> gst-launch-1.0 audiotestsrc ! faac ! "audio/mpeg, mpegversion=4" ! udpsink
> host=224.0.0.1 port=8001
>
> Still the same error. I tried to write the incoming Bytes to an aac file and
> play it, but it didn't work. So I think the Android media extractor isn't
> the problem. 

That's not putting it into a container, it's still a raw AAC stream.
And depending on the stream format, this is also very hard to play back
as you will lose all packetization when writing it to a file.

Use some streamable container here, I guess if you need Android to read
the file your best choice is AAC in MPEG-TS, especially as you want to
transmit it over UDP.

(Before you ask, MP4 is not a streamable container format)


It might also be an option to just use GStreamer in the Android app
too, you're much more flexible then with regard to supported codecs,
container formats and transport protocols.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[de_ninja]

In Android Media Codec it sais:
 "audio/mp4a-latm" - AAC audio (note, this is raw AAC packets, not packaged in LATM!)
Android Media Codec
I think this means that raw AAC is playable, am I wrong?

Is this a container format I could use?:
https://developer.ridgerun.com/wiki/index.php/Gstreamer_MPEG2_TS_pipelines
I'm not sure how to install this plugin in gstreamer, which command do I have to use?

I thought about using GStreamer in the Android app too. But as you probably noticed I am not very advanced in programming so I would have to learn programming with Gstreamer from the beginning and build my whole projekt new. This would cost a lot of time. For me using GStreamer would be the last option.

[Sebastian Dröge-3]

On Di, 2016-05-03 at 06:44 -0700, de_ninja wrote:
> In Android Media Codec it sais:
>  "audio/mp4a-latm" - AAC audio (note, this is raw AAC packets, not packaged
> in LATM!)
> Android Media Codec
>   
> I think this means that raw AAC is playable, am I wrong?

In LATM stream format (or LOAS) I would say, not any of the others.

> Is this a container format I could use?:
> https://developer.ridgerun.com/wiki/index.php/Gstreamer_MPEG2_TS_pipelines
>   
> I'm not sure how to install this plugin in gstreamer, which command do I
> have to use?

mpegtsmux can create MPEG-TS streams.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[de_ninja]

Now I used mpegtsmus to create a MPEG-TS stream and the extractor works! Thanks!

This is the pipeline i set:

gst-launch-1.0 audiotestsrc ! faac ! mux. mpegtsmux name=mux ! udpsink host=224.0.0.1 port=8001

An aac-decoder gets created!
Now the decoder fails with the following error:
aacDecoder_DecodeFrame decoderErr = 0x4004
AAC decoder returned error 0x4004, substituting silence

I searched for the error 0x4004 and it means: AAC_DEC_DECODE_FRAME_ERROR  < The parsed bitstream value is out of range. Most probably the bitstream is corrupt, or the system crashed.

Do you have an idea what could be wrong?

[Sebastian Dröge-3]

On Di, 2016-05-03 at 09:43 -0700, de_ninja wrote:

> Now I used mpegtsmus to create a MPEG-TS stream and the extractor works!
> Thanks!
>
> This is the pipeline i set:
>
> gst-launch-1.0 audiotestsrc ! faac ! mux. mpegtsmux name=mux ! udpsink
> host=224.0.0.1 port=8001 
>
> An aac-decoder gets created!
> Now the decoder fails with the following error:
> aacDecoder_DecodeFrame decoderErr = 0x4004
> AAC decoder returned error 0x4004, substituting silence
>
> I searched for the error 0x4004 and it means: AAC_DEC_DECODE_FRAME_ERROR  <
> The parsed bitstream value is out of range. Most probably the bitstream is
> corrupt, or the system crashed. 
>
> Do you have an idea what could be wrong? 

Can you play the file with another software? It might be that the AAC
stream is in a profile that can't be handled by Android. Try enforcing
the LC (low complexity) profile, or alternatively try using voaacenc
instead of faac.

--
Sebastian Dröge, Centricular Ltd · http://www.centricular.com

_______________________________________________
gstreamer-devel mailing list
[hidden email]
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel

[de_ninja]

Sebastian Dröge-3 wrote

Can you play the file with another software?

Yes! It was playable with the normal Android Music App and it sounds good, so I think the part of the programm where I receive the data works well.

Sebastian Dröge-3 wrote

Try enforcing the LC (low complexity) profile

In the faac wiki it sais that the default value of the profile is "LC", so I think I have nothing to change here.
faac wiki page

I also tried my code with an AAC sample track instead of the received file but there is the same problem, the extractor works but the decoder doesn't.
I think something has to be wrong with the code of my decoder.

I know this isn't the right forum for this, but maybe you can have a look at my decoder code:


            format = extractor.getTrackFormat(0);
            String mime = format.getString(MediaFormat.KEY_MIME);
            decoder = MediaCodec.createDecoderByType(mime);
            decoder.configure(format, null, null, 0);


            while (!stopTrack) {


                multicastSocket.receive(packet);

                decoder.start();
                data = packet.getData();

                inputBuffers = decoder.getInputBuffers();
                outputBuffers = decoder.getOutputBuffers();
                inputBufferIndex = decoder.dequeueInputBuffer(-1);

                if (inputBufferIndex >= 0)
                {
                    inputBuffer = inputBuffers[inputBufferIndex];
                    inputBuffer.clear();

                    inputBuffer.put(data);

                    decoder.queueInputBuffer(inputBufferIndex, 0, data.length, 0, 0);
                }

                bufferInfo = new MediaCodec.BufferInfo();
                outputBufferIndex = decoder.dequeueOutputBuffer(bufferInfo, 0);

                while (outputBufferIndex >= 0)
                {
                    outputBuffer = outputBuffers[outputBufferIndex];

                    outputBuffer.position(bufferInfo.offset);
                    outputBuffer.limit(bufferInfo.offset + bufferInfo.size);

                    outData = new byte[bufferInfo.size];
                    outputBuffer.get(outData);

                    //  Log.d("AudioDecoder", outData.length + " bytes decoded");

                   track.write(outData, 0, outData.length);

                    decoder.releaseOutputBuffer(outputBufferIndex, false);
                    outputBufferIndex = decoder.dequeueOutputBuffer(bufferInfo, 0);

                }


Any idea how I could test what is wrong?

By the way: Thanks for all your support!

[de_ninja]

I looked at the error messages again and realized the problem might not be the decoder. As stupid as I am I only looked for the last error message. These are all the messages I receive in the logcat:


D/ExtendedUtils: Try creating ExtendedExtractor
E/MM_OSAL: FileSource::FileSource
E/MM_OSAL: FileSource::FileSource m_bEveryThingOK 1
E/MM_OSAL: MM_File_Create File Name /data/mmosal_logmask.cfg , Mode 0
E/MM_OSAL: MM_File_Create failed .Efs Error No -1 , File Name /data/mmosal_logmask.cfg , Mode 0
E/MM_OSAL: Open or read fail on /data/mmosal_logmask.cfg. Possible permission denied issue. Looking for /data/misc/media/mmosal_logmask.cfg
E/MM_OSAL: MM_File_Create File Name /data/misc/media/mmosal_logmask.cfg , Mode 0
E/MM_OSAL: MM_File_Create failed .Efs Error No -1 , File Name /data/misc/media/mmosal_logmask.cfg , Mode 0
E/MMParserExtractor:  FileSourceWrapper::New returned NULL
D/ExtendedUtils: default extractor is NULL, return extended extractor
E/MM_OSAL: FileSource::FileSource
E/MM_OSAL: FileSource::FileSource m_bEveryThingOK 1
E/MM_OSAL: MM_File_Create File Name /data/mmosal_logmask.cfg , Mode 0
E/MM_OSAL: MM_File_Create failed .Efs Error No -1 , File Name /data/mmosal_logmask.cfg , Mode 0
E/MM_OSAL: Open or read fail on /data/mmosal_logmask.cfg. Possible permission denied issue. Looking for /data/misc/media/mmosal_logmask.cfg
E/MM_OSAL: MM_File_Create File Name /data/misc/media/mmosal_logmask.cfg , Mode 0
E/MM_OSAL: MM_File_Create failed .Efs Error No -1 , File Name /data/misc/media/mmosal_logmask.cfg , Mode 0
E/MM_OSAL: parseTransportStreamPacket isPESPacket failed 18954
E/MM_OSAL: parseTransportStreamPacket isPESPacket failed 18783
E/MM_OSAL: parseTransportStreamPacket isPESPacket failed 18954
E/MM_OSAL: parseTransportStreamPacket isPESPacket failed 18783
I/OMXClient: Using client-side OMX mux.
E/OMXMaster: A component of name 'OMX.qcom.audio.decoder.aac' already exists, ignoring this one.
D/MediaCodec: MediaCodec[kWhatConfigure]: video-output-protection: 00000000, audio-output-protection: 00000000
E/Format:: audio/mp4a-latm
W/SoftAAC2: aacDecoder_DecodeFrame decoderErr = 0x4004
W/SoftAAC2: AAC decoder returned error 0x4004, substituting silence
I/SoftAAC2: Reconfiguring decoder: 0->44100 Hz, 0->0 channels