A new video codec API arrived with the Jelly Bean release of Android (API level 16) called MediaCodec. With this new feature, you can access hardware-level encode / decode functionality from within the Java application space. Additionally, because you are given access to the input and output buffers of the hardware components, the MediaCodec API provides greater flexibility with regard to what input sources can be used given that the image data can come from any source you choose (individual image frames or an actual video stream similar to MediaRecorder). Also, in the case of using camera preview frames, it is possible to manipulate the images prior to handing the data off to MediaCodec for encoding. In contrast, the MediaRecoder API puts the camera into video capture mode and the image data is fed directly to the encoder and no post-processing can occur. While this enhancement is a welcome addition, there are important features that MediaCodec does not currently support; some of which are accessible via the Khronos OpenMAX framework directly in the native C++ layer. In this post, I’ll discuss some of these differences with the intent of enabling you to decide which route to take when implementing your application.
Static Configuration Features
Configuration of the hardware encoder components can be broken down into two specific categories, parameters that are set statically when the component is initialized, and those that can be manipulated dynamically after the component is already in use. In the case of static configuration, neither MediaCodec nor direct OpenMAX calls have any functionality advantage as both frameworks support the following features:
- Codec and profile / level selection
- Frame Rate
- Bit Rate
In contrast, neither MediaCodec nor Qualcomm Innovation Center’s (QuIC) implementation of the OpenMAX Integration Layer (IL) client currently support the following static configuration features:
- Rate Control Mode (Constant Bit Rate, Variable Frame Rate)
- Slice Size
- Group of Picture (GOP) Size
- Long Term Reference Picture (LTRP) Signals
- Hierarchical P Coding
Dynamic Configuration Features
While both frameworks mirror each other when it comes to static configuration, only the OpenMAX IL client currently supports dynamic IDR frame insertion during encoding. Depending on your specific use-case, this may or may not be important. In video telephony applications, for example, periodically inserting IDR frames into the data stream helps to reestablish a baseline for the decoder in the event that previous frames were lost in transit.
Finally, the following is a list of dynamic configuration features not currently supported by either MediaCodec or the QuIC OpenMAX IL client:
- Bit Rate
- Frame Rate
- LTRP Signal
- Hierarchical P Bit Rate Per Layer Reconfiguration
On the decode side of things, neither MediaCodec nor the OpenMAX IL client currently has any functionality advantage of the other. Both support port reconfiguration events to respond to asynchronous sequence parameter sets (SPS) and picture parameter sets (PPS) embedded into the video stream. Also, neither framework currently supports selection for error concealment mode while decoding.
If you’re looking to build an application to take advantage of the low-level multimedia components on devices featuring Qualcomm® Snapdragon™ 800 processors, including the Snapdragon 800 Mobile Development Platforms, using MediaCodec and the OpenMAX IL client are both good options. If you need the ability to manually trigger an IDR frame, only the latter will presently meet your needs.
There is a video codec sample code for Android available that can help get you up and running. And if you have questions, you can visit the Multimedia Optimization Forum and we’ll be happy to help you.