Snapdragon and Qualcomm branded products are products of
Qualcomm Technologies, Inc. and/or its subsidiaries.
Video processing. Computer vision.
You can handle tasks like these on the CPU, but with a powerful and efficient Hexagon™ 680 DSP in the chipset, why would you want to?
Developers have been using the Hexagon SDK over the last couple of years to customize the DSP for audio, imaging and computation. At the HotChips conference a couple of weeks ago, we offered an early peek at the new Hexagon DSP inside the upcoming Snapdragon 820 processor. Dubbed the Hexagon 680 DSP, it lets you take advantage of completely new DSP use cases with its Hexagon Vector eXtensions (HVX) core for wide-vector processing.
The addition of VX means that you can now offload video and camera tasks from the CPU to the Hexagon DSP for fast image processing with low power consumption. We’ve released the HVX Add-on to Hexagon SDK 2.0 so you can drop your own algorithms into the pipeline and process them on the DSP.
New Hexagon VX Core For Wide-Vector Processing
Camera, computer vision (CV) and video have long been compelling use cases for offload from the CPU because they consume a lot of power. With the HVX core, Hexagon’s designers have added wide-vector processing for higher performance and lower power consumption.
You can take advantage of Hexagon VX in three main paths:
- Video Processing – Enhance the quality of decoded video output with your own algorithms. The add-on supplements the SDK with a framework that allows developers to create end-to-end applications for processing decoded video frames. The add-on example we provide focuses on two elements:
- The Android-side Video Processing application, which you can use as a starting point for integration with the Android multimedia pipeline of other Android video applications.
- The Hexagon-side Video Processing algorithm, which you can use as a starting point to create your own algorithms executing on the Hexagon and Hexagon VX.
- Camera Post-Processing and CV – We’ve included several examples in the add-on, including an end-to-end .apk for taking live camera input, running CV processing on it and sending it out to the display. We’ve provided a live app around a sample algorithm you can replace with whichever algorithm you want to run in the camera preview path.
- Camera Streaming – The sensor and camera front end used to be directly connected, but Hexagon VX includes a hardware protocol interface among the sensor, the Hexagon DSP and the camera front end. That interface lets you use Hexagon VX to introduce DSP processing (for instance, to use a non-standard sensor or perform pixel correction) between the sensor and the camera front-end hardware. An example in the SDK shows you how to build the required Hexagon library and plug it into the Android camera pipeline.
In all three use cases, your algorithm doesn’t have to chug along in software anymore. Plus, you can integrate your DSP customization to your OEM’s build process more smoothly.
In short, the add-on is a formal way for developers to add their own post-processing algorithms, given that the rest of the pipeline is ready. It’s been possible to process video, CV and camera on Hexagon before, but Hexagon VX offers a big improvement in the size and resolution of images you can process and how fast the algorithms run (see charts below).
Next Steps With The Hexagon VX Add-On To Hexagon SDK 2.0
Ready to try your own image processing algorithms on Hexagon VX? Commercial devices running the Snapdragon 820 will be available soon; meanwhile, you can test your builds in the simulator that comes with the Hexagon SDK.
And keep an eye out for upcoming hardware releases: a new Mobile Development Platform (MDP) and a new DragonBoard Development Kit are expected to be available soon for purchase from Intrinsyc.
If you already develop on the Hexagon DSP, keep an eye out for news on obtaining the Hexagon VX Add-on to Hexagon SDK 2.0. If not, then apply for access to the Hexagon SDK. And get your image processing off of ARM and onto the DSP.
