"We're always trying to improve the performance of iOnRoad; up until now, we had never seen such a jump in performance as with FastCV. It usually takes weeks of work to get the kind of performance improvement we saw within a couple of days of integrating FastCV."
-Ram Dayan, Head of Software, iOnRoad
iOnRoad Achieves Faster Image Conversion Using FastCV™ Computer Vision
iOnRoad application was honored with a CES Award for Design and Engineering in the Software & Mobile Apps category from the Consumer Electronics Association (CEA). iOnRoad utilizes computer vision (CV), technology that uses video input and high-speed computation to identify shapes in a field of view. iOnRoad uses CV with the camera in a mobile phone to detect neighboring objects.
- FastCV for Snapdragon
- 10-15% overall performance increase
- 30% faster conversion of YUV420 images to RGB
- Simple, quick integration of computer vision to existing code
- Market opening to middle-tier devices with sub-1GHz processors
- Performance improvement in a few days of engineering effort instead of weeks
Taking computer vision mobile
Dayan explains, "From our experience on previous CV projects, we knew the amount of work this application would demand of the device: capturing the preview image, converting it into a more convenient format, processing it, then performing all of our calculations on the resulting data. We figured that, with the evolution of application processors, mobile devices were mature and powerful enough to handle it."
Bottleneck: YUV420-to-RGB conversion
The preview and processing in CV can take 30 to 80 milliseconds for every frame received from the camera in an Android device. The overhead of that pipeline plus the data transfer to the native code of the app can add up to a stiff burden on the CPU.
Image conversion like the YUV420-to-RGB step in iOnRoad is especially compute-intensive, but it's often necessary for handling preview images in Android. Even when optimized, a software implementation of the conversion takes 10-25 milliseconds at best, or 30 percent of the total computation time.
One option is to modify the code to offload the work to the ARM NEON vector processing engine on the chipset, but now there is a simpler approach to running CV on hardware.
Dayan learned about the FastCV SDK, a library containing hardware-accelerated versions of the most frequently used vision processing functions for mobile devices. He decided to use the fcvColorYUV420toRGB8888u8()function in FastCV instead of the conversion function iOnRoad had been running in software.
"We integrated FastCV very easily and quickly," says Dayan. "I downloaded the FastCV SDK and went through the Getting Started Guide on the website. I set up the developer environment in Eclipse, installed the SDK, then compiled and ran the app. "
"All I had to change was one line of code to call the hardware-accelerated function, then recompile, and I was ready to test a FastCV-integrated version of iOnRoad on my HTC Sensation 4G, which has a Snapdragon S3 processor. We went from about 22ms with our own YUV420-to-RGB function down to just 16ms for the FastCV function. That's 30 percent less time for the image conversion, and an overall performance improvement of 10 to 15 percent for the entire algorithm."
To take advantage of the accelerated conversion process, iOnRoad's engineers changed processes that worked on BGR channel ordering to take RGB channel ordering. They also tweaked the code to provide for snapshots and direct access to pixels, but Dayan considers these small accommodations given the performance boost and ease of implementing FastCV.
New metrics and markets for computer vision
Because CV apps are so compute-intensive, iOnRoad keeps an eye on two metrics:
- Performance – iOnRoad reports a 10-15% overall improvement using FastCV
- Battery life – A FastCV-enabled app can run more frames per second yet do it more efficiently than with raw CPU cycles. The resulting app can identify shapes more dependably than the software-only version, and it may also consume less power.
Because of the processing overhead involved, many CV app developers target devices with at least a 1GHz processor, but FastCV opens up the possibility of reaching markets and customers on mid-tier devices with sub-gigahertz processors.
"We proved that FastCV is very simple and powerful," concludes Dayan, "and we plan to keep looking for additional parts of our algorithm to optimize. I expect iOnRoad's performance to increase with more hardware acceleration, which will allow us to enrich the app with new CV-related features."
See For Yourself
- Learn more about FastCV
- Read Computer Vision Comes to Mobile Apps with FastCV from the QDevNet Newsletter
- Review the FastCV Getting Started Guide
- Browse the FastCV API documentation
iOnRoad is developed, owned and distributed by iOnRoad Technologies Ltd.