With the recent launch of the Snapdragon™ Neural Processing Engine (NPE) SDK, we previously highlighted the importance of on-device processing for time-critical AI applications. For example, the ability for self-driving cars make instant decisions - identifying and avoiding an obstacle in the road as near to the time it was detected, rather than following a trip across the network to a server in the cloud.
Image: AI powered car detecting an object ahead.
AI processing at the edge is inherently complex, but the Qualcomm® Snapdragon mobile platform has the power and flexibility to help developers with an advanced, heterogeneous computational architecture, supporting optimized tools, libraries and AI frameworks. With the Snapdragon Neural Processing Engine (NPE) SDK, developers can further tune the performance of AI applications relying on Caffe/Caffe2 and TensorFlow deep learning frameworks, with less effort.
Now we’d like to take this topic one step further by exploring some example AI applications where on-device computation can provide differentiation and critical advantage.
Evolving from Object Recognition to Object Tracking and Predictive Modelling
Modern, cloud-based photo sharing and storage services offer object and facial recognition as a standard feature, allowing you to quickly filter huge image libraries. It’s a remarkable feat requiring significant computing power. Life would be simpler if every photo comprised a single object of interest, but that’s rarely the case. Not only must multiple objects in a photo be identified and labelled, judgement is required to select the most important elements in the image for classification.
Object tracking and the predictive modelling required to support real-time, intelligent decision-making from a range of sensor inputs offers an even greater challenge. Imagine the possibilities:
- Security cameras that can instantly tell the difference between family members and strangers. A system that can rapidly identify and alert homeowners to a pattern of suspicious behavior, such as the same person trying door handles at the front and back of the house or an attempt to force open a window or a door.
- An application for the vision impaired that can detect obstacles in their path, sudden level changes such as steps or curbs or automatically detect and trigger traffic lights for road crossings.
- Object detection and avoidance for self-driving cars or drones.
- Dash cams that can recognize and warn drivers about potential risks from other vehicles, such as aggressive driving, tired drivers losing concentration or drivers under the influence.
Devices delivering these types of application will require the processing power to detect, understand and react to a host of environments and events with critical pace and precision.
Natural Language Processing, Speech and Speaker Recognition
Advances in speech recognition allow us to control car navigation, in-home entertainment and use voice-activated personal assistants for simple tasks such as information queries, calendar management and device control. Speech recognition accurately identifies words being spoken, but on its own it lacks important context. Recognizing the speaker can significantly improve the utility of personal assistants, particularly in secure applications, while natural language understanding goes beyond speech recognition to support deeper comprehension of what is being communicated.
Enhanced applications could include:
- Voice-activated interfaces that can operate in emergencies, where power and Internet access have been lost.
- Real-time translation for travellers or business meetings, supporting identification of complex linguistic context such as sarcasm or humor.
- Voice-controlled locks that can identify and grant access only to family members.
- Voice-controlled passwords for system access or in-store/online payments.
These examples demonstrate compelling use cases where on-device AI is advantageous compared to traditional, cloud-based processing. The Snapdragon Neural Processing Engine (NPE) allows developers to harness deep learning on-device using models trained with TensorFlow or Caffe2, achieving superior performance through the Snapdragon mobile platform heterogeneous computing architecture.
Together, these technologies greatly enhance delivery of today’s AI-driven applications, such as real-time image processing and language translation. They also open the door to innovation, giving developers the time, tools and processing power to realize tomorrow’s intelligent user experiences.
According to IDC, less than 1% of software developers incorporate cognitive capabilities into their applications today. That figure is expected to exceed 50% by 2018. Advanced mobile applications and automated system features, such as voice and speech recognition, object recognition and tracking - that typically require datacenter processing - could be fully supported on-device, with low-latency sensing and inference.
To take your development to the edge, be sure to find out more about our collaborations with Google and Facebook, download the Snapdragon NPE SDK and stay tuned for more discussion on AI development here at QDN.