Snapdragon and Qualcomm branded products are products of
Qualcomm Technologies, Inc. and/or its subsidiaries.
The 2021 ARM DevSummit was held from October 19 to 21, 2021. At the Summit, I took part in a presentation with Marcus Perryman, Principal Engineer – Windows Team at Microsoft, titled AI Accelerated Experiences & Security with Windows on Snapdragon. Our presentation provided insight into using AI in apps for always-on, always-connected, Windows PCs powered by our Snapdragon® compute platforms.
Our presentation acknowledged how society’s dependence on mobile devices continues to increase, particularly on PCs. Many aspects of our lives are experiencing this impact ranging from remote and hybrid working environments to staying connected with loved ones, anytime, anywhere. Fulfilling these requirements has fueled the demand for powerful yet energy-efficient platforms that can run apps like Teams for video chats and productivity packages like Microsoft 365 from virtually anywhere with all-day battery life.
AI is the key for Rich, Personal, and Connected User Experiences
Marcus and I discussed how the key to providing rich, personal, and connected user experiences in today’s remote climate is to augment apps with AI. Real-time AI calculations can be accelerated using on-device hardware engines when the utmost speed and performance are required with minimal power consumption. While cloud-based inference, accessible over high-speed links like 5G and Wi-Fi 6, can complement on-device AI with near-real-time performance.
For example, the Microsoft SQ1 processor, developed in collaboration with Qualcomm Technologies and used in the Microsoft Surface Pro X, runs on-device, hardware-accelerated inference for real-time gaze correction. This feature tracks a meeting participant’s eyes and digitally corrects them each frame to make the participant appear as if they’re looking at the camera, resulting in more personal and connected interaction, as shown in Figure 1:
Figure 1 – Example of real-time gaze correction using AI to make a meeting participant appear as if they’re looking at the camera.Marcus noted that the traditional power consumption using a CPU and GPU to perform this feat is around 15 W. In comparison, the SQ1’s AI acceleration can do it with around 0.3 W of power – 50 times more efficiently. Moreover, the performance of the neural processing unit (NPU) is up to 10x faster than traditional CPUs because it is custom built to compute the complex multi-dimensional mathematical formulas required by AI neural networks.
During our presentation, we also noted numerous other existing and potential uses for AI in Windows apps, such as:
- Backgrounds and video enhancement and replacement for video conferencing
- Natural language processing (NLP) tasks such as real-time voice translation in video conferences, predictive suggestions in Word documents, or voice commands for voice UIs
- Audio de-noising and cleanup to capture only the speaker’s voice
- Security including biometrics, and behavioral analysis for threat detection
- Computer vision tasks, including user and object detection, which can automatically lock content when a user walks away from their PC
- Photo app filtering (e.g., changing contrasts, converting photos to sketches, etc.)
Marcus talked about how Windows 11, and apps like Office 365, are aligned to support such experiences on high-performance devices like Snapdragon compute platforms. These devices are always-connected, up-to-date, and capable of maintaining long battery life.
Resources for Developers
Developers can start by visiting the Windows on Snapdragon page on QDN to access additional information. Here we have put together a list of additional tools for Windows development on QDN that you should check out.
To allow developers to convert their AI models for optimal performance on Windows PCs built around Snapdragon processors, developers can use the Qualcomm® Neural Processing SDK for artificial intelligence (AI). With this SDK, developers can import models built with TensorFlow, PyTorch, Caffe, and ONNX to take advantage of hardware-accelerated, on-device AI across a variety of verticals.
Microsoft provides Windows on ARM tools and Windows on ARM documentation which developers should use to get started. And as noted in the presentation, Visual Studio 2022, currently in pre-release, will support Windows 11 and be updated monthly.
Additional Blogs
- Windows on ARM development – How do I get started with the tools?
- On-device AI with Developer-Ready Software Stacks
- Neural Network Optimization with AIMET
Snapdragon and Qualcomm Neural Processing SDK are products of Qualcomm Technologies, Inc. and/or its subsidiaries.
