Snapdragon and Qualcomm branded products are products of
Qualcomm Technologies, Inc. and/or its subsidiaries.
From health care to education and manufacturing, AI use cases grow exponentially. Developers choose popular AI Frameworks like PyTorch and TensorFlow to analyze predictions, train models, leverage data, and improve future outcomes. In this article, we explain how to improve the performance of framework trained models. Read on to learn more about a powerful platform complemented by high performance AI software development kits.
Introduction to Windows on Snapdragon
For the past few years, Qualcomm Technologies and Microsoft have been partnering to create a powerful platform called Windows on Snapdragon, designed to provide high performance and energy efficiency for running your AI workloads. The Snapdragon processor, developed by Qualcomm Technologies, is at the heart of this platform. The processor can run AI applications on Windows PC without developers having to worry about performance issues or battery life. Windows on Snapdragon handles your AI workloads with speed, efficiency and versatility.
Back in October, Qualcomm Technologies introduced Snapdragon X Elite processor, setting a new bar in performance. The processor can outperform x86 processors by up to 2X in both CPU and GPU performance, thanks to its 4nm process node and 136GB/s memory bandwidth. It can also run over 13 billion parameter large language models and generate 30 tokens per second for up to 7 billion tokens, making it a generative AI powerhouse. Whether natural language processing, computer vision, or any other AI domain, Snapdragon X Elite processor can be trusted to deliver fast and accurate results.
But performance is not the only thing that matters. You also want a platform that can enhance user experience, save battery life, and adapt to your needs. Windows on Snapdragon platform supports various user experience enhancements, such as lightning-fast 5G, Wi-Fi 7, advanced camera ISP, seamless security from chip-to-cloud, and more. It also consumes 68% less power than its competitors, while delivering peak PC performance. The platform also allows the CPU, GPU, and DSP to balance application workloads, providing a seamless experience for users. The best part? The platform can scale across different designs and form factors, enabling a variety of devices and applications to leverage its capabilities.
Accelerate AI workloads with high performing SDKs
To help our customers take advantage of on-device AI, Qualcomm Technologies has developed a suite of AI toolchains that simplifies and optimizes the deployment of AI models on our Windows on Snapdragon platforms. With these AI toolchains, our customers can run their AI workloads on-device, delivering immediate, reliable, and secure AI experiences for their users.
A big part of running AI models on-device is quantization. The Qualcomm AI Research team has developed an open-sourced library, AI Model Efficiency Toolkit (AIMET), that provides advanced model compression and quantization techniques to shrink models while maintaining task accuracy. Developers can incorporate the advanced model compression and quantization algorithms of AIMET into their PyTorch and TensorFlow model-building pipelines for automated post-training optimization, as well as for model fine-tuning.
When it comes to inference, the Qualcomm AI Stack provides high performance runtimes to help developers optimize and deploy AI models quickly by supporting AI frameworks, developing libraries, system software and popular operating systems. As part of the Qualcomm AI Stack, you get access to two SDKs, Qualcomm Neural Processing SDK and Qualcomm Direct AI Engine SDK, that provides full access and ultimate flexibility for best-in-class AI performance.
The Qualcomm Neural Processing SDK is an all-in-one SDK that supports heterogenous computing, system-level configurations and designed to direct AI workloads to all accelerator cores on Qualcomm Technologies’ platforms. It provides developers flexibility, including inter-core collaboration support and other advanced features.
On the other hand, the Qualcomm AI Engine Direct SDK provides lower-level, highly customizable unified APIs that speeds up AI models on all AI accelerator cores with individual libraries. It can be used directly to target a specific accelerator core or delegate workloads from popular runtimes including Qualcomm Neural Processing SDK, TensorFlow Lite and ONNX runtime.
Apart from these SDKs, Microsoft and Qualcomm Technologies have also collaborated to bring DirectML support to Qualcomm Technologies’ GPUs. DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning on Windows. It provides Qualcomm Technologies’ GPU acceleration for common machine learning frameworks and workloads, including image classification, object detection, and style transfer.
Customer story: Camo transforms their video experience
Teams around the world meet, stream and record with video thanks to Camo, an app from Reincubate that delivers professional picture quality and powerful effects using any device they choose. Over the past several months, the developers at Reincubate worked to port the app over to Windows on Snapdragon, resulting in an unparalleled video experience.
With Camo running on Snapdragon dedicated NPU, users enjoy longer battery life and unburdened CPUs, while AI-enabled image processing gives them instant, real-time access to segmentation effects like Portrait Mode and Virtual green screen, AR effects like emoji hands and responsive face tracking.
To try the app, download Camo Studio on your devices with Windows on Snapdragon.
Figure 1 Camo running on Snapdragon dedicated NPU“With Qualcomm Technologies, we’re using the cutting-edge chip architecture to put AI-powered video tools in the hands of creators, gamers, and professionals everywhere. Camo on Windows on Snapdragon does it all: it’s faster, it has lower latency, it’s more efficient and it just looks better.” – Aidan Fitzpatrick, CEO Reincubate
What’s Next?
Looking to accelerate your AI use cases on Windows on Snapdragon? We’re here to help you. Explore Qualcomm support options and get the resources you need to help you build your next innovation.
In our next blog, we will discuss how you can get started with the Qualcomm Neural Processing SDK. You will learn how to bring your AI models and run inference seamlessly on Windows on Snapdragon devices. Stay tuned!
Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc. AIMET is a product of Qualcomm Innovation Center, Inc. Qualcomm branded products are products of Qualcomm Technologies, Inc. and/or its subsidiaries.
