Snapdragon and Qualcomm branded products are products of
Qualcomm Technologies, Inc. and/or its subsidiaries.
Motion-based security systems are known for their high rate of false positives from events such as passing animals and the wind rustling nearby trees. It’s a use case ripe for AI, which can use computer vision and inference to weed out both false positives and false negatives. The result is more-accurate detection that increases profits by reducing the expenses associated with making in-person security checks.
With the proliferation of high-resolution cameras and IoT devices, and increasing data volumes, CVEDIA believes the notable shift towards using AI on edge devices offers clear advantages in the evolving landscape of smart technologies.
By processing and storing data closer to its source, edge deployments enhance efficiency, reduce latency, minimize bandwidth requirements, improve privacy and enable shorter response times. Those advantages are attractive to retailers and security operators. Together with Qualcomm Technologies edge-related products and solutions, CVEDIA is meeting those challenges by providing AI-based video analytics specifically for edge devices.
Surveillance moves to edge devices
Most video management systems (VMS) store feeds as data in the cloud, and some also perform analytics on the feeds and notify retailers and security personnel of potential incidents taking place at the monitored location. These types of VMSes that are dependent on the cloud come with bandwidth limitations, network latency problems and steadily increasing costs. That’s why edge devices, designed to keep storage and compute close to the data source, are growing as a viable alternative.
With years of expertise in Edge AI deployments, CVEDIA had successfully integrated its computer vision AI into VMSes and applied their analytics to detect incidents. They discovered an obstacle: retailers and security operators had to invest in additional engineering work, first to connect their cameras to the VMS, then to enable computer vision on multiple feeds.
To address this challenge, CVEDIA began to work on a solution that:
- Improved accuracy of threat detection
- Decreased false alerts
- Alleviated the onboarding challenge to business owners
- Optimized computing power, storage and memory
The AI: SecuRT and Network Optix
In approaching this challenge, CVEDIA looked to its trusted partners. CVEDIA’s expertise in the VMS world included its work with Network Optix, a leading cross-platform enterprise video platform that offers intelligent video services as a full-stack video platform and ecosystem.
Most Network Optix customers use cameras to record events continuously through a subscription, amassing huge amounts of video. Human review of all that video is expensive, so most VMSes use motion detection to look for trespassers or people entering a surveillance zone. But, as noted above, the false alarms associated with motion detection raise manual security expenses and eat into profits.
SecuRT is CVEDIA’s AI-powered video analytics solution that yields more-reliable detections in IP camera feeds by greatly reducing the number of false alarms. SecuRT sits inside the camera settings within Nx Witness, enabling existing and future Network Optix customers to run SecuRT without having to waste time with additional configurations. As the VMS manages the video input, CVEDIA takes the multiple video feeds, pulls them in, processes them, extracts all the objects from the frame, categorizes and classifies all of the objects using AI and then places them back simultaneously. Because CVEDIA classifies the objects, their software filters the objects using categories such as classes, attributes, dwell time and direction to make their customers' video feeds searchable, actionable and quantifiable.
Within the shaded bounding box imposed on the video feed (image below), CVEDIA’s algorithms perform AI analytics.
Users can configure categories of detection such as intrusions, loitering, crowding, line crossing and people entering/exiting the zone. As shown below, from simple menus and dialog boxes they can specify the types of notifications they want to receive upon detection. They can easily opt to detect people, vehicles and animals within the zone of interest.
The combination of the VMS and the computer vision AI allows customers to monitor, analyze and react to incidents in real time:
The edge device: Thundercomm EB5
The next step was to identify suitable hardware for running AI algorithms on camera video at the network edge. Through their relationship with Qualcomm Technologies, CVEDIA turned to Thundercomm, a provider of IoT products and solutions built around edge AI stations.
Thundercomm’s EB5 Edge AI Station is based on the Qualcomm Robotics RB5 Platform, equipped with the Qualcomm AI Engine and Qualcomm Hexagon Tensor Accelerator (HTA). The EB5 delivers up to 11 trillion operations per second (teraOPS) for AI and deep learning workloads at the edge. It can handle up to 24 streams at full-HD video processing.
CVEDIA chose the EB5 as a good hardware match for Nx Witness VMS. In addition to all-in-one analytics of security camera feeds, the EB5 allows device management, video management, AI algorithm configuration, application upgrades, customization and multi-cloud connectivity. The software-hardware combination is ideal in settings like smart retail, smart buildings and smart transportation.
The acceleration: The AI processor and the Neural Processing Engine SDK
Working together, CVEDIA and Thundercomm ran the VMS on the EB5. Their goal was to market a deeply integrated product so that business owners could make high-level configuration changes without needing to pay for extensive engineering.
Once the software was working on the EB5, they turned their attention to processing data streams more quickly than the 2 to 3 frames per second (fps) they were achieving. That meant taking advantage of the Hexagon™ DSP with Hexagon Vector eXtensions (HVX) and Hexagon Tensor Accelerator, the acceleration engines on the Qualcomm Technologies chipset inside the EB5. Offloading the work of AI inference from the CPU and GPU to dedicated AI processing cores in the hardware would speed up the analysis of video frames.
Qualcomm Technologies introduced CVEDIA’s engineers to the Qualcomm® Neural Processing Engine SDK. The Qualcomm Neural Processing Engine SDK provides tools for model conversion and APIs for executing on the core with the optimal power and performance profile for the workload. CVEDIA used the tools in the Qualcomm Neural Processing Engine SDK to compress the layers in their existing Yolo object detection models and quantize/convert them from FP32 to INT8 format. They then ran the converted models on the Qualcomm Neural Processing Engine SDK runtime inside the EB5.
CVEDIA was able to perform the conversion with little guidance from Qualcomm Technologies. They found the Qualcomm Neural Processing Engine SDK reference guide impressively thorough and the workflow straightforward.
The results: Performance boost and lower power consumption
The transition from execution on the CPU and GPU to the Hexagon DSP made big differences in the overall solution.
- Higher performance – Running their algorithms on DSP, CVEDIA saw processing jump from 2-3 fps to 35-60 fps.
- Power-efficiency – They reported a 75% increase in efficiency of power consumption, with the DSP running on a maximum of 1 watt. “Having tested dozens of chip manufacturers and accelerators,” says Natalia Simanovsky, head of partnerships at CVEDIA, “we believe Qualcomm Technologies makes the most power-efficient system-on-chip in the market today.”
- Time to market – CVEDIA’s video analytics solutions are powered by synthetic data, which had already cut time to market by 90%. Since deploying on the EB5 with the Qualcomm Neural Processing Engine SDK, CVEDIA has seen time to market shrink by a further 25%.
From the business point of view, Simanovsky points to three noteworthy results:
- The solution makes it easier for business owners to deploy smart security. They can purchase the hardware from Thundercomm and license the AI software from CVEDIA, then convert their existing monitoring system with, say, 10 cameras into a smart system.
- CVEDIA-RT includes a collection of the most useful security and intelligent traffic video analytics, such as vehicle classification, people counting, zone crossing/tripwire, intrusion and detecting vulnerable road users.
- The CVEDIA-Thundercomm-Qualcomm Technologies-Network Optix solution will run on a full range of edge AI products powered by compatible Qualcomm Technologies’ chipsets.
Takeaways and next steps
The shift toward edge deployment of VMS offers several advantages to retailers and security operators. This turnkey solution is an example of combining AI-based video analytics for smart surveillance on edge devices. It is also a solution accessible to any embedded systems developer focused on IoT and on deploying AI at the edge.
The solution follows a straightforward recipe that any embedded developer can readily follow:
- Purchase the Qualcomm Robotics RB5 development kit from Thundercomm.
- Obtain and install the Qualcomm Neural Processing Engine SDK.
- Study the documentation for the SDK.
- Learn how to use the Qualcomm Neural Processing Engine SDK to convert trained models and deploy on the Robotics RB5.
- Adapt the sample applications in the documentation for specific use cases.
- Once the application runs as desired, deploy it to the optimal EBx/RB5-based AI box.
Qualcomm branded products are products of Qualcomm Technologies, Inc. and/or its subsidiaries.
