Snapdragon and Qualcomm branded products are products of
Qualcomm Technologies, Inc. and/or its subsidiaries.
When it comes to real-time edge computing augmented by cloud backends, there is perhaps no better mission-critical use cases than those found in medical technology or MedTech.
Imagine surgically repairing a muscle using a robotic-assisted surgical instrument. Your view consists of high-frame-rate HD video streams of the muscle, overlaid with real-time insights and advisories inferenced from computer vision AI models. These are executed in real-time on a powerful, small-form-factor edge device running a MedTech platform that intelligently controls the instrument, augmented by cloud compute. Such capabilities are now possible, and they are powered by the same energy-efficient compute technology found in your smartphone, thanks to forward-thinking companies like Innominds, headquartered in San Jose, CA.
With their mission to engineer the digital future, Innominds develops platforms and works with clients to build devices for MedTech, smart cities, and several other verticals. Innominds has now developed iMedVision (iDhi), a vision-based AI-on-the-edge platform powered by our Qualcomm QCS8250 System-on-Chip (SoC), for building next-generation, chip-to-cloud MedTech devices and applications.
Here's a great video that provides an overview of the iMedVision Development Kit
We recently caught up with Raj Ganti, President of Strategic Accounts at Innominds, to learn more about iDhi.
A Small Platform for Big Innovations
iDhi consists of a carrier board (see Figure 1) with a rich set of I/O connections and a system on module (SOM) built around the QCS8250. The SOM plugs into the carrier board via a customized Smart Mobility Architecture Connector (SMARC) interface.
Figure 1 – Images of the iDhi carrier board and QCS8250-powered SOM.
iDhi brings together computer vision, cloud connectivity, 4G/5G, WiFi6, Bluetooth 5.1, and multiple camera streams, all powered by heterogeneous low-power compute so that developers can build small-form-factor medical devices with it. The SOM itself measures a mere 50mm by 100mm.
Raj says iDhi is targeted at the following MedTech domains:
- Diagnostic equipment: Equipment to help physicians evaluate specific medical conditions based on visual images.
- Surgical assistance equipment: Assists surgeons in operating environments to perform specific surgical procedures.
- Surgical-first equipment: Performs procedures with minimal supervision.
The Android-based platform has a rich set of hardware and software specifications. It supports multiple simultaneous camera streams, video encoding at up to 4K 120 FPS in 10-bit HDR, output to multiple screens, and a rich set of I/O connections (e.g., GPIO, USB, MIPI, PCIe, GbE, and UARTs). The QCS8250 provides the heterogenous compute power (multi-core ARM-Kryo, DSP, GPU, and neural processing) necessary to perform real-time, AI-based computer vision inferencing on these streams. The result is glass-to-glass latencies as low as 150ms, giving surgeons real-time insights as they work. It’s further augmented by cloud connectivity over 5G and Wi-Fi, opening up a wide range of possibilities, including remote operations.
On the software side, the platform supports OpenCL, OpenGL, and Vulkan and has integrations for AWS IoT and Azure IoT. It also supports cloud-based ML services, including AWS Sagemaker, Azure ML, and iFusion. Collectively, these provide developers with rich heterogeneous, rendering, and cloud compute functionality.
The platform can also make use of several SDKs from Qualcomm Technologies, Inc. The Qualcomm Hexagon DSP SDK provides heterogeneous processing for audio, imaging, and embedded vision. The Qualcomm Neural Processing SDK for AI optimizes deep learning models for execution on the edge. And the Qualcomm Computer Vision (FastCV) SDK drives computer vision algorithms like object detection, tracking, and segmentation.
Inspiration
Figure 2 – iDhi casing.
Raj says the company is inspired by the application of mobile, low-power compute in healthcare. This comes as AI and ML have entered surgical rooms, and vision-based ML inference contributes to the success of critical surgical procedures. Up until recently, most MedTech hardware was built around larger form factors that consumed high power, generated more heat, and was not developed with portability in mind. Thankfully low-power, highly-integrated mobile compute platforms now facilitate smaller form factors, thereby ensuring portability and allowing surgeons and healthcare professionals the much-needed flexibility.
MedTech Use Cases
Raj says the platform, which is fully integrated with their medical-grade, HIPAA/GDPR-compliant cloud backend, helps medical original equipment manufacturers (OEMs) and solution providers build systems to:
- Integrate with existing medical cameras such as laparoscopes and endoscopes to acquire high bandwidth video feeds.
- Perform multimedia operations such as transcoding and preparing the video stream for ingestion into ML models.
- Perform real-time AI-modeled inferences and display the results (e.g., image segmentation of tissues) overlaid on interactive displays to guide surgeons.
- Perform remote device management via management platforms powered by hyper-scale cloud services (e.g., AWS and Azure).
Developer Tips and Lessons Learned
In reflecting on the ten months it took to develop iDhi, Raj shared several key learnings.
For starters, Raj says that the current supply chain environment meant longer lead times to acquire the EM526 high-speed material needed for PCB fabrication, so designers should plan accordingly.
On the performance side, Raj recommends calculating the glass-to-glass delay and latency requirements early on. Then, optimize the camera path and the required compute at the beginning of the project. Achieving high frame rates in the USB camera path was a challenge, and best handled in the design stage in coordination with Qualcomm Technologies.
Due to the complexity of their image processing algorithms, the Innominds team needed to optimize ML techniques for best performance across the QCS8250 multiple heterogeneous compute blocks. To accomplish this, they used our SDKs, in conjunction with OpenGL, OpenCL, as well as tools for the SoC’s Qualcomm Adreno GPU, Hexagon DSPs, and the Qualcomm Neural Processing Unit. This allowed them to build image and video processing algorithms and maximize frame rates and overall performance.
One key challenge was achieving up to 120 FPS video output. The team utilized the QCS8250 heterogeneous design for concurrent CPU, GPU, and DSP processing to achieve this. A key focus of this effort was memory management and buffer sharing.
Raj says the QCS8250 is their preferred platform. In particular, the Qualcomm Neural Processing SDK for AI allowed them to optimize their deep learning pipelines for multimedia. They also praise the computer vision support for both OpenGL and OpenCL. He also said the SoC’s Wi-Fi features are robust compared to solutions from competitors, and its massive processing power is a significant benefit.
Looking Forward
Raj anticipates continued growth in telemedicine, augmentation and upskilling of healthcare professionals, personalized treatment, and wearables. These will come by using AI on the edge, cloud computing, extended reality (XR), and IoT to develop and deliver new treatments and services.
He also foresees technology impacting hospital automation. The use of robots to automate hospital auxiliary and back-office activities can save money and time and increase dependability. Nurses and other medical personnel can call robots for certain duties by just tapping a screen. For example, the robots can distribute drugs, transfer blood samples, gather diagnostic data, and schedule linen and food supplies, either as a planned task or in response to a real-time request. In addition, certain hospital revenue cycle, accounting, and finance tasks (e.g., scheduling and claims processing) may benefit from robotic procedures.
With the potential to power so many use cases, the company has plans to sell iDhi as a stand-alone development kit for building MedTech solutions. Raj also says their technology could also be applied to other use cases requiring the fusion of edge compute with multiple cameras. This includes automotive safety systems, warehouse robots, behavioral analysis and authentication, worker safety in industrial and construction environments, and smart cities.
Regardless of what Innominds’ customers build with iDhi, we’re proud to be playing a part in powering such innovative technologies.
Qualcomm QCS8250, Qualcomm Hexagon, Qualcomm Neural Processing SDK, Qualcomm Neural Processing Unit, Qualcomm Adreno, and Qualcomm Computer Vision (FastCV) SDK are products of Qualcomm Technologies, Inc. and/or its subsidiaries.
