Snapdragon and Qualcomm branded products are products of
Qualcomm Technologies, Inc. and/or its subsidiaries.
At Qualcomm Developer Network, we love hearing from developers who are using our hardware platforms and software tools in their development. Not only does it give us an understanding of what tools are helpful but how they are being used. This insight helps our product teams build future products and tools that are designed to meet the needs of our developer community. Here is one such company and their experience in designing a new product with resources from Qualcomm Developer Network.
Philipp von Weitershausen, Silk Labs Engineer & Evangelist
Who We Are And What We Were Trying To Build
At Silk Labs we build software to power the next generation of intelligent devices for the home. We use computer vision, sound, voice recognition, and machine learning to offer consumers an almost magical experience. At the heart of Silk Lab’s vision is the belief that functionality should be simple, relevant, and not require a behavior change in how we interact with connected products. Imagine you don’t have to use your cell phone as a remote control for the appliances in your home. Instead, your home interacts with you and your device automatically via presence, gestures, and voice.
Bringing this vision to reality is the Silk device OS. Silk is a new platform that functions like an operating system for intelligent devices. It’s built from the ground up and uses near-AI technology and the ability to run local applications, making your connected home more intelligent and secure. The first product to utilize Silk is called Sense.
Sense is a product we decided to build around the Snapdragon® processor. With the excellent compute power of the Snapdragon processor and Adreno™ GPU we could perform our computer vision and machine learning tasks locally without having to stream sensitive data from the living room to the cloud.
Challenges In New Product Development
Developing a new device with a brand new chipset is always a challenge. We were still several months away from having the first prototype PCBs back from our manufacturer. But we were able to use the DragonBoard 410c development board for our development in the meantime which has been invaluable to us. The chipset on the board isn’t the same as in our final product, but it's a close enough match for our purposes. The board comes with the connectivity that we needed (WiFi, Bluetooth) and can drive a camera sensor, so we can test the camera that will be used in our final product.What We Were Able To Do
Our Silk platform that powers Sense uses the Android Linux kernel and user space. We run a Node.js-based stack on top of this foundation, which allows us to implement much of the functionality in JavaScript. Third-party developers can easily extend Sense and the Silk platform using small JavaScript programs, and they’ll be able to share their extensions with other developers and consumers. It’s an IoT operating system as smart as your smartphone, complete with apps!Having all the Android kernel and userspace open source available from codeaurora.org with matching binaries from the Qualcomm Developer Network makes debugging issues a breeze (see Snapdragon Debugger and Snapdragon Profiler). We were also particularly happy about OpenCL on the DragonBoard. We accelerate our computer vision pipeline with the Adreno GPU using OpenCL, and we can test and optimize for Adreno months before we get our hands on the final PCBs. This is a massive improvement from traditional hardware/software development, where key software development often has to be delayed until the hardware is ready.
Join In The Fun
If you are interested in Sense and want to experience it in your own home, please head over to our Kickstarter campaign. You will also find further information about Sense and the Silk IoT platform. Be sure to check out our demo video that shows how you can easily extend Sense and detect cats in the field of view with a few lines of JavaScript :)
