One of the techniques used in the highest-quality animated films has come to mobile devices. Motorola has created a new form of entertainment, “Spotlight Stories,” which are interactive animated short films for their Moto X smartphones. These, interactive stories feature a Pixar technology known as “subdivision of surfaces” along with host of other technologies including sensors to produce a new creative canvas for storytelling. The first Spotlight Story is called “Windy Day” and features a mouse called “Pepé” in a forest, and his red hat. As the story unfolds, Moto X users get a 360-degree immersive experience, as graphics are rendered in real-time based on how you move the phone, giving each person the opportunity to explore the story differently.
Remember how impressed you were the first time you saw animation from Pixar? Subdivision of surface, or subdiv, is a key technology Pixar uses to give animators more granular control over model quality in their films. Pixar developed the technology in every film since “Gerry’s Game” and recently open-sourced it as OpenSubdiv. Motorola then brought OpenSubdiv to mobile, through OpenCL and the Qualcomm® Adreno™ GPU inside the Moto X.
A job for heterogeneous computing
Surface subdivision is a technique of subdividing polygons into smaller and smaller parts, producing high-quality and smooth surfaces for more natural models with reduced facets on curves. In most games and visual simulations you might do something similar with MIP mapping – creating several versions of the same element – say, a nose or a ball – showing more detail on zoom-in and less detail on zoom-out. Instead, subdivision surface recomputes the shape of that element for each frame at any level of detail. The result is much smoother, higher-resolution graphics without the performance overhead of constantly loading different versions from memory, but of course it requires a powerful GPU, like the Adreno 320 in the Moto X.
Motorola wanted to accomplish several things with this project:
- create a new canvas for high quality, interactive storytelling
- create silky smooth animations and detailed models using OpenSubdiv in real time
- surprise and delight users of the Moto X phone with Spotlight Stories, cleverly pushed to their phone
With these ambitious goals, the engineers at Motorola worked with Qualcomm Technologies engineers, on a heterogeneous approach: using OpenSubdiv implemented in OpenCL together with OpenGL to fully utilize the power of the Adreno 320 GPU.
A world’s-first on mobile
Here are some of the moving parts inside this heterogeneous computing project:
- Subdividing surfaces involves dividing the polygons that make up a surface into smaller and smaller polygons. The recursive nature of the OpenSubdiv algorithm is well suited to the parallel architecture of the GPU.
- OpenCL is a programming interface that allows you to use the GPU or other processors for general purpose computing tasks (like the subdivision), freeing the CPU for other work, or for enhanced performance and power efficiency.
- Activity on the GPU tends to be bursty, with opportunities to take advantage of idle time; for example, while the CPU is executing game play logic in a gaming app. OpenCL is a good way to balance the system by offloading tasks that the CPU might have handled to the otherwise idle GPU.
- Under OpenCL on the GPU, Motorola’s OpenSubDiv algorithm computes the triangles and vertices and stores them in GPU memory. Next, OpenGL ES, also running on the GPU, picks them up and renders the high-resolution shapes. The tight integration between these repetitive operations on the GPU reduces back-and-forth with the CPU and makes for smooth, fast, high-resolution graphics.
Motorola used the Adreno SDK to work with the OpenGL ES and OpenCL APIs, and the Adreno Profiler for OpenCL debugging. They demonstrated their first OpenSubdiv stories, “Windy Day,” at SIGGRAPH 2013, and released it to consumers last month.
Baback Elmieh, technical program lead on the Motorola Mobility side, is pleased with the project: “We believe this use case we have for OpenSubdiv is the world’s first to use OpenCL and OpenGL ES interoperatively on a mobile device. It’s never been used outside of movies.”
I am pleased to have been a part of this project, and congratulate Motorola on this ground breaking new technology, and the delightful result that everyone can enjoy.
The project is a good example of two converging trends: OpenSubdiv in animation and heterogeneous computing in mobile devices. Which one do you want to try in your app development?