Screenovate

The smartphone keeps getting smarter, but the screen will always be small because it needs to fit in your pocket. What if you could take that smartphone out of your pocket and beam its display to a large screen for easy viewing and sharing?

That's the technology developed by Screenovate, an Israeli startup. Screenovate "borrows" large screens – televisions, laptops, desktops, tablets and multimedia displays in automobiles, airplanes and exercise equipment – by beaming the display from Android smartphones to them over Wi-Fi. When the display is on a much larger screen, the phone realizes more of its potential as a computing device. Users can play interactive mobile games on big-screen TVs and run mapping applications on in-dash multimedia centers in cars.

Screenovate captures the display from the phone's operating system (Android™ 1.5 to 2.3) and creates a frame of the bits going to the phone's screen. The software encodes the frame using a proprietary codec and sends it via Wi-Fi to the external screen. Watch Screenovate's video on connecting smartphones to big screens.

That's a lot to do in software, so the company looked for a mobile processor that could perform the work in hardware instead. Screenovate was attracted to the Snapdragon platform for two important reasons – hardware acceleration capabilities, and industry leadership in the Android ecosystem – so they contacted Qualcomm.

Hardware acceleration in three different places

Qualcomm works with application developers all the time. However, the Screenovate story is different because it's more than an application for a user to download and run; it's a technology that a device manufacturer must integrate and ship on the smartphone.

"They could have just pointed us to an SDK, or told us to buy the Snapdragon MDP and left us to it," says Gilad Ben-Yossef, CTO and co-founder of Screenovate, "but they have gone to great lengths to help us."

Screenovate posed an interesting use case for the Snapdragon chipset and its multiple engines. Because the application's tasks are so varied and compute-intensive, Qualcomm's experts first helped Screenovate carve up all the work and point each task at the right engine.

"This was a great leap for us," continues Ben-Yossef. "The implementation we came up with together was to use the Linux frame buffer to get the display, the media processing engine for color space conversion and the JPEG encoder to speed up compression of the images we send. Qualcomm supplied information on how to use each engine and walked us through the APIs we needed to invoke."

The result? An optimized user experience, especially in the areas that Screenovate wanted to focus on: real-time beaming with the lower latency, higher frame-rate, lower CPU usage and lower power consumption.

The part you can't do on your own

"Then came the part that you can't do on your own," continues Ben-Yossef. "Qualcomm invited us to work alongside their engineers on the software that runs on the Snapdragon platform, and together we spun a special build that integrated our hardware-accelerated optimizations on the Snapdragon MDP. This was something that both of us could show to our prospects and customers, and tell them: 'If you choose Screenovate and Snapdragon, this is how good the performance can be.'"

Screenovate found that it had doubled the frame-rate (to 30 frames per second) and halved the latency (to 60 milliseconds) compared to the version that performs all the encoding in software running on the CPU. The demo platform included the targeted hardware acceleration model, the special integration build on the Snapdragon MDP and Android 2.2.

With a sample device in hand, Screenovate introduced its hardware-accelerated version in Qualcomm's booth at CTIA. The two companies share customers among device manufacturers, so Screenovate successfully demonstrated that the software they sell to OEMs runs well on the hardware that Qualcomm sells to those same OEMs.

Tools and a special codec

Besides the Snapdragon MDP and the side-by-side integration help, Screenovate also used Qualcomm's Trepn Profiler to measure the changes in CPU load and power draw as they moved the application onto dedicated hardware.

"It took about two months to get from first contact to an agreement to work together," Ben-Yossef recalls, "then another four months to arrive at a working prototype of the application on the Snapdragon MDP. We got an immense amount of work done in those six months. We had approached several other chipmakers previously, but we were pleased that Qualcomm was forthcoming in telling us what they could do to help us."

Screenovate uses its own codec for advantages like lower latency than MPEG-4. The codec and communication protocol also support features not yet available in standard technologies:

• a back channel for connecting additional input devices – mouse, keyboard, etc. – to the external screen
• ability to stream phone UI to a browser through an HTML5 client
• "beam on stream," a picture-in-picture mode in which a movie streaming from the Web or an SD card occupies most of the screen, with an Android app in a small window inside it.

Hardware acceleration makes these features practical and keeps them from bogging down the rest of the system.

The Qualcomm boost

Screenovate plans to extend its success to date by integrating with the recently launched dual-core version of the Snapdragon MDP running Android 2.3, for even higher frame-rates.

"Our work with Qualcomm has been gratifying," concludes Ben-Yossef. "There's no way for an outside developer to integrate as part of the build without inside help, but once we had approached Qualcomm and told them what we wanted to do, they said, 'Let's find a way to do it.' And together we did.

"At CTIA, we were able to show our prospects that we had an agreement for cooperation with Qualcomm, and they could see that we had a presence at their booth. That has been a big boost to our credibility and given OEMs the confidence to work with us."

See For Yourself:

• Watch Screenovate's video on connecting smartphones to big screens and other use-case videos.