Extended Reality with 5G

Friday 4/16/21 09:17am
Posted By Ana Schafer
  • Up0
  • Down0

Snapdragon and Qualcomm branded products are products of
Qualcomm Technologies, Inc. and/or its subsidiaries.

We’ve teamed up with the IMGA (International Mobile Gaming Awards) to bring you a whitepaper on the 5G Future of Mobile Games. In it, we discuss with game industry influencers how 5G is impacting the mobile gaming space and areas of opportunity for developers such as Cloud Gaming, Mobile Multi-player gaming (MMP), Mobile eSports, extended reality (XR), and the future of storytelling (pervasive gaming) with AI in games.

In this blog, we do a deeper dive into the Extended Reality (XR) space.

From the earliest cyberpunk books in the 80s to the current Sci-Fi movies and games, we have been enticed by the idea of being able to step inside those 3D virtual worlds. Little by little game developers have worked on these concepts to make our fantasies a reality. However, developers have been constricted by power and throughput limitations to bring immersive experiences to life that seamlessly integrate into our real world. The convergence of 5G and several new technologies may be about to change that.

So let’s look at how mobile technology innovations from Qualcomm Technologies, Inc. (QTI) along with advances in XR and 5G technology support and grow the ecosystem needed for you to create this Sci-Fi world of extended reality (XR) that gamers are craving.

Extended What?
Before we dive into the exciting tech coming your way, let’s review XR. XR is the umbrella term used to describe the ecosystem that comprises augmented reality (AR), virtual reality (VR), and mixed realities (MR).

There were many little steps, some successful and some not, that primed gamers for the immersive experiences of the future. Most gamers are familiar with Oculus VR games and heads-up displays (HUDs) even if they haven't played with them. And many had their first AR experience playing Pokémon Go on their smartphones. Wii games took them out of their chair and showed them that they could become the controller, and Google Glasses dipped them into the concept of smart glasses. These steps confirmed the appetite for more immersive gaming experiences.

XR viewers, a lightweight form factor equipped with various sensors, are providing gamers with a new level of development opportunities such as eye-tracking capabilities . The high-resolution display supplies photo-realistic graphics with six degrees of freedom (6DoF). Combining on-device processing with cloud processing provides advanced imagery along with real-time responses that will help gamers experience the ultimate goal of full immersion.

Bring It On (Power and Speed)
“…if you had a powerful internet connection, you could stream your camera content and get help from the cloud in order to create fantastic things where you see other people playing in the streets and their virtual experiences, too…”
- from Mr, Tommy Palm, co-founder of Resolution Games

It's hard to think of a technology that is poised to benefit more from 5G than XR. Immersive XR game experiences are data-heavy, bandwidth-intensive, and intolerant of lag. 5G delivers with increased bandwidth, lower latencies, and real-time simultaneous connectivity that your audience requires for a seamless, richer XR gaming experience.

I think that the use case for VR and AR will come from many other areas. First, before we see the first really mainstream large games make it onto those platforms. But, who knows how it will work out, whereas 5G has so many advantages to it from a gamer perspective in terms of just much faster latency anywhere. That dream of being able to just get together with friends anywhere.
- Kristian Segerstrale

At QTI, we’re offering new powerful Snapdragon® XR2 5G Platforms, and Snapdragon 5G Mobile Platforms with cutting-edge Qualcomm® Snapdragon Elite Gaming™ features that help you to build unrivaled XR experiences. Your gamers will be able to explore every angle of their virtual world in a 360° spherical view that captures the scene with vivid HDR rendering and display. These powerful SoCs are designed specifically to support your advanced game development ideas with XR streaming capabilities, multi-camera support, and 8K. We even offer a Qualcomm®3D Audio Plugin for Unity and Qualcomm®3D Audio Tools, so you can develop experiences with high-fidelity sound in all directions to give your XR gamers a competitive edge.

The Low down on Latency
“… We’re confident that 5G networks can accelerate our vision of bridging the physical and digital worlds with advanced AR functionality such as real-time synchronous multiplayer AR experiences in a shared environment…”
-from Mr. Omar Tellez, Niantic Labs’ VP of Growth

Latency is a critical concern for any interactive XR gaming experience. For venues trying to support multiplayer XR games that push real-time simultaneous connections to their limits, it can be a deal-breaker. mmWave technology, with its breakthroughs in spectrum use and advances in antenna technology and signal processing, is helping transform how wireless communications are utilized to increase coverage and connectivity with reduced interference.

Even in densely-populated venues, mmWave can provide multi-Gigabit throughput speeds. This delivers the extreme capacity, ultra-high throughput, and ultra-low latency needed to support your multiplayer games or make eSports even more nail-biting.

Latency can also contribute to motion-to-photon latency in XR experiences, especially in VR. Our SoCs are engineered to be optimized end-to-end for latency, which is especially helpful in reducing lag, which can cause cybersickness for some users of HMDs.

Send in the Clouds
Beyond the simple introduction of AR into games like that of Pokémon Go, creating a truly immersive experience in XR games is a challenge. It requires images and characters that look like they belong in their extended world and appear physically present in the gamers’ surroundings. Reactions to the lighting, casting shadows in the correct direction, and even a character’s awareness of weather conditions add to the illusion of “realness,” which increases the immersion.

“Industry initiatives such as Niantic’s Real World Platform, based on 5G, will boost location-based gaming in combination with AR, using the capacities of real-time rendering that 5G can bring.”
-from The 5G Future of Mobile Games Whitepaper

Cloud gaming over 5G is used to achieve some of these XR effects. For example, ray-tracing, the technology that creates these effects for games, had only been possible through the GPU power found on standalone PCs. Today, 5G’s streaming provides the ability for graphics to be rendered in the cloud, so you can deliver a rich, immersive experience.

Moreover, with graphics being rendered in the cloud, gamers can access game content on the screen of their choice, making gaming truly platform agnostic. Of course, with the GPU power of premium mobile platforms like our latest Snapdragon XR2 5G Platform and Snapdragon 865+ 5G Mobile Platform that brings PC and console-quality graphics to mobile, rendering is moving closer to the edge all the time.

5G, especially high bandwidth mmWave, will bring us more realistic 3D experiences in XR. We’ll soon experience more real-time rendering of ‘skins’ on buildings, people, and moving objects, all done in the cloud, on the edge, or via a mixture of both, and have unique experiences in sport and concert venues. We can only imagine the artistic possibilities gaming developers will create!

To hear more on 5G mmWave, watch our RCR Wireless webinar and visit our mmWave page.

To get started with your XR project, check out our XR Development Kits. If you are planning on using OpenXR, read OpenXR and an Introduction to the Khronos Group. The Adopters Program for OpenXR 1.0 has been launched and includes a comprehensive test suite to ensure implementations conform to the cross-platform API specification.

To learn more about XR Viewers, watch our webinar How to take advantage of XR over 5G: Understanding XR Viewers, and have a look at our Q&A with Sajith Balraj, Director of Product Management here at QTI.

The 5G Future of Gaming Whitepaper includes many more exciting details and ideas on how 5G is poised to drive the future of mobile gaming. And for additional information and context, we’ve put together the following series of blogs that provide a deeper dive into the Whitepaper’s topics:

Snapdragon, Snapdragon Elite Gaming, Qualcomm 3D Audio Plugin, and Qualcomm 3D Audio Tools are products of Qualcomm Technologies, Inc. and/or its subsidiaries.