A lot of developers like to learn by doing - diving into new technologies and using cutting-edge tools that help bring inspirations to life. The ForwardXP team shares this same mindset, and turned to the mobile innovation and extended realities (XR) expertise from Qualcomm Technologies to develop new, immersive experiences in the Think F.A.S.T. VR medical training demo.

Think F.A.S.T. is a powerfully engaging VR simulation that puts viewers in the middle of a medical emergency. In this state-of-the-art virtual training center, medical students use VR to educate themselves on how to recognize symptoms of a stroke using the F.A.S.T. (Face, Arms, Speech, Time) system while interacting in real-time with a virtual patient.

Qualcomm Technologies selected ForwardXP to bring their expertise in video game development to healthcare VR by showcasing breakthrough realities in medical training in the Think F.A.S.T demo. The hardware and software are the yin and yang that makes XR truly immersive which is why seamless integration of both was essential in the training simulation.

Image: Radiation oncologist using the Snapdragon 835 VR development kit to interact with the Think FAST training simulation.

Hardware

A critical aspect of VR is immersion, which comes from three key areas: visual, auditory and interactive. To achieve this vital goal, ForwardXP used the Snapdragon® 835 VR development kit; a standalone all-in-one mobile XR headset with a single panel AMOLED head mounted display (HMD) and an integrated Snapdragon 835 mobile platform.

It includes dual front-facing cameras for six degrees of freedom (6DoF), which is done at a fast 800Hz rate using the integrated Qualcomm Hexagon™ DSP. This allows for a user interaction where they are free to explore as they see fit, and provides a natural feeling to the environment.

The truly mobile nature of the Snapdragon 835 VR development kit allows users to be immersed in a virtual world without tethered cables, which could distract from examining a patient from all angles to ensure an accurate diagnosis. Not only that, but now medical providers can be trained anywhere without cumbersome and costly setups.

While the Hexagon DSP offloads the 6DoF calculations, the robust Qualcomm Adreno™ 540 GPU provides a high framerate experience that is designed to help reduce motion sickness. The ForwardXP team was able to throw high resolution models and textures at the GPU without any degradation in performance.

When compared to other XR solutions, the Snapdragon 835 VR development kit delivers a robust solution for this type of implementation.

Source: QTI internal testing

Software

ForwardXP used Unity 3D to build their training platform, which is compiled to an Android package and deployed to the platform. Unlike other solutions, the Snapdragon 835 VR development kit is a standalone product that doesn’t need to worry about phone calls, multitasking or other burdens that one may get using a smartphone VR solution.

The Snapdragon VR SDK exposes the high precision 6DoF tracking to Unity3D, which delivers a more immersive experience. The power of the Adreno 540 GPU allows artists the freedom to provide a scene that is highly detailed and realistic looking. When dealing with patient diagnostic tools, the realism should be preserved so the training carries forward into real life.

Image: Jah Raphael developing the ThinkFAST VR simulation program in Unity 3D.

Think F.A.S.T. has also integrated precise hand movement tracking from Leap Motion to help increase the sense of immersion. Since the training is geared toward medical professionals who may have never used a gamepad before, the natural hand motion is a key component.

To further increase the natural user interaction, ForwardXP has also integrated proprietary voice control. This allows the user to interact with the virtual patient in a comfortable way as if they were talking to one of their own patients.

The Medium Matters

When it comes to stroke victims, minutes can save lives. And practical hands on training can help reduce the time it takes to make an accurate diagnosis.

Interested in VR development? Check out these resources on Qualcomm Developer Network:

• Snapdragon VR SDK - Provides developer access to optimized, advanced VR features on Snapdragon VR devices.
• 3D Audio Tools - Suite of tools designed to help create true-to-life immersive 3D audio experiences including support for scene-based audio.
• Snapdragon 835 VR Development Kit - Standalone and untethered all-in-one VR headset with 6DoF support, powered by the Snapdragon 835 mobile platform.

Hardware makers can also take advantage of our HMD Accelerator Program to utilize Qualcomm Technologies reference designs and/or work with a turnkey original design manufacturer.

Whatever your role in XR development, and whoever you’re building immersive realities for, Qualcomm Developer Network has the tech, tools, and resources to help. Sign up for email updates to keep up with the latest from QDN.