Snapdragon and Qualcomm branded products are products of
Qualcomm Technologies, Inc. and/or its subsidiaries.
5G mmWave networks have set the bar high with promises of low latency, high bandwidth, and a level of performance to allow for real-time operations and processes to be controlled over the air. While there has been a lot of hype around this, the real proof comes from seeing it in practice. So if you’re going to build a proof of concept for 5G, why not also make it fun? That’s what we did at the recent Snapdragon® Tech Summit 2020, where we demonstrated the possibilities of 5G with our 5G radio controlled (RC) vehicle demo (RC demo).
The RC demo consisted of two RC vehicles whose stock receivers and transmitters have been replaced with 5G phones powered by the Snapdragon® 888 5G Mobile Platform, and communicate as peer-to-peer devices. One was mounted on the RC vehicle to control the servos, and the peer was used by a driver, located one mile away, to send steering, throttle, and brake commands to the vehicle over 5G networks.
The two drivers were also equipped with a PC and monitor that received a streamed video feed from their respective RC vehicle’s phone to engage with the RC vehicle based on a first-person view. As you can imagine, this circular dependency between the view streamed from the vehicle and the transmission of the drivers’ inputs back to the vehicle, requires very high-speed communications to operate successfully in real-time. A perfect test for 5G.
So, how did it all come together? Be sure to check out the video from our Tech Summit keynote below, and then in the rest of the blog we’ll share some of the behind-the-scenes details that went into making this demo.
Architecture
The RC demo took place across two separate campuses. The racecourse was on the outside of one campus, while the drivers themselves were located one mile away in a separate building on a different campus. Both locations are equipped with 5G mmWave network over which communication between the drivers and their respective vehicles occurred. The figure below provides an overview of the system architecture and how it’s distributed across the two campuses:
Using the application running on the phone, drivers would tilt their phones like steering wheels to control direction. The application also read the onboard gyroscope sensor to derive a steering angle, while on-screen “joysticks” interpreted finger gestures as throttle and brake/reverse duty cycles in the range of 0 to 100%. The application also offers the option to use an external steering wheel connected to the phone via USB, though this wasn’t used in the demo. The driver’s inputs were sent to the RC vehicle’s phone over 5G, and the final pulse width modulation (PWM) signals were then sent directly from that phone to the RC’s servos.
The phones mounted on the RC vehicles, and additional phones placed around the racecourse, streamed video feeds of the action over 5G. To handle communications, network infrastructure from Ericsson and Verizon along with technology by Tension OnSite streamed data and videos with low latency. At the protocol level, TCP guarantees delivery of all packets to avoid unexpected driving characteristics while the large amount of bandwidth provided by 5G mmWave, makes data compression unnecessary.
Both drivers’ PCs received their respective video feeds from the network. In addition to this video data, the RC vehicles also collected and transmitted:
- Full gyroscope data from the vehicle so that the PCs could render a 3D model of a vehicle at the proper angle
- GPS location with lane-level assist to enhance differential GNSS (eDGNSS)
- Vehicle speed telemetry
The network also carried video over 5G to a separate phone powered by the Snapdragon 888, for a spectator to follow the action. In the demo, this spectator was none other than Cristiano Amon.
Project and Application Development of the Demo
Building a new network for the demo required many moving pieces like allocating spectrum, determining the location of the antennas, integrating systems, and lots of testing.
The project took several months to complete, but we had early prototypes working within a month, so the remaining time was primarily spent trying to achieve the desired level of performance. Everything was custom made, which also took a lot of effort and time. This included everything from the control boards that control the RC vehicles’ electric motors to the phone holders mounted within the RC vehicles which were custom designed and 3D printed to fit our 5G prototype devices.
The app was developed in-house to gather data from drivers’ phone’s accelerometer and on-screen “joysticks” and compile that information to send over 5G to the vehicles’ phones. The app also bridges with the Tension camera app, by sending throttle and steering position and gyroscope data for the drivers to receive telemetry information of the car while driving. In addition, the app provides options to connect to different cars, set wheel alignment trim, lower engine power for training mode, and optionally use an external steering wheel device instead of the phone.
The drivers’ video feeds included a mini-map of the race course showing the position of the RC vehicles. Since the race course had a relatively small width of approximately two meters, lane-level vehicle navigation (with meter-Level positioning) was incorporated to ensure that the position was not rendered outside of the race course graphic. This lane-level navigation combines the following technologies: dual-frequency GNSS, Qualcomm® Sensor-Assisted GNSS and Qualcomm® Enhanced DGNSS(2) where differential GNSS corrections are received by the device over a 5G-NR connection.
When driving vehicles at a reasonable speed, any latency on the input can yield a big difference between the vehicles’ desired and actual path. That’s why low latency was one of the key differentiators for this demonstration and allowed for such precise driving over the network.
Another important ingredient for this demo was the sheer bandwidth capacity provided by 5G mmWave. The drivers relied on video feeds directly from the RC vehicles, which required a lot of bandwidth. The demo setup included one camera per car, in addition to multiple cameras throughout the track, all streaming at the same time. Not only did those cameras require a lot of bandwidth, they also had to work in low latency, otherwise the drivers would base their reactions on past video frames which wouldn’t represent the vehicles’ current positions and orientations.
Conclusion
5G technologies are being deployed throughout the world and their promises of high bandwidth and low latency are starting to come to fruition. Moreover, these aspects of 5G are now paving the way for new real-time uses cases as shown in the RC demo.
We hope the RC demo has inspired you to create your own high-performance, real-time 5G solutions. Developers interested in learning more should check out our Snapdragon 888 press note and stay tuned to our Snapdragon Developer Tools page on QDN as the Snapdragon 888 is rolled in the first quarter of 2021.
For additional behind-the-scenes footage of the RC demo, be sure to check out this inside look video:
Driving cars with smartphones and Snapdragon 888
Snapdragon, Qualcomm Sensor-Assisted GNSS, and Qualcomm Enhanced DGNSS(2) are products of Qualcomm Technologies, Inc. and/or its subsidiaries.
