Snapdragon and Qualcomm branded products are products of
Qualcomm Technologies, Inc. and/or its subsidiaries.
What will it take for headphone audio to feel and sound like real-world audio?
The engineers at IDUN audio in Denmark have an answer: dynamic spatial audio. They’ve implemented an interactive acoustics simulation in software running on the Qualcomm QCC5171 and Qualcomm QCC5141 chipsets, which manufacturers build into wireless headphones and earbuds. In this post, you’ll learn about the problems inherent to headphone audio, IDUN audio’s approach to addressing them and what it’s like to implement an audio algorithm.
Making headphones feel like the real world
Humans, like most animals, naturally hear the direction from which a sound is coming, and how far away the source of the sound is. We turn our head (“head tracking”) toward sounds not only to hear and interpret them better, but also to capture useful context like direction, distance, and velocity.
But most of that context vanishes when we listen to digital audio with headphones or other hearables. Because the sound is inside our head, we can’t use the auditory capabilities we take for granted in the real world. In fact, consuming almost any kind of audio — conversations, podcasts, concerts, movies, games — through headphones is an unnatural experience, causing fatigue with prolonged listening.
IDUN audio has developed a virtual and augmented reality (VR/AR) audio algorithm - dynamic spatial audio - that allows listeners to use their natural hearing abilities when listening to audio through headphones. The simulated acoustic environment is fully interactive, allowing listeners to move around and turn their heads, while the sound at the ears is updated just like in real life. Simulation can be configured to improve the listening experience in many everyday scenarios such as:
- During online meetings. It is much more natural to hear individual participants talking from different directions than inside the head. Speech intelligibility also improves a lot when people talk at the same time.
- When listening to music, movies, audio books or podcasts, the sound image can be placed in front of and around the listener, for a heightened sense of engagement.
- While gaming, dynamic spatial audio can be advantageous, since players can hear sounds from all directions and the head tracking allows them to locate the source of the sounds more precisely.
The key, of course, is to get the technology into the headphones.
Implementing spatial audio in software
Software from IDUN audio creates a three-dimensional, acoustic space in which listeners can enjoy a heightened sense of engagement because they consume digital content with their natural, spatial hearing. The acoustic space can be adapted with a sophisticated room-simulation model to create anything from a small, dry room to a large concert hall.
IDUN audio optimized the entire spatialization algorithm, with its large number of configuration options, for extremely low processing power and memory requirements. But instead of implementing dynamic spatial audio on a computer or a smartphone, they prefer to run their algorithm in the hearable. That allows listeners to enjoy spatialization independent of the playback device. It also eliminates any latency between the hearable and the playback device, enabling more-responsive head tracking. Updating the audio at the listener’s ears according to their head movements in real time is essential for providing a precise, stable sound image.
Utilizing hardware from Qualcomm Technologies, Inc.
IDUN audio has implemented their dynamic spatial audio software on the Qualcomm Kalimba digital signal processor (DSP) in the QCC5171, an audio platform chosen by manufacturers of wireless hearables.
Battery power is a precious element in wireless gear, and the IDUN audio solution uses only a fraction of one of the two DSPs on the QCC5171. The solution’s low power consumption and small memory footprint are appealing characteristics for manufacturers that want to differentiate their product offerings.
To take advantage of head tracking, IDUN audio uses an inertial measurement unit (IMU) consisting of an accelerometer and a gyroscope. IMUs have become relatively inexpensive, despite their small size and high accuracy. So adding an IMU to headphones is straightforward. Adding it to earbuds, however, is different. Since signals are processed separately on the IMU in each earbud, a communication protocol is required to synchronize head tracking across the pair.
To add their dynamic spatial audio to both forms of hearable, IDUN audio has capitalized on Qualcomm technology in the QCC5171. The first advantage is sensor fusion, which includes a library for converting the accelerometer and gyroscope data into a combined format that can be used for head tracking. Another big advantage is the synching mechanism for aligning the head tracking data between the two earbuds. Those features result in the stable, reliable head tracking needed for dynamic spatial audio.
Lessons learned on the QCC5171
The engineers at IDUN audio worked closely with our team to receive the hardware and support they needed and were able to complete their implementation within just a few months.
It’s important to IDUN to be able to run their spatial audio software on a wide range of DSPs and microcontroller units (MCUs). They write their code in pure C so that it does not depend on external libraries.
Generally, it’s useful to have an implementation consisting of floating-point operations that are applied on a sample-by-sample basis. However, for the QCC5171 they refactored the algorithm to use fixed-point operations applied on a block-by-block basis, with custom block operations written at the bare-metal level. They found that the latter approach was less complex, offered higher performance and led to gains on other platforms as well.
Spatial audio and the future
Our 2022 State of Sound Report points out the importance of spatial audio. More than half of respondents claimed spatial audio will have an influence on their decision to buy their next pair of true wireless earbuds. Moreover, 41 percent said they would be willing to spend more for the feature.
As headphones and earbuds enjoy rapid adoption, they, like many wearables, are getting more processing power and more sensors. The trend will lead users to depend less on computers and smartphones as processing hubs; instead, the processing will be pushed to edge devices like hearables. The edge devices will, in turn, connect directly to the Internet, rather than depending on a connected device by default.
IDUN audio believes that is why processing in the headphone is so important. They also believe that combined technologies like dynamic spatial audio and the QCC5171 or QCC5141 enable headphone manufacturers to customize user experiences and differentiate their products from those of competitors.
For more information about the great innovation IDUN is working on, be sure to visit their website.
Snapdragon and Qualcomm branded products are products of Qualcomm Technologies, Inc. and/or its subsidiaries.
