Apple Glass could make AR believable with accurate 3D audio
Apple's AR headset or "Apple Glass" smart glasses may offer realistic sound effects to go with its digital imagery laid over a real-world view, by accurately reproducing a sound in such a way that it matches where the sound would come from in 3D space.
Gaming and virtual reality systems all offer some level of audio feedback to the user to accompany visuals. In the case of augmented reality, the experience of seeing a digitally-created item in the real-world view could easily be accentuated by a sound effect when it is interacted with in some way.
While audio can help sell the illusion of video, there are some limitations. A typical stereo headphone setup may provide some limited left and right differentiation at a basic level, while virtual sound systems are able to fake 3D positions by altering the properties of the sound effect to mimic distortions based on its positioning.
For example, a sound played behind the user's head could be muffled, while one from the front could be much clearer. However, while this can be useful, the effect isn't truly believable as differences in physiological elements like ears, the head, and body can alter what a person expects to hear from a noise in a specific position.
In a patent granted to Apple on Tuesday titled "Head related transfer function selection for binaural sound reproduction," Apple believes it can make the effect of virtual 3D sound positioning better by having what it refers to as the Head related transfer function (HRTF). These are effectively a pair of acoustic filters for each ear that characterize the transmission of sound from one place in an environment to the ear.
Each HRTF varies on a number of factors, so as to match differences between people in terms of spatial hearing expectations. While it is relatively time-consuming or impractical to create a HRTF for an individual user using consumer electronics with any real level of accuracy, there is the possibility of using either a generic "average" HRTF, or selecting a HRTF that is closest to the user's needs.
The HRTF is then used as part of a binaural simulation digital signal processing algorithm to reproduce an audio recording as binaural sound, which in turn is played through headphones.
The patent centers around the idea of selecting the nearest HRTF to what is needed, by whittling down the database to a select few. The pruning process can include taking away options that are far from basic characteristics of the user, such as height, gender, race, and age.
Cutting down the database to a smaller number increases the chance of the most-correct HRTF being used, the patent advises. After the initial settings, the system could then narrow down its choice to the most generic from the reduced list, to select the most appropriate HRTF for use.
To help the system more, it is suggested that a binaural measurement could be taken by the audio device for the user. If the device is being used by only one user, the measurement would only need to be performed one time, but it would offer potentially other information about which HRTF would be best.
It is also suggested that, by grouping users, the mass usage of different HRTFs could be monitored to help improve selection. For example, knowing that users aged 20-25 prefer HRTFs with certain properties will mean the system would err towards those particular filters if it encounters someone matching those settings in the future.
The filing lists its inventors as Darius A. Satongar, Jonathan D. Sheaffer, Martin E. Johnson, and Peter V. Jupin.
Apple files numerous patent applications on a weekly basis. While the existence of a filing indicates areas of interest for the company's research and development efforts, they do not guarantee the concept will be used in a future product or service.
Apple's audio processing has resulted in products like the HomePod, as well as its pass-through noise cancellation feature in the AirPods Pro, but it is still continuing to develop more sound-related ideas.
In December 2019, it was granted a patent for a "System to move a virtual sound away from a listener using a crosstalk canceller," which performs a similar spatial audio trick in making a sound appear to emanate from a different place than the device's speakers.
One suitable place the patent could be applied is in "Apple Glass," Apple's long-rumored AR glasses. As it is thought to use AR, and would apply digital items on a real-world environment for the user to see and interact with, perfecting the audio playback could help make the non-existent item more believable to the wearer.
A related patent
called "Display Devices with Multimodal Audio" suggests a headset could
have earpieces that adjust to suit the type of audio being presented.
Apple Glass can make AR more immersive with 3D Audio support
The Apple Glass, an AR headset, may offer a few interesting features. To make the whole experience more immersive it could offer an accurate reproduction of sounds on a 3D plane, according to a report from AppleInsider.
Virtual reality in gaming and augmented reality has become increasingly more popular in recent times, and the people behind such technology are constantly trying to make the world you see more believable. The latter in particular has revolved around bringing more to the mundane world around the user. Thus, the added realism from the accurate audio could make digitally created structures seem more genuine.
his could also help improve interactions with said artificially produced objects in the users’ vision. While most stereo headphones offer basic left and right differentiation, a virtual sound system can reproduce audio positioning on a 3D plane by altering the properties of the sound effects to mimic the distortions caused by different factors.
The Apple patent titled “Head related transfer function selection for binaural sound reproduction,” points at such a system. In simpler terms, the Cupertino based giant seeks to improve 3D sound positioning by having Head related transfer function (HRTF), which is basically a pair of acoustic filters for each ear. These help in differentiating the transmission of sounds from various positions in the environment around the user.
Unfortunately, the Apple patent does show the company’s interest in this field. Although, it does not provide any guarantee that the company would work on such a system. Rather, it could be their efforts in covering various bases for future possibilities which are again unlikely.
Post a Comment
If you have any doubts, please let me know