audio-AR exploration
Kunsthochschule Weißensee
2023
My decision to explore audio-augmented reality and see if it could be of interest to interaction designers was influenced by my thoughts on smartphones and their attention-grabbing screen displays. Smartphones allow us to do all sorts of things digitally. In and of themselves, they already augment our reality, providing digital help to solve our real-world problems. But they also demand a lot of attention and make it quite difficult to focus on the real world since we have to look at those tiny screens every time we need help. My first idea was to write a thesis that criticized the smartphone from a designer‘s perspective. But then I was wondering, what practical alternatives could I propose as a designer?
I‘ve always been interested in using augmented reality with the phone, but it was mostly an unsatisfying experience as all the digital objects were only visible on the small screen. I wanted to know how audio could be used to convey the same information that images normally do. What spatial, narrative and structural guidelines are necessary to communicate the information in an understandable way and where are the limits of all of this?
1. Exploring head tracking and spatial audio technology
Audio-augmented reality is a rapidly growing field that merges virtual audio information with the real-world environment. The recent popularity of wireless headphones, audio con- tent, and improvements in technologies like transparent hearing and spatial audio, open up new possibilities for audio-augmented reality and help it become more accessible to the general public. With more technical development, AAR could be used to create complex audio interfaces and develop screen-less, hands-free, and eyes-free interactions.
I used Unity and VR/AR outlets to test and explore spatial audio technologies.
How can spatial audio information be meaningfully arranged in 3D space? What significance can the spatial position or movement of a sound have in order to convey information to the listener that goes beyond the content of the audio files? For example, how can I show that a sound information is very important without having to mention this in the audio file itself? The interaction patterns cover four areas: interfaces, spatial parameters, audio formations and gesture control. All four serve the purpose of better interactions with the spatialized sound that is mapped to the real world and also with headphones.