Sound of vision 3D virtual training environments-a gamification approach for visual to audio-haptic sensory substitution

The purpose of this paper is to introduce our training strategy, based on gamified 3D virtual environments, for improving sound localization and navigational skills of visually impaired people (VIP). During training, the VIPs will use an advanced sensory substitution device (SSD) which conveys information about the environment. An example of such device is the Sound of Vision device (Sound of Vision website) which performs a visual to audio-haptic encoding of the environment. In this paper we explain how a virtual environment can provide, in such cases, the same scenarios and outputs as real life situations, within full safety and control. Most serious games are specifically designed for training or educating users, rather than entertaining them, and in most cases, users play them out of necessity to gain a practical skill set-which is also the case for our visually impaired users. However, in this paper, we argue that a serious game success is highly dependent upon creating game specific entertainment and even fun, immersion and motivation for the user. Serious games' effectiveness comes from many related factors, such as sense of presence, mental and emotional involvement, motivation, sense and flow achievement, player's feeling of uniqueness and individualization in the environment. In this paper we explain our efforts in designing a "serious-but-entertaining" game with hierarchical levels of difficulty that would provide auditory and haptic stimuli and tune in unique ways general game design and learning principles, such as learning by doing or experimenting, reflection and meta-reflection for transferring the learning outcomes from virtual to live contexts. Particular attention is given to level design and interaction (which, unlike in almost all other cases, is based only on audio-haptic outputs) to maximize immersion and motivation through all available means and taking into account the specificity of our users group.