019-22 – Behavioural improvement and functional brain changes in multisensory integration after audio-visual training in virtual reality

Introduction and background:
When interacting with external events, humans use multiple sensory systems simultaneously to perceive and respond. Understanding cross-modal learning effects in virtual environments can help determine how information is integrated across sensory modalities. Learning more about the plasticity of human cortical audio-visual integration could enable us to design more effective learning environments that provide better learning outcomes. Among the key questions in this area is how virtual reality training alters brain functions that correlate with cognitive performance.
Methodology:
In order to investigate the association between neuroplastic changes and behavioural performance, twenty healthy participants followed a home-based audio-visual training program on virtual reality for 30 minutes daily, five days a week, for four weeks. During training, performance data was logged on the VR system, and more formal laboratory tests, including voluntary and involuntary eye movements, and target detection were conducted. Imaging data (fMRI) was collected at baseline, after two and four weeks of continuous training, and four weeks after training ended.
Results:
Participants showed significant improvements in behavioural performance (faster task completion time and higher scores over time). Functional activation increases (BOLD signal) were observed in multisensory brain regions which are involved in early-stage audio-visual processing during the bimodal fMRI task in comparison to the unimodal task.
Conclusion:
Adding spatial audio cues to voluntary visual training in VR shows brain activation increase in multisensory integration, providing measurable performance gains, which applies to involuntary and visual search conditions. This could be helpful in optimising rehabilitative training programmes.