012-23 – Enhancing Learning Outcomes through Multisensory Integration: An fMRI and DTI Study of Audio-Visual Training in Virtual Reality

Virtual reality (VR) technology offers a unique opportunity to study the integration of information from different sensory modalities, a fundamental process in the brain that enhances the perception and modifies behaviour. Recent research has shown that the brain’s microstructure can undergo experience-dependent plasticity even in adulthood. This study aimed to investigate the effects of an audio-visual training program using VR on brain activity, microstructure, and cognitive performance. Twenty healthy participants completed a 30-minute daily training program on VR, employing a ‘scanning training’ paradigm commonly used in hemianopia rehabilitation. Throughout the study, neuroimaging data (fMRI and DTI), performance data, and laboratory tests were collected at different time points: baseline, after two and four weeks of training, and four weeks post-training. The results showed significant performance gains when spatial auditory cues were incorporated into voluntary visual training in VR, benefiting both involuntary and visual search conditions. Post hoc comparisons on VR data analyses revealed a significant decrease in mean response time from 1.63 (±0.4) seconds on day one to 1.23 (±0.14) seconds on day 20 (t(361)= 9.24; p = 4.21e-16). Furthermore, neuroimaging analyses revealed increased activation in key multisensory brain regions involved in early-stage audio-visual processing with microstructural changes in the optic radiation and superior longitudinal fasciculus II, crucial white matter tracts related to multisensory integration. The study sheds light on the neural mechanisms underlying multisensory integration in virtual environments, providing insights into the benefits of incorporating spatial auditory cues into voluntary visual training. This research contributes to the growing body of evidence supporting the efficacy of VR-based interventions for cognitive enhancement and rehabilitation purposes.