Behavioral Analysis in Immersive Environments

My internship was part of an ongoing research project on behavioral ana- lysis in immersive environments, specifically within a virtual reality driving simulation. The goal was to analyze and model users’ emotional reactions using multimodal data (text, audio, video, heart rate, and eye-tracking) to enhance real-time emotional adaptation in VR experiences. I collaborated closely with a PhD researcher working on this topic, con- tributing to his research with the aim of publishing the findings at the end of his study. My role involved developing multimodal AI models based on Transformer architectures (A-MulT and Husformer) to predict valence and arousal. Experiments were conducted on state-of-the-art datasets (MSP- Podcast, IEMOCAP) as well as an internally collected ALTEN dataset. The results highlighted the importance of multimodal fusion, with eye-tracking and physiological signals playing a significant role in emotion detection.