Empowering human-computer interactions with advanced brain imaging and real-time virtual reality interfaces

A Brain-Computer Interface (BCI) is a communication system that enables the users to control a machine with a set of predetermined commands by means of their neural activity. The BCI systems are usually composed of three modules: (i) a brain acquisition system, which includes some sensors to collect data from the neural activity of the user; (ii) a computational system, which processes the acquired data in order to determine the task executed by the user; (iii) an interactive system, which reacts to the commands received and provides a feedback to the user. The electroencephalography (EEG), that is one of the most commonly used acquisition systems, consists in the measurements of the electrical activity of the brain by means of several electrodes positioned on the scalp of the user. In this work we focus particularly on a BCI paradigm that exploits brain signals that are triggered by external stimuli, categorised as evoked potentials in literature. Specifically, we developed a BCI pipeline based on Steady-State Visual Evoked Potentials (SSVEP), where the user focuses on a visual stimulus flickering at a constant frequency leading to have a similar signal projected in the electrical activity that can be observed from the EEG. We exploited this paradigm to enable the users to navigate in a virtual environment: some visual stimuli are proposed to the user that can focus on them to issue commands to the interactive application. A framework containing several tests and application examples was developed and analysed on real-data experiments. Furthermore, the possibility of the integration in a Virtual Reality (VR) system has been evaluated. Finally, a part of the work was included in a framework that lets the users see the real-time electrical activity of their brains in an immersive VR environment. Among other things, this impacts very favourably neurofeedback, a therapeutic practice that allows the patients to self-regulate their brain activity thanks to the real-time feedback provided.