HMI and VR for trustable and comfortable driving on autonomous vehicles: Multimodal interfaces in a virtual driving simulation

Automotive companies are investing a lot resources in the field of self-driving cars, which represent one of the key technological challenges of the coming years. The lack of trust on the side of users is a critical aspect for the technology’s success. The design of a user interface that deliver a proper amount of context information is certainly one of the most important factors that can improve trust. Context information allows the occupants to be aware of what is going on outside the vehicle as well as the of the vehicle’s own decisions. Once the users have gained a certain level of confidence, they can engage in secondary tasks without having to constantly monitor the driving system’s actions. This thesis specifically focuses on prototyping using virtual reality (VR) technology a new interface based on current state of the art in the field. The proposed interface consists of different elements. First, it includes an Augmented Reality Head-up Display (AR-HUD) that has the function of alerting the driver of dangers such as roadworks or obstacles (which will require human intervention), or warning him or her of what is present in the surrounding environment, from area signs to speed limits. The interface also exploits a static HUD to provide the user with quantitative information, such as speed or proximity. Finally, the interface features a dashboard which includes general vehicle information, supports a possible secondary task (like, e.g., watching a video), and also incorporates a world-in-miniature visualization, showing the car’s perception of the environment and its actions. Visual information is accompanied by several auditory outputs or voice feedback to inform the driver about the vehicle’s actions or to report cases of takeover requests. The thesis has prepared the essential material that will be later needed to perform the experiments planned as future work for validating the devised interface.