Video Synthetic Data for In-cabin Sensing

Vehicles become more connected and autonomous and the occupant experience inside the vehicle becomes even more important. Advanced driver assistance systems (ADAS) are emerging as crucial components moving towards higher level of autonomous driving vehicles, aiming to enhance road safety by continuously assessing and analyzing driver behavior, actions, and overall state while behind the wheel. The types of sensors that could be used for the purpose of monitoring the driver are multiple: cameras, infrared sensors, and eye tracking devices, to detect and monitor a range of driver-related parameters, such as head position, eye gaze, facial expressions, and physiological signals. According to EURONCAP latest implementation of the assessment protocol for safety assist (December 2023) the driver monitoring system (DMS) implementeation from car manufacturers should demonstrate the capabilities of the system in various driving conditions and how it can identify various driver impairments. • Sensing: The dossier should provide evidence that the DSM system can accurately detect a wide range of drivers, including those with different facial features, hairstyles, and headwear. Additionally, it should show that the system can operate effectively in a variety of lighting conditions, including bright sunlight, dimly lit environments, and nighttime conditions. • Driver State: The dossier should detail the specific elements of driver impairment that the DSM system can identify. This includes drowsiness, distraction, and unresponsiveness. For each impairment, the dossier should explain the specific sensor inputs that the system uses to detect the impairment and the algorithms that are used to analyze these inputs. • Vehicle Response: The dossier should describe how the vehicle responds to a detected driver impairment. This includes the types of warnings that are provided to the driver, the potential for automatic interventions, and the specific actions that the vehicle may take in response to an impaired driver. The thesis goal is to create camera simulations allowing a virtual replication of the driving experience to create a synthetic dataset, this dataset would be exploited to test Machine Learning model to recognize dangerous in-cabin situations and detect driver impairment. The same model has been already trained and validated on real data coming from Drive&Act dataset.