Deep learning (DL) models are becoming so effective at classifying human emotions from facial expressions that they are approaching, and in some cases surpassing, human performance. However, the decisional process behind the models’ output remains unclear. On the assumption that there are some similarities in the operativity of DL models and the human brain in the visual analysis of emotional facial expressions, this work aims to compare seven DL models with the behavioural responses of a group of 54 human subjects on a task of emotion recognition using Ekman’s basic emotions facial expressions. Human visual attention has been studied using eye-tracking analysis recorded during a facial emotion recognition task, while DL models were trained and tested on the same dataset of images.