Neural networks for the recognition of 3D facial expressions

Facial expression recognition technology is a branch of face recognition technology. With the development of artificial intelligence technology and computer technology, expression recognition technology can analyze the image to determine the psychological emotions of the recognized humans and realize the computer’s understanding of facial expressions. Moreover facial expression recognition can fundamentally change the relationship between people and computers so that computers can better serve humans and achieve better human-computer interaction. In applications, facial expression recognition technology is used in psychology and intelligent robots; and it has great potential application value in intelligent monitoring. For example, in the retail industry, it is possible to identify the customer’s expression to obtain his preferences for products or to intelligently recommend suitable advertisements through expression recognition to achieve precise marketing and interactive experience. It can also be applied in teaching; online education monitors students’ class status by identifying students’ facial expressions. In terms of traffic safety, it can effectively reduce the development of fatigue in driving and other situations by identifying the status of drivers. At present, facial expression recognition is mostly based on 2D images. The research on 2D face recognition is relatively long, and the method is relatively mature. However, due to the limitation of depth data loss in 2D information, it is impossible to express the real face fully. It is easily affected by posture and illumination, resulting in low recognition and live detection accuracy. The 3D face data can show the characteristics of different face angles, which has more one-dimensional depth information than the 2D face data. Regarding recognition accuracy and living body detection accuracy, 3D facial expression recognition has advantages over 2D facial expression recognition. Therefore, this thesis studies 3D facial expression recognition, focusing on recognizing the six basic emotions cited by Paul Ekman (i.e. anger, disgust, enjoyment, fear, sadness and surprise), except for neutral emotion. Two 3D facial expression databases are used in this study, BU3D-FE (Binghamton University 3D Facial Expression) and CalD3r, where CalD3r is a spontaneous expression database. And the content of the method is divided into three steps. Firstly, preprocess the data of the two 3D facial expression datasets; for the BU3D-FE datasets, once WRL files are converted into PCD (Point Cloud Data) files, data augmentation and dataset division are performed; The difference between the CalD3r database and the BU3D-FE database is that the depth images of the CalD3r database are converted into PCD files, and then denoised. Secondly, once the point cloud data points are randomly extracted from the PCD samples obtained in the last step, the three-coordinate value of each point is obtained, and 1D-CNN (one-dimensional convolution neural network) is used to extract key features of the coordinate values in the three directions of x, y, and z. Thirdly is classification, six outputs are obtained through the fully connected layer, and the SoftMax function is used to obtain the probability that the sample belongs to each expression; combined with the loss function, backpropagation is performed in the training phases, and then the model parameters are updated to improve the classification accuracy.