Evaluation of synthetic ECG signals generated with a Generative Adversarial Network

Developing machine learning (ML) algorithms for rare disease detection, such as Brugada syndrome (BrS), can be challenging because of the lack of data. For this reason, generative adversarial networks (GAN) can be employed to perform data augmentation, to obtain a larger dataset. In order to adopt GAN-generated data in training ML algorithms, their quality and physiological reliability must be investigated. In literature, many metrics are used for this purpose, but none of them is commonly recognized as a standardized evaluation criterion. The first purpose of this work is the evaluation of ECG signals previously generated with a state-of-the-art GAN. Six ECG morphological features are used to assess the physiological reliability of the generated signals. Twelve of the most used state-of-the-art similarity metrics are implemented in order to assess the similarity between real and fake signals. Additionally, in the second part of the work, the most widely used metrics for ECG similarity estimation are compared based on five criteria: their ability to evaluate the temporal evolution of the signals, their dependency on the signal length, the possibility to evaluate pairs of signals of different lengths, their sensibility to peak misalignment and to the presence of artefacts and noise. The final results lead to the conclusion that the generated signals are physiologically reliable according to the calculated morphological features. In addition, based on the distributions of the similarity metrics the generated ECGs can be considered quite similar to the original real ECGs. However, based on the results of our analysis, each of the implemented metrics has strengths and weaknesses that should be considered while evaluating synthetic ECG signals, especially in the case of multi-beat ECGs.