Design and implementation of an advanced monitoring system for alert management and log analysis in a mobile payment app

The thesis work has been conducted with the collaboration of NEXI Digital S.R.L., inside a team adopting the SAFe framework for Agile development methodology, which is a methodology used in large organizations with multiple teams cooperating among themselves, to develop complex products in a very quick and continuous way, using a specific set of technologies. The case study is about the MyPayments application, which manages merchants’ data and different kinds of PoS implementations. This application is in the digital payments market and allows to manage physical PoS terminals, but also soft PoS terminals, which are increasing over time also because they work on common smartphones. Although the application has a very extended backend, including databases, the whole system has some technical issues to be solved, like absences of services or high waiting times inside the application under specific conditions. Specifically, the objective of the work consists in identifying these anomalies and improving the reaction time to these issues, thanks to a real-time alerting system, also in order to increase the user experience. The methodology applied in this work consists in analyzing potential solutions with different approaches, and then suitable technologies for the chosen approach; describing how the parsing of the raw log data has been done, the partial output structure, which is used by the machine learning model to be trained, and how the training and execution of the model are implemented, using two different scripts. The results obtained are positive and respect the given requirements, since the anomalies have been successfully identified with very high recall. The benefits of the results in terms of user experience and reactiveness to issues are evident, because the anomaly alert would be received by the team as soon as an issue is identified, so that it can be solved in the least possible time. As a conclusion, the proposed system can be implemented inside the NEXI infrastructure in the future, both inside the Cloud or in any virtual machine. The analysis and design of the entire system consists in a RabbitMQ implementation, including the scripts defined to parse raw logs and to execute the XGBoost model, with the script to send email alerts. It will also be possible to extend the machine learning model to identify new anomalies or malicious patterns which may be potentially insecure for the backend servers and applications.