Data Mining for Fault Management in Telecommunication Networks: Clustering-Aided Network Supervision

Big Data and Machine Learning are nowadays interesting and fashionable topics. Their usage is increasingly massive in our society and many companies are following this direction to streamline their business and maximize revenues. TIM SpA is no exception: it has been working in collaboration with the Polytechnic of Turin for some years with the aim to build an intelligent alarm management system capable to handle and predict the failures raised from networks located along the national territory. In the mentioned context, this project aimed to analyze TIM alarms raised from both the transmissive domain (TX) and the Mobile Access Network domain (MOB) with the purpose of finding spatial-temporal correlations between them. For this purpose, starting from provided company’s datasets, failures are firstly grouped inside clusters according to specified requirements, secondly processed by a Pattern Mining technique to extrapolate statistically meaningful patterns and finally visualized through explorative dashboards in form of web application. This work is therefore based on two main processes: a “back-end” process which takes as input the TIM raw alarms and returns failures clusters and related patterns, and a “front-end” process which takes as input the “back-end” output to visually display and easily navigate between the returned results. As regard the “back-end” process, after an initial cleaning phase, alarms are grouped inside clusters according to two different strategies: the first strategy is thought to groups failures belonging to the same device whenever their first occurrence fall in a maximum delta time of five minutes; hence, the algorithm returns group of alarms potentially correlated inside the same device along the temporal axis, reason why these clusters are here called “InDevice-clusters”. The second strategy returns the so called “AOU-clusters”, aggregating upon the temporal groups over a geographical dimension. The spatial dimension identified for this kind of aggregation is the “aou” field, already used and well-known by TIM staff for their operations. This last strategy has returned quite interesting results, especially for massive failures conditions, since it collects alarms of a larger pool than the temporal strategy which, although it has returned remarkable results at the experts’ eyes, seems less interesting in view of a large-scale application. Once clusters are computed, they are processed by FP-Growth Pattern Mining algorithm: the idea is trying to find meaningful associations between alarms especially exploiting as main information their probable cause, obtaining a list of frequent itemsets as frequent slogans occurring together, and related association rules. Many experiments have been carried out, collecting clusters and patterns for any single TX network, for the MOB network, among the whole TX domain and on both TX and MOB domains bonded together. The “front-end” process takes as input clusters and extracted patterns in order to visualize and explore them exploiting their intrinsic relations: indeed, given a certain association rule, there will be at least one itemset from which it has been generated (the one composed by the antecedent and the consequent and its superset) and, chosen in turn a certain itemset there will be at least one cluster from whose basket of alarms that itemset has been extrapolated. Moreover, it is also possible to navigate within a specific cluster, viewing the alarms included and some descriptive and statistical characteristics.