

This chapter deals with the core of the project: the AI model designed to recognise malicious traffic from the benign one. It starts with an introduction about the AI, follows with the examination of the dataset and goes on with a dissertation about the ML algorithms considered and the chosen one. Afterwards a section about the output files and finally an analysis of the accuracy of the model. 
The realisation of the AI model has been supported by a previous research of the supervisor Borja Bordel.
\section{Artificial Intelligence}
As exhaustively explained in the Chapter DoS attacks, some of the denial of service attacks are particularly hard to recognize, with a special attention to slow attacks, which can camouflage in the normal traffic. Traditional methods appear to be ineffective and resource consuming, especially in relation to the new technology that exploit compromised devices or that spoofs the IP addresses. Artificial Intelligence (AI) offers an alternative that provide an automatic and robust solution for the detection of DoS attacks, with a minimal human intervention \cite{jarrett2021}.

AI is the general field about the creation of a machine capable of performing task that typically would require human intelligence. Traditional AI methods relies on explicit programming to simulate the intelligent behaviour, however they struggle in task whose understanding and generalising result too vast and complex.
AI in fact, emerges as good solution in terms of scalability (for the possibility of handling huge volume of data), pattern recognition, automation (reducing the human intervention and minimising errors), adaptability (allowing an evolution of the model together with the evolution of data with a minimal effort) and finally accuracy.


Machine Learning is a branch of AI, that aims in recognising patterns and, most of all, extracting knowledge from data without depending on predefined rule, just as traditional AI. The learning activity relies on statistical techniques to extract patterns from large datasets and making decision or prediction without explicitly programming them for each task.
ML includes a variety of algorithms like:
\begin{itemize}
    \item Supervised Learning: training a model on a labelled dataset, where the correct output is provided for each input (e.g. Regression, Classification, SVM)
    \item Unsupervised Learning: finding hidden pattern or intrinsic structures without any answer label (e.g. Clustering, K-means, PCA)
    \item Reinforcement Learning: training of agents to make a sequence of decision by rewarding or penalizing them, based in their desirability.
\end{itemize}
Deep Learning (DL) is a further subset of ML that exploits the deep neural network (DNN) technologies to solve complex problems, especially in presence of raw unstructured data, like images, audio and text. The DNNs are composed of multiple layers of interconnected nodes, called neurons. Each layer perform a specific transformation on the input data, progressively extracting high-level features. The data path goes by 3 main categories of nodes. The first one is the Input Layer thanks to the DNN receives the raw data. Next, there are the Hidden Layers, which are responsible for performing transformation and computation aimed at obtaining the prediction or classification. The term deep in DNN refers to the presence of many hidden layers, that capture different levels of abstractions. Finally, the output layer provides the result, as an instance classifying an input with the right label.
Deep learning would offer the possibility to learn and extract relevant data straight from raw network traffic data. Moreover, variants like Recurrent Neural Network (RNN), result particularly effective for dataset with sequential entries, especially temporal patterns like traffic records, and high-dimensional data.
Once deployed, the model can undergo periodic retraining to ensure an evolution of the model together with the evolution of the attacks \cite{janiesch2021}. 


The implementation process goes by several phases, so it is essential to gather a substantial set of sample in order to train the model. Of course, the performance of the model depend on the quality of data too.
After the data collection, follows the data preparation, in which data are cleaned of noises and data type inconsistency. Finally, the dataset is analysed in order to understand the relation that exist between attributes of the dataset.
At this point, it is possible to choose the most suitable model and its architecture design. In order to train and test the model, the dataset is divided in two part: training and testing.
Once the model is trained by means of the training set, it can be tested and validated with metrics like the accuracy \cite{janiesch2021}.


\section{Dataset}
As treated in the Chapter 1, DoS attack leave a concrete but hard to detect trace on the network and nodes. To sum up, these traces can be related to unusual traffic patterns, packet rates, resource performance and service performance. The realisation of an AI model goes by the definition of a dataset that make use of some of these metrics. The dataset could either be realised or be recovered from other sources. Since the building of a dataset may result too much demanding, it has been opted for a ready-made dataset. 
In order to choose the dataset, it is indispensable to understand if it is going to be used structured or unstructured dataset. The structured dataset is organised in a fixed format, such as a table, in which each column is an attribute and each row is an entry. As an instance it would be acceptable to use data related to each connection, in which each entry is a connection and the attributes relative to that connection may be the duration of the connection, the source IP address, quantity of data exchanged and so on. In case of unstructured data, it may be possible to use raw network packets without any extensive manual feature extraction.
For a continuity reason it has been chosen, the same dataset elaborated and used by professor Bordel's team in their previous project. This dataset has been extracted and then elaborated from  another dataset, created by the Canadian Institute for Cybersecurity, that provide both raw data and structured data relative to the traffic exchanged with a server during two days. 
The CIC-IDS2017 dataset contains benign and common attack, in the format of true real-world data (PCAP). The data capturing period is composed of 5 days, in which the first one is characterised of only normal activity, the third of DoS attacks plus normal activity and the other remaining days of other attacks plus normal activity. Since the dataset of interest treats only the DoS attack, the only section selected from the CIC-IDS2017 was the day related to the DoS attack \cite{sharafaldin2018} \cite{CIC2017}. 
Next, the dataset has been manipulated and transformed, in order to match specific parameters. The parameters have been oriented to match the metrics recoverable with tools coming from the BCC library for python, tcpLife and tcpTracer, whose details will be covered in the chapter Data Collector. 

The parameters to recognise the attack, have been chosen among the possible parameters, to match metrics that could be independent from easy tweakable parameters, such as the IP address, that can be spoofed by the attacker, the content of the payload or the headers and others. 

\begin{figure}[h]
    \centering
    \includegraphics[width=1\textwidth]{4theModel/tcptoolsTablemetrics.png}
    \caption{Metrics obtainable from tcptools}
    \label{fig:tcptoolsMetrics}
\end{figure}


The chosen feature so, are related to the number of bytes exchanged, that takes note indirectly of the length of the header or the payload, the duration of the connection, that takes note of the rapidity of transmission and in general how much the server is kept busy, the number of open and closed connection and the time interval between one open/closed/accept event and the other. Using these parameters, the model should be able to spot the presence of an attack, by recognising general behaviour.
The dataset is composed of 2523 entries described by the attributes in the table \ref{tab:dataset-description}

\begin{table}[h!]
\centering
\begin{tabular}{p{4cm} p{9cm}}
\toprule
\textbf{Column Name} & \textbf{Description} \\
\midrule
mediaBT & Average bytes transmitted \\
mediaRX & Average bytes received \\
varBT & Variance of bytes transmitted \\
varRX & Variance of bytes received \\
medialat & Average latency \\
varlat & Variance of latency \\
Open & Number of open connections \\
Close & Number of closed connections \\
mediadift & Average difference of time \\
varDifT & Variance of difference of time \\
Attack & Attack label \\
\bottomrule
\end{tabular}
\caption{Description of features of the dataset.}
\label{tab:dataset-description}
\end{table}

The first six attributes are collected from the script tcpLife, while the seconds from tcpTracer. The tag \cmd{Attack} indicates whether the time interval has been characterised by malicious or benign traffic. To be more precise, both the script are tools that run perpetually, until the input of \cmd{CTRL+C}. For this reason, to collect the data, they need to be associated to a time interval. With this aim, the output of the script has been collected in a different list every 0.5 seconds and afterwards computed the average and variance of each value. The only values that are treated differently belong to the attributes Open and Close, which are counters. How it is noticeable from the table \ref{tab:dataset-description} there are two time referred attributes. The latency is referred to the duration of the connection, while the time difference is the interval of time between one event and the other, in which with event it is indicated the opening, accepting or closing of a connection.

\label{par:datasetAnalysis}
The initial step of the data analysis involves the importing and the preprocessing of data. Using the Seaborn's \cmd{describe} function, key statistics are obtained: count, mean, standard deviation, minimum, first quartile, median, third quartile and maximum. In this way are analysed the Central Tendency like mean and median (which is the middle value when data are ordered), the spread by means of std (standard deviation) and range (range = max - min), the presence of outliers with high value of std or range, the analysis of quartiles and finally the skewness that indicates the asymmetry of the distribution (positive means that distribution tail is on the right). 
Another fundamental step is the analysis of the relationships between the variables in the dataset, in order to understand the interplay between the different network parameters. To do so, it is computed the correlation matrix, whose calculation outputs the result shown by figure \ref{fig:heatmap}.

\begin{figure}[h]
    \centering
    \includegraphics[width=0.7\textwidth]{4theModel/heatmap.png}
    \caption{Heatmap of the correlation matrix of the dataset.}
    \label{fig:heatmap}
\end{figure}

 Each cell represents the correlation between two variables, with the colour indicating the strength and direction of the correlation. The correlation is a statistical measure that quantifies the strength and direction of the linear relationship between two variables. The correlation coefficient is represented in a range that goes from -1 to 1. Minus one means negative correlation, which is inverse proportionality, zero means no correlation, so no linear relationship between variables and finally positive one means direct proportionality.
 From the heatmap \ref{fig:heatmap} emerges a strong correlation between open and close suggesting that the variable move together, which is true in healthy traffic. Many variables, like \cmd{varlat}, express nearly zero correlation with most of the variable and a very low or moderate with the others. This means that the chosen attributes don't present multicollinearity, that occurs when independent variables are highly correlated, destabilising the model. Usually, this kind of situation is managed by combining highly correlated attributes or regularising them with specific techniques.

 Another way to analyse the dataset is by means of the scatter matrix, enhanced with the Kernel Density Estimate (KDE) on the diagonal. In this way, it is visually shown the distribution among couples of the various features, giving another perspective of the correlation. Each cell in the matrix corresponds to the scatter graphic of one variable against the other. An uptrend or downtrend highlights a positive or negative correlation, while if there is no discernible trend, it means no correlation. In addition to the correlation, the scatter plot is capable to indicate also specific correlation for specific intervals of the data (e.g the first quartile has a positive correlation and the rest of the quartile does not). The KDE on the diagonal is a non-parametric way to estimate the probability density function, which substitute the more traditional histogram with a smoother function. The peaks of the function indicate high concentration point revealing the most probable data, while the spread and shape indicates the variability.

\begin{figure}[h]
    \centering
    \includegraphics[width=\textwidth]{4theModel/scatterMatrix.png}
    \caption{Scatter matrix plot with kernel density estimate on the diagonal.}
    \label{fig:scatter_matrix}
\end{figure}

To conclude, the scatter matrix gives information about the correlation of the variables, their distribution and the detection of outliers.

Afterwards, the model gets partitioned into training and testing sets.
To improve the performance of many machine learning algorithms, it is required the standardisation of the dataset, which means transforming the feature to have a mean of zero and a std of one. In fact, features with large scales can dominate the distance calculation, while the standardisation ensures that each feature contributes equally to the model. It also fastens the convergence, since in the gradient descent optimisation the step size can be affected by the scale of the features and, finally, makes easier the comparison of the importance of different features. 
The standardisation process involves two main operations: 
\begin{itemize}
    \item Fitting: during this step, the mean and standard deviation of each feature in the training set are computed. This is performed only on the training set to avoid data leakage from the testing set to the training one, implying a different and more optimistic training of the model.
    \item Transforming: during this step, each feature is standardised by subtracting its means and dividing by the standard deviation. The transformation is applied both to the training and test sets, ensuring consistency between the training and test sets. 
\end{itemize}
Consistency in handling new data is mandatory, meaning that when a new entry is entered or predicted, it should be standardised with the same scaler used for the training data.

\section{Model}
%https://ieeexplore.ieee.org/abstract/document/8692706
It appears that Deep Learning algorithm perform better in relation to Machine Learning in the Network Security field, as demonstrated by V.Khetani and Y.Gandhi \ref{tab:algorithm_performance} \cite{khetani2023}.

\begin{table}[ht]
\centering
\resizebox{\textwidth}{!}{%
\begin{tabular}{|c|c|c|c|c|c|}
\hline
\multirow{} & \multirow{2}{*}{\textbf{Algorithm}} & \multicolumn{4}{c|}{\textbf{Accuracy}} \\ \cline{3-6}
 &  & \textbf{Healthcare} & \textbf{NLP} & \textbf{Financial Services} & \textbf{Network Security} \\ \hline
\multirow{4}{*}{\centering ML} & \centering Random Forest & 0.87 & 0.78 & 0.92 & 0.88 \\ \cline{2-6}
 & \centering SVM & 0.82 & 0.81 & 0.91 & 0.9 \\ \cline{2-6}
 & \centering LR & 0.81 & 0.84 & 0.88 & 0.85 \\ \cline{2-6}
 & \centering GBM & 0.88 & 0.83 & 0.93 & 0.89 \\ \hline
\multirow{4}{*}{\centering DL} & \centering CNN & 0.92 & 0.89 & 0.87 & 0.92 \\ \cline{2-6}
 & \centering LSTM & 0.88 & 0.87 & 0.89 & 0.88 \\ \cline{2-6}
 & \centering GRU & 0.86 & 0.85 & 0.92 & 0.87 \\ \cline{2-6}
 & \centering Transformer & 0.89 & 0.88 & 0.9 & 0.95 \\ \hline
\end{tabular}
}
\caption{Performance of various algorithms in different domains
\cite{khetani2023}.}
\label{tab:algorithm_performance}
\end{table}

For this reason, it appeared a good idea to concentrate the energies and resources on the realisation of a deep learning algorithm, to be confronted with a baseline constituted by a machine learning algorithm.
As deep learning algorithm has been chosen the Multilayer Perceptrons (MLP), that offer many upper hands for the detection of DoS attacks, especially when dealing with complex and potentially high-dimensional data, providing, in this way, scalability and better performances, allowing the upgrade of the dataset without having to change the strategy. MLP results, thanks to the multiple layer and non-linear activation functions, ideal for modelling Non-Linear Patterns, like the ones of the network. 
When selecting a ML algorithm baseline instead, the target should be oriented towards simplicity, interpretability, accuracy, computational efficiency and performance on structured data. Among the list of ML algorithms, has been chosen the Random Forest, for its ability to handle non-linear relationships and reducing the risk of overfitting \cite{Oukhouya2023}.

\subsection{Random Forest}
\label{par:ranfomF}
Random Forest stands as a strong and adaptable learning method in the machine learning field, offering advantages in predictive accuracy, interpretability and resilience against overfitting. As the name suggests, Random Forest is a collection of decision trees, in which each of them is trained independently on a subset of training data, called bootstrap aggregating or bagging. The bootstrap set is generating sampling with replacement to the original set, which means that a specific data point can be selected multiple times, making each bootstrap aggregation independent from the other. This operation involves a bigger variability, allowing the capture of different patterns and structure in the data, making the model more robust. Despite this, it contemporarily reduces the variance, leading to a more accurate prediction. Moreover, it also helps to prevent and reducing overfitting, since it generalises the dataset.
A decision tree is built by recursively splitting the dataset into subsets, based on the values of input of features: at each decision point of the tree, the algorithm chooses a feature and a threshold value to split the data, creating child nodes that are more homogeneous. In the Random Forest, when considering a split, the algorithm does not evaluate all the available features, but a random subset. Finally, within it, potential splits are evaluated and the one that best separates the data is chosen. The choice is made on the based of criteria like Gini impurity, information gain (based on entropy) or variance reduction. The chosen split is the one with the highest purity increase, for classification tasks, or the greatest reduction in variance, for regression tasks.
To conclude, the choice of Random Forest is motivated for its ability to increase diversity among the trees, reduce the overfitting and its computational efficiency.
As baseline has been chosen a random forest classifier, with \cmd{n\_estimators = 10} (number of decision trees in the forest), as evaluation criteria of the split has been chosen \cmd{entropy} and with \cmd{max\_depth = 1}, so that each tree makes a decision based on a single feature.

\begin{lstlisting}[caption=Training a Random Forest Classifier]
from sklearn.ensemble import RandomForestClassifier

# Initialize the classifier
classifier = RandomForestClassifier(n_estimators=10, criterion='entropy', random_state=0, max_depth=1)

# Train the classifier
classifier.fit(X_train, y_train)
\end{lstlisting}

The split criteria chosen, \cmd{entropy}, measures the amount of uncertainty or impurity in a dataset in which:
\[ H(p) = -p \log_2(p) - (1 - p) \log_2(1 - p) \]
where \cmd{p} is the proportion of one class in the dataset. When a node is pure (all instances belong to a single class), entropy goes to zero, differently if it is impure entropy reaches the maximum value of 1 of a binary classification problem (e.g. defining if the connection data are referred to malicious traffic or not). Entropy's advantages are related to the possibility of calculating the information gain, which measure the reduction in entropy after a split, the facilitation to balance the splits and its robustness to noise, especially compared to other criteria.


\subsection{MultiLayer Perceptrons}
Among the deep learning algorithm, stands the Multilayer Perceptrons (MLP). It belongs to the class of the feedforward neural network. It consists of multiple layers of nodes, called neurons, densely connected to the next. As the other neural networks, its infrastructure relies on three types of layers: input, hidden, output. The input layer takes raw features of the data as input and pass them to the hidden layers, where the computations and transformations occur. Each neuron in a layer processes the weighted sum of its inputs, applies an activation function to introduce non-linearity and, finally, passes the result to the next layer, until the output layer is reached. The main strength of MLP relies on its ability to deal with non-linearity in data relationships. The complex function, in this way, is approximated, making it suitable for tasks in which linearity models would fail.
Training this kind of model consist in adjusting the weight and biases of the neurons to minimise the difference between predicted outputs and actual targets, using optimisation techniques like gradient descent. 
Despite its effectiveness, MLP algorithm needs a large amount of labelled training data and the high number of hidden layers may lead to overfitting. During the recent years, the development in deep learning introduced new way to mitigate the overfitting and improve the training stability, as for an instance regularisation, dropout and batch normalisation.
\label{par:MLP}
To implement an MLP, it is possible to rely on TensorFlow, which is an open-source machine learning framework developed by Google. In TensorFlow, it is integrated a high-level neural network API called Keras, that simplifies the process of building a neural network. One of the simplest type of neural network that Keras presents is the Sequential Model, that permits the creation of an MLP. It allows layer to be added one after the other, in a system that offers as input the output of the previous layer. The \ref{fig:model_definition} shows how the MLP has been built and which kind of layers have been inserted.

\begin{figure}[htb]
    \centering
    \includegraphics[trim=0in 4in 0in 0in, clip, width=0.95\textwidth]{4theModel/MLPsummary.pdf}
    \caption{MLP definition}
    \label{fig:model_definition}
\end{figure}

A Dense Layer (or fully connected layer) is a node of the neural network that is connected to every node of the previous layer. In this case, have been chosen 6 neurons for each of the first two layers and one for the last dense layer. The activation functions, as previously explained, are crucial for the introducing of non-linearity. They are mathematical functions applied to the output of each neuron. The two function used are the:
\begin{itemize}
    \item ReLU (Rectified Linear Unit): used widely because it helps in overcoming the vanishing gradient problem. It consists in the slowing down of the updating process of the model's parameters, due to a very small gradient of the loss function. It is defined as \begin{equation} f(x) = \max(0, x) \end{equation}
    \item Sigmoid: defined as \begin{equation} f(x) = \frac{1}{1 + e^{-x}} \end{equation} Its main task, especially if placed as last node, is to prepare the values to the binary classification. In fact, it takes the input and squashes it in a range between 0 and 1.
\end{itemize}  

The chosen optimizer, which determines the strategy for adjusting the neural network's weights to minimise the loss function, is the \cmd{adam} optimizer. It works on the gradients of the parameters, improving and making faster the convergence during training. As loss function has been chosen the \cmd{binary\_crossentropy}, that calculates the difference between the predicted probabilities and the true binary labels. As a metric for the evaluation, that provides human readability and a measure for the model improvement, \cmd{acc} (accuracy) has been selected.
The Adam optimiser is one of the most used in training the neural networks. Adam adapt dynamically the learning rate for each parameter individually and differently from the traditional fixed learning rate. To do so, it relies on two mechanism, the moments of estimation and the bias correction. The moments help in understanding the magnitude and variability of the gradients, by dealing with mean and variance of the gradients. To counteract the bias introduced in these steps, a bias of correction is applied. Finally, the parameters of the model are updated individually using the calculated weight.


\begin{lstlisting}[language=Python], label={lst:model_compilation}, caption={Model Compilation}]
model.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['acc']) 
\end{lstlisting}

Once the model is compiled, it is ready to learn from the training data using the \cmd{fit} method.

\begin{lstlisting}[language=Python, label={lst:model_training}, caption={Model Training}]
history = model.fit(X_train, y_train, batch_size = 100, epochs = 200)
\end{lstlisting}

Usually the fit stage involves the processing of data in batches, balancing the computational efficiency and gradient accuracy, influencing how smoothly the model converges. As an epoch is meant the number of time the model will iterate over the entire dataset. Each epoch consists of multiple batches, and after each batch the model's parameters are updated based on the computed gradients from the loss function.
To sum up the train dataset is fed into the model in batches, the predictions are compared with the original labels using the specified loss function(\cmd{binary\_crossentropy}), and the weights are adjusted via backpropagation to minimise the loss. The process goes on for the number of epoch specified.

\begin{figure}[h]
  \centering
  \includegraphics[width=0.5\textwidth]{4theModel/trainingMLP.png} 
  \caption{Accuracy through epochs of MLP}
  \label{fig:trainingMPL}
\end{figure}

\subsection{Comparison of results}
At this step of the compilation, it is possible to compare the results of the two models, by means of the confusion matrix and metrics like accuracy, precision, recall, $F_1$-score and AUC\_ROC score.

\begin{center}
\begin{tabular}{|c|c|c|}
\hline
\textbf{Model} & \textbf{Random Forest} & \textbf{MLP (Multi-Layer Perceptron)} \\
\hline
\textbf{Confusion Matrix} & 
$\begin{bmatrix}
363 & 0 \\
2 & 266
\end{bmatrix}$ &
$\begin{bmatrix}
294 & 0 \\
2 & 209
\end{bmatrix}$ \\
\hline
\textbf{Accuracy} & 0.9968 & 0.9960 \\
\hline
\textbf{Precision} & 1.0 & 1.0 \\
\hline
\textbf{Recall (Sensitivity)} & 0.9925 & 0.9905 \\
\hline
\textbf{F1-score} & 0.9963 & 0.9952 \\
\hline
\textbf{AUC\_ROC score} & 0.9963 & 0.9953 \\
\hline
\end{tabular}
\end{center}

A confusion matrix is a table that summarizes the performance of a classification algorithm by displaying the counts of true positive (TP), true negative (TN), false positive (FP), and false negative (FN) predictions made by the model on a set of test data. It is typically represented as follows:
\begin{figure}
\centering
\noindent
\renewcommand\arraystretch{1.5}
\setlength\tabcolsep{0pt}
\begin{tabular}{c >{\bfseries}r @{\hspace{0.7em}}c @{\hspace{0.4em}}c @{\hspace{0.7em}}l}
  \multirow{10}{*}{\rotatebox{90}{\parbox{1.1cm}{\bfseries\centering actual\\ value}}} & 
    & \multicolumn{2}{c}{\bfseries Prediction outcome} & \\
  & & \bfseries p & \bfseries n & \bfseries total \\
  & p$'$ & \MyBox{True}{Positive} & \MyBox{False}{Negative} & P$'$ \\[2.4em]
  & n$'$ & \MyBox{False}{Positive} & \MyBox{True}{Negative} & N$'$ \\
  & total & P & N &
\end{tabular}
\end{figure}

Both models have a very high accuracy with Random Forest being slightly higher than MLP with 0,9968 compared to 0.9960: They both have  a perfect precision of 1.0 meaning there are no false positives. Recall measures the proportion of actual positive instances (true positives) that are correctly identified by the model. It is calculated as:
\[
\text{Recall} = \frac{\text{True Positives}}{\text{True Positives} + \text{False Negatives}}
\]
where:
\begin{itemize}
  \item $\text{True Positives (TP)}$: The number of instances that are positive and are correctly predicted as positive by the model.
  \item $\text{False Negatives (FN)}$: The number of instances that are positive but are incorrectly predicted as negative by the model.
\end{itemize}
The random forest shows a slightly higher recall of 0.9925 compared to MLP 0.9905.

F1-score, instead, is the mean of precision and recall. It provides a single metric that balances both precision and recall. The formula for $F_1$-score is:
\[
\text{$F_1$-score} = 2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}
\]
where the $\text{precision}$ is the proportion of true positive predictions among all positive predictions, calculated as $\frac{\text{True Positives}}{\text{True Positives} + \text{False Positives}}$.
F1-score ranges from 0 to 1, where 1 indicates the best possible performance and 0 indicates the worst. 
Even in this metric the Random Forest appeared to perform better with a score of 0.9963 in comparison to MLP 0.9952

Finally, for the AUC-ROC score too, the Random Forest performed better with 0.9963 against 0.9953.
The AUC-ROC score is a performance metric used to evaluate the ability of a binary classification model to distinguish between classes. It represents the area under the Receiver Operating Characteristic (ROC) curve.

The ROC curve plots the True Positive Rate (Recall) against the False Positive Rate (FPR) at various threshold settings. The AUC-ROC score ranges from 0 to 1, where:
\begin{itemize}
  \item A score of 1 indicates perfect discrimination, where the model perfectly separates the positive and negative classes.
  \item A score of 0.5 indicates that the model performs no better than random chance.
  \item A score below 0.5 suggests that the model is worse than random guessing, where predictions are flipped.
\end{itemize}

The AUC-ROC score provides a single value that summarizes the model's ability to rank true positives higher than false positives across all possible thresholds.

Overall appeared that the Random Forest performs better, even if the results of the 2 model are very similar. 
Since it is expected that a bigger dataset would bring to better performance for the MLP, it has been decided to proceed with the neural network model. With a larger dataset, of course, the training is going to require significant computational resources, with the possible employment of the GPU.



