Detecting compromise in TEE applications at runtime

The complexity of modern applications poses significant challenges in maintaining system security and trustworthiness. Applications across various domains, ranging from Cloud Computing to the Internet of Things, often rely on processing sensitive data and hence require execution within a secure environment that isolates them from other untrusted applications. As a result, in recent years Trusted Execution Environments (TEEs) have been developed to offer a secure area where data and code can be securely processed and stored, providing strong isolation guarantees. This thesis focuses specifically on Keystone Enclave, an open-source Trusted Execution Environment framework built upon the RISC-V Instruction Set Architecture. Keystone Enclave aims to address by design the limitations observed in other existing TEE technologies. This framework provides a set of components that enable the developers to customize the trusted environment, according to the security requirements of the specific domain. Among the numerous security features offered, it includes a binary measurement mechanism during the loading phase. This process verifies the integrity of an application and determines whether it can be considered trustworthy or not. However, it’s important to notice that this strategy only ensures the application’s state at boot time. Vulnerabilities present in the application code can still be exploited by attackers during the execution, potentially compromising the integrity and confidentiality guarantees of the trusted environment. Being able to identify an application that behaves in an unexpected way during its entire lifecycle is significant in this context, since it enables the framework to enforce the defined security policies at any given moment, although introducing some associated computational overhead. Therefore, the primary objective of this thesis is to design and implement a run-time monitoring solution capable of detecting compromised applications running within Trusted Execution Environments. To this end, the core TEE concepts and technologies are initially introduced, with a particular emphasis on their architecture, characteristics and relevant use cases. Subsequently, the most significant attacks against Trusted Execution Environment are classified and discussed, highlighting the weaknesses of certain TEE implementations. Afterwards, a detailed analysis of Keystone Enclave is conducted, including the relevant RISC-V ISA features within this context. Considering the current limitations of the Keystone framework, the proposed run-time monitoring solution is then described. Its objective is to initially identify the memory regions of the enclave (the trusted environment) that needs to be verified. This process requires a proper configuration of the Runtime (a RISC-V S-mode software module) page tables, which are located in the enclave memory. During the enclave execution, the Security Monitor performs measurements of the previously identified memory pages, in order to check whether the trusted application behaves as expected, and eventually enforce appropriate countermeasures.